Python Socket网络编程
Python Socket网络编程
在网络通信中socket几乎无处不在,它可以看成是应用层与TCP/IP协议簇通信的中间软件抽象层,是两个应用程序彼此进行通信的接口,并且把复杂的TCP/IP协议细节隐藏在接口之后。Python提供了socket模块,可以非常方便的进行socket编程。
创建一个server socket
使用socket
方法创建一个新的socket,通常提供两个参数,第一个参数是address family, 第二个是socket type。
#create an INET, STREAMing socket s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
以上创建了一个address family为IP协议,并且传输协议为TCP的socket。
服务器端在创建一个socket之后,需要绑定到一个IP地址与端口,提供服务,这就要用到bind
方法。
# bind the socket to all available interfaces on port 8888 s.bind(('', 8888))
使用listen
方法,将socket设置为监听状态。listen方法后面跟一个参数,表示最多可以在队列里面接收的请求数目。
#become a server socket serversocket.listen(5)
现在,我们已经创建了一个server socket,然后编写一个无限循环体,接收来自客户端的连接,利用accept
方法,它返回一个新的socket,表示与远端的连接,同时返回一个地址对,表示远端连接的IP地址与端口号。
# enter the main loop of the web server while 1: #accept connections from outside, return a pair (conn, addr) (clientsocket, address) = serversocket.accept() #now do something with the clientsocket #in this case, we'll pretend this is a threaded server ct = client_thread(clientsocket) ct.run()
通常,循环体中的server socket不会发送和接收任何数据,而仅仅是接收来自远端的连接,将新的连接交给其他的线程去处理,然后就继续侦听更多其他的连接,否则的话在处理一个连接的时候,就无法接受其他新的连接了。
接下来,我们新建一个线程对连接进行处理。首先,使用recv
方法接受来自socket的数据,后面带着一个参数表示一次最多可以接受数据的大小。然后使用sendall
方法回复socket,sendall不断发送数据直到所有数据发送完毕或者发生了异常。最后,需要记得把socket关闭,因为每个socket就代表了系统中的一个文件描述符,如果没有及时关闭,可能会超过系统最大文件描述符的数量,导致无法创建新的socket。
def handle_request(sock, addr): print "Accept new connection from {addr}".format(addr = addr) request_data = client_connection.recv(1024) print request_data.decode() http_response = "Hello, world!" # send response data to the socket sock.sendall(http_response) sock.close()
总结server socket主要由以下几个步骤:
- 创建一个新的server socket;
- 将socket绑定到一个地址和端口;
- 侦听远程进来的连接;
- 接受连接, 分发给其他线程处理。
以下是完整的server socket示例代码:
import socket import threading SERVER_ADDRESS = (HOST, PORT) = '', 8888 REQUEST_QUEUE_SIZE = 1024 def handle_request(sock, addr): print "Accept new connection from {addr}".format(addr = addr) request_data = client_connection.recv(1024) print request_data.decode() http_response = "Hello, world!" # send response data to the socket sock.sendall(http_response) sock.close() def serve_forever(): server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # reuse socket immediately server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server_socket.bind(SERVER_ADDRESS) server_socket.listen(REQUEST_QUEUE_SIZE) print 'Serving HTTP on port {port} ...'.format(port = PORT) while True: try: client_connection, client_address = server_socket.accept() except IOError as e: code, msg = e.args # restart 'accept' if it was interrupted if code == errno.EINTR: continue else: raise # create a thread to handle this request t = threading.Thread(target=handle_request, args=(sock, addr)) t.start() if __name__ == '__main__': serve_forever()
创建一个client socket
客户端的socket就比较简单,主要包括以下几个步��:
- 创建一个socket;
- 连接到服务器;
- 给服务器发送数据;
- 接收服务器返回的数据;
- 关闭socket。
import socket HOST = 'localhost' # the remote host PORT = 8888 # port used by server client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # connect to server client_socket.connect((HOST, PORT)) # send something to the server client_socket.sendall("Hello, world") data = client_socket.recv(1024) client_socket.close() print 'Received', repr(data)
IO复用
IO多路复用是指,先构造一张有关描述符的列表,然后调用一个函数,直到这些描述符中的一个已经准备好进行I/O时,该函数才返回。在返回时,它告诉进程哪些描述符已经准备好可以进行I/O。
Python的select模块提供了IO复用的功能,如select和poll等都能够执行I/O多路复用。
select
select方法的调用形式如下:
select.select(rlist, wlist, xlist[, timeout]
前面3个参数表示我们正在等待的对象,即我们所关心的文件描述符:
- rlist: 读事件,等待直到可读
- wlist: 写事件,等待直到可写
- xlist: 错误事件,等待直到发生了异常
第4个参数是可选的超时时间的秒数,如果没有选择超时参数,那么一直等待直到至少可以文件描述符准备就绪。如果超时时间设置为0,那么完全不等待,测试所有指定的描述符并立即返回,而不阻塞。
select方法返回的是一个三元组的列表表示准备好的对象。 一个示例程序: server端等待至少一个socket就绪:
import socket import select import sys import Queue # create a server socket server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.setblocking(0) server.bind(('localhost', 8888)) server.listen(5) # socketss ready for reading rlist = [server] # sockets ready for writing wlist = [] # exception elist =[] # each connection needs a queue to act as a buffer for the data to be sent # through it message_queues = {} # main loop while rlist: # wait for at least one socket is ready print "waiting for the next event" readable, writable, exceptional = select.select(rlist, wlist, elist) # handle readable lists for sock in readable: # server socket: ready to accept another connection if sock == server: conn, addr = sock.accept() print "new connection from {addr}".format(addr = addr) conn.setblocking(0) # add new connection to rlist rlist.append(conn) # give the connection a queue for data to send message_queues[conn] = Queue.Queue() # established connection from client else: request_data = sock.recv(1024) if request_data: print "received {data} from {addr}".format( data = request_data, addr = sock.getpeername()) message_queues[sock].put(request_data) # add to response if sock not in wlist: wlist.append(sock) # without data should be closed else: print "closing {addr} after no reading data".format(addr = sock.getpeername()) if sock in wlist: wlist.remove(sock) rlist.remove(sock) sock.close() del message_queues[sock] # handle writable lists for sock in writable: try: next_msg = message_queues[sock].get_nowait() except Queue.Empty: # no message to send print "output queue for {addr} is empty".format(addr = sock.getpeername()) wlist.remove(sock) else: print "sending {msg} to {addr}".format(msg = next_msg, addr = sock.getpeername()) sock.send(next_msg) # handle exceptional lists for sock in exceptional: print "handling exeptional condition for", sock.getpeername() rlist.remove(sock) if sock in wlist: wlist.remove(sock) sock.close() del message_queues[sock]
client端,创建两个socket向server发送消息:
import socket import sys # create 2 sockets for clients clients = [socket.socket(socket.AF_INET, socket.SOCK_STREAM), socket.socket(socket.AF_INET, socket.SOCK_STREAM) ] # tow messages to send messages = ["Hello, server.", "This is my message"] # connect to the server server_address = ('localhost', 8888) print "connecting to {addr}".format(addr = server_address) for sock in clients: sock.connect(server_address) # send one piece of messages once for msg in messages: # send message to the server for sock in clients: print "{addr} : sending {msg}".format(addr = sock.getpeername(), msg = msg) sock.send(msg) # read response from the server for sock in clients: response_data = sock.recv(1024) print "{addr} : received {data}".format(addr = sock.getpeername(), data = response_data) if not response_data: print "closing socket {addr}".format(addr = sock.getpeername()) sock.close()
poll
Unix系统的中poll()比select()有着更好的可扩展性,也就是说同时可以支持更多的客户端连接。因为poll()系统调用只需真正感兴趣的那些文件描述符,而select()是通过位图将感兴趣的文件描述符在位图中对应的bit置1,然后需要线性扫描位图中所有的bit。所以select()复杂度是O(N),而poll()是O(M),其中N是最大文件描述符个数,M是目前的文件描述符个数,通常M < N。
- 使用
select.poll()
新建一个polling对象poller,可以在这个对象上面注册(register)或者取消注册(unregister)文件描述符。 - 使用
poll.register
函数将socket注册到polling对象上poll.register(fd[, eventmask])
,这样只要有新的连接或者新的数据过来都会触发一个事件。register函数带有两个参数,第一个参数是文件描述符(整数),第二个参数是一个关于事件比特掩码,可以由POLLIN、POLLOUT和POLLERR等常量表示,用来表示你要检查的感兴趣的事件,如果没有显示指定,那么默认会检查所有3种类型。 poll.poll([timeout])
函数返回由(fd, event)二元组组成的列表,包含了有事件或者错误需要报告的描述符。event是一个bitmask,该描述符需要被报告的事件(events)对应的位被设置为1。POLLIN
:读,等待输入;POLLOUT
:写,描述符可以被写入,依次类推。如果返回的是一个空的列表,那么表示调用超时并且没有文件描述符有任何事件需要报告。如果给出了timeout参数,那么系统会在返回之前等待timeout的时间,如果没有给出timeout参数,那么调用会阻塞直到这个poll对象有新的事件到来。poll.modify(fd, eventmask)
函数可以修改已经注册过的fd,这样我们就能利用这个方法更改某个fd的eventmask,如从读改为写。poll.unregister(fd)
从polling对象中移除fd。
一个示例程序:
import select import socket import sys import Queue # create a server socket server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.setblocking(0) server.bind(('localhost', 8888)) server.listen(5) # queuses for message to send message_queues = {} # eventmask is an optional bitmask describing the type of events you want to # check for READ_ONLY = select.POLLIN | select.POLLPRI | select.POLLHUP | select.POLLERR READ_WRITE = READ_ONLY | select.POLLOUT # Returns a polling object, which supports registering and unregistering file # descriptors, and then polling them for I/O events poller = select.poll() # Register a file descriptor with the polling object poller.register(server, READ_ONLY) # map file desriptor to socket fd_to_socket = {server.fileno(): server, } # timeout pass to poll(), ms TIMEOUT = 1000 # in the loop calls poll() and process returned events while True: # wait for at least one socket be ready print "waiting for the next event" events = poller.poll(TIMEOUT) for fd, flag in events: # retrieve socket from fd s = fd_to_socket[fd] # handle read if flag & (select.POLLIN | select.POLLPRI): if s is server: # a readable server is ready to accept a connection conn, addr = s.accept() print "new connection from {addr}".format(addr = addr) conn.setblocking(0) fd_to_socket[conn.fileno()] = conn poller.register(conn, READ_ONLY) # give the connection a queue for sending messages message_queues[conn] = Queue.Queue() # otherwise receive data waiting to be read else: request_data = s.recv(1024) if request_data: print "received {data} from {addr}".format(data = request_data, addr = s.getpeername()) message_queues[s].put(request_data) # add to writable for response poller.modify(s, READ_WRITE) # get empty data means to close the connection else: print "closing {addr} after reading no data".format(addr = s.getpeername()) # stop listening for readable poller.unregister(s) s.close() del message_queues[s] # a client hang up the connection elif flag & select.POLLHUP: print "closing {addr} after receiving HUP".format(addr = s.getpeername()) # stop listening for readable poller.unregister(s) s.close() del message_queues[s] # any event with POLLERR cause to close the socket elif flag & select.POLLERR: print "handling exception for {addr}".format(addr = s.getpeername()) poller.unregister(s) s.close() del message_queues[s]
其他
Python的select模块中提供的select()和poll()函数几乎可以被大多数操作系统支持,此外模块还提供了epoll()和kqueue()函数,但是仅仅限于一部分的操作系统,epoll()仅支持Linux 2.5.44及之后的系统,而kqueue()运行于BSD系统。
本文示例的完整代码,可在GitHub下载查看。
下面关于Python的文章您也可能喜欢,不妨看看:
Python:在指定目录下查找满足条件的文件
Python2.7.7源码分析
无需操作系统直接运行 Python 代码
CentOS上源码安装Python3.4
《Python核心编程 第二版》.(Wesley J. Chun ).[高清PDF中文版]
《Python开发技术详解》.( 周伟,宗杰).[高清PDF扫描版+随书视频+代码]
Python脚本获取Linux系统信息
在Ubuntu下用Python搭建桌面算法交易研究环境
Python 语言的发展简史
Python 的详细介绍:请点这里
Python 的下载地址:请点这里
本文永久更新链接地址:
评论暂时关闭