本文实例讲述了python多进程机制。分享给大家供大家参考。具体如下：
在以前只是接触过PYTHON的多线程机制，今天搜了一下多进程，相关文章好像不是特别多。看了几篇，小试了一把。程序如下，主要内容就是通过PRODUCER读一个本地文件，一行一行的放到队列中去。然后会有相应的WORKER从队列中取出这些行。
import multiprocessingimport osimport sysimport Queueimport timedef writeQ(q,obj): q.put(obj,True,None) print "put size: ",q.qsize()def readQ(q): ret = q.get(True,1) print "get size: ",q.qsize() return retdef producer(q): time.sleep(5) #让进行休息几秒 方便ps命令看到相关内容 pid = os.getpid() handle_file = '/home/dwapp/joe.wangh/test/multiprocess/datafile' with open(handle_file,'r') as f: #with...as... 这个用法今天也是第一次看到的 for line in f: print "producer is doing: ",line writeQ(q,line.strip()) q.close()def worker(q): time.sleep(5) #让进行休息几秒 方便ps命令看到相关内容 pid = os.getpid() empty_count = 0 while True: try: task = readQ(q) print "worker is doing: " ,task ''' 如果这里不休眠的话 一般情况下所有行都会被同一个子进程读取到 为了使实验效果更加清楚 在这里让每个进程读取完一行内容时候休眠5s 这样就可以让其他的进程到队列中进行读取 ''' time.sleep(5) except Queue.Empty: empty_count += 1 if empty_count == 3: print "queue is empty, quit" q.close() sys.exit(0)def main(): concurrence = 3 q = multiprocessing.Queue(10) funcs = [producer , worker] for i in range(concurrence-1): funcs.append(worker) for item in funcs: print str(item) nfuncs = range( len(funcs) ) processes = [] for i in nfuncs: p = multiprocessing.Process(target=funcs[i] , args=(q,)) processes.append(p) print "concurrence worker is : ",concurrence," working start" for i in nfuncs: processes[i].start() for i in nfuncs: processes[i].join() print "all DONE"if __name__ == '__main__': main()
登录后复制
实验结果如下：
dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>python 1.py concurrence worker is : 3 working startproducer is doing: line 1put size: 1producer is doing: line 2put size: 2producer is doing: line 3put size: 3producer is doing: line 4put size: 3producer is doing: line 5get size: 3put size: 4worker is doing: line 1get size: 3worker is doing: line 2get size: 2worker is doing: line 3get size: 1worker is doing: line 4get size: 0worker is doing: line 5queue is empty, quitqueue is empty, quitqueue is empty, quitall DONE
登录后复制
程序运行期间在另外一个窗口进行ps命令 可以观测到一些进程的信息
dwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp 13735 11830 0 Nov20 pts/12 00:00:05 pythondwapp 28319 27481 8 14:04 pts/0 00:00:00 python 1.pydwapp 28320 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28321 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28323 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28325 27849 0 14:04 pts/13 00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp 13735 11830 0 Nov20 pts/12 00:00:05 python #此时28320进程 也就是PRODUCER进程已经结束dwapp 28319 27481 1 14:04 pts/0 00:00:00 python 1.pydwapp 28321 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28323 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28328 27849 0 14:04 pts/13 00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp 13735 11830 0 Nov20 pts/12 00:00:05 pythondwapp 28319 27481 0 14:04 pts/0 00:00:00 python 1.pydwapp 28321 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28322 28319 0 14:04 pts/0 00:00:00 python 1.pydwapp 28323 28319 0 14:04 pts/0 00:00:00 [python] #这里应该是代表28323进程（WORKER）已经运行结束了dwapp 28331 27849 0 14:04 pts/13 00:00:00 grep pythondwapp@pttest1:/home/dwapp/joe.wangh/test/multiprocess>ps -ef | grep pythondwapp 13735 11830 0 Nov20 pts/12 00:00:05 pythondwapp 28337 27849 0 14:05 pts/13 00:00:00 grep python
登录后复制
希望本文所述对大家的Python程序设计有所帮助。
版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容， 请发送邮件至253000106@qq.com举报，一经查实，本站将立刻删除。
发布者：PHP中文网，转转请注明出处：https://www.chuangxiangniao.com/p/2299723.html
