multiprocessing模块实现了对多进程编程的封装,让我们可以非常方便的使用多进程进行编程。它的使用方法非常类似threading模块。
1.创建一个进程
import multiprocessing def worker():
"""worker function"""
print 'Worker'
return if __name__ == '__main__':
jobs = []
for i in range(5):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()
输出结果:
$ python multiprocessing_simple.py Worker
Worker
Worker
Worker
Worker
2.传递参数给子进程
import multiprocessing def worker(num):
"""thread worker function"""
print 'Worker:', num
return if __name__ == '__main__':
jobs = []
for i in range(5):
p = multiprocessing.Process(target=worker, args=(i,))
jobs.append(p)
p.start()
运行结果:
$ python multiprocessing_simpleargs.py Worker: 0
Worker: 1
Worker: 2
Worker: 3
Worker: 4
注意:所传递的参数必需是可以序列化的
3.后台进程
默认情况下,主进程会等待所有子进程运行完毕后退出。如果需要让子进程独立在后台运行,可以如下操作。
import multiprocessing
import time
import sys def daemon():
p = multiprocessing.current_process()
print 'Starting:', p.name, p.pid
sys.stdout.flush()
time.sleep(2)
print 'Exiting :', p.name, p.pid
sys.stdout.flush() def non_daemon():
p = multiprocessing.current_process()
print 'Starting:', p.name, p.pid
sys.stdout.flush()
print 'Exiting :', p.name, p.pid
sys.stdout.flush() if __name__ == '__main__':
d = multiprocessing.Process(name='daemon', target=daemon)
d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon)
n.daemon = False d.start()
time.sleep(1)
n.start()
运行结果:
$ python multiprocessing_daemon.py Starting: daemon 13866
Starting: non-daemon 13867
Exiting : non-daemon 13867
可以看到,daemon并没有exiting输出。
4.等待进程
import multiprocessing
import time
import sys def daemon():
print 'Starting:', multiprocessing.current_process().name
time.sleep(2)
print 'Exiting :', multiprocessing.current_process().name def non_daemon():
print 'Starting:', multiprocessing.current_process().name
print 'Exiting :', multiprocessing.current_process().name if __name__ == '__main__':
d = multiprocessing.Process(name='daemon', target=daemon)
d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon)
n.daemon = False d.start()
time.sleep(1)
n.start() d.join()
n.join()
join方法会等待对应的进程执行完毕,然后进行下一步操作。
运行结果:
$ python multiprocessing_daemon_join.py Starting: non-daemon
Exiting : non-daemon
Starting: daemon
Exiting : daemon
join方法还可以指定timeout参数,等待一段时间,如果子进程还没有执行完毕,那么就会返回。
import multiprocessing
import time
import sys def daemon():
print 'Starting:', multiprocessing.current_process().name
time.sleep(2)
print 'Exiting :', multiprocessing.current_process().name def non_daemon():
print 'Starting:', multiprocessing.current_process().name
print 'Exiting :', multiprocessing.current_process().name if __name__ == '__main__':
d = multiprocessing.Process(name='daemon', target=daemon)
d.daemon = True n = multiprocessing.Process(name='non-daemon', target=non_daemon)
n.daemon = False d.start()
n.start() d.join(1)
print 'd.is_alive()', d.is_alive()
n.join()
5.关闭子进程
import multiprocessing
import time def slow_worker():
print 'Starting worker'
time.sleep(0.1)
print 'Finished worker' if __name__ == '__main__':
p = multiprocessing.Process(target=slow_worker)
print 'BEFORE:', p, p.is_alive() p.start()
print 'DURING:', p, p.is_alive() p.terminate()
print 'TERMINATED:', p, p.is_alive() p.join()
print 'JOINED:', p, p.is_alive()
输出:
$ python multiprocessing_terminate.py BEFORE: <Process(Process-1, initial)> False
DURING: <Process(Process-1, started)> True
TERMINATED: <Process(Process-1, started)> True
JOINED: <Process(Process-1, stopped[SIGTERM])> False
6.进程返回状态
0,没有错误产生
>0,进程产生了一个错误
<0,进程被杀死
import multiprocessing
import sys
import time def exit_error():
sys.exit(1) def exit_ok():
return def return_value():
return 1 def raises():
raise RuntimeError('There was an error!') def terminated():
time.sleep(3) if __name__ == '__main__':
jobs = []
for f in [exit_error, exit_ok, return_value, raises, terminated]:
print 'Starting process for', f.func_name
j = multiprocessing.Process(target=f, name=f.func_name)
jobs.append(j)
j.start() jobs[-1].terminate() for j in jobs:
j.join()
print '%s.exitcode = %s' % (j.name, j.exitcode)
输出:
$ python multiprocessing_exitcode.py Starting process for exit_error
Starting process for exit_ok
Starting process for return_value
Starting process for raises
Starting process for terminated
Process raises:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python
2.7/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python
2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "multiprocessing_exitcode.py", line 24, in raises
raise RuntimeError('There was an error!')
RuntimeError: There was an error!
exit_error.exitcode = 1
exit_ok.exitcode = 0
return_value.exitcode = 0
raises.exitcode = 1
terminated.exitcode = -15