近日学习python的多进程,遇到了一个坑,在此记录一下。
代码如下:
from multiprocessing import Manager, Pool, Lock
def worker(l):
for i in range(5):
l.append(i)
print(l)
def callback(arg):
print('......exec done ')
if __name__ == '__main__':
pool = Pool()
m = Manager()
l = m.list()
for _ in range(4):
pool.apply_async(worker, args=(l,), callback=callback)
pool.close()
pool.join()
print('Main Process End.')
其中一次输出:
[0, 1, 2, 3, 0, 4, 1]
......exec done:
[0, 1, 2, 3, 0, 4, 1, 2, 3, 0, 4]
......exec done:
[0, 1, 2, 3, 0, 4, 1, 2, 3, 0, 4, 1, 2, 3, 0, 4, 1]
......exec done:
[0, 1, 2, 3, 0, 4, 1, 2, 3, 0, 4, 1, 2, 3, 0, 4, 1, 2, 3, 4]
......exec done:
Main Process End.
emmm....好像哪里有些不对....
于是就想到了加锁,修改代码如下:
def worker(l, lock):
with lock:
for i in range(5):
l.append(i)
print(l)
....
if __name__ == '__main__':
pool = Pool()
m = Manager()
l = m.list()
lock = Lock()
for _ in range(4):
pool.apply_async(worker, args=(l,lock), callback=callback)
pool.close()
pool.join()
print('Main Process End.')
结果发现程序直接就执行了最后一句print('Main Process End.')
...
查了一下发现似乎要用Manage,再次修改代码将lock = Lock()
替换为lock=m.Lock()
[0, 1, 2, 3, 4]
......exec done:
[0, 1, 2, 3, 4, 0, 1, 2, 3, 4]
......exec done:
[0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4]
......exec done:
[0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 1, 2, 3, 4]
......exec done:
Main Process End.
这次执行结果就和我们想要的一样了。