使用带有进程池的队列似乎是一种常见的模式,即。

Pool(2).map(f, xs)

但是f的主体可以附加到被映射的项上,例如。
from multiprocessing import Pool

xs = [0]

def f(n):
    global xs
    if n < 10:
        xs.append(n + 1)
    return n

Pool(2).map(f, xs)

期望返回[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
我意识到用mt提供的原语构建这个“manually”是可能的,但它似乎是一个足够通用的模式,必须有一个通用的解决方案。你知道吗?

最佳答案

根据@martineau的建议,您的代码可以更新为:

import multiprocessing as mp


def f(n, xs, xn):
    if n < 10:
        xn.append(n)
        xs.append(n + 1)
        xn.append(n)
        xs.append(n + 2)


if __name__ == '__main__':
    with mp.Manager() as manager:
        xs = manager.list()
        xn = manager.list()
        with mp.Pool(processes=2) as pool:
            pool.starmap(f, [(n, xs, xn) for n in range(20)])
        print(xn)
        print(xs)

这个指纹
[3, 0, 3, 0, 4, 1, 4, 1, 5, 2, 5, 2, 6, 6, 7, 9, 7, 9, 8, 8]
[4, 1, 5, 2, 5, 2, 6, 3, 6, 3, 7, 4, 7, 8, 8, 10, 9, 11, 9, 10]

你所看到的,不能保证n个值的产生顺序得到保留。
编辑:
import multiprocessing as mp


def f(n):
    thresh = 10
    if max(xs) <= thresh and n < thresh:
        xs.append(n + 1)


if __name__ == '__main__':
    with mp.Manager() as manager:
        xs = manager.list([0])
        with mp.Pool(processes=2) as pool:
            pool.map(f, range(20))
        print(sorted(xs))

这张照片
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

10-06 05:19
查看更多