如何在并行进程(python)中将项目附加到列表中?

问题描述

这是我正在使用的更复杂代码的一个非常简单的版本.问题是我想将在 method() 函数中计算的项目附加到一些我可以稍后显示的列表中.然而，当这段代码运行时，列表对象是空的，而结果数组是满的.

this is a really simplistic version of a more complicated code I am working with. The problem is that I would like to append items calculated in the method() function to some list that I can display later. However, when this code is run, the list object is empty, while the results array is full.

    import multiprocessing as mp
    global list
    list = []
    def add(thing):
        list.append(thing)
    def method():
        global list
        add(8) #doesn't work as wanted
        return 7
    def logResult(result):
        results.append(result)

    if (__name__ == '__main__'):
        results = []
        cpu = mp.cpu_count()
        pool = mp.Pool(processes=cpu)
        for x in range(0, 2000):
            pool.apply_async(method,callback=logResult)
        pool.close()
        pool.join()
        print list
        print results

输出:

    []
    [7,7,7,7,7,7,7,7,7....] and so on.

我知道 add 方法似乎是多余的，但是 method() 函数中的简单 list.append() 也不起作用.add 方法旨在反映 logResult method().我明白为什么它不起作用，但我不知道如何解决这个问题.如果没有并行化，程序可以按预期工作，但是我的项目需要并行化，因为完成的计算比 method() 函数要繁琐得多.所需的输出是

I know that the add method seems redundant, but a simple list.append() inside the method() function doesn't work either. The add method is meant to mirror the logResult method(). I can see why it doesn't work, but I don't know how to fix this. Without the parallelization, the program works as desired, but parallelization is needed for my project as the calculation done is much more tedious than in the method() function. The desired output would be

    [8,8,8,8,8,8,8,8,8,8,8,8,...]
    [7,7,7,7,7,7,7,7,7,7,7,7,...] and so on.

提前致谢.

推荐答案

该列表应驻留在共享内存中，以便从工作子进程访问.考虑 multiprocessing.Manager().list().

The list should reside in the shared memory in order to be accessible from the worker subprocesses. Consider multiprocessing.Manager().list().

这篇关于如何在并行进程(python)中将项目附加到列表中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！