问题描述
我有一个运行蒙特卡罗模拟的python程序,可以找到概率问题的答案.我正在使用多重处理,这是在伪代码中
I have a python program that runs a Monte Carlo simulation to find answers to probability questions. I am using multiprocessing and here it is in pseudo code
import multiprocessing
def runmycode(result_queue):
print "Requested..."
while 1==1:
iterations +=1
if "result found (for example)":
result_queue.put("result!")
print "Done"
processs = []
result_queue = multiprocessing.Queue()
for n in range(4): # start 4 processes
process = multiprocessing.Process(target=runmycode, args=[result_queue])
process.start()
processs.append(process)
print "Waiting for result..."
result = result_queue.get() # wait
for process in processs: # then kill them all off
process.terminate()
print "Got result:", result
我想扩展此范围,以便可以对已运行的迭代次数进行统一计数.就像线程1运行了100次,线程2运行了100次一样,我想显示总共200次迭代,作为打印到控制台的信息.我指的是线程过程中的iterations
变量.如何确保所有线程都添加到同一变量?我以为使用iterations
的Global
版本会起作用,但没有效果.
I'd like to extend this so that I can keep a unified count of the number of iterations that have been run. Like if thread 1 has run 100 times and thread 2 has run 100 times then I want to show 200 iterations total, as a print to the console. I am referring to the iterations
variable in the thread process. How can I make sure that ALL threads are adding to the same variable? I thought that using a Global
version of iterations
would work but it does not.
推荐答案
正常的全局变量不在进程之间共享,就像它们在线程之间共享一样.您需要使用流程感知的数据结构.对于您的用例, multiprocessing.Value
应该可以正常工作:
Normal global variables are not shared between processes the way they are shared between threads. You need to use a process-aware data structure. For your use-case, a multiprocessing.Value
should work fine:
import multiprocessing
def runmycode(result_queue, iterations):
print("Requested...")
while 1==1: # This is an infinite loop, so I assume you want something else here
with iterations.get_lock(): # Need a lock because incrementing isn't atomic
iterations.value += 1
if "result found (for example)":
result_queue.put("result!")
print("Done")
if __name__ == "__main__":
processs = []
result_queue = multiprocessing.Queue()
iterations = multiprocessing.Value('i', 0)
for n in range(4): # start 4 processes
process = multiprocessing.Process(target=runmycode, args=(result_queue, iterations))
process.start()
processs.append(process)
print("Waiting for result...")
result = result_queue.get() # wait
for process in processs: # then kill them all off
process.terminate()
print("Got result: {}".format(result))
print("Total iterations {}".format(iterations.value))
一些注意事项:
- 我将
Value
明确传递给子代,以使代码与Windows兼容,Windows无法在父子代之间共享读/写全局变量. - 我用锁保护了增量,因为它不是原子操作,并且容易受竞争条件的影响.
- 我添加了
if __name__ == "__main__":
防护,再次提供了与Windows兼容的帮助,并且这是一般的最佳做法.
- I explicitly passed the
Value
to the children, to keep the code compatible with Windows, which can't share read/write global variables between parent and children. - I protected the increment with a lock, because its not an atomic operation, and is susceptible to race conditions.
- I added an
if __name__ == "__main__":
guard, again to help with Windows compatibility, and just as a general best practice.
这篇关于在多处理过程中保持统一计数?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!