问题描述
对于C++,我们可以使用OpenMP进行并行编程;但是,OpenMP 不适用于 Python.如果我想并行处理我的 Python 程序的某些部分,我该怎么办?
For C++, we can use OpenMP to do parallel programming; however, OpenMP will not work for Python. What should I do if I want to parallel some parts of my python program?
代码的结构可以认为是:
The structure of the code may be considered as:
solve1(A)
solve2(B)
其中solve1
和solve2
是两个独立的函数.如何并行而不是顺序运行这种代码以减少运行时间?代码是:
Where solve1
and solve2
are two independent function. How to run this kind of code in parallel instead of in sequence in order to reduce the running time?The code is:
def solve(Q, G, n):
i = 0
tol = 10 ** -4
while i < 1000:
inneropt, partition, x = setinner(Q, G, n)
outeropt = setouter(Q, G, n)
if (outeropt - inneropt) / (1 + abs(outeropt) + abs(inneropt)) < tol:
break
node1 = partition[0]
node2 = partition[1]
G = updateGraph(G, node1, node2)
if i == 999:
print "Maximum iteration reaches"
print inneropt
其中setinner
和setouter
是两个独立的函数.这就是我想要并行的地方...
Where setinner
and setouter
are two independent functions. That's where I want to parallel...
推荐答案
您可以使用 multiprocessing模块.对于这种情况,我可能会使用处理池:
You can use the multiprocessing module. For this case I might use a processing pool:
from multiprocessing import Pool
pool = Pool()
result1 = pool.apply_async(solve1, [A]) # evaluate "solve1(A)" asynchronously
result2 = pool.apply_async(solve2, [B]) # evaluate "solve2(B)" asynchronously
answer1 = result1.get(timeout=10)
answer2 = result2.get(timeout=10)
这将产生可以为您完成通用工作的进程.由于我们没有通过 processes
,它会为您机器上的每个 CPU 内核生成一个进程.每个 CPU 内核可以同时执行一个进程.
This will spawn processes that can do generic work for you. Since we did not pass processes
, it will spawn one process for each CPU core on your machine. Each CPU core can execute one process simultaneously.
如果要将列表映射到单个函数,可以这样做:
If you want to map a list to a single function you would do this:
args = [A, B]
results = pool.map(solve1, args)
不要使用线程,因为 GIL 会锁定对 Python 对象的任何操作.
Don't use threads because the GIL locks any operations on python objects.
这篇关于如何在 Python 中进行并行编程?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!