我正在尝试从standard python documentation中提取代码片段,以了解如何使用多处理模块。该代码将粘贴在此消息的末尾。
我在四核计算机上的Ubuntu 11.04上使用Python 2.7.1(根据系统监视器,由于超线程,它为我提供了八个核)
问题:尽管启动了多个进程,但似乎所有工作负载都只计划在一个核心上,利用率接近100%。有时,所有工作负载都会迁移到另一个核心,但是工作负载永远不会在其中分布。
有任何想法为什么会这样吗?
此致,
保罗
#
# Simple example which uses a pool of workers to carry out some tasks.
#
# Notice that the results will probably not come out of the output
# queue in the same in the same order as the corresponding tasks were
# put on the input queue. If it is important to get the results back
# in the original order then consider using `Pool.map()` or
# `Pool.imap()` (which will save on the amount of code needed anyway).
#
# Copyright (c) 2006-2008, R Oudkerk
# All rights reserved.
#
import time
import random
from multiprocessing import Process, Queue, current_process, freeze_support
#
# Function run by worker processes
#
def worker(input, output):
for func, args in iter(input.get, 'STOP'):
result = calculate(func, args)
output.put(result)
#
# Function used to calculate result
#
def calculate(func, args):
result = func(*args)
return '%s says that %s%s = %s' % \
(current_process().name, func.__name__, args, result)
#
# Functions referenced by tasks
#
def mul(a, b):
time.sleep(0.5*random.random())
return a * b
def plus(a, b):
time.sleep(0.5*random.random())
return a + b
def test():
NUMBER_OF_PROCESSES = 4
TASKS1 = [(mul, (i, 7)) for i in range(500)]
TASKS2 = [(plus, (i, 8)) for i in range(250)]
# Create queues
task_queue = Queue()
done_queue = Queue()
# Submit tasks
for task in TASKS1:
task_queue.put(task)
# Start worker processes
for i in range(NUMBER_OF_PROCESSES):
Process(target=worker, args=(task_queue, done_queue)).start()
# Get and print results
print 'Unordered results:'
for i in range(len(TASKS1)):
print '\t', done_queue.get()
# Add more tasks using `put()`
for task in TASKS2:
task_queue.put(task)
# Get and print some more results
for i in range(len(TASKS2)):
print '\t', done_queue.get()
# Tell child processes to stop
for i in range(NUMBER_OF_PROCESSES):
task_queue.put('STOP')
test()
最佳答案
尝试用实际需要CPU的东西替换time.sleep
,您会看到multiprocess
正常工作!例如:
def mul(a, b):
for i in xrange(100000):
j = i**2
return a * b
def plus(a, b):
for i in xrange(100000):
j = i**2
return a + b
关于Python多处理仅利用一个核心,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/6905264/