问题描述
我正在尝试编写测试以比较python中进程间通信的方法,而OSX和Windows给我的第一个示例带来了问题.这几乎是使用队列的手动示例.
I'm trying to write a test to compare methods for inter process communication in python, and OSX and windows are giving me problem with my first sample. it is pretty much the manual sample for using Queues.
我用python 2.7.9和3.4.1在linux上编写了所有代码.而且效果很好.
i wrote all the code on linux with python 2.7.9 and 3.4.1. And it works fine.
#!/usr/bin/env python
import multiprocessing as MP
...
self.mpq_main = MP.Queue()
self.mpq_net = MP.Queue()
self.mpt_net = MP.Process( target=self.start_t_net, args=(self.mpq_main, self.mpq_net) )
...
def start_t_net(self, qin, qout):
self.NET = gcbtestnet.GCBTestNET(qin, qout);
然后在osx和Windows上测试我得到了错误.这个问题是针对Windows的:
Then testing on osx and windows i get errors. this question is for the windows one:
C:\Documents and Settings\user1\Desktop>C:\Python27\python.exe gcbtest.py
Traceback (most recent call last):
File "gcbtest.py", line 82, in <module>
theclass = GCBTest()
File "gcbtest.py", line 31, in __init__
self.mpt_fs.start();
File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
self._popen = Popen(self)
File "C:\Python27\lib\multiprocessing\forking.py", line 277, in __init__
dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\lib\pickle.py", line 224, in dump
self.save(obj)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\multiprocessing\forking.py", line 67, in dispatcher
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 401, in save_reduce
save(args)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 548, in save_tuple
save(element)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 725, in save_inst
save(stuff)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "C:\Python27\lib\pickle.py", line 419, in save_reduce
save(state)
File "C:\Python27\lib\pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "C:\Python27\lib\pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
File "C:\Python27\lib\pickle.py", line 681, in _batch_setitems
save(v)
File "C:\Python27\lib\pickle.py", line 313, in save
(t.__name__, obj))
pickle.PicklingError: Can't pickle '_subprocess_handle' object: <_subprocess_han
dle object at 0x00ABD190>
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1378, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 858, in load
dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 880, in load_eof
raise EOFError
EOFError
UI check
UI check
UI check
...
如果您尝试将对象引用作为过程参数传递,通常会显示该错误,但是我仅传递multiprocess.Queues
实例.
That error usually shows up if you try to pass a object reference as the process argument, but i am only passing multiprocess.Queues
instances.
推荐答案
我发现了(严重)错误.当我将流程从简单方法移动到对象时,我忘了将所述对象的初始化移动到.Process()
调用,并将其保留在类方法中.由于某种原因,它可以在Linux上运行,但不能在Windows上运行.
I found my (gross) mistake. When i moved the Processes from simple methods to objects, i forgot to move the initialization of said objects to the .Process()
call and kep it in a class method. For some reason it worked on linux, but not on windows.
这里是带有修复程序的差异 https://github.com/gcb/python_multiprocess_test/commit /8565384e5cc6cadf959c335b5db0693e646f6777
here is the diff with the fix https://github.com/gcb/python_multiprocess_test/commit/8565384e5cc6cadf959c335b5db0693e646f6777
-self.mpt_net = MP.Process( target=self.start_t_net, args=(self.mpq_main, self.mpq_net) )
+self.mpt_net = MP.Process( target=gcbtestnet.GCBTestNET, args=(self.mpq_main, self.mpq_net) )
...
-def start_t_net(self, qin, qout):
- self.NET = gcbtestnet.GCBTestNET(qin, qout);
这篇关于pickle.PicklingError:无法腌制'_subprocess_handle'对象:< _subprocess_han dle对象位于0x00AAAAAAA>的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!