本文介绍了使用copy_reg处理类方法的酸洗问题的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
在进行多处理时遇到腌制错误:
I met a pickling error when dealing with multiprocessing:
from multiprocessing import Pool
def test_func(x):
return x**2
class Test:
@classmethod
def func(cls, x):
return x**2
def mp_run(n, func, args):
return Pool(n).map(func, args)
if __name__ == '__main__':
args = range(1,6)
print mp_run(5, test_func, args)
# [1, 4, 9, 16, 25]
print mp_run(5, Test.func, args)
"""
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.6/threading.py", line 484, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib64/python2.6/multiprocessing/pool.py", line 225, in _handle_tasks
put(task)
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
"""
我在此处,该解决方案非常适合这些自我"风格的实例方法,但在将配方应用于@classmethod时遇到问题:
def _pickle_method(method):
func_name = method.im_func.__name__
obj = method.im_self
cls = method.im_class
return _unpickle_method, (func_name, obj, cls)
def _unpickle_method(func_name, obj, cls):
try:
for cls in cls.mro():
try:
func = cls.__dict__[func_name]
except KeyError:
pass
else:
break
except AttributeError:
func = cls.__dict__[func_name]
return func.__get__(obj, cls)
copy_reg.pickle(MethodType, _pickle_method, _unpickle_method)
new_func = pickle.loads(pickle.dumps(Test.func))
"""
Traceback (most recent call last):
File "test3.py", line 45, in <module>
new_func = pickle.loads(pickle.dumps(Test.func))
File "/usr/lib64/python2.6/pickle.py", line 1366, in dumps
Pickler(file, protocol).dump(obj)
File "/usr/lib64/python2.6/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib64/python2.6/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/usr/lib64/python2.6/pickle.py", line 401, in save_reduce
save(args)
File "/usr/lib64/python2.6/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib64/python2.6/pickle.py", line 562, in save_tuple
save(element)
File "/usr/lib64/python2.6/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/lib64/python2.6/pickle.py", line 748, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <type 'classobj'>: it's not found as __builtin__.classobj
"""
Any ways to alter a few lines to get it work for classmethod?
推荐答案
我修改了配方,使其可以与classmethod一起使用.这是代码.
I modified the recipe to make it work with classmethod.Here's the code.
import copy_reg
import types
def _pickle_method(method):
func_name = method.im_func.__name__
obj = method.im_self
cls = method.im_class
if func_name.startswith('__') and not func_name.endswith('__'):
#deal with mangled names
cls_name = cls.__name__.lstrip('_')
func_name = '_%s%s' % (cls_name, func_name)
return _unpickle_method, (func_name, obj, cls)
def _unpickle_method(func_name, obj, cls):
if obj and func_name in obj.__dict__:
cls, obj = obj, None # if func_name is classmethod
for cls in cls.__mro__:
try:
func = cls.__dict__[func_name]
except KeyError:
pass
else:
break
return func.__get__(obj, cls)
copy_reg.pickle(types.MethodType, _pickle_method, _unpickle_method)
这篇关于使用copy_reg处理类方法的酸洗问题的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!