问题描述
我正在用pygtk编写一个工具，该工具需要递归解析大型目录，并将生成的匹配文件添加到列表中.这个过程显然导致用户界面挂断，因此我尝试使用多处理库.
I'm writing a tool in pygtk which needs to deal with recursively parsing large directories, and adding resulting matched files to a list. This process obviously causes the user interface to hang up, and so I have attempted to use the multiprocessing library.
在我寻求一些解决方案之前，请先了解更多背景知识:-该程序有两个主要的类，一个用于完成所有繁重工作并与UI对话的控制器类，另一个是用于处理工具所需的所有数据的Model类.
Some more background before I ask for some solutions:- the program has two main Classes, a controller class that does all the intensive work and speaks to the UI, and a Model class, for dealing with all the data the tool needs.
import sys
import os
import pygtk
import fnmatch
from multiprocessing import Pool
pygtk.require("2.0")
#try:
from gi.repository import Gtk
from gi.repository import GObject
#except:
# print("GTK Not Availible")
# sys.exit(1)
class Controller(object):
def __init__(self,builder,model):
self.builder=builder
self.model=model
def btn_pass_clicked(self, *args,**kwargs):
print "it's working!, its woooooorkkinnnnggg!"
spinnywheel= self.builder.get_object("activitySpinner")
spinnywheel.start()
def btn_fail_clicked(self, *args, **kwargs):
print "stopping spinnywheel!"
spinnywheel=self.builder.get_object("activitySpinner")
spinnywheel.stop()
def quit(self,*args,**kwargs):
print "iamquit"
Gtk.main_quit()
def file_menu_open(self,*args,**kwargs):
print "file->open"
self.builder.get_object("openDialogue").show()
def opendialogue_btnOpen_clicked(self,*args,**kwargs):
rootdir = os.path.expanduser(self.builder.get_object("openDialogue_entryBox").get_text())
self.builder.get_object("openDialogue").hide()
self.builder.get_object("openDialogue_entryBox").set_text("")
if os.path.exists(rootdir):
self.builder.get_object("activitySpinner").start()
print "pooling workers and walking ",rootdir
p = Pool(None)
p.apply_async(self.walk_for_files,rootdir,None,self.finished_recurse)
else:
print "Path does not exist!"
def walk_for_files(self,rootdir):
for root,dirs,files in os.walk(rootdir):
for extension in ['c','cpp']:
for filename in fnmatch.filter(files,'*.'+extension):
self.model.add_single_file(os.path.join(root,filename))
def finished_recurse(self,*args,**kargs):
print "workers finished parsing dirs!"
self.builder.get_object("activitySpinner").stop()
class Model(object):
def __init__(self):
self.fileList=[]
def add_single_file(self,file):
self.fileList.append(file)
print "added ",file
class Scrutiny(object):
def __init__(self):
builder = Gtk.Builder()
builder.add_from_file("scrutinydev.ui")
model_object=Model()
controller_object=Controller(builder,model_object)
builder.connect_signals(controller_object)
builder.get_object("windowMain").show()
builder.get_object("listView")
GObject.threads_init()
Gtk.main()
if __name__ == "__main__":
scrutiny = Scrutiny()
现在，这是我的问题.
如您所见，使用pool()生成的工作程序需要执行回调finish_recurse，以便在其他UI工作中停止GtkSpinner.
As you can see, the workers spawned with pool() need to execute the callback finish_recurse so that I can stop the GtkSpinner amongst other UI work.
在代码处于当前状态的情况下，我收到一个腌制错误，
With the code in its current state, I get a pickling error,
PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
我了解这是因为我无法序列化回调，并且希望获得解决方法/修复程序的建议，以实现所需的功能.
I understand that this is because I'm unable to serialize the callback, and would like suggestions for workarounds/fixes in order to achieve what I need.
推荐答案
我不太了解GTK，但是我认为您的问题更多是关于酸洗，而不是关于多处理.
I don't know well GTK, but I think your problem is more about pickling than about multiprocessing.
The __getstate__ and __setstate__ methods of the pickle module lets you customize the pickling process for any object.
这是一个简单的示例，展示了其工作原理:
Here is a trivial example which show how it works :
from pickle import dumps, loads
class NotPickable(object):
def __init__(self, x):
self.attr = x
ffile = open('/tmp/filesarenotpickable', 'r+w')
o = NotPickable(ffile)
dumps(o)
# => TypeError: can't pickle file objects
class Pickable(NotPickable):
attr = open('/tmp/a_file_on_an_other_system', 'r+w')
def __getstate__(self):
return self.attr.read()
def __setstate__(self, state):
self.attr.write(state)
o = Pickable(ffile)
dumps(o)
# OUT: 'ccopy_reg\n_reconstructor\np0\n(c__main__\nPickable\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n.'
o2 = loads(dumps(o))
o2.attr
# OUT: <open file '/tmp/a_file_on_an_other_system', mode 'r+w' at 0x18ad4b0>
当然，开发人员有责任代表并正确还原对象的状态.
Of course, it remains the responsibility of the developer to represent and properly restore the state of objects.
这篇关于多线程处理时出现酸洗错误-设计或代码问题?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！