我正在抓取网页中的一些数据,然后将其保存到ZODB数据库中。
报废:
STARTING_URL = "https://persoanefizice.otpbank.ro/ro/curs-valutar"
page = requests.get(STARTING_URL)
tree = html.fromstring(page.text)
z = tree.xpath('//tr/td[position()<5]/text()')
爬取是在脚本中完成的,保存到数据库中的操作是通过另一个脚本来完成的,如下所示:
datainput.main(pres,nume,val_c,val_v,data_de_azi,'OTP')
尝试调用时出现错误。给出以下代码:
STARTING_URL = "https://persoanefizice.otpbank.ro/ro/curs-valutar"
page = requests.get(STARTING_URL)
tree = html.fromstring(page.text)
z = tree.xpath('//tr/td[position()<5]/text()')
contor = 0
while contor < len(z) :
pres = z[contor]
nume = z[contor+1]
val_c = z[contor+2]
val_v = z[contor+3]
# pdb.set_trace()
datainput.main(pres,nume,val_c,val_v,data_de_azi,'OTP')
print z[contor],z[contor+1],z[contor+2],z[contor+3],data_de_azi,'OTP'
contor = contor + 4
我收到此错误消息:
TypeError: can't pickle ElementBase objects
完整回溯:
> datainput.main(pres,nume,val_c,val_v,data_de_azi,'OTP') File "/home/iulian/workspace/Python/Crawler/datainput.py", line 36, in main
> transaction.commit() File "/usr/local/lib/python2.7/dist-packages/transaction/_manager.py", line
> 111, in commit
> return self.get().commit() File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py",
> line 280, in commit
> reraise(t, v, tb) File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py",
> line 271, in commit
> self._commitResources() File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py",
> line 417, in _commitResources
> reraise(t, v, tb) File "/usr/local/lib/python2.7/dist-packages/transaction/_transaction.py",
> line 391, in _commitResources
> rm.commit(self) File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 572,
> in commit
> self._commit(transaction) File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 628,
> in _commit
> self._store_objects(ObjectWriter(obj), transaction) File "/usr/local/lib/python2.7/dist-packages/ZODB/Connection.py", line 663,
> in _store_objects
> p = writer.serialize(obj) # This calls __getstate__ of obj File "/usr/local/lib/python2.7/dist-packages/ZODB/serialize.py", line 419,
> in serialize
> return self._dump(meta, obj.__getstate__()) File "/usr/local/lib/python2.7/dist-packages/ZODB/serialize.py", line 428,
> in _dump
> self._p.dump(state) File "/usr/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
> raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle ElementBase objects
最佳答案
解决我的问题的方法是将z []数据转换为字符串,然后再将其保存到这样的变量中:
pres = str(z[contor])
nume = str(z[contor+1])
val_c = str(z[contor+2])
val_v = str(z[contor + 3])
关于python - TypeError : cant pickle ElementBase objects,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/33346973/