python - python 3.2 pickle.load随机导致EOFError

我遇到一个非常奇怪的错误。我有一个每天运行的cronjob:它打开一个 pickle 的文件，加载该文件，然后处理数据。在脚本末尾，它将数据保存到该文件中以供第二天处理。

大多数情况下，此脚本运行良好，但是打开该文件时会花费一些时间(我每月要说两次)，引发EOFError:

Traceback (most recent call last):
  File "read.py", line 11, in <module>
    SellerDictionarycf=pickle.load(SellerDict)
EOFError

我坚信没有其他进程正在处理该文件，并且我100％确定已写入数据，请在读取文件之前将其关闭。

我的代码片段在脚本开头读取了文件:

SellerDict=open('/home/hostadl/SellerDictab','rb')
SellerDictionaryab=pickle.load(SellerDict)
SellerDict.close()

我的代码嗅探在脚本末尾写入文件:

SellerDict=open('/home/hostadl/SellerDictab','wb')
pickle.dump(SellerDictionaryab,SellerDict)
SellerDict.flush()
SellerDict.close()

我保存了损坏文件的副本(读取时引发EOFError的文件)，然后使用pickletools对其进行了检查，这是错误消息:

python3.2 -m pickletools -o test.txt SellerDictab
Traceback (most recent call last):
  File "/usr/local/lib/python3.2/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python3.2/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.2/pickletools.py", line 2403, in <module>
    args.indentlevel, annotate)
  File "/usr/local/lib/python3.2/pickletools.py", line 1963, in dis
    for opcode, arg, pos in genops(pickle):
  File "/usr/local/lib/python3.2/pickletools.py", line 1874, in genops
    arg = opcode.arg.reader(pickle)
  File "/usr/local/lib/python3.2/pickletools.py", line 474, in read_unicodestring4
    "remain" % (n, len(data)))
ValueError: expected 2254 bytes in a unicodestring4, but only 0 remain

test.txt仍然有数据，下面是头和尾:

head test.txt
    0: \x80 PROTO      3
    2: }    EMPTY_DICT
    3: q    BINPUT     0
    5: (    MARK
    6: X        BINUNICODE 'bradey4357604769'
   27: q        BINPUT     1
   29: ]        EMPTY_LIST
   30: q        BINPUT     2
   32: ]        EMPTY_LIST
   33: q        BINPUT     3

tail test.txt
18933166: e                APPENDS    (MARK at 18930621)
18933167: ]            EMPTY_LIST
18933168: r            LONG_BINPUT 174342
18933173: (            MARK
18933174: X                BINUNICODE 'HP PhotoSmart C4480 All-in-One Printer'
18933217: r                LONG_BINPUT 174343
18933222: G                BINFLOAT   45.0
18933231: G                BINFLOAT   84104.0
18933240: X                BINUNICODE 'Salt Lake City,  UT '
18933265: r                LONG_BINPUT 174344

对于好的文件(没有EOFError)，下面是标题:

head testgood.txt
    0: \x80 PROTO      3
    2: }    EMPTY_DICT
    3: q    BINPUT     0
    5: (    MARK
    6: X        BINUNICODE 'bj8016541577'
   23: q        BINPUT     1
   25: ]        EMPTY_LIST
   26: q        BINPUT     2
   28: (        MARK
   29: ]            EMPTY_LIST

tail testgood.txt
16569368: X                BINUNICODE 'Bought this beautiful dinner set but never opened it, I would love to get rod of it pleas tex with an offer...'
16569483: r                LONG_BINPUT 161843
16569488: J                BININT     20177894
16569493: X                BINUNICODE 'antonio8016497082'
16569515: r                LONG_BINPUT 161844
16569520: e                APPENDS    (MARK at 16569283)
16569521: e            APPENDS    (MARK at 16568508)
16569522: u        SETITEMS   (MARK at 16088830)
16569523: .    STOP

那么，可能是此问题的原因(例如字符串数据中的奇怪字符)？我对这个问题一无所知。

最佳答案

看起来您已经用完了磁盘空间(或者由于类似的原因而无法写入完整的文件)。

更加惯用的版本(您无需担心调用close/flush)将是:

with open('/home/hostadl/SellerDictab','rb') as SellerDict:
    SellerDictionaryab=pickle.load(SellerDict)

with open('/home/hostadl/SellerDictab','wb') as SellerDict:
    pickle.dump(SellerDictionaryab,SellerDict)