问题描述
Keras fit_generator()具有参数pickle_safe，其默认值为False.
如果是 pickle_safe，则可以更快地进行训练，并因此将标志设置为True吗?
Keras fit_generator() has a parameter pickle_safe which defaults to False.
Training can run faster if it is pickle_safe, and accordingly set the flag to True ?
根据 Kera的文档:
我不明白这是什么意思.
如何确定我的论点是否为pickle_safe ?
I don't understand exactly what this is saying.
How can I determine if my arguments are pickle_safe or not ??
如果相关:
-我要传递自定义生成器
-生成器函数采用以下参数:X_train，y_train，batch_size，p_keep;
它们的类型为np.array，int，float)
-我没有使用GPU
-另外，我使用的是Keras 1.2.1，尽管我认为此参数的行为与keras 2中的行为相同
If it's relevant:
- I'm passing in a custom generator
- the generator function takes arguments: X_train, y_train, batch_size, p_keep;
they are of type np.array, int, float)
- I'm not using a GPU
- Also, I'm using Keras 1.2.1, though I believe this argument behaves the same as in keras 2
推荐答案
我对keras不熟悉，但是从文档看，pickle_safe仅表示生成器生成的元组必须可拾取". .
I have no familiarity with keras, but from a glance at the documentation, pickle_safe just means that the tuples produced by your generator must be "picklable".
pickle是一个标准的python模块，用于序列化和反序列化对象.标准的multiprocessing实现使用pickle机制在不同进程之间共享对象-由于两个进程不共享相同的地址空间，因此它们无法直接看到相同的python对象.因此，要将对象从进程A发送到进程B，将它们在A中腌制(产生以特定的众所周知格式的字节序列)，然后通过进程间通信机制将腌制的格式发送到B，然后在B中解开，在B的地址空间中产生A的原始对象的副本.
pickle is a standard python module that is used to serialize and unserialize objects. The standard multiprocessing implementation uses the pickle mechanism to share objects between different processes -- since the two processes don't share the same address space, they cannot directly see the same python objects. So, to send objects from process A to process B, they're pickled in A (which produces a sequence of bytes in a specific well-known format), the pickled format is then sent via an interprocess-communication mechanism to B, and unpickled in B, producing a copy of A's original object in B's address space.
因此，要发现您的对象是否可腌制，只需对它们调用pickle.dumps即可.
So, to discover if your objects are picklable, just invoke, say, pickle.dumps on them.
>>> import pickle
>>> class MyObject:
... def __init__(self, a, b, c):
... self.a = a
... self.b = b
... self.c = c
...
>>> foo = MyObject(1, 2, 3)
>>> pickle.dumps(foo)
b'\x80\x03c__main__\nMyObject\nq\x00)\x81q\x01}q\x02(X\x01\x00\x00\x00cq\x03K\x03X\x01\x00\x00\x00aq\x04K\x01X\x01\x00\x00\x00bq\x05K\x02ub.'
>>>
dumps产生一个字节字符串.现在，我们可以使用loads从字节字符串中将foo对象重构为bar:
dumps produces a byte string. We can now reconstitute the foo object from the byte string as bar using loads:
>>> foo_pick = pickle.dumps(foo)
>>> bar = pickle.loads(foo_pick)
>>> bar
<__main__.MyObject object at 0x7f5e262ece48>
>>> bar.a, bar.b, bar.c
(1, 2, 3)
如果某些东西无法腌制，您将得到一个例外.例如，不能腌制lambda:
If something is not picklable, you'll get an exception. For example, lambdas can't be pickled:
>>> class MyOther:
... def __init__(self, a, b, c):
... self.a = a
... self.b = b
... self.c = c
... self.printer = lambda: print(self.a, self.b, self.c)
...
>>> other = MyOther(1, 2, 3)
>>> other_pick = pickle.dumps(other)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: Can't pickle local object 'MyOther.__init__.<locals>.<lambda>'
有关更多信息，请参阅文档: https://docs .python.org/3.5/library/pickle.html?highlight = pickle#什么可以被腌制和未腌制
See the documentation for more info:https://docs.python.org/3.5/library/pickle.html?highlight=pickle#what-can-be-pickled-and-unpickled
这篇关于Keras"pickle_safe":“腌制安全"或“不可腌制"是什么意思?在Python中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！