问题描述
我在行动中看到的 NeuralDataSet 对象除了XOR外只有两个小数据阵列......我还没能从 MLDataSet 上的文档中找出任何内容。
The NeuralDataSet objects that I've seen in action haven't been anything but XOR which is just two small data arrays... I haven't been able to figure out anything from the documentation on MLDataSet.
似乎必须立即加载所有内容。但是，我想循环训练数据，直到我达到EOF然后将其计为1个纪元。但是，我所看到的所有数据必须从头开始加载到1个2D数组中。我怎样才能解决这个问题？
It seems like everything must be loaded at once. However, I would like to loop through training data until I reach EOF and then count that as 1 epoch.. However, everything I've seen all the data must be loaded into 1 2D array from the beginning. How can I get around this?
。除此之外，我还没有在这里找到类似的问题。
I've read this question, and the answers didn't really help me. And besides that, I haven't found a similar question asked on here.
推荐答案
这是可能的，你可以使用支持流操作的数据集的现有实现，或者您可以在任何源上实现自己的数据集。查看和代码作为示例。如果您有特定的格式，则必须实现编解码器。对于CSV已经有一个实现，我还没有检查它是否是基于内存的。
This is possible, you can either use an existing implementation of a data set that supports streaming operation or you can implement your own on top of whatever source you have. Check out the BasicMLDataSet interface and the SQLNeuralDataSet code as an example. You will have to implement a codec if you have a specific format. For CSV there is an implementation already, I haven't checked if it is memory based though.
请记住，在执行此操作时，您的数据将为每个时期完全流式传输根据我的经验，这是一个比实际网络计算更高的瓶颈。
Remember when doing this that your data will be streamed fully for each epoch and from my experience that is a much higher bottleneck than the actual computation of the network.
这篇关于Encog - 如何加载神经网络的训练数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！