张量流基本lstm_cell中的trainable_weights和trainable_variables有什么区别?

问题描述

当尝试使用记录的在此，我偶然发现了trainable_weights和trainable_variables属性.

源代码还没有真可悲的是，对于像我这样的菜鸟来说，这确实很有用.一点点的实验确实产生了以下信息:两者的布局完全相同，是长度为2的列表，其中第一个条目为tf.形状的变量为(2 * num_units，4 * num_units)，列表的第二个条目的形状为(4 * num_units，)，其中num_units是初始化BasicLSTMCell的num_units.现在，对我而言，直观的猜测是，第一个列表项是lstm的四个内部层的权重的串联，第二个项是各个偏差的串联，显然适合这些偏差的预期大小.

现在的问题是，两者之间实际上是否有区别?我认为它们可能仅仅是从rnn_cell类继承这些结果的结果吗?

解决方案

从 RNNCell 继承的 Layer 类的源代码中:

  @propertydef trainable_variables(自己):返回 self.trainable_weights 

请参见此处.RNN类似乎并没有覆盖该定义-我会假设它存在于具有可训练变量的特殊图层类型中，这些变量并不完全符合权重"的要求.批量标准化将浮现在脑海，但是不幸的是，在我的源代码中，我找不到任何提及 trainable_variables 的内容( GraphKeys.TRAINABLE_VARIABLES 除外).>

While trying to copy the weights of a LSTM Cell in Tensorflow using the Basic LSTM Cell as documented here, i stumbled upon both the trainable_weights and trainable_variables property.

Source code has not really been informative for a noob like me sadly. A little bit of experimenting did yield the following information though:Both have the exact same layout, being a list of length two, where the first entry is a tf.Variable of shape: (2*num_units, 4*num_units), the second entry of the list is of shape (4*num_units,), where num_units is the num_units from initializing the BasicLSTMCell.The intuitive guess for me is now, that the first list item is a concatenation of the weights of the four internal layers of the lstm, the second item being a concatenation of the respective biases, fitting the expected sizes of these obviously.

Now the question is, whether there is actually any difference between these? I assume they might just be a result of inheriting these from the rnn_cell class?

解决方案

From the source code of the Layer class that RNNCell inherits from:

@property
def trainable_variables(self):
    return self.trainable_weights

See here. The RNN classes don't seem to overwrite this definition -- I would assume it's there for special layer types that have trainable variables that don't quite qualify as "weights". Batch normalization would come to mind, but unfortunately I can't find any mention of trainable_variables in that one's source code (except for GraphKeys.TRAINABLE_VARIABLES which is different).

这篇关于张量流基本lstm_cell中的trainable_weights和trainable_variables有什么区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！