问题描述
在 DQN 上的OpenAI基准代码中，在构建操作图的过程中，在目标网络的q值上使用tf.stop_gradient可以防止目标q值对最小化损耗的影响. (第213行)
In OpenAI baselines code on DQN, tf.stop_gradient is used on the q values of the target network during building the operation graph to prevent the contributions of the target q values to the minimization of the loss. (line 213)
但是，在调用minimize时，仅将var_list指定为范围在要优化的q网络下的tf.Variable，不包括目标q网络下具有范围的变量. (第223行)
However, when calling minimize, the var_list is specified as only the tf.Variable with scope that falls under the q network being optimized, excluding the variables with scope under the target q network. (line 223)
我不确定为什么两者都做.两种方法似乎都能达到相同的结果.
I'm not sure why they do both. The two approaches seem to achieve the same result.
推荐答案
这是多余的. IMO代码的读法更好-您知道该表达式不会流过渐变，而且您确切知道哪些变量会受到影响.
It's redundant. IMO code reads better - you know that gradient will not flow through that expression, and also you know exactly which variables will be affected.
一个人确实足以达到同等的效果.
One would indeed suffice to achieve equivalent effect.
这篇关于OpenAI基准:为什么同时使用`tf.stop_gradient`并指定`var_list`?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！