本文介绍了OpenAI基准:为什么同时使用`tf.stop_gradient`并指定`var_list`?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

DQN 上的OpenAI基准代码中,在构建操作图的过程中,在目标网络的q值上使用tf.stop_gradient可以防止目标q值对最小化损耗的影响. (第213行)

In OpenAI baselines code on DQN, tf.stop_gradient is used on the q values of the target network during building the operation graph to prevent the contributions of the target q values to the minimization of the loss. (line 213)

但是,在调用minimize时,仅将var_list指定为范围在要优化的q网络下的tf.Variable,不包括目标q网络下具有范围的变量. (第223行)

However, when calling minimize, the var_list is specified as only the tf.Variable with scope that falls under the q network being optimized, excluding the variables with scope under the target q network. (line 223)

我不确定为什么两者都做.两种方法似乎都能达到相同的结果.

I'm not sure why they do both. The two approaches seem to achieve the same result.

推荐答案

这是多余的. IMO代码的读法更好-您知道该表达式不会流过渐变,而且您确切知道哪些变量会受到影响.

It's redundant. IMO code reads better - you know that gradient will not flow through that expression, and also you know exactly which variables will be affected.

一个人确实足以达到同等的效果.

One would indeed suffice to achieve equivalent effect.

这篇关于OpenAI基准:为什么同时使用`tf.stop_gradient`并指定`var_list`?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

07-07 18:27