问题描述
我正在使用一个简单的网络,我正在尝试使用 AdamOptimizer 来最小化 Q 学习环境中的损失.
I'm using a simple network and i'm trying to use AdamOptimizer to minimize the loss in a Q learning contexte.
代码如下:
### DATASET IMPORT
from DataSet import *
### NETWORK
state_size = STATE_SIZE
stack_size = STACK_SIZE
action_size = ACTION_SIZE
learning_rate = LEARNING_RATE
hidden_tensors = HIDDEN_TENSORS
gamma = GAMMA
import tensorflow as tf
import numpy as np
class NNetwork:
def __init__(self, name='NNetwork'):
# Initialisations
self.state_size = state_size
self.action_size = action_size
self.model = tf.keras.models.Sequential()
self.optimizer = tf.keras.optimizers.Adam(learning_rate)
# Network shaping
self.model.add(tf.keras.layers.Dense(self.state_size, activation='relu', kernel_initializer='glorot_uniform'))
self.model.add(tf.keras.layers.Dense(hidden_tensors, activation='relu', kernel_initializer='glorot_uniform'))
self.model.add(tf.keras.layers.Dense(action_size, activation='linear', kernel_initializer='glorot_uniform'))
# Prediction function (return Q_values)
def get_outputs(self, inputs):
inputs = tf.convert_to_tensor(inputs, dtype=tf.float32)
return self.model.predict(inputs)
# Optimization of the network
def optimize(self, state, action, reward, next_state):
next_Q_values = self.get_outputs(next_state)
target_Q = reward + gamma * np.max(next_Q_values)
curent_Q = tf.reduce_sum(tf.multiply(self.get_outputs(state), action))
loss = tf.square(target_Q - curent_Q)
self.optimizer.minimize(tf.convert_to_tensor(loss), self.model.trainable_variables)
B = NNetwork('b')
print(B.get_outputs([[0.12, 0.59]]))
B.optimize([[0.12, 0.59]], [1, 0, 0, 0, 0, 0, 0], 100000000, [[0.13, 0.58]])
print(B.get_outputs([[0.12, 0.59]]))
所以我的问题是:
当我执行这段代码时,我得到了这个:
When i execute this code i got this :
[[-0.00105272 0.02356465 -0.01908724 -0.03868931 0.01585
0.024270340.00203115]] 回溯(最近一次调用最后一次):文件.\DQNet.py",第 69 行,在B.optimize([[0.12, 0.59]], [1, 0, 0, 0, 0, 0, 0], 100000000, [[0.13, 0.58]]) 文件.\DQNet.py",第62行,在优化tf.keras.optimizers.Adam(learning_rate).minimize(tf.convert_to_tensor(10),self.model.trainable_variables) 文件C:\Users\Odeven poste1\Documents[Python-3.6.8\python-3.6.8.amd64\lib\site-packages\tensorflow\python\keras\optimizer_v2\optimizer_v2.py",第 296 行,最小化loss, var_list=var_list, grad_loss=grad_loss) 文件C:\Users\Odeven poste1\Documents[Python-3.6.8\python-3.6.8.amd64\lib\site-packages\tensorflow\python\keras\optimizer_v2\optimizer_v2.py",第 328 行,在 _compute_gradients 中loss_value = loss() TypeError: 'tensorflow.python.framework.ops.EagerTensor' 对象不可调用
这意味着我的网络可以正常运行,因为我得到了 Q 值,但是当我尝试调用我的优化"函数时,我在线上出现错误:
That mean that my network is functional because i got Q values but when i try to call my 'optimize' function i got an error on the line :
self.optimizer.minimize(tf.convert_to_tensor(loss), self.model.trainable_variables)
我真的不明白为什么我会收到这个错误:
And i really don't understand why i got this error :
'tensorflow.python.framework.ops.EagerTensor' 对象不可调用
因为我很确定我必须给最小化函数的损失"参数应该是张量...
Because i'm pretty sure that the 'loss' parameter i have to give to the minimize function should be a Tensor...
推荐答案
在 TF2 中,最小化方法的 loss
参数必须是 Python 可调用的.
In TF2 the loss
parameter of the minimize method must be a Python callable.
因此,您可以将损失定义更改为:
Thus, you can change your loss definition to:
def loss():
return tf.square(target_Q - curent_Q)
并在不将其转换为张量的情况下使用它:
and use it without converting it to a Tensor:
self.optimizer.minimize(loss, self.model.trainable_variables)
这篇关于AdamOptimizer 的问题的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!