官方ZeroOut渐变示例错误:AttributeError:'列表'对象没有属性'eval'

问题描述

我遵循了tensorflow网站的官方教程: https://www.tensorflow.org/extend /adding_an_op 在本教程的下面的这段简短代码片段中，我还将尝试介绍如何调用示例ZeroOut的渐变.

我在这里找到了代码: https://github.com/MatteoRagni/tf. ZeroOut.gpu

import numpy as np
import tensorflow as tf
from tensorflow.python.framework import ops
from tensorflow.python.ops import array_ops
from tensorflow.python.ops import sparse_ops
zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
     to_zero = op.inputs[0]
     shape = array_ops.shape(to_zero)
     index = array_ops.zeros_like(shape)
     first_grad = array_ops.reshape(grad, [-1])[0]
     to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)
     return [to_zero_grad]  # List of one Tensor, since we have one input

t_in = tf.placeholder(tf.int32, [None,None])
ret = zero_out_module.zero_out(t_in)
grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session(''):
    feed_dict = {t_in: [[1, 2], [3, 4]]}
    print "ret val: ", ret.eval(feed_dict=feed_dict)
    print "grad: ", grad
    print "grad: ", grad.eval(feed_dict=feed_dict)

我遇到了这个错误...

AttributeError: 'list' object has no attribute 'eval'

...但是我可以做ret.eval().

为什么我不能调用grad.eval()?我想在梯度张量中看到这些值.如何调试渐变?

解决方案

老问题的答案

实施

def _zero_out_grad(op, *grads):
    topdiff = grads[0]
    bottom = op.inputs[0]

    shape = array_ops.shape(bottom)
    index = array_ops.zeros_like(shape)
    first_grad = array_ops.reshape(topdiff, [-1])[0]
    to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)

    return to_zero_grad

在这里工作得很好.您确定在tf.Session()之前执行"@ ops.RegisterGradient("ZeroOut")"吗?

通常是

zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
    # ...

被放置在另一个文件中，并且刚刚被导入. 此处.

回答完全更改的问题

您的渐变函数返回一个列表，而Python列表中没有'eval()'.尝试之一:

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)[0]

或遵循最佳做法并使用

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session() as sess:
    sess.run(grad, feed_dict=feed_dict)

请不要更改您的整个问题

I followed the official tutorial of the tensorflow website: https://www.tensorflow.org/extend/adding_an_opThere is also described how to call the gradient of the example ZeroOut in the tutorial that I want to try in this short code snippet underneath.

I have found the code here: https://github.com/MatteoRagni/tf.ZeroOut.gpu

import numpy as np
import tensorflow as tf
from tensorflow.python.framework import ops
from tensorflow.python.ops import array_ops
from tensorflow.python.ops import sparse_ops
zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
     to_zero = op.inputs[0]
     shape = array_ops.shape(to_zero)
     index = array_ops.zeros_like(shape)
     first_grad = array_ops.reshape(grad, [-1])[0]
     to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)
     return [to_zero_grad]  # List of one Tensor, since we have one input

t_in = tf.placeholder(tf.int32, [None,None])
ret = zero_out_module.zero_out(t_in)
grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session(''):
    feed_dict = {t_in: [[1, 2], [3, 4]]}
    print "ret val: ", ret.eval(feed_dict=feed_dict)
    print "grad: ", grad
    print "grad: ", grad.eval(feed_dict=feed_dict)

I got this error ...

AttributeError: 'list' object has no attribute 'eval'

... but I can do ret.eval().

Why I cant call grad.eval()? I want to see these values inside the grad tensor. How to debug gradient?

解决方案

Answer to old question

The implementation

def _zero_out_grad(op, *grads):
    topdiff = grads[0]
    bottom = op.inputs[0]

    shape = array_ops.shape(bottom)
    index = array_ops.zeros_like(shape)
    first_grad = array_ops.reshape(topdiff, [-1])[0]
    to_zero_grad = sparse_ops.sparse_to_dense([index], shape, first_grad, 0)

    return to_zero_grad

works quite nicely here. Are you sure "@ops.RegisterGradient("ZeroOut")" is executed before the tf.Session()?

Usually the

zero_out_module = tf.load_op_library('./libzeroout.so')
@ops.RegisterGradient("ZeroOut")
def _zero_out_grad(op, grad):
    # ...

is placed in a different file and just imported. A full working example even with the recent TensorFlow version is here.

Answer to completely changed question

Your gradient function returns a list and a Python list has no 'eval()'. Try either:

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)[0]

Or follow best practice and use

grad = tf.gradients(ys=tf.reduce_sum(ret), xs=t_in)
with tf.Session() as sess:
    sess.run(grad, feed_dict=feed_dict)

Please do not change your entire question

这篇关于官方ZeroOut渐变示例错误:AttributeError:'列表'对象没有属性'eval'的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！