本文介绍了如果索引叶变量用于梯度更新,如何解决就地操作错误?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
当我尝试索引叶子变量以使用自定义的收缩功能更新渐变时,遇到了就地操作错误.我无法解决.非常感谢您的帮助!
I am encountering In place operation error when I am trying to index a leaf variable to update gradients with customized Shrink function. I cannot work around it. Any help is highly appreciated!
import torch.nn as nn
import torch
import numpy as np
from torch.autograd import Variable, Function
# hyper parameters
batch_size = 100 # batch size of images
ld = 0.2 # sparse penalty
lr = 0.1 # learning rate
x = Variable(torch.from_numpy(np.random.normal(0,1,(batch_size,10,10))), requires_grad=False) # original
# depends on size of the dictionary, number of atoms.
D = Variable(torch.from_numpy(np.random.normal(0,1,(500,10,10))), requires_grad=True)
# hx sparse representation
ht = Variable(torch.from_numpy(np.random.normal(0,1,(batch_size,500,1,1))), requires_grad=True)
# Dictionary loss function
loss = nn.MSELoss()
# customized shrink function to update gradient
shrink_ht = lambda x: torch.stack([torch.sign(i)*torch.max(torch.abs(i)-lr*ld,0)[0] for i in x])
### sparse reprsentation optimizer_ht single image.
optimizer_ht = torch.optim.SGD([ht], lr=lr, momentum=0.9) # optimizer for sparse representation
## update for the batch
for idx in range(len(x)):
optimizer_ht.zero_grad() # clear up gradients
loss_ht = 0.5*torch.norm((x[idx]-(D*ht[idx]).sum(dim=0)),p=2)**2
loss_ht.backward() # back propogation and calculate gradients
optimizer_ht.step() # update parameters with gradients
ht[idx] = shrink_ht(ht[idx]) # customized shrink function.
RuntimeError Traceback (most recent call last) in ()
15 loss_ht.backward() # back propogation and calculate gradients
16 optimizer_ht.step() # update parameters with gradients
—> 17 ht[idx] = shrink_ht(ht[idx]) # customized shrink function.
18
19
/home/miniconda3/lib/python3.6/site-packages/torch/autograd/variable.py in setitem(self, key, value)
85 return MaskedFill.apply(self, key, value, True)
86 else:
—> 87 return SetItem.apply(self, key, value)
88
89 def deepcopy(self, memo):
RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.
具体来说,下面的这一行代码似乎在给出错误,因为它同时索引并更新了叶子变量.
Specifically, this line of code below seems give error as it index and update leaf variable at the same time.
ht[idx] = shrink_ht(ht[idx]) # customized shrink function.
谢谢.
W.S.
推荐答案
我刚刚发现:要更新变量,应使用ht.data [idx].使用数据直接访问张量.
I just find out: to update the Variable, should use ht.data[idx]. use data to access tensor directly.
这篇关于如果索引叶变量用于梯度更新,如何解决就地操作错误?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!