问题描述
简而言之:
在pytorch中使用adadelta优化器时，我无法绘制lr/时期曲线，因为 optimizer.param_groups [0] ['lr'] 始终返回相同的值.
I can't draw lr/epoch curve when using adadelta optimizer in pytorch because optimizer.param_groups[0]['lr'] always return the same value.
详细信息:
Adadelta可以仅使用一阶信息就可以动态适应时间变化，并且具有超出香草随机梯度下降的最小计算开销[1].
Adadelta can dynamically adapts over time using only first order information andhas minimal computational overhead beyond vanilla stochastic gradient descent [1].
在pytorch中，Adadelta的源代码在这里 https://pytorch.org/docs/stable/_modules/torch/optim/adadelta.html#Adadelta
In pytorch, the source code of Adadelta is here https://pytorch.org/docs/stable/_modules/torch/optim/adadelta.html#Adadelta
由于不需要手动调整学习率，据我所知，在声明优化器后，我们不必设置任何时间表
Since it requires no manual tuning of learning rate, in my knowledge, we don't have to set any schedular after declare the optimizer
self.optimizer = torch.optim.Adadelta(self.model.parameters()，lr = 1)
检查学习率的方法是
current_lr = self.optimizer.param_groups [0] ['lr']
问题在于它总是返回1(初始lr).
有人可以告诉我如何获得真正的学习率，以便可以画出lr/epch曲线吗?
Could anyone tell me how can I get the true learning rate so that can I draw a lr/epch curve?
[1] https://arxiv.org/pdf/1212.5701.pdf
推荐答案
检查: self.optimizer.state .这使用lr进行了优化，并在优化过程中使用.
Check: self.optimizer.state. This is optimized with the lr and used in optimization process.
在文档中，lr只是:
https://pytorch.org/docs/stable/_modules/torch/optim/adadelta.html
您可能会在self.optimizer.state值中找到acc_delta值，但需要仔细阅读此词典中包含的字典:
Edited: you may find acc_delta values in self.optimizer.state values but you need to go through dictionaries contained by this dictionary:
dict_with_acc_delta = [self.optimizer.state[i] for i in self.optimizer.state.keys() if "acc_delta" in self.optimizer.state[i].keys()]
acc_deltas = [i["acc_delta"] for i in dict_with_acc_delta]
我有八层，acc_deltas列表中的元素形状如下
I have eight layers and shapes of elements in the acc_deltas list are following
[torch.Size([25088]),
torch.Size([25088]),
torch.Size([4096, 25088]),
torch.Size([4096]),
torch.Size([1024, 4096]),
torch.Size([1024]),
torch.Size([102, 1024]),
torch.Size([102])]
这篇关于如何打印“实际"标签pytorch中Adadelta的学习率的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！