我正在尝试用千层面做一个简单的多元线性回归。
这是我的输入:
x_train = np.array([[37.93, 139.5, 329., 16.64,
16.81, 16.57, 1., 707.,
39.72, 149.25, 352.25, 16.61,
16.91, 16.60, 40.11, 151.5,
361.75, 16.95, 16.98, 16.79]]).astype(np.float32)
y_train = np.array([37.92, 138.25, 324.66, 16.28, 16.27, 16.28]).astype(np.float32)
对于这两个数据点,网络应该能够完美地学习
y
。这是模型:
i1 = T.matrix()
y = T.vector()
lay1 = lasagne.layers.InputLayer(shape=(None,20),input_var=i1)
out1 = lasagne.layers.get_output(lay1)
lay2 = lasagne.layers.DenseLayer(lay1, 6, nonlinearity=lasagne.nonlinearities.linear)
out2 = lasagne.layers.get_output(lay2)
params = lasagne.layers.get_all_params(lay2, trainable=True)
cost = T.sum(lasagne.objectives.squared_error(out2, y))
grad = T.grad(cost, params)
updates = lasagne.updates.sgd(grad, params, learning_rate=0.1)
f_train = theano.function([i1, y], [out1, out2, cost], updates=updates)
执行多次后
f_train(x_train,y_train)
成本激增到无限。知道这里出了什么问题吗?
谢谢!
最佳答案
对于单个训练实例,网络具有太多的容量。您将需要应用一些强大的正则化功能,以防止培训出现分歧。另外,希望更现实一些,为其提供更复杂的训练数据(许多实例)。
对于单个实例,仅使用一个输入(而不是20个输入)即可解决任务,并且禁用DenseLayer
的偏差:
import numpy as np
import theano
import lasagne
import theano.tensor as T
def compile():
x, z = T.matrices('x', 'z')
lh = lasagne.layers.InputLayer(shape=(None, 1), input_var=x)
ly = lasagne.layers.DenseLayer(lh, 6, nonlinearity=lasagne.nonlinearities.linear,
b=None)
y = lasagne.layers.get_output(ly)
params = lasagne.layers.get_all_params(ly, trainable=True)
cost = T.sum(lasagne.objectives.squared_error(y, z))
updates = lasagne.updates.sgd(cost, params, learning_rate=0.0001)
return theano.function([x, z], [y, cost], updates=updates)
def main():
f_train = compile()
x_train = np.array([[37.93]]).astype(theano.config.floatX)
y_train = np.array([[37.92, 138.25, 324.66, 16.28, 16.27, 16.28]])\
.astype(theano.config.floatX)
for _ in xrange(100):
print f_train(x_train, y_train)
main()
注意,学习速度也需要大量降低以防止发散。
关于python - 线性回归千层面/Theano,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/34830499/