我一直在尝试完成Andrew Ng的机器学习课程，现在我参与逻辑回归。我正在尝试发现参数，并在不使用MATLAB函数fminunc的情况下计算成本。但是，我没有收敛于使用fminunc完成作业的其他学生所发布的正确结果。具体来说，我的问题是：
参数theta不正确
我的成本似乎在上涨
我的成本向量中有很多NaN（我只是创建了一个成本向量以进行跟踪）
我尝试通过“梯度下降”来发现参数，就像我对内容的理解一样。但是，我的实现似乎仍然给我错误的结果。
dataset = load('dataStuds.txt');
x = dataset(:,1:end-1);
y = dataset(:,end);
m = length(x);
% Padding the the 1's (intercept term, the call it?)
x = [ones(length(x),1), x];
thetas = zeros(size(x,2),1);
% Setting the learning rate to 0.1
alpha = 0.1;
for i = 1:100000
% theta transpose x (tho why in MATLAB it needs to be done the other way
% round? :)
ttrx = x * thetas;
% the hypothesis function h_x = g(z) = sigmoid(-z)
h_x = 1 ./ (1 + exp(-ttrx));
error = h_x - y;
% the gradient (aka the derivative of J(\theta) aka the derivative
% term)
for j = 1:length(thetas)
gradient = 1/m * (h_x - y)' * x(:,j);
% Updating the parameters theta
thetas(j) = thetas(j) - alpha * gradient;
end
% Calculating the cost, just to keep track...
cost(i) = 1/m * ( -y' * log(h_x) - (1-y)' * log(1-h_x) );
end
% Displaying the final theta's that I obtained
thetas
我得到的参数
theta是：thetas =
-482.8509
3.7457
2.6976
下面的结果来自我下载的一个示例，但是作者为此使用了fminunc。
Cost at theta found by fminunc: 0.203506
theta:
-24.932760
0.204406
0.199616
数据：
34.6236596245170 78.0246928153624 0
30.2867107682261 43.8949975240010 0
35.8474087699387 72.9021980270836 0
60.1825993862098 86.3085520954683 1
79.0327360507101 75.3443764369103 1
45.0832774766834 56.3163717815305 0
61.1066645368477 96.5114258848962 1
75.0247455673889 46.5540135411654 1
76.0987867022626 87.4205697192680 1
84.4328199612004 43.5333933107211 1
95.8615550709357 38.2252780579509 0
75.0136583895825 30.6032632342801 0
82.3070533739948 76.4819633023560 1
69.3645887597094 97.7186919618861 1
39.5383391436722 76.0368108511588 0
53.9710521485623 89.2073501375021 1
69.0701440628303 52.7404697301677 1
67.9468554771162 46.6785741067313 0
70.6615095549944 92.9271378936483 1
76.9787837274750 47.5759636497553 1
67.3720275457088 42.8384383202918 0
89.6767757507208 65.7993659274524 1
50.5347882898830 48.8558115276421 0
34.2120609778679 44.2095285986629 0
77.9240914545704 68.9723599933059 1
62.2710136700463 69.9544579544759 1
80.1901807509566 44.8216289321835 1
93.1143887974420 38.8006703371321 0
61.8302060231260 50.2561078924462 0
38.7858037967942 64.9956809553958 0
61.3792894474250 72.8078873131710 1
85.4045193941165 57.0519839762712 1
52.1079797319398 63.1276237688172 0
52.0454047683183 69.4328601204522 1
40.2368937354511 71.1677480218488 0
54.6351055542482 52.2138858806112 0
33.9155001090689 98.8694357422061 0
64.1769888749449 80.9080605867082 1
74.7892529594154 41.5734152282443 0
34.1836400264419 75.2377203360134 0
83.9023936624916 56.3080462160533 1
51.5477202690618 46.8562902634998 0
94.4433677691785 65.5689216055905 1
82.3687537571392 40.6182551597062 0
51.0477517712887 45.8227014577600 0
62.2226757612019 52.0609919483668 0
77.1930349260136 70.4582000018096 1
97.7715992800023 86.7278223300282 1
62.0730637966765 96.7688241241398 1
91.5649744980744 88.6962925454660 1
79.9448179406693 74.1631193504376 1
99.2725269292572 60.9990309984499 1
90.5467141139985 43.3906018065003 1
34.5245138532001 60.3963424583717 0
50.2864961189907 49.8045388132306 0
49.5866772163203 59.8089509945327 0
97.6456339600777 68.8615727242060 1
32.5772001680931 95.5985476138788 0
74.2486913672160 69.8245712265719 1
71.7964620586338 78.4535622451505 1
75.3956114656803 85.7599366733162 1
35.2861128152619 47.0205139472342 0
56.2538174971162 39.2614725105802 0
30.0588224466980 49.5929738672369 0
44.6682617248089 66.4500861455891 0
66.5608944724295 41.0920980793697 0
40.4575509837516 97.5351854890994 1
49.0725632190884 51.8832118207397 0
80.2795740146700 92.1160608134408 1
66.7467185694404 60.9913940274099 1
32.7228330406032 43.3071730643006 0
64.0393204150601 78.0316880201823 1
72.3464942257992 96.2275929676140 1
60.4578857391896 73.0949980975804 1
58.8409562172680 75.8584483127904 1
99.8278577969213 72.3692519338389 1
47.2642691084817 88.4758649955978 1
50.4581598028599 75.8098595298246 1
60.4555562927153 42.5084094357222 0
82.2266615778557 42.7198785371646 0
88.9138964166533 69.8037888983547 1
94.8345067243020 45.6943068025075 1
67.3192574691753 66.5893531774792 1
57.2387063156986 59.5142819801296 1
80.3667560017127 90.9601478974695 1
68.4685217859111 85.5943071045201 1
42.0754545384731 78.8447860014804 0
75.4777020053391 90.4245389975396 1
78.6354243489802 96.6474271688564 1
52.3480039879411 60.7695052560259 0
94.0943311251679 77.1591050907389 1
90.4485509709636 87.5087917648470 1
55.4821611406959 35.5707034722887 0
74.4926924184304 84.8451368493014 1
89.8458067072098 45.3582836109166 1
83.4891627449824 48.3802857972818 1
42.2617008099817 87.1038509402546 1
99.3150088051039 68.7754094720662 1
55.3400175600370 64.9319380069486 1
74.7758930009277 89.5298128951328 1
最佳答案
我运行了您的代码，它运行正常。但是，关于梯度下降的棘手事情是确保您的成本不会分散到无穷大。如果查看成本阵列，您会发现成本肯定有差异，这就是为什么您未获得正确结果的原因。
消除这种情况的最佳方法是降低学习率。通过实验，我发现alpha = 0.003的学习率最适合您的问题。我还将迭代次数增加到200000。更改这两件事为我提供了以下参数和相关成本：
>> format long g;
>> thetas
thetas =
-17.6287417780435
0.146062780453677
0.140513170941357
>> cost(end)
ans =
0.214821863463963
这或多或少与您使用
fminunc时看到的参数大小一致。但是，由于实际的最小化方法本身，它们得到的参数略有不同，成本也有所不同。 fminunc使用L-BFGS的变体，可以更快地找到解决方案。最重要的是实际精度本身。请记住，要对示例属于标签0还是1进行分类，请对参数和示例进行加权求和，然后通过S型函数和0.5的阈值进行运算。我们找到每个预期标签和预期标签匹配的平均次数。
使用我们在梯度下降中发现的参数，可以得到以下精度：
>> ttrx = x * thetas;
>> h_x = 1 ./ (1 + exp(-ttrx)) >= 0.5;
>> mean(h_x == y)
ans =
0.89
这意味着我们已经达到89％的分类准确率。使用
fminunc提供的标签还可以得到：>> thetas2 = [-24.932760; 0.204406; 0.199616];
>> ttrx = x * thetas2;
>> h_x = 1 ./ (1 + exp(-ttrx)) >= 0.5;
>> mean(h_x == y)
ans =
0.89
因此，我们可以看到精度是相同的，因此我不必担心参数的大小，但是它与我们比较两个实现之间的成本时所看到的更加一致。
最后，我建议您查看我的这篇文章，以获取有关如何使Logistic回归长期运行的一些技巧。我绝对建议在查找参数之前先对功能进行标准化，以使算法运行更快。它还解决了为什么您找到错误的参数（即成本增加）的原因：Cost function in logistic regression gives NaN as a result。
关于matlab - 吴恩达的Logistic回归可以轻松执行，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/49315963/