【第二课】kaggle案例分析二

Evernote Export

推荐系统比赛(常见比赛)

推荐系统分类

最能变现的机器学习应用

• 基于应用领域分类：电子商务推荐，社交好友推荐，搜索引擎推荐，信息内容推荐等
• **基于设计思想：**基于协同过滤的推荐，基于内容的推荐，基于知识的推荐，混合推荐
• 基于使用何种数据：基于用户行为数据的推荐，基于用户标签的推荐，基于社交网络数据，基于上下信息(时间上下文，地点上下文等)
  示例说明：Netflix比赛
• Netflix是一家在线影片租赁提供商。公司能够提供超大数量的DVD，而且能够让顾客快速方便的挑选影片，同时免费递送。Netflix已经连续五次被评为顾客最满意的网站，可以通过PC，TV及iPad、iphone等收看电视节目
• Netflix大奖赛从2006年10月份开始，Netfilx公开了大约1个亿1-5的匿名影片评级。数据集仅包含了影片名称，评价星级和评级日期，没有任何文本评价的内容。比赛要求参赛者预测Netflix的客户分别喜欢什么影片，要把预测的效率提高10%以上。
• 对推荐算法发展有着深远影响，比如对LFM的追捧使其快速进入大众视野，对LFM提出了很多改进方法。

推荐系统的算法

• 基于物品的协同过滤算法ItemCF：基于item的系统算法，通过用户对不同item的评分来评测item之间的相似性，基于item之间的相似性做出推荐
• 简单来说就是：给用户推荐他之前喜欢的物品或相似的物品
• IitemCF算例：
• 基于ItemCF的原理和基于UserCF类似，只是在计算邻居时采用物品本身，而不是从用户的角度，即基于用户对于物品的偏好找到类似的物品，然后根据用户的历史偏好，推荐相似的物品给他。
• 从计算的角度看，就是将所有用户对某个物品的偏好作为一个向量来计算物品之间的相似度，得到物品的相似物品之后，根据用户历史的偏好预测当前用户还没有表示偏好的物品，计算得到一个排序的物品列表作为推荐。
• 同现矩阵X评分矩阵=推荐结果
• 本质是猜测评分矩阵的空缺元素
• LFM(latent factor model)
• 关键公式
  LFM通过如下公式计算用户u对物品的兴趣

Preference(u,i)=rw​i=pMT​qi​=f=1∑F​PM​qi​,k

这个公式中，Pw​,k和qi​,k是模型的参数，其中Pw​,k度量了用户u的兴趣和第k个隐类的关系，而qi​,k度量了第k个隐类和物品i之间的关系。那么，下面的问题就是如何计算这两个参数。
选择一个训练集，用适当的数学算法计算出上述公式里待确定的权重

• 源于对SVD(奇异值分解)方法的改进，传统SVD是线性代数典型问题
• SVD可用于推荐系统评分矩阵补全，但是由于计算量太大，实际上只是适用于规模很小的系统
• Simon Funk改进SVD(Funk-SVD),后来被称为Latent Factor Model
• 常见同类问题求解思路
• 最大似然(类似训练贝叶斯信念网络和EM算法)
• 转化为求解损失函数极小值问题

C=(u,i)ϵK∑​(rui​−r^ui​)2=(u,i)ϵK∑​(rui​−f=1∑F​Pu,k​qi,k​)2+λ∣∣pu​∣∣2+λ∣∣qi​∣∣2

LFM损失函数极值用梯度下降法求解
上面定义的损失函数里有两组参数Puf​和qif​，随机梯度下降法需要首先对它们进行分贝求偏导数

∂pik​∂C​=−2qik​+2λpuk​

∂qik​∂C​=−2puk​+2λqik​

然后，根据随机梯度下降法，需要将参数沿着最速下降方向向前推进，因此可以得到如下递推公式：

Puk​=Puk​+α(qik​−λpuk​)

qik​=qik​+α(puk​−λqik​)

其中,α是学习速率(learning rate)，它的选取需要通过反复实现获得

%23%23%23%20%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E6%AF%94%E8%B5%9B(%E5%B8%B8%E8%A7%81%E6%AF%94%E8%B5%9B)%0A%23%23%23%23%20%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E5%88%86%E7%B1%BB%0A**%E6%9C%80%E8%83%BD%E5%8F%98%E7%8E%B0%E7%9A%84%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%BA%94%E7%94%A8**%0A*%20%E5%9F%BA%E4%BA%8E%E5%BA%94%E7%94%A8%E9%A2%86%E5%9F%9F%E5%88%86%E7%B1%BB%EF%BC%9A%E7%94%B5%E5%AD%90%E5%95%86%E5%8A%A1%E6%8E%A8%E8%8D%90%EF%BC%8C%E7%A4%BE%E4%BA%A4%E5%A5%BD%E5%8F%8B%E6%8E%A8%E8%8D%90%EF%BC%8C%E6%90%9C%E7%B4%A2%E5%BC%95%E6%93%8E%E6%8E%A8%E8%8D%90%EF%BC%8C%E4%BF%A1%E6%81%AF%E5%86%85%E5%AE%B9%E6%8E%A8%E8%8D%90%E7%AD%89%0A*%20**%E5%9F%BA%E4%BA%8E%E8%AE%BE%E8%AE%A1%E6%80%9D%E6%83%B3%EF%BC%9A**%E5%9F%BA%E4%BA%8E%E5%8D%8F%E5%90%8C%E8%BF%87%E6%BB%A4%E7%9A%84%E6%8E%A8%E8%8D%90%EF%BC%8C%E5%9F%BA%E4%BA%8E%E5%86%85%E5%AE%B9%E7%9A%84%E6%8E%A8%E8%8D%90%EF%BC%8C%E5%9F%BA%E4%BA%8E%E7%9F%A5%E8%AF%86%E7%9A%84%E6%8E%A8%E8%8D%90%EF%BC%8C%E6%B7%B7%E5%90%88%E6%8E%A8%E8%8D%90%0A*%20%E5%9F%BA%E4%BA%8E%E4%BD%BF%E7%94%A8%E4%BD%95%E7%A7%8D%E6%95%B0%E6%8D%AE%EF%BC%9A%E5%9F%BA%E4%BA%8E%E7%94%A8%E6%88%B7%E8%A1%8C%E4%B8%BA%E6%95%B0%E6%8D%AE%E7%9A%84%E6%8E%A8%E8%8D%90%EF%BC%8C%E5%9F%BA%E4%BA%8E%E7%94%A8%E6%88%B7%E6%A0%87%E7%AD%BE%E7%9A%84%E6%8E%A8%E8%8D%90%EF%BC%8C%E5%9F%BA%E4%BA%8E%E7%A4%BE%E4%BA%A4%E7%BD%91%E7%BB%9C%E6%95%B0%E6%8D%AE%EF%BC%8C%E5%9F%BA%E4%BA%8E%E4%B8%8A%E4%B8%8B%E4%BF%A1%E6%81%AF(%E6%97%B6%E9%97%B4%E4%B8%8A%E4%B8%8B%E6%96%87%EF%BC%8C%E5%9C%B0%E7%82%B9%E4%B8%8A%E4%B8%8B%E6%96%87%E7%AD%89)%0A%E7%A4%BA%E4%BE%8B%E8%AF%B4%E6%98%8E%EF%BC%9A**Netflix%E6%AF%94%E8%B5%9B**%0A*%20Netflix%E6%98%AF%E4%B8%80%E5%AE%B6%E5%9C%A8%E7%BA%BF%E5%BD%B1%E7%89%87%E7%A7%9F%E8%B5%81%E6%8F%90%E4%BE%9B%E5%95%86%E3%80%82%E5%85%AC%E5%8F%B8%E8%83%BD%E5%A4%9F%E6%8F%90%E4%BE%9B%E8%B6%85%E5%A4%A7%E6%95%B0%E9%87%8F%E7%9A%84DVD%EF%BC%8C%E8%80%8C%E4%B8%94%E8%83%BD%E5%A4%9F%E8%AE%A9%E9%A1%BE%E5%AE%A2%E5%BF%AB%E9%80%9F%E6%96%B9%E4%BE%BF%E7%9A%84%E6%8C%91%E9%80%89%E5%BD%B1%E7%89%87%EF%BC%8C%E5%90%8C%E6%97%B6%E5%85%8D%E8%B4%B9%E9%80%92%E9%80%81%E3%80%82Netflix%E5%B7%B2%E7%BB%8F%E8%BF%9E%E7%BB%AD%E4%BA%94%E6%AC%A1%E8%A2%AB%E8%AF%84%E4%B8%BA%E9%A1%BE%E5%AE%A2%E6%9C%80%E6%BB%A1%E6%84%8F%E7%9A%84%E7%BD%91%E7%AB%99%EF%BC%8C%E5%8F%AF%E4%BB%A5%E9%80%9A%E8%BF%87PC%EF%BC%8CTV%E5%8F%8AiPad%E3%80%81iphone%E7%AD%89%E6%94%B6%E7%9C%8B%E7%94%B5%E8%A7%86%E8%8A%82%E7%9B%AE%0A*%20Netflix%E5%A4%A7%E5%A5%96%E8%B5%9B%E4%BB%8E2006%E5%B9%B410%E6%9C%88%E4%BB%BD%E5%BC%80%E5%A7%8B%EF%BC%8CNetfilx%E5%85%AC%E5%BC%80%E4%BA%86%E5%A4%A7%E7%BA%A61%E4%B8%AA%E4%BA%BF1-5%E7%9A%84%E5%8C%BF%E5%90%8D%E5%BD%B1%E7%89%87%E8%AF%84%E7%BA%A7%E3%80%82%E6%95%B0%E6%8D%AE%E9%9B%86%E4%BB%85%E5%8C%85%E5%90%AB%E4%BA%86%E5%BD%B1%E7%89%87%E5%90%8D%E7%A7%B0%EF%BC%8C%E8%AF%84%E4%BB%B7%E6%98%9F%E7%BA%A7%E5%92%8C%E8%AF%84%E7%BA%A7%E6%97%A5%E6%9C%9F%EF%BC%8C%E6%B2%A1%E6%9C%89%E4%BB%BB%E4%BD%95%E6%96%87%E6%9C%AC%E8%AF%84%E4%BB%B7%E7%9A%84%E5%86%85%E5%AE%B9%E3%80%82%E6%AF%94%E8%B5%9B%E8%A6%81%E6%B1%82%E5%8F%82%E8%B5%9B%E8%80%85%E9%A2%84%E6%B5%8BNetflix%E7%9A%84%E5%AE%A2%E6%88%B7%E5%88%86%E5%88%AB%E5%96%9C%E6%AC%A2%E4%BB%80%E4%B9%88%E5%BD%B1%E7%89%87%EF%BC%8C%E8%A6%81%E6%8A%8A%E9%A2%84%E6%B5%8B%E7%9A%84%E6%95%88%E7%8E%87%E6%8F%90%E9%AB%9810%25%E4%BB%A5%E4%B8%8A%E3%80%82%0A*%20%E5%AF%B9%E6%8E%A8%E8%8D%90%E7%AE%97%E6%B3%95%E5%8F%91%E5%B1%95%E6%9C%89%E7%9D%80%E6%B7%B1%E8%BF%9C%E5%BD%B1%E5%93%8D%EF%BC%8C%E6%AF%94%E5%A6%82%E5%AF%B9LFM%E7%9A%84%E8%BF%BD%E6%8D%A7%E4%BD%BF%E5%85%B6%E5%BF%AB%E9%80%9F%E8%BF%9B%E5%85%A5%E5%A4%A7%E4%BC%97%E8%A7%86%E9%87%8E%EF%BC%8C%E5%AF%B9LFM%E6%8F%90%E5%87%BA%E4%BA%86%E5%BE%88%E5%A4%9A%E6%94%B9%E8%BF%9B%E6%96%B9%E6%B3%95%E3%80%82%0A%0A**%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E7%9A%84%E7%AE%97%E6%B3%95**%0A*%20%E5%9F%BA%E4%BA%8E%E7%89%A9%E5%93%81%E7%9A%84%E5%8D%8F%E5%90%8C%E8%BF%87%E6%BB%A4%E7%AE%97%E6%B3%95**ItemCF**%EF%BC%9A%E5%9F%BA%E4%BA%8Eitem%E7%9A%84%E7%B3%BB%E7%BB%9F%E7%AE%97%E6%B3%95%EF%BC%8C%E9%80%9A%E8%BF%87%E7%94%A8%E6%88%B7%E5%AF%B9%E4%B8%8D%E5%90%8Citem%E7%9A%84%E8%AF%84%E5%88%86%E6%9D%A5%E8%AF%84%E6%B5%8Bitem%E4%B9%8B%E9%97%B4%E7%9A%84%E7%9B%B8%E4%BC%BC%E6%80%A7%EF%BC%8C%E5%9F%BA%E4%BA%8Eitem%E4%B9%8B%E9%97%B4%E7%9A%84%E7%9B%B8%E4%BC%BC%E6%80%A7%E5%81%9A%E5%87%BA%E6%8E%A8%E8%8D%90%0A*%20%E7%AE%80%E5%8D%95%E6%9D%A5%E8%AF%B4%E5%B0%B1%E6%98%AF%EF%BC%9A%E7%BB%99%E7%94%A8%E6%88%B7%E6%8E%A8%E8%8D%90%E4%BB%96%E4%B9%8B%E5%89%8D%E5%96%9C%E6%AC%A2%E7%9A%84%E7%89%A9%E5%93%81%E6%88%96%E7%9B%B8%E4%BC%BC%E7%9A%84%E7%89%A9%E5%93%81%0A*%20IitemCF%E7%AE%97%E4%BE%8B%EF%BC%9A%0A*%20%E5%9F%BA%E4%BA%8EItemCF%E7%9A%84%E5%8E%9F%E7%90%86%E5%92%8C%E5%9F%BA%E4%BA%8EUserCF%E7%B1%BB%E4%BC%BC%EF%BC%8C%E5%8F%AA%E6%98%AF%E5%9C%A8%E8%AE%A1%E7%AE%97%E9%82%BB%E5%B1%85%E6%97%B6%E9%87%87%E7%94%A8%E7%89%A9%E5%93%81%E6%9C%AC%E8%BA%AB%EF%BC%8C%E8%80%8C%E4%B8%8D%E6%98%AF%E4%BB%8E%E7%94%A8%E6%88%B7%E7%9A%84%E8%A7%92%E5%BA%A6%EF%BC%8C%E5%8D%B3%E5%9F%BA%E4%BA%8E%E7%94%A8%E6%88%B7%E5%AF%B9%E4%BA%8E%E7%89%A9%E5%93%81%E7%9A%84%E5%81%8F%E5%A5%BD%E6%89%BE%E5%88%B0%E7%B1%BB%E4%BC%BC%E7%9A%84%E7%89%A9%E5%93%81%EF%BC%8C%E7%84%B6%E5%90%8E%E6%A0%B9%E6%8D%AE%E7%94%A8%E6%88%B7%E7%9A%84%E5%8E%86%E5%8F%B2%E5%81%8F%E5%A5%BD%EF%BC%8C%E6%8E%A8%E8%8D%90%E7%9B%B8%E4%BC%BC%E7%9A%84%E7%89%A9%E5%93%81%E7%BB%99%E4%BB%96%E3%80%82%0A*%20%E4%BB%8E%E8%AE%A1%E7%AE%97%E7%9A%84%E8%A7%92%E5%BA%A6%E7%9C%8B%EF%BC%8C%E5%B0%B1%E6%98%AF%E5%B0%86%E6%89%80%E6%9C%89%E7%94%A8%E6%88%B7%E5%AF%B9%E6%9F%90%E4%B8%AA%E7%89%A9%E5%93%81%E7%9A%84%E5%81%8F%E5%A5%BD%E4%BD%9C%E4%B8%BA%E4%B8%80%E4%B8%AA%E5%90%91%E9%87%8F%E6%9D%A5%E8%AE%A1%E7%AE%97%E7%89%A9%E5%93%81%E4%B9%8B%E9%97%B4%E7%9A%84%E7%9B%B8%E4%BC%BC%E5%BA%A6%EF%BC%8C%E5%BE%97%E5%88%B0%E7%89%A9%E5%93%81%E7%9A%84%E7%9B%B8%E4%BC%BC%E7%89%A9%E5%93%81%E4%B9%8B%E5%90%8E%EF%BC%8C%E6%A0%B9%E6%8D%AE%E7%94%A8%E6%88%B7%E5%8E%86%E5%8F%B2%E7%9A%84%E5%81%8F%E5%A5%BD%E9%A2%84%E6%B5%8B%E5%BD%93%E5%89%8D%E7%94%A8%E6%88%B7%E8%BF%98%E6%B2%A1%E6%9C%89%E8%A1%A8%E7%A4%BA%E5%81%8F%E5%A5%BD%E7%9A%84%E7%89%A9%E5%93%81%EF%BC%8C%E8%AE%A1%E7%AE%97%E5%BE%97%E5%88%B0%E4%B8%80%E4%B8%AA%E6%8E%92%E5%BA%8F%E7%9A%84%E7%89%A9%E5%93%81%E5%88%97%E8%A1%A8%E4%BD%9C%E4%B8%BA%E6%8E%A8%E8%8D%90%E3%80%82%0A*%20%E5%90%8C%E7%8E%B0%E7%9F%A9%E9%98%B5X%E8%AF%84%E5%88%86%E7%9F%A9%E9%98%B5%3D%E6%8E%A8%E8%8D%90%E7%BB%93%E6%9E%9C%0A*%20%E6%9C%AC%E8%B4%A8%E6%98%AF%E7%8C%9C%E6%B5%8B%E8%AF%84%E5%88%86%E7%9F%A9%E9%98%B5%E7%9A%84%E7%A9%BA%E7%BC%BA%E5%85%83%E7%B4%A0%0A*%20LFM(latent%20factor%20model)%0A*%20**%E5%85%B3%E9%94%AE%E5%85%AC%E5%BC%8F**%0ALFM%E9%80%9A%E8%BF%87%E5%A6%82%E4%B8%8B%E5%85%AC%E5%BC%8F%E8%AE%A1%E7%AE%97%E7%94%A8%E6%88%B7u%E5%AF%B9%E7%89%A9%E5%93%81%E7%9A%84%E5%85%B4%E8%B6%A3%0A%24%24Preference(u%2Ci)%3Dr_wi%3Dp%5ET_Mq_i%3D%5Csum%5EF_%7Bf%3D1%7DP_Mq_i%2Ck%24%24%0A%E8%BF%99%E4%B8%AA%E5%85%AC%E5%BC%8F%E4%B8%AD%EF%BC%8C%24P_w%2Ck%24%E5%92%8C%24q_i%2Ck%24%E6%98%AF%E6%A8%A1%E5%9E%8B%E7%9A%84%E5%8F%82%E6%95%B0%EF%BC%8C%E5%85%B6%E4%B8%AD%24P_w%2Ck%24%E5%BA%A6%E9%87%8F%E4%BA%86%E7%94%A8%E6%88%B7u%E7%9A%84%E5%85%B4%E8%B6%A3%E5%92%8C%E7%AC%ACk%E4%B8%AA%E9%9A%90%E7%B1%BB%E7%9A%84%E5%85%B3%E7%B3%BB%EF%BC%8C%E8%80%8C%24q_i%2Ck%24%E5%BA%A6%E9%87%8F%E4%BA%86%E7%AC%ACk%E4%B8%AA%E9%9A%90%E7%B1%BB%E5%92%8C%E7%89%A9%E5%93%81i%E4%B9%8B%E9%97%B4%E7%9A%84%E5%85%B3%E7%B3%BB%E3%80%82%E9%82%A3%E4%B9%88%EF%BC%8C%E4%B8%8B%E9%9D%A2%E7%9A%84%E9%97%AE%E9%A2%98%E5%B0%B1%E6%98%AF%E5%A6%82%E4%BD%95%E8%AE%A1%E7%AE%97%E8%BF%99%E4%B8%A4%E4%B8%AA%E5%8F%82%E6%95%B0%E3%80%82%0A**%E9%80%89%E6%8B%A9%E4%B8%80%E4%B8%AA%E8%AE%AD%E7%BB%83%E9%9B%86%EF%BC%8C%E7%94%A8%E9%80%82%E5%BD%93%E7%9A%84%E6%95%B0%E5%AD%A6%E7%AE%97%E6%B3%95%E8%AE%A1%E7%AE%97%E5%87%BA%E4%B8%8A%E8%BF%B0%E5%85%AC%E5%BC%8F%E9%87%8C%E5%BE%85%E7%A1%AE%E5%AE%9A%E7%9A%84%E6%9D%83%E9%87%8D**%0A*%20%E6%BA%90%E4%BA%8E%E5%AF%B9SVD(%E5%A5%87%E5%BC%82%E5%80%BC%E5%88%86%E8%A7%A3)%E6%96%B9%E6%B3%95%E7%9A%84%E6%94%B9%E8%BF%9B%EF%BC%8C%E4%BC%A0%E7%BB%9FSVD%E6%98%AF%E7%BA%BF%E6%80%A7%E4%BB%A3%E6%95%B0%E5%85%B8%E5%9E%8B%E9%97%AE%E9%A2%98%0A*%20SVD%E5%8F%AF%E7%94%A8%E4%BA%8E%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E8%AF%84%E5%88%86%E7%9F%A9%E9%98%B5%E8%A1%A5%E5%85%A8%EF%BC%8C%E4%BD%86%E6%98%AF%E7%94%B1%E4%BA%8E%E8%AE%A1%E7%AE%97%E9%87%8F%E5%A4%AA%E5%A4%A7%EF%BC%8C%E5%AE%9E%E9%99%85%E4%B8%8A%E5%8F%AA%E6%98%AF%E9%80%82%E7%94%A8%E4%BA%8E%E8%A7%84%E6%A8%A1%E5%BE%88%E5%B0%8F%E7%9A%84%E7%B3%BB%E7%BB%9F%0A*%20Simon%20Funk%E6%94%B9%E8%BF%9BSVD(Funk-SVD)%2C%E5%90%8E%E6%9D%A5%E8%A2%AB%E7%A7%B0%E4%B8%BALatent%20Factor%20Model%0A*%20%E5%B8%B8%E8%A7%81%E5%90%8C%E7%B1%BB%E9%97%AE%E9%A2%98%E6%B1%82%E8%A7%A3%E6%80%9D%E8%B7%AF%0A*%20%E6%9C%80%E5%A4%A7%E4%BC%BC%E7%84%B6(%E7%B1%BB%E4%BC%BC%E8%AE%AD%E7%BB%83%E8%B4%9D%E5%8F%B6%E6%96%AF%E4%BF%A1%E5%BF%B5%E7%BD%91%E7%BB%9C%E5%92%8CEM%E7%AE%97%E6%B3%95)%0A*%20%E8%BD%AC%E5%8C%96%E4%B8%BA%E6%B1%82%E8%A7%A3%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E6%9E%81%E5%B0%8F%E5%80%BC%E9%97%AE%E9%A2%98%0A%0A%24%24C%3D%20%5Csum_%7B(u%2Ci)%20%5Cepsilon%20K%7D(r_%7Bui%7D-%5Chat%20r_%7Bui%7D)%5E2%3D%5Csum_%7B(u%2Ci)%5Cepsilon%20K%7D(r_%7Bui%7D-%5Csum%5EF_%7Bf%3D1%7DP_%7Bu%2Ck%7Dq_%7Bi%2Ck%7D)%5E2%2B%5Clambda%7C%7Cp_u%7C%7C%5E2%2B%5Clambda%7C%7Cq_i%7C%7C%5E2%24%24%0A**LFM%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E6%9E%81%E5%80%BC%E7%94%A8%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%E6%B1%82%E8%A7%A3**%0A%E4%B8%8A%E9%9D%A2%E5%AE%9A%E4%B9%89%E7%9A%84%E6%8D%9F%E5%A4%B1%E5%87%BD%E6%95%B0%E9%87%8C%E6%9C%89%E4%B8%A4%E7%BB%84%E5%8F%82%E6%95%B0%24P_%7Buf%7D%24%E5%92%8C%24q_%7Bif%7D%24%EF%BC%8C%E9%9A%8F%E6%9C%BA%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%E9%9C%80%E8%A6%81%E9%A6%96%E5%85%88%E5%AF%B9%E5%AE%83%E4%BB%AC%E8%BF%9B%E8%A1%8C%E5%88%86%E8%B4%9D%E6%B1%82%E5%81%8F%E5%AF%BC%E6%95%B0%0A%24%24%5Cfrac%7B%5Cpartial%20C%7D%7B%5Cpartial%20p_%7Bik%7D%7D%20%3D%20-2q_%7Bik%7D%2B2%5Clambda%20p_%7Buk%7D%24%24%0A%24%24%5Cfrac%7B%5Cpartial%20C%7D%7B%5Cpartial%20q_%7Bik%7D%7D%3D%20-2p_%7Buk%7D%2B2%5Clambda%20q_%7Bik%7D%24%24%0A%E7%84%B6%E5%90%8E%EF%BC%8C%E6%A0%B9%E6%8D%AE%E9%9A%8F%E6%9C%BA%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%EF%BC%8C%E9%9C%80%E8%A6%81%E5%B0%86%E5%8F%82%E6%95%B0%E6%B2%BF%E7%9D%80%E6%9C%80%E9%80%9F%E4%B8%8B%E9%99%8D%E6%96%B9%E5%90%91%E5%90%91%E5%89%8D%E6%8E%A8%E8%BF%9B%EF%BC%8C%E5%9B%A0%E6%AD%A4%E5%8F%AF%E4%BB%A5%E5%BE%97%E5%88%B0%E5%A6%82%E4%B8%8B%E9%80%92%E6%8E%A8%E5%85%AC%E5%BC%8F%EF%BC%9A%0A%24%24P_%7Buk%7D%3DP_%7Buk%7D%2B%5Calpha%20(q_%7Bik%7D-%5Clambda%20p_%7Buk%7D)%24%24%0A%24%24q_%7Bik%7D%3Dq_%7Bik%7D%2B%5Calpha(p_%7Buk%7D-%5Clambda%20q_%7Bik%7D)%24%24%0A%E5%85%B6%E4%B8%AD%2C%24%5Calpha%24%E6%98%AF%E5%AD%A6%E4%B9%A0%E9%80%9F%E7%8E%87(learning%20rate)%EF%BC%8C%E5%AE%83%E7%9A%84%E9%80%89%E5%8F%96%E9%9C%80%E8%A6%81%E9%80%9A%E8%BF%87%E5%8F%8D%E5%A4%8D%E5%AE%9E%E7%8E%B0%E8%8E%B7%E5%BE%97%0A**%E5%9F%BA%E4%BA%8E%E8%AF%84%E5%88%86%E7%9F%A9%E9%98%B5%E7%9A%84%E6%96%B9%E6%B3%95%E5%87%BA%E7%8E%B0%E7%9A%84%E9%97%AE%E9%A2%98**%0A%E4%B8%8D%E5%90%8C%E7%94%A8%E6%88%B7%E8%AF%84%E5%88%86%E7%9A%84%E6%A0%87%E5%87%86%E4%B8%8D%E4%B8%80%EF%BC%8C%E6%9C%89%E4%BA%BA%E5%AE%BD%E6%9D%BE%EF%BC%8C%E6%9C%89%E4%BA%BA%E4%B8%A5%E8%8B%9B%EF%BC%8C%E6%9C%89%E7%9A%84%E8%BF%98%E6%98%AF%E7%B3%BB%E7%BB%9F%E7%BC%BA%E7%9C%81%E8%87%AA%E5%8A%A8%E8%AF%84%E5%88%86%EF%BC%8C%E9%80%9A%E7%94%A8%E6%80%A7%E3%80%81%E5%AE%A2%E8%A7%82%E6%80%A7%E5%AD%98%E7%96%91%0A%E5%AF%B9%E4%BA%8E%E8%AF%84%E5%88%86%E4%BB%A5%E5%A4%96%E7%9A%84%E5%85%B6%E4%BB%96%E7%94%A8%E6%88%B7%E5%B1%9E%E6%80%A7%E4%B8%8D%E4%BE%BF%E4%BA%8E%E5%88%A9%E7%94%A8%0A%E6%B7%B7%E5%90%88%E5%A4%9A%E9%87%8D%E7%94%A8%E6%88%B7%E5%B1%9E%E6%80%A7%E7%9A%84%E8%AF%84%E5%88%86%EF%BC%8C%E5%85%B6%E8%AF%84%E5%88%86%E5%85%AC%E5%BC%8F%E7%9A%84%E5%88%B6%E5%AE%9A%E5%AE%B9%E6%98%93%E6%AD%A6%E6%96%AD%0A%E8%AF%84%E5%88%86%E7%9F%A9%E9%98%B5%E6%99%AE%E9%81%8D%E5%BE%88%E7%A8%80%E7%96%8F%EF%BC%8C%E5%BD%B1%E5%93%8D%E6%8E%A8%E8%8D%90%E7%BB%8F%E5%BA%A6%0A%E5%86%B7%E5%90%AF%E5%8A%A8%E9%97%AE%E9%A2%98%EF%BC%9A%E6%96%B0%E9%97%BB%E6%8E%A8%E8%8D%90%E9%83%BD%E6%98%AF%E5%86%B7%E5%90%AF%E5%8A%A8%E7%9A%84%EF%BC%8C%E5%88%9A%E4%B8%8A%E7%BA%BF%E6%97%B6%E4%B8%8D%E8%83%BD%E6%8E%A8%E8%8D%90%E7%BB%99%E4%BB%BB%E4%BD%95%E4%BA%BA%0A**%E6%B7%B7%E5%90%88%E6%8E%A8%E8%8D%90%EF%BC%9A**%E6%8E%A8%E8%8D%90%E7%9A%84%E6%97%B6%E5%80%99%E9%9C%80%E8%A6%81%E5%B0%86%E5%95%86%E5%93%81%E7%9A%84%E5%B1%9E%E6%80%A7%E5%92%8C%E5%AE%A2%E6%88%B7%E7%9A%84%E5%B1%9E%E6%80%A7%E6%9C%80%E7%BB%88%E7%BB%93%E5%90%88%E8%B5%B7%E6%9D%A5%0A%0A*%20*%20*%0A%23%23%23%20%E6%A1%88%E4%BE%8B%E8%83%8C%E6%99%AF%E4%BB%8B%E7%BB%8D%0A%3E%E8%A5%BF%E7%8F%AD%E7%89%99%E5%9B%BD%E9%99%85%E9%93%B6%E8%A1%8C%E4%B8%BA%E5%B9%BF%E5%A4%A7%E5%AE%A2%E6%88%B7%E6%8F%90%E4%BE%9B%E4%BA%86%E5%90%84%E7%A7%8D%E9%87%91%E8%9E%8D%E6%9C%8D%E5%8A%A1%EF%BC%8C%E4%BD%86%E6%98%AF%E6%AF%94%E8%BE%83%E9%81%97%E6%86%BE%E7%9A%84%E6%98%AF%EF%BC%8C%E5%B0%B1%E7%9B%AE%E5%89%8D%E7%9A%84%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E6%9D%A5%E8%AF%B4%EF%BC%8C%E5%8F%AA%E6%9C%89%E5%B0%91%E9%83%A8%E5%88%86%E5%AE%A2%E6%88%B7%E5%8F%AF%E4%BB%A5%E5%9C%A8%E6%AF%8F%E4%B8%AA%E6%9C%88%E6%94%B6%E5%88%B0%E9%87%91%E8%9E%8D%E4%BA%A7%E5%93%81%E7%9A%84%E6%8E%A8%E8%8D%90%E4%BF%A1%E6%81%AF%EF%BC%8C%E5%A4%A7%E9%83%A8%E5%88%86%E5%AE%A2%E6%88%B7%E9%83%BD%E6%97%A0%E6%B3%95%E4%BA%AB%E5%8F%97%E8%BF%99%E6%A0%B7%E7%9A%84%E5%BE%85%E9%81%87%EF%BC%8C%E4%B8%BA%E4%BA%86%E6%9B%B4%E5%8A%A0%E5%B9%B3%E7%AD%89%E7%9A%84%E5%AF%B9%E5%BE%85%E6%AF%8F%E4%B8%AA%E5%AE%A2%E6%88%B7%EF%BC%8C%E8%A5%BF%E7%8F%AD%E7%89%99%E5%9B%BD%E9%99%85%E9%93%B6%E8%A1%8C%E6%89%93%E7%AE%97%E6%9E%84%E5%BB%BA%E4%B8%80%E4%B8%AA%E6%96%B0%E7%9A%84%E4%BA%A7%E5%93%81%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9F%E3%80%82%0A%3E%E5%9C%A8%E8%BF%99%E6%AC%A1%E6%AF%94%E8%B5%9B%E4%B8%AD%EF%BC%8C%E8%A5%BF%E7%8F%AD%E7%89%99%E5%9B%BD%E9%99%85%E9%93%B6%E8%A1%8C%E5%B8%8C%E6%9C%9B%E6%A0%B9%E6%8D%AE%E5%90%84%E4%B8%AA%E5%AE%A2%E6%88%B7%E8%BF%87%E5%8E%BB%E7%9A%84%E8%A1%8C%E4%B8%BA%EF%BC%8C%E9%A2%84%E6%B5%8B%E5%87%BA%E5%AE%A2%E6%88%B7%E4%B8%8B%E4%B8%AA%E6%9C%88%E4%BC%9A%E6%96%B0%E5%A2%9E%E5%8A%A0(%E8%B4%AD%E4%B9%B0)%E7%9A%84%E9%87%91%E8%9E%8D%E4%BA%A7%E5%93%81%0A%0A%23%23%23%23%20%E6%95%B0%E6%8D%AE%E6%8E%A2%E7%B4%A2%0A%E8%A7%A3%E5%86%B3%E6%80%9D%E8%B7%AF%EF%BC%9A%E9%80%9A%E8%BF%87%E6%97%B6%E9%97%B4%E5%BA%8F%E5%88%97%E7%9A%84%E6%A8%A1%E5%9E%8B%E7%BB%84%E5%90%88%EF%BC%8C%E5%B0%8616%E4%B8%AA%E6%A8%A1%E5%9E%8B%E7%BB%93%E6%9E%9C%E8%BF%9B%E8%A1%8C%E5%8A%A0%E6%9D%83%E5%B9%B3%E5%9D%87%EF%BC%8C%E6%9C%80%E7%BB%88%E7%BB%93%E6%9E%9C%E4%B8%BA%E8%AF%A5%E4%BA%A7%E5%93%81%E7%9A%84%E8%B4%AD%E4%B9%B0%E6%A6%82%E7%8E%87%0A