问题描述
是否有办法在使用Vowpal Wabbit进行回归分析时使用梯度增强?我使用Vowpal Wabbit随附的各种有用的技术.我想同时尝试梯度增强,但是我找不到在大众汽车上实现梯度增强的方法.
Is there a way to use gradient boosting on regression using Vowpal Wabbit? I use various techniques that come with Vowpal Wabbit that are helpful. I want to try gradient boosting along with that, but I can't find a way to implement gradient boosting on VW.
推荐答案
梯度增强的想法是一个整体模型是根据黑匣子弱模型构建的.您当然可以将VW用作黑匣子,但请注意,VW不提供决策树,而决策树是黑匣子弱势模型在提升方面的最受欢迎选择.一般而言,提升可以降低偏差(并增加方差),因此您应确保大众模型的方差低(不过度拟合).参见偏差方差权衡.
The idea of gradient boosting is that an ensemble model is built from black-box weak models. You can surely use VW as the black box, but note that VW does not offer decision trees, which are the most popular choice for the black-box weak models in boosting. Boosting in general decreases bias (and increases variance), so you should make sure that the VW models have low variance (no overfitting). See bias-variance tradeoff.
与大众化相关的一些减少措施:
There are some reductions related to boosting and bagging in VW:
-
--autolink N
添加了具有多项式N的链接函数,这可以被视为一种简单的提升方法. -
--log_multi K
是用于K级分类的在线增强算法.参见本文.您甚至可以将其用于二进制分类(K = 2),但不能用于回归. 通过在线重要性重采样进行 -
--bootstrap M
M方式引导.使用--bs_type=vote
进行分类,使用--bs_type=mean
进行回归.请注意,这是装袋,而不是加强 . -
--boosting N
(于2015-06-17添加)在N个弱学习者的在线推动下,请参阅理论论文
--autolink N
adds a link function with polynomial N, which can be considered a simple way of boosting.--log_multi K
is an online boosting algorithm for K-class classification. See the paper. You can use it even for binary classification (K=2), but not for regression.--bootstrap M
M-way bootstrap by online importance resampling. Use--bs_type=vote
for classification and--bs_type=mean
for regression. Note that this is bagging, not boosting.--boosting N
(added on 2015-06-17) online boosting with N weak learners, see a theoretic paper
这篇关于Vowpal Wabbit的梯度提升的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!