从FlinkML多元线性回归中提取权重

问题描述

我正在为Flink(0.10-SNAPSHOT)运行示例多元线性回归.我无法弄清楚如何提取权重(例如，斜率和截距，beta0-beta1，以及您想称呼的权重).我在Scala中经验不足，这可能是我问题的一半.

I am running the example multiple linear regression for Flink (0.10-SNAPSHOT). I can't figure out how to extract the weights (e.g. slope and intercept, beta0-beta1, what ever you want to call them). I'm not super seasoned in Scala, that is probably half my problem.

感谢任何人可以提供的任何帮助.

Thanks for any help any one can give.

object Job {
 def main(args: Array[String]) {
    // set up the execution environment
    val env = ExecutionEnvironment.getExecutionEnvironment

    val survival = env.readCsvFile[(String, String, String, String)]("/home/danger/IdeaProjects/quickstart/docs/haberman.data")

    val survivalLV = survival
      .map{tuple =>
      val list = tuple.productIterator.toList
      val numList = list.map(_.asInstanceOf[String].toDouble)
      LabeledVector(numList(3), DenseVector(numList.take(3).toArray))
    }

    val mlr = MultipleLinearRegression()
      .setStepsize(1.0)
      .setIterations(100)
      .setConvergenceThreshold(0.001)

    mlr.fit(survivalLV)
    println(mlr.toString())     // This doesn't do anything productive...
    println(mlr.weightsOption)  // Neither does this.

  }
}

推荐答案

问题是您仅构造了Flink作业(DAG)，该作业将计算权重，但尚未执行.触发执行的最简单方法是使用collect方法，该方法将把DataSet的结果返回给客户端.

The problem is that you've only constructed the Flink job (DAG) which will calculate the weights but it is not yet executed. The easiest way to trigger the execution is to use the collect method which will retrieve the result of the DataSet back to your client.

mlr.fit(survivalLV)

val weights = mlr.weightsOption match {
  case Some(weights) => weights.collect()
  case None => throw new Exception("Could not calculate the weights.")
}

println(weights)

这篇关于从FlinkML多元线性回归中提取权重的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！