使用 Scala 将 Array[DenseVector] 转换为 CSV

问题描述

我在 Scala 中使用 Kmeans Spark 函数，我需要将获得的集群中心保存到 CSV 中.这个 val 是类型:Array[DenseVector].

I am using Kmeans Spark function with Scala and I need to save the Cluster Centers obtained into a CSV. This val is type: Array[DenseVector].

val clusters = KMeans.train(parsedData, numClusters, numIterations)
val centers = clusters.clusterCenters

我试图将 centers 转换为 RDD 文件，然后从 RDD 转换为 DF，但我遇到了很多问题(例如，import spark.implicits._/SQLContext.implicits._ 不是工作，我不能使用 .toDF).我想知道是否有另一种方法可以使 CSV 更容易.

I was trying converting centers to a RDD file and then from RDD to DF, but I get a lot of problems (e.g, import spark.implicits._ / SQLContext.implicits._ is not working and I cannot use .toDF). I was wondering if there is another way to make a CSV easier.

有什么建议吗?

推荐答案

无需使用外部库，您只需通过 Java 方式写入文件即可.

Without use of external libraries you can do that by simply writing to the file Java way.

import java.io.{ PrintWriter, File, FileOutputStream }

...

val pw = new PrintWriter(
    new File( "KMeans_centers.csv" )
)

centers
.foreach( vec =>
        pw.write( vec.toString.drop( 1 ).dropRight( 1 ) + "\n" )
    )

pw.close()

结果文件

0.1,0.1,0.1
9.1,9.1,9.1

需要

drop 和 dropRight 来移除转换后的向量周围的 [].

drop and dropRight are needed to remove [] around the converted vector.

代码和数据取自官方示例.

这篇关于使用 Scala 将 Array[DenseVector] 转换为 CSV的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！