创建空的数据框Java Spark

问题描述

关于如何使用Spark Scala/Python创建空数据框/数据集的示例很多.但是我想知道如何在Java Spark中创建一个空的数据框/数据集.

There are many examples on how to create empty dataframe/Dataset using Spark Scala/Python. But I would like to know how to create an empty dataframe/Dataset in Java Spark.

我必须创建一个只有标题为Column_1的一列的空数据框，然后键入String.

I have to create an empty dataframe with just one column with header as Column_1 and type String.

推荐答案

Alternative-1使用用户定义的架构

创建空数据框

// alternative - 1
        StructType s = new StructType()
                .add(new StructField("Column_1", DataTypes.StringType, true, Metadata.empty()));
        Dataset<Row> csv = spark.read().schema(s).csv(spark.emptyDataset(Encoders.STRING()));
        csv.show(false);
        csv.printSchema();
        /**
         * +--------+
         * |Column_1|
         * +--------+
         * +--------+
         *
         * root
         *  |-- Column_1: string (nullable = true)
         */

替代2使用 null 值和用户定义的架构

创建数据框

Alternative-2 create dataframe with null value and user defined schema

 Dataset<Row> df4 = spark.sql("select cast(null  as string) Column_1");
        df4.show(false);
        df4.printSchema();
        /**
         * +--------+
         * |Column_1|
         * +--------+
         * |null    |
         * +--------+
         *
         * root
         *  |-- Column_1: string (nullable = true)
         */

替代3

 ClassTag<Row> rowTag = scala.reflect.ClassTag$.MODULE$.apply(Row.class);
        Dataset<Row> df5 = spark.createDataFrame(spark.sparkContext().emptyRDD(rowTag),
                new StructType()
                        .add(new StructField("Column_1", DataTypes.StringType, true, Metadata.empty())));
        df5.show(false);
        df5.printSchema();
        /**
         * +--------+
         * |Column_1|
         * +--------+
         * +--------+
         *
         * root
         *  |-- Column_1: string (nullable = true)
         */

spark.emptyDataframe 创建没有任何列和值的数据框

spark.emptyDataframe to create dataframe without any column and value

 Dataset<Row> rowDataset = spark.emptyDataFrame();
        rowDataset.show(false);
        rowDataset.printSchema();
        /**
         * ++
         * ||
         * ++
         * ++
         *
         * root
         */

这篇关于创建空的数据框Java Spark的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！