python - Pyspark将数据写入配置单元

下面是我将数据写入Hive的代码

from pyspark import since,SparkContext as sc
from pyspark.sql import SparkSession
from pyspark.sql.functions import _functions , isnan
from pyspark.sql import SQLContext
from pyspark.sql.types import *
from pyspark import HiveContext as hc


spark = SparkSession.builder.appName("example-spark").config("spark.sql.crossJoin.enabled","true").config('spark.sql.warehouse.dir',"file:///C:/spark-2.0.0-bin-hadoop2.7/bin/metastore_db/spark-warehouse").config('spark.rpc.message.maxSize','1536').getOrCreate()
Name= spark.read.csv("file:///D:/valid.csv", header="true",inferSchema =
True,sep=',')

join_df=join_df.where("LastName != ''").show()
join_df.registerTempTable("test")
hc.sql("CREATE TABLE dev_party_tgt_repl STORED AS PARQUETFILE AS SELECT * from dev_party_tgt")

执行上面的代码后，我得到以下错误

Traceback (most recent call last):
File "D:\01 Delivery Support\01
easyJet\SparkEclipseWorkspace\SparkTestPrograms\src\NameValidation.py", line
22, in <module>
join_df.registerTempTable("test")
AttributeError: 'NoneType' object has no attribute 'test'

我的系统环境详细信息:

操作系统:Windows

Eclipse霓虹灯

Spark版本:2.0.0

最佳答案

试试这个:

join_df.where("LastName != ''").write.saveAsTable("dev_party_tgt_repl")

关于python - Pyspark将数据写入配置单元，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/44757972/