问题描述
我正在使用带有Spark 2.0.0的预览" Google DataProc Image 1.1.要完成我的一项操作，我必须完成笛卡尔乘积.从2.0.0版开始，已经创建了一个火花配置参数(spark.sql.cross Join.enabled)，该参数禁止笛卡尔乘积，并引发Exception.如何设置spark.sql.crossJoin.enabled = true，最好使用初始化操作?spark.sql.crossJoin.enabled=true
I am using the 'preview' Google DataProc Image 1.1 with Spark 2.0.0. To complete one of my operations I have to complete a cartesian product. Since version 2.0.0 there has been a spark configuration parameter created (spark.sql.cross Join.enabled) that prohibits cartesian products and an Exception is thrown. How can I set spark.sql.crossJoin.enabled=true, preferably by using an initialization action?spark.sql.crossJoin.enabled=true
推荐答案
要更改Dataproc中配置设置的默认值，您甚至不需要执行初始化操作，可以使用-properties标志:
For changing default values of configuration settings in Dataproc, you don't even need an init action, you can use the --properties flag when creating your cluster from the command-line:
gcloud dataproc clusters create --properties spark:spark.sql.crossJoin.enabled=true my-cluster ...
这篇关于适用于Spark 2.x的spark.sql.crossJoin.enabled的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！