问题描述
explode函数和explode运算符之间有什么区别?
What's the difference between explode function and explode operator?
推荐答案
spark.sql.functions.explode
explode函数为给定数组或映射列(在DataFrame中)中的每个元素创建一个新行.
spark.sql.functions.explode
explode function creates a new row for each element in the given array or map column (in a DataFrame).
val signals: DataFrame = spark.read.json(signalsJson)
signals.withColumn("element", explode($"data.datapayload"))
explode创建一个列.
请参见功能对象以及如何在DataFrame中展开数组(从JSON)的示例?
explode运算符几乎是explode函数的.
explode operator is almost the explode function.
在scaladoc中:
From the scaladoc:
ds.flatMap(_.words.split(" "))
请注意(再次引用scaladoc):
Please note that (again quoting the scaladoc):
请参见数据集API 和如何使用键入的数据集将多值列拆分为单独的行中的示例?
explode(然后我们可以将主要问题转换为explode函数和flatMap运算符之间的区别)，但不同之处在于前者是一个函数，而后者是一个运算符.它们具有不同的签名，但可以给出相同的结果.这通常会导致讨论什么更好，并且通常归结为个人喜好或编码风格.
Despite explode being deprecated (that we could then translate the main question to the difference between explode function and flatMap operator), the difference is that the former is a function while the latter is an operator. They have different signatures, but can give the same results. That often leads to discussions what's better and usually boils down to personal preference or coding style.
也可以说，考虑到flatMap在Scala编程中的普遍性(主要隐藏在要理解).
One could also say that flatMap (i.e. explode operator) is more Scala-ish given how ubiquitous flatMap is in Scala programming (mainly hidden behind for-comprehension).
这篇关于爆炸函数和运算符有什么区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！