如何比较共享相同内容的两个 StructType?

问题描述

看起来 StructType 保留了顺序，所以两个包含相同 StructField 的 StructType 不被认为是等效的.

It seems like StructType preserves order, so two StructType containing same StructFields are not considered equivalent.

例如:

val st1 = StructType(
StructField("ii",StringType,true) ::
StructField("i",StringType,true) :: Nil)

val st2 = StructType(
StructField("i",StringType,true) ::
StructField("ii",StringType,true) :: Nil)

println(st1 == st2)

返回 false 即使它们都有 StructField("i",StringType,true) 和 StructField("ii",StringType,true)，只是顺序不同.

returns false even though they both have StructField("i",StringType,true) and StructField("ii",StringType,true), just in different order.

我需要一个测试来证明这两个是等价的，因为就我的目的而言，这两个没有区别.

I need a test that can say that these two are equivalent because for my purpose, these two are not different.

val schema1 = StructType(StructField("A",ArrayType(st1,true),true) :: Nil)

val schema2 = StructType(StructField("A",ArrayType(st2,true),true) :: Nil)

val final_schema = StructType((schema1 ++ schema2).distinct)

final_schmea 的结果应该只有一个 A 的 StructType 而不是两个，但是 distinct 认为这些两个 StructType 不同，所以我最终得到两个不同的 StructField，名为 A.所以我的问题是，有没有办法根据两个 StructType 的内容而不是订单来比较它们?

The result of final_schmea should only have one StructType of A instead of two, but distinct considers these two StructType as different, so I end up getting two different StructField named A. So my question is, is there a way to compare two StructTypes based on their contents, not on orders?

经过进一步调查，由于StructType基本上是Seq，我可以做到内容比较适用于Seq，但我我试图想出一种方法，我可以最有效地对嵌入的 StructType 进行比较.

After further investigation, since StructType is basically Seq<StructField>, I can do content comparison for that works for Seq, but I am trying to think of a way I can do comparison for embedded StructType most efficiently.

推荐答案

这可能可以清理，但它可以工作并处理嵌套的 StructType:

This can probably be cleaned up, but it works and handles nested StructType:

def isEqual(struct1: StructType, struct2: StructType): Boolean = {
  struct1.headOption match {
    case Some(field) => {
      if(field.dataType.typeName != "struct") {
        struct2.find(_ == field) match {
         case Some(matchedField) => isEqual(StructType(struct1.filterNot(_ == field)), StructType(struct2.filterNot(_ == field)))
         case None => false
        }
      } else {
        val isEqualContents = struct2.find(x => x.name == field.name && x.nullable == field.nullable && x.dataType.typeName == "struct") match {
          case Some(matchedField) => isEqual(field.dataType.asInstanceOf[StructType], matchedField.dataType.asInstanceOf[StructType])
          case None => false
        }
        if(isEqualContents) isEqual(StructType(struct1.filterNot(_ == field)), StructType(struct2.filterNot(_ == field))) else false
      }
    }
    case None => struct2.size == 0
  }
}

val st1 = StructType(
StructField("ii",StringType,true) ::
StructField("i",StringType,true) ::
StructField("iii", StructType(StructField("iv", StringType, true) :: Nil), true) :: Nil)

val st2 = StructType(
StructField("i",StringType,true) ::
StructField("ii",StringType,true) ::
StructField("iii", StructType(StructField("v", StringType, true) :: Nil), true) :: Nil)

isEqual(st1, st2)

它也可以使用更多的爱来变成尾递归.

It could also use a little more love to become tail recursive, too.

这篇关于如何比较共享相同内容的两个 StructType?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！