来自嵌套数组的 JOLT 连接值 (Apache NiFi)

问题描述

我有一个 JSON:

{报告":[{列标题":{度量标头":{metricHeaderEntries":[{名称":ga:会话"，类型":整数"},{名称":ga:bounces"，类型":整数"},{名称":ga:sessionDuration"，类型":时间"},{名称":ga:pageviews"，类型":整数"}]}},数据":{行":[{指标":[{值":[25"，18"，1269.0"，27"]}]}],总计":[{值":[25"，18"，1269.0"，27"]}],行数":1，最低限度":[{值":[25"，18"，1269.0"，27"]}],最大值":[{值":[25"，18"，1269.0"，27"]}],isDataGolden":真}}]}

它们的

metricHeaderEntries 和 values 是分开的.值在 data.totals 数组中(顺序正确保存).我想修改 JSON 并获得以下结构(或与此类似，我只需要对 metric.name = metric.value):

{指标":[{名称":ga:会话"，值":25"},{名称":ga:bounces"，类型":18"},{名称":ga:sessionDuration"，类型":1269.0"；},{名称":ga:pageviews"，类型":27"}],isDataGolden":真}

JOLT 可以吗?之前我只使用 shift 规范来完成一些非常简单的任务.以下规格:

[{操作":移位"，规格":{报告":{*":{列标题":{度量标头":{metricHeaderEntries":{*":{姓名":@(1,name)"}}}},isDataGolden":isDataGolden"}}}}]

返回:

{ga:sessions":ga:会话"，ga:bounces": "ga:bounces",ga:sessionDuration":ga:sessionDuration"，ga:pageviews":ga:浏览量"}

几乎".当然不是我想要的.我需要一个数组 metrics 与字段 name 和 value 如上所述.但我不知道如何从 data.totals 获取这些值并将它们放入指标.而且 isDataGolden 也消失了.我阅读了一些关于 modify-overwrite-beta 的内容，我可以将它用于我的案例吗?

解决方案

可以使用executegroovyscript

导入 groovy.json.*def ff=session.get()如果(！ff)返回//读取流文件内容并解析def body = ff.read().withReader(UTF-8"){reader->新的 JsonSlurper().parse(reader)}def rep0=body.reports[0]定义结果 = [指标:rep0.columnHeader.metricHeader.metricHeaderEntries.indexed().collect{i,m->[姓名 : m.name,值:rep0.data.totals[0].values[i]]},isDataGolden : rep0.data.isDataGolden]//写入新的流文件内容ff.write(UTF-8"){writer->新的 JsonBuilder(result).writeTo(writer)}//转移REL_SUCCESS <<ff

I have a JSON:

{
  "reports": [
    {
      "columnHeader": {
        "metricHeader": {
          "metricHeaderEntries": [
            {
              "name": "ga:sessions",
              "type": "INTEGER"
            },
            {
              "name": "ga:bounces",
              "type": "INTEGER"
            },
            {
              "name": "ga:sessionDuration",
              "type": "TIME"
            },
            {
              "name": "ga:pageviews",
              "type": "INTEGER"
            }
          ]
        }
      },
      "data": {
        "rows": [
          {
            "metrics": [
              {
                "values": [
                  "25",
                  "18",
                  "1269.0",
                  "27"
                ]
              }
            ]
          }
        ],
        "totals": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "rowCount": 1,
        "minimums": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "maximums": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "isDataGolden": true
      }
    }
  ]
}

metricHeaderEntries and values for them are separated. Values are in data.totals array (order is saved correctly).I want to modify JSON and get following structure (or similar to this, I only need pairs metric.name = metric.value):

{
  "metrics": [
            {
              "name": "ga:sessions",
              "value": "25"
            },
            {
              "name": "ga:bounces",
              "type": "18"
            },
            {
              "name": "ga:sessionDuration",
              "type": "1269.0"
            },
            {
              "name": "ga:pageviews",
              "type": "27"
            }
          ],
    "isDataGolden": true
}

Is it possible with JOLT?Before I only used shift spec for some very easy tasks. Following spec:

[
  {
    "operation": "shift",
    "spec": {
      "reports": {
        "*": {
          "columnHeader": {
            "metricHeader": {
              "metricHeaderEntries": {
                "*": {
                  "name": "@(1,name)"
                }
              }
            }
          },
          "isDataGolden": "isDataGolden"
        }
      }
    }
  }
]

Returns:

{
  "ga:sessions" : "ga:sessions",
  "ga:bounces" : "ga:bounces",
  "ga:sessionDuration" : "ga:sessionDuration",
  "ga:pageviews" : "ga:pageviews"
}

"Almost". Not what i wanted of course. I need an array metrics with fields name and value as I described above. But I don't know how to get these values from data.totals and put them to metrics. And also isDataGolden disappeared. I read a little bit about modify-overwrite-beta, can i use it for my case?

解决方案

you can use executegroovyscript

import groovy.json.*

def ff=session.get()
if(!ff)return

//read flow file content and parse it
def body = ff.read().withReader("UTF-8"){reader->
    new JsonSlurper().parse(reader)
}

def rep0=body.reports[0]

def result = [
    metrics : rep0.columnHeader.metricHeader.metricHeaderEntries.indexed().collect{i,m->
            [
                name : m.name,
                value: rep0.data.totals[0].values[i]
            ]
        },
    isDataGolden : rep0.data.isDataGolden
]

//write new flow file content
ff.write("UTF-8"){writer->
    new JsonBuilder(result).writeTo(writer)
}
//transfer
REL_SUCCESS << ff

这篇关于来自嵌套数组的 JOLT 连接值 (Apache NiFi)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！