elasticsearch - 自定义日志字符串的ElasticSearch Grok模式问题

我正在尝试在Elasticsearch上为以下基于JSON的自定义日志创建完整的GROK模式:

------------------------DEBUG----------------------------
Date   : 2019-12-26 12:18:21,498
METHOD NAME:   xyz
{
  "methodName": "SMS_POOL_IN",
  "Tran_Type": "Response",
  "URL": "xyz.abcL",
  "ApiResult": "Success",
  "Date": "2019/12/26 12:18:21",
  "ErrorCode": "00",
  "ErrorReason": "Success",
  "Msisdn": "9999999",
  "CNIC": "99999999",
  "RequestID": "1111",
  "SR_TranID": "2222",
  "Channel": "abc"
}

但是，无论何时我解析它，我都只会从希腊人那里得到时间戳。

我正在使用grok调试器进行测试。每当我使用greedydata时，我只会得到第一个json参数，其余的都将被忽略，我在这里缺少什么吗？我如何才能从这些日志中找到一个骗子？任何帮助之手将不胜感激

我在下面创建了

%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}%{SPACE \s}%{GREEDYDATA.*}

并获得以下结果。

{
  "GREEDYDATA": [
    [
      "------------------------DEBUG----------------------------",
      "Date   : 2019-12-26 12:18:21,498 ",
      "METHOD NAME:   xyz",
      "{",
      ""methodName": "SMS_POOL_IN",",
      ""Tran_Type": "Response",",
      ""URL": "xyz.abcL",",
      ""ApiResult": "Success",",
      ""Date": "2019/12/26 12:18:21",",
      ""ErrorCode": "00",",
      ""ErrorReason": "Success",",
      ""Msisdn": "9999999",",
      ""CNIC": "99999999",",
      ""RequestID": "1111",",
      ""SR_TranID": "2222",",
      ""Channel": "abc"",
      "} ",
      "",
      ""
    ]
  ],
  "SPACE": [
    [
      "\n",
      "\n",
      "\n",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n  ",
      "\n",
      "",
      ""
    ]
  ]
}

我需要显示所有这些json标签，因为我需要将它们填充在ELK的单独标签中

最佳答案

我自己创建了grok，唯一的问题是我试图构建grok的语法。下面是上面阅读的正确的grok语法

%{TIMESTAMP_ISO8601:date_time}\s*%{GREEDYDATA:Method}\n%{GREEDYDATA:Bracket}\s*\"methodName\"\:\s\"%{DATA:methodName}\s*\"Tran_Type\"\:\s\"%{DATA:Tran_Type}\s*\"URL\"\:\s\"%{DATA:URL}\s*\"ApiResult\"\:\s\"%{DATA:ApiResult}\s*\"Date\"\:\s\"%{DATA:Date}\s*\"ErrorCode\"\:\s\"%{DATA:ErrorCode}\s*\"ErrorReason\"\:\s\"%{DATA:ErrorReason}\s*\"Msisdn\"\:\s\"%{DATA:Msisdn}\s*\"CNIC\"\:\s\"%{DATA:CNIC}\s*\"RequestID\"\:\s\"%{DATA:RequestID}\s*\"SR_TranID\"\:\s\"%{DATA:SR_TranID}\s*\"Channel\"\:\s\"%{DATA:Channel}\s

首先，我拿起时间戳，然后在GREEDYDATA中拿起json字符串之外的所有内容，然后使用DATA关键字分隔json标签。

以上结果是

{
  "date_time": [
    [
      "2019-12-26 12:18:21,498"
    ]
  ],
  "YEAR": [
    [
      "2019"
    ]
  ],
  "MONTHNUM": [
    [
      "12"
    ]
  ],
  "MONTHDAY": [
    [
      "26"
    ]
  ],
  "HOUR": [
    [
      "12",
      null
    ]
  ],
  "MINUTE": [
    [
      "18",
      null
    ]
  ],
  "SECOND": [
    [
      "21,498"
    ]
  ],
  "ISO8601_TIMEZONE": [
    [
      null
    ]
  ],
  "Method": [
    [
      "METHOD NAME:   xyz"
    ]
  ],
  "Bracket": [
    [
      "{"
    ]
  ],
  "methodName": [
    [
      "SMS_POOL_IN","
    ]
  ],
  "Tran_Type": [
    [
      "Response","
    ]
  ],
  "URL": [
    [
      "xyz.abcL","
    ]
  ],
  "ApiResult": [
    [
      "Success","
    ]
  ],
  "Date": [
    [
      "2019/12/26 12:18:21","
    ]
  ],
  "ErrorCode": [
    [
      "00","
    ]
  ],
  "ErrorReason": [
    [
      "Success","
    ]
  ],
  "Msisdn": [
    [
      "9999999","
    ]
  ],
  "CNIC": [
    [
      "99999999","
    ]
  ],
  "RequestID": [
    [
      "1111","
    ]
  ],
  "SR_TranID": [
    [
      "2222","
    ]
  ],
  "Channel": [
    [
      "abc""
    ]
  ]
}

关于elasticsearch - 自定义日志字符串的ElasticSearch Grok模式问题，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/59489383/