如何在grok/logstash中匹配换行符?

问题描述

我有一台结合多行事件并通过伐木工人协议发送事件的远程机器.

I have a remote machine that combines multiline events and sends them across the lumberjack protocol.

进来的东西看起来像这样:

What comes in is something that looks like this:

{
     "message" => "2014-10-20T20:52:56.133+0000 host 2014-10-20 15:52:56,036 [ERROR   ][app.logic     ] Failed to turn message into JSON\nTraceback (most recent call last):\n  File \"somefile.py", line 249, in _get_values\n    return r.json()\n  File \"/path/to/env/lib/python3.4/site-packages/requests/models.py\", line 793, in json\n    return json.loads(self.text, **kwargs)\n  File \"/usr/local/lib/python3.4/json/__init__.py\", line 318, in loads\n    return _default_decoder.decode(s)\n  File \"/usr/local/lib/python3.4/json/decoder.py\", line 343, in decode\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n  File \"/usr/local/lib/python3.4/json/decoder.py\", line 361, in raw_decode\n    raise ValueError(errmsg(\"Expecting value\", s, err.value)) from None\nValueError: Expecting value: line 1 column 1 (char 0), Failed to turn message into JSON"
}

当我尝试将邮件与

grok {
    match => [ "message", "%{TIMESTAMP_ISO8601:timestamp} \[%LOGLEVEL:loglevel}%{    SPACE}\]\[%{NOTSPACE:module}%{SPACE}\]%{GREEDYDATA:message}" ]
}

GREEDYDATA并没有我想要的那么贪婪.

the GREEDYDATA is not nearly as greedy as I would like.

因此，我尝试使用gsub:

So then I tried to use gsub:

mutate {
    gsub => ["message", "\n", "LINE_BREAK"]
}
# Grok goes here
mutate {
    gsub => ["message", "LINE_BREAK", "\n"]
}

但那个人没有工作

The Quick brown fox
jumps over the lazy
groks

我知道了

The Quick brown fox\njumps over the lazy\ngroks

所以...

如何将换行符添加回数据中，使GREEDYDATA与我的换行符匹配，或者以其他方式获取消息的相关部分?

How do I either add the newline back to my data, make the GREEDYDATA match my newlines, or in some other way grab the relevant portion of my message?

推荐答案

所有GREEDYDATA是.*，但是.与换行符不匹配，因此您可以将%{GREEDYDATA:message}替换为(?<message>(.|\r|\n)*)并获取才是真正的贪婪.

All GREEDYDATA is is .*, but . doesn't match newline, so you can replace %{GREEDYDATA:message} with (?<message>(.|\r|\n)*)and get it to be truly greedy.

这篇关于如何在grok/logstash中匹配换行符?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！