从/var/log/messages过滤文本并以logstash转发

问题描述

我正在使用Logstash，但无法弄清楚如何进行过滤.我想从/var/log/messages中过滤以下日志，并仅转发其中包含[INFO]的行.

I am using logstash and was not able to figure out how to filter. I want to filter the following log from /var/log/messages and forward only the line which contain [INFO] in it.

mingus  sshd[INFO]: Failed password for illegal user user from 219.117.251.250 port 44741 ssh2
mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44817 ssh2
mingus  sshd[2264]: Failed password for root from 219.117.251.250 port 44866 ssh2
mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44918 ssh2
mingus  sshd[2268]: Illegal user test from 219.117.251.250
mingus  sshd[2268]: Failed password for illegal user test from 219.117.251.250 port 44997 ssh2

我想知道是否有可能.

推荐答案

是的，这是可能的！使用 grok 过滤器提取所需的信息.您可以使用它来获取包含"sshd [(此处为某些字符串)]"内部的字段.示例:

Yes, this is possible! Use the grok filter to extract the information you need. You can use this to get a field containing the inside of "sshd[(some string here)]". Example:

filter {
    grok {
        match => ["message", "mingus  sshd\[%{WORD:messagetype}\]: %{GREEDYDATA}"]
    }
}

完成此操作后，可以在输出中使用条件语句，以便仅传递包含INFO的行.示例:

Once you've done that, you can use a conditional over the output such that only lines containing INFO will be passed. Example:

output {
    if [messagetype] == "INFO" {
        stdout {
            codec => "rubydebug"
        }
    }
}

我希望这对您有帮助！

这是我的配置文件:

input {
    stdin {}
}
filter {
    grok {
        match => ["message", "mingus  sshd\[%{WORD:messagetype}\]: %{GREEDYDATA}"]
    }
}
output {
    if [messagetype] == "INFO" {
        stdout {
            codec => "rubydebug"
        }
    }
}

使用stdin和stdout的选择很容易使调试容易.使用提供的日志行的片段，我输入以下内容:

The choice to use stdin and stdout was simple to make debugging easy. Using the snippet of log lines provided, I put in this input:

mingus  sshd[INFO]: Failed password for illegal user user from 219.117.251.250 port 44741   ssh2
mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44817 ssh2
mingus  sshd[2264]: Failed password for root from 219.117.251.250 port 44866 ssh2
mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44918 ssh2
mingus  sshd[2268]: Illegal user test from 219.117.251.250
mingus  sshd[2268]: Failed password for illegal user test from 219.117.251.250 port 44997 ssh2

并收到以下输出:

{
        "message" => "mingus  sshd[INFO]: Failed password for illegal user user from         219.117.251.250 port 44741 ssh2",
       "@version" => "1",
     "@timestamp" => "2014-07-31T13:59:34.376Z",
           "host" => "cmssrv221.fnal.gov",
    "messagetype" => "INFO"
}
{
        "message" => "mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44817 ssh2",
       "@version" => "1",
     "@timestamp" => "2014-07-31T13:59:34.378Z",
           "host" => "cmssrv221.fnal.gov",
    "messagetype" => "INFO"
}
{
        "message" => "mingus  sshd[INFO]: Failed password for root from 219.117.251.250 port 44918 ssh2",
       "@version" => "1",
     "@timestamp" => "2014-07-31T13:59:34.387Z",
           "host" => "cmssrv221.fnal.gov",
    "messagetype" => "INFO"
}

根据您提供给我的信息，这似乎是您想要的.如果这不是您想要的，则必须更加具体.

This, based on the information you gave me, appears to be what you want. If this not what you wanted, you'll have to be more specific.

顺便说一句，我使用的是logstash 1.4.1，以防万一您使用的是完全不同的版本.

By the way, I am using logstash 1.4.1, just in case you are using a vastly different version.

这篇关于从/var/log/messages过滤文本并以logstash转发的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！