如何从logstash输出中grep particulr字段

问题描述

我正在尝试仅从logstash 1.repositories#create 2.\"repo \":\"username/reponame \"的输出中grep几个字段.请分享您的想法，以从该输出中提取特定信息，并将其分配给另一个变量

I am trying to grep only few fields from this output from logstash 1.repositories#create 2.\"repo\":\"username/reponame\" . please share your ideas to grep particular info from this outpput and assign this to another variable

消息" =>< 190> 11月1日20:35:15 10-254-128-66 github_audit:{\" actor_ip \:\" 192.168.1.1 \，\" from \: \"repositories#create \"，\"actor \":\"myuserid \"，\"repo \":\"username/reponame \"，\"action \":\"staff.repo_route \"，\"created_at \:1516286634991，\" repo_id \:44743，\" actor_id \:1033，\" data \:{\" actor_location \:{\" location \:{\" lat \:null ，\"lon \":null}}}}，

"message" => "<190>Nov 01 20:35:15 10-254-128-66 github_audit: {\"actor_ip\":\"192.168.1.1\",\"from\":\"repositories#create\",\"actor\":\"myuserid\",\"repo\":\"username/reponame\",\"action\":\"staff.repo_route\",\"created_at\":1516286634991,\"repo_id\":44743,\"actor_id\":1033,\"data\":{\"actor_location\":{\"location\":{\"lat\":null,\"lon\":null}}}}",

我正在使用此syslog.conf文件获取输出.

I am using this syslog.conf file to get the output.

input {
  tcp {
    port => 8088
    type => syslog
  }
  udp {
    port => 8088
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp}"
    }
    grep {
      match => { "message" => "repositories#create" }
    }
  }
}

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

我无法为您的回复添加评论，非常感谢您的回复.

I am not able to add my comments for your reply, thank you so much for your reply.

请分享您的想法以获取用户名:和回购:仅从此输出中，我正尝试从该特定输出中分配值，再次感谢

could you please share your ideas to get username: and repo: only from this output , i m trying assign the values from this particular output, thanks again

消息:"github_audit:{" actor_ip:" 192.168.1.1，"来自:" repositories#create，" actor:"用户名，" repo:"用户名/logstashrepo，"用户":用户名"，"created_at":1416299104782，操作":"repo.create"，"user_id":1033，"repo_id":44744，"actor_id":1033，"data":{"actor_location":{"location:{" lat:null，" lon:null}}}}"，@version:"1"，@timestamp:"2014-11-18T08:25:05.427Z"，主持人:"15-274-145-63"，类型:"syslog"，syslog5424_pri:"190"，时间戳:"11月18日00:25:05"，actor_ip:"10.239.37.185"，来自:"repositories#create"，演员:用户名"，回购:用户名/logstashrepo"，用户:用户名"，created_at:1416299104782，行动:"repo.create"，user_id:1033，repo_id:44744，actor_id:1033，

message: "github_audit: {"actor_ip":"192.168.1.1","from":"repositories#create","actor":"username","repo":"username/logstashrepo","user":"username","created_at":1416299104782,"action":"repo.create","user_id":1033,"repo_id":44744,"actor_id":1033,"data":{"actor_location":{"location":{"lat":null,"lon":null}}}}",@version: "1",@timestamp: "2014-11-18T08:25:05.427Z",host: "15-274-145-63",type: "syslog",syslog5424_pri: "190",timestamp: "Nov 18 00:25:05",actor_ip: "10.239.37.185",from: "repositories#create",actor: "username",repo: "username/logstashrepo",user: "username",created_at: 1416299104782,action: "repo.create",user_id: 1033,repo_id: 44744,actor_id: 1033,

推荐答案

使用 grok过滤器将JSON有效内容提取到单独的字段中，然后使用 json过滤器从JSON对象中提取字段.下面的示例有效，但仅从前缀为"github_audit:"的消息中提取JSON有效负载.我还猜测时间戳后面的字段是主机名，应该覆盖主机"字段中当前可能存在的任何内容.不要忘记添加日期过滤器来解析时间戳记"中的字符串"字段中输入"@timestamp".

Use a grok filter to extract the JSON payload into a separate field, then use a json filter to extract the fields from the JSON object. The example below works but only extracts the JSON payload from messages prefixed with "github_audit: ". I'm also guessing that the field after the timestamp is a hostname that should overwrite whatever might currently be in the "host" field. Don't forget to add a date filter to parse the string in the "timestamp" field into "@timestamp".

filter {
  grok {
    match => [
      "message",
      "%{SYSLOG5424PRI}%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{GREEDYDATA:message}"
    ]
    overwrite => ["host", "message"]
  }
  if [message] =~ /^github_audit: / {
    grok {
      match => ["message", "^github_audit: %{GREEDYDATA:json_payload}"]
    }
    json {
      source => "json_payload"
      remove_field => "json_payload"
    }
  }
}

这篇关于如何从logstash输出中grep particulr字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！