使用grok将日志文件名添加为logstash中的字段

问题描述

我正在使用 Grok & Logstash将访问日志从Nginx发送到Elastic搜索.我正在为Logstash提供我的所有访问日志(使用通配符，效果很好)，我想获取文件名(准确地说是其中的一部分)并将其用作字段. /p>

我的配置如下:

input {
  file {
    path => "/var/log/nginx/*.access.log"
    type => "nginx_access"
  }
}

filter {
  if [type] == "nginx_access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
      match => { "path" => "%{GREEDYDATA}/%{GREEDYDATA:app}.access.log" }
      add_field => { "app" => "%{app}" }
    }
  }
}
output{
   # whatever
}

但是它似乎不起作用:添加了app字段，但其值为%{app}(未替换).

我尝试了不同的尝试，但无济于事.我可能会遗漏某些东西... 有什么想法吗?

非常感谢

解决方案

好，找到了. grok默认情况下在匹配时中断.因此，第一场比赛很好，它跳过了第二场比赛.

我这样解决了它:

filter {
  if [type] == "nginx_access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
      match => { "path" => "%{GREEDYDATA}/%{GREEDYDATA:app}.access.log" }
      break_on_match => false
    }
  }
}

I'm using Grok & Logstash to send access logs from Nginx to Elastic search. I'm giving Logstash all my access logs (with a wildcard, works well) and I would like to get the filename (some part of it, to be exact) and use it as a field.

My config is as follows :

input {
  file {
    path => "/var/log/nginx/*.access.log"
    type => "nginx_access"
  }
}

filter {
  if [type] == "nginx_access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
      match => { "path" => "%{GREEDYDATA}/%{GREEDYDATA:app}.access.log" }
      add_field => { "app" => "%{app}" }
    }
  }
}
output{
   # whatever
}

But it doesn't seem to work : the app field is added, but has a value of %{app} (not replaced).

I tried different things but to no avail. I may be missing something ... Any ideas ?

Thanks a lot

解决方案

Ok, found it. grok breaks on match by default. So the first match being good, it skips the second one.

I solved it like that :

filter {
  if [type] == "nginx_access" {
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
      match => { "path" => "%{GREEDYDATA}/%{GREEDYDATA:app}.access.log" }
      break_on_match => false
    }
  }
}

这篇关于使用grok将日志文件名添加为logstash中的字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！