Logstash是否可能将相同内容从日志文件推送到ElasticSearch

问题描述

logstash 配置将日志文件设置为输入源，然后将内容发送到 ElasticSearch .

The logstash config sets log files as input source and then sends the content to ElasticSearch.

input 部分如下所示

input{
    file{
        path => "/data/logs/backend.log*"
        start_position => "beginning"
    }
}

然后，日志文件将按大小滚动，这意味着日志文件名首先为 backend.log ，当文件大小达到10M时，将其重命名为 backend.log.1 ，并创建一个新的空backend.log来记录内容.

Then the log file will be rolling by size, which means at first the log file name is backend.log, when the file reaches size 10M, then it is renamed to backend.log.1, and a new empty backend.log is created to log content.

所以问题是 logstash 是否会将内容从 backend.log.1 发送到es服务器?还是 ElasticSearch 能够区分已经收到 backend.log.1 中的内容，尽管这似乎效率不高.

So the question is whether logstash will send the content from backend.log.1 to es server? Or is ElasticSearch able to distinguish that the content from backend.log.1 already received, although this seems to be not efficient.

推荐答案

file 输入文档包含有关如何处理旋转

The file input documentation contains a whole paragraph about how well it handles rotation

由于默认为 tail 模式，因此 path 参数应确保使用glob模式来捕获所有文件，就像您所做的一样.所以，您都准备好了.拖尾很开心！

Since the tail mode is the default, your path parameter should make sure to use a glob pattern to catch all files, exactly as you did. So you're all set. Happy tailing!

这篇关于Logstash是否可能将相同内容从日志文件推送到ElasticSearch的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！