elasticsearch使用logstash和csv删除文档

问题描述

有什么方法可以使用Logstash和csv文件从ElasticSearch中删除文档吗?我阅读了Logstash文档，却一无所获，并尝试了一些配置，但是使用操作删除"没有任何反应

Is there any way to delete documents from ElasticSearch using Logstash and a csv file?I read the Logstash documentation and found nothing and tried a few configs but nothing happened using action "delete"

output {
    elasticsearch{
        action => "delete"
        host => "localhost"
        index => "index_name"
        document_id => "%{id}"
    }
}

有人尝试过吗?我应该在配置的输入和过滤器部分添加一些特殊的东西吗?我使用文件插件作为输入，使用csv插件作为过滤器.

Has anyone tried this? Is there anything special that I should add to the input and filter sections of the config? I used file plugin for input and csv plugin for filter.

推荐答案

绝对可以按照您的建议进行操作，但是如果您使用的是Logstash 1.5，则需要使用transport协议，因为存在错误在通过HTTP协议执行delete时在Logstash 1.5中使用(请参阅 issue# 195 )

It is definitely possible to do what you suggest, but if you're using Logstash 1.5, you need to use the transport protocol as there is a bug in Logstash 1.5 when doing deletes over the HTTP protocol (see issue #195)

因此，如果您的delete.csv CSV文件的格式如下:

So if your delete.csv CSV file is formatted like this:

id
12345
12346
12347

您的delete.conf Logstash配置如下:

And your delete.conf Logstash config looks like this:

input {
    file {
        path => "/path/to/your/delete.csv"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}
filter {
    csv {
        columns => ["id"]
    }
}
output {
    elasticsearch{
        action => "delete"
        host => "localhost"
        port => 9300                         <--- make sure you have this
        protocol => "transport"              <--- make sure you have this
        index => "your_index"                <--- replace this
        document_type => "your_doc_type"     <--- replace this
        document_id => "%{id}"
    }
}

然后，在运行bin/logstash -f delete.conf时，您将能够删除其CSV文件中指定了ID的所有文档.

Then when running bin/logstash -f delete.conf you'll be able to delete all the documents whose id is specified in your CSV file.

这篇关于elasticsearch使用logstash和csv删除文档的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！