问题描述
我希望以不同的方式处理不同的字段.
I want different fields to be processed in different way.
我有两个管道.一种是处理布尔值,另一种是将字符串转换为数组.
I have two pipelines. One is to process boolean values, another one is to convert a string to array.
output {
stdout {
codec => rubydebug
}
elasticsearch {
action => "index"
hosts => ["127.0.0.1:9200"]
index => "mini_system"
document_id => "%{mini_system_key}"
if [source] == "secure_flag" {
pipeline => "bool-pipeline"
} else if "partners" == %{FIELD} {
pipeline => "partners-pipeline"
}
}
}
我正在尝试这样做.但是我无法做到这一点,也找不到参考.
I am trying to do this. But I am not able to achieve this and couldn't find a reference also.
示例文档:
key,partners,secure_flag,date_added
5369922730525,"1002300,1009747,12359,2285459",FALSE,2020-03-31T14:00:00Z
2218100624,,FALSE,2020-03-31T14:00:00Z
在这里
"1002300,1009747,12359,2285459"是合作伙伴.FALSE是secure_flag.
"1002300,1009747,12359,2285459" is partners.FALSE is secure_flag.
合作伙伴管道:
{
"description": "Converts \"a,b,c\" to [\"a\", \"b\",\"c\"]",
"processors" : [
{
"split" : {
"field" : "partners",
"separator": ",",
"ignore_missing": true
}
}
]
}
推荐答案
您不能在插件配置中应用逻辑,但是使用if/else逻辑肯定可以得到多个输出:
You cannot apply logic inside plugin configurations, but you can definitely have several output using if/else logic:
output {
stdout {
codec => rubydebug
}
if [source] == "secure_flag" {
elasticsearch {
action => "index"
hosts => ["127.0.0.1:9200"]
index => "mini_system"
document_id => "%{mini_system_key}"
pipeline => "bool-pipeline"
}
} else if [field_xyz] == "partners" {
elasticsearch {
action => "index"
hosts => ["127.0.0.1:9200"]
index => "mini_system"
document_id => "%{mini_system_key}"
pipeline => "partners-pipeline"
}
}
}
更新:
您实际上不需要任何逻辑,只需将两个处理器添加到同一管道中即可.
You don't actually need any logic, but simply add both of your processors in the same pipeline:
PUT _ingest/pipeline/mini-pipeline
{
"processors" : [
{
"convert" : {
"field" : "secure_flag",
"type": "boolean",
"ignore_missing": true
}
},
{
"split" : {
"field" : "partners",
"separator": ",",
"ignore_missing": true
}
}
]
}
然后只需使用此配置
output {
stdout {
codec => rubydebug
}
elasticsearch {
action => "index"
hosts => ["127.0.0.1:9200"]
index => "mini_system"
document_id => "%{mini_system_key}"
pipeline => "mini-pipeline"
}
}
这篇关于Logstash-指定多个管道的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!