本文介绍了弹性搜索批量API,管道和地理IP的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

  {index:{_ index: w,_ type:日志,_ id:325d05bb6900440e}} 
{id:325d05bb6900440e,country:US,ip ,协议:HTTP / 1.1,方法:GET,主机:xxxxx,user_agent:Mozilla / 5.0(Windows NT 10.0; WOW64)AppleWebKit / 537.36(KHTML, )Chrome / 54.0.2840.99 Safari / 537.36,uri:/?a =>< script> alert(1)< / script>,request_duration:1999872,triggered_rule_ids:[100030 ], 行动 : 挑战 cloudflare_location: 未知, occurred_at: 2017-01-23T17:38:58.46Z, rule_detail:[{ ID: ,描述:ARGS:A}],rule_message:Generic XSS Probing,type:waf,rule_id:100030}
pre>

我有一个ip在数据中,我想使用GEOIP插件转入经度和纬度。



我创建了一个pipleine:

  PUT _ingest / pipeline / geoip-info 
{
description:添加地理位置信息,
游标:[
{
geoip:{
field:ip,
target_field:client_geoip,
properties [location],
ignore_failure:true
}
}
]
}`

但是,当导入数据时,管道被忽略可以有人解释我如何修改批量API以通过管道传递信息,以便为我添加long和lat创建地图。



谢谢

解决方案

没有管道名称

  here 
|
V
{index:{_ index:waf,_ type:logs,_ id:325d05bb6900440e,pipeline:geoip-info}}
{id:325d05bb6900440e,country:US,ip:1.1.1.1,protocol:HTTP / 1.1主机:xxxxx,user_agent:Mozilla / 5.0(Windows NT 10.0; WOW64)AppleWebKit / 537.36(KHTML,像Gecko)Chrome / 54.0.2840.99 Safari / 537.36,uri:/?a = ><脚本>警报(1) - ; /脚本> 中, request_duration:1999872, triggered_rule_ids:[ 100030], 行动: 挑战, cloudflare_location: 未知, __,rule_message:Generic XSS Probing(通用XSS探测) ,type:waf,rule_id:100030}

或者你也可以在大量网址中设置

  POST _bulk?pipeline = geoip-info 
/ pre>

I import data to my ELK stack using the Bulk API.

{"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e"}}
{"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}

I have an ip in the data that i want to turn in to longitude and latitude using the GEOIP addon.

I have created a pipleine:

PUT _ingest/pipeline/geoip-info
{
"description": "Add geoip info",
"processors": [
{
  "geoip": {
    "field": "ip",
    "target_field": "client_geoip",
    "properties": ["location"],
    "ignore_failure": true
  }
}
]
}`

However when I import the data the pipeline is ignored can someone explain how i modify the bulk API to pass the information through a pipeline in order to add long and lat for me to create maps.

Thanks

解决方案

In your bulk call you're missing the pipeline name

                                                                    here
                                                                      |
                                                                      V
{"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e", "pipeline": "geoip-info"}}
{"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}

Or you can also set it in the bulk URL

POST _bulk?pipeline=geoip-info

这篇关于弹性搜索批量API,管道和地理IP的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-20 01:30