无法使用logstash从外部文件加载索引到弹性搜索

本文介绍了无法使用logstash从外部文件加载索引到弹性搜索的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述 29岁程序员，3月因学历无情被辞！ 我在同一台计算机上并行运行一个弹性实例和一个logstash。 当尝试将文件加载到弹性文件中时，使用正在运行的logstash下面的配置文件，我得到以下输出msgs弹性和没有文件加载（当输入被配置为stdin一切似乎工作正常） 任何想法？ [2014-06-17 22：42：24,748] [INFO] [集群。服务] [Masked Marvel]删除{[logstash-Eitan-PC-5928-2010] [Ql5fyvEGQyO96R9NIeP32g] [Eitan-PC] [inet [Eitan-PC / 10.0.0.5：9301]] {client = true，data = false} ，}，原因：zen-disco-node_failed（[logstash-Eitan-PC-5928-2010] [Ql5fyvEGQyO96R9NIeP32g] [Eitan-PC] [inet [Eitan-PC / 10.0.0.5：9301]] {client = true，data = false}），原因运输断开（wi 验证连接） [2014-06-17 22： 43：00,686] [INFO] [cluster.service] [Masked Marvel]添加了{[logstash-Eitan-PC-5292-4014] [m0Tg-fcmTHW9aP6zHe UqTA] [Eitan-PC] [inet [/10.0.0.5:9301]] {client = true，data = false}，}，reason：zen-disco-receive（join from node [[logstash-Eitan-PC-5292在[/ code> et [/10.0.0.5:9301]] {client = true，data = false}]中，[m0Tg-fcmTHW9aP6zHeUqTA] [Eitan-PC]] p> 配置文件： 输入{文件{ path => c：\testLog.txt} } 输出{ elasticsearch {host => localhost index => amat1 } } 解决方案> 当您使用弹性搜索作为输出时 http://logstash.net/ docs / 1.4.1 / outputs / elasticsearch 而不是elasticsearch_http http： //logstash.net/docs/1.4.1/outputs/elasticsearch_http 您将要设置协议。 原因是它可以有3个不同的值，每个都有不同行为的节点，http或传输，默认选择没有很好的记录。 从外观您的日志文件似乎正在尝试使用节点协议，因为我看到端口9301上的连接尝试，其指示（以及其他日志条目）logstash尝试将该集群作为节点加入。这可能会导致任何数量的原因，包括群集名称上的不匹配。 我建议将协议设置为http - 以前更改已经修复了类似的问题。 / p> 另请参见： http://logstash.net/docs/1.4.1/outputs/elasticsearch#cluster http://logstash.net/docs/1.4.1/outputs/elasticsearch#protocol 编辑： 我在配置中看到的其他一些问题 - 您的主机和索引应该是字符串，在logstash config 文件中应该用双引号localhost和amat1包装。 没有报价可以工作，但他们建议您使用报价。 http://logstash.net/docs/1.4.1/configuration#string 如果你不要使用http作为协议，也不要使用elasticsearch_http作为输出，您应该将集群设置为您的ES集群名称（因为它将试图成为集群）。 您应该将start_position设置为文件中的开始。 否则它将默认从文件的末尾读取，而将看不到任何数据。这是Windows的一个特别的问题，现在作为跟踪文件中位置的另一种方式，sincedb是在Windows上断开： https://logstash.jira.com/browse/LOGSTASH-1587 http://logstash.net/docs /1.4.1/inputs/file#start_position 您应该将您的日志文件路径更改为： C：/testLog.txt。 Logstash更喜欢Windows下的斜杠和大写驱动器号。 https：//logstash.jira。 com / browse / LOGSTASH-430 i am running one instance of elastic and one of logstash in parallel on the same computer.when trying to load a file into elastic, using logstash that is running the config file below, i get the follwing output msgs on elastic and no file is loaded(when input is configured to be stdin everything seems to be working just fine)any ideas?" [2014-06-17 22:42:24,748][INFO ][cluster.service ] [Masked Marvel] removed {[logstash- Eitan-PC-5928-2010][Ql5fyvEGQyO96R9NIeP32g][Eitan-PC][inet[Eitan-PC/10.0.0.5:9301]]{client=true, data=false},}, reason: zen-disco-node_failed([logstash-Eitan-PC-5928-2010][Ql5fyvEGQyO96R9NIeP32g][Eitan-PC][inet[Eitan-PC/10.0.0.5:9301]]{client=true, data=false}), reason transport disconnected (with verified connect)[2014-06-17 22:43:00,686][INFO ][cluster.service ] [Masked Marvel] added {[logstash-Eitan-PC-5292-4014][m0Tg-fcmTHW9aP6zHeUqTA][Eitan-PC][inet[/10.0.0.5:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[logstash-Eitan-PC-5292-4014][m0Tg-fcmTHW9aP6zHeUqTA][Eitan-PC][inet[/10.0.0.5:9301]]{client=true, data=false}])"config file: input { file { path => "c:\testLog.txt" } } output { elasticsearch { host => localhost index=> amat1 }} 解决方案 When you use "elasticsearch" as your output http://logstash.net/docs/1.4.1/outputs/elasticsearch as opposed to "elasticsearch_http" http://logstash.net/docs/1.4.1/outputs/elasticsearch_http you are going to want to set "protocol".The reason is that it can have 3 different values, "node", "http" or "transport" with different behavior for each and the default selection is not well documented.From the look of your log files it appears it's trying to use "node" protocol as I see connection attempts on port 9301 which indicates (along with other log entries) that logstash is trying to join the cluster as a node. This can fail for any number of reasons including mismatch on the cluster name.I'd suggest setting protocol to "http" - that change has fixed similar issues before.See also:http://logstash.net/docs/1.4.1/outputs/elasticsearch#clusterhttp://logstash.net/docs/1.4.1/outputs/elasticsearch#protocolEDIT:A few other issues I see in your config -Your host and index should be strings, which in a logstash configfile should be wrapped with double quotes, "localhost" and "amat1".No quotes may work but they recommend you use quotes.http://logstash.net/docs/1.4.1/configuration#stringIf you don't use "http" as the protocol or don't use"elasticsearch_http" as the output you should set cluster equal toyour ES cluster name (as it will be trying to become a node of thecluster).You should set start_position under file in input to "beginning".Otherwise it will default to reading from the end of the file and youwon't see any data. This a particular problem with Windows right nowas the other way of tracking position within a file, sincedb, isbroken on Windows:https://logstash.jira.com/browse/LOGSTASH-1587http://logstash.net/docs/1.4.1/inputs/file#start_positionYou should change your path to your log file to this:"C:/testLog.txt". Logstash prefers forward slashes and upper casedrive letters under Windows.https://logstash.jira.com/browse/LOGSTASH-430 这篇关于无法使用logstash从外部文件加载索引到弹性搜索的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！ 上岸，阿里云！