elasticsearch - 配置 Elasticsearch 输出时，似乎未启动Logstash

我在下面放了这个docker-compose.yml文件。
实际上，我需要使用http输入插件来馈送csv文件，这就是它出现在配置中的原因。

我正在使用该简单的curl行进行测试，只是为了测试基础知识是否正常，然后继续处理csv文件和其余配置。

version: '3.3'
services:
  logstash:
    image: docker.elastic.co/logstash/logstash:6.7.0
    command: bash -c 'bin/logstash-plugin install logstash-input-http && bin/logstash-plugin install logstash-output-elasticsearch && bin/logstash -e "input { http { port => 5044 } } filter { split {} csv {} } output { elasticsearch { hosts => '127.0.0.1:9200' } }"'
    links:
      - elasticsearch
    ports:
      - 5044:5044
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9200:9200"
  kibana:
    image: docker.elastic.co/kibana/kibana:6.7.0
    ports:
      - "5601:5601"

configs:
  logstash_config:
    file: ./configs/logstash.conf

volumes:
  esdata1:
    driver: local

当我尝试下面的命令时，它失败，如下所示:(Windows 10)

curl 'http://127.0.0.1:5044/twitter/tweet/2' -Method Put -Body 'hello2'
curl : The underlying connection was closed: The connection was closed unexpectedly.
At line:1 char:1
+ curl 'http://127.0.0.1:5044/twitter/tweet/2' -Method Put -Body 'hello ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

但是如果在使用此docker_compose.yml文件时可以工作:

version: '3.3'
services:
  logstash:
    image: docker.elastic.co/logstash/logstash:6.7.0
    command: bash -c 'bin/logstash -e "input { http { port => 5044 } } filter { split {} csv {} } output { stdout { codec => rubydebug} }" && bin/logstash-plugin install logstash-input-http && bin/logstash-plugin install logstash-output-elasticsearch'
    links:
      - elasticsearch
    ports:
      - 5044:5044
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - "9200:9200"
  kibana:
    image: docker.elastic.co/kibana/kibana:6.7.0
    ports:
      - "5601:5601"

configs:
  logstash_config:
    file: ./configs/logstash.conf

volumes:
  esdata1:
    driver: local

我启动相同的命令，现在它可以工作了。

 curl 'http://127.0.0.1:5044/twitter/tweet/2' -Method Put -Body 'hello2'


StatusCode        : 200
StatusDescription : OK
Content           : ok
RawContent        : HTTP/1.1 200 OK
                    Content-Length: 2
                    Content-Type: text/plain

                    ok
Forms             : {}
Headers           : {[Content-Length, 2], [Content-Type, text/plain]}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 2

我迫切需要将数据传递给 flex
任何关于出什么问题的想法都值得欢迎

提前致谢。

最佳答案

首先，您不必安装logstash插件。包括电池。可以列出插件:bin/logstash-plugin list
接下来，利用docker服务发现。您的服务elasticsearch可从您的logstash容器中解析。容器中的Localhost解析为-容器localhost，而不是主机localhost。

HOST localhost =! CONTAINER localhost (in bridge mode)

一个工作的docker-compose.yml

version: '3.3'
services:
  logstash:
    image: docker.elastic.co/logstash/logstash:6.7.0
    command: bash -c 'bin/logstash -e "input { http { port => 5044 } } filter { split {} csv {} } output { elasticsearch { hosts => [\"elasticsearch:9200\"] } }"'
    networks:
      - elastic-net
    ports:
      - 5044:5044

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    networks:
      - elastic-net
    ports:
      - "9200:9200"
  kibana:
    image: docker.elastic.co/kibana/kibana:6.7.0
    networks:
      - elastic-net
    ports:
      - "5601:5601"

networks:
  elastic-net:

configs:
  logstash_config:
    file: ./configs/logstash.conf

volumes:
  esdata1:
    driver: local

links是docker旧版构造。而是使用网络构造。

祝你好运。

关于elasticsearch - 配置 Elasticsearch 输出时，似乎未启动Logstash，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/59401447/