elasticsearch - 增加日志消息的最大大小

我想通过elasticsearch增加在Graylog中保存的消息的最大大小。

邮件大小上限为32 kb

因此，我更新了映射以删除full_message上的索引:
graylog-custom-mapping.json:

{
  "template": "graylog_*",
  "mappings": {
    "message": {
      "properties": {
        "full_message": {
          "index": "no",
          "doc_values": false,
          "type": "string"
        }
      }
    }
  }
}

curl -X PUT -d @'graylog-custom-mapping.json' 'http://localhost:9200/_template/graylog-custom-mapping?pretty'

{
  "acknowledged" : true
}

我从Graylog界面(系统>索引>维护>手动循环偏转器)创建了一个新的Graylog索引(graylog_5)。

但是我的映射似乎不被考虑:
curl -X GET 'http://localhost:9200/graylog_5/_mapping/message'

...
"full_message": {
    "type": "string",
    "analyzer": "standard"
},
...

我的 Activity 模板:

{
  "graylog-internal": {
    "order": -2147483648,
    "template": "graylog_*",
    "settings": {
      "index": {
        "analysis": {
          "analyzer": {
            "analyzer_keyword": {
              "filter": "lowercase",
              "tokenizer": "keyword"
            }
          }
        }
      }
    },
    "mappings": {
      "message": {
        "_source": {
          "enabled": true
        },
        "dynamic_templates": [
          {
            "internal_fields": {
              "mapping": {
                "index": "not_analyzed",
                "type": "string"
              },
              "match": "gl2_*"
            }
          },
          {
            "store_generic": {
              "mapping": {
                "index": "not_analyzed"
              },
              "match": "*"
            }
          }
        ],
        "properties": {
          "full_message": {
            "analyzer": "standard",
            "index": "analyzed",
            "type": "string"
          },
          "streams": {
            "index": "not_analyzed",
            "type": "string"
          },
          "source": {
            "analyzer": "analyzer_keyword",
            "index": "analyzed",
            "type": "string"
          },
          "message": {
            "analyzer": "standard",
            "index": "analyzed",
            "type": "string"
          },
          "timestamp": {
            "format": "yyyy-MM-dd HH:mm:ss.SSS",
            "type": "date"
          }
        }
      }
    },
    "aliases": {}
  },
  "graylog-custom-mapping": {
    "order": 0,
    "template": "graylog_*",
    "settings": {},
    "mappings": {
      "message": {
        "properties": {
          "full_message": {
            "index": "no",
            "type": "string",
            "doc_values": false
          }
        }
      }
    },
    "aliases": {}
  }
}

我的配置有什么问题？

Graylog 2.1.2 + ES 2.4.2

我有以下日志:

[2018-02-16 16:26:36,598][INFO ][cluster.metadata         ] [Zero] [graylog_5] creating index, cause [api], templates [graylog-internal, graylog-custom-mapping], shards [4]/[0], mappings [message]
[2018-02-16 16:26:37,091][INFO ][cluster.routing.allocation] [Zero] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[graylog_5][1], [graylog_5][2], [graylog_5][0], [graylog_5][2], [graylog_5][0]] ...]).
[2018-02-16 16:27:03,665][INFO ][cluster.metadata         ] [Zero] [graylog_5] update_mapping [message]
[2018-02-16 16:27:03,816][INFO ][cluster.metadata         ] [Zero] [graylog_5] update_mapping [message]

谢谢

最佳答案

问题不是Elasticsearch索引编制，而是从rsyslog udp协议(protocol)上载的最大邮件大小。

要解决这个问题:
/etc/rsyslog.conf
并定义
$MaxMessageSize 256k
在第一行。

关于elasticsearch - 增加日志消息的最大大小，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/48857477/