es 报错 Data too large 触发断路器

文章目录

事出有因

Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [<transport_request>] would be [1021339792/974mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1021337736/974mb], new bytes reserved: [2056/2kb], usages [request=0/0b, fielddata=1730/1.6kb, in_flight_requests=88822/86.7kb, accounting=12643373/12mb]

curl -s -XGET '10.190.107.78:19200/_cat/nodes?v=true&h=name,node*,heap*' 

name id   node.role heap.current heap.percent heap.max 
es-3 q8nd dilm           935.3mb           91      1gb 
es-2 U5GI dilm           700.2mb           68      1gb 
es-1 vgkL dilm           825mb             80      1gb 
es-0 UH0L dilm           511.6mb           49      1gb 

解决思路

ES Circuit Breaker 断路器

jvm 常见的垃圾收集器的组合方式

效果展示

curl -s -XGET '10.190.107.78:19200/_cat/nodes?v=true&h=name,node*,heap*' 

name id   node.role heap.current heap.percent heap.max 
es-1 vgkL dilm           431.6mb           43  989.8mb 
es-0 UH0L dilm           337.1mb           34  989.8mb 
es-2 U5GI dilm           500.5mb           50  989.8mb 
es-3 q8nd dilm           730.3mb           73  989.8mb

关于重启

curl -XPUT 10.190.107.78:19200/_cluster/settings -H 'Content-Type: application/json' -d '
{
    "transient": {
        "cluster.routing.allocation.enable": "none"
    }
}'

curl -XPUT 10.190.107.78:19200/_cluster/settings -H 'Content-Type: application/json' -d '
{
    "transient": {
        "cluster.routing.allocation.enable": "all"
    }
}'

课外扩展

限制内存使用