retrying failed action with response code: 403

0x00 Event

[2019-09-24T19:22:31,655][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2019-09-24T19:22:31,655][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2019-09-24T19:22:31,655][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

[2019-09-24T19:22:31,655][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})

0x01 Solution

$ curl -XPUT -H 'Content-Type: application/json' http://192.168.1.146:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

Official Solution:

https://www.elastic.co/guide/en/elasticsearch/reference/6.4/disk-allocator.html