现在开始在业务上使用ES,记录一些踩坑经历,做点笔记.
2018-11-13
source不返回问题
使用了角色校验,客户端插入成功之后获取数据没有source,和查询参数无关.
检查mapping,发现获取mapping也是空...
如下:
{'test_index': {'mappings': {'test_doc': {'properties': {}}}}}
排查了一会儿..找不出原因.
后来要到了一个高权限的账号去kibana看了眼...发现
能获取的fields为空... ...
emmmmm....
设置为*后解决
参考链接:
https://www.elastic.co/guide/en/elastic-stack-overview/6.4/field-level-security.html
2018-11-16
termQuery不返回结果
emmmmm 一个id是用了dynamic mapping插入,默认使用的是standard分词器.
一些id有中划线,分词结果:
GET /_analyze
{
"analyzer": "standard",
"text": "1111-2222-4444-3333-55555"
}
{
"tokens": [
{
"token": "1111",
"start_offset": 0,
"end_offset": 4,
"type": "<NUM>",
"position": 0
},
{
"token": "2222",
"start_offset": 5,
"end_offset": 9,
"type": "<NUM>",
"position": 1
},
{
"token": "4444",
"start_offset": 10,
"end_offset": 14,
"type": "<NUM>",
"position": 2
},
{
"token": "3333",
"start_offset": 15,
"end_offset": 19,
"type": "<NUM>",
"position": 3
},
{
"token": "55555",
"start_offset": 20,
"end_offset": 25,
"type": "<NUM>",
"position": 4
}
]
}
然后在代码里使用的是termQuery:
SearchRequest searchRequest = new SearchRequest(INDEX_NAME);
searchRequest.types(TYPE_NAME);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.termQuery(ID, id));
termQuery是不对搜索词分词的...
于是就什么都查不出来..
解决方式是用dynamic mapping自动生成的keyword field.
参考:
https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-analyzers.html
2018-12-14
处理深分页
默认的from+size限制是10000,大于这个会报错:
{
"error": {
"root_cause": [
{
"type": "query_phase_execution_exception",
"reason": "Result window is too large, from + size must be less than or equal to: [1000000] but was [1000001]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."
}
],
深分页的操作对coordinator会造成很大的影响,会占用大量heap存放数据并进行排序操作.
业务上没有注意,结果就超了... ...
可以设定index.max_result_window来提高上限.
但如果真的要有这么深,还是使用search after比较可靠.又因为是实时业务查询,所以用scroll是不合适的.
参考:
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html