在摄取管道中使用搜索模板

问题描述

ElasticSearch 摄取管道可以使用搜索模板作为其脚本吗?

Can an ElasticSearch ingest pipeline use a search template as its script?

具体来说，我想配置一个摄取管道，以便每当特定类型的数据进来时，我们查询 ElasticSearch 以获取一些相关数据，并根据结果在原始数据上填充一个(或多个)字段，在加载之前.

Specifically, I'd like to configure an ingest pipeline so that whenever data of a particular type come in, we query ElasticSearch for some related data, and populate a field (or fields) on the original data based on the results, before loading it in.

我看到摄取管道可以使用脚本(https://www.elastic.co/guide/en/elasticsearch/reference/master/script-processor.html)，并且该脚本可以包含搜索模板(https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html)，但我无法找到有关可能将两者结合使用的任何信息.

I see that ingest pipelines can use scripts (https://www.elastic.co/guide/en/elasticsearch/reference/master/script-processor.html), and that scripts can include search templates (https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html), but I haven't been able to find any information on potentially combining the two.

推荐答案

实际上，您要求的功能是通过新的 丰富处理器，提供索引时JOIN能力.

Actually, the feature you're asking for is coming in 7.5 via the new enrich processor, which kind of provides index-time JOIN capability.

主要思想是建立一个丰富策略，它将从您的相关索引中获取数据到一个新的丰富索引"中，然后您可以使用 丰富处理器，以便用相关字段丰富您的文档.

The main idea is to set up an enrich policy that will source data from your related indexes into a new "enrich index" and then you can leverage that "enrich index" in your ingest pipeline using an enrich processor in order to enrich your documents with related fields.

因此，在不涉及太多细节的情况下，以下是它在实践中的工作原理:

So, without going in too many details, here is how it works in practice:

1. 你有一个索引 A 和字段 (a, b, c, d)，你想用它来丰富你的传入文档
2. 您根据该索引 A 和加入"字段 a
定义丰富策略
3. 您定义了一个带有丰富处理器的摄取管道，它将尝试将传入文档的字段 z 与丰富索引的字段 A.a 进行匹配
4. 如果找到匹配项，您的传入文档将从索引 A 中获取字段 b、c 和 d代码>.请注意，它还会获取匹配字段 a，您可以根据需要使用 remove 处理器删除该字段.

1. You have an index A with fields (a, b, c, d) that you'd like to use for enriching your incoming documents
2. You define an enrich policy based on that index A and the "join" field a
3. You define an ingest pipeline with an enrich processor that will try to match field z of the incoming document against field A.a of the enrich index
4. If a match is found, your incoming document will get fields b, c and d from the index A. Note that it will also get the match field a that you can remove using a remove processor if needed.

这正是您所期望的.您可以在此处找到完整示例.一开始，它适用于精确匹配(即 term 查询)和地理匹配(即 geo_shape 查询)，但它们可能会添加新类型的匹配(如 range 匹配)在不久的将来.

That's pretty much what you expect. You can find a complete example here. At the beginning, it will work for exact matches (i.e. term query) and geo matches (i.e. geo_shape query), but they will probably add new kind of matches (like range matches) in the near future.

这篇关于在摄取管道中使用搜索模板的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！