对Elasticsearch和mongodb来说是新手，
我在批量索引中将数据从mongodb导入es时遇到了问题。
要使用mongo-connector和elastic2-doc-manager同步数据。
我能够成功地从一个集合中获取数据。但是，涉及到mongodb中django2_tables创建的元数据集合。 Elasticsearch出错了。
顺便说一句，它较早就可以正常工作，不确定为什么以及如何。也许我缺少或忘记了一些大容量配置。
任何帮助将不胜感激。
谢谢，
附加来自elasticsearch和mongo-connector的日志。
来自elasticsearch的日志:
[2020-01-24T12:55:04,898][DEBUG][o.e.a.b.TransportShardBulkAction] [Qasims-MBP.hub] [djangodb01][0] failed to execute bulk item (index) index {[djangodb01][auth_permission][5e2a32540c31b2edc41dd9e6], source[{"id": 32, "name": "Can view project", "content_type_id": 8, "codename": "view_project"}]}
java.lang.IllegalArgumentException: Rejecting mapping update to [djangodb01] as the final mapping would have more than 1 type: [spiderCollection1, auth_permission]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:272) ~[elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:238) ~[elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:702) ~[elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:324) ~[elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:219) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.MasterService.access$000(MasterService.java:73) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:151) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.5.1.jar:7.5.1]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.5.1.jar:7.5.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:830) [?:?]
来自mongo-connector的日志:
2020-01-24 12:55:04,909 [ERROR] mongo_connector.doc_managers.elastic2_doc_manager:505 - Bulk request failed with exception
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/mongo_connector/doc_managers/elastic2_doc_manager.py", line 497, in send_buffered_operations
successes, errors = bulk(self.elastic, action_buffer)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/elasticsearch/helpers/__init__.py", line 257, in bulk
for ok, item in streaming_bulk(client, actions, **kwargs):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/elasticsearch/helpers/__init__.py", line 188, in streaming_bulk
for data, (ok, info) in zip(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/elasticsearch/helpers/__init__.py", line 137, in _process_bulk_chunk
raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
elasticsearch.helpers.BulkIndexError: ('40 document(s) failed to index.', [{'index': {'_index': 'djangodb01', '_type': 'django_content_type', '_id': '5e2a32540c31b2edc41dd9bf', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Rejecting mapping update to [djangodb01] as the final mapping would have more than 1 type: [spiderCollection1, django_content_type]'}, 'data': {'id': 1, 'app_label': 'admin', 'model': 'logentry'}}}, {'index': {'_index': 'djangodb01', '_type': 'auth_permission', '_id': '5e2a32540c31b2edc41dd9c0', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Rejecting mapping update to [djangodb01] as the final mapping would have more than 1 type: [spiderCollection1, auth_permission]'}, 'data': {'id': 1, 'name': 'Can add log entry', 'content_type_id': 1, 'codename': 'add_logentry'}}}, {'index': {'_index': 'djangodb01', '_type': 'auth_permission', '_id': '5e2a32540c31b2edc41dd9c1', 'status': 400, 'error': {'type': 'illegal_argument_exception', 'reason': 'Rejecting mapping update to [djangodb01] as the final mapping would have more than 1 type: [spiderCollection1, auth_permission]'}, 'data': {'id': 2, 'name': 'Can change log entry', 'content_type_id': 1, 'codename': 'change_logentry'}}},
最佳答案
在最新版本中，elasticsearch每个索引仅支持一种映射类型。 Here是详细信息。
您面临的问题仍未解决:https://github.com/yougov/mongo-connector/issues/770
Here is a potential workaround。如果这行不通，请考虑将ES降级为5.X版本，其中允许多个映射。从mongo导入后，您将能够升级到版本6.X。这是some background。