Solr Mongo/DocDB Delta 导入查询不起作用

问题描述

我正在尝试将数据从文档数据库导入到 solr-5.4.1.完全导入正常执行，但增量导入不起作用.当我执行 delta import 时什么也没发生

I am trying to import data from Document DB to solr-5.4.1. Full import is executing properly but delta import is not working. When I execute delta import nothing happens

这是当前的增量配置

deltaQuery="{'lastUpdatedDate':{$exists:true},'lastUpdatedDate':{$gt: Date('${dih.last_index_time}')}}"
            deltaImportQuery="{'_id':'${dih.delta._id}'}"> 

整个 db-data-config.xml

whole db-data-config.xml

<dataConfig>
   <propertyWriter dateFormat="yyyy-MM-dd'T'HH:mm:ss.SSSXXX" type="SimplePropertiesWriter" filename="dataimport.properties"/>

   <document name="tvTitleSearch">
        <entity name="tvTitleSearch"
            processor="MongoEntityProcessor"
            query=""
            collection="tvTitleSearch" 
            datasource="MyMongo"
            transformer="MongoMapperTransformer"
            deltaQuery="{'lastUpdatedDate':{$exists:true},'lastUpdatedDate':{$gt: Date('${dih.last_index_time}')}}"
            deltaImportQuery="{'_id':'${dih.delta._id}'}"> 

                       <field name="id" column="_id" indexed="true" type="uuid" stored="true" mongoField="_id"/>
                      <field column="lastUpdatedDate" sourceColName="lastUpdatedDate" dateTimeFormat="yyyy-MM-dd'T'HH:mm:ss.SSSXXX" locale="en"/>

         </entity>
    </document>

</dataConfig>

来自托管架构文件的示例

sample from managed-schema file

<field type="date" name="modifiedDate"  indexed="true" stored="true"/>
<field type="date" name="lastUpdatedDate"  indexed="true" stored="true"/>

存储在 DocumentDB 中的值:

the value stored in DocumentDB:

lastUpdatedDate:2019-11-24T11:43:46.045+00:00

我尝试遵循存储在 DocumentDB 中的日期时间格式，但仍然没有奏效，此外，我尝试了此处提出的其他类似问题的建议，但没有成功.我在增量查询中尝试了以下代码段

I tried to follow the date-time format as stored in DocumentDB, but still, it did not work, also, I tried suggestions from other similar question asked here but no luck. I tried below snippet in delta query

{lastUpdatedDate : {$gt: ISODate('${dih.last_index_time}')}} , but I got JSON Prasing Excpetion

但低于异常

java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: com.mongodb.util.JSONParseException: 
{'lastUpdatedDate':{$exists:true},'lastUpdatedDate':{$gt: ISODate('2019-11-20T13:14:30.576-08:00')}}

我也尝试过以下配置

deltaQuery="{'lastUpdatedDate':{$exists:true},'lastUpdatedDate':{$gt: 
{$date:'${dih.last_index_time}'}}}"

有人可以帮助我或提供任何建议来触发增量查询

could someone please help me or provide any suggestions to trigger delta query

推荐答案

使用deltaQuery的正确方法是

The correct way to use the deltaQuery is

deltaQuery="{'lastUpdatedDate':{$exists:true},'lastUpdatedDate':{$gt: {$date:'2020-03-13T08:44:06.959Z'}}}"

如您所见此处

但是，SOLR 将使用 ${dih.last_index_time}，默认格式为 2020-03-13 08:44:06.您需要通过在 <dateConfig> 元素中添加类似以下设置的内容来更改格式.

However, the SOLR will use ${dih.last_index_time} that is by default formated as 2020-03-13 08:44:06. You'll need to change the format by adding something like the following setting inside your <dateConfig> element.

<propertyWriter dateFormat="yyyy-MM-dd'T'HH:mm:ss.SSSXXX" type="SimplePropertiesWriter" filename="my_dih.properties" locale="en-US"  />

这篇关于Solr Mongo/DocDB Delta 导入查询不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！