问题描述
我有以下代码.
async function bulkInsert(db, collectionName, documents) {
try {
const cosmosResults = await db.collection(collectionName).insertMany(documents);
console.log(cosmosResults);
return cosmosResults
} catch (e) {
console.log(e)
}
}
如果我使用大量文档来运行它，我会(并非意外)
If I run it with a large array of documents I get ( not unexpectedly)
{ MongoError: Message: {"Errors":["Request rate is large"]}
ActivityId: b3c83c38-0000-0000-0000-000000000000,
Request URI: /apps/DocDbApp/services/DocDbServer24/partitions/a4cb4964-38c8-11e6-8106-8cdcd42c33be/replicas/1p/,
RequestStats: , SDK: Microsoft.Azure.Documents.Common/1.19.102.5
at G:\Node-8\NodeExample\node_modules\oracle-movie-ticket-demo\node_modules\mongodb-core\lib\connection\pool.js:596:61
at authenticateStragglers (G:\Node-8\NodeExample\node_modules\oracle-movie-ticket-demo\node_modules\mongodb-core\lib\connection\pool.js:514:16)
at Connection.messageHandler (G:\Node-8\NodeExample\node_modules\oracle-movie-ticket-demo\node_modules\mongodb-core\lib\connection\pool.js:550:5)
at emitMessageHandler (G:\Node-8\NodeExample\node_modules\oracle-movie-ticket-demo\node_modules\mongodb-core\lib\connection\connection.js:309:10)
at TLSSocket.<anonymous> (G:\Node-8\NodeExample\node_modules\oracle-movie-ticket-demo\node_modules\mongodb-core\lib\connection\connection.js:452:17)
at emitOne (events.js:116:13)
at TLSSocket.emit (events.js:211:7)
at addChunk (_stream_readable.js:263:12)
at readableAddChunk (_stream_readable.js:250:11)
at TLSSocket.Readable.push (_stream_readable.js:208:10)
name: 'MongoError',
message: 'Message: {"Errors":["Request rate is large"]}\r\nActivityId: b3c83c38-0000-0000-0000-000000000000,
Request URI: /apps/DocDbApp/services/DocDbServer24/partitions/a4cb4964-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, RequestStats: , SDK: Microsoft.Azure.Documents.Common/1.19.102.5',
_t: 'OKMongoResponse',
ok: 0,
code: 16500,
errmsg: 'Message: {"Errors":["Request rate is large"]}\r\nActivityId: b3c83c38-0000-0000-0000-000000000000,
Request URI: /apps/DocDbApp/services/DocDbServer24/partitions/a4cb4964-38c8-11e6-8106-8cdcd42c33be/replicas/1p/,
RequestStats: ,
SDK: Microsoft.Azure.Documents.Common/1.19.102.5',
'$err': 'Message: {"Errors":["Request rate is large"]}\r\nActivityId: b3c83c38-0000-0000-0000-000000000000,
Request URI: /apps/DocDbApp/services/DocDbServer24/partitions/a4cb4964-38c8-11e6-8106-8cdcd42c33be/replicas/1p/, RequestStats: ,
SDK: Microsoft.Azure.Documents.Common/1.19.102.5' }
看来我正在处理的740条记录中的某些(大约165条)已被加载.他们似乎都被分配了"_id"属性.
It appears that some (approx. 165) of the 740 records I was processing have been loaded. All of them appear to have been assigned '_id' attributes.
是否有人知道如何处理(或至少告诉我们插入了哪些记录，哪些不是进程)...
Does anyone have any idea how to handle this (or at least tell which records were inserted and which were not processes)...
推荐答案
使用cosmosdb的请求需要使用RU.显然，您的插入请求超出了RU吞吐量，并且发生了错误代码16500.
Requests with cosmosdb need to consume RUs. Obviously, your insert request exceeded the RU throughput and error code 16500 occurred.
您可以从官方文件.
您可以按照以下方法尝试解决此问题:
You could follow the ways as below to try to solve the issue:
-
批量导入数据以降低吞吐量.
Import your data in batches to reduce throughput.
在您的应用程序中添加您自己的重试逻辑.
Add your own retry logic in your application.
增加集合的保留吞吐量.当然，这会增加您的成本.
Increasing the reserved throughput for the collection. Of course, it increases your cost.
您可以参考文章.
希望它对您有帮助.
更新答案:
Update Answer:
看来您的文件不是唯一可识别的.因此，我认为Cosmos DB自动生成的"_id"属性无法确定已插入哪些文档和尚未插入哪些文档.
It looks like your documents are not uniquely identifiable. So I think the "_id" attribute which automatically generated by Cosmos DB cannot determine which documents have been inserted and which documents have not been inserted.
我建议您增加吞吐量设置，清空数据库，然后批量导入数据.
I suggest you increasing throughput settings, empty the database and then bulk import the data.
考虑到费用，请参阅此文档设置适当的RU.
Considering the cost , please refer to this document for setting the appropriate RU.
或者您可以通过 Cosmos DB在本地测试批量导入操作模拟器.
这篇关于Cosmos DB Mongo API如何管理“请求率很大"健康)状况的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！