问题描述
我正在使用 MongoDB 生成这种格式的唯一 ID:
{ID TYPE}{ZONE}{ALPHABET}{YY}{XXXX}
这里 ID TYPE 将是 {U, E, V} 中的字母表，取决于输入，区域将来自 {N, S, E, W}, YY 将是当前年份的最后 2 位，XXXXX 将是从 0 开始的 5 位数字(将填充0 使其长度为 5 位数字).当 XXXXX 到达 99999 时，ALPHABET 部分将递增到下一个字母表(从 A 开始).
我将接收 ID TYPE 和 ZONE 作为输入，并且必须提供生成的唯一 ID 作为输出.每次，我都必须生成一个新 ID，我将读取为给定的 ID TYPE 和 ZONE 生成的最后一个 ID，将数字部分增加 1 (XXXXX + 1) 和然后将新生成的 ID 保存在 MongoDB 中，并将输出返回给用户.
此代码将在单个 NodeJS 服务器上运行，并且可以有多个客户端调用此方法如果我只运行单个服务器实例，是否有可能出现如下所述的竞争条件:
- 第一个客户端读取最后生成的 ID 为
USA2100000 - 第二个客户端读取最后生成的 ID 为
USA2100000 - 第一个客户端生成新 ID 并将其保存为
USA2100001 - 第二个客户端生成新 ID 并将其保存为
USA2100001
由于 2 个客户端生成了 ID，因此 DB 应该具有 USA2100002.
为了克服这个问题，我使用了 MongoDB 事务.我在 Typescript 中使用 Mongoose 作为 ODM 的代码是这样的:
session = await startSession();session.startTransaction();lastId = await GeneratedId.findOne({ key: idKeyStr }, "value").valuelastId = createNextId(lastId);const newIdObj: 任何 = {键:`类型:${idPrefix}_Zone:${zone_letter}`，值:lastId，};等待 GeneratedId.findOneAndUpdate({ key: idKeyStr }, newIdObj, {upsert:真的，新:真实，});等待 session.commitTransaction();session.endSession();- 我想知道当我遇到这种情况时到底会发生什么这段代码会发生上述情况吗?
- 第二个客户端的事务是否会抛出异常，我必须在我的代码中中止或重试事务，还是由它自己处理重试?
- MongoDB 或其他数据库如何处理事务?MongoDB 是否锁定事务中涉及的文档?是否有排他锁(甚至不允许其他客户端读取)?
- 如果同一个客户端一直未能提交其事务，则该客户端将被饿死.如何应对这种饥饿?
您正在使用 MongoDB 来存储 ID.这是一种状态.ID的生成是一个函数.当 mongodb 进程接受函数的参数并返回生成的 ID 时，您使用 Mongodb 生成 ID.这不是你在做什么.您正在使用 nodejs 生成 ID.
线程数，或者说事件循环是至关重要的，因为它定义了架构，但无论哪种方式，您都不需要事务.mongodb 中的事务被称为多文档事务".正是为了强调它们旨在同时更新多个文档.
如果您可能需要扩展到 1 个以上的 nodejs 进程来处理更多并发请求或添加另一个主机以在将来实现冗余，那么您将需要同步 ID 的生成，并且您可以使用 Mongodb唯一索引.函数本身没有太大变化，您仍然像在单线程架构中一样生成 ID，但添加了一个额外的步骤来将 ID 保存到 mongo.文档应该在 ID 字段上具有唯一索引，因此在并发更新的情况下，一个查询将成功添加文档，另一个将失败并显示E11000 重复键错误".您在 nodejs 端捕获此类错误并再次选择下一个数字重复该函数:
I am using MongoDB to generate unique IDs of this format:
{ID TYPE}{ZONE}{ALPHABET}{YY}{XXXX}
Here ID TYPE will be an alphabet from {U, E, V} depending on the input, zone will be from the set {N, S, E, W}, YY will be the last 2 digits of the current year and XXXXX will be a 5 digit number beginning from 0 (willbe padded with 0s to make it 5 digits long). When XXXXX reaches 99999, the ALPHABET part will be incremented to the next alphabet (starting from A).
I will receive ID TYPE and ZONE as input and will have to give the generated unique ID as output. Everytime, I have to generate a new ID, I will read the last generated for the given ID TYPE and ZONE, increment the number part by 1 (XXXXX + 1) and then save the new generated ID in MongoDB and return the output to the user.
This code will be run on a single NodeJS server and there can be multiple clients calling this methodIs there a possibility of a race condition like the once described below if I am ony running a single server instance:
- First client reads last generated ID as
USA2100000 - Second client reads last generated ID as
USA2100000 - First client generates the new ID and saves it as
USA2100001 - Second client generates the new ID and saves it as
USA2100001
Since 2 clients have generated IDs, finally the DB should have had USA2100002.
To overcome this, I am using MongoDB transactions. My code in Typescript using Mongoose as ODM is something like this:
session = await startSession();
session.startTransaction();
lastId = await GeneratedId.findOne({ key: idKeyStr }, "value").value
lastId = createNextId(lastId);
const newIdObj: any = {
key: `Type:${idPrefix}_Zone:${zone_letter}`,
value: lastId,
};
await GeneratedId.findOneAndUpdate({ key: idKeyStr }, newIdObj, {
upsert: true,
new: true,
});
await session.commitTransaction();
session.endSession();
- I want to know what exactly will happen when the situation Idescribed above happens with this code?
- Will the second client's transaction throw an exception and I have to abort or retry the transaction in my code or will it handle the retry on its own?
- How does MongoDB or other DBs handle transactions? Does MongoDB lock the documents involved in the transaction? Are the exclusive locks (wont even allow other clients to read)?
- If the same client keeps failing to commit its transaction, this client would be starved. How to deal with this starvation?
You are using MongoDB to store the ID. It's a state. Generation of the ID is a function. You use Mongodb to generate the ID when mongodb process takes arguments of the function and returns the generated ID. It's not what you are doing. You are using nodejs to generate the ID.
Number of threads, or rather event loops is critical as it defines the architecture but in either way you don't need transactions. Transactions in mongodb are being called "multi-document transactions" exactly to highlight they are intended for consistent update of several documents at once. The very first paragraph of https://docs.mongodb.com/manual/core/transactions/ warns you that if you update a single document there is no room for transactions.
A single threaded application does not require any synchronisation. You can reliably read the latest generated ID on start and guarantee the ID is unique within the nodejs process. If you exclude mongodb and other I/O from the generation function you will make it synchronous so you can maintain state of the ID within nodejs process and guarantee its uniqueness. Once generated you can persist in in the db asynchronously. In the worst case scenario you may have a gap in the sequential numbers but no duplicates.
If there is a slighteest chance that you may need to scale up to more than 1 nodejs process to handle more simultaneous requests or add another host for redundancy in the future you will need to sync generation of the ID and you can employ Mongodb unique indexes for that. The function itself doesn't change much you still generate the ID as in a single-threaded architecture but add an extra step to save the ID to mongo. The document should have unique index on the ID field, so in case of concurrent updates one of the query will successfully add the document and another will fail with "E11000 duplicate key error". You catch such errors on nodejs side and repeat the function again picking the next number:
这篇关于使用 MongoDB + NodeJS 生成唯一 ID 时处理竞争条件和饥饿的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！