Ndb强一致性和频繁写入

问题描述

我正在尝试使用 python 与 ndb 实现强一致性.看起来我错过了一些东西，因为我的阅读表现得好像不是很一致.

I'm trying to achieve strong consistency with ndb using python.And looks like I'm missing something as my reads behave like they're not strongly consistent.

查询是:

links = Link.query(ancestor=lead_key).filter(Link.last_status ==
None).fetch(keys_only=True)

if links:
    do_action()

关键结构是:

Lead root (generic key) -> Lead -> Website (one per lead) -> Link

我有许多使用 TaskQueue 并发执行的任务，并且在每个任务结束时执行此查询.有时我在更新 last_status 字段时会遇到竞争过多"异常，但我使用重试来处理它.能否打破强一致性?

I have many tasks that are executed concurrently using TaskQueue and this query is performed at the end of every task. Sometimes I'm getting "too much contention" exception when updating the last_status field but I deal with it using retries. Can it break strong consistency?

预期行为是 do_action() 在没有链接时调用 last_status 等于 None.实际行为不一致:有时 do_action() 被调用两次，有时根本不调用.

The expected behavior is having do_action() called when there are no links left with last_status equal to None. The actual behavior is inconsistent: sometimes do_action() is called twice and sometimes not called at all.

推荐答案

使用祖先键来获得强一致性有一个限制:每个实体组每秒只能更新一次.解决此问题的一种方法是对实体组进行分片.分片计数器 描述了该技术.这是一篇旧文章，但据我所知，建议仍然合理.

Using an ancestor key to get strong consistency has a limitation: you're limited to one update per second per entity group. One way to work around this is to shard the entity groups. Sharding Counters describes the technique. It's an old article, but as far as I know, the advise is still sound.

这篇关于Ndb强一致性和频繁写入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！