问题描述
目前我正在从事需要实现一些后台作业(主要用于电子邮件发送和大量数据库更新)的 python 项目.我使用 Redis 作为任务代理.所以在这一点上我有两个候选人:Celery 和 RQ.我对这些作业队列有一些经验，但我想请你们分享使用这些工具的经验.所以.
Currently I'm working on python project that requires implement some background jobs (mostly for email sending and heavily database updates). I use Redis for task broker. So in this point I have two candidates: Celery and RQ. I had some experience with these job queues, but I want to ask you guys to share you experience of using this tools. So.
- 使用 Celery 与 RQ 的优缺点是什么.
- 适合使用 Celery 与 RQ 的任何项目/任务示例.
Celery 看起来很复杂，但它是功能齐全的解决方案.实际上，我认为我不需要所有这些功能.从另一方面看，RQ 非常简单(例如配置、集成)，但它似乎缺少一些有用的功能(例如任务撤销、代码自动重新加载)
Celery looks pretty complicated but it's full featured solution. Actually I don't think that I need all these features. From other side RQ is very simple (e.g configuration, integration), but it seems that it lacks some useful features (e.g task revoking, code auto-reloading)
推荐答案
这是我在尝试回答这个完全相同的问题时发现的.它可能并不全面，甚至在某些方面可能不准确.
Here is what I have found while trying to answer this exact same question. It's probably not comprehensive, and may even be inaccurate on some points.
简而言之，RQ 被设计得更简单.Celery 被设计得更健壮.他们都很优秀.
In short, RQ is designed to be simpler all around. Celery is designed to be more robust. They are both excellent.
- 文档.RQ 的文档 全面而不复杂，反映了项目的整体简单性 - 您永远不会感到迷茫或困惑.Celery 的文档 也很全面，但希望重新- 当你第一次设置的时候经常访问它，因为有太多的选项需要内化
- Documentation. RQ's documentation is comprehensive without being complex, and mirrors the project's overall simplicity - you never feel lost or confused. Celery's documentation is also comprehensive, but expect to be re-visiting it quite a lot when you're first setting things up as there are too many options to internalize
Monitoring. Celery's Flower and the RQ dashboard are both very simple to setup and give you at least 90% of all information you would ever want
经纪人支持.Celery 是明显的赢家，RQ 只支持 Redis.这意味着关于什么是代理"的文档较少，但也意味着如果 Redis 不再适合您，您将来无法切换代理.例如，Instagram 考虑了 Redis和 RabbitMQ 与芹菜.这很重要，因为不同的经纪人有不同的保证，例如Redis 不能(在撰写本文时)保证您的消息 100% 已送达.
Broker support. Celery is the clear winner, RQ only supports Redis. This means less documentation on "what is a broker", but also means you cannot switch brokers in the future if Redis no longer works for you. For example, Instagram considered both Redis and RabbitMQ with Celery. This is important because different brokers have different guarantees e.g. Redis cannot (as of writing) guarantee 100% that your messages are delivered.
优先队列.RQ 的优先级队列模型简单有效 - worker 按顺序从队列中读取.Celery 需要启动多个 worker 以从不同的队列中消费.两种方法都有效
Priority queues. RQs priority queue model is simple and effective - workers read from queues in order. Celery requires spinning up multiple workers to consume from different queues. Both approaches work
操作系统支持.Celery 显然是赢家，因为 RQ 只在支持 fork 的系统上运行，例如Unix 系统
OS Support. Celery is the clear winner here, as RQ only runs on systems that support fork e.g. Unix systems
语言支持.RQ 仅支持 Python，而 Celery 允许您将任务从一种语言发送到另一种语言
Language support. RQ only supports Python, whereas Celery lets you send tasks from one language to a different language
API.Celery 非常灵活(多个结果后端、漂亮的配置格式、工作流画布支持)，但这种功能自然会令人困惑.相比之下，RQ api 很简单.
API. Celery is extremely flexible (multiple result backends, nice config format, workflow canvas support) but naturally this power can be confusing. By contrast, the RQ api is simple.
子任务支持.Celery 支持子任务(例如，从现有任务中创建新任务).我不知道 RQ 有没有
Subtask support. Celery supports subtasks (e.g. creating new tasks from within existing tasks). I don't know if RQ does
社区和稳定.Celery 可能更成熟，但它们都是活跃的项目.截至撰写本文时，Celery 在 Github 上拥有约 3500 颗星，而 RQ 拥有约 2000 颗星，并且两个项目都显示出积极的发展
Community and Stability. Celery is probably more established, but they are both active projects. As of writing, Celery has ~3500 stars on Github while RQ has ~2000 and both projects show active development
在我看来，Celery 并不像它的声誉让您相信的那么复杂，但您必须使用 RTFM.
In my opinion, Celery is not as complex as its reputation might lead you to believe, but you will have to RTFM.
那么，为什么有人愿意用(可以说是功能更齐全的)芹菜换取 RQ?在我看来，这一切都归结为简单.通过将自己限制在 Redis+Unix，RQ 提供了更简单的文档、更简单的代码库和更简单的 API.这意味着您(和您的项目的潜在贡献者)可以专注于您关心的代码，而不必在您的工作记忆中保留有关任务队列系统的详细信息.我们都对一次可以在我们脑海中保留多少细节有限制，并且通过消除将任务队列细节保留在那里 RQ 的需要，让我们回到您关心的代码.这种简单性是以牺牲语言间任务队列、广泛的操作系统支持、100% 可靠的消息保证以及轻松切换消息代理的能力为代价的.
So, why would anyone be willing to trade the (arguably more full-featured) Celery for RQ? In my mind, it all comes down to the simplicity. By restricting itself to Redis+Unix, RQ provides simpler documentation, simpler codebase, and a simpler API. This means you (and potential contributors to your project) can focus on the code you care about, instead of having to keep details about the task queue system in your working memory. We all have a limit on how many details can be in our head at once, and by removing the need to keep task queue details in there RQ lets get back to the code you care about. That simplicity comes at the expense of features like inter-language task queues, wide OS support, 100% reliable message guarantees, and ability to switch message brokers easily.
这篇关于使用 Celery 与 RQ 的利弊的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！