python - 如何给 celery 中的每个节点命名

我们用西芹做烧瓶。我们的各种任务有许多队列，我们使用supervisord来运行队列。我们使用cloudamqp
经纪人。
管理器配置示例如下：

[program:my-queue]
command=/home/ubuntu/opt/proect/venv/bin/celery -A async_runner worker -Q my_queue --loglevel=INFO --without-gossip --without-mingle --autoscale=1,1  -c 1
environment=PYTHONPATH=/home/ubuntu/opt/project/,PRODUCTION_ENVIRONMENT=true
directory=/home/ubuntu/opt/project/app

process_name = %(program_name)s_%(process_num)02d
user=ubuntu
numprocs=2
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
priority=998

我们得到以下错误和队列刚刚停止。

[2017-04-06 12:43:06,759: WARNING/MainProcess] /home/ubuntu/opt/project/venv/local/lib/python2.7/site-packages/kombu/pidbox.py:75: UserWarning: A node named celery@ip-xxx-yy-yy-yy is already using this process mailbox!

Maybe you forgot to shutdown the other node or did not do so properly?
Or if you meant to start multiple nodes on the same host please make sure
you give each node a unique node name!

  warnings.warn(W_PIDBOX_IN_USE.format(node=self))

问题：如何给每个节点命名？
当我运行celery -A async_runner status时，它会给出5个节点的联机消息。

celery@ip-ip_value_here-: OK
celery@ip-ip_value_here-: OK
celery@ip-ip_value_here-: OK
celery@ip-ip_value_here-: OK
celery@ip-ip_value_here-: OK

5 nodes online.

最佳答案

下面是我们如何为公司的每个队列运行一个芹菜工人。入口点包含以下内容：

echo QUEUES: ${QUEUES}  # comma separated list of queue names
export NUM_QUEUES=$(python -c "print len('$QUEUES'.split(','))")
echo NUM_QUEUES: ${NUM_QUEUES}

supervisord &

管理器配置文件包含以下内容：

[program:worker]
command=/path/to/celery_worker.sh %(process_num)s
# supervisor uses the special ENV_ prefix to get the environment variable set above
numprocs=%(ENV_NUM_QUEUES)s

最后celery_worker.sh包含如下内容：

QUEUE=$(python -c "print '$QUEUES'.split(',')[$1]")  # $1 = process_num above
celery worker -n "worker.${QUEUE}" -Q ${QUEUE}  # -n sets the name of the worker

关于python - 如何给 celery 中的每个节点命名，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/43255831/