问题描述
我在使用 KafaConsumer
时遇到麻烦,无法使其从开头或从其他任何显式偏移量读取.
I am having trouble with KafaConsumer
to make it read from the beginning, or from any other explicit offset.
针对相同的主题为使用者运行命令行工具,我确实看到带有-from-beginning
选项的消息,否则挂起
Running the command line tools for the consumer for the same topic , I do see messages with the --from-beginning
option and it hangs otherwise
$ ./kafka-console-consumer.sh --zookeeper {localhost:port} --topic {topic_name} --from-beginning
如果我通过python运行它,它会挂起,我怀疑是由错误的使用者配置引起的
If I run it through python, it hangs, which I suspect to be caused by incorrect consumer configs
consumer = KafkaConsumer(topic_name,
bootstrap_servers=['localhost:9092'],
group_id=None,
auto_commit_enable=False,
auto_offset_reset='smallest')
print "Consuming messages from the given topic"
for message in consumer:
print "Message", message
if message is not None:
print message.offset, message.value
print "Quit"
输出:
消费来自给定主题的消息(之后挂起)
Output:
Consuming messages from the given topic(hangs after that)
我正在使用kafka-python 0.9.5,并且代理运行kafka 8.2.不知道确切的问题是什么.
I am using kafka-python 0.9.5 and the broker runs kafka 8.2. Not sure what the exact problem is.
按照dpkp的建议设置_group_id = None_来模拟控制台使用者的行为.
Set _group_id=None_ as suggested by dpkp to emulate the behavior of console consumer.
推荐答案
控制台消费者和您发布的python消费者代码之间的区别是python消费者使用消费者组来保存偏移量: group_id =测试消费者组"
.相反,如果您设置group_id = None,则应该看到与控制台使用者相同的行为.
The difference between the console-consumer and the python consumer code you have posted is the python consumer uses a consumer group to save offsets: group_id="test-consumer-group"
. If instead you set group_id=None, you should see the same behavior as the console consumer.
这篇关于kafka-python使用者未收到消息的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!