本文介绍了kafka最早和最新的偏移值有什么区别的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

producer 发送消息 1、2、3、4

producer sends messages 1, 2, 3, 4

consumer 接收消息 1、2、3、4

consumer receives messages 1, 2, 3, 4

consumer 崩溃/断开连接

producer 发送消息 5, 6, 7

producer sends messages 5, 6, 7

consumer 回来了,应该接收从 5 开始的消息,而不是从 7 开始

consumer comes back up and should receive messages starting from 5 instead of 7

对于这种结果,我必须使用哪个 offset 值以及需要做哪些其他更改/配置

For this kind of result, which offset value I have to use and what are the other changes/configurations need to do

推荐答案

当消费者加入消费者组时,它将获取最后提交的偏移量,因此如果在崩溃之前提交了最新的提交,它将重新从 5、6、7 开始读取偏移量(所以 4).auto.offset.reset 属性的 earliestlatest 值在消费者启动时使用,但分配的分区没有提交的偏移量.在这种情况下,您可以选择是要从头(最早)还是在最后一个(最新)之后重新阅读所有消息.

When a consumer joins a consumer group it will fetch the last committed offset so it will restart to read from 5, 6, 7 if before crashing it committed the latest offset (so 4).The earliest and latest values for the auto.offset.reset property is used when a consumer starts but there is no committed offset for the assigned partition. In this case you can chose if you want to re-read all the messages from the beginning (earliest) or just after the last one (latest).

这篇关于kafka最早和最新的偏移值有什么区别的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

10-19 20:22