事务性生产者与Java幂等生产者(OutOfOrderSequenceException异常)

本文介绍了事务性生产者与Java幂等生产者(OutOfOrderSequenceException异常)的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我将spring-kafka用于幂等生产者配置:

I use spring-kafka with idempotent producer configuration:

这些是我的配置道具:

   Properties props = new Properties();
    props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, Joiner.on(",").join(appProps.getBrokers()));

    //configure the following three settings for SSL Encryption
    props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SSL");
    props.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, appProps.getJksLocation());
    props.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG,  appProps.getJksPassword());
    props.put(ProducerConfig.ACKS_CONFIG, "all");
    props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG,true);
    props.put(ProducerConfig.RETRIES_CONFIG, 5);
    props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");
    props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringSerializer");

我的卡夫卡生产者抛出OutOfOrderSequenceException:

My kafka producer throws OutOfOrderSequenceException:

我不确定为什么会引发此异常.我找不到对此的具体答案.异常的官方javadoc声明以下内容:

I am not sure why this exception is being thrown. I couldn't find a concrete answer to this. The official javadoc for the exception states the following:

这是否意味着我需要使用事务性生产者来避免此问题?

Does that mean I need to use a transactional producer to avoid this issue?

KafkaProducer文档陈述了使上述说法含糊的内容: https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html

KafkaProducer doc states something that makes the above statement ambiguous: https://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html

要利用幂等生成器，必须避免重新发送应用程序级别，因为这些应用程序无法重复删除.因此，如果应用程序启用幂等性，建议不要设置重试配置，因为它将默认为Integer.MAX_VALUE.此外，如果send(ProducerRecord)即使无限次重试也返回错误(例如，如果消息在发送之前已在缓冲区中过期)，则建议关闭生产者并检查最后产生的消息的内容，以确保不能重复.最后，生产者只能保证在单个会话中发送的消息具有幂等性.

To take advantage of the idempotent producer, it is imperative to avoid application level re-sends since these cannot be de-duplicated. As such, if an application enables idempotence, it is recommended to leave the retries config unset, as it will be defaulted to Integer.MAX_VALUE. Additionally, if a send(ProducerRecord) returns an error even with infinite retries (for instance if the message expires in the buffer before being sent), then it is recommended to shut down the producer and check the contents of the last produced message to ensure that it is not duplicated. Finally, the producer can only guarantee idempotence for messages sent within a single session.

上面的声明清楚地表明，我需要一个幂等生成器，只是使用enable.idempotence属性.但是，该异常指出我必须使用该transactional.id属性.

The above statement clearly states, all I need for an idempotent producer is to just use enable.idempotence property. However, the exception states that I have to use that transactional.id property.

创建幂等异步生成器而不必处理致命的OutOfOrderSequenceException的正确方法是什么.

What is the right way to create an idempotent async producer without having to deal with the fatal OutOfOrderSequenceException.

事务性生产者

事务性生产者与Java幂等生产者(OutOfOrderSequenceException异常)

问题描述

推荐答案