Kafka Streams 2.5.0 需要输入主题

问题描述

从 Kafka Streams 2.5.0 开始，拓扑似乎必须包含输入主题.在 Kafka 2.4.1(及更早版本)中，情况并非如此.

Starting with Kafka Streams 2.5.0 it seems like a topology must include an input topic. In Kafka 2.4.1 (and earlier) that is not the case.

我有一个应用程序，其中拓扑只是创建一些全局状态存储，这些存储从其他应用程序写入的主题中读取数据.

I have an application where the topology is just creating a few global state stores that read in data from topics written to by other applications.

使用 Kafka 2.5.0 我得到这个错误:

With Kafka 2.5.0 I get this error:

13:24:27.161 [<redacted>-7cf1b5c9-4a6e-4bf2-9f77-f7f85f2df3bb-StreamThread-1] ERROR o.a.k.s.p.internals.StreamThread - stream-thread [<redacted>-7cf1b5c9-4a6e-4bf2-9f77-f7f85f2df3bb-StreamThread-1] Encountered the following error during processing:
java.lang.IllegalStateException: Consumer is not subscribed to any topics or assigned any partitions
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1228)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216)
    at org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:853)
    at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:753)
    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:697)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:670)

如果我添加一个虚拟输入主题(例如通过 streamsBuilder.stream(Pattern.compile("hack"));)，应用程序启动正常.

If I add a dummy input topic (e.g. via streamsBuilder.stream(Pattern.compile("hack"));) the application starts fine.

这种行为是意料之中的还是 Kafka Streams 2.5.0 中的无意更改?

Is this behavior to be expected or is it an unintentional change in Kafka Streams 2.5.0?

更多细节:上面的用例可能看起来有点奇怪，我不得不同意.首先这样做的原因是交互式查询的一个缺点，即应用程序在一段时间内无法回答查询.我看到该问题已在 Kafka Streans 2.5.0 中通过 KIP-535 很棒.希望以后再研究智商.

More details: The use case above may seem a bit weird and I would have to agree. The reason for doing it in the first place was a shortcoming of Interactive Queries where for periods of time the application could not answer queries. I see that issue has been fixed in Kafka Streans 2.5.0 via KIP-535 which is great. I hope to look into IQ again later.

推荐答案

当我们切换(返回)使用 收藏订阅.刚刚合并了一个 fix，所以你应该升级到 2.5.1 或 2.6被释放.

There was a regression introduced in 2.5.0 when we switched (back) to using collection subscription. A fix was just merged, so you should upgrade to 2.5.1 or 2.6 when they are released.

这篇关于Kafka Streams 2.5.0 需要输入主题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！