KIP-429 — Kafka Consumer Incremental Rebalance Protocol

Accepted Kafka 2.4 Consumer

Introduces an incremental cooperative rebalance protocol for the Kafka consumer client, where members surrender only the partitions that need to move rather than releasing all partitions on each rebalance. The existing eager stop-the-world rebalance causes all consumers in a group to pause processing while the coordinator reassigns all partitions, creating significant latency spikes for stateful applications like Kafka Streams.

Protocol Impact

JoinGroup

Details