conduktor.io ↗

KIP-983 — Full speed async processing during rebalance

Discussion Consumer

Proposes a new Java consumer API mode where partitions not being revoked during a cooperative rebalance continue processing at full speed while the application asynchronously drains revoked partitions before committing. The current asynchronous processing pattern blocks the entire poll loop during rebalance to prevent offset commit races, stalling throughput on all partitions even those unaffected by the rebalance.

Details

AuthorErik van Oosten
StatusDiscussion
WikiView on Apache Wiki
Created2023-09-23
Last Modified2023-10-14
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.