conduktor.io ↗

KIP-387 — Fair Message Consumption Across Partitions in KafkaConsumer

Discussion Consumer

Proposes a round-robin partition-interleaved fetch strategy in `KafkaConsumer.poll()` so that `max.poll.records` is distributed evenly across all assigned partitions rather than filled greedily from the last-fetched partition. Under the current implementation, a partition with a large backlog can monopolize the entire `max.poll.records` budget, starving other partitions and breaking fairness guarantees.

Details

AuthorCHIENHSING WU
StatusDiscussion
JIRAKAFKA-3932
WikiView on Apache Wiki
Created2018-11-05
Last Modified2018-11-05
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.