KIP-881 — Rack-aware Partition Assignment for Kafka Consumers

Accepted Kafka 3.4 Consumer

Extends the Kafka consumer partition assignment algorithms to prefer assigning partitions to consumers whose rack matches the replica rack, minimizing cross-AZ fetches when the replication factor is lower than the number of AZs. The existing rack-aware fetch (KIP-392) only helped when every AZ had a local replica; with fewer replicas than AZs some consumers were always assigned partitions with no local replica.

Protocol Impact

ConsumerProtocolAssignment · ConsumerProtocolSubscription

Details