KIP-1101 — Trigger rebalance on rack topology changes

Accepted Kafka 4.0 Consumer

Replaces per-partition rack topology data (the full set of replica racks) in the ConsumerGroup subscribed topic metadata with a compact hash, and triggers rebalances when the hash changes due to partition additions or rack changes. Storing the full rack set per partition for groups with thousands of members and thousands of partitions consumed disproportionate memory (up to 79% of ConsumerGroup object size in production cases) after the new consumer group protocol (KIP-848) was introduced.

Protocol Impact

Heartbeat · ConsumerGroupHeartbeat

Details