KIP-509 — Rebalance and restart Producers

Discussion Producer

KIP-509 proposes a producer-side load-balancing mechanism analogous to consumer group rebalancing, where producers coordinate which source partitions each instance is responsible for without requiring Kafka Connect. In Kubernetes-style environments where producers autoscale horizontally, there is no built-in coordination primitive to shard source partitions across producer replicas.

Details