conduktor.io ↗

KIP-253 — Support in-order message delivery with partition expansion

Discussion ProducerBroker

Proposes a mechanism to guarantee key-ordered delivery even when a topic's partition count changes, by maintaining a stable mapping from key to old partition during the transition window. Expanding partitions today causes records with the same key to be routed to a different partition (since `hash(key) % numPartitions` changes), breaking the ordering guarantee relied upon by stateful consumers.

Protocol Impact

Produce · Fetch · Metadata · LeaderAndIsr · StopReplica · UpdateMetadata · SyncGroup

Details

AuthorDong Lin
StatusDiscussion
WikiView on Apache Wiki
Created2018-02-09
Last Modified2018-07-21
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.