conduktor.io ↗

KIP-669 — Preserve Source Partition in Kafka Streams from context

Discussion Streams

Adds a `preserve.source.partition` configuration to Kafka Streams sink nodes that instructs the built-in `StreamPartitioner` to route output records to the same partition number as the source partition rather than computing a partition from the record key. In pipeline use cases where input and output topics have the same partition count, data locality and ordering guarantees depend on routing to the matching partition, but the key-based partitioner cannot guarantee this when the source partitioner logic is unknown.

Details

Authorsatya k
StatusDiscussion
JIRAKAFKA-10448
WikiView on Apache Wiki
Created2020-09-03
Last Modified2020-09-10
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.