conduktor.io ↗

KIP-759 — Unneeded repartition canceling

Accepted Streams

Adds a `repartitionRequired` flag to Kafka Streams' internal graph optimization so that a `selectKey`/`map` followed by a `groupByKey` does not insert an unnecessary repartition topic when the upstream stream is already partitioned by the new key. Every key-changing operation currently forces a repartition before any stateful aggregation, even when the resulting key is identical to the existing partition key, wasting network bandwidth and adding end-to-end latency.

Details

AuthorIvan Ponomarev
StatusAccepted
JIRAKAFKA-4835
WikiView on Apache Wiki
Created2021-06-24
Last Modified2023-08-03
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.