conduktor.io ↗

KIP-878 — Autoscaling for Stateless & Statically Partitioned Streams

Accepted Kafka 3.5 Streams

Enables Kafka Streams to detect when user input topic partition counts have changed and automatically resize internal repartition and changelog topics to match. Streams currently treats a partition count change as a fatal topology mismatch and shuts down, preventing any horizontal scaling of partitioned topologies without a full state migration.

Details

AuthorA. Sophie Blee-Goldman
StatusAccepted
Kafka Version3.5
JIRAKAFKA-14318
WikiView on Apache Wiki
Created2022-10-18
Last Modified2023-02-23
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.