KIP-287 — Support partition and processor expansion for stateful processing jobs

Discussion Streams

Proposes a safe partition and processor expansion protocol for stateful Kafka Streams jobs by coordinating a two-phase migration that ensures all in-flight keyed messages drain before the partition/processor count changes. After a partition count increase, existing stateful stream tasks may receive records for keys whose state lives on a different task instance, producing incorrect aggregation results.

Details