conduktor.io ↗

KIP-695 — Further Improve Kafka Streams Timestamp Synchronization

Accepted Kafka 3.0 Streams

Improves Kafka Streams timestamp synchronization by using the last-seen record timestamp as a lower bound for a partition's unknown future timestamp when deciding which task input to process next. The existing `max.task.idle.ms` idle wait is symmetric—it waits the same duration regardless of how far ahead other partitions are—leading to excessive waiting or out-of-order processing when partitions have very different timestamps.

Details

AuthorJohn Roesler
StatusAccepted
Kafka Version3.0
JIRAKAFKA-10091
WikiView on Apache Wiki
Created2020-11-18
Last Modified2021-02-22
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.