conduktor.io ↗

KIP-129 — Streams Exactly-Once Semantics

Accepted Kafka 1.0 StreamsTransactions

Adds exactly-once processing semantics to Kafka Streams by building on KIP-98 transactions and idempotent producer: each stream task wraps its consume-process-produce cycle in a transaction so output records and consumer offsets are committed atomically. Without this, a Streams task crash could re-process input after recovery and produce duplicate output records, violating at-least-once at the application level.

Details

AuthorGuozhang Wang
StatusAccepted
Kafka Version1.0
JIRAKAFKA-4923
WikiView on Apache Wiki
Created2017-02-28
Last Modified2017-03-30
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.