conduktor.io ↗

KIP-381 — Connect: Tell about records that had their offsets flushed in callback

Discussion Connect

Adds an `offsetsFlushed` callback and a `commitRecord(SourceRecord, RecordMetadata)` overload to the Connect `SourceTask` API so connectors know precisely which records had their source offsets durably committed. Without this, a SourceTask has no way to track whether a given polled record's offset was actually flushed, making it impossible to maintain correct state for exactly-once or at-least-once semantics.

Details

AuthorPer Steffensen
StatusDiscussion
JIRAKAFKA-5716
WikiView on Apache Wiki
Created2018-10-10
Last Modified2018-12-12
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.