KIP-381 — Connect: Tell about records that had their offsets flushed in callback
Discussion Connect
Adds an `offsetsFlushed` callback and a `commitRecord(SourceRecord, RecordMetadata)` overload to the Connect `SourceTask` API so connectors know precisely which records had their source offsets durably committed. Without this, a SourceTask has no way to track whether a given polled record's offset was actually flushed, making it impossible to maintain correct state for exactly-once or at-least-once semantics.
Details
| Author | Per Steffensen |
| Status | Discussion |
| JIRA | KAFKA-5716 |
| Wiki | View on Apache Wiki |
| Created | 2018-10-10 |
| Last Modified | 2018-12-12 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.