conduktor.io ↗

KIP-1158 — Improve Connect SourceTask commit()

Discussion Connect

Proposes extending SourceTask with a commit(Map offsets) hook that receives the specific offsets being committed, enabling source connectors to know exactly which records are durably committed to Kafka before signaling progress to the external source system. The existing commit() hook fires periodically but without the committed offset values, and commitRecord() fires per-record but without strict ordering guarantees or batch-level completion signals.

Details

AuthorSudesh Wasnik
StatusDiscussion
JIRAKAFKA-19106
WikiView on Apache Wiki
Created2025-04-07
Last Modified2025-04-15
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.