conduktor.io ↗

KIP-89 — Allow sink connectors to decouple flush and offset commit

Accepted Kafka 1.0 Connect

KIP-89 adds a `SinkTask.preCommit()` method and a `SinkTaskContext.requestCommit()` method to decouple offset commits from periodic buffer flushes in Kafka Connect sink connectors. Previously, the runtime called `flush()` on the fixed `offset.flush.interval.ms` schedule and committed offsets immediately after, forcing connectors that manage their own buffering (size-based, time-based, or not at all) to flush unnecessarily.

Details

AuthorShikhar Bhushan
StatusAccepted
Kafka Version1.0
JIRAKAFKA-4161
WikiView on Apache Wiki
Created2016-11-04
Last Modified2017-01-02
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.