conduktor.io ↗

KIP-275 — Indicate "isClosing" in the SinkTaskContext

Discarded Connect

KIP-275 proposed adding an `isClosing()` flag to `SinkTaskContext` to signal to a sink task that the worker is shutting down, enabling the task to flush and commit a partial batch before termination. Without this signal, tasks that accumulate large in-memory batches (e.g., S3 connector) lose all buffered records whenever a task restart or rebalance occurs mid-batch.

Details

AuthorMatt Farmer
StatusDiscarded
JIRAKAFKA-6725
WikiView on Apache Wiki
Created2018-03-29
Last Modified2020-06-14
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.