KIP-762 — Delete Committed Connect Records

Discussion Connect

Extends Kafka Connect to automatically delete records from source topics once all SinkConnectors that subscribe to those topics have processed them, enabling low-latency storage-efficient pipelines. Without this, Connect pipelines that use Kafka purely as a transport medium (not for long-term retention) must either over-provision retention or risk SinkConnectors falling behind a short retention window.

Details