conduktor.io ↗

KIP-943 — Add independent “offset.storage.segment.bytes” for connect-distributed.properties

Discussion Connect

Adds a dedicated `offset.storage.segment.bytes` configuration for Kafka Connect's offset storage internal topic, independent of the broker's global `segment.bytes`. At startup, Connect workers consume the entire offset storage topic from the earliest offset; with a large segment size (e.g., default 1 GB × 25 partitions = up to tens of GB of uncompacted active segments), startup time becomes prohibitively long.

Details

Authorhudeqi
StatusDiscussion
JIRAKAFKA-15086
WikiView on Apache Wiki
Created2023-06-27
Last Modified2025-01-02
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.