KIP-943 — Add independent “offset.storage.segment.bytes” for connect-distributed.properties
Discussion Connect
Adds a dedicated `offset.storage.segment.bytes` configuration for Kafka Connect's offset storage internal topic, independent of the broker's global `segment.bytes`. At startup, Connect workers consume the entire offset storage topic from the earliest offset; with a large segment size (e.g., default 1 GB × 25 partitions = up to tens of GB of uncompacted active segments), startup time becomes prohibitively long.
Details
| Author | hudeqi |
| Status | Discussion |
| JIRA | KAFKA-15086 |
| Wiki | View on Apache Wiki |
| Created | 2023-06-27 |
| Last Modified | 2025-01-02 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.