conduktor.io ↗

KIP-158 — Kafka Connect should allow source connectors to set topic-specific settings for new topics

Accepted Kafka 2.6 Connect

KIP-158 enables Kafka Connect source connectors to declare per-topic configuration (replication factor, partition count, cleanup policy, etc.) so the Connect runtime creates destination topics automatically if they do not exist, controlled by a new `topic.creation.enable` worker config. Previously, Connect relied on broker auto-topic-creation or required manual pre-creation, which was error-prone and impossible when `auto.create.topics.enable=false`.

Details

AuthorRandall Hauch
StatusAccepted
Kafka Version2.6
JIRAKAFKA-5295
WikiView on Apache Wiki
Created2017-05-19
Last Modified2020-06-05
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.