KIP-793 — Allow sink connectors to be used with topic-mutating SMTs
Accepted Kafka 3.6 Connect
Adds originalTopic, originalKafkaPartition, and originalKafkaOffset fields to SinkRecord so that sink tasks overriding preCommit() can return the pre-transform offsets that the framework expects. When a topic-mutating SMT (e.g., RegexRouter) changes the topic name, the connector only sees the post-transform topic/partition/offset, causing preCommit() to return wrong offsets and breaking at-least-once offset tracking.
Details
| Author | Diego Erdody |
| Status | Accepted |
| Kafka Version | 3.6 |
| JIRA | KAFKA-13431 |
| Wiki | View on Apache Wiki |
| Created | 2021-11-03 |
| Last Modified | 2023-07-07 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.