KIP-644 — Support sources which can't give out changed data in kafka connect
Discussion Connect
KIP-644 proposes extending the `SourceTask` contract to allow the task to return both output records and a state snapshot per `poll()` call, with Connect persisting the state to an internal topic and providing `SourceTaskContext` access to the previous state snapshot on restart. This enables sources that can only provide full snapshots (not change feeds) to be supported in Connect by having the framework compute the delta between the current and previous snapshots.
Details
| Author | Nikhil Kumar |
| Status | Discussion |
| Wiki | View on Apache Wiki |
| Created | 2020-07-16 |
| Last Modified | 2020-07-16 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.