KIP-213 — Support non-key joining in KTable
Accepted Kafka 2.4 Streams
Adds foreign-key KTable-to-KTable joins to the Kafka Streams DSL through `join`/`leftJoin` overloads taking a `foreignKeyExtractor`, implemented with subscription records over repartition topics, a `CombinedKey` (primary key plus foreign key) prefix-scan store on the right-hand table, and an original-value hash carried round-trip so stale results are discarded when the extracted foreign key changes. KTables could previously only be joined on their primary key, so the one-to-many relationships that arrive from CDC pipelines (JDBC connect, Debezium, Maxwell) had to be resolved with a poorly supported group-by/join/lateral-view workaround at odds with record-based processing.
Details
| Author | Jan Filipiak |
| Status | Accepted |
| Kafka Version | 2.4 |
| JIRA | KAFKA-3705 |
| Wiki | View on Apache Wiki |
| Created | 2017-10-25 |
| Last Modified | 2021-05-21 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.