KIP-868 — Metadata Transactions
Accepted Kafka 3.6 KRaftProtocol
Introduces metadata transactions in KRaft so the controller can atomically commit a set of metadata records larger than the Raft fetch size limit (currently hard-coded at 8 KB) by spanning multiple batches with transaction start/end markers. Creating a topic with thousands of partitions generates a `TopicRecord` + N `PartitionRecord`s that can exceed 8 KB, but partial commits during a controller failover would leave the metadata store in an inconsistent state.
Details
| Author | David Arthur |
| Status | Accepted |
| Kafka Version | 3.6 |
| JIRA | KAFKA-14305 |
| Wiki | View on Apache Wiki |
| Created | 2022-09-07 |
| Last Modified | 2022-10-14 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.