KIP-500 — Replace ZooKeeper with a Self-Managed Metadata Quorum

Accepted Kafka 2.8 KRaft

Replaces Apache ZooKeeper as Kafka's metadata store with a self-managed Raft-based metadata quorum (KRaft) embedded in the Kafka brokers, where one or more broker nodes act as the metadata log leader and followers. ZooKeeper is an external dependency that limits metadata scalability (partition count), complicates deployment, and creates an operational boundary between Kafka and its metadata store.

Protocol Impact

BrokerRegistration · AddOffsetsToTxn · AddPartitionsToTxn · AlterConfigs · AlterPartitionReassignments · AlterUserScramCredentials · CreateAcls · CreateDelegationToken · CreatePartitions · DeleteAcls · DescribeAcls · DescribeClientQuotas · DescribeConfigs · DescribeDelegationToken · DescribeUserScramCredentials · ElectLeaders · EndTxn · ExpireDelegationToken · InitProducerId · ListPartitionReassignments · Metadata · RenewDelegationToken · TxnOffsetCommit · UpdateFeatures · BrokerHeartbeat · Fetch · LeaderAndIsr

Details