conduktor.io ↗

KIP-64 — Allow underlying distributed filesystem to take over replication depending on configuration

Discussion Broker

KIP-64 proposes a broker config to delegate replication entirely to the underlying distributed filesystem (e.g. NFS/SMB-mounted HDFS or GlusterFS), treating a successful `flush()` as a signal that all ISR replicas have the data and disabling Kafka's own replication traffic. This eliminates redundant data copies and inter-broker replication overhead when the storage layer already provides its own fault-tolerant replication.

Details

AuthorArvind Kandhare
StatusDiscussion
JIRAKAFKA-3867
WikiView on Apache Wiki
Created2016-06-18
Last Modified2016-06-19
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.