KIP-101 — Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation

Accepted Kafka 1.0 BrokerProtocol

KIP-101 replaces the high watermark (HWM) as the truncation reference point for follower log recovery with a per-epoch `OffsetForLeaderEpoch` RPC, so followers truncate to the first offset of the next leader epoch rather than their locally cached HWM. The HWM-based truncation allowed data loss and log divergence in scenarios where a follower became leader before it had received the HWM update (e.g. during rapid double failovers).

Protocol Impact

Produce · Fetch · LeaderAndIsr · OffsetForLeaderEpoch

Details