conduktor.io ↗

KIP-320 — Allow fetchers to detect and handle log truncation

Discussion Kafka 2.2 BrokerConsumer

Adds client-side log truncation detection to the consumer and replica fetcher: when a fetch response reveals that the broker's log end offset has moved backward (indicating leader truncation after unclean election), the client detects the divergence using epoch-fenced offset validation and resets to the last safe offset. Without this, consumers silently re-read or skip records after a leader failover with unclean election enabled, causing data corruption or loss that is invisible to the application.

Protocol Impact

Fetch v9

Details

AuthorJason Gustafson
StatusDiscussion
Kafka Version2.2
JIRAKAFKA-6880
WikiView on Apache Wiki
Created2018-06-20
Last Modified2019-08-23
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.