conduktor.io ↗

KIP-360 — Improve reliability of idempotent/transactional producer

Accepted Kafka 2.5 ProducerTransactions

Refines the producer's handling of `UNKNOWN_PRODUCER_ID` errors by adding a `producerEpoch` bump mechanism so the producer can recover from log compaction or `DeleteRecords` without treating the error as fatal. The previous recovery (resetting sequence to 0 while keeping the epoch) was unsafe on retried requests—if the first attempt had succeeded, resetting the sequence caused duplicate detection to fail and could yield duplicates or data loss.

Protocol Impact

InitProducerId

Details

AuthorJason Gustafson
StatusAccepted
Kafka Version2.5
JIRAKAFKA-8710
WikiView on Apache Wiki
Created2018-08-20
Last Modified2020-04-17
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.