conduktor.io ↗

KIP-68 — Add a consumed log retention before log retention

Discussion BrokerConsumer

Proposes a `log.consumed.retention.ms` config that delays log segment deletion until the segment's messages have been consumed by at least one consumer group, preventing data loss when retention expires faster than consumers can process. Current time-based and size-based retention deletes segments regardless of consumer progress, risking message loss for slow consumers without a way to track consumption-based retention.

Details

Authorpengwei
StatusDiscussion
WikiView on Apache Wiki
Created2016-06-30
Last Modified2016-11-13
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.