conduktor.io ↗

KIP-534 — Reorganize checkpoint system in log cleaner to per partition

Accepted Kafka 3.1 Broker

Fixes log compaction to ensure that tombstone records and transaction abort markers are retained for at least `delete.retention.ms` milliseconds from the point the compacting segment was written, not just from the segment roll time. Under the prior implementation, tombstones could be physically removed before `delete.retention.ms` had elapsed from the last possible read of the preceding non-tombstone record, violating the semantic guarantee that consumers reading within the retention window will observe deletions.

Details

AuthorRichard Yu
StatusAccepted
Kafka Version3.1
JIRAKAFKA-8522
WikiView on Apache Wiki
Created2019-09-02
Last Modified2019-10-18
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.