conduktor.io ↗

KIP-33 — Add a time based log index

Accepted Kafka 1.0 Broker

Adds a time-based index (.timeindex file) per log segment that maps message timestamps to file offsets, enabling accurate offset-by-timestamp lookups (ListOffsets by timestamp) and time-based log rolling/retention at per-message-timestamp granularity. The previous implementation relied on file modification times for time-based operations, which broke on replica reassignment and provided only segment-level (coarse) timestamp resolution.

Details

AuthorJiangjie Qin
StatusAccepted
Kafka Version1.0
JIRAKAFKA-3163
WikiView on Apache Wiki
Created2015-09-10
Last Modified2016-09-01
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.