KIP-33 — Add a time based log index
Accepted Kafka 1.0 Broker
Adds a time-based index (.timeindex file) per log segment that maps message timestamps to file offsets, enabling accurate offset-by-timestamp lookups (ListOffsets by timestamp) and time-based log rolling/retention at per-message-timestamp granularity. The previous implementation relied on file modification times for time-based operations, which broke on replica reassignment and provided only segment-level (coarse) timestamp resolution.
Details
| Author | Jiangjie Qin |
| Status | Accepted |
| Kafka Version | 1.0 |
| JIRA | KAFKA-3163 |
| Wiki | View on Apache Wiki |
| Created | 2015-09-10 |
| Last Modified | 2016-09-01 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.