KIP-1075 — Introduce delayed remote list offsets purgatory to make LIST_OFFSETS async
Accepted Kafka 4.0 Tiered StorageBroker
KIP-1075 introduces a delayed purgatory (`DelayedRemoteListOffsets`) and a new `TimeoutMs` field in `ListOffsetsRequest` so timestamp-based offset lookups that require fetching remote indexes are handled asynchronously by dedicated remote-log-reader threads rather than blocking request-handler threads. `ListOffsets` requests against remote-storage-enabled topics require non-deterministic latency to fetch remote offset and time indexes; when many such requests arrive concurrently, request-handler thread exhaustion starves higher-priority `FETCH` and `PRODUCE` requests.
Protocol Impact
ListOffsets v10
Details
| Author | Kamal Chandraprakash |
| Status | Accepted |
| Kafka Version | 4.0 |
| JIRA | KAFKA-15859 |
| Wiki | View on Apache Wiki |
| Created | 2024-08-02 |
| Last Modified | 2024-08-29 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.