KIP-1018 — Introduce max remote fetch timeout config for DelayedRemoteFetch requests
Accepted Kafka 3.8 Tiered StorageBroker
Adds a dedicated remote.fetch.max.wait.ms broker configuration for the DelayedRemoteFetchPurgatory timeout used when fetching from tiered (remote) storage, independent of the consumer's fetch.max.wait.ms. The shared fetch.max.wait.ms default (500ms) was too short for remote storage fetches during degraded conditions, causing excessive purgatory expiration and retry storms against the remote storage layer.
Details
| Author | Kamal Chandraprakash |
| Status | Accepted |
| Kafka Version | 3.8 |
| JIRA | KAFKA-15776 |
| Wiki | View on Apache Wiki |
| Created | 2024-01-30 |
| Last Modified | 2024-06-05 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.