KIP-1254 — Kafka Consumer Support for Remote Tiered Storage Fetch

Discussion Tiered StorageConsumer

Describes the client-side protocol changes required for Kafka consumers to fetch data directly from remote tiered storage, complementing the broker-side protocol introduced in KIP-1248. When brokers redirect consumers to tiered storage, consumers must resolve the remote segment location, authenticate to the object store, parse Kafka log segment format from raw object storage bytes, and handle transactions correctly—none of which the current consumer client supports.

Protocol Impact

Fetch

Details