KIP-405 — Kafka Tiered Storage

Accepted Kafka 3.5 Tiered Storage

Introduces tiered storage for Kafka, offloading older log segments to remote object storage (S3, GCS, etc.) while keeping only recent segments on local broker disks, governed by a new RemoteLogManager and RemoteStorageManager plugin interface. Local disk capacity proportional to retention × throughput limited Kafka's ability to serve as a long-term data store, requiring expensive broker hardware just to maintain retention.

Protocol Impact

Fetch v14 · ListOffsets v8

Details