conduktor.io ↗

KIP-952 — Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

Discussion Tiered Storage

KIP-952 changes the Remote Log Manager to retroactively generate missing producer snapshot files when uploading a log segment to remote storage if the segment predates Kafka 2.8.0 (which introduced segment-aligned snapshot files). Users upgrading from pre-2.8.0 Kafka and enabling Tiered Storage immediately encounter `NullPointerException`s because old segments lack snapshot files that the RSM API contract requires, blocking all remote log uploads.

Details

AuthorChristo Lolov
StatusDiscussion
JIRAKAFKA-15195
WikiView on Apache Wiki
Created2023-07-14
Last Modified2023-07-17
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.