KIP-1333 — Support log segments larger than 2 GB

Discussion BrokerTiered Storage

Widens log.segment.bytes and the topic-level segment.bytes from INT to LONG and introduces a 12-byte offset index entry format so physical file positions can exceed 4-byte signed range, both gated behind MetadataVersion IBP_4_4_IV1. The 2 GB segment ceiling forces roughly 20,000 open files for a 10 TB partition, a segment roll every few seconds at 500 MB/s ingest, extra compaction cycles over many small segment groups, and one tiered-storage copy or delete unit per segment.

Details