KIP-1346 — Parallelize loadSegmentFiles

Discussion BrokerMetrics

Parallelizes LogLoader.loadSegmentFiles() with a per-data-directory executor sized by the new num.segment.loading.threads.per.data.dir broker config (default 1, preserving sequential loading), and adds a kafka.log:type=LogManager,name=RemainingSegmentsToLoad gauge tagged by dir and threadNum for startup progress. LogManager already loads partitions concurrently via num.recovery.threads.per.data.dir, but segments within a partition are opened and sanity-checked one at a time, so broker startup scales linearly with the largest partition's segment count no matter how many cores or recovery threads are available.

Details