KIP-1197 — Introduce new method to improve the TopicBasedRemoteLogMetadataManager's initialization

Discussion Kafka 4.2 Tiered Storage

Replaces the polling/retry initialization loop in TopicBasedRemoteLogMetadataManager with a direct dependency on broker readiness, adding a new initialization hook to avoid retrying before the broker can handle __remote_log_metadata requests. The default 2-minute retry timeout was insufficient for clusters that take more than 2 minutes to start, causing tiered storage initialization failures that left local log segments undeleted and produced excessive WARN logs.

Details