KIP-1178 — Introduce remote.max.partition.fetch.bytes.config in Consumer

Discussion ConsumerTiered Storage

Introduces a new consumer config remote.max.partition.fetch.bytes that controls the maximum bytes per partition returned from tiered remote storage in a Fetch request, propagated to the broker via a new FetchRequest v18 field. The existing max.partition.fetch.bytes applied to both local and remote reads uniformly, but remote storage plugins are often tuned for larger read chunks (e.g., 4MB), and reading only 1MB per remote fetch increased round-trip overhead without benefiting local read performance.

Protocol Impact

Fetch

Details