max.partition.fetch.bytes — Kafka Consumer Configuration
The maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer.
Description
The maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer. If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. The maximum record batch size accepted by the broker is defined via message.max.bytes (broker config) or max.message.bytes (topic config). See fetch.max.bytes for limiting the consumer request size.
Default Values by Kafka Version
| Kafka Version | Default Value |
|---|---|
| 0.9.0 | 1048576 |
| 0.10.0 | 1048576 |
| 0.10.1 | 1048576 |
| 0.10.2 | 1048576 |
| 0.11.0 | 1048576 |
| 1.0 | 1048576 |
| 1.1 | 1048576 |
| 2.0 | 1048576 |
| 2.1 | 1048576 |
| 2.2 | 1048576 |
| 2.3 | 1048576 |
| 2.4 | 1048576 |
| 2.5 | 1048576 |
| 2.6 | 1048576 (1 mebibyte) |
| 2.7 | 1048576 (1 mebibyte) |
| 2.8 | 1048576 (1 mebibyte) |
| 3.0 | 1048576 (1 mebibyte) |
| 3.1 | 1048576 (1 mebibyte) |
| 3.2 | 1048576 (1 mebibyte) |
| 3.3 | 1048576 (1 mebibyte) |
| 3.4 | 1048576 (1 mebibyte) |
| 3.5 | 1048576 (1 mebibyte) |
| 3.6 | 1048576 (1 mebibyte) |
| 3.7 | 1048576 (1 mebibyte) |
| 3.8 | 1048576 (1 mebibyte) |
| 3.9 | 1048576 (1 mebibyte) |
| 4.0 | 1048576 (1 mebibyte) |
| 4.1 | 1048576 (1 mebibyte) |
| 4.2 | 1048576 (1 mebibyte) |
Tuning Recommendation
| Profile | Recommended | Why |
|---|---|---|
| consumer / throughput | 4194304 | Increasing the per-partition fetch limit from 1MB to 4MB reduces the number of FetchRequests needed to drain a partition's backlog by 4x. Especially impactful for consumers with few partitions or when catching up after lag accumulation. |
Related Configs
fetch.min.bytes · fetch.max.bytes · max.poll.records · receive.buffer.bytes
Manage Kafka configs across all your clusters with Conduktor Console — view, compare, and update configurations in one place.