max.poll.records — Kafka Consumer Configuration
The maximum number of records returned in a single call to poll(). Note, that max.
Description
The maximum number of records returned in a single call to poll(). Note, that max.poll.records does not impact the underlying fetching behavior. The consumer will cache the records from each fetch request and returns them incrementally from each poll.
Default Values by Kafka Version
| Kafka Version | Default Value |
|---|---|
| 0.10.0 | 2147483647 |
| 0.10.1 | 500 |
| 0.10.2 | 500 |
| 0.11.0 | 500 |
| 1.0 | 500 |
| 1.1 | 500 |
| 2.0 | 500 |
| 2.1 | 500 |
| 2.2 | 500 |
| 2.3 | 500 |
| 2.4 | 500 |
| 2.5 | 500 |
| 2.6 | 500 |
| 2.7 | 500 |
| 2.8 | 500 |
| 3.0 | 500 |
| 3.1 | 500 |
| 3.2 | 500 |
| 3.3 | 500 |
| 3.4 | 500 |
| 3.5 | 500 |
| 3.6 | 500 |
| 3.7 | 500 |
| 3.8 | 500 |
| 3.9 | 500 |
| 4.0 | 500 |
| 4.1 | 500 |
| 4.2 | 500 |
Tuning Recommendation
| Profile | Recommended | Why |
|---|---|---|
| consumer / throughput | 2000 | Raising max.poll.records from 500 to 2000 increases the batch size returned per poll() call, amortizing the per-poll overhead (offset tracking, deserializer invocation, application loop overhead) across 4x more records. Directly increases records/sec throughput in tight poll loops. |
| consumer / latency | 100 | Lowering max.poll.records from 500 to 100 reduces the batch size returned per poll(), so the application processes records faster and returns to poll() sooner. This tightens the processing loop and reduces the delay between a record arriving at the broker and being processed by the application. |
| consumer / durability | 100 | Smaller batches reduce the reprocessing blast radius: if processing fails midway through a 500-record batch, all 500 records must be reprocessed. With 100-record batches, at most 100 records are reprocessed on failure. This directly limits the duplicates-per-failure window in at-least-once systems. |
Related Configs
fetch.max.bytes · max.partition.fetch.bytes · max.poll.interval.ms · enable.auto.commit · check.crcs
Manage Kafka configs across all your clusters with Conduktor Console — view, compare, and update configurations in one place.