buffer.memory — Kafka Producer Configuration
The total bytes of memory the producer can use to buffer records waiting to be sent to the server. If records are sent faster than they can be delivered to the server the producer will block for max.
Description
The total bytes of memory the producer can use to buffer records waiting to be sent to the server. If records are sent faster than they can be delivered to the server the producer will block for max.block.ms after which it will throw an exception.
Default Values by Kafka Version
| Kafka Version | Default Value |
|---|---|
| 0.8.1 | 33554432 |
| 0.8.2 | 33554432 |
| 0.9.0 | 33554432 |
| 0.10.0 | 33554432 |
| 0.10.1 | 33554432 |
| 0.10.2 | 33554432 |
| 0.11.0 | 33554432 |
| 1.0 | 33554432 |
| 1.1 | 33554432 |
| 2.0 | 33554432 |
| 2.1 | 33554432 |
| 2.2 | 33554432 |
| 2.3 | 33554432 |
| 2.4 | 33554432 |
| 2.5 | 33554432 |
| 2.6 | 33554432 |
| 2.7 | 33554432 |
| 2.8 | 33554432 |
| 3.0 | 33554432 |
| 3.1 | 33554432 |
| 3.2 | 33554432 |
| 3.3 | 33554432 |
| 3.4 | 33554432 |
| 3.5 | 33554432 |
| 3.6 | 33554432 |
| 3.7 | 33554432 |
| 3.8 | 33554432 |
| 3.9 | 33554432 |
| 4.0 | 33554432 |
| 4.1 | 33554432 |
| 4.2 | 33554432 |
Tuning Recommendation
| Profile | Recommended | Why |
|---|---|---|
| producer / throughput | 134217728 | Expanding the accumulator buffer from 32MB to 128MB prevents send() from blocking during traffic bursts, sustaining throughput without back-pressure stalls when brokers are temporarily slower. |
| producer / durability | 67108864 | Doubling the accumulator buffer to 64MB provides headroom for records queued during a retry window (broker unavailability lasting 10-30s at typical rates); without this, buffer.memory exhausts and max.block.ms triggers, causing send() to throw before the retry window expires. |
Related Configs
batch.size · linger.ms · max.block.ms · delivery.timeout.ms
Manage Kafka configs across all your clusters with Conduktor Console — view, compare, and update configurations in one place.