KIP-126 — Allow KafkaProducer to split and resend oversized batches.
Accepted Kafka 1.0 Producer
Adds logic to `KafkaProducer` to detect when an estimated-compressed batch exceeds `max.request.size` or `message.max.bytes` after actual compression and automatically split and retry the oversized batch. The producer's compression ratio estimator can underestimate compression for a specific batch, causing a `RecordTooLargeException` at send time even though individual records were within limits.
Details
| Author | Jiangjie Qin |
| Status | Accepted |
| Kafka Version | 1.0 |
| JIRA | KAFKA-3995 |
| Wiki | View on Apache Wiki |
| Created | 2017-02-21 |
| Last Modified | 2017-05-08 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.