Kafka Exception RecordTooLargeException
org.apache.kafka.common.errors.RecordTooLargeException
Non-retriable
Producer
This record is larger than the maximum allowable size
Common Causes
- Producer message or batch exceeds broker's 'message.max.bytes' (default 1MB); common with large Avro/JSON payloads, images, or aggregated records
- Topic-level 'max.message.bytes' is set lower than the broker default and the producer is unaware of the per-topic limit
- Compression not enabled on the producer, causing uncompressed payload to exceed the limit when the original data would fit compressed
Solutions
- Increase limits consistently across all three layers: broker 'message.max.bytes', topic 'max.message.bytes', and consumer 'max.partition.fetch.bytes' — mismatching any one causes failures
- Enable compression on the producer ('compression.type=lz4' or 'snappy') to reduce payload size before the limit check; or split large payloads into smaller records with application-level chunking
- For topic-specific limit: kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name <topic> --alter --add-config max.message.bytes=10485760
Example Stack Trace
org.apache.kafka.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept.Diagnostic Commands
# Check topic message size limits
kafka-configs.sh --bootstrap-server localhost:9092 --entity-type topics --entity-name <topic> --describe | grep message.bytes
# Check broker message size limits
kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-name <broker-id> --describe | grep message.maxRelated
Protocol error: MESSAGE_TOO_LARGE (code 10)
Related Producer exceptions: BufferExhaustedException · InvalidProducerEpochException · OutOfOrderSequenceException · RecordBatchTooLargeException · TransactionAbortedException · UnknownProducerIdException · UnsupportedForMessageFormatException
Hitting
RecordTooLargeException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.