Kafka Error CORRUPT_MESSAGE
Error code 2 · Retriable Broker
This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt.
Common Causes
- Log segment or index corruption on disk after storage failure, filesystem corruption, or an unclean shutdown damaged local files
- Producer sent a record with a null key to a compacted topic (cleanup.policy includes compact), which Kafka rejects as an invalid/corrupt record
- A buggy or non-standard client produced a malformed record batch (bad CRC, bad length fields, or invalid compressed batch wrapper)
Solutions
- If broker logs mention a null key on a compacted topic, fix the producer to always set a key for that topic or remove compaction from the topic's cleanup.policy
- If broker logs mention CRC or log corruption, identify the bad replica with kafka-dump-log.sh or broker logs, stop the affected broker, remove only the corrupt replica copy after confirming another healthy replica exists, and let replication rebuild it
- If only one client or library triggers this error, compare its compression.type and client version with a standard Kafka client and upgrade or replace that client before retrying
Diagnostic Commands
# Scan log segment for corruption or errors
kafka-dump-log.sh --files /var/kafka-logs/<topic>-<partition>/00000000000000000000.log --print-data-log 2>&1 | grep -i 'corrupt\|invalid\|error'
# Inspect log directories on the broker
kafka-log-dirs.sh --bootstrap-server localhost:9092 --topic-list <topic> --describeRelated APIs
This error can be returned by: Produce · WriteTxnMarkers
Debugging Kafka errors? Conduktor Console gives you real-time visibility into your cluster. Explore all errors in the Error Decoder.