Kafka Exception DataException
org.apache.kafka.connect.errors.DataException
Non-retriable
Connect
Base class for all Kafka Connect data API exceptions.
Common Causes
- Converter mismatch: the configured key/value converter doesn't match the bytes on the topic — e.g. a JSON/Avro/JSON-Schema converter reading data that wasn't written in that format, producing 'Converting byte[] to Kafka Connect data failed due to serialization error'.
- Schema-Registry magic-byte mismatch: plain JSON (or data without the 5-byte registry prefix) is fed to AvroConverter / JsonSchemaConverter, yielding `SerializationException: Unknown magic byte!` wrapped as DataException.
- Schemaless JSON with no envelope: JsonConverter (or a schema-requiring sink) gets a bare JSON object and can't build a schema → 'Cannot infer mapping without schema' / type errors, especially when `schemas.enable` doesn't match the producer.
- Struct/field access errors inside connectors or SMTs: reading a field that doesn't exist in the schema, a type mismatch, or null where a non-optional value is required.
Solutions
- Match converters to the actual on-wire format: Avro → `io.confluent.connect.avro.AvroConverter` + `value.converter.schema.registry.url`; JSON-Schema → `JsonSchemaConverter`; raw bytes/CSV → `ByteArrayConverter`.
- For plain JSON written without a schema envelope, set `value.converter=org.apache.kafka.connect.json.JsonConverter` and `value.converter.schemas.enable=false`.
- 'Unknown magic byte!' means the data wasn't serialized through Schema Registry — either switch to the converter matching how it was actually produced, or re-produce the data with the schema-aware serializer.
- Quarantine poison records instead of crashing the task: `errors.tolerance=all` + a dead-letter queue (`errors.deadletterqueue.topic.name`), then inspect DLQ record headers for the original error.
Example Stack Trace
org.apache.kafka.connect.errors.DataException: Converting byte[] to Kafka Connect data failed due to serialization error of topic connect-topic1:
at org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:333)
at org.apache.kafka.connect.storage.Converter.toConnectData(Converter.java:87)
at org.apache.kafka.connect.runtime.WorkerSinkTask.convertValue(WorkerSinkTask.java:545)
at org.apache.kafka.connect.runtime.WorkerSinkTask.convertAndTransformRecord(WorkerSinkTask.java:501)
at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:478)
Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte!Diagnostic Commands
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic connect-topic1 --from-beginning --max-messages 1 | xxd | head # inspect first bytes (0x00 + schema id = registry-serialized)
curl -s http://localhost:8081/subjects | jq . # check which subjects exist in Schema Registry
curl -s http://localhost:8083/connectors/<name>/config | jq '{key:."key.converter", value:."value.converter"}' # confirm configured convertersRelated
Related Connect exceptions: AlreadyExistsException · ConnectException · IllegalWorkerStateException · NotFoundException · SchemaBuilderException · SchemaProjectorException
Hitting
DataException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.