Kafka Exception StreamsException
org.apache.kafka.streams.errors.StreamsException
Non-retriable
Streams
StreamsException is the top-level exception type generated by Kafka Streams, and indicates errors have occurred during a StreamThread StreamThread's processing. It is guaranteed that any exception thrown up to the StreamsUncaughtExceptionHandler will be of the type StreamsException. For example, any user exceptions will be wrapped as a StreamsException.
Common Causes
- A user lambda/Processor/Transformer threw an unchecked exception (NPE, ClassCastException, arithmetic) during processing; Kafka Streams wraps it as a StreamsException and surfaces it to the StreamsUncaughtExceptionHandler.
- A serialization/deserialization failure (poison-pill record) where the configured DeserializationExceptionHandler is the default FAIL handler, so the SerializationException bubbles up wrapped in a StreamsException.
- A non-retriable infrastructure error — authorization failure (TopicAuthorizationException), missing source topic, incompatible topology change on restart, or repeated TimeoutException exceeding task.timeout.ms.
- An underlying TaskMigratedException / TaskCorruptedException / state-store error that Streams could not internally recover from after exhausting retries, re-thrown as the top-level StreamsException.
Solutions
- Install a StreamsUncaughtExceptionHandler and branch on exception.getCause(): return REPLACE_THREAD for transient causes, SHUTDOWN_CLIENT/SHUTDOWN_APPLICATION for fatal ones (handler must be set while client is in CREATED state).
- Implement a max-failures policy inside your StreamsUncaughtExceptionHandler (replace the thread up to N times within a window, then SHUTDOWN_CLIENT) so transient flaps self-heal but a hard loop doesn't spin forever — Kafka ships the StreamsUncaughtExceptionHandler interface, not a bundled max-failures handler.
- For deserialization poison pills, configure default.deserialization.exception.handler=org.apache.kafka.streams.errors.LogAndContinueExceptionHandler (or a DLQ handler) instead of relying on the uncaught handler — REPLACE_THREAD would re-hit the same record because Streams doesn't commit past it.
- Inspect the wrapped cause and full stack trace before retrying; for production-grade error handling route bad records to a dead-letter topic and alert on the handler firing.
Example Stack Trace
org.apache.kafka.streams.errors.StreamsException: Exception caught in process. taskId=0_2, processor=KSTREAM-SOURCE-0000000000, topic=orders, partition=2, offset=98123, stacktrace=java.lang.NullPointerException
at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:756)
at org.apache.kafka.streams.processor.internals.TaskExecutor.processTask(TaskExecutor.java:95)
at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:777)
at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:617)
at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:579)
Caused by: java.lang.NullPointerException
at com.example.OrderProcessor.process(OrderProcessor.java:33)Diagnostic Commands
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-app # check stuck offset / lag at the failing partition
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic orders --partition 2 --offset 98123 --max-messages 1 # inspect the exact record at the offset in the stack traceRelated
Related Streams exceptions: BrokerNotFoundException · InternalTopicsAlreadySetupException · InvalidStateStoreException · InvalidStateStorePartitionException · LockException · MisconfiguredInternalTopicException · MissingInternalTopicsException · MissingSourceTopicException
Hitting
StreamsException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.