What causes LockException?

Two Kafka Streams instances (or two StreamThreads) point at the SAME state.dir on disk: the second one cannot acquire the per-task .lock file under / / /.lock that the first still holds. Classic in Kubernetes when one PersistentVolume is mounted across multiple pods instead of one PVC per pod. Task migration during a rebalance: a task moves from one thread/instance to another while the previous owner has not yet closed its StateManager and released the directory lock, so the new owner logs 'Will retry' then throws LockException for that task (e.g. task [0_18] Failed to lock the state directory). Default /tmp/kafka-streams state dir gets auto-deleted (OS tmp cleanup or app cleanUp()) underneath a running/restarting instance during a rolling bounce, so the thread re-creates the directory while a lock from the shutting-down process is still in flight, producing RocksDB 'IO error: ... LOCK: No locks available'. Standby-task / restoring-task directory livelocking (KAFKA-12679): the assignor hands a task to a new thread before the old thread releases the lock; without proper backoff the lock-retry loop can spin and starve CPU, with a steady stream of 'Failed to lock the state directory for task' debug lines.

How do I fix LockException?

Give every instance/replica its OWN state.dir. On Kubernetes, deploy Streams as a StatefulSet with volumeClaimTemplates (one PVC per pod) plus pod anti-affinity — never share a single PV. Set state.dir to a stable, persistent path (e.g. /var/lib/kafka-streams), NOT the default /tmp/kafka-streams which the OS can wipe. Make custom Processor/Transformer close() idempotent so it doesn't throw on the second close during rebalance, which would leave the StateManager (and its lock) un-closed. Treat 'Will retry' LockException warnings during rebalance as transient — Streams retries and the lock frees once the prior owner finishes. Only persistent loops indicate a config problem (shared dir) or a known bug. Upgrade Kafka Streams: the core locking bugs (KAFKA-5070/5167/5562, fixed by 0.10.2; livelock backoff KAFKA-12679) are resolved in modern releases. On old releases a workaround was per-instance distinct state.dir.

Is LockException retriable?

No — LockException is not retriable. Retrying the same call will not help; the underlying cause must be fixed first.

Kafka Exception `LockException`

org.apache.kafka.streams.errors.LockException

Non-retriable Streams

Indicates that the state store directory lock could not be acquired because another thread holds the lock.

Common Causes

Two Kafka Streams instances (or two StreamThreads) point at the SAME state.dir on disk: the second one cannot acquire the per-task .lock file under <state.dir>/<application.id>/<taskId>/.lock that the first still holds. Classic in Kubernetes when one PersistentVolume is mounted across multiple pods instead of one PVC per pod.
Task migration during a rebalance: a task moves from one thread/instance to another while the previous owner has not yet closed its StateManager and released the directory lock, so the new owner logs 'Will retry' then throws LockException for that task (e.g. task [0_18] Failed to lock the state directory).
Default /tmp/kafka-streams state dir gets auto-deleted (OS tmp cleanup or app cleanUp()) underneath a running/restarting instance during a rolling bounce, so the thread re-creates the directory while a lock from the shutting-down process is still in flight, producing RocksDB 'IO error: ... LOCK: No locks available'.
Standby-task / restoring-task directory livelocking (KAFKA-12679): the assignor hands a task to a new thread before the old thread releases the lock; without proper backoff the lock-retry loop can spin and starve CPU, with a steady stream of 'Failed to lock the state directory for task' debug lines.

Solutions

Give every instance/replica its OWN state.dir. On Kubernetes, deploy Streams as a StatefulSet with volumeClaimTemplates (one PVC per pod) plus pod anti-affinity — never share a single PV. Set state.dir to a stable, persistent path (e.g. /var/lib/kafka-streams), NOT the default /tmp/kafka-streams which the OS can wipe.
Make custom Processor/Transformer close() idempotent so it doesn't throw on the second close during rebalance, which would leave the StateManager (and its lock) un-closed.
Treat 'Will retry' LockException warnings during rebalance as transient — Streams retries and the lock frees once the prior owner finishes. Only persistent loops indicate a config problem (shared dir) or a known bug.
Upgrade Kafka Streams: the core locking bugs (KAFKA-5070/5167/5562, fixed by 0.10.2; livelock backoff KAFKA-12679) are resolved in modern releases. On old releases a workaround was per-instance distinct state.dir.

Example Stack Trace

org.apache.kafka.streams.errors.LockException: stream-thread [my-app-StreamThread-1] standby-task [0_18] Failed to lock the state directory: /opt/kafka-streams/my-app/0_18
	at org.apache.kafka.streams.processor.internals.StateDirectory.lock(StateDirectory.java:213)
	at org.apache.kafka.streams.processor.internals.ProcessorStateManager.<init>(ProcessorStateManager.java:122)
	at org.apache.kafka.streams.processor.internals.StreamTask.<init>(StreamTask.java:188)
	at org.apache.kafka.streams.processor.internals.ActiveTaskCreator.createTask(ActiveTaskCreator.java:124)
	at org.apache.kafka.streams.processor.internals.TaskManager.handleAssignment(TaskManager.java:288)
	at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:778)
Caused by: org.rocksdb.RocksDBException: IO error: lock /opt/kafka-streams/my-app/0_18/LOCK: No locks available

Diagnostic Commands

ls -la <state.dir>/<application.id>/*/.lock   # find which task dirs hold locks; lsof on the .lock file shows the owning PID

kafka-streams-application-reset.sh --application-id <app-id> --bootstrap-server <b>:9092 --input-topics <topics>   # after a full stop, clears local state so locks are recreated cleanly

lsof +D <state.dir>   # on Linux, list every process/file handle still open under the state directory

Hitting LockException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.

Kafka Exception LockException

Common Causes

Solutions

Example Stack Trace

Diagnostic Commands

Related

Kafka Exception `LockException`