Kafka Exception StreamsRebalancingException
org.apache.kafka.streams.errors.StreamsRebalancingException
Non-retriable
Streams
Indicates that Kafka Streams is in state REBALANCING REBALANCING and thus cannot be queried by default. You can retry to query after the rebalance finished. As an alternative, you can also query (potentially stale) state stores during a rebalance via enableStaleStores().
Common Causes
- An Interactive Query was issued while KafkaStreams.State is REBALANCING (instance startup, scaling event, or a member join/leave), and stale reads were not enabled, so the query is rejected by default.
- Frequent rebalances (caused by long GC pauses, max.poll.interval.ms exceeded, or session timeouts) keep the instance in REBALANCING long enough that queries repeatedly hit this exception.
- A large state restore after a rebalance keeps the assigned task in restoring/REBALANCING for a long time, widening the unavailability window for IQ.
- Adding/removing instances or threads triggers a cooperative rebalance; during the settling period the store is intentionally not queryable for consistency.
Solutions
- Pass StoreQueryParameters with enableStaleStores() (KIP-535) to allow reads from standby/restoring replicas during the rebalance if the use case (cache, dashboard) can tolerate eventual consistency.
- Retry the query with backoff until the state returns to RUNNING; combine with a KafkaStreams.StateListener so you only retry while REBALANCING and fail fast on ERROR.
- Reduce rebalance frequency/duration: tune max.poll.interval.ms, session.timeout.ms, and use static group membership (group.instance.id) so transient restarts don't trigger full rebalances.
- Configure standby replicas (num.standby.replicas >= 1) so a standby can serve stale reads and shorten failover, keeping IQ available during rebalances.
Example Stack Trace
org.apache.kafka.streams.errors.StreamsRebalancingException: Cannot process while rebalancing; the state store is not available until the rebalance completes
at org.apache.kafka.streams.KafkaStreams.validateIsRunningOrRebalancing(KafkaStreams.java:1460)
at org.apache.kafka.streams.KafkaStreams.store(KafkaStreams.java:1798)
at com.example.QueryService.lookup(QueryService.java:41)Diagnostic Commands
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-app --state # confirm group is in PreparingRebalance/CompletingRebalance
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-app --members --verbose # spot the member causing repeated rebalancesRelated
Related Streams exceptions: BrokerNotFoundException · InternalTopicsAlreadySetupException · InvalidStateStoreException · InvalidStateStorePartitionException · LockException · MisconfiguredInternalTopicException · MissingInternalTopicsException · MissingSourceTopicException
Hitting
StreamsRebalancingException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.