Kafka Error REBALANCE_IN_PROGRESS
Error code 27 · Non-retriable Consumer
The group is rebalancing, so a rejoin is needed.
Common Causes
- Consumer missed heartbeat deadline due to long poll processing time exceeding max.poll.interval.ms
- New consumer joining or existing consumer leaving the group triggers rebalance for all members
- Broker-side session timeout (session.timeout.ms) elapsed while consumer was GC-paused or CPU-starved
Solutions
- Increase max.poll.interval.ms or reduce records processed per poll via max.poll.records to avoid heartbeat starvation
- Enable static group membership (group.instance.id) to avoid full rebalances on restarts and rolling deploys
- Use incremental cooperative rebalancing (partition.assignment.strategy=CooperativeStickyAssignor) to minimize partition movement during rebalances
Diagnostic Commands
# Check consumer group state and members
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group <group-id> | grep -E 'STATE|CONSUMER-ID|HOST'
# Look for rebalance events in logs
grep 'Attempt to heartbeat failed\|poll interval exceeded\|rebalance' /var/log/kafka-consumer.log | tail -50
Debugging Kafka errors? Conduktor Console gives you real-time visibility into your cluster. Explore all errors in the Error Decoder.