What causes IllegalGenerationException?

Consumer took too long to process records between poll() calls, causing the broker-side session to expire and trigger a new rebalance with an incremented generation ID Consumer crashed or was killed during a rebalance and the group committed offsets with an outdated generation ID after rejoining Concurrent consumer group management from multiple application instances incorrectly sharing the same group.id with mismatched session state

How do I fix IllegalGenerationException?

Increase 'max.poll.interval.ms' (default 300000ms) or reduce processing time per poll batch by decreasing 'max.poll.records' (default 500) to prevent session timeout during processing Ensure offset commits happen within the poll loop and before max.poll.interval.ms elapses; use async commits for performance but sync commit before shutdown Check for duplicate group.id across different applications; each consumer group should have a unique, stable group.id — audit with kafka-consumer-groups.sh --list

Is IllegalGenerationException retriable?

No — IllegalGenerationException is not retriable. Retrying the same call will not help; the underlying cause must be fixed first.

Kafka Exception `IllegalGenerationException`

org.apache.kafka.common.errors.IllegalGenerationException

Non-retriable Consumer

IllegalGenerationException is a Kafka exception (org.apache.kafka.common.errors.IllegalGenerationException).

Common Causes

Consumer took too long to process records between poll() calls, causing the broker-side session to expire and trigger a new rebalance with an incremented generation ID
Consumer crashed or was killed during a rebalance and the group committed offsets with an outdated generation ID after rejoining
Concurrent consumer group management from multiple application instances incorrectly sharing the same group.id with mismatched session state

Solutions

Increase 'max.poll.interval.ms' (default 300000ms) or reduce processing time per poll batch by decreasing 'max.poll.records' (default 500) to prevent session timeout during processing
Ensure offset commits happen within the poll loop and before max.poll.interval.ms elapses; use async commits for performance but sync commit before shutdown
Check for duplicate group.id across different applications; each consumer group should have a unique, stable group.id — audit with kafka-consumer-groups.sh --list

Example Stack Trace

org.apache.kafka.common.errors.IllegalGenerationException: Specified group generation id is not valid.

Diagnostic Commands

# Describe consumer group offsets and lag
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group <group> --describe

# Look for rebalance events in logs
grep -E 'IllegalGeneration|Rebalance|JoinGroup|generation' /var/log/kafka/server.log | tail -30

Protocol error: ILLEGAL_GENERATION (code 22)

Hitting IllegalGenerationException in production? Conduktor Console gives you real-time visibility into clients, consumer groups, and broker health. Browse every Kafka exception or protocol error code.

Kafka Exception IllegalGenerationException

Common Causes

Solutions

Example Stack Trace

Diagnostic Commands

Related

Kafka Exception `IllegalGenerationException`