What is OffsetCommit?
Every consumer sends this after processing records to checkpoint its position into __consumer_offsets. Commit failures during rebalances (REBALANCE_IN_PROGRESS, ILLEGAL_GENERATION) are the root cause of duplicate processing in at-least-once consumers.
| Version | Kafka | Changes |
|---|---|---|
| OffsetCommitRequest | ||
| v2 | 2.2 | + RetentionTimeMs int64adds retention time. It removes the commit timestamp added in version 1. |
| v3 | 2.2 | are the same as version 2. |
| v4 | 2.2 | are the same as version 2. |
| v5 | 2.2 | - RetentionTimeMs int64removes the retention time, which is now controlled only by a broker configuration. |
| v6 | 2.2 | + CommittedLeaderEpoch int32adds the leader epoch for fencing. |
| v7 | 2.3 | + GroupInstanceId stringadds a new field called groupInstanceId to indicate member identity across restarts. |
| v8 | 2.4 | ~ Flexible encoding enabled FLEXIBLE is the first flexible version. |
| v9 | 3.6 | is the first version that can be used with the new consumer group protocol (KIP-848). The request is the same as version 8. |
| v10 | 4.1 | + TopicId uuid- Name stringadds support for topic ids and removes support for topic names (KIP-848). |
| OffsetCommitResponse | ||
| v2 | 2.2 | are the same as version 0. |
| v3 | 2.2 | + ThrottleTimeMs int32adds the throttle time to the response. |
| v4 | 2.2 | on quota violation, brokers send out responses before throttling. |
| v5 | 2.2 | are the same as version 4. |
| v6 | 2.2 | are the same as version 4. |
| v7 | 2.3 | offsetCommitRequest supports a new field called groupInstanceId to indicate member identity across restarts. |
| v8 | 2.4 | ~ Flexible encoding enabled FLEXIBLE is the first flexible version. |
| v9 | 3.6 | is the first version that can be used with the new consumer group protocol (KIP-848). The response is the same as version 8 but can return STALE_MEMBER_EPOCH when the new consumer group protocol is used and GROUP_ID_NOT_FOUND when the group does not exist for both protocols. |
| v10 | 4.1 | + TopicId uuid- Name stringadds support for topic ids and removes support for topic names (KIP-848). |