Kafka Configs Kafka Protocol Wire Format API Versions KIPs Errors Kafka Upgrade Config Advisor Flink SQL
conduktor.io ↗

Kafka OffsetCommit Request Wire Format v10 — Binary Protocol Layout

What is OffsetCommit?

Every consumer sends this after processing records to checkpoint its position into __consumer_offsets. Commit failures during rebalances (REBALANCE_IN_PROGRESS, ILLEGAL_GENERATION) are the root cause of duplicate processing in at-least-once consumers.

Related Errors

COORDINATOR_NOT_AVAILABLE · GROUP_AUTHORIZATION_FAILED · INVALID_GROUP_ID · OFFSET_METADATA_TOO_LARGE · STALE_MEMBER_EPOCH · TOPIC_AUTHORIZATION_FAILED · UNKNOWN_TOPIC_ID · UNKNOWN_TOPIC_OR_PARTITION

Wire Diagram

Request Header · flexible
message_size int32 · 4B
api_key int16 · 2B
api_version int16 · 2B
correlation_id int32 · 4B
client_id string (compact)
tagged var
OffsetCommitRequest v10
GroupId string (compact)
GenerationIdOrMemberEpoch int32 · 4B
MemberId string (compact)
GroupInstanceId? string (compact)
Topics array
tagged var
TopicId uuid · 16B
Partitions array
tagged var
PartitionIndex int32 · 4B
CommittedOffset int64 · 8B
CommittedLeaderEpoch int32 · 4B
CommittedMetadata? string (compact)
tagged var

Schema & Example

Schema
{
  "GroupId": string,
  "GenerationIdOrMemberEpoch": int32,
  "MemberId": string,
  "GroupInstanceId": string?,
  "Topics": [{
      "TopicId": uuid,
      "Partitions": [{
          "PartitionIndex": int32,
          "CommittedOffset": int64,
          "CommittedLeaderEpoch": int32,
          "CommittedMetadata": string?
      }]
  }]
}
Example
{
  "GroupId": "order-processors",
  "GenerationIdOrMemberEpoch": 12,
  "MemberId": "consumer-1-abc123",
  "GroupInstanceId": "instance-0",
  "Topics": [{
      "TopicId": "550e8400-e29b-41d4-a716-446655440000",
      "Partitions": [{
          "PartitionIndex": 3,
          "CommittedOffset": 150382,
          "CommittedLeaderEpoch": 12,
          "CommittedMetadata": ""
      }]
  }]
}