Kafka Configs Kafka Protocol Wire Format API Versions KIPs Errors Kafka Upgrade Config Advisor Flink SQL
conduktor.io ↗

Kafka Fetch Response Wire Format v16 — Binary Protocol Layout

What is Fetch?

Consumers and follower replicas both issue this continuously for log replication and consumption. When a consumer's fetch stalls with empty responses despite records existing, check isolation.level: read_committed consumers are blocked by open transactions until EndTxn.

Related Errors

FENCED_LEADER_EPOCH · FETCH_SESSION_ID_NOT_FOUND · FETCH_SESSION_TOPIC_ID_ERROR · INVALID_FETCH_SESSION_EPOCH · KAFKA_STORAGE_ERROR · LEADER_NOT_AVAILABLE · NETWORK_EXCEPTION · NOT_LEADER_OR_FOLLOWER · OFFSET_MOVED_TO_TIERED_STORAGE · OFFSET_NOT_AVAILABLE · OFFSET_OUT_OF_RANGE · THROTTLING_QUOTA_EXCEEDED · TOPIC_AUTHORIZATION_FAILED · UNKNOWN_LEADER_EPOCH · UNKNOWN_TOPIC_ID · UNKNOWN_TOPIC_OR_PARTITION

Wire Diagram

Response Header · flexible
message_size int32 · 4B
correlation_id int32 · 4B
tagged var
FetchResponse v16
ThrottleTimeMs int32 · 4B
ErrorCode int16 · 2B
SessionId int32 · 4B
Responses array
NodeEndpoints array
tagged var
TopicId uuid · 16B
Partitions array
tagged var
PartitionIndex int32 · 4B
ErrorCode int16 · 2B
HighWatermark int64 · 8B
LastStableOffset int64 · 8B
LogStartOffset int64 · 8B
DivergingEpoch struct
CurrentLeader struct
SnapshotId struct
AbortedTransactions? array
PreferredReadReplica int32 · 4B
Records? records (compact)
tagged var
Epoch int32 · 4B
EndOffset int64 · 8B
tagged var
LeaderId int32 · 4B
LeaderEpoch int32 · 4B
tagged var
EndOffset int64 · 8B
Epoch int32 · 4B
tagged var
ProducerId int64 · 8B
FirstOffset int64 · 8B
tagged var
NodeId int32 · 4B
Host string (compact)
Port int32 · 4B
Rack? string (compact)
tagged var

Schema & Example

Schema
{
  "ThrottleTimeMs": int32,
  "ErrorCode": int16,
  "SessionId": int32,
  "Responses": [{
      "TopicId": uuid,
      "Partitions": [{
          "PartitionIndex": int32,
          "ErrorCode": int16,
          "HighWatermark": int64,
          "LastStableOffset": int64,
          "LogStartOffset": int64,
          "DivergingEpoch": {
            "Epoch": int32,
            "EndOffset": int64
          },
          "CurrentLeader": {
            "LeaderId": int32,
            "LeaderEpoch": int32
          },
          "SnapshotId": {
            "EndOffset": int64,
            "Epoch": int32
          },
          "AbortedTransactions": [{
              "ProducerId": int64,
              "FirstOffset": int64
          }]?,
          "PreferredReadReplica": int32,
          "Records": records?
      }]
  }],
  "NodeEndpoints": [{
      "NodeId": int32,
      "Host": string,
      "Port": int32,
      "Rack": string?
  }]
}
Example
{
  "ThrottleTimeMs": 0,
  "ErrorCode": 0,
  "SessionId": 1847,
  "Responses": [{
      "TopicId": "550e8400-e29b-41d4-a716-446655440000",
      "Partitions": [{
          "PartitionIndex": 3,
          "ErrorCode": 0,
          "HighWatermark": 284729,
          "LastStableOffset": 284729,
          "LogStartOffset": 284729,
          "DivergingEpoch": {
            "Epoch": 17,
            "EndOffset": 150382
          },
          "CurrentLeader": {
            "LeaderId": 1,
            "LeaderEpoch": 17
          },
          "SnapshotId": {
            "EndOffset": 150382,
            "Epoch": 17
          },
          "AbortedTransactions": [{
              "ProducerId": 0,
              "FirstOffset": 150382
          }]?,
          "PreferredReadReplica": 2,
          "Records": "<binary>"
      }]
  }],
  "NodeEndpoints": [{
      "NodeId": 1,
      "Host": "broker-1.kafka.local",
      "Port": 9092,
      "Rack": "us-east-1a"
  }]
}