Kafka Configs Kafka Protocol Wire Format API Versions KIPs Errors Kafka Upgrade Config Advisor Flink SQL
conduktor.io ↗

Kafka Fetch Request Wire Format v18 — Binary Protocol Layout

What is Fetch?

Consumers and follower replicas both issue this continuously for log replication and consumption. When a consumer's fetch stalls with empty responses despite records existing, check isolation.level: read_committed consumers are blocked by open transactions until EndTxn.

Related Errors

FENCED_LEADER_EPOCH · FETCH_SESSION_ID_NOT_FOUND · FETCH_SESSION_TOPIC_ID_ERROR · INVALID_FETCH_SESSION_EPOCH · KAFKA_STORAGE_ERROR · LEADER_NOT_AVAILABLE · NETWORK_EXCEPTION · NOT_LEADER_OR_FOLLOWER · OFFSET_MOVED_TO_TIERED_STORAGE · OFFSET_NOT_AVAILABLE · OFFSET_OUT_OF_RANGE · THROTTLING_QUOTA_EXCEEDED · TOPIC_AUTHORIZATION_FAILED · UNKNOWN_LEADER_EPOCH · UNKNOWN_TOPIC_ID · UNKNOWN_TOPIC_OR_PARTITION

Wire Diagram

Request Header · flexible
message_size int32 · 4B
api_key int16 · 2B
api_version int16 · 2B
correlation_id int32 · 4B
client_id string (compact)
tagged var
FetchRequest v18
ClusterId? string (compact)
ReplicaState struct
MaxWaitMs int32 · 4B
MinBytes int32 · 4B
MaxBytes int32 · 4B
IsolationLevel int8 · 1B
SessionId int32 · 4B
SessionEpoch int32 · 4B
Topics array
ForgottenTopicsData array
RackId string (compact)
tagged var
ReplicaId int32 · 4B
ReplicaEpoch int64 · 8B
tagged var
TopicId uuid · 16B
Partitions array
tagged var
Partition int32 · 4B
CurrentLeaderEpoch int32 · 4B
FetchOffset int64 · 8B
LastFetchedEpoch int32 · 4B
LogStartOffset int64 · 8B
PartitionMaxBytes int32 · 4B
ReplicaDirectoryId uuid · 16B
HighWatermark int64 · 8B
tagged var
TopicId uuid · 16B
Partitions []int32
tagged var

Schema & Example

Schema
{
  "ClusterId": string?,
  "ReplicaState": {
    "ReplicaId": int32,
    "ReplicaEpoch": int64
  },
  "MaxWaitMs": int32,
  "MinBytes": int32,
  "MaxBytes": int32,
  "IsolationLevel": int8,
  "SessionId": int32,
  "SessionEpoch": int32,
  "Topics": [{
      "TopicId": uuid,
      "Partitions": [{
          "Partition": int32,
          "CurrentLeaderEpoch": int32,
          "FetchOffset": int64,
          "LastFetchedEpoch": int32,
          "LogStartOffset": int64,
          "PartitionMaxBytes": int32,
          "ReplicaDirectoryId": uuid,
          "HighWatermark": int64
      }]
  }],
  "ForgottenTopicsData": [{
      "TopicId": uuid,
      "Partitions": [int32]
  }],
  "RackId": string
}
Example
{
  "ClusterId": "dQw4w9WgXcQ",
  "ReplicaState": {
    "ReplicaId": -1,
    "ReplicaEpoch": 12
  },
  "MaxWaitMs": 500,
  "MinBytes": 65536,
  "MaxBytes": 65536,
  "IsolationLevel": 1,
  "SessionId": 1847,
  "SessionEpoch": 12,
  "Topics": [{
      "TopicId": "550e8400-e29b-41d4-a716-446655440000",
      "Partitions": [{
          "Partition": 0,
          "CurrentLeaderEpoch": 12,
          "FetchOffset": 150382,
          "LastFetchedEpoch": 12,
          "LogStartOffset": 284729,
          "PartitionMaxBytes": 65536,
          "ReplicaDirectoryId": "550e8400-e29b-41d4-a716-446655440000",
          "HighWatermark": 284729
      }]
  }],
  "ForgottenTopicsData": [{
      "TopicId": "550e8400-e29b-41d4-a716-446655440000",
      "Partitions": [1, 2, 3]
  }],
  "RackId": "abc-123"
}