What is Fetch?
Consumers and follower replicas both issue this continuously for log replication and consumption. When a consumer's fetch stalls with empty responses despite records existing, check isolation.level: read_committed consumers are blocked by open transactions until EndTxn.
| Version | Kafka | Changes |
|---|---|---|
| FetchRequest | ||
| v4 | 2.2 | + IsolationLevel int8adds IsolationLevel. Starting in version 4, the requestor must be able to handle Kafka log message format version 2. |
| v5 | 2.2 | + LogStartOffset int64adds LogStartOffset to indicate the earliest available offset of partition data that can be consumed. |
| v6 | 2.2 | is the same as version 5. |
| v7 | 2.2 | + SessionId int32+ SessionEpoch int32+ ForgottenTopicsData []ForgottenTopic+ Topic string+ Partitions []int32adds incremental fetch request support. |
| v8 | 2.2 | is the same as version 7. |
| v9 | 2.2 | + CurrentLeaderEpoch int32adds CurrentLeaderEpoch, as described in KIP-320. |
| v10 | 2.2 | indicates that we can use the ZStd compression algorithm, as described in KIP-110. |
| v11 | 2.3 | + RackId string |
| v12 | 2.7 | + ClusterId string+ LastFetchedEpoch int32~ Flexible encoding enabled FLEXIBLE adds flexible versions support as well as epoch validation through the `LastFetchedEpoch` field |
| v13 | 3.1 | + TopicId uuid+ TopicId uuid- Topic string- Topic stringreplaces topic names with topic IDs (KIP-516). May return UNKNOWN_TOPIC_ID error code. |
| v14 | 3.5 | is the same as version 13 but it also receives a new error called OffsetMovedToTieredStorageException(KIP-405) |
| v15 | 3.5 | + ReplicaState ReplicaState+ ReplicaId int32+ ReplicaEpoch int64- ReplicaId int32adds the ReplicaState which includes new field ReplicaEpoch and the ReplicaId. Also, deprecate the old ReplicaId field and set its default value to -1. (KIP-903) |
| v16 | 3.7 | is the same as version 15 (KIP-951). |
| v17 | 3.9 | + ReplicaDirectoryId uuidadds directory id support from KIP-853 |
| v18 | 4.1 | + HighWatermark int64adds high-watermark from KIP-1166 |
| FetchResponse | ||
| v4 | 2.2 | + LastStableOffset int64+ AbortedTransactions []AbortedTransaction+ ProducerId int64+ FirstOffset int64adds features for transactional consumption. |
| v5 | 2.2 | + LogStartOffset int64adds LogStartOffset to indicate the earliest available offset of partition data that can be consumed. |
| v6 | 2.2 | we may return KAFKA_STORAGE_ERROR as an error code. |
| v7 | 2.2 | + ErrorCode int16+ SessionId int32adds incremental fetch request support. |
| v8 | 2.2 | on quota violation, brokers send out responses before throttling. |
| v9 | 2.2 | is the same as version 8. |
| v10 | 2.2 | indicates that the response data can use the ZStd compression algorithm, as described in KIP-110. |
| v11 | 2.3 | + PreferredReadReplica int32 |
| v12 | 2.7 | + DivergingEpoch EpochEndOffset+ Epoch int32+ EndOffset int64+ CurrentLeader LeaderIdAndEpoch+ LeaderId int32+ LeaderEpoch int32+ SnapshotId SnapshotId~ Flexible encoding enabled FLEXIBLE adds support for flexible versions, epoch detection through the `TruncationOffset` field, and leader discovery through the `CurrentLeader` field |
| v13 | 3.1 | + TopicId uuid- Topic stringreplaces the topic name field with topic ID (KIP-516). |
| v14 | 3.5 | is the same as version 13 but it also receives a new error called OffsetMovedToTieredStorageException (KIP-405) |
| v15 | 3.5 | is the same as version 14 (KIP-903). |
| v16 | 3.7 | + NodeEndpoints []NodeEndpoint+ NodeId int32+ Host string+ Port int32+ Rack stringadds the 'NodeEndpoints' field (KIP-951). |
| v17 | 3.9 | no changes to the response (KIP-853). |
| v18 | 4.1 | no changes to the response (KIP-1166) |