Kafka 3.6 → 3.7 Upgrade Guide

Kafka 3.7 brings 24 new configs, removes 0, and changes 2 defaults compared to 3.6.

+24 configs 2 defaults changed 28 KIPs

New Configurations

Config	Category	Default
eligible.leader.replicas.enable	broker	false
group.consumer.assignors	broker	org.apache.kafka.coordinator.group.assignor.UniformAssignor,org.apache.kafka.coordinator.group.assignor.RangeAssignor
group.consumer.heartbeat.interval.ms	broker	5000 (5 seconds)
group.consumer.max.heartbeat.interval.ms	broker	15000 (15 seconds)
group.consumer.max.session.timeout.ms	broker	60000 (1 minute)
group.consumer.max.size	broker	2147483647
group.consumer.min.heartbeat.interval.ms	broker	5000 (5 seconds)
group.consumer.min.session.timeout.ms	broker	45000 (45 seconds)
group.consumer.session.timeout.ms	broker	45000 (45 seconds)
group.coordinator.rebalance.protocols	broker	classic
group.coordinator.threads	broker	1
remote.log.index.file.cache.total.size.bytes	broker	1073741824 (1 gibibyte)
ssl.allow.dn.changes	broker	false
ssl.allow.san.changes	broker	false
telemetry.max.bytes	broker	1048576 (1 mebibyte)
enable.metrics.push	consumer	true
group.protocol	consumer	classic
group.remote.assignor	consumer	null
retry.backoff.max.ms	consumer	1000 (1 second)
enable.metrics.push	producer	true
retry.backoff.max.ms	producer	1000 (1 second)
retry.backoff.max.ms	connect_default	1000 (1 second)
dsl.store.suppliers.class	stream	org.apache.kafka.streams.state.BuiltInDslStoreSuppliers$RocksDBDslStoreSuppliers
enable.metrics.push	stream	true

Changed Defaults

Config	Old Default	New Default
inter.broker.protocol.version	3.6-IV2	3.7-IV4
state.dir	/var/folders/3j/8r9d0znd5pzgp8ww95yn8g140000gp/T//kafka-streams	/var/folders/tj/qjgd_zb13w19nkmt8rfpnssc0000gn/T//kafka-streams

Protocol Changes

Produce v10 · Produce v10 · Fetch v16 · Fetch v16 · OffsetFetch v9 · OffsetFetch v9 · DescribeCluster v1 · DescribeCluster v1 · BrokerRegistration v2 · BrokerRegistration v2 · BrokerHeartbeat v1 · BrokerHeartbeat v1

KIPs Released

KIP-580 Exponential Backoff for Kafka Clients
KIP-589 Add API to update Replica state in Controller
KIP-714 Client metrics and observability
KIP-833 Mark KRaft as Production Ready
KIP-858 Handle JBOD broker disk failure in KRaft
KIP-896 Remove old client protocol API versions in Kafka 4.0
KIP-919 Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration
KIP-923 Add A Grace Period to Stream Table Join
KIP-938 Add more metrics for measuring KRaft performance
KIP-941 Range queries to accept null lower and upper bounds
KIP-951 Leader discovery optimizations for the client
KIP-954 expand default DSL store configuration to custom types
KIP-959 Add BooleanConverter to Kafka Connect
KIP-960 Support single-key_single-timestamp Interactive Queries (IQv2) for Versioned State Stores
KIP-962 Relax non-null key requirement in Kafka Streams
KIP-963 Additional metrics in Tiered Storage
KIP-966 Eligible Leader Replicas
KIP-968 Support single-key_multi-timestamp Interactive Queries (IQv2) for Versioned State Stores 
KIP-975 Docker Image for Apache Kafka
KIP-976 Cluster-wide dynamic log adjustment for Kafka Connect
KIP-979 Allow independently stop KRaft processes
KIP-980 Allow creating connectors in a stopped state
KIP-985 Add reverseRange and reverseAll query over kv-store in IQv2
KIP-988 Streams Standby Task Update Listener
KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery
KIP-1000 List Client Metrics Configuration Resources
KIP-1001 Add CurrentControllerId Metric
KIP-1014 Managing Unstable Features in Apache Kafka