Kafka 2.8 → 3.0 Upgrade Guide

Kafka 3.0 brings 28 new configs, removes 13, and changes 15 defaults compared to 2.8.

+28 configs −13 configs 15 defaults changed 27 KIPs

Removed Configurations

Config	Category
advertised.host.name	broker
advertised.port	broker
host.name	broker
port	broker
quota.consumer.default	broker
quota.producer.default	broker
internal.key.converter	connect_default
internal.value.converter	connect_default
rest.host.name	connect_default
rest.port	connect_default
default.windowed.key.serde.inner	stream
default.windowed.value.serde.inner	stream
partition.grouper	stream

New Configurations

Config	Category	Default
broker.heartbeat.interval.ms	broker	2000 (2 seconds)
broker.session.timeout.ms	broker	9000 (9 seconds)
controller.listener.names	broker	null
controller.quorum.append.linger.ms	broker	25
controller.quorum.election.backoff.max.ms	broker	1000 (1 second)
controller.quorum.election.timeout.ms	broker	1000 (1 second)
controller.quorum.fetch.timeout.ms	broker	2000 (2 seconds)
controller.quorum.request.timeout.ms	broker	2000 (2 seconds)
controller.quorum.retry.backoff.ms	broker	20
controller.quorum.voters	broker
initial.broker.registration.timeout.ms	broker	60000 (1 minute)
metadata.log.dir	broker	null
metadata.log.max.record.bytes.between.snapshots	broker	20971520
metadata.log.segment.bytes	broker	1073741824 (1 gibibyte)
metadata.log.segment.ms	broker	604800000 (7 days)
metadata.max.retention.bytes	broker	-1
metadata.max.retention.ms	broker	604800000 (7 days)
node.id	broker	-1
process.roles	broker
sasl.mechanism.controller.protocol	broker	GSSAPI
local.retention.bytes	topic	-2
local.retention.ms	topic	-2
remote.storage.enable	topic	false
default.list.key.serde.inner	stream	null
default.list.key.serde.type	stream	null
default.list.value.serde.inner	stream	null
default.list.value.serde.type	stream	null
windowed.inner.class.serde	stream	null

Changed Defaults

Config	Old Default	New Default
inter.broker.protocol.version	2.8-IV1	3.0-IV1
listeners	null	PLAINTEXT://:9092
log.message.format.version	2.8-IV1	3.0-IV1
principal.builder.class	null	org.apache.kafka.common.security.authenticator.DefaultKafkaPrincipalBuilder
partition.assignment.strategy	class org.apache.kafka.clients.consumer.RangeAssignor	class org.apache.kafka.clients.consumer.RangeAssignor,class org.apache.kafka.clients.consumer.CooperativeStickyAssignor
session.timeout.ms	10000 (10 seconds)	45000 (45 seconds)
acks	1	all
enable.idempotence	false	true
message.format.version	2.8-IV1	3.0-IV1
connector.client.config.override.policy	None	All
listeners	null	http://:8083
default.key.serde	org.apache.kafka.common.serialization.Serdes$ByteArraySerde	null
default.value.serde	org.apache.kafka.common.serialization.Serdes$ByteArraySerde	null
replication.factor	1	-1
state.dir	/var/folders/st/wn8xlbk16ml31qrqpyh28rlc0000gn/T//kafka-streams	/var/folders/5w/m48dfpps5fj1byw1ldmq3v5w0000gp/T//kafka-streams

Protocol Changes

ListOffsets v7 · ListOffsets v7 · OffsetFetch v8 · OffsetFetch v8 · FindCoordinator v4 · FindCoordinator v4

KIPs Released

KIP-216 IQ should throw different exceptions for different errors
KIP-466 Add support for List<T> serialization and deserialization
KIP-622 Add currentSystemTimeMs and currentStreamTimeMs to ProcessorContext
KIP-633 Deprecate 24-hour Default Grace Period for Windowed Operations in Streams
KIP-635 GetOffsetShell: support for multiple topics and consumer configuration override
KIP-666 Add Instant-based methods to ReadOnlySessionStore
KIP-681 Rename master key in delegation token feature
KIP-695 Further Improve Kafka Streams Timestamp Synchronization
KIP-699 Update FindCoordinator to resolve multiple Coordinators at a time
KIP-707 The future of KafkaFuture
KIP-716 Allow configuring the location of the offset-syncs topic with MirrorMaker2
KIP-720 Deprecate MirrorMaker v1
KIP-721 Enable connector log contexts in Connect Log4j configuration
KIP-722 Enable connector client overrides by default
KIP-724 Drop support for message formats v0 and v1
KIP-730 Producer ID generation in KRaft mode
KIP-732 Deprecate eos-alpha and replace eos-beta with eos-v2
KIP-733 change Kafka Streams default replication factor config
KIP-734 Improve AdminClient.listOffsets to return timestamp and offset for the record with the largest timestamp
KIP-735 Increase default consumer session timeout
KIP-737 Add canTrackSource to ReplicationPolicy
KIP-738 Removal of Connect's internal converter properties
KIP-740 Clean up public API in TaskId
KIP-743 Remove config value 0.10.0-2.4 of Streams built-in metrics version config
KIP-744 Migrate TaskMetadata and ThreadMetadata to an interface with internal implementation
KIP-745 Connect API to restart connector and tasks
KIP-746 Revise KRaft Metadata Records