Kafka 2.0 → 2.1 Upgrade Guide

Kafka 2.1 brings 8 new configs, removes 21, and changes 4 defaults compared to 2.0.

+8 configs −21 configs 4 defaults changed 24 KIPs

Removed Configurations

Config	Category
auto.commit.enable	consumer
consumer.id	consumer
consumer.timeout.ms	consumer
dual.commit.enabled	consumer
fetch.message.max.bytes	consumer
fetch.wait.max.ms	consumer
num.consumer.fetchers	consumer
offsets.channel.backoff.ms	consumer
offsets.channel.socket.timeout.ms	consumer
offsets.commit.max.retries	consumer
offsets.storage	consumer
queued.max.message.chunks	consumer
rebalance.backoff.ms	consumer
rebalance.max.retries	consumer
refresh.leader.backoff.ms	consumer
socket.receive.buffer.bytes	consumer
socket.timeout.ms	consumer
zookeeper.connect	consumer
zookeeper.connection.timeout.ms	consumer
zookeeper.session.timeout.ms	consumer
zookeeper.sync.time.ms	consumer

New Configurations

Config	Category	Default
connection.failed.authentication.delay.ms	broker	100
kafka.metrics.polling.interval.secs	broker	10
kafka.metrics.reporters	broker
client.dns.lookup	consumer	default
client.dns.lookup	producer	default
delivery.timeout.ms	producer	120000
client.dns.lookup	connect_default	default
max.task.idle.ms	stream	0

Changed Defaults

Config	Old Default	New Default
inter.broker.protocol.version	2.0-IV1	2.1-IV2
log.message.format.version	2.0-IV1	2.1-IV2
retries	0	2147483647
message.format.version	2.0-IV1	2.1-IV2

KIPs Released

KIP-206 Add support for UUID serialization and deserialization
KIP-211 Revise Expiration Semantics of Consumer Group Offsets
KIP-231 Improve the Required ACL of ListGroups API
KIP-235 Add DNS alias support for secured connection
KIP-270 A Scala Wrapper Library for Kafka Streams
KIP-302 Enable Kafka clients to use all DNS resolved IP addresses
KIP-306 Configuration for Delaying Response to Failed Authentication
KIP-312 Add Overloaded StreamsBuilder Build Method to Accept java.util.Properties
KIP-319 Replace segments with segmentInterval in WindowBytesStoreSupplier
KIP-321 Update TopologyDescription to better represent Source and Sink Nodes
KIP-324 Add method to get metrics() in AdminClient
KIP-330 Add retentionPeriod in SessionBytesStoreSupplier
KIP-332 Update AclCommand to use AdminClient API
KIP-338 Support to exclude the internal topics in kafka-topics.sh command
KIP-340 Allow kafka-reassign-partitions.sh and kafka-log-dirs.sh to take admin client property file
KIP-342 Add support for custom SASL extensions in OAuthBearer authentication
KIP-346 Improve LogCleaner behavior on error
KIP-353 Improve Kafka Streams Timestamp Synchronization
KIP-356 Add withCachingDisabled() to StoreBuilder
KIP-357 Add support to list ACLs per principal
KIP-358 Migrate Streams API to Duration instead of long ms times
KIP-365 Materialized, Serialized, Joined, Consumed and Produced with implicit Serde
KIP-366 Make FunctionConversions deprecated
KIP-372 Naming Repartition Topics for Joins and Grouping