Kafka 2.6 → 2.7 Upgrade Guide

Kafka 2.7 brings 24 new configs, removes 0, and changes 3 defaults compared to 2.6.

+24 configs 3 defaults changed 21 KIPs

New Configurations

Config	Category	Default
controller.quota.window.num	broker	11
controller.quota.window.size.seconds	broker	1
max.connection.creation.rate	broker	2147483647
socket.connection.setup.timeout.max.ms	broker	127000 (127 seconds)
socket.connection.setup.timeout.ms	broker	10000 (10 seconds)
ssl.keystore.certificate.chain	broker	null
ssl.keystore.key	broker	null
ssl.truststore.certificates	broker	null
socket.connection.setup.timeout.max.ms	consumer	127000 (127 seconds)
socket.connection.setup.timeout.ms	consumer	10000 (10 seconds)
ssl.keystore.certificate.chain	consumer	null
ssl.keystore.key	consumer	null
ssl.truststore.certificates	consumer	null
socket.connection.setup.timeout.max.ms	producer	127000 (127 seconds)
socket.connection.setup.timeout.ms	producer	10000 (10 seconds)
ssl.keystore.certificate.chain	producer	null
ssl.keystore.key	producer	null
ssl.truststore.certificates	producer	null
socket.connection.setup.timeout.max.ms	connect_default	127000 (127 seconds)
socket.connection.setup.timeout.ms	connect_default	10000 (10 seconds)
ssl.keystore.certificate.chain	connect_default	null
ssl.keystore.key	connect_default	null
ssl.truststore.certificates	connect_default	null
task.timeout.ms	stream	300000 (5 minutes)

Changed Defaults

Config	Old Default	New Default
inter.broker.protocol.version	2.6-IV0	2.7-IV2
log.message.format.version	2.6-IV0	2.7-IV2
message.format.version	2.6-IV0	2.7-IV2

Protocol Changes

Fetch v12 · Fetch v12 · CreateTopics v6 · CreateTopics v6 · DeleteTopics v5 · DeleteTopics v5 · InitProducerId v4 · InitProducerId v4 · AddPartitionsToTxn v2 · AddPartitionsToTxn v2 · AddOffsetsToTxn v2 · AddOffsetsToTxn v2 · EndTxn v2 · EndTxn v2 · CreatePartitions v3 · CreatePartitions v3

KIPs Released

KIP-431 Support of printing additional ConsumerRecord fields in DefaultMessageFormatter
KIP-446 Add changelog topic configuration to KTable suppress
KIP-450 Sliding Window Aggregations in the DSL
KIP-478 Strongly typed Processor API
KIP-497 Add inter-broker API to alter ISR
KIP-545 support automated consumer offset sync across clusters in MM 2.0
KIP-554 Add Broker-side SCRAM Config API
KIP-584 Versioning scheme for features
KIP-590 Redirect Zookeeper Mutation Protocols to The Controller
KIP-595  A Raft Protocol for the Metadata Quorum
KIP-597 MirrorMaker2 internal topics Formatters
KIP-599 Throttle Create Topic, Create Partition and Delete Topic Operations
KIP-607 Add Metrics to Kafka Streams to Report Properties of RocksDB
KIP-612 Ability to Limit Connection Creation Rate on Brokers
KIP-616 Rename implicit Serdes instances in kafka-streams-scala
KIP-617 Allow Kafka Streams State Stores to be iterated backwards
KIP-626 Rename StreamsConfig config variable name
KIP-629 Use racially neutral terms in our codebase
KIP-651 Support PEM format for SSL certificates and private key
KIP-654 Aborted transaction with non-flushed data should throw a non-fatal exception
KIP-662 Throw Exception when Source Topics of a Streams App are Deleted