KIP-447 — Producer scalability for exactly once semantics

Accepted Kafka 2.5 TransactionsProducer

Scales exactly-once semantics (EOS) to support more concurrent transactional producers by introducing a shared transactional ID prefix and allowing epoch-based producer fencing without requiring a globally unique transactional ID per producer instance. The original EOS design (KIP-98) required one unique transactional ID per producer, forcing Kafka Streams to pre-allocate a fixed pool of IDs and limiting parallelism for applications like Kafka Streams with many tasks.

Protocol Impact

OffsetCommit · TxnOffsetCommit

Details