conduktor.io ↗

KIP-262 — Metadata should include number of state stores for task

Discarded Streams

Proposes including the number of state stores per task in the Streams partition assignor's subscription metadata so that the assignor can factor stateful task weight into assignment decisions, preventing stateful tasks from being over-concentrated on a subset of workers (rejected). Without state store counts in metadata, `StreamPartitionAssignor` treats stateful and stateless tasks identically, potentially assigning all RocksDB-backed tasks to the same worker.

Details

AuthorRichard Yu
StatusDiscarded
JIRAKAFKA-6583
WikiView on Apache Wiki
Created2018-02-24
Last Modified2019-01-03
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.