KIP-262 — Metadata should include number of state stores for task
Discarded Streams
Proposes including the number of state stores per task in the Streams partition assignor's subscription metadata so that the assignor can factor stateful task weight into assignment decisions, preventing stateful tasks from being over-concentrated on a subset of workers (rejected). Without state store counts in metadata, `StreamPartitionAssignor` treats stateful and stateless tasks identically, potentially assigning all RocksDB-backed tasks to the same worker.
Details
| Author | Richard Yu |
| Status | Discarded |
| JIRA | KAFKA-6583 |
| Wiki | View on Apache Wiki |
| Created | 2018-02-24 |
| Last Modified | 2019-01-03 |
Explore how this KIP affects the Kafka protocol in the Protocol Explorer, or see the full KIP database.