Kafka Configs Kafka Protocol Wire Format API Versions KIPs Errors Kafka Upgrade Config Advisor Policy Pack
Debezium
Config Matrix Event Envelope Snapshot Modes Error Reference
Flink SQL
conduktor.io ↗
Home · Kafka Policy Pack

Kafka Policy Pack

Copy-pasteable Conduktor ResourcePolicy YAML for common Kafka governance concerns. These are the defaults the Conduktor platform team runs against its own clusters, written down so you can run them against yours.

Starter

Ten essentials. Start here if you have nothing in place today.

10 policies
View preset →

Production Hardening

Starter plus 15 extra rules for clusters carrying real traffic.

25 policies
View preset →

Financial Services

Defaults aligned with SOC2, PCI, GDPR, and HIPAA controls.

17 policies
View preset →

Multi-Tenant SaaS

Tenant prefixes, quota tiers, and chargeback labels for shared clusters.

18 policies
View preset →

Technical & Governance Policies

What you can protect against, by category. Pick a category, browse the rules, copy the YAML you need.

Naming

Topic and consumer-group naming conventions that make a cluster searchable and ownable.

    01
    Topic Name Convention
    Topic names must follow <env>.<domain>.<entity>.<version>.
    →     02
    Consumer Group Prefix
    Consumer group IDs must be prefixed by their owning team.
    →     03
    Tenant-ID Topic Prefix
    Topic names must begin with a tenant identifier prefix t-<6 alphanum>. Every topic belongs to exactly one tenant.
    →     04
    Tenant Label Must Match Topic Prefix
    metadata.labels.tenant-id is mandatory and must equal the tenant segment embedded in the topic name.
    →

Replication

Replication factor floors, ISR alignment, rack-awareness.

    01
    Minimum Replication Factor
    Production topics must have replication factor ≥ 3.
    →     02
    ISR Alignment with Replication Factor
    min.insync.replicas must equal replicationFactor − 1.
    →     03
    Minimum min.insync.replicas for Durability
    min.insync.replicas must be >= 2 and <= replicationFactor − 1.
    →     04
    min.insync.replicas Floor for Regulated Topics
    Topics labeled data-criticality C2/C3 must set min.insync.replicas >= 3 and replicationFactor >= 5.
    →     05
    min.insync.replicas Floor by Tenant Tier
    min.insync.replicas floor is tied to tenant-tier: free >= 1, paid >= 2, enterprise >= 2 with RF >= 3.
    →

Retention

Retention bounds, coherence between bytes and time, compacted-topic rules.

    01
    Maximum Retention Bound
    retention.ms must not exceed 30 days unless explicitly justified.
    →     02
    retention.bytes Required Alongside retention.ms
    Every non-compacted topic must declare a retention.bytes cap.
    →     03
    segment.bytes Within Sane Operational Bounds
    segment.bytes must be between 64 MiB and 2 GiB.
    →     04
    Audit Retention Floor (No Infinite, 7y for Audit-Tagged)
    retention.ms must be a finite value; topics labeled audit=true must retain at least 7 years (~220752000000 ms).
    →     05
    Retention Capped by Tenant Tier
    retention.ms is capped by tenant-tier: free <= 7d, paid <= 30d, enterprise <= 90d.
    →

Schema Enforcement

Schema-required topics, compatibility modes, subject naming.

    01
    Schema Required for Non-Internal Topics
    Every non-internal topic must have at least one registered schema subject.
    →     02
    Subject Name Must Follow -key / -value Convention
    Reject subjects that do not end in -key or -value (TopicNameStrategy).
    →     03
    Subject Compatibility Cannot Be NONE
    Forbid spec.compatibility: NONE and require an explicit non-NONE mode.
    →     04
    Schema Compatibility = FULL for Regulated Subjects
    Subjects labeled regulated=true must declare compatibility FULL or FULL_TRANSITIVE. No breaking changes allowed.
    →

Security & ACLs

No wildcard principals, SASL allowlists, prefix-based ACLs.

    01
    No Wildcard ACLs in Production
    Production ACLs must not grant `*` principal or `*` resource pattern.
    →     02
    ApplicationGroup Cannot Grant Wildcard Write Permissions
    Reject ApplicationGroup permissions that combine name="*" with write/create/delete on topics or connectors.
    →     03
    Production ApplicationGroups Must Sync from SSO
    For groups targeting prod instances, forbid spec.members and require externalGroups/externalGroupRegex.
    →     04
    No Wildcard Subject Access in ApplicationGroups
    SUBJECT permissions must be prefix-scoped (patternType=PREFIXED, name!="*").
    →     05
    ApplicationGroup Cannot Span Tenants
    ApplicationGroup must carry labels.tenant-id and every permission resource name must start with that tenant id. No wildcards.
    →

Operational Hygiene

Avoid auto-create, reserved prefixes, opinionated defaults.

    01
    Topic Must Declare owner and data-criticality Labels
    Every topic must carry `owner` (email) and `data-criticality` (C0..C3) labels.
    →     02
    Topic Name Must Include Declared domain Label as Prefix
    Topic name must start with the `domain` label value.
    →     03
    Data Criticality Label Required
    Every topic must carry a data-criticality label (C0/C1/C2/C3) so encryption, retention, and access policies can target it.
    →     04
    Data Residency Label Required
    Every topic must declare a data-residency region (eu, us, uk, apac, global). GDPR and equivalents require it.
    →     05
    Cost-Center Label Required (Chargeback)
    Every topic must carry metadata.labels.cost-center and metadata.labels.quota-tier in {bronze, silver, gold}.
    →     06
    Topic Owner Label Must Be a Team Slug
    Every topic must carry an owner label that resolves to a real team in the IdP / Backstage catalog.
    →

Connectors

Kafka Connect plugin allowlists, parallelism caps, DLQ wiring.

    01
    Only Allow Vetted connector.class Plugins
    Reject any connector whose connector.class is not on the platform allowlist.
    →     02
    connector tasks.max Within [1, 16]
    Cap connector parallelism. Runaway tasks.max is the #1 way to DoS your own Connect cluster.
    →     03
    errors.tolerance=all Requires a DLQ
    If a connector sets errors.tolerance=all, it must also configure a dead-letter-queue topic.
    →     04
    No Inline Credentials in Connector Config
    Connector spec.config must not embed raw passwords, JAAS strings, or AWS keys. Only ${secret:...} or ${vault:...} references.
    →     05
    JDBC Connector Host Allowlist
    JDBC source/sink connectors may only target hosts under an approved internal domain. No public DBs, no shadow-IT endpoints.
    →     06
    Connector Topics Must Match Owning Tenant
    Connector's spec.config["topics"] / ["topics.regex"] must be confined to the connector's own tenant prefix.
    →

Enforce, don't just publish.

Conduktor Console turns these YAMLs into pre-commit guardrails with audit history and ApplicationInstance bindings. No more Slack tickets relayed to Kafka admins to approve a topic config.

See Conduktor Console →