Topic Compression Must Be producer (No Broker Recompression)

compression.type on the topic must be `producer` (or unset). Real compression is enforced at the producer.

Rationale

Setting `compression.type` to a concrete codec (lz4, zstd, gzip, snappy) at the topic level tells the broker to *re-compress* every batch after producer-side compression — paying CPU twice, increasing append latency, and breaking idempotence guarantees on retries. The Kafka-idiomatic value is `producer`: the broker stores producer-compressed batches byte-for-byte. Compression as a *policy* must be enforced where compression actually happens — in the producer — via the `producer-compression-required` Gateway interceptor.

Pattern

!(compression.type in spec.configs) OR compression.type == "producer"

Examples

Parameters

Governs

This policy relates to the following Kafka configuration keys:

• compression.type

Implementation

Drop this YAML into Conduktor Console as a ResourcePolicy, then link it from an ApplicationInstance, Application, or KafkaCluster.

# Conduktor self-service ResourcePolicy
# Schema: https://docs.conduktor.io/platform/reference/resource-reference/self-service/#resourcepolicy
# Paired with producer-compression-required (Gateway interceptor) which forces
# the producer to actually compress with lz4/zstd before bytes hit the broker.
---
apiVersion: self-serve/v1
kind: ResourcePolicy
metadata:
  name: topic-compression-passthrough
spec:
  targetKind: Topic
  description: compression.type on the topic must be "producer" (or unset). Compression is enforced producer-side via Gateway.
  rules:
    - condition: '!has(spec.configs) || !("compression.type" in spec.configs) || spec.configs["compression.type"] == "producer"'
      errorMessage: "compression.type must be \"producer\" (or unset) — broker recompression wastes CPU. Enforce lz4/zstd at the producer via the Gateway producer-compression-required interceptor."

Related policies

• Producer Must Compress with lz4 or zstd (Gateway)