Producer Must Compress with lz4 or zstd (Gateway)

Every produce request must arrive compressed with LZ4 or ZSTD. Enforced at the Gateway.

Rationale

Compression has to happen at the producer (broker recompression is a CPU tax — see topic-compression-passthrough), but nothing in Kafka itself forces a producer to set `compression.type` correctly. The Gateway ProducePolicyPlugin inspects the compression codec advertised in the produce request batch header and rejects NONE / GZIP / SNAPPY. LZ4 and ZSTD give the best throughput/CPU tradeoff on modern Kafka; GZIP is slow and CPU-heavy, SNAPPY is acceptable but consistently beaten by LZ4 in benchmarks.

Pattern

produce.compressions in {LZ4, ZSTD}

Examples

Parameters

Governs

This policy relates to the following Kafka configuration keys:

• compression.type

Implementation

Drop this YAML into Conduktor Console as a ResourcePolicy, then link it from an ApplicationInstance, Application, or KafkaCluster.

# Conduktor Gateway Interceptor — wire-layer enforcement.
# Plugin: io.conduktor.gateway.interceptor.safeguard.ProducePolicyPlugin
# Docs:   https://docs.conduktor.io/guide/use-cases/apply-traffic-control-policies
# Pairs with topic-compression-passthrough (compression.type=producer on the
# broker side) — broker stores bytes verbatim, Gateway forces the codec choice.
---
apiVersion: gateway/v2
kind: Interceptor
metadata:
  name: producer-compression-required
  scope:
    vCluster: passthrough
spec:
  pluginClass: io.conduktor.gateway.interceptor.safeguard.ProducePolicyPlugin
  priority: 100
  config:
    topic: ".*"
    compressions:
      value:
        - LZ4
        - ZSTD
      action: BLOCK

Related policies

• Topic Compression Must Be producer (No Broker Recompression)