segment.bytes Within Sane Operational Bounds

segment.bytes must be between 64 MiB and 2 GiB.

Rationale

Tiny segments (some teams set 1MB "for faster deletion") explode open-file count and cripple controller failover. Huge segments (>2GB) delay retention by hours because nothing rolls. Both are common foot-guns from copy-pasted blog configs.

Pattern

67108864 (64 MiB) <= segment.bytes <= 2147483648 (2 GiB)

Examples

Parameters

Governs

This policy relates to the following Kafka configuration keys:

• segment.bytes
• segment.ms

Implementation

Drop this YAML into Conduktor Console as a ResourcePolicy, then link it from an ApplicationInstance, Application, or KafkaCluster.

# Conduktor self-service ResourcePolicy
# Topics that don't override segment.bytes inherit broker default and pass vacuously.
---
apiVersion: self-serve/v1
kind: ResourcePolicy
metadata:
  name: segment-bytes-sane-bound
spec:
  targetKind: Topic
  description: segment.bytes must be in [64 MiB, 2 GiB]
  rules:
    - condition: '!("segment.bytes" in spec.configs) || (int(string(spec.configs["segment.bytes"])) >= 67108864 && int(string(spec.configs["segment.bytes"])) <= 2147483648)'
      errorMessage: "segment.bytes must be between 64MiB and 2GiB — tiny segments blow up file count, huge segments delay retention"

Related policies

• Maximum Retention Bound
• retention.bytes Required Alongside retention.ms
• Audit Retention Floor (No Infinite, 7y for Audit-Tagged)
• Retention Capped by Tenant Tier