Skip to content

Production checklist

Before onboarding a shared workload:

  • Pin the Kollect chart and image version, and review release notes before upgrades.
  • Restrict watched namespaces and allowed GVKs with least-privilege RBAC and KollectScope.
  • Keep sink credentials in Secrets; prefer workload identity or short-lived tokens.
  • Configure TLS trust explicitly and keep connection tests enabled during rollout.
  • Select a sink for the consumer: Git for history, a database for queries, or an event sink for reactions.
  • Partition inventories so payloads and reconcile work remain bounded.
  • Set CPU and memory requests, disruption protection, and leader election for the environment.
  • Alert on Degraded, export failures, workqueue backlog, and stale successful export timestamps.
  • Test restore and credential rotation for every durable sink.
  • Verify the NetworkPolicy allows only the API server, DNS, and intended sink endpoints.

See security architecture, performance, and metrics.

Platform-oriented guidance for designing Kollect scope, sinks, and multi-cluster topology. For install defaults, see Operator manual.

Assumptions

Read Understand the basics and Platform decisions before changing production values. Sink role taxonomy: ADR-0401.

Pre-1.0 API

Pre-1.0 API: Kollect is a public alpha with a v1alpha1 API. Minor releases may change the API, defaults and export layout; every breaking change is in the upgrade guide and the release notes. Only the latest minor is supported. See Stability and support.

Scope design

Install one operator per tenant boundary with namespaced RBAC and a restricted informer cache (ADR-0203):

# kollect-doc: ignore Helm values, not a kollect CR
tenantMode: true
watchNamespaces:
  - team-a
featureGates:
  inventoryHttp:
    enabled: false

Place KollectProfile, family sinks (KollectSnapshotSink / KollectDatabaseSink / KollectEventSink), KollectTarget, and KollectInventory in the team namespace. Portal read paths should use Postgres or Kafka sink export — not the optional HTTP inventory API in production.

Same-namespace references

Namespaced pipeline objects must reference peers in the same namespace:

Field Must resolve in
KollectTarget.spec.profileRef Target namespace
KollectInventory.spec.*SinkRefs Inventory namespace
KollectConnectionTest.spec.sinkRef Test namespace

Cluster-wide rollup uses KollectClusterInventory with spec.sinkNamespace instead.

Watch labels and KollectScope

  • Exclude platform namespaces with Target excludedNamespaces or Scope deniedNamespaces — no Namespace patch RBAC (ADR-0207).
  • Trivy HIGH-only collection: resourceRules with label match OR CEL matchPolicy — see examples/kollecttarget_trivy-high.yaml.
  • Run platform targets with watchMode: All; let teams opt out noisy namespaces via kollect.dev/namespace-watch: disabled (ANNOTATIONS-LABELS.md).
  • Use watchMode: OptIn only in shared clusters where most tenants should be ignored by default.
  • Enforce policy with KollectScope — violations set Degraded=True and block export (hard degrade, not silent skip).

Scope vs Helm watchNamespaces

Helm watchNamespaces limits what the operator informer cache sees. KollectScope limits what a tenant may configure. Use both for defense in depth.

Sink roles (ADR-0401)

Classify sinks by role, not vendor. The in-memory snapshot per KollectInventory is the canonical artifact; every sink is a projection of it.

Role Backends Answers Deletes
Snapshot store Git/GitLab (yaml/json/ndjson), S3/GCS (json, parquet, csv) Current state, written whole each cycle Free — absent from snapshot = deleted
Relational SoR Postgres Queryable current state, SQL joins for portals Requires delete reconciliation
Event emitter NATS JetStream (lean default), Kafka/Redpanda Change stream for downstream integration Tombstone (consumer-owned)

Choosing a sink

Need Prefer
Audit trail, GitOps-friendly history Git snapshot
Queryable inventory without running a DB S3/GCS Parquet snapshot
Rich relational portal, BI joins Postgres SoR
Event-driven integrations, fan-out NATS or Kafka emitter

Not a sink type

Prometheus metrics come from the operator /metrics endpoint only — not a family sink type (there is no prometheus sink kind; see ADR-0601).

Postgres and event emitters

  • Postgres must delete rows (or tombstone) for resources no longer in the snapshot — upsert-only drifts stale (ADR-0401).
  • Set spec.cluster on sinks in multi-cluster installs so the backend primary key merges rows across clusters.
  • Tune per-sink exportMinInterval on structured *SinkRefs[] entries before adding more backends — portal Postgres at 30s + Git audit at 1h is the default sample (ADR-0413). See Performance tuning.

Connection testing

Keep spec.connectionTest: false in Git-managed manifests. Probe on demand with the kollect.dev/test-connection annotation or a KollectConnectionTest CR (Connection test example).

Multi-cluster fleet (shared sink)

Default multi-cluster path: each cluster runs the same single-mode operator and exports to a shared sink (Postgres, Git, Kafka, NATS) with spec.cluster set (or {cluster} in pathTemplate). The backend primary key merges rows across clusters — no aggregation tier inside the operator (ADR-0501, ADR-0401).

flowchart LR
  subgraph clusters [Per-cluster operators]
    S1[Kollect cluster A]
    S2[Kollect cluster B]
  end
  subgraph backend [Shared backend]
    PG[(Snapshot · Database · Event sinks)]
  end
  S1 -->|export + cluster id| PG
  S2 -->|export + cluster id| PG

Walkthrough: Multi-cluster fleet.

Operational checklist

Area Practice
Upgrades Two-step: kubectl apply -f dist/install-crds.yaml, then helm upgrade — never delete CRDs
Secrets Credentials in Kubernetes Secrets only; restrict operator SA Secret access
HTTP API Keep featureGates.inventoryHttp.enabled: false unless debugging
Observability Watch ConnectionVerified, SinkReachable, Synced; alert on Degraded=True
Performance Restrict watchNamespaces, narrow target selectors, raise exportMinInterval under load