Skip to content

ADR-0407: Git and object-store export layout and workflow

Where exported inventory lands in a repo or bucket, and how the Git/GitLab sinks commit, branch, and open merge requests.

Theme: 04 · Export & sinks · Status: Current (see the 2026-09-03 note below)

Context

The Git, GitLab, S3, and GCS backends are snapshot stores (ADR-0401): each export writes the whole inventory document to a path. The path layout is a contract (consumers and GitOps tooling depend on it) and the Git write workflow (commit vs branch vs MR) is an ergonomics decision. Both were implemented (internal/sink/git, internal/sink/gitlab) without an ADR.

Decision

Object path layout

The inventory controller passes a canonical object path; backends map it to their store:

inventory/<inventory-namespace>/<inventory-name>.json
  • Git/GitLab/S3/GCS write the JSON payload (ADR-0405) at that path.
  • Multi-cluster fan-in disambiguates by cluster in the path/key (clusters/<cluster>/… or a cluster column/header — ADR-0501), so concurrent spokes never collide on the same object.

Git write workflow

internal/sink/git/export_file.go clones a single branch into a temp dir, writes the file, and:

  • Commit identity defaults to kollect <kollect@kollect.dev>; message templates and rich defaults are defined in ADR-0415.
  • No-op guard: if git add produces no change, the export errors rather than pushing an empty commit (idempotent exports stay quiet — ADR-0406).
  • Push policy : spec.git.pushPolicy defaults to Commit (append-only pushes). Optional ForcePush restores snapshot semantics (push --force) for demo repos.
  • Empty-remote bootstrap: a fresh/empty repo is git init'd and the branch created, so the first export works against a bare repository.
  • Custom CA: trusted via resolved caPEM from BuildContext (ADR-0104); no insecureSkipVerify for Git. Amended 2026-09-03: that second clause was wrong — the Git sink does honour spec.tls.insecureSkipVerify, and for an ssh:// remote it disables host-key verification as well. See insecureSkipVerify does apply to Git — corrected 2026-09-03 below.

insecureSkipVerify does apply to Git — corrected 2026-09-03

no insecureSkipVerify for Git recorded an intent, not the implementation. The Git sink reads spec.tls.insecureSkipVerify through git.TLSConfigFromSpec and acts on it in three places:

Path Effect when the flag is set
internal/sink/git/export.go → effectiveSSHConfig → sshAuthMethod go-git installs ssh.InsecureIgnoreHostKey for an ssh:// remote
internal/sink/git/cli_env.go → effectiveSSHConfig → buildGitSSHCommand the git-CLI machinery (the git ls-remote probe) gets GIT_SSH_COMMAND=... -o StrictHostKeyChecking=no (emitted at two points: keyless and keyed). GIT_SSL_NO_VERIFY=true is also added unconditionally — it is what disables HTTP(S) certificate checking and is simply inert for ssh://
internal/sink/git/connection.go (lsRemoteUncached) reaches the same newCLIEnv at :143, so the connection test skips host-key verification too. Its own GIT_SSL_NO_VERIFY=true re-append is redundant — applyCLIEnv set it one line earlier

internal/sink/git/mirror.go, sync_remote.go and the fetch/push options in export.go also pass it to go-git as InsecureSkipTLS, which is TLS-only.

The correct rule is therefore: Git prefers a trusted CA and known_hosts; insecureSkipVerify is an explicit, default-off development escape hatch that disables verification for whichever transport the endpoint selects. Since 2026-10-01 (K-14) the flag also requires the manager to run with --allow-insecure-sinks, and the TLSInsecure status condition is set whenever it is in effect, whatever the connection test does (it was previously written only when a connection test ran and succeeded). Git remotes also no longer follow HTTP redirects (http.followRedirects=false), so spec.endpoint must be the final URL. The endpoint is a single URL and its scheme picks the transport, so the flag never covers HTTPS and SSH at once. Guard rails, their conditions, and why the escape hatch is one flag rather than a separate SSH field are recorded in ADR-0104; internal/sink/git/insecure_hostkey_contract_test.go pins the behaviour.

GitLab merge-request mode

internal/sink/gitlab adds a review workflow on top of the Git backend:

  • Direct mode: commit straight to the target branch (same as Git).
  • Branch+MR mode (MergeRequestModeBranchMR): push to a per-inventory feature branch (BranchNameForExport(prefix, kind, ns, name): prefix/<ns>/<name>, or prefix/_cluster/<name> for a KollectClusterInventory) off the target branch, then EnsureMergeRequest opens or updates one MR per inventory via the GitLab API. Before writing, the export is also checked against the target branch's ownership records, so a feature branch never claims a path the target already gives another inventory (ADR-0422). One stable branch/MR per inventory keeps churn reviewable instead of spawning an MR per export.

Consequences

  • Predictable, diffable layout for GitOps and audit (ADR-0103).
  • Force-push means Git holds current state with audit trail, not a queryable history — relational history is Postgres' job (ADR-0401).
  • One branch/MR per inventory bounds review noise but means concurrent exports to the same inventory serialize on the remote.

Open questions

  • DECIDED : Make the object path a spec.pathTemplate (e.g. {cluster}/{namespace}/{name}.json, default inventory/{namespace}/{name}.json) so layout is configurable per sink.
  • DECIDED : spec.git adds pushPolicy, branch, auth, commitMessage, commitBody, commitTrailers, author, cloneDepth, and prune on snapshot sinks — see ADR-0415.
  • RESOLVED: S3 and GCS both support serialization.format: parquet; JSON remains the default.