# terdut-operator build roadmap This is the staging plan for implementing the operator against `DESIGN.md`'s settled decisions. It exists for the same reason `DESIGN.md` and `SERVICE-ACCOUNTS.md` do: so each stage starts from an agreed sequencing instead of re-litigating "what do we build first" mid-PR. ## Sequencing call this roadmap makes **The operator creates and owns every `TerdutServer` it manages — it never adopts one deployed independently, by hand or by `charts/terdut-server`** (`DESIGN.md` §1). An earlier version of this roadmap staged `TerdutServer`'s Deployment/Service/bootstrap takeover separately (old Stage 5), behind a hand-deployed server the simpler CRDs could be proven against first. That staging existed only because a credential-less operator couldn't `/api/bootstrap` its way into a server something else had already bootstrapped (`DESIGN.md` §6's original gap). With no server to adopt at all, that split has nothing left to justify it: `TerdutServer` now builds its full lifecycle — Deployment, Service, database wiring, bootstrap, credentials — in one stage, Stage 1, since bootstrap only has something to bootstrap once the Deployment exists. ## Stage 0 — Scaffolding & CI - `go.mod` (`git.ryuvia.com/niklas/terdut-operator`) + Kubebuilder v4 scaffold (`cmd/main.go`, `config/`, `Makefile`, `PROJECT`), matching terdut-server's Go toolchain and house style (§3). Kubebuilder's own scaffolded `Makefile` already wires `manifests`/`generate` (`controller-gen`) and `setup-envtest` into `test`, and `golangci-lint` into `lint`, all fetched on demand into `bin/` — no separate install step needed beyond what `make test`/`make lint` already do. - `.release.conf` deliberately **not** added yet: it names a `HELM_CHART` this repo doesn't have until Stage 5. Adding it now would either be a stub that lies about what's releasable or dead config nobody can run — it lands in Stage 5, alongside the chart it describes. - Gitea Actions CI calling `fmt lint test`, mirroring terdut-server's `ci.yaml` convention (its `CLAUDE.md`: "a green gate here and a green pipeline are the same code, not two descriptions of it") minus the `chart`/`security` jobs, which need a chart (Stage 5) and real controller code (Stage 1+) respectively to have anything to check. - Drop kubebuilder's default `.github/workflows/*` scaffold — this org runs on Gitea, not GitHub; `.gitea/workflows/ci.yaml` is the only CI this repo has. - Housekeeping: drop the stray `.DESIGN.md.swp` (leftover vim swapfile, shouldn't be committed); correct `DESIGN.md` §6/§13's "v1-blocking, not v1-shippable" language — the service-account feature it was blocking on has since shipped in terdut-server. **Done when:** CI is green on an otherwise-empty scaffold. ## Stage 1 — `TerdutServer`, full lifecycle Supersedes the Stage 1 shipped before this redesign (commit `1be7cf2`) outright — that `TerdutServerSpec`/`Status`/controller/tests implemented the now-removed bring-your-own path and get replaced wholesale, not extended. New commits build forward over the old ones; no git history rewrite. - Full §4.1 spec: `image`, `replicas`, `networking`, `database`, `sweeper`, `deadman`, `notify`, `oidc`, `passwordLogin`, `allowedTeams`, all together — no narrowing, since bootstrap needs the Deployment it's narrowed away from in the version this replaces. - Controller manages the Deployment + Service, both Postgres paths from §8 at once (bring-your-own DSN *and* the Zalando `postgres-operator` `postgresClusterRef` integration — not sequenced, per the user's call), and bootstrap/credentials per §6's self-registration flow: `/api/bootstrap` once the Deployment has a ready replica, checkpoint the admin key, mint the instance-scoped service account, generated credentials Secret in the operator's own namespace, `status.credentialsSecretRef`. Finalizer cleans up that Secret (and the checkpoint, if one's still there) on delete — there's no server-side row to clean up alongside it: terdut-server's API has no way to delete a user or a service account, only to revoke individual keys, so there's nothing to undo there regardless. - RBAC: read-only watch on `postgresql.acid.zalan.do`, degrading gracefully if that CRD isn't installed (§8, §9). - Shipped, scoped down from §8's full ambition in two ways, both called out in code rather than silently dropped: no live watch on the Zalando- generated credentials Secret for rotation (relies on the periodic resync to notice eventually, higher latency than a watch); no Gateway API `HTTPRoute` creation from `spec.networking.hostname`/`gatewayListener` (needs the Gateway API types as a new dependency, and nothing about proving a `TerdutServer` boots and bootstraps a real server depends on external ingress existing). Both are near-term follow-ups, not deferred to a later stage. - `envtest` covering Deployment/Service reconciliation and both database paths — the Zalando path needs that CRD's schema vendored into the test environment (there's no real `postgres-operator` controller in `envtest`, only the CRD shape to create fixture objects against) — plus the self-registration flow against an `httptest.Server` fake of `/api/bootstrap` and `/api/service-accounts` (§11), including the adopt-on-409 recovery path and the one fail-closed case (`BootstrapStateLost`), not just the happy path. - **Done, 2026-10-01**: a real `kind` end-to-end pass (bring-your-own DSN, real terdut-server `v0.33.0` image, operator built into a real image and deployed as a real Pod, not `go run` against the cluster). `TerdutServer` went `Ready`; the generated credential authenticated and exercised its real capability against the actual server (`GET`/`POST /api/teams` → `200`/`201`, confirmed from terdut-server's own access log). Caught two real bugs no `envtest` suite could have (its client bypasses RBAC): `.dockerignore`'s `!**/*.go` not working under podman, and missing RBAC for `events.k8s.io` (the new events API `GetEventRecorder` uses) — both fixed. The Zalando path can additionally be validated for real against the org's own cluster later, where `postgres-operator` already runs, rather than only in a disposable `kind` stand-in — not done in this pass. ## Stage 2 — `TerdutTeam` - §4.2: `serverRef` resolution, real update-in-place (POST create / PUT rename / PUT oidc-groups), team-scoped service-account minting once `Ready` (§6 point 3), finalizer that DELETEs the team server-side and its credential Secret. - First place the "every child resolves its own `teamRef` → `TerdutTeam.status`, never chains up to `TerdutServer`" pattern (§5) gets proven end to end. ## Stage 3 — `TerdutEscalationRule` + `TerdutDeadmanSwitch` - Built together: both stay same-namespace-as-their-`TerdutTeam` (§1), so neither exercises cross-namespace complexity, but together they cover the two different reconciliation shapes §5's table calls out — whole-policy PUT-upsert for the escalation policy (no separate create step at all), real create/update-in-place/delete for the dead man's switch (PUT added in terdut-server `v0.33.0` specifically for this operator) — against the same shared create/finalizer/resync scaffolding Stage 2 already built. - New shared `resolveTeamAndClient` helper (`childref.go`) implements §5's "every child resolves its own `teamRef` → `TerdutTeam.status`, never chains up to `TerdutServer`" rule once, for both controllers — `TerdutTeam.status.serverEndpoint`, added in this stage, is what makes that literally true rather than just a stated intent. - `TerdutEscalationRule` resolves each "user" target's username to a user_id via `GET /api/users` (confirmed open to any authenticated caller) and reports `Ready: False, reason: UnknownUser` if it doesn't resolve. No `DELETE` exists for this resource, so its delete path `PUT`s an empty policy as the closest available undo. - `TerdutDeadmanSwitch` has no unique-name constraint server-side, so its idempotent-create is `GET`-list-and-match-by-name rather than adopt-on-409 (unlike every other resource in this operator). - **Done, 2026-10-01**: `envtest` coverage for both controllers' happy path, `TeamRefNotFound`/`WaitingForTeam`, `UnknownUser`, list-and-match adoption, update-in-place on spec drift, and deletion. `make fmt lint test build` all clean; `internal/controller` envtest coverage 50.5% → 71.7%. No `kind` e2e pass for this stage — Stage 1's already proved the real-cluster mechanics (RBAC, image, bootstrap) these two controllers reuse unchanged, and neither introduces a new mechanism that pass would exercise differently (same reasoning Stage 2 used to skip one). ## Stage 4 — `TerdutAlertSource` - Last of the children on purpose: it has the subtlest failure mode of the four. Covers webhook Secret generation/ownership (§4.5, §7), the `WebhookSecretLost` fail-closed condition + `Warning` event (§5, added 2026-09-30), and the kind-change delete-and-recreate rotation path — all easier to get right with the other three controllers' patterns already in place to build on. - Idempotent-create here is neither adopt-on-409 (Team/service-account) nor list-and-match-by-name (`TerdutDeadmanSwitch`): terdut-server shows the webhook key exactly once, at creation, and never again, so no server-side lookup could ever recover it after a crash. The generated webhook Secret itself — written immediately after the POST, before `status` is ever touched — is this CR's only durable record that a create already succeeded; found on a later reconcile with `status.integrationID` still unset, it's read back directly rather than POSTing again. Found missing with `status.integrationID` *set*, that's the already-designed `WebhookSecretLost` fail-closed case instead. - **Done, 2026-10-01**: `envtest` coverage for the happy path, rename (PATCH, no key rotation), a kind change (delete-and-recreate, new id and key), crash recovery between POST and the Secret write, `WebhookSecretLost`, `TeamRefNotFound`/`WaitingForTeam`, and deletion. `make fmt lint test build` all clean; `internal/controller` envtest coverage holds at 71.6%. No `kind` e2e pass for this stage, same reasoning as Stage 3 (reuses Stage 1's already-proven real-cluster mechanics unchanged). ## Stage 5 — Installer chart + real release - Package CRDs + the operator's own Deployment/RBAC into the installer chart §10 describes; wire `.release.conf`/release-vars the same way terdut-server does; run it through the `release` skill for a real first cut. - Full `kind` end-to-end test per §11: create `TerdutServer` → `TerdutTeam` → one of each child kind → verify via terdut-server's own API that each object exists with the right shape → delete the CR → verify the server-side object is gone. - Chart built via kubebuilder's own `helm/v2-alpha` plugin from `config/`'s kustomize output (`charts/terdut-operator`), not hand-rolled -- CRDs + manager Deployment/RBAC come from the same markers/manifests every other stage already generates, so there's exactly one source of truth for them. Hand-added on top: the optional `terdutServer` values block (§10's "helm install and get a server" path), `.release.conf`, and the `release-vars`/`helm-lint`/`push`/`helm-package`/`helm-push`/`release` Makefile targets `.gitea/workflows/release.yaml` calls, mirroring terdut-server's own shape end to end (same registry/namespace convention, same multi-arch buildx push, same trivy/govulncheck/gitleaks scans). Also fixed while wiring this: the Dockerfile's builder stage didn't pin `--platform=$BUILDPLATFORM`, which would have made a multi-arch release build fail outright on this org's runners (no binfmt registration) -- caught before it ever shipped, not discovered mid-release; and govulncheck surfaced one real, reachable finding (`google.golang.org/grpc` v1.82.1, transitive via controller-runtime's otel exporter), fixed by bumping to v1.83.1. - **Done, 2026-10-01**: the full golden-path pass above, run for real against a `kind` cluster, installed via `helm install` (not raw kustomize/kubectl apply -- the first time the chart itself, not just `config/`, was exercised): `TerdutServer` (real terdut-server `v0.33.0` image, bring-your-own DSN against a throwaway in-cluster Postgres) → `TerdutTeam` → one `TerdutEscalationRule` + `TerdutDeadmanSwitch` + `TerdutAlertSource`, each confirmed `Ready` and then confirmed a second way, independent of the operator's own status: a `curl` pod inside the cluster, authenticated with the generated team credential, hit terdut-server's real API directly (`GET /api/teams/{id}/escalation`, `.../deadman/switches`, `.../integrations`) and got back exactly the policy/switch/integration each spec declared. Deleting every CR in reverse order was verified the same way: the escalation policy came back empty (its only available "undo"), the switch and the integration were both gone from their list endpoints, the team no longer resolved by name, and the Deployment/Service/every generated Secret were gone from the cluster. No new bugs found this pass -- Stage 1's own kind e2e pass already caught the two issues (`events.k8s.io` RBAC, the podman `.dockerignore` fix) a real cluster catches and `envtest` can't, and nothing since has touched that surface. - Not done in this pass, deliberately: an actual tagged release. `make release-vars`/`helm-lint`/`push`/`helm-package`/`helm-push` all work locally and `.gitea/workflows/release.yaml` is wired, but `release-preflight` found there is no `terdut-operator/` entry under `Ryuvia/charts` yet to bump -- every other onboarded repo had that one-time wrapper-chart bootstrap done for it before its own first release, and this one doesn't, since deploying this operator for real is a decision for whoever runs the cluster, not a side effect of finishing this stage. Cutting the first real release (and creating that wrapper entry) is therefore the next action, not yet taken. ## Deferred (§13, unchanged by this roadmap) Cross-namespace `allowedTeams` exercised against a real second namespace, CloudNativePG support, narrower-than-namespace Secret RBAC for the webhook Secret, gitops-managed team membership, automatic Deployment restart on upstream Postgres credential rotation, admission webhooks/CEL-only validation limits, OLM packaging.