Let TerdutServer customize its pod, and never manage its own ingress

spec.pod (api/v1alpha1/terdutserver_types.go): annotations, nodeSelector,
tolerations, affinity, topologySpreadConstraints, resources, pod and
container securityContext, serviceAccountName, extraEnv/extraEnvFrom,
extraVolumes/extraVolumeMounts, imagePullSecrets, and an optional
disruptionBudget. All direct corev1 passthrough -- no wrapper types buy
anything for any of these, matching how CloudNativePG and the Zalando
postgres-operator both expose the same knobs, and matching this repo's
own SweeperSpec precedent ("wrap only when a round-trip through a
different type buys something"). affinity is pure user-supplied
passthrough, not a toggle-plus-generated-default the way a multi-replica
cluster operator's pod anti-affinity usually is: this operator never
auto-generates one, since spec.replicas above 1 isn't a supported
topology (the sweeper/notifier singleton constraint). Considered and
declined for this round: priorityClassName, pod labels beyond
annotations, and a HorizontalPodAutoscaler -- the last of those would
directly contradict the singleton constraint above.

disruptionBudget is the one field here that isn't a plain PodTemplateSpec
knob: when set, the controller now reconciles a PodDisruptionBudget
selecting the TerdutServer's own pods (new terdutserver_pdb.go); clearing
it deletes any it previously created. New RBAC marker on
poddisruptionbudgets to match.

Driven by a public-release pass: looking past this project's own use case
at what a mature, general-purpose operator CRD exposes here (researched
against Zalando postgres-operator and CloudNativePG specifically), not
just the fields this install happened to need.

Separately, and found while answering a question about exposing
TerdutServer through Istio instead of Gateway API: spec.networking's own
doc comment quietly promised a Gateway API HTTPRoute this operator would
build eventually ("a near-term follow-up, not deferred"). That promise is
wrong for a public release -- an operator managing someone's ingress
mechanism for them is a worse default than not touching it at all, and a
surprise HTTPRoute appearing once that follow-up eventually landed would
have been exactly backwards for an Istio (or plain-Ingress, or
intentionally-unexposed) install. Made the non-goal explicit and
permanent instead (DESIGN.md §1), removed the dead `gatewayListener`
field it was the only consumer of (zero runtime call sites anywhere --
setting it already had no effect, so this is a schema cleanup, not a
behavior change), and corrected ROADMAP.md's framing. hostname/servicePort
stay: both are live (TERDUT_PUBLIC_URL, container/Service port), this
operator just never acts on hostname for exposure. Added
examples/networking (Gateway API HTTPRoute, Istio VirtualService) showing
how to expose the plain ClusterIP Service the operator already creates --
outside the operator itself, as illustrations, not as something
examples/demo applies automatically.

No new terdut-server version requirement: both changes are CRD/controller-
only, nothing about the API this operator's bootstrap flow depends on
changed.
This commit is contained in:
Niklas Ye
2026-10-02 18:50:38 +02:00
parent d9315322fc
commit 6a699d4341
20 changed files with 8631 additions and 96 deletions
+60 -8
View File
@@ -50,6 +50,14 @@ it just created, never a server something else already bootstrapped first.
that needs a live look at another object (e.g. "does this teamRef exist")
is a status condition, not an admission rejection — keeps v1 to a
controller-only deployment with no cert-manager/webhook dependency.
- **Never manages external exposure/ingress for `TerdutServer`, in any
form — a permanent non-goal, not a staged one.** The operator creates a
plain `ClusterIP` Service (§4.1) and stops there: no `Ingress`, no
Gateway API `HTTPRoute`, no Istio `VirtualService`, nothing. Some
installs won't expose `TerdutServer` outside the cluster at all; others
will use whichever of those mechanisms already fits their cluster. That
choice belongs to whoever deploys it, not to this operator — see
`examples/networking` for worked (but not operator-managed) examples.
## 2. The README's open questions, resolved
@@ -149,7 +157,6 @@ spec:
networking:
hostname: terdut.example.com
servicePort: 8080
gatewayListener: "" # same semantics as chart's networking.listener
database:
dsn: "postgres://terdut@terdut-postgres:5432/terdut?sslmode=require" # mutually exclusive with postgresClusterRef
passwordSecretRef: {name: terdut.terdut-postgres.credentials.postgresql.acid.zalan.do, key: password}
@@ -188,6 +195,20 @@ spec:
# selector: # required, and only meaningful, when from: Selector
# matchLabels:
# terdut.ryuvia.com/allowed: "true"
# Pod-level customization of the Deployment -- all optional, direct corev1
# passthrough throughout (see PodSpec's own doc comment). A representative
# subset:
pod:
resources:
requests: {cpu: 100m, memory: 128Mi}
limits: {memory: 256Mi}
tolerations:
- key: dedicated
operator: Equal
value: terdut
effect: NoSchedule
disruptionBudget:
minAvailable: 1 # mutually exclusive with maxUnavailable
status:
conditions: [...] # Ready, DatabaseReady, Bootstrapped
observedGeneration: 3
@@ -214,6 +235,25 @@ per-name allowlist (no "and only these teams") — namespace-level consent is
the right granularity here, same reasoning as `ListenerSet`: the namespace
is the tenancy boundary, not the object.
`spec.pod` is pod-level customization of the Deployment, all optional and
directly reusing corev1 types wherever corev1 already models the knob
exactly (`tolerations`, `affinity`, `topologySpreadConstraints`,
`resources`, `securityContext`/`containerSecurityContext`, `extraEnv`/
`extraEnvFrom`, `extraVolumes`/`extraVolumeMounts`, `imagePullSecrets`) —
no custom wrapper buys anything for any of these, matching how
CloudNativePG and the Zalando postgres-operator both expose the same
knobs. `affinity` is pure user-supplied passthrough, not a
toggle-plus-generated-default the way a multi-replica-aware operator's
pod anti-affinity typically is: this operator never auto-generates
affinity of its own, since `replicas` above 1 isn't a supported topology
(the sweeper/notifier singleton constraint, §4.1's own illustrative YAML
comment). `spec.pod.disruptionBudget` is the one field here that isn't a
straight PodTemplateSpec knob — when set, the controller reconciles a
`PodDisruptionBudget` selecting this `TerdutServer`'s pods; clearing it
deletes any it previously created (§7). `minAvailable`/`maxUnavailable`
are mutually exclusive, `+kubebuilder:validation:XValidation`-guarded the
same way as `spec.database`'s own `dsn`/`postgresClusterRef` rule.
### 4.2 `TerdutTeam`
```yaml
@@ -591,9 +631,15 @@ when nothing ever crosses into a tenant namespace in the first place.
## 7. Ownership, status, garbage collection
- Every generated object that lives in the *same* namespace as the CR that
caused it (Deployment, Service, webhook Secret) carries a standard
`metav1.OwnerReference` — GC handles these, no finalizer needed. The two
credential Secrets from §6 are the one exception: they live in the
caused it (Deployment, Service, webhook Secret, and `TerdutServer`'s own
PodDisruptionBudget) carries a standard `metav1.OwnerReference` — GC
handles these, no finalizer needed. PodDisruptionBudget is the one
member of that list that's conditionally created/deleted rather than
always present: it exists only while `spec.pod.disruptionBudget` is set,
and the controller deletes it itself the moment that field is cleared
(it doesn't wait on GC for that case, only for the `TerdutServer` being
deleted outright). The two credential Secrets from §6 are the one
exception to OwnerReference-based cleanup generally: they live in the
operator's own namespace regardless of where their owning CR lives, so
`OwnerReference` doesn't apply (cross-namespace) and cleanup instead runs
through that CR's finalizer directly, alongside the server-side DELETE
@@ -640,10 +686,10 @@ documented and tested operationally:
- The operator's own ServiceAccount needs, per namespace it's granted:
`get/list/watch/create/update/patch/delete` on `Deployments`, `Services`
it owns, and `get/list/watch` on `postgresql.acid.zalan.do` (optional,
degrade gracefully if absent per §8), plus cluster-wide `get/list` on
`Namespace` (labels only, for `allowedTeams: {from: Selector}` evaluation
— §4.1, §4.6).
and `PodDisruptionBudgets` it owns, and `get/list/watch` on
`postgresql.acid.zalan.do` (optional, degrade gracefully if absent per
§8), plus cluster-wide `get/list` on `Namespace` (labels only, for
`allowedTeams: {from: Selector}` evaluation — §4.1, §4.6).
- **Two different `Secret` scopes, not one — corrected from an earlier draft
of this section.** That earlier draft said `Secret` access was "scoped to
the operator's own namespace only... nowhere else," reasoning that with
@@ -784,6 +830,12 @@ what it was, a separate install, until someone deletes it.
a one-off exercise.
- Gitops-managed team *membership* (see §4.2).
- Automatic Deployment restart on upstream Postgres credential rotation.
- `spec.pod.priorityClassName`, pod-label passthrough beyond
`spec.pod.annotations`, and a HorizontalPodAutoscaler for `TerdutServer`
— all considered alongside §4.1's `spec.pod` and explicitly left out of
that round: an HPA in particular would actively contradict
`spec.replicas`'s own stance that this operator doesn't support more
than one replica (the sweeper/notifier singleton constraint).
- Admission webhooks / CEL-only validation limits (e.g. verifying a
`teamRef` exists at admission time rather than surfacing it as a status
condition after the fact).