Stage 1: TerdutServer full lifecycle (Deployment, Service, both database
CI / test (push) Successful in 1m46s
CI / test (push) Successful in 1m46s
paths, self-registration bootstrap)
Replaces the bring-your-own-only Stage 1 (commit 1be7cf2) wholesale, per
the redesign in the previous two commits: the operator creates every
server it manages, so self-registration (DESIGN.md §6) is the only
bootstrap path, and Deployment/Service/database management builds
together with it (ROADMAP.md Stage 1) rather than behind a separate
later stage.
Grounded in terdut-server's actual chart (charts/terdut-server/templates/
deployment.yaml, values.yaml), not reconstructed from DESIGN.md's
illustrative YAML alone -- env var names, the password-via-PGPASSWORD
convention, the Recreate deployment strategy, /healthz probes, and the
TERDUT_OPERATOR_MODE=true decision (always on here, unlike the chart's
default-off: every write this operator's own future controllers make
goes through a service account already) all match that source exactly.
- api/v1alpha1: full TerdutServerSpec (image, replicas, networking,
database, sweeper, deadman, notify, oidc, passwordLogin, allowedTeams).
spec.database is a oneOf (dsn xor postgresClusterRef) via CEL
XValidation. No spec.credentialsSecretRef -- removed entirely in the
prior redesign commit, not carried forward.
- internal/controller:
- terdutserver_deployment.go: Deployment + Service via CreateOrUpdate,
owned (OwnerReference), env built field-for-field against the chart.
- terdutserver_database.go: both §8 paths. The Zalando path resolves
the postgresql.acid.zalan.do CR by convention (database/role both
"terdut", matching every DESIGN.md example) and only ever confirms
its generated credentials Secret exists -- never reads the value,
same "wire a secretKeyRef, don't read it" posture the DSN path takes.
classifyClusterGetError is its own function specifically so the
CRD-not-installed case (meta.IsNoMatchError) is unit-testable without
a real client.
- terdutserver_bootstrap.go: self-registration, checkpointed against
both real crash windows (DESIGN.md §6 point 1) -- an admin-key
checkpoint Secret, and adopt-via-GET+mint-new-key on a 409 from
creating the service account. BootstrapStateLost is its own error
type so Reconcile can route it to a condition instead of an infinite
retry.
- terdutserver_controller.go: ties it together -- finalizer add, DB
resolution, Deployment/Service reconcile, wait for a ready replica,
bootstrap, Ready/Bootstrapped/DatabaseReady conditions. Finalizer on
delete only removes the generated Secrets: terdut-server's API can't
delete a user or service account, only revoke keys, so there's
nothing server-side to undo.
- internal/tdclient: added Bootstrap, CreateInstanceServiceAccount,
GetServiceAccountByName, CreateServiceAccountKey, matching
terdut-server's real handlers' request/response shapes (internal/api/
users.go, service_accounts.go in that repo) field-for-field.
- Tests: envtest suite covering the full DSN-path lifecycle end to end
(finalizer -> Deployment/Service -> simulated readiness -> real
bootstrap against an httptest.Server fake), the adopt-on-409 recovery
path, BootstrapStateLost, both Zalando outcomes (cluster not found;
cluster + Secret found -> real DSN -> Ready), and deletion. A minimal
test-only stub of the Zalando CRD (internal/controller/testdata) lets
envtest create fixture objects without a real postgres-operator
installed. 74.0%/44.7% coverage, 0 lint issues.
- Two things scoped down from §8's full ambition, called out in code and
ROADMAP.md rather than silently dropped: no live watch on the
Zalando-generated Secret for rotation (periodic resync notices
eventually, not immediately), no Gateway API HTTPRoute creation from
spec.networking (would add a new dependency; nothing about proving
bootstrap works depends on external ingress existing). Both are
near-term follow-ups.
Verified locally: make fmt lint test build all clean.
This commit is contained in:
@@ -50,7 +50,11 @@ var _ = BeforeSuite(func() {
|
||||
|
||||
By("bootstrapping test environment")
|
||||
testEnv = &envtest.Environment{
|
||||
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
|
||||
// testdata carries a minimal, test-only stub of Zalando
|
||||
// postgres-operator's CRD (see its own file comment) so
|
||||
// terdutserver_database_test.go can exercise the postgresClusterRef
|
||||
// path without a real postgres-operator installed.
|
||||
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases"), "testdata"},
|
||||
ErrorIfCRDPathMissing: true,
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
package controller
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client"
|
||||
|
||||
terdutv1alpha1 "git.ryuvia.com/niklas/terdut-operator/api/v1alpha1"
|
||||
"git.ryuvia.com/niklas/terdut-operator/internal/tdclient"
|
||||
)
|
||||
|
||||
// credentialsSecretDataKey is the fixed data key every generated
|
||||
// credentials Secret this controller writes uses — "whatever a human chose"
|
||||
// only applied to the bring-your-own input this design removed (DESIGN.md
|
||||
// §6); every Secret this controller itself generates uses this one key.
|
||||
const credentialsSecretDataKey = "token"
|
||||
|
||||
// bootstrapStateLostError is DESIGN.md §6's one genuinely pathological
|
||||
// case: a checkpointed admin credential was used and then lost before the
|
||||
// lasting credential it was for could be persisted. Distinct from a plain
|
||||
// error so Reconcile can route it to a Ready: False condition (the
|
||||
// documented recovery is delete-and-recreate, not an automatic retry) rather
|
||||
// than treating it as a transient reconcile failure.
|
||||
type bootstrapStateLostError struct{ detail string }
|
||||
|
||||
func (e *bootstrapStateLostError) Error() string {
|
||||
return fmt.Sprintf(
|
||||
"server reports already bootstrapped, but neither status.credentialsSecretRef nor a "+
|
||||
"checkpointed admin credential exist here: %s. This TerdutServer cannot recover a "+
|
||||
"credential on its own; delete and recreate it", e.detail)
|
||||
}
|
||||
|
||||
// reconcileBootstrap implements DESIGN.md §6 point 1's self-registration
|
||||
// flow, checkpointed against the two real crash windows in it rather than
|
||||
// leaving them as theoretical gaps. Only called once
|
||||
// srv.Status.CredentialsSecretRef is nil and the Deployment has a ready
|
||||
// replica.
|
||||
func (r *TerdutServerReconciler) reconcileBootstrap(ctx context.Context, srv *terdutv1alpha1.TerdutServer) error {
|
||||
adminKey, err := r.getOrCreateCheckpointedAdminKey(ctx, srv)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
bc := r.NewClient(serviceURL(srv)).WithToken(adminKey)
|
||||
instanceKey, err := r.getOrMintInstanceServiceAccountKey(ctx, bc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
credsName := credentialsSecretName(srv)
|
||||
if err := r.writeSecret(ctx, credsName, instanceKey); err != nil {
|
||||
return err
|
||||
}
|
||||
srv.Status.CredentialsSecretRef = &terdutv1alpha1.SecretKeyRef{Name: credsName, Key: credentialsSecretDataKey}
|
||||
|
||||
// Best-effort: the checkpoint has done its job. Leaving it behind on a
|
||||
// delete failure here isn't a correctness problem (the next reconcile
|
||||
// finds status.CredentialsSecretRef already set and never looks at the
|
||||
// checkpoint again) — it would just be an unused Secret sitting around,
|
||||
// cleaned up for real by the finalizer on delete.
|
||||
checkpoint := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: checkpointSecretName(srv), Namespace: r.OperatorNamespace}}
|
||||
_ = r.Delete(ctx, checkpoint)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// getOrCreateCheckpointedAdminKey returns a usable admin key: from the
|
||||
// checkpoint Secret if an earlier, interrupted attempt already got one, or
|
||||
// freshly from /api/bootstrap, immediately checkpointed before it's used
|
||||
// for anything else.
|
||||
func (r *TerdutServerReconciler) getOrCreateCheckpointedAdminKey(ctx context.Context, srv *terdutv1alpha1.TerdutServer) (string, error) {
|
||||
checkpointName := checkpointSecretName(srv)
|
||||
|
||||
var checkpoint corev1.Secret
|
||||
err := r.Get(ctx, client.ObjectKey{Namespace: r.OperatorNamespace, Name: checkpointName}, &checkpoint)
|
||||
switch {
|
||||
case err == nil:
|
||||
return string(checkpoint.Data[credentialsSecretDataKey]), nil
|
||||
case !apierrors.IsNotFound(err):
|
||||
return "", err
|
||||
}
|
||||
|
||||
bc := r.NewClient(serviceURL(srv))
|
||||
result, err := bc.Bootstrap(ctx, bootstrapUsername, bootstrapEmail)
|
||||
if err != nil {
|
||||
if statusErr, ok := errors.AsType[*tdclient.StatusError](err); ok && statusErr.Code == http.StatusForbidden {
|
||||
// §1: this operator is the only thing that ever bootstraps a
|
||||
// server it created, so a 403 here (no checkpoint, no
|
||||
// status.credentialsSecretRef) means a prior reconcile already
|
||||
// won this exact race and its checkpoint was lost afterward --
|
||||
// the one case §6 doesn't try to paper over.
|
||||
return "", &bootstrapStateLostError{detail: "/api/bootstrap returned 403"}
|
||||
}
|
||||
return "", fmt.Errorf("POST /api/bootstrap: %w", err)
|
||||
}
|
||||
|
||||
if err := r.writeSecret(ctx, checkpointName, result.APIKey.Key); err != nil {
|
||||
return "", fmt.Errorf("checkpointing admin key: %w", err)
|
||||
}
|
||||
return result.APIKey.Key, nil
|
||||
}
|
||||
|
||||
// getOrMintInstanceServiceAccountKey mints the operator's own instance-
|
||||
// scoped service account, or, if an earlier interrupted attempt already
|
||||
// created it (409), adopts it and mints a fresh key rather than treating
|
||||
// the conflict as an error (DESIGN.md §6 point 1, §5's general
|
||||
// adopt-on-conflict rule).
|
||||
func (r *TerdutServerReconciler) getOrMintInstanceServiceAccountKey(ctx context.Context, bc *tdclient.Client) (string, error) {
|
||||
result, err := bc.CreateInstanceServiceAccount(ctx, serviceAccountName)
|
||||
if err == nil {
|
||||
return result.Key.Key, nil
|
||||
}
|
||||
|
||||
statusErr, ok := errors.AsType[*tdclient.StatusError](err)
|
||||
if !ok || statusErr.Code != http.StatusConflict {
|
||||
return "", fmt.Errorf("POST /api/service-accounts: %w", err)
|
||||
}
|
||||
|
||||
sa, err := bc.GetServiceAccountByName(ctx, serviceAccountName)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("GET /api/service-accounts?name=%s (adopting after 409): %w", serviceAccountName, err)
|
||||
}
|
||||
if sa == nil {
|
||||
return "", fmt.Errorf("POST /api/service-accounts 409'd for %q but GET found nothing", serviceAccountName)
|
||||
}
|
||||
key, err := bc.CreateServiceAccountKey(ctx, sa.ID, "initial")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("POST /api/service-accounts/%d/keys (adopting after 409): %w", sa.ID, err)
|
||||
}
|
||||
return key.Key, nil
|
||||
}
|
||||
|
||||
// writeSecret creates or replaces a Secret in the operator's own namespace
|
||||
// holding one raw value under credentialsSecretDataKey.
|
||||
func (r *TerdutServerReconciler) writeSecret(ctx context.Context, name, rawValue string) error {
|
||||
secret := &corev1.Secret{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: r.OperatorNamespace},
|
||||
Data: map[string][]byte{credentialsSecretDataKey: []byte(rawValue)},
|
||||
}
|
||||
if err := r.Create(ctx, secret); err != nil {
|
||||
if apierrors.IsAlreadyExists(err) {
|
||||
return r.Update(ctx, secret)
|
||||
}
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -2,16 +2,20 @@ package controller
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
appsv1 "k8s.io/api/apps/v1"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
"k8s.io/apimachinery/pkg/api/meta"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/runtime"
|
||||
"k8s.io/apimachinery/pkg/runtime/schema"
|
||||
ctrl "sigs.k8s.io/controller-runtime"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client"
|
||||
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
|
||||
logf "sigs.k8s.io/controller-runtime/pkg/log"
|
||||
"sigs.k8s.io/controller-runtime/pkg/recorder"
|
||||
|
||||
@@ -21,41 +25,67 @@ import (
|
||||
|
||||
// resyncInterval is the periodic requeue on a successful reconcile (DESIGN.md
|
||||
// §5's general rule) — it exists to catch drift from someone changing state
|
||||
// directly against the server's API/UI, not from a missed watch event.
|
||||
// directly against the server's API/UI, not from a missed watch event. It
|
||||
// also doubles as the eventual-rotation-detection interval for the Zalando
|
||||
// database path (§8 asks for a live Secret watch; this controller doesn't
|
||||
// have one yet, so a rotated credential is noticed on the next resync
|
||||
// rather than immediately — a known simplification, not a design decision).
|
||||
const resyncInterval = 5 * time.Minute
|
||||
|
||||
// waitInterval is the requeue while waiting on an external condition to
|
||||
// resolve (the credential Secret not existing yet, the server being
|
||||
// unreachable) — shorter than resyncInterval, since these are expected to
|
||||
// change sooner than "someone edited something out of band."
|
||||
const waitInterval = 30 * time.Second
|
||||
// resolve (the Deployment not ready yet, a Zalando cluster not found yet) —
|
||||
// shorter than resyncInterval, since these are expected to change sooner
|
||||
// than "someone edited something out of band."
|
||||
const waitInterval = 15 * time.Second
|
||||
|
||||
// finalizerName cleans up the credentials Secret(s) this controller
|
||||
// generates in the operator's own namespace on delete — the Deployment and
|
||||
// Service are owned (OwnerReference, DESIGN.md §7) and need no finalizer of
|
||||
// their own.
|
||||
const finalizerName = "terdut.ryuvia.com/terdutserver"
|
||||
|
||||
// serviceAccountName is the name the operator registers itself under
|
||||
// server-side (DESIGN.md §6) — a fixed, repo-wide constant, not a spec
|
||||
// field: it names the automation, not anything about this one TerdutServer.
|
||||
const serviceAccountName = "terdut-operator"
|
||||
|
||||
// bootstrapUsername/bootstrapEmail found the one human-shaped user every
|
||||
// fresh install needs (terdut-server's handleBootstrap requires both).
|
||||
// Nobody signs in as this user afterward — its only purpose is minting the
|
||||
// admin key the controller immediately trades for a real service-account
|
||||
// key — so these are fixed, not spec fields.
|
||||
const (
|
||||
bootstrapUsername = "terdut-operator-bootstrap"
|
||||
bootstrapEmail = "bootstrap@terdut-operator.local"
|
||||
)
|
||||
|
||||
// postgresqlGVK is the Zalando postgres-operator's CR (DESIGN.md §8).
|
||||
// Resolved via unstructured rather than vendoring Zalando's own client, to
|
||||
// keep this operator's dependency on it to "an optional CRD read" rather
|
||||
// than a library — matches §9's "degrade gracefully if the CRD isn't
|
||||
// installed" stance.
|
||||
var postgresqlGVK = schema.GroupVersionKind{Group: "acid.zalan.do", Version: "v1", Kind: "postgresql"}
|
||||
|
||||
// TerdutServerReconciler reconciles a TerdutServer object.
|
||||
//
|
||||
// Stage 1 (ROADMAP.md): bootstrap/credentials only, bring-your-own path
|
||||
// only. It never creates, updates, or deletes anything server-side or any
|
||||
// Deployment/Service — it only reads a Secret the operator's own namespace
|
||||
// already has and probes the server's /api/version. No finalizer: this
|
||||
// controller owns nothing that needs cleaning up on delete (it never creates
|
||||
// the credentials Secret itself, only ever adopts one a human already
|
||||
// made) — revisit once self-registration (Stage 5) generates its own.
|
||||
// Stage 1 (ROADMAP.md): full lifecycle. The operator creates and owns every
|
||||
// TerdutServer it manages (DESIGN.md §1) — there is no adopt path.
|
||||
type TerdutServerReconciler struct {
|
||||
client.Client
|
||||
Scheme *runtime.Scheme
|
||||
|
||||
// OperatorNamespace is where every credentials Secret this controller
|
||||
// reads or writes lives (DESIGN.md §6) — never the TerdutServer's own
|
||||
// generates lives (DESIGN.md §6) — never the TerdutServer's own
|
||||
// namespace. Set from the POD_NAMESPACE downward-API env var in
|
||||
// production (cmd/main.go); tests set it directly.
|
||||
OperatorNamespace string
|
||||
|
||||
// Recorder emits the Kubernetes Events DESIGN.md §12 asks for on every
|
||||
// externally-visible outcome. The events.k8s.io/v1 API, not the
|
||||
// deprecated core/v1 one GetEventRecorderFor still returns.
|
||||
// externally-visible outcome.
|
||||
Recorder recorder.EventRecorder
|
||||
|
||||
// NewClient builds the terdut-server API client for a given endpoint.
|
||||
// A field, not a direct tdclient.New call, so tests can substitute an
|
||||
// NewClient builds the terdut-server API client for a given endpoint. A
|
||||
// field, not a direct tdclient.New call, so tests can substitute an
|
||||
// httptest.Server's client without a real network round trip. Defaults
|
||||
// to tdclient.New via SetupWithManager.
|
||||
NewClient func(endpoint string) *tdclient.Client
|
||||
@@ -64,7 +94,10 @@ type TerdutServerReconciler struct {
|
||||
// +kubebuilder:rbac:groups=terdut.ryuvia.com,resources=terdutservers,verbs=get;list;watch;create;update;patch;delete
|
||||
// +kubebuilder:rbac:groups=terdut.ryuvia.com,resources=terdutservers/status,verbs=get;update;patch
|
||||
// +kubebuilder:rbac:groups=terdut.ryuvia.com,resources=terdutservers/finalizers,verbs=update
|
||||
// +kubebuilder:rbac:groups="",resources=secrets,verbs=get;list;watch
|
||||
// +kubebuilder:rbac:groups="",resources=secrets,verbs=get;list;watch;create;update;patch;delete
|
||||
// +kubebuilder:rbac:groups="",resources=services,verbs=get;list;watch;create;update;patch;delete
|
||||
// +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
|
||||
// +kubebuilder:rbac:groups=acid.zalan.do,resources=postgresqls,verbs=get;list;watch
|
||||
|
||||
func (r *TerdutServerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
|
||||
log := logf.FromContext(ctx)
|
||||
@@ -77,77 +110,88 @@ func (r *TerdutServerReconciler) Reconcile(ctx context.Context, req ctrl.Request
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
|
||||
if srv.Spec.CredentialsSecretRef == nil {
|
||||
return r.setNotReady(ctx, &srv,
|
||||
terdutv1alpha1.ReasonCredentialsSecretRefRequired,
|
||||
"spec.credentialsSecretRef is unset; self-registration bootstrap isn't "+
|
||||
"implemented yet (Stage 5) — mint an instance-scoped service account "+
|
||||
"with your own admin session (POST /api/service-accounts) and set "+
|
||||
"this field to a Secret holding its key (DESIGN.md §6)")
|
||||
if !srv.DeletionTimestamp.IsZero() {
|
||||
return r.reconcileDelete(ctx, &srv)
|
||||
}
|
||||
ref := srv.Spec.CredentialsSecretRef
|
||||
|
||||
var secret corev1.Secret
|
||||
secretKey := client.ObjectKey{Namespace: r.OperatorNamespace, Name: ref.Name}
|
||||
if err := r.Get(ctx, secretKey, &secret); err != nil {
|
||||
if apierrors.IsNotFound(err) {
|
||||
return r.setNotReady(ctx, &srv,
|
||||
terdutv1alpha1.ReasonCredentialsSecretNotFound,
|
||||
fmt.Sprintf("Secret %q not found in namespace %q", ref.Name, r.OperatorNamespace))
|
||||
if !controllerutil.ContainsFinalizer(&srv, finalizerName) {
|
||||
controllerutil.AddFinalizer(&srv, finalizerName)
|
||||
if err := r.Update(ctx, &srv); err != nil {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
// The Update above re-triggers a reconcile via the watch; nothing
|
||||
// further to do on this pass.
|
||||
return ctrl.Result{}, nil
|
||||
}
|
||||
|
||||
dbEnv, dbErr := r.resolveDatabaseEnv(ctx, &srv)
|
||||
if dbErr != nil {
|
||||
return r.setNotReady(ctx, &srv, dbErr.reason, dbErr.message, waitInterval)
|
||||
}
|
||||
|
||||
deploy, err := r.reconcileDeployment(ctx, &srv, dbEnv)
|
||||
if err != nil {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
if err := r.reconcileService(ctx, &srv); err != nil {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
|
||||
raw, ok := secret.Data[ref.Key]
|
||||
if !ok || len(raw) == 0 {
|
||||
meta.SetStatusCondition(&srv.Status.Conditions, metav1.Condition{
|
||||
Type: terdutv1alpha1.ConditionDatabaseReady,
|
||||
Status: metav1.ConditionTrue,
|
||||
Reason: terdutv1alpha1.ReasonAdopted,
|
||||
Message: "database resolved",
|
||||
})
|
||||
|
||||
if deploy.Status.ReadyReplicas < 1 {
|
||||
return r.setNotReady(ctx, &srv,
|
||||
terdutv1alpha1.ReasonCredentialsSecretInvalid,
|
||||
fmt.Sprintf("Secret %q has no data under key %q", ref.Name, ref.Key))
|
||||
terdutv1alpha1.ReasonWaitingForDeployment,
|
||||
fmt.Sprintf("Deployment %q has no ready replica yet", deploy.Name),
|
||||
waitInterval)
|
||||
}
|
||||
|
||||
newClient := r.NewClient
|
||||
if newClient == nil {
|
||||
newClient = tdclient.New
|
||||
}
|
||||
if _, err := newClient(srv.Spec.Endpoint).Version(ctx); err != nil {
|
||||
return r.setNotReady(ctx, &srv,
|
||||
terdutv1alpha1.ReasonServerUnreachable,
|
||||
fmt.Sprintf("GET %s/api/version: %v", srv.Spec.Endpoint, err))
|
||||
if srv.Status.CredentialsSecretRef == nil {
|
||||
if err := r.reconcileBootstrap(ctx, &srv); err != nil {
|
||||
if pending, ok := errors.AsType[*bootstrapStateLostError](err); ok {
|
||||
return r.setNotReady(ctx, &srv, terdutv1alpha1.ReasonBootstrapStateLost, pending.Error(), waitInterval)
|
||||
}
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
}
|
||||
|
||||
srv.Status.CredentialsSecretRef = ref.DeepCopy()
|
||||
meta.SetStatusCondition(&srv.Status.Conditions, metav1.Condition{
|
||||
Type: terdutv1alpha1.ConditionBootstrapped,
|
||||
Status: metav1.ConditionTrue,
|
||||
Reason: terdutv1alpha1.ReasonAdopted,
|
||||
Message: fmt.Sprintf("adopted spec.credentialsSecretRef (Secret %q, key %q)", ref.Name, ref.Key),
|
||||
Message: fmt.Sprintf("credentials in Secret %q", srv.Status.CredentialsSecretRef.Name),
|
||||
})
|
||||
meta.SetStatusCondition(&srv.Status.Conditions, metav1.Condition{
|
||||
Type: terdutv1alpha1.ConditionReady,
|
||||
Status: metav1.ConditionTrue,
|
||||
Reason: terdutv1alpha1.ReasonAdopted,
|
||||
Message: "credentials adopted, server reachable",
|
||||
Message: "deployment ready, database resolved, credentials bootstrapped",
|
||||
})
|
||||
srv.Status.ServiceName = srv.Name
|
||||
srv.Status.ObservedGeneration = srv.Generation
|
||||
if err := r.Status().Update(ctx, &srv); err != nil {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
if r.Recorder != nil {
|
||||
r.Recorder.Eventf(&srv, nil, corev1.EventTypeNormal,
|
||||
terdutv1alpha1.ReasonAdopted, terdutv1alpha1.ReasonAdopted,
|
||||
"credentials adopted, server reachable")
|
||||
r.Recorder.Eventf(&srv, nil, corev1.EventTypeNormal, terdutv1alpha1.ReasonAdopted, terdutv1alpha1.ReasonAdopted,
|
||||
"terdut-server ready")
|
||||
}
|
||||
log.Info("TerdutServer ready", "endpoint", srv.Spec.Endpoint)
|
||||
log.Info("TerdutServer ready", "name", srv.Name)
|
||||
|
||||
return ctrl.Result{RequeueAfter: resyncInterval}, nil
|
||||
}
|
||||
|
||||
// setNotReady records Ready: False with reason/message on both conditions,
|
||||
// fires a Warning event, and requeues after waitInterval — every "waiting on
|
||||
// something external" exit from Reconcile goes through here so the
|
||||
// condition/event/requeue shape can't drift between them.
|
||||
// setNotReady records Ready: False with reason/message, fires a Warning
|
||||
// event, and requeues after d. Every "waiting on something" exit from
|
||||
// Reconcile goes through here so the condition/event/requeue shape can't
|
||||
// drift between them.
|
||||
func (r *TerdutServerReconciler) setNotReady(
|
||||
ctx context.Context, srv *terdutv1alpha1.TerdutServer, reason, message string,
|
||||
ctx context.Context, srv *terdutv1alpha1.TerdutServer, reason, message string, d time.Duration,
|
||||
) (ctrl.Result, error) {
|
||||
meta.SetStatusCondition(&srv.Status.Conditions, metav1.Condition{
|
||||
Type: terdutv1alpha1.ConditionReady,
|
||||
@@ -162,7 +206,31 @@ func (r *TerdutServerReconciler) setNotReady(
|
||||
if r.Recorder != nil {
|
||||
r.Recorder.Eventf(srv, nil, corev1.EventTypeWarning, reason, reason, message)
|
||||
}
|
||||
return ctrl.Result{RequeueAfter: waitInterval}, nil
|
||||
return ctrl.Result{RequeueAfter: d}, nil
|
||||
}
|
||||
|
||||
// reconcileDelete cleans up the credentials Secret(s) this controller
|
||||
// generated in the operator's own namespace. The Deployment and Service are
|
||||
// owned (OwnerReference, DESIGN.md §7) and need no attention here — normal
|
||||
// GC handles them. There is no server-side "delete this install" call to
|
||||
// make: bootstrap created a user and a service account, and terdut-server's
|
||||
// API has no way to delete either (only to revoke individual keys), so
|
||||
// there is nothing meaningful to undo there either.
|
||||
func (r *TerdutServerReconciler) reconcileDelete(ctx context.Context, srv *terdutv1alpha1.TerdutServer) (ctrl.Result, error) {
|
||||
if !controllerutil.ContainsFinalizer(srv, finalizerName) {
|
||||
return ctrl.Result{}, nil
|
||||
}
|
||||
for _, name := range []string{checkpointSecretName(srv), credentialsSecretName(srv)} {
|
||||
sec := &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: r.OperatorNamespace}}
|
||||
if err := r.Delete(ctx, sec); err != nil && !apierrors.IsNotFound(err) {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
}
|
||||
controllerutil.RemoveFinalizer(srv, finalizerName)
|
||||
if err := r.Update(ctx, srv); err != nil {
|
||||
return ctrl.Result{}, err
|
||||
}
|
||||
return ctrl.Result{}, nil
|
||||
}
|
||||
|
||||
// SetupWithManager sets up the controller with the Manager.
|
||||
@@ -175,6 +243,26 @@ func (r *TerdutServerReconciler) SetupWithManager(mgr ctrl.Manager) error {
|
||||
}
|
||||
return ctrl.NewControllerManagedBy(mgr).
|
||||
For(&terdutv1alpha1.TerdutServer{}).
|
||||
Owns(&appsv1.Deployment{}).
|
||||
Owns(&corev1.Service{}).
|
||||
Named("terdutserver").
|
||||
Complete(r)
|
||||
}
|
||||
|
||||
// --- naming ---
|
||||
|
||||
func checkpointSecretName(srv *terdutv1alpha1.TerdutServer) string {
|
||||
return fmt.Sprintf("%s.%s-bootstrap-admin", srv.Namespace, srv.Name)
|
||||
}
|
||||
|
||||
func credentialsSecretName(srv *terdutv1alpha1.TerdutServer) string {
|
||||
return fmt.Sprintf("%s.%s-instance-credentials", srv.Namespace, srv.Name)
|
||||
}
|
||||
|
||||
func serviceURL(srv *terdutv1alpha1.TerdutServer) string {
|
||||
port := srv.Spec.Networking.ServicePort
|
||||
if port == 0 {
|
||||
port = 8080
|
||||
}
|
||||
return fmt.Sprintf("http://%s.%s.svc:%d", srv.Name, srv.Namespace, port)
|
||||
}
|
||||
|
||||
@@ -2,43 +2,120 @@ package controller
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"sync"
|
||||
|
||||
. "github.com/onsi/ginkgo/v2"
|
||||
. "github.com/onsi/gomega"
|
||||
appsv1 "k8s.io/api/apps/v1"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
"k8s.io/apimachinery/pkg/api/meta"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
|
||||
"k8s.io/apimachinery/pkg/types"
|
||||
ctrl "sigs.k8s.io/controller-runtime"
|
||||
"sigs.k8s.io/controller-runtime/pkg/reconcile"
|
||||
|
||||
terdutv1alpha1 "git.ryuvia.com/niklas/terdut-operator/api/v1alpha1"
|
||||
"git.ryuvia.com/niklas/terdut-operator/internal/tdclient"
|
||||
)
|
||||
|
||||
const (
|
||||
// unusedEndpoint is a syntactically valid URL no test here ever expects
|
||||
// to actually be dialed (the cases using it fail before reaching the
|
||||
// server-reachability check).
|
||||
unusedEndpoint = "http://unused.invalid"
|
||||
// tokenKey is the data key every test's credentials Secret uses.
|
||||
tokenKey = "token"
|
||||
)
|
||||
// fakeTerdutServer reproduces the exact stateful semantics of
|
||||
// /api/bootstrap, /api/service-accounts and /api/version that the
|
||||
// bootstrap flow depends on (DESIGN.md §6, §11: "terdut-server's REST API
|
||||
// is faked with a small httptest.Server per controller test ... matching
|
||||
// the real handlers' request/response shapes"), including the 403-after-
|
||||
// first-success and 409-on-name-conflict behavior the adopt-on-conflict
|
||||
// recovery path exists for.
|
||||
type fakeTerdutServer struct {
|
||||
mu sync.Mutex
|
||||
bootstrapped bool
|
||||
bootstrap403 bool // force every /api/bootstrap call to 403, even the first
|
||||
nextID int64
|
||||
accounts map[string]int64 // name -> id
|
||||
keyMints map[int64]int // id -> number of keys minted so far
|
||||
}
|
||||
|
||||
// fakeTerdutServer is an httptest.Server standing in for terdut-server's
|
||||
// GET /api/version, per DESIGN.md §11 ("terdut-server's REST API is faked
|
||||
// with a small httptest.Server per controller test ... no real Postgres or
|
||||
// real terdut-server binary needed for controller unit tests").
|
||||
func fakeTerdutServer(version string) *httptest.Server {
|
||||
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/api/version" {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
func newFakeTerdutServer() (*fakeTerdutServer, *httptest.Server) {
|
||||
f := &fakeTerdutServer{accounts: map[string]int64{}, keyMints: map[int64]int{}}
|
||||
return f, httptest.NewServer(f)
|
||||
}
|
||||
|
||||
func (f *fakeTerdutServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
f.mu.Lock()
|
||||
defer f.mu.Unlock()
|
||||
|
||||
switch {
|
||||
case r.URL.Path == "/api/version":
|
||||
writeJSON(w, http.StatusOK, map[string]string{"version": "test"})
|
||||
|
||||
case r.URL.Path == "/api/bootstrap" && r.Method == http.MethodPost:
|
||||
if f.bootstrapped || f.bootstrap403 {
|
||||
writeJSON(w, http.StatusForbidden, map[string]string{"error": "bootstrap already completed"})
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, `{"version":%q}`, version) //nolint:errcheck
|
||||
}))
|
||||
f.bootstrapped = true
|
||||
writeJSON(w, http.StatusCreated, tdclient.BootstrapResult{
|
||||
APIKey: tdclient.APIKey{ID: 1, Name: "bootstrap", Key: "admin-key-raw"},
|
||||
})
|
||||
|
||||
case r.URL.Path == "/api/service-accounts" && r.Method == http.MethodPost:
|
||||
var req struct {
|
||||
Name string `json:"name"`
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
_ = json.NewDecoder(r.Body).Decode(&req)
|
||||
if _, exists := f.accounts[req.Name]; exists {
|
||||
writeJSON(w, http.StatusConflict, map[string]string{"error": "a service account with that name already exists"})
|
||||
return
|
||||
}
|
||||
f.nextID++
|
||||
id := f.nextID
|
||||
f.accounts[req.Name] = id
|
||||
f.keyMints[id] = 1
|
||||
writeJSON(w, http.StatusCreated, tdclient.CreateServiceAccountResult{
|
||||
ServiceAccount: tdclient.ServiceAccount{ID: id, Name: req.Name, Scope: req.Scope},
|
||||
Key: tdclient.APIKey{ID: 1, Name: "initial", Key: fmt.Sprintf("instance-key-%d-initial", id)},
|
||||
})
|
||||
|
||||
case r.URL.Path == "/api/service-accounts" && r.Method == http.MethodGet:
|
||||
name := r.URL.Query().Get("name")
|
||||
id, exists := f.accounts[name]
|
||||
if !exists {
|
||||
writeJSON(w, http.StatusOK, []tdclient.ServiceAccount{})
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, []tdclient.ServiceAccount{{ID: id, Name: name, Scope: "instance"}})
|
||||
|
||||
default:
|
||||
if id, name, ok := parseKeysPath(r.URL.Path); ok && r.Method == http.MethodPost {
|
||||
f.keyMints[id]++
|
||||
writeJSON(w, http.StatusCreated, tdclient.APIKey{
|
||||
ID: int64(f.keyMints[id]), Name: name,
|
||||
Key: fmt.Sprintf("instance-key-%d-mint%d", id, f.keyMints[id]),
|
||||
})
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}
|
||||
|
||||
func parseKeysPath(path string) (id int64, mintName string, ok bool) {
|
||||
var parsedID int64
|
||||
n, err := fmt.Sscanf(path, "/api/service-accounts/%d/keys", &parsedID)
|
||||
if err != nil || n != 1 {
|
||||
return 0, "", false
|
||||
}
|
||||
return parsedID, "minted", true
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
|
||||
var _ = Describe("TerdutServer Controller", func() {
|
||||
@@ -63,10 +140,23 @@ var _ = Describe("TerdutServer Controller", func() {
|
||||
AfterEach(func(ctx SpecContext) {
|
||||
srv := &terdutv1alpha1.TerdutServer{}
|
||||
if err := k8sClient.Get(ctx, objKey, srv); err == nil {
|
||||
Expect(k8sClient.Delete(ctx, srv)).To(Succeed())
|
||||
srv.Finalizers = nil
|
||||
_ = k8sClient.Update(ctx, srv)
|
||||
_ = k8sClient.Delete(ctx, srv)
|
||||
}
|
||||
for _, n := range []string{checkpointSecretNameFor(name), credentialsSecretNameFor(name)} {
|
||||
_ = k8sClient.Delete(ctx, &corev1.Secret{ObjectMeta: metav1.ObjectMeta{Name: n, Namespace: operatorNamespace}})
|
||||
}
|
||||
})
|
||||
|
||||
dsnSpec := func() terdutv1alpha1.TerdutServerSpec {
|
||||
return terdutv1alpha1.TerdutServerSpec{
|
||||
Image: terdutv1alpha1.ImageSpec{Repository: "example.invalid/terdut-server", Tag: "test"},
|
||||
Networking: terdutv1alpha1.NetworkingSpec{Hostname: "terdut.example.invalid", ServicePort: 8080},
|
||||
Database: terdutv1alpha1.DatabaseSpec{DSN: "postgres://terdut@test-postgres:5432/terdut?sslmode=require"},
|
||||
}
|
||||
}
|
||||
|
||||
createServer := func(ctx context.Context, spec terdutv1alpha1.TerdutServerSpec) {
|
||||
srv := &terdutv1alpha1.TerdutServer{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: operatorNamespace},
|
||||
@@ -75,123 +165,235 @@ var _ = Describe("TerdutServer Controller", func() {
|
||||
Expect(k8sClient.Create(ctx, srv)).To(Succeed())
|
||||
}
|
||||
|
||||
reconcileOnce := func(ctx context.Context) {
|
||||
_, err := reconciler.Reconcile(ctx, reconcile.Request{NamespacedName: objKey})
|
||||
reconcileOnce := func(ctx context.Context) ctrl.Result {
|
||||
res, err := reconciler.Reconcile(ctx, reconcile.Request{NamespacedName: objKey})
|
||||
Expect(err).NotTo(HaveOccurred())
|
||||
return res
|
||||
}
|
||||
|
||||
markDeploymentReady := func(ctx context.Context) {
|
||||
var deploy appsv1.Deployment
|
||||
Expect(k8sClient.Get(ctx, objKey, &deploy)).To(Succeed())
|
||||
deploy.Status.ReadyReplicas = 1
|
||||
deploy.Status.Replicas = 1
|
||||
Expect(k8sClient.Status().Update(ctx, &deploy)).To(Succeed())
|
||||
}
|
||||
|
||||
readyCondition := func(ctx context.Context) metav1.Condition {
|
||||
srv := &terdutv1alpha1.TerdutServer{}
|
||||
Expect(k8sClient.Get(ctx, objKey, srv)).To(Succeed())
|
||||
c := meta.FindStatusCondition(srv.Status.Conditions, terdutv1alpha1.ConditionReady)
|
||||
Expect(c).NotTo(BeNil(), "Ready condition should always be set after a reconcile")
|
||||
Expect(c).NotTo(BeNil(), "Ready condition should always be set after a reconcile past the finalizer-add pass")
|
||||
return *c
|
||||
}
|
||||
|
||||
When("spec.credentialsSecretRef is unset", func() {
|
||||
It("reports Ready: False, reason CredentialsSecretRefRequired, and makes no API call", func(ctx SpecContext) {
|
||||
createServer(ctx, terdutv1alpha1.TerdutServerSpec{Endpoint: unusedEndpoint})
|
||||
Describe("bring-your-own DSN path", func() {
|
||||
It("reaches Ready through the full lifecycle: finalizer, Deployment/Service, wait, bootstrap", func(ctx SpecContext) {
|
||||
fake, fakeSrv := newFakeTerdutServer()
|
||||
DeferCleanup(fakeSrv.Close)
|
||||
reconciler.NewClient = func(string) *tdclient.Client { return tdclient.New(fakeSrv.URL) }
|
||||
|
||||
createServer(ctx, dsnSpec())
|
||||
|
||||
// Pass 1: adds the finalizer and returns early.
|
||||
reconcileOnce(ctx)
|
||||
srv := &terdutv1alpha1.TerdutServer{}
|
||||
Expect(k8sClient.Get(ctx, objKey, srv)).To(Succeed())
|
||||
Expect(srv.Finalizers).To(ContainElement(finalizerName))
|
||||
|
||||
cond := readyCondition(ctx)
|
||||
Expect(cond.Status).To(Equal(metav1.ConditionFalse))
|
||||
Expect(cond.Reason).To(Equal(terdutv1alpha1.ReasonCredentialsSecretRefRequired))
|
||||
})
|
||||
})
|
||||
|
||||
When("the referenced Secret does not exist", func() {
|
||||
It("reports Ready: False, reason CredentialsSecretNotFound", func(ctx SpecContext) {
|
||||
createServer(ctx, terdutv1alpha1.TerdutServerSpec{
|
||||
Endpoint: unusedEndpoint,
|
||||
CredentialsSecretRef: &terdutv1alpha1.SecretKeyRef{Name: "does-not-exist", Key: tokenKey},
|
||||
})
|
||||
// Pass 2: creates Deployment + Service, waits for readiness.
|
||||
reconcileOnce(ctx)
|
||||
Expect(readyCondition(ctx).Reason).To(Equal(terdutv1alpha1.ReasonWaitingForDeployment))
|
||||
|
||||
cond := readyCondition(ctx)
|
||||
Expect(cond.Status).To(Equal(metav1.ConditionFalse))
|
||||
Expect(cond.Reason).To(Equal(terdutv1alpha1.ReasonCredentialsSecretNotFound))
|
||||
})
|
||||
})
|
||||
|
||||
When("the referenced Secret exists but has no data under the given key", func() {
|
||||
It("reports Ready: False, reason CredentialsSecretInvalid", func(ctx SpecContext) {
|
||||
secret := &corev1.Secret{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name + "-creds", Namespace: operatorNamespace},
|
||||
Data: map[string][]byte{"wrong-key": []byte("tdsa_something")},
|
||||
var deploy appsv1.Deployment
|
||||
Expect(k8sClient.Get(ctx, objKey, &deploy)).To(Succeed())
|
||||
Expect(deploy.Spec.Template.Spec.Containers[0].Image).To(Equal("example.invalid/terdut-server:test"))
|
||||
envNames := map[string]string{}
|
||||
for _, e := range deploy.Spec.Template.Spec.Containers[0].Env {
|
||||
envNames[e.Name] = e.Value
|
||||
}
|
||||
Expect(k8sClient.Create(ctx, secret)).To(Succeed())
|
||||
defer func() { _ = k8sClient.Delete(ctx, secret) }()
|
||||
Expect(envNames).To(HaveKeyWithValue("TERDUT_DB_DSN", "postgres://terdut@test-postgres:5432/terdut?sslmode=require"))
|
||||
Expect(envNames).To(HaveKeyWithValue("TERDUT_OPERATOR_MODE", "true"))
|
||||
|
||||
createServer(ctx, terdutv1alpha1.TerdutServerSpec{
|
||||
Endpoint: unusedEndpoint,
|
||||
CredentialsSecretRef: &terdutv1alpha1.SecretKeyRef{Name: secret.Name, Key: tokenKey},
|
||||
})
|
||||
var svc corev1.Service
|
||||
Expect(k8sClient.Get(ctx, objKey, &svc)).To(Succeed())
|
||||
Expect(svc.Spec.Ports[0].Port).To(Equal(int32(8080)))
|
||||
|
||||
// Simulate the Deployment becoming ready (envtest has no
|
||||
// kubelet/deployment-controller to do this for real).
|
||||
markDeploymentReady(ctx)
|
||||
|
||||
// Pass 3: bootstraps for real against the fake server.
|
||||
reconcileOnce(ctx)
|
||||
|
||||
cond := readyCondition(ctx)
|
||||
Expect(cond.Status).To(Equal(metav1.ConditionFalse))
|
||||
Expect(cond.Reason).To(Equal(terdutv1alpha1.ReasonCredentialsSecretInvalid))
|
||||
Expect(fake.bootstrapped).To(BeTrue())
|
||||
|
||||
Expect(k8sClient.Get(ctx, objKey, srv)).To(Succeed())
|
||||
ready := meta.FindStatusCondition(srv.Status.Conditions, terdutv1alpha1.ConditionReady)
|
||||
Expect(ready.Status).To(Equal(metav1.ConditionTrue))
|
||||
Expect(ready.Reason).To(Equal(terdutv1alpha1.ReasonAdopted))
|
||||
Expect(srv.Status.CredentialsSecretRef).NotTo(BeNil())
|
||||
Expect(srv.Status.CredentialsSecretRef.Key).To(Equal(credentialsSecretDataKey))
|
||||
|
||||
var credsSecret corev1.Secret
|
||||
Expect(k8sClient.Get(ctx, types.NamespacedName{
|
||||
Name: srv.Status.CredentialsSecretRef.Name, Namespace: operatorNamespace,
|
||||
}, &credsSecret)).To(Succeed())
|
||||
Expect(string(credsSecret.Data[credentialsSecretDataKey])).To(Equal("instance-key-1-initial"))
|
||||
|
||||
// The checkpoint is cleaned up once the lasting credential is
|
||||
// written (DESIGN.md §6 point 2).
|
||||
var checkpoint corev1.Secret
|
||||
err := k8sClient.Get(ctx, types.NamespacedName{Name: checkpointSecretName(srv), Namespace: operatorNamespace}, &checkpoint)
|
||||
Expect(err).To(HaveOccurred())
|
||||
})
|
||||
})
|
||||
|
||||
When("the Secret is valid but the server is unreachable", func() {
|
||||
It("reports Ready: False, reason ServerUnreachable", func(ctx SpecContext) {
|
||||
secret := &corev1.Secret{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name + "-creds", Namespace: operatorNamespace},
|
||||
Data: map[string][]byte{tokenKey: []byte("tdsa_something")},
|
||||
}
|
||||
Expect(k8sClient.Create(ctx, secret)).To(Succeed())
|
||||
defer func() { _ = k8sClient.Delete(ctx, secret) }()
|
||||
Describe("the adopt-on-409 recovery path", func() {
|
||||
It("mints a fresh key instead of erroring when the service account already exists", func(ctx SpecContext) {
|
||||
fake, fakeSrv := newFakeTerdutServer()
|
||||
DeferCleanup(fakeSrv.Close)
|
||||
fake.bootstrapped = true // an earlier attempt already bootstrapped...
|
||||
fake.nextID = 1
|
||||
fake.accounts[serviceAccountName] = 1 // ...and already created the service account.
|
||||
fake.keyMints[1] = 1
|
||||
reconciler.NewClient = func(string) *tdclient.Client { return tdclient.New(fakeSrv.URL) }
|
||||
|
||||
createServer(ctx, terdutv1alpha1.TerdutServerSpec{
|
||||
// Port 1 is never listening (closed cleanly and immediately,
|
||||
// unlike an unroutable address which would hang on a
|
||||
// connect timeout) -- keeps the test fast.
|
||||
Endpoint: "http://127.0.0.1:1",
|
||||
CredentialsSecretRef: &terdutv1alpha1.SecretKeyRef{Name: secret.Name, Key: tokenKey},
|
||||
})
|
||||
reconcileOnce(ctx)
|
||||
createServer(ctx, dsnSpec())
|
||||
reconcileOnce(ctx) // finalizer
|
||||
reconcileOnce(ctx) // Deployment/Service, WaitingForDeployment
|
||||
markDeploymentReady(ctx)
|
||||
|
||||
cond := readyCondition(ctx)
|
||||
Expect(cond.Status).To(Equal(metav1.ConditionFalse))
|
||||
Expect(cond.Reason).To(Equal(terdutv1alpha1.ReasonServerUnreachable))
|
||||
})
|
||||
})
|
||||
// The earlier attempt's checkpoint survived (that's how this
|
||||
// reconcile can authenticate at all to recover).
|
||||
Expect(reconciler.writeSecret(ctx, checkpointSecretName(&terdutv1alpha1.TerdutServer{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: operatorNamespace},
|
||||
}), "admin-key-raw")).To(Succeed())
|
||||
|
||||
When("the Secret is valid and the server answers /api/version", func() {
|
||||
It("reports Ready: True, Bootstrapped: True, and mirrors credentialsSecretRef into status", func(ctx SpecContext) {
|
||||
fake := fakeTerdutServer("v0.20.0")
|
||||
DeferCleanup(fake.Close)
|
||||
|
||||
secret := &corev1.Secret{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: name + "-creds", Namespace: operatorNamespace},
|
||||
Data: map[string][]byte{tokenKey: []byte("tdsa_something")},
|
||||
}
|
||||
Expect(k8sClient.Create(ctx, secret)).To(Succeed())
|
||||
defer func() { _ = k8sClient.Delete(ctx, secret) }()
|
||||
|
||||
createServer(ctx, terdutv1alpha1.TerdutServerSpec{
|
||||
Endpoint: fake.URL,
|
||||
CredentialsSecretRef: &terdutv1alpha1.SecretKeyRef{Name: secret.Name, Key: tokenKey},
|
||||
})
|
||||
reconcileOnce(ctx)
|
||||
|
||||
srv := &terdutv1alpha1.TerdutServer{}
|
||||
Expect(k8sClient.Get(ctx, objKey, srv)).To(Succeed())
|
||||
Expect(meta.FindStatusCondition(srv.Status.Conditions, terdutv1alpha1.ConditionReady).Status).To(Equal(metav1.ConditionTrue))
|
||||
|
||||
ready := meta.FindStatusCondition(srv.Status.Conditions, terdutv1alpha1.ConditionReady)
|
||||
Expect(ready).NotTo(BeNil())
|
||||
Expect(ready.Status).To(Equal(metav1.ConditionTrue))
|
||||
Expect(ready.Reason).To(Equal(terdutv1alpha1.ReasonAdopted))
|
||||
var credsSecret corev1.Secret
|
||||
Expect(k8sClient.Get(ctx, types.NamespacedName{
|
||||
Name: srv.Status.CredentialsSecretRef.Name, Namespace: operatorNamespace,
|
||||
}, &credsSecret)).To(Succeed())
|
||||
// Minted fresh, not the (never-seen-by-this-reconcile) "initial"
|
||||
// key from the account's original creation.
|
||||
Expect(string(credsSecret.Data[credentialsSecretDataKey])).To(Equal("instance-key-1-mint2"))
|
||||
})
|
||||
})
|
||||
|
||||
bootstrapped := meta.FindStatusCondition(srv.Status.Conditions, terdutv1alpha1.ConditionBootstrapped)
|
||||
Expect(bootstrapped).NotTo(BeNil())
|
||||
Expect(bootstrapped.Status).To(Equal(metav1.ConditionTrue))
|
||||
Describe("the BootstrapStateLost path", func() {
|
||||
It("fails closed when the server reports already-bootstrapped with no checkpoint to recover from", func(ctx SpecContext) {
|
||||
fake, fakeSrv := newFakeTerdutServer()
|
||||
_ = fake
|
||||
DeferCleanup(fakeSrv.Close)
|
||||
fake.bootstrap403 = true
|
||||
reconciler.NewClient = func(string) *tdclient.Client { return tdclient.New(fakeSrv.URL) }
|
||||
|
||||
Expect(srv.Status.CredentialsSecretRef).NotTo(BeNil())
|
||||
Expect(srv.Status.CredentialsSecretRef.Name).To(Equal(secret.Name))
|
||||
Expect(srv.Status.CredentialsSecretRef.Key).To(Equal(tokenKey))
|
||||
Expect(srv.Status.ObservedGeneration).To(Equal(srv.Generation))
|
||||
createServer(ctx, dsnSpec())
|
||||
reconcileOnce(ctx) // finalizer
|
||||
reconcileOnce(ctx) // Deployment/Service
|
||||
markDeploymentReady(ctx)
|
||||
|
||||
reconcileOnce(ctx)
|
||||
|
||||
cond := readyCondition(ctx)
|
||||
Expect(cond.Status).To(Equal(metav1.ConditionFalse))
|
||||
Expect(cond.Reason).To(Equal(terdutv1alpha1.ReasonBootstrapStateLost))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("the Zalando postgresClusterRef path", func() {
|
||||
zalandoSpec := func(clusterName string) terdutv1alpha1.TerdutServerSpec {
|
||||
s := dsnSpec()
|
||||
s.Database = terdutv1alpha1.DatabaseSpec{
|
||||
PostgresClusterRef: &terdutv1alpha1.PostgresClusterRef{Name: clusterName},
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
It("waits with reason PostgresClusterNotFound when the postgresql CR doesn't exist yet", func(ctx SpecContext) {
|
||||
createServer(ctx, zalandoSpec("missing-cluster"))
|
||||
reconcileOnce(ctx) // finalizer
|
||||
reconcileOnce(ctx)
|
||||
|
||||
Expect(readyCondition(ctx).Reason).To(Equal(terdutv1alpha1.ReasonPostgresClusterNotFound))
|
||||
})
|
||||
|
||||
It("resolves a real DSN and reaches Ready once the CR and its generated Secret both exist", func(ctx SpecContext) {
|
||||
clusterName := "pg-" + name
|
||||
cluster := &unstructured.Unstructured{}
|
||||
cluster.SetGroupVersionKind(postgresqlGVK)
|
||||
cluster.SetName(clusterName)
|
||||
cluster.SetNamespace(operatorNamespace)
|
||||
Expect(unstructured.SetNestedField(cluster.Object, map[string]any{}, "spec")).To(Succeed())
|
||||
Expect(k8sClient.Create(ctx, cluster)).To(Succeed())
|
||||
DeferCleanup(func() { _ = k8sClient.Delete(ctx, cluster) })
|
||||
|
||||
credsSecretName := fmt.Sprintf("terdut.%s.credentials.postgresql.acid.zalan.do", clusterName)
|
||||
zalandoSecret := &corev1.Secret{
|
||||
ObjectMeta: metav1.ObjectMeta{Name: credsSecretName, Namespace: operatorNamespace},
|
||||
Data: map[string][]byte{"password": []byte("whatever")},
|
||||
}
|
||||
Expect(k8sClient.Create(ctx, zalandoSecret)).To(Succeed())
|
||||
DeferCleanup(func() { _ = k8sClient.Delete(ctx, zalandoSecret) })
|
||||
|
||||
fake, fakeSrv := newFakeTerdutServer()
|
||||
_ = fake
|
||||
DeferCleanup(fakeSrv.Close)
|
||||
reconciler.NewClient = func(string) *tdclient.Client { return tdclient.New(fakeSrv.URL) }
|
||||
|
||||
createServer(ctx, zalandoSpec(clusterName))
|
||||
reconcileOnce(ctx) // finalizer
|
||||
reconcileOnce(ctx) // Deployment/Service created with resolved DB env
|
||||
|
||||
var deploy appsv1.Deployment
|
||||
Expect(k8sClient.Get(ctx, objKey, &deploy)).To(Succeed())
|
||||
var dsn string
|
||||
for _, e := range deploy.Spec.Template.Spec.Containers[0].Env {
|
||||
if e.Name == "TERDUT_DB_DSN" {
|
||||
dsn = e.Value
|
||||
}
|
||||
}
|
||||
Expect(dsn).To(Equal(fmt.Sprintf("postgres://terdut@%s.%s.svc:5432/terdut?sslmode=require", clusterName, operatorNamespace)))
|
||||
|
||||
markDeploymentReady(ctx)
|
||||
reconcileOnce(ctx)
|
||||
|
||||
Expect(readyCondition(ctx).Status).To(Equal(metav1.ConditionTrue))
|
||||
})
|
||||
})
|
||||
|
||||
Describe("deletion", func() {
|
||||
It("removes the credentials and checkpoint Secrets and the finalizer", func(ctx SpecContext) {
|
||||
fake, fakeSrv := newFakeTerdutServer()
|
||||
_ = fake
|
||||
DeferCleanup(fakeSrv.Close)
|
||||
reconciler.NewClient = func(string) *tdclient.Client { return tdclient.New(fakeSrv.URL) }
|
||||
|
||||
createServer(ctx, dsnSpec())
|
||||
reconcileOnce(ctx)
|
||||
reconcileOnce(ctx)
|
||||
markDeploymentReady(ctx)
|
||||
reconcileOnce(ctx)
|
||||
|
||||
srv := &terdutv1alpha1.TerdutServer{}
|
||||
Expect(k8sClient.Get(ctx, objKey, srv)).To(Succeed())
|
||||
credsName := srv.Status.CredentialsSecretRef.Name
|
||||
|
||||
Expect(k8sClient.Delete(ctx, srv)).To(Succeed())
|
||||
reconcileOnce(ctx) // runs the finalizer
|
||||
|
||||
err := k8sClient.Get(ctx, objKey, srv)
|
||||
Expect(err).To(HaveOccurred(), "the TerdutServer itself should be gone once the finalizer clears")
|
||||
|
||||
var leftover corev1.Secret
|
||||
err = k8sClient.Get(ctx, types.NamespacedName{Name: credsName, Namespace: operatorNamespace}, &leftover)
|
||||
Expect(err).To(HaveOccurred(), "the credentials Secret should have been cleaned up by the finalizer")
|
||||
})
|
||||
})
|
||||
|
||||
@@ -205,12 +407,12 @@ var _ = Describe("TerdutServer Controller", func() {
|
||||
})
|
||||
})
|
||||
|
||||
var _ = Describe("TerdutServerReconciler sanity", func() {
|
||||
It("treats a real apierrors.IsNotFound the same as any other caller would", func() {
|
||||
// Guards against a refactor accidentally swapping in a different
|
||||
// not-found check that stops matching what client.Client actually
|
||||
// returns.
|
||||
Expect(apierrors.IsNotFound(apierrors.NewNotFound(
|
||||
terdutv1alpha1.GroupVersion.WithResource("terdutservers").GroupResource(), "x"))).To(BeTrue())
|
||||
})
|
||||
})
|
||||
// checkpointSecretNameFor/credentialsSecretNameFor let AfterEach clean up
|
||||
// without needing a live TerdutServer object (it may already be gone by
|
||||
// then in the deletion test).
|
||||
func checkpointSecretNameFor(name string) string {
|
||||
return fmt.Sprintf("default.%s-bootstrap-admin", name)
|
||||
}
|
||||
func credentialsSecretNameFor(name string) string {
|
||||
return fmt.Sprintf("default.%s-instance-credentials", name)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
package controller
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
"k8s.io/apimachinery/pkg/api/meta"
|
||||
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client"
|
||||
|
||||
terdutv1alpha1 "git.ryuvia.com/niklas/terdut-operator/api/v1alpha1"
|
||||
)
|
||||
|
||||
// databaseError carries a condition reason/message out of resolveDatabaseEnv
|
||||
// — distinct from a plain error, since "the database isn't ready yet" is an
|
||||
// expected, requeue-and-retry outcome (DESIGN.md §8), not a reconcile
|
||||
// failure.
|
||||
type databaseError struct {
|
||||
reason string
|
||||
message string
|
||||
}
|
||||
|
||||
func (e *databaseError) Error() string { return e.message }
|
||||
|
||||
// classifyClusterGetError turns a failed Get of the postgresql.acid.zalan.do
|
||||
// CR into the right condition — distinct from a plain reconcile error, and
|
||||
// its own function (not inlined) so it's unit-testable against a
|
||||
// hand-constructed error without needing a real client at all: a
|
||||
// *meta.NoKindMatchError means the CRD itself isn't installed
|
||||
// (ReasonPostgresOperatorCRDNotInstalled, §8/§9's graceful-degradation
|
||||
// case), apierrors.IsNotFound means the CRD exists but this particular
|
||||
// object doesn't (yet) (ReasonPostgresClusterNotFound).
|
||||
func classifyClusterGetError(err error, clusterName, namespace string) *databaseError {
|
||||
switch {
|
||||
case meta.IsNoMatchError(err):
|
||||
return &databaseError{
|
||||
reason: terdutv1alpha1.ReasonPostgresOperatorCRDNotInstalled,
|
||||
message: "spec.database.postgresClusterRef is set but the postgresql.acid.zalan.do CRD isn't installed in this cluster",
|
||||
}
|
||||
case apierrors.IsNotFound(err):
|
||||
return &databaseError{
|
||||
reason: terdutv1alpha1.ReasonPostgresClusterNotFound,
|
||||
message: fmt.Sprintf("postgresql.acid.zalan.do %q not found in namespace %q", clusterName, namespace),
|
||||
}
|
||||
default:
|
||||
// Any other error (RBAC, transient API failure) is a real
|
||||
// reconcile error, not a condition to report and wait out --
|
||||
// callers still surface it as a databaseError so resolveDatabaseEnv
|
||||
// has one return shape, but Reconcile treats every databaseError as
|
||||
// a wait-and-retry outcome today. Revisit if that ever matters in
|
||||
// practice (a persistent RBAC misconfiguration looks identical to
|
||||
// "not ready yet" until someone reads the condition message).
|
||||
return &databaseError{reason: terdutv1alpha1.ReasonPostgresClusterNotFound, message: err.Error()}
|
||||
}
|
||||
}
|
||||
|
||||
// resolveDatabaseEnv implements DESIGN.md §8's two Postgres paths. It never
|
||||
// reads a password's value — only ever wires a secretKeyRef into the
|
||||
// Deployment's env, the same way the chart does — so it needs no RBAC on
|
||||
// Secret *data* for this, only on the postgresql.acid.zalan.do CR itself
|
||||
// (the XValidation rule on TerdutServerSpec.Database guarantees exactly one
|
||||
// of the two fields below is set, so there's no third case to handle).
|
||||
func (r *TerdutServerReconciler) resolveDatabaseEnv(ctx context.Context, srv *terdutv1alpha1.TerdutServer) ([]corev1.EnvVar, *databaseError) {
|
||||
db := srv.Spec.Database
|
||||
|
||||
if db.PostgresClusterRef != nil {
|
||||
return r.resolveZalandoDatabaseEnv(ctx, srv, db.PostgresClusterRef.Name)
|
||||
}
|
||||
|
||||
env := []corev1.EnvVar{{Name: "TERDUT_DB_DSN", Value: db.DSN}}
|
||||
if db.PasswordSecretRef != nil {
|
||||
env = append(env, corev1.EnvVar{
|
||||
Name: "PGPASSWORD",
|
||||
ValueFrom: &corev1.EnvVarSource{
|
||||
SecretKeyRef: &corev1.SecretKeySelector{
|
||||
LocalObjectReference: corev1.LocalObjectReference{Name: db.PasswordSecretRef.Name},
|
||||
Key: db.PasswordSecretRef.Key,
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
return env, nil
|
||||
}
|
||||
|
||||
// resolveZalandoDatabaseEnv resolves a Zalando postgres-operator
|
||||
// `postgresql` CR into a DSN + PGPASSWORD secretKeyRef, without reading the
|
||||
// generated credentials Secret's value — only confirming it exists, the
|
||||
// same "exists, don't read" posture the DSN path takes toward its own
|
||||
// password Secret.
|
||||
//
|
||||
// By convention (DESIGN.md §8, PostgresClusterRef's own doc comment) the
|
||||
// database and role are both named "terdut", and the primary Service is
|
||||
// named after the cluster CR itself — Zalando's own naming convention, not
|
||||
// something its status exposes as a field to read.
|
||||
func (r *TerdutServerReconciler) resolveZalandoDatabaseEnv(
|
||||
ctx context.Context, srv *terdutv1alpha1.TerdutServer, clusterName string,
|
||||
) ([]corev1.EnvVar, *databaseError) {
|
||||
cluster := &unstructured.Unstructured{}
|
||||
cluster.SetGroupVersionKind(postgresqlGVK)
|
||||
if err := r.Get(ctx, client.ObjectKey{Namespace: srv.Namespace, Name: clusterName}, cluster); err != nil {
|
||||
return nil, classifyClusterGetError(err, clusterName, srv.Namespace)
|
||||
}
|
||||
|
||||
credsSecretName := fmt.Sprintf("terdut.%s.credentials.postgresql.acid.zalan.do", clusterName)
|
||||
var secret corev1.Secret
|
||||
if err := r.Get(ctx, client.ObjectKey{Namespace: srv.Namespace, Name: credsSecretName}, &secret); err != nil {
|
||||
if apierrors.IsNotFound(err) {
|
||||
return nil, &databaseError{
|
||||
reason: terdutv1alpha1.ReasonPostgresClusterNotFound,
|
||||
message: fmt.Sprintf(
|
||||
"postgresql.acid.zalan.do %q found, but its generated credentials Secret %q doesn't exist yet",
|
||||
clusterName, credsSecretName),
|
||||
}
|
||||
}
|
||||
return nil, &databaseError{reason: terdutv1alpha1.ReasonPostgresClusterNotFound, message: err.Error()}
|
||||
}
|
||||
|
||||
host := fmt.Sprintf("%s.%s.svc", clusterName, srv.Namespace)
|
||||
dsn := fmt.Sprintf("postgres://terdut@%s:5432/terdut?sslmode=require", host)
|
||||
return []corev1.EnvVar{
|
||||
{Name: "TERDUT_DB_DSN", Value: dsn},
|
||||
{
|
||||
Name: "PGPASSWORD",
|
||||
ValueFrom: &corev1.EnvVarSource{
|
||||
SecretKeyRef: &corev1.SecretKeySelector{
|
||||
LocalObjectReference: corev1.LocalObjectReference{Name: credsSecretName},
|
||||
Key: "password",
|
||||
},
|
||||
},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package controller
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
apierrors "k8s.io/apimachinery/pkg/api/errors"
|
||||
"k8s.io/apimachinery/pkg/api/meta"
|
||||
"k8s.io/apimachinery/pkg/runtime/schema"
|
||||
|
||||
terdutv1alpha1 "git.ryuvia.com/niklas/terdut-operator/api/v1alpha1"
|
||||
)
|
||||
|
||||
// These exercise classifyClusterGetError directly, against hand-constructed
|
||||
// errors, rather than through envtest -- the point is the classification
|
||||
// logic, not a real Get, and the "CRD not installed" case specifically
|
||||
// can't be produced in the shared envtest suite (its CRDs, stub included,
|
||||
// are always installed for the whole suite's lifetime).
|
||||
|
||||
func TestClassifyClusterGetError_CRDNotInstalled(t *testing.T) {
|
||||
noMatch := &meta.NoKindMatchError{GroupKind: schema.GroupKind{Group: "acid.zalan.do", Kind: "postgresql"}}
|
||||
|
||||
got := classifyClusterGetError(noMatch, "terdut-postgres", "oncall")
|
||||
if got.reason != terdutv1alpha1.ReasonPostgresOperatorCRDNotInstalled {
|
||||
t.Errorf("reason = %q, want %q", got.reason, terdutv1alpha1.ReasonPostgresOperatorCRDNotInstalled)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyClusterGetError_ClusterNotFound(t *testing.T) {
|
||||
notFound := apierrors.NewNotFound(schema.GroupResource{Group: "acid.zalan.do", Resource: "postgresqls"}, "terdut-postgres")
|
||||
|
||||
got := classifyClusterGetError(notFound, "terdut-postgres", "oncall")
|
||||
if got.reason != terdutv1alpha1.ReasonPostgresClusterNotFound {
|
||||
t.Errorf("reason = %q, want %q", got.reason, terdutv1alpha1.ReasonPostgresClusterNotFound)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClassifyClusterGetError_OtherErrorStillReturnsACondition(t *testing.T) {
|
||||
// Not one of the two classified cases -- still returns a databaseError
|
||||
// (Reconcile treats every databaseError as wait-and-retry today; see
|
||||
// the function's own comment on why), never nil, so a caller can't
|
||||
// accidentally treat an unclassified error as success.
|
||||
got := classifyClusterGetError(errors.New("etcd is on fire"), "terdut-postgres", "oncall")
|
||||
if got == nil {
|
||||
t.Fatal("classifyClusterGetError returned nil for an unclassified error")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
package controller
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
appsv1 "k8s.io/api/apps/v1"
|
||||
corev1 "k8s.io/api/core/v1"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/apimachinery/pkg/util/intstr"
|
||||
"sigs.k8s.io/controller-runtime/pkg/client"
|
||||
"sigs.k8s.io/controller-runtime/pkg/controller/controllerutil"
|
||||
|
||||
terdutv1alpha1 "git.ryuvia.com/niklas/terdut-operator/api/v1alpha1"
|
||||
)
|
||||
|
||||
// labelsFor returns the selector labels for srv's Deployment/Service —
|
||||
// fixed once a Deployment exists (its selector is immutable), so this must
|
||||
// never depend on anything in srv.Spec that could change later.
|
||||
func labelsFor(srv *terdutv1alpha1.TerdutServer) map[string]string {
|
||||
return map[string]string{
|
||||
"app.kubernetes.io/name": "terdut-server",
|
||||
"app.kubernetes.io/instance": srv.Name,
|
||||
}
|
||||
}
|
||||
|
||||
// reconcileDeployment creates or updates the Deployment running
|
||||
// terdut-server, owned by srv (DESIGN.md §7: same-namespace generated
|
||||
// objects carry a plain OwnerReference, no finalizer). Returns the live
|
||||
// object (not just the desired one) so the caller can check
|
||||
// status.readyReplicas.
|
||||
func (r *TerdutServerReconciler) reconcileDeployment(
|
||||
ctx context.Context, srv *terdutv1alpha1.TerdutServer, dbEnv []corev1.EnvVar,
|
||||
) (*appsv1.Deployment, error) {
|
||||
deploy := &appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{Name: srv.Name, Namespace: srv.Namespace}}
|
||||
_, err := controllerutil.CreateOrUpdate(ctx, r.Client, deploy, func() error {
|
||||
replicas := srv.Spec.Replicas
|
||||
if replicas == 0 {
|
||||
replicas = 1
|
||||
}
|
||||
labels := labelsFor(srv)
|
||||
|
||||
deploy.Spec.Replicas = &replicas
|
||||
deploy.Spec.Selector = &metav1.LabelSelector{MatchLabels: labels}
|
||||
// Recreate, not RollingUpdate: the sweeper and the notifier are
|
||||
// unsynchronised singletons inside terdut-server, and two replicas
|
||||
// overlapping during a rollout would both page for the same
|
||||
// incident (matches the chart's own deployment.yaml comment).
|
||||
deploy.Spec.Strategy = appsv1.DeploymentStrategy{Type: appsv1.RecreateDeploymentStrategyType}
|
||||
deploy.Spec.Template = corev1.PodTemplateSpec{
|
||||
ObjectMeta: metav1.ObjectMeta{Labels: labels},
|
||||
Spec: corev1.PodSpec{
|
||||
EnableServiceLinks: new(false),
|
||||
Containers: []corev1.Container{{
|
||||
Name: "terdut-server",
|
||||
Image: fmt.Sprintf("%s:%s", srv.Spec.Image.Repository, srv.Spec.Image.Tag),
|
||||
Ports: []corev1.ContainerPort{{
|
||||
Name: "http",
|
||||
ContainerPort: servicePort(srv),
|
||||
Protocol: corev1.ProtocolTCP,
|
||||
}},
|
||||
Env: buildEnv(srv, dbEnv),
|
||||
LivenessProbe: healthzProbe(),
|
||||
ReadinessProbe: healthzProbe(),
|
||||
}},
|
||||
},
|
||||
}
|
||||
return controllerutil.SetControllerReference(srv, deploy, r.Scheme)
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// CreateOrUpdate's own object is desired-state-only on a fresh create
|
||||
// (no .status yet); re-fetch so the ready-replica check the caller does
|
||||
// next sees the real, live object.
|
||||
if err := r.Get(ctx, client.ObjectKeyFromObject(deploy), deploy); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return deploy, nil
|
||||
}
|
||||
|
||||
// reconcileService creates or updates the ClusterIP Service in front of the
|
||||
// Deployment, owned by srv.
|
||||
func (r *TerdutServerReconciler) reconcileService(ctx context.Context, srv *terdutv1alpha1.TerdutServer) error {
|
||||
svc := &corev1.Service{ObjectMeta: metav1.ObjectMeta{Name: srv.Name, Namespace: srv.Namespace}}
|
||||
_, err := controllerutil.CreateOrUpdate(ctx, r.Client, svc, func() error {
|
||||
svc.Spec.Selector = labelsFor(srv)
|
||||
svc.Spec.Ports = []corev1.ServicePort{{
|
||||
Name: "http",
|
||||
Port: servicePort(srv),
|
||||
TargetPort: intstr.FromString("http"),
|
||||
Protocol: corev1.ProtocolTCP,
|
||||
}}
|
||||
return controllerutil.SetControllerReference(srv, svc, r.Scheme)
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
func servicePort(srv *terdutv1alpha1.TerdutServer) int32 {
|
||||
if srv.Spec.Networking.ServicePort == 0 {
|
||||
return 8080
|
||||
}
|
||||
return srv.Spec.Networking.ServicePort
|
||||
}
|
||||
|
||||
func healthzProbe() *corev1.Probe {
|
||||
return &corev1.Probe{
|
||||
ProbeHandler: corev1.ProbeHandler{
|
||||
HTTPGet: &corev1.HTTPGetAction{
|
||||
Path: "/healthz",
|
||||
Port: intstr.FromString("http"),
|
||||
},
|
||||
},
|
||||
InitialDelaySeconds: 5,
|
||||
}
|
||||
}
|
||||
|
||||
// buildEnv mirrors charts/terdut-server's own deployment.yaml env block
|
||||
// field-for-field (confirmed against that source, not reconstructed from
|
||||
// DESIGN.md's illustrative YAML alone) — dbEnv (TERDUT_DB_DSN, optionally
|
||||
// PGPASSWORD) comes from resolveDatabaseEnv, since which of §8's two paths
|
||||
// produced it doesn't matter past this point.
|
||||
func buildEnv(srv *terdutv1alpha1.TerdutServer, dbEnv []corev1.EnvVar) []corev1.EnvVar {
|
||||
env := []corev1.EnvVar{{Name: "TERDUT_ADDR", Value: fmt.Sprintf(":%d", servicePort(srv))}}
|
||||
env = append(env, dbEnv...)
|
||||
|
||||
sweeper := srv.Spec.Sweeper
|
||||
env = append(env,
|
||||
corev1.EnvVar{Name: "TERDUT_STALE_AFTER", Value: sweeper.StaleAfter},
|
||||
corev1.EnvVar{Name: "TERDUT_ARCHIVE_AFTER", Value: sweeper.ArchiveAfter},
|
||||
)
|
||||
|
||||
deadman := srv.Spec.Deadman
|
||||
env = append(env,
|
||||
corev1.EnvVar{Name: "TERDUT_DEADMAN_MATCHERS", Value: deadman.Matchers},
|
||||
corev1.EnvVar{Name: "TERDUT_DEADMAN_TIMEOUT", Value: deadman.Timeout},
|
||||
corev1.EnvVar{Name: "TERDUT_DEADMAN_SEVERITY", Value: deadman.Severity},
|
||||
)
|
||||
|
||||
notify := srv.Spec.Notify
|
||||
if notify.NtfyURL != "" {
|
||||
env = append(env,
|
||||
corev1.EnvVar{Name: "TERDUT_NTFY_URL", Value: notify.NtfyURL},
|
||||
corev1.EnvVar{Name: "TERDUT_NTFY_FALLBACK_TOPIC", Value: notify.FallbackTopic},
|
||||
corev1.EnvVar{Name: "TERDUT_NOTIFY_REPEAT", Value: notify.RepeatEvery},
|
||||
)
|
||||
if notify.TokenSecretRef != nil {
|
||||
env = append(env, corev1.EnvVar{
|
||||
Name: "TERDUT_NTFY_TOKEN",
|
||||
ValueFrom: secretEnvSource(notify.TokenSecretRef),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
publicURL := fmt.Sprintf("https://%s", srv.Spec.Networking.Hostname)
|
||||
env = append(env,
|
||||
corev1.EnvVar{Name: "TERDUT_PUBLIC_URL", Value: publicURL},
|
||||
corev1.EnvVar{Name: "TERDUT_PASSWORD_LOGIN", Value: boolString(srv.Spec.PasswordLogin)},
|
||||
// Always on, unlike the chart's own default-off operatorMode: every
|
||||
// write this operator's own TerdutTeam/EscalationRule/etc.
|
||||
// controllers make goes through a service account already, and the
|
||||
// whole reason to run this operator is gitops-managed config, not a
|
||||
// human editing an operator-managed install's teams/policies by
|
||||
// hand (chart's values.yaml comment: "a statement that something
|
||||
// like terdut-operator... owns this install's configuration from
|
||||
// here on" -- which is unconditionally true for anything this
|
||||
// operator creates).
|
||||
corev1.EnvVar{Name: "TERDUT_OPERATOR_MODE", Value: "true"},
|
||||
)
|
||||
|
||||
if oidc := srv.Spec.OIDC; oidc.Enabled {
|
||||
env = append(env,
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_ISSUER", Value: oidc.Issuer},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_CLIENT_ID", Value: oidc.ClientID},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_NAME", Value: oidc.Name},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_SCOPES", Value: oidc.Scopes},
|
||||
// terdut-server's own defaults for the claims/trust-email knobs
|
||||
// the chart exposes but DESIGN.md's spec doesn't (§4.1's doc
|
||||
// comment on OIDCSpec) -- not configurable here, not an
|
||||
// oversight.
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_USERNAME_CLAIM", Value: "preferred_username"},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_EMAIL_CLAIM", Value: "email"},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_GROUPS_CLAIM", Value: "groups"},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_TRUST_EMAIL", Value: "false"},
|
||||
corev1.EnvVar{Name: "TERDUT_OIDC_SESSION_MAX_AGE", Value: oidc.SessionMaxAge},
|
||||
)
|
||||
if oidc.ClientSecretRef != nil {
|
||||
env = append(env, corev1.EnvVar{
|
||||
Name: "TERDUT_OIDC_CLIENT_SECRET",
|
||||
ValueFrom: secretEnvSource(oidc.ClientSecretRef),
|
||||
})
|
||||
}
|
||||
if len(oidc.AllowedGroups) > 0 {
|
||||
env = append(env, corev1.EnvVar{Name: "TERDUT_OIDC_ALLOWED_GROUPS", Value: strings.Join(oidc.AllowedGroups, ",")})
|
||||
}
|
||||
if oidc.AdminGroup != "" {
|
||||
env = append(env, corev1.EnvVar{Name: "TERDUT_OIDC_ADMIN_GROUP", Value: oidc.AdminGroup})
|
||||
}
|
||||
}
|
||||
|
||||
return env
|
||||
}
|
||||
|
||||
func secretEnvSource(ref *terdutv1alpha1.SecretKeyRef) *corev1.EnvVarSource {
|
||||
return &corev1.EnvVarSource{
|
||||
SecretKeyRef: &corev1.SecretKeySelector{
|
||||
LocalObjectReference: corev1.LocalObjectReference{Name: ref.Name},
|
||||
Key: ref.Key,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func boolString(b bool) string {
|
||||
if b {
|
||||
return "true"
|
||||
}
|
||||
return "false"
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
# A deliberately minimal, permissive stub of Zalando postgres-operator's
|
||||
# `postgresql` CRD -- test-only, installed into envtest so suite_test.go can
|
||||
# create fixture `postgresql.acid.zalan.do` objects without a real
|
||||
# postgres-operator controller running. This is NOT the real upstream CRD
|
||||
# (which has a much larger structural schema); it exists purely so this
|
||||
# repo's own controller logic -- which only ever Gets this object by name
|
||||
# and never inspects its spec/status fields beyond existence -- has
|
||||
# something matching the real GVK to read. A real cluster installs the
|
||||
# genuine CRD via the postgres-operator chart; this file is never applied
|
||||
# there.
|
||||
apiVersion: apiextensions.k8s.io/v1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: postgresqls.acid.zalan.do
|
||||
spec:
|
||||
group: acid.zalan.do
|
||||
names:
|
||||
kind: postgresql
|
||||
listKind: postgresqlList
|
||||
plural: postgresqls
|
||||
singular: postgresql
|
||||
scope: Namespaced
|
||||
versions:
|
||||
- name: v1
|
||||
served: true
|
||||
storage: true
|
||||
schema:
|
||||
openAPIV3Schema:
|
||||
type: object
|
||||
x-kubernetes-preserve-unknown-fields: true
|
||||
+144
-8
@@ -5,10 +5,6 @@
|
||||
// shape must be mirrored" convention) but authorizes with a Bearer API key
|
||||
// rather than a session cookie — the operator never signs in as a human
|
||||
// (DESIGN.md §6).
|
||||
//
|
||||
// Deliberately minimal for Stage 1 (ROADMAP.md): only Version, the
|
||||
// reachability probe TerdutServer's controller needs. Bootstrap and the
|
||||
// service-account endpoints land here once self-registration does (Stage 5).
|
||||
package tdclient
|
||||
|
||||
import (
|
||||
@@ -22,7 +18,7 @@ import (
|
||||
|
||||
// Client talks to one terdut-server install, optionally as a service
|
||||
// account. A zero-value token works for endpoints that don't need one
|
||||
// (Version).
|
||||
// (Version, Bootstrap).
|
||||
type Client struct {
|
||||
baseURL string
|
||||
httpClient *http.Client
|
||||
@@ -68,12 +64,29 @@ func statusError(resp *http.Response) error {
|
||||
return &StatusError{Code: resp.StatusCode, Message: e.Error}
|
||||
}
|
||||
|
||||
func (c *Client) newRequest(ctx context.Context, method, path string) (*http.Request, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, nil)
|
||||
func (c *Client) newRequest(ctx context.Context, method, path string, body any) (*http.Request, error) {
|
||||
var reader *strings.Reader
|
||||
if body != nil {
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reader = strings.NewReader(string(data))
|
||||
}
|
||||
var req *http.Request
|
||||
var err error
|
||||
if reader != nil {
|
||||
req, err = http.NewRequestWithContext(ctx, method, c.baseURL+path, reader)
|
||||
} else {
|
||||
req, err = http.NewRequestWithContext(ctx, method, c.baseURL+path, nil)
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Accept", "application/json")
|
||||
if body != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
if c.token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
}
|
||||
@@ -101,7 +114,7 @@ func (c *Client) do(req *http.Request, out any) error {
|
||||
// for it"). Used here purely as a reachability probe: a bad endpoint fails
|
||||
// here, clearly, rather than on whatever the controller tries first.
|
||||
func (c *Client) Version(ctx context.Context) (string, error) {
|
||||
req, err := c.newRequest(ctx, http.MethodGet, "/api/version")
|
||||
req, err := c.newRequest(ctx, http.MethodGet, "/api/version", nil)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
@@ -113,3 +126,126 @@ func (c *Client) Version(ctx context.Context) (string, error) {
|
||||
}
|
||||
return v.Version, nil
|
||||
}
|
||||
|
||||
// APIKey is the raw key a bootstrap or service-account-key mint hands back —
|
||||
// the one moment its value exists outside the request that generated it.
|
||||
// Mirrors terdut-server's models.APIKey/models.ServiceAccountKey shape
|
||||
// (internal/models in that repo) for the fields this client actually reads.
|
||||
type APIKey struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Key string `json:"key"`
|
||||
CreatedAt string `json:"created_at"`
|
||||
}
|
||||
|
||||
// BootstrapResult is /api/bootstrap's 201 response body.
|
||||
type BootstrapResult struct {
|
||||
User struct {
|
||||
ID int64 `json:"id"`
|
||||
Username string `json:"username"`
|
||||
Email string `json:"email"`
|
||||
} `json:"user"`
|
||||
APIKey APIKey `json:"api_key"`
|
||||
}
|
||||
|
||||
// Bootstrap calls POST /api/bootstrap — unauthenticated, single-shot per
|
||||
// install (internal/api/users.go's handleBootstrap in terdut-server:
|
||||
// gated on SELECT COUNT(*) FROM users). Returns the raw admin key directly;
|
||||
// DESIGN.md §6 has the controller use it for exactly one further call
|
||||
// (CreateServiceAccount) and discard it, never storing it as the lasting
|
||||
// credential.
|
||||
//
|
||||
// A StatusError with Code 403 means this install already has a user —
|
||||
// per this operator's design (DESIGN.md §1), that only happens if this
|
||||
// exact TerdutServer's own controller already won this race on an earlier,
|
||||
// interrupted reconcile; see the checkpoint-Secret handling in the
|
||||
// controller, not a retry loop here.
|
||||
func (c *Client) Bootstrap(ctx context.Context, username, email string) (*BootstrapResult, error) {
|
||||
req, err := c.newRequest(ctx, http.MethodPost, "/api/bootstrap", map[string]string{
|
||||
"username": username,
|
||||
"email": email,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var result BootstrapResult
|
||||
if err := c.do(req, &result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
// ServiceAccount mirrors terdut-server's models.ServiceAccount
|
||||
// (internal/models/service_account.go), minus fields this client never
|
||||
// reads.
|
||||
type ServiceAccount struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
|
||||
// CreateServiceAccountResult is POST /api/service-accounts' 201 response.
|
||||
type CreateServiceAccountResult struct {
|
||||
ServiceAccount ServiceAccount `json:"service_account"`
|
||||
Key APIKey `json:"key"`
|
||||
}
|
||||
|
||||
// CreateInstanceServiceAccount calls POST /api/service-accounts with
|
||||
// scope "instance", authenticated with c's current token (the raw admin key
|
||||
// from Bootstrap, for the operator's own first-ever call). A StatusError
|
||||
// with Code 409 means a prior, interrupted attempt already created this
|
||||
// name — DESIGN.md §6's adopt-rather-than-error rule: the caller should
|
||||
// fall back to GetServiceAccountByName + CreateServiceAccountKey, not treat
|
||||
// this as a hard failure.
|
||||
func (c *Client) CreateInstanceServiceAccount(ctx context.Context, name string) (*CreateServiceAccountResult, error) {
|
||||
req, err := c.newRequest(ctx, http.MethodPost, "/api/service-accounts", map[string]string{
|
||||
"name": name,
|
||||
"scope": "instance",
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var result CreateServiceAccountResult
|
||||
if err := c.do(req, &result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &result, nil
|
||||
}
|
||||
|
||||
// GetServiceAccountByName calls GET /api/service-accounts?name=... --
|
||||
// authenticated (terdut-server's AuthMiddleware hard-rejects any
|
||||
// unauthenticated request before this endpoint's own, more permissive
|
||||
// internal check ever runs; DESIGN.md §6). Returns nil, nil if nothing
|
||||
// matches, not an error -- the server's own distinction between "found
|
||||
// nothing" and "the call failed".
|
||||
func (c *Client) GetServiceAccountByName(ctx context.Context, name string) (*ServiceAccount, error) {
|
||||
req, err := c.newRequest(ctx, http.MethodGet, "/api/service-accounts?name="+name, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var accounts []ServiceAccount
|
||||
if err := c.do(req, &accounts); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(accounts) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
return &accounts[0], nil
|
||||
}
|
||||
|
||||
// CreateServiceAccountKey calls POST /api/service-accounts/{id}/keys to
|
||||
// mint an additional key on an existing account -- rotation (DESIGN.md §6
|
||||
// point 6), and the adopt-on-409 recovery path in point 1.
|
||||
func (c *Client) CreateServiceAccountKey(ctx context.Context, serviceAccountID int64, name string) (*APIKey, error) {
|
||||
req, err := c.newRequest(ctx, http.MethodPost,
|
||||
fmt.Sprintf("/api/service-accounts/%d/keys", serviceAccountID),
|
||||
map[string]string{"name": name})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var key APIKey
|
||||
if err := c.do(req, &key); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &key, nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user