terdut-server/SERVICE-ACCOUNTS.md

# Service accounts: a scoped, non-human credential type

This is a design note for a feature, not an implementation plan — it exists to
propose the shape before writing code. It's raised directly by `terdut-operator`
(a separate repo, no shared code — see its `DESIGN.md` §6, §9, §13), which needs
a credential for unattended, repeatable API access and currently has no good one
available. Anything automating terdut-server long-term (this operator, CI, future
integrations) hits the same gap, so this is written as a general primitive, not
operator-specific.

## The problem

terdut-server has two credential types today, and neither fits "an unattended
process that manages teams/schedules/policies on someone's behalf":

- **User API keys** (`api_keys`, `internal/api/users.go`) are always tied to a
  real `users` row and carry that user's full rights — every team they're a
  member of, their admin flag if set. There's no `kind`/`service` marker
  distinguishing "a human's personal automation key" from "a login session," and
  no way to mint one scoped to less than the full user.
- **Integration keys** (`integrations`, `internal/api/*teams*.go`) are team-scoped,
  but narrowly: they authenticate exactly one inbound Alertmanager webhook call
  (`POST /api/integrations/{key}/alertmanager`) and nothing else. They're not a
  general management-API credential and shouldn't become one — overloading a
  narrow, one-way ingestion credential with broad read/write access would weaken
  the one property that makes it safe to embed in an Alertmanager config today.

The result: any automation that needs to create teams, set escalation policies,
manage dead-man switches, or rotate integration keys has to hold a real human
admin's or team owner's API key. That key is exactly as powerful as that person
logging in — full team access, and full instance access if they're an admin.
`terdut-operator`'s design ran directly into this (its DESIGN.md §6): its
described bootstrap/rotation flow assumed a repeatable, identity-scoped way to
get a credential, and `/api/bootstrap`'s actual behavior (single-shot per
install, gated on `COUNT(*) FROM users`, confirmed via `internal/api/users.go`
and `charts/terdut-server/templates/bootstrap-job.yaml`) doesn't provide one —
it mints exactly one founding admin, once, ever.

## Goals

- A credential type that isn't a human: doesn't touch OIDC group sync, login,
  session, or the `is_admin`/account-management semantics that come with a real
  `users` row.
- Two scopes matching the two shapes automation actually needs: instance-wide
  (create/list teams — what a server-owning controller needs) and team-scoped
  (manage one team's escalation policy, dead-man switches, integrations,
  schedule, OIDC group bindings — what a per-team controller or integration
  needs).
- Repeatable issuance and rotation — unlike `/api/bootstrap`, callable more than
  once, by anything that already holds admin rights, without destroying and
  recreating state to get a fresh credential.
- Visibly distinct from a human in every place identity shows up (audit trails,
  timeline entries, UI attribution) — a service account acting on a team should
  never be indistinguishable from a person.

## Non-goals

- Not a general OAuth2/OIDC client-credentials flow — this is a bearer-token
  primitive matching the shape `api_keys` already uses (SHA-256 hash stored,
  raw key shown once at creation), not a new auth protocol.
- Not replacing integration keys — those stay as the narrow, one-way webhook
  credential they are today.
- Not modeling per-endpoint or per-verb permissions within a scope — `instance`
  and `team` are the only two scopes for now; finer-grained scoping is future
  work if a real need shows up.

## Proposed shape

### Schema

```sql
CREATE TABLE service_accounts (
    id          BIGSERIAL PRIMARY KEY,
    name        TEXT NOT NULL UNIQUE,       -- e.g. "terdut-operator"
    scope       TEXT NOT NULL CHECK (scope IN ('instance', 'team')),
    team_id     BIGINT REFERENCES teams(id) ON DELETE CASCADE,
    -- team_id required iff scope = 'team'; NULL iff scope = 'instance'
    created_by  BIGINT REFERENCES users(id),
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE service_account_keys (
    id                  BIGSERIAL PRIMARY KEY,
    service_account_id  BIGINT NOT NULL REFERENCES service_accounts(id) ON DELETE CASCADE,
    key_hash            TEXT NOT NULL UNIQUE,
    name                TEXT NOT NULL,       -- e.g. "initial", "2026-Q4-rotation"
    created_at           TIMESTAMPTZ NOT NULL DEFAULT now(),
    last_used_at         TIMESTAMPTZ
);
```

Deliberately not a `users` row: no `password_hash`, no `is_admin`, no
`user_identities` linkage, so it's structurally impossible for a service account
to be pulled into OIDC group sync or password login. Multiple keys per account
(mirroring `api_keys`' existing one-user-many-keys shape) so rotation is "mint a
new key, revoke the old one," not "recreate the account."

### Endpoints

- `POST /api/service-accounts` — instance-scope/admin-only. Body:
  `{"name": ..., "scope": "instance"|"team", "teamID": ... }` (teamID required
  iff scope=team, and caller must be that team's owner or a system admin).
  Returns the account plus its first raw key (shown once, same pattern as
  `POST /api/users/{id}/api-keys`). Safe to call again with the same `name` —
  see "idempotent lookup" below — unlike `/api/bootstrap`, which is inherently
  one-shot by design (it's answering "does any user exist yet," a question with
  no analogue once one already does).
- `POST /api/service-accounts/{id}/keys` — mint an additional key on an existing
  account (self-service-equivalent: instance admin for `instance` scope, team
  owner or system admin for `team` scope). Enables rotation without recreating
  the account or losing its identity/audit history.
- `DELETE /api/service-accounts/{id}/keys/{keyID}` — revoke one key, mirroring
  `DELETE /api/users/{id}/api-keys/{keyID}`.
- `GET /api/service-accounts?name=` — look up an existing account by name.
  This is what turns "I tried to create my account and got a conflict" into a
  normal flow instead of an error: a controller that expects to have already
  registered itself calls this first, and only falls through to `POST` if
  nothing comes back.

### Auth middleware

`internal/api/middleware.go`'s existing dual resolution (`Authorization: Bearer`
→ `apiKeyUser()`, or session cookie → `sessionUser()`, both landing on the same
`models.User` + team-membership context) gains a third path: a bearer token that
hashes to a `service_account_keys.key_hash` resolves to a distinct principal
type, not a synthesized `models.User`. `requireTeamMember`/`requireTeamOwner`
treat a matching team-scoped service account as owner-equivalent for that one
team (satisfies the same checks a real team owner would), and an instance-scoped
one as satisfying `AdminOnly` for team-creation/listing purposes **and** for
minting a `team`-scoped service account against any team (`POST
/api/service-accounts {"scope":"team","teamID":...}`) — this second permission
is what lets an operator-style caller create a team, then immediately mint that
team its own narrower credential, without a human in the loop for every team.
Neither permission extends to user-management endpoints (`POST /api/users`,
`PUT /api/users/{id}/admin`, etc.), which stay human-admin-only. Anywhere
identity is recorded for a human (incident timeline
`acknowledged_by`/`assigned_to`, audit-relevant fields), a service-account
principal is stored and displayed distinctly, e.g. `service-account:terdut-operator`,
never coerced into a `user_id` FK.

**Team scope, as implemented, is owner-equivalent for every `requireTeamOwner`
endpoint, membership and invites included — nothing server-side carves those
two out.** That's broader than what `terdut-operator`'s CRDs actually need
(escalation/deadman/integrations/OIDC-bindings only; membership is explicitly
never gitops-managed, see its DESIGN.md §4.2), a gap acknowledged rather than
closed here: narrowing this to exclude
`POST/DELETE /api/teams/{teamID}/members*` and
`.../invites*` specifically for a service-account caller is a small, isolated
follow-up (special-case those handlers rather than `requireTeamOwner` itself,
which every other owner-gated endpoint still wants shared). Until then, what
actually keeps membership out of automation's hands is that no operator built
against this scope should ever call those two endpoints — not a server-side
refusal.

## What this unblocks

Directly resolves `terdut-operator` DESIGN.md §6's two broken assumptions:
1. **Bootstrap becomes single-purpose again.** `/api/bootstrap` mints exactly
   the founding human admin, once. The operator's actual first-reconcile flow:
   call `/api/bootstrap` only on a genuinely empty install; otherwise (or
   immediately after, if it won the bootstrap race) call
   `GET /api/service-accounts?name=terdut-operator`, and `POST` one if it
   doesn't exist yet. From then on the operator never touches `/api/bootstrap`
   again.
2. **Rotation becomes real.** `POST /api/service-accounts/{id}/keys` + revoke the
   old one — no destructive DB-level workaround, no re-triggering a single-shot
   endpoint that can't fire twice.
3. **Cross-namespace credential mirroring is no longer needed at all.**
   `terdut-operator`'s current design holds every credential — instance- and
   team-scoped alike — privately in the operator's own namespace, never in
   the namespace of the CR each one authenticates for; reconciliation happens
   entirely inside the operator's controller loop, so no CR owner ever needs
   read access to a terdut-server credential regardless of same- or
   cross-namespace `serverRef`. Team scoping is still what bounds the blast
   radius of any individual credential: a leaked team-scoped key exposes
   exactly one team's resources, never the whole server, which is what makes
   holding many credentials in one place (the operator's namespace) an
   acceptable trade rather than reintroducing the mirrored design's
   server-admin-equivalent-everywhere problem.

## Suggested sequencing

Land this before `terdut-operator` implements any bootstrap/credential-handling
code — that code would otherwise be written against the current one-shot,
user-only credential model as a known-temporary workaround, which is wasted
effort on a repo that currently has zero implementation to begin with.