terdut-server/SERVICE-ACCOUNTS.md

# Service accounts: a scoped, non-human credential type

This is a design note for a feature, not an implementation plan — it exists to
propose the shape before writing code. It's raised directly by `terdut-operator`
(a separate repo, no shared code — see its `DESIGN.md` §6, §9, §13), which needs
a credential for unattended, repeatable API access and currently has no good one
available. Anything automating terdut-server long-term (this operator, CI, future
integrations) hits the same gap, so this is written as a general primitive, not
operator-specific.

## The problem

terdut-server has two credential types today, and neither fits "an unattended
process that manages teams/schedules/policies on someone's behalf":

- **User API keys** (`api_keys`, `internal/api/users.go`) are always tied to a
  real `users` row and carry that user's full rights — every team they're a
  member of, their admin flag if set. There's no `kind`/`service` marker
  distinguishing "a human's personal automation key" from "a login session," and
  no way to mint one scoped to less than the full user.
- **Integration keys** (`integrations`, `internal/api/*teams*.go`) are team-scoped,
  but narrowly: they authenticate exactly one inbound Alertmanager webhook call
  (`POST /api/integrations/{key}/alertmanager`) and nothing else. They're not a
  general management-API credential and shouldn't become one — overloading a
  narrow, one-way ingestion credential with broad read/write access would weaken
  the one property that makes it safe to embed in an Alertmanager config today.

The result: any automation that needs to create teams, set escalation policies,
manage dead-man switches, or rotate integration keys has to hold a real human
admin's or team owner's API key. That key is exactly as powerful as that person
logging in — full team access, and full instance access if they're an admin.
`terdut-operator`'s design ran directly into this (its DESIGN.md §6): its
described bootstrap/rotation flow assumed a repeatable, identity-scoped way to
get a credential, and `/api/bootstrap`'s actual behavior (single-shot per
install, gated on `COUNT(*) FROM users`, confirmed via `internal/api/users.go`
and `charts/terdut-server/templates/bootstrap-job.yaml`) doesn't provide one —
it mints exactly one founding admin, once, ever.

## Goals

- A credential type that isn't a human: doesn't touch OIDC group sync, login,
  session, or the `is_admin`/account-management semantics that come with a real
  `users` row.
- Two scopes matching the two shapes automation actually needs: instance-wide
  (create/list teams — what a server-owning controller needs) and team-scoped
  (manage one team's escalation policy, dead-man switches, integrations,
  schedule, OIDC group bindings — what a per-team controller or integration
  needs).
- Repeatable issuance and rotation — unlike `/api/bootstrap`, callable more than
  once, by anything that already holds admin rights, without destroying and
  recreating state to get a fresh credential.
- Visibly distinct from a human in every place identity shows up (audit trails,
  timeline entries, UI attribution) — a service account acting on a team should
  never be indistinguishable from a person.

## Non-goals

- Not a general OAuth2/OIDC client-credentials flow — this is a bearer-token
  primitive matching the shape `api_keys` already uses (SHA-256 hash stored,
  raw key shown once at creation), not a new auth protocol.
- Not replacing integration keys — those stay as the narrow, one-way webhook
  credential they are today.
- Not modeling per-endpoint or per-verb permissions within a scope — `instance`
  and `team` are the only two scopes for now; finer-grained scoping is future
  work if a real need shows up.

## Proposed shape

### Schema

```sql
CREATE TABLE service_accounts (
    id          BIGSERIAL PRIMARY KEY,
    name        TEXT NOT NULL UNIQUE,       -- e.g. "terdut-operator"
    scope       TEXT NOT NULL CHECK (scope IN ('instance', 'team')),
    team_id     BIGINT REFERENCES teams(id) ON DELETE CASCADE,
    -- team_id required iff scope = 'team'; NULL iff scope = 'instance'
    created_by  BIGINT REFERENCES users(id),
    created_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE service_account_keys (
    id                  BIGSERIAL PRIMARY KEY,
    service_account_id  BIGINT NOT NULL REFERENCES service_accounts(id) ON DELETE CASCADE,
    key_hash            TEXT NOT NULL UNIQUE,
    name                TEXT NOT NULL,       -- e.g. "initial", "2026-Q4-rotation"
    created_at           TIMESTAMPTZ NOT NULL DEFAULT now(),
    last_used_at         TIMESTAMPTZ
);
```

Deliberately not a `users` row: no `password_hash`, no `is_admin`, no
`user_identities` linkage, so it's structurally impossible for a service account
to be pulled into OIDC group sync or password login. Multiple keys per account
(mirroring `api_keys`' existing one-user-many-keys shape) so rotation is "mint a
new key, revoke the old one," not "recreate the account."

### Endpoints

- `POST /api/service-accounts` — instance-scope/admin-only. Body:
  `{"name": ..., "scope": "instance"|"team", "teamID": ... }` (teamID required
  iff scope=team, and caller must be that team's owner or a system admin).
  Returns the account plus its first raw key (shown once, same pattern as
  `POST /api/users/{id}/api-keys`). Safe to call again with the same `name` —
  see "idempotent lookup" below — unlike `/api/bootstrap`, which is inherently
  one-shot by design (it's answering "does any user exist yet," a question with
  no analogue once one already does).
- `POST /api/service-accounts/{id}/keys` — mint an additional key on an existing
  account (self-service-equivalent: instance admin for `instance` scope, team
  owner or system admin for `team` scope). Enables rotation without recreating
  the account or losing its identity/audit history.
- `DELETE /api/service-accounts/{id}/keys/{keyID}` — revoke one key, mirroring
  `DELETE /api/users/{id}/api-keys/{keyID}`.
- `GET /api/service-accounts?name=` — look up an existing account by name.
  This is what turns "I tried to create my account and got a conflict" into a
  normal flow instead of an error: a controller that expects to have already
  registered itself calls this first, and only falls through to `POST` if
  nothing comes back.

### Auth middleware

**Revised** (this section originally described an aspiration that didn't
match what shipped — `TEAM-LOOKUP.md` already caught one instance of that,
and a fuller audit found three more; this is the corrected, as-built
description, not the original proposal).

`internal/api/middleware.go`'s dual resolution (`Authorization: Bearer` →
`apiKeyUser()`, or session cookie → `sessionUser()`) and the service-account
path (`serviceAccountFor()`) both resolve into one `Caller` type
(`internal/api/caller.go`), not two parallel, un-unified context
representations the way an earlier version of this server kept them. Every
authorization predicate reads `Caller`'s methods:

- `Caller.IsAdmin()` — true **only** for a human system administrator, never
  for a service account of either scope, under any circumstance. `AdminOnly`
  and `requireSelfOrAdmin` key on this alone — user management
  (`POST /api/users`, `PUT /api/users/{id}/admin`, etc.) and
  `GET/PUT /api/admin/settings` stay human-only, forever. The original text
  here claimed an instance-scoped service account satisfies `AdminOnly` "for
  team-creation/listing purposes" — that was never true of the shipped code
  (`TEAM-LOOKUP.md` caught the listing half; the creation half was always a
  separate, bespoke check in `handleCreateTeam`, not `AdminOnly` itself) and
  is not being made true now. Don't widen `AdminOnly`: every time this has
  come up, the fix has been a narrower, purpose-built capability instead
  (`?name=` lookups for teams and service accounts; now
  `terdut-operator`'s own invite-minting feature for the one real gap this
  boundary left — how a human ever gets a first login on a no-OIDC,
  operator-managed install. See the bottom of "What this unblocks.")
- `Caller.IsInstanceServiceAccount()` — true only for an instance-scoped
  service account, never for a human (including a human admin).
  `handleCreateTeam` uses exactly this: a human creates a team by being a
  human (and becomes its owner); an instance-scoped service account creates
  one with no human owner at all. The two paths are not interchangeable, so
  this predicate deliberately does not also admit a human admin.
- `Caller.Role(teamID)`/`TeamIDs()` — a human's real `team_members` rows, or
  a team-scoped service account's single synthetic owner membership
  (`serveAsServiceAccount`). This is what makes `requireTeamMember`/
  `requireTeamOwner` treat a team-scoped service account as owner-equivalent
  for that one team, with no separate branch needed in either function.
- `Caller.ServiceAccountID()` — used by `OperatorModeBlock` ("any service
  account passes") and by `callerMayManageServiceAccount`'s self-rotation
  check.
- `Caller.AsHuman()` — the accessor every handler that needs a real
  `user_id` to act on behalf of must call and check, instead of reading a
  user off context unconditionally. Before the `Caller` type existed, four
  handlers did the latter and silently misbehaved for a service-account
  caller: `handleMe` and `handleTestNotification` 500'd (a zero-value user id
  that matches no row), `handleDismissOnboarding` silently no-op'd (`UPDATE
  ... WHERE id = 0` affects nothing, still returns 204), and
  `handleCreateInvite` wrote that same zero value into `invites.created_by`
  — a real foreign-key violation, not just a wrong answer, since that column
  is nullable but was never passed as `nil`. All four now call `AsHuman()`
  and return an explicit 403 ("this endpoint is for human accounts only")
  or, for the invite case, leave `created_by` `NULL` the same way
  `handleCreateServiceAccount` already did for the analogous situation.

**Team scope is owner-equivalent for every `requireTeamOwner` endpoint,
membership and invites included — by design, not by an unclosed gap.** An
earlier version of this document flagged this as "acknowledged rather than
closed," kept in check only by the social convention that nobody *builds*
automation against those two routes. That convention is retired:
`terdut-operator`'s `TerdutTeam` controller now mints and revokes its own
team's invite link through exactly this capability (its existing
team-scoped credential, `POST`/`DELETE /api/teams/{teamID}/invites`), which
is the real fix for the human-onboarding gap below — not a narrower
carve-out of this capability. `service_accounts_test.go`'s
`TestServiceAccount_TeamScopeManagesItsOwnInvites` pins it.

**A team-scoped account can also mint another service account scoped to its
own team** (`handleCreateServiceAccount`'s `callerOwnsTeam` branch, which a
team-scoped caller already satisfies for its own team via the synthetic
membership above). Kept, not restricted, for the same reason: a team-scoped
credential is that team's owner's reach, full stop — carving this one
capability out while leaving membership/invites alone would be an arbitrary
asymmetry. Pinned by
`TestServiceAccount_TeamScopeCanMintAnotherAccountForItsOwnTeam`.

**`callerMayManageServiceAccount` gained the one load-bearing fix this
redesign exists for:** an instance-scoped service account may manage
(mint/revoke a key on) *any* team-scoped account, not only one admin, that
team's human owner, or the account itself. `handleCreateServiceAccount`
already let an instance-scoped caller *create* a team-scoped account for
any team; this closes the gap where adopting or rotating one it didn't just
create in the same call — exactly `terdut-operator`'s documented
adopt-on-409 crash-window recovery (its own `DESIGN.md` §5) — 403'd forever
instead of succeeding (`terdut-operator#3`). Pinned by
`TestServiceAccount_InstanceScopeAdoptsAnExistingTeamScopedAccountsKey`.

Anywhere identity is recorded for a human (incident timeline
`acknowledged_by`/`assigned_to`, audit-relevant fields), a service-account
caller is still coerced into a bare `user_id` of `0` today — `Caller`'s new
`Identity()` accessor exists for exactly this follow-up, but wiring it in
needs a schema migration (an actor-attribution column distinct from
`user_id`) and is deliberately out of scope here. Tracked separately, not by
this document.

## What this unblocks

Directly resolves `terdut-operator` DESIGN.md §6's two broken assumptions:
1. **Bootstrap becomes single-purpose again.** `/api/bootstrap` mints exactly
   the founding human admin, once. The operator's actual first-reconcile flow:
   call `/api/bootstrap` only on a genuinely empty install; otherwise (or
   immediately after, if it won the bootstrap race) call
   `GET /api/service-accounts?name=terdut-operator`, and `POST` one if it
   doesn't exist yet. From then on the operator never touches `/api/bootstrap`
   again.
2. **Rotation becomes real.** `POST /api/service-accounts/{id}/keys` + revoke the
   old one — no destructive DB-level workaround, no re-triggering a single-shot
   endpoint that can't fire twice.
3. **Cross-namespace credential mirroring is no longer needed at all.**
   `terdut-operator`'s current design holds every credential — instance- and
   team-scoped alike — privately in the operator's own namespace, never in
   the namespace of the CR each one authenticates for; reconciliation happens
   entirely inside the operator's controller loop, so no CR owner ever needs
   read access to a terdut-server credential regardless of same- or
   cross-namespace `serverRef`. Team scoping is still what bounds the blast
   radius of any individual credential: a leaked team-scoped key exposes
   exactly one team's resources, never the whole server, which is what makes
   holding many credentials in one place (the operator's namespace) an
   acceptable trade rather than reintroducing the mirrored design's
   server-admin-equivalent-everywhere problem.
4. **A human can get a first login on a no-OIDC, operator-managed install —
   without ever touching `AdminOnly` or `/api/admin/settings`.** This was
   filed as `terdut-server#23` ("no API path to create a human login after
   bootstrap") and diagnosed, at the time, as this server needing to let a
   service account through `AdminOnly`. It doesn't: the fix lives entirely
   in `terdut-operator`, because a team-scoped credential was *already*
   owner-equivalent for `POST /api/teams/{teamID}/invites`, and invite
   redemption (`POST /api/signup` with an `invite` token) bypasses
   `signup_mode` entirely — `terdut-operator` just never grew a feature to
   use either fact. Its `TerdutTeam` controller now mints and surfaces one
   via its own existing team-scoped credential (`spec.invite`,
   `status.inviteSecretRef`, see that repo's own docs), so a human joins a
   CRD-managed team by a real invite link, the same way anyone else would.
   `terdut-server#23` is closed with this note once that feature ships — its
   named routes stay human-only, correctly, not a gap.

## Suggested sequencing

Land this before `terdut-operator` implements any bootstrap/credential-handling
code — that code would otherwise be written against the current one-shot,
user-only credential model as a known-temporary workaround, which is wasted
effort on a repo that currently has zero implementation to begin with.