Teams: scope data per team, per-team integration keys, per-team dead-man switches #4

Closed
opened 2026-09-20 05:58:54 +00:00 by niklas · 1 comment
Owner

Part of #1. Depends on Postgres and the admin role. This is the core of the multi-tenancy work.

Today

There is no team, org or tenant concept. alerts, incidents, schedule_entries and api_keys are global, and ingest comes through a single unauthenticated POST /api/alertmanager/webhook (internal/api/alertmanager.go).

Scope

Model

  • teams and team_members (user, team, role owner|member).
  • A team_id on incidents, alerts, schedule_entries, integrations, and the notification outbox as needed.
  • Migration: create a default team, move all existing rows into it, make every existing user a member, and owners where that makes sense.

Ingestion

  • An integrations table: team, kind (alertmanager to start), name, key hash, created/last-used.
  • POST /api/integrations/{key}/alertmanager replaces the global webhook, and resolves the team from the key. This closes the unauthenticated-ingest gap.
  • Decide what happens to the old path: kept as deprecated and routed to the default team for one release, or removed outright.

Scoping

  • Every read and write in internal/api/ filters by the caller's team membership. The incident, alert, schedule and on-call queries in incident_store.go all need it.
  • An API key is scoped to one team.
  • owner gates team config (escalation, schedules, integrations, fallback topic, dead-man, membership); member covers incident work (ack, assign, snooze, resolve, note).
  • Incidents never move between teams; the team is fixed at creation.

Schedules

  • Scope schedule_entries to a team, and allow several schedules per team. Keep the one-person-per-UTC-day model; rotations are a separate issue.

Dead-man switches

  • Move from TERDUT_DEADMAN_MATCHERS/_TIMEOUT/_SEVERITY into per-team rows. A heartbeat belongs to the team whose integration received it, and a missed one opens an incident there (internal/api/deadman.go, archiver.go).
  • Migrate the existing env config into the default team.

UI

  • One combined queue across the user's teams, with a team badge per incident and a team filter (internal/web/static/js/queue.js, incident.js, oncall.js).
  • Per-team settings pages.

Done when

Two teams can coexist with no data leaking between them, an alert posted to a team's integration key opens an incident only in that team, and the existing install migrates into a default team with nothing lost.

Part of #1. Depends on Postgres and the admin role. This is the core of the multi-tenancy work. ### Today There is no team, org or tenant concept. `alerts`, `incidents`, `schedule_entries` and `api_keys` are global, and ingest comes through a single **unauthenticated** `POST /api/alertmanager/webhook` (`internal/api/alertmanager.go`). ### Scope **Model** - [ ] `teams` and `team_members` (user, team, role `owner`|`member`). - [ ] A `team_id` on `incidents`, `alerts`, `schedule_entries`, `integrations`, and the notification outbox as needed. - [ ] Migration: create a default team, move all existing rows into it, make every existing user a member, and owners where that makes sense. **Ingestion** - [ ] An `integrations` table: team, kind (`alertmanager` to start), name, key hash, created/last-used. - [ ] `POST /api/integrations/{key}/alertmanager` replaces the global webhook, and resolves the team from the key. This closes the unauthenticated-ingest gap. - [ ] Decide what happens to the old path: kept as deprecated and routed to the default team for one release, or removed outright. **Scoping** - [ ] Every read and write in `internal/api/` filters by the caller's team membership. The incident, alert, schedule and on-call queries in `incident_store.go` all need it. - [ ] An API key is scoped to one team. - [ ] `owner` gates team config (escalation, schedules, integrations, fallback topic, dead-man, membership); `member` covers incident work (ack, assign, snooze, resolve, note). - [ ] Incidents never move between teams; the team is fixed at creation. **Schedules** - [ ] Scope `schedule_entries` to a team, and allow several schedules per team. Keep the one-person-per-UTC-day model; rotations are a separate issue. **Dead-man switches** - [ ] Move from `TERDUT_DEADMAN_MATCHERS`/`_TIMEOUT`/`_SEVERITY` into per-team rows. A heartbeat belongs to the team whose integration received it, and a missed one opens an incident there (`internal/api/deadman.go`, `archiver.go`). - [ ] Migrate the existing env config into the default team. **UI** - [ ] One combined queue across the user's teams, with a team badge per incident and a team filter (`internal/web/static/js/queue.js`, `incident.js`, `oncall.js`). - [ ] Per-team settings pages. ### Done when Two teams can coexist with no data leaking between them, an alert posted to a team's integration key opens an incident only in that team, and the existing install migrates into a default team with nothing lost.
niklas added the databaseteamsui labels 2026-09-20 06:13:02 +00:00
Author
Owner

Merged to main as of 05f8222 (#10, #11, #13), and the gate is green there: make fmt lint test helm-lint with -race against Postgres 17.

One clause of "Done when" is not met yet, and cannot be from a merge:

...and the existing install migrates into a default team with nothing lost.

Nothing is released or deployed. The running install is still on v0.11.1, which predates teams — migrations 002_admin_role, 003_teams and 004_team_deadman have only ever run against test databases and a scratch one. Closing this by the Closes #4 line is therefore accurate about the code and ahead of the deployment, the same way #2's auto-close was.

Left to do, and worth tracking wherever it belongs rather than being assumed:

  • Release a version carrying these migrations, and bump the wrapper chart.
  • After Flux reconciles, confirm the default team exists, holds the existing data, and that the current users are owners of it.
  • Point Alertmanager at a team integration key. Until that happens the deprecated unauthenticated POST /api/alertmanager/webhook is what keeps delivery working, routed to the oldest team.
  • TERDUT_DEADMAN_MATCHERS and friends stay in the chart: they seed the default team's switches on first start, after which the team owns them.
Merged to `main` as of `05f8222` (#10, #11, #13), and the gate is green there: `make fmt lint test helm-lint` with `-race` against Postgres 17. **One clause of "Done when" is not met yet, and cannot be from a merge:** > ...and the existing install migrates into a default team with nothing lost. Nothing is released or deployed. The running install is still on **v0.11.1**, which predates teams — migrations `002_admin_role`, `003_teams` and `004_team_deadman` have only ever run against test databases and a scratch one. Closing this by the `Closes #4` line is therefore accurate about the code and ahead of the deployment, the same way #2's auto-close was. Left to do, and worth tracking wherever it belongs rather than being assumed: - Release a version carrying these migrations, and bump the wrapper chart. - After Flux reconciles, confirm the default team exists, holds the existing data, and that the current users are owners of it. - Point Alertmanager at a team integration key. Until that happens the deprecated unauthenticated `POST /api/alertmanager/webhook` is what keeps delivery working, routed to the oldest team. - `TERDUT_DEADMAN_MATCHERS` and friends stay in the chart: they seed the default team's switches on first start, after which the team owns them.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: niklas/terdut-server#4