Give each team its own dead man's switches, and the UI a team to show #12

2026-09-20T13:18:37Z

niklas commented

2026-09-20 13:18:37 +00:00

Closes #4. Stacked on #11, which is stacked on #10 — merge in order and each retargets.

Two halves that belong together because they are the same sentence from opposite ends: a team decides which of its alerts are heartbeats, and the UI has to be able to say which team it is talking about.

Dead man's switches become a team's own

They were three environment variables, which made them one setting for the whole install — the last piece of the alerting path a team could not control. It could take its own alerts on its own key and still not say which of them were heartbeats, or how long a silence had to last.

Now a row per team, edited by an owner through PUT /api/teams/{teamID}/deadman, with the sweeper running each team against its own matchers, timeout and severity.

The environment variables become the starting point, not the setting. Every team without a configuration is seeded from them at startup, so an upgrade keeps watching exactly what it was watching, and seeding never overwrites — a redeploy must not put the environment's value back over an owner's edit.

A team created later watches nothing until somebody says otherwise. Inheriting an install-wide heartbeat would page a new team about a source it has never heard of, and a switch nobody chose is the kind that gets muted rather than fixed.

A matcher string with no alertname is refused rather than stored — storing it produces a switch that watches nothing silently, which is the exact failure the feature exists to prevent.

NewRouter and Sweep lose their DeadmanConfig parameter: there is no longer one answer to hand them.

The UI gets a team

A team badge on each queue row.
A team chip per team in the filter row, defaulting to all of them.
"On call now" as one card per team.

All three appear only when the viewer is in more than one team. Otherwise they are the same word repeated down a list — noise rather than information — and the single-team install reads exactly as it did before teams existed.

Verified

make fmt lint test helm-lint green with -race against Postgres 17, plus three new tests: the same alert treated as a heartbeat in one team and an ordinary problem in another (and silence paging only the watching team); owner-only editing with member-readable access; and an unusable matcher rejected with 400.

Also driven against a live two-team server: the combined queue labelled by team, the team_id filter, the default team seeded from the environment, and a newly created team starting with empty switches.

https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7

Closes #4. **Stacked on #11**, which is stacked on #10 — merge in order and each retargets. Two halves that belong together because they are the same sentence from opposite ends: a team decides which of its alerts are heartbeats, and the UI has to be able to say which team it is talking about. ### Dead man's switches become a team's own They were three environment variables, which made them one setting for the whole install — the last piece of the alerting path a team could not control. It could take its own alerts on its own key and still not say which of them were heartbeats, or how long a silence had to last. Now a row per team, edited by an owner through `PUT /api/teams/{teamID}/deadman`, with the sweeper running each team against its own matchers, timeout and severity. **The environment variables become the starting point, not the setting.** Every team without a configuration is seeded from them at startup, so an upgrade keeps watching exactly what it was watching, and seeding never overwrites — a redeploy must not put the environment's value back over an owner's edit. **A team created later watches nothing** until somebody says otherwise. Inheriting an install-wide heartbeat would page a new team about a source it has never heard of, and a switch nobody chose is the kind that gets muted rather than fixed. **A matcher string with no `alertname` is refused rather than stored** — storing it produces a switch that watches nothing silently, which is the exact failure the feature exists to prevent. `NewRouter` and `Sweep` lose their `DeadmanConfig` parameter: there is no longer one answer to hand them. ### The UI gets a team - A team badge on each queue row. - A team chip per team in the filter row, defaulting to all of them. - "On call now" as one card per team. All three appear **only when the viewer is in more than one team**. Otherwise they are the same word repeated down a list — noise rather than information — and the single-team install reads exactly as it did before teams existed. ### Verified `make fmt lint test helm-lint` green with `-race` against Postgres 17, plus three new tests: the same alert treated as a heartbeat in one team and an ordinary problem in another (and silence paging only the watching team); owner-only editing with member-readable access; and an unusable matcher rejected with `400`. Also driven against a live two-team server: the combined queue labelled by team, the `team_id` filter, the default team seeded from the environment, and a newly created team starting with empty switches. https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7

niklas added 1 commit 2026-09-20 13:18:38 +00:00

Give each team its own dead man's switches, and the UI a team to show

CI / chart (pull_request) Successful in 1s

Details

CI / security (pull_request) Successful in 14s

Details

CI / test (pull_request) Successful in 1m57s

Details

74359c72ab

The rest of #4. Two halves that belong together because they are the
same sentence from opposite ends: a team decides which of its alerts are
heartbeats, and the UI has to be able to say which team it is talking
about.

Switches were three environment variables, which made them one setting
for the whole install. That was the last piece of the alerting path a
team could not control: it could take its own alerts on its own key and
still not say which of them were heartbeats, or how long a silence had
to last. They are a row per team now, edited by an owner through
PUT /api/teams/{teamID}/deadman, and the sweeper runs each team against
its own matchers, timeout and severity.

The environment variables become the starting point rather than the
setting. Every team without a configuration is seeded from them at
startup, so an upgrade keeps watching exactly what it was watching, and
SeedDeadmanConfigs never overwrites -- a redeploy must not put the
environment's value back over an owner's edit. A team created later
watches nothing until somebody says otherwise: inheriting an
install-wide heartbeat would page a new team about a source it has never
heard of, and a switch nobody chose is the kind that gets muted rather
than fixed.

A matcher string with no alertname in it is refused at the door instead
of stored. Storing it would produce a switch that watches nothing
silently, which is the exact failure the feature exists to prevent.

NewRouter and Sweep lose their DeadmanConfig parameter -- there is no
longer one answer to hand them. The type stays, because parsing a
matcher string is still parsing a matcher string.

The UI side: rows in the queue carry a team badge, the filter row gains
a team chip per team, and "on call now" shows one card per team. All
three appear only when the viewer is in more than one team -- otherwise
they are the same word repeated down a list, which is noise rather than
information, and the single-team install reads exactly as it did before
teams existed.

Verified against a live two-team server as well as in tests: the
combined queue labelled by team, the team_id filter, a heartbeat that is
a heartbeat in one team and an ordinary alert in another, and a new
team's switches starting empty while the upgraded team keeps the
environment's.

Claude-Session: https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7

niklas merged commit 5227eb0d5f into teams

2026-09-20 13:28:31 +00:00

niklas referenced this pull request

2026-09-20 13:29:04 +00:00

Per-team dead man's switches, and the UI's team badge, filter and cards #13

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: niklas/terdut-server#12