15 Commits

Author	SHA1	Message	Date
Niklas Ye	a27ff49171	Sign in through an OpenID Connect provider, and from a terminal ... terdut can now sign people in through any OIDC provider (written against Authentik), and let groups at the provider decide who may sign in, which teams they belong to and whether they administer the install. Password login keeps working alongside it; TERDUT_PASSWORD_LOGIN=false turns it off, and is refused at startup unless SSO is configured. With no TERDUT_OIDC_* setting nothing changes, so every existing install behaves as before. Identity is (issuer, subject), never email or username: those are mutable at the provider and a recycled address must not inherit an account. An existing user is linked by email only when the provider marks it verified, or TERDUT_OIDC_TRUST_EMAIL is set, which Authentik needs. Group grants are marked source='oidc' on team_members and users, and the sync changes only those rows. Hand-made memberships and administrators are left alone, and the sync bypasses the last-owner and last-admin guards because the provider is the source of truth for what it grants. Editing managed access by hand is refused with 409, since the next sign-in would undo it. The web UI badges it as SSO and disables the controls. Groups are read only at sign-in, so an SSO session carries a hard ceiling (sessions.max_expires_at, 12h by default) that sliding never extends. There is no refresh token, which means API keys of somebody removed at the provider stay valid until an administrator disables the user. That is accepted and documented, not fixed. A client with no browser, the TUI over SSH, signs in with a device code run by terdut itself (POST /api/oidc/device and /device/token), so the terminal never talks to the provider and ends up with the ordinary terdut_session cookie. Only a browser session can approve a code; an API key cannot. /device?code= sends a signed-out visitor through sign-in and back, which is what oidc_logins.next is for. oauth2 is pinned to v0.36.0: v0.37 needs Go 1.26 and the Dockerfile builds on 1.25. Migrations 011 and 012 add tables and defaulted columns only.	2026-09-26 21:37:40 +02:00
Niklas Ye	60ebb75cd2	Show notes from similar earlier incidents ... Each incident gets a signature: the alert name plus the group labels that say what is broken, minus the ones that only say where it ran (instance, pod, container, ...). GET /api/incidents/{id}/similar returns resolved incidents in the same team with the same signature that have notes. Notes can be marked as the resolution note, "what fixed it", either with a resolution field on resolve or pinned on a note. Those lead the similar list, show on the incident page as "Seen before", and the triggered notification carries the latest one. Claude-Session: https://claude.ai/code/session_01MMados3BD1oSjevHxbmVqU	2026-09-25 15:42:25 +02:00
Niklas Ye	d728af53b1	Put a team's own settings in the web UI ... Closes #17. Everything a team owner configures was API-only: escalation, integrations, dead man's switches, membership, and the rota -- which the on-call view still described as the TUI's job, and the TUI has been broken against this server since teams landed. Setting up the feature this whole line of work exists for meant using curl. A Team tab now holds all of it, one team at a time, with a picker for somebody in more than one. An owner edits; a member sees the same page without the controls, because the server refuses their writes anyway -- hiding a button is a courtesy to the reader, not the thing enforcing anything. The escalation editor holds a draft and sends the whole ladder, because the API replaces it wholesale: the levels are an order, and patching one rung leaves the numbering of the others undecided. Adding a level defaults to five minutes and the rota, which is the shape almost every ladder starts as. An integration key is returned exactly once, so creating one opens a panel that says so, shows the URL large with a copy button, and renders the Alertmanager receiver snippet with the URL already in it -- the next thing anybody does with that key is paste it into a config. The panel stays until it is dismissed rather than disappearing on the next re-render. The incident view gains where an incident is on the ladder and when the next page is due, which is the question somebody looking at an unacknowledged incident actually has. The API carries it: the incident payload now includes escalation_level and escalation_due_at, the latter computed in the incident SELECT by joining the level's timeout, so a list costs no extra queries. Verified against a live server by making every call the page makes, including the writes: the six reads the Team tab issues, a two-level ladder saved and read back, an integration created and its key returned once, three days of rota assigned, switches set, and an incident showing level 1 with a due time five minutes out. Claude-Session: https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7	2026-09-20 21:22:24 +02:00
Niklas Ye	b0a02c010b	Add an admin page, and move the behaviour settings into the database ... Closes #5. Three of the server's tunables were environment variables, which meant changing how long an incident waits before being paged again required editing a chart, merging it and waiting for a reconcile. They are behaviour rather than infrastructure, and the difference is who needs to change them and how often. The split is by who owns the value. What stays in the environment is where the server is plugged in: the listen address, the DSN, the ntfy URL and token, the public URL. Those are needed before the database is open and two of them are credentials -- the settings endpoint reports that ntfy is configured and that a token is set, and never what either is. What moves is how it behaves: the notify repeat interval, the stale window and the archive window. The environment variable becomes the seed rather than the setting, written once on first start and never overwritten, so a redeploy cannot put a chart's default back over an administrator's edit -- the rule the per-team dead man's switches already follow. The loops read the current value per tick, so a change at 02:00 is obeyed at 02:00. Key/value rather than a column per knob: #6 and #7 will both add settings, and a table shaped one-column-per-setting needs a migration for each. The cost is that values are text and the accessor has to say what type it wanted, which settings.go does in one place. Unknown keys are refused rather than stored -- a typo that wrote notify_repeat_second would otherwise sit in the table looking like configuration and doing nothing -- and each value has bounds loose enough to catch a slipped decimal point without having an opinion about anybody's rota. Disabling an account is new, and is not deleting one. Deleting a user nulls acknowledged_by and assigned_to, which quietly rewrites who did what during an incident months after the fact. A disabled user cannot authenticate by either credential, loses their sessions immediately, and stays the name on every acknowledgement they made. The check is part of the lookup in serveAs rather than a test afterwards, so there is no path where the row is loaded and the flag is then forgotten. The page itself is a fourth tab, shown only to an administrator and only as a courtesy: every endpoint under it is refused with 403 regardless, so somebody who types /admin gets an explanation rather than a blank screen. It lists teams with their size and open-incident count, users with their flags, and the settings with their bounds -- plus the environment half, read-only, so somebody hunting for the ntfy URL learns where it lives instead of concluding the server has none. Delete is disabled rather than offered-and-refused for a team with open incidents, and neither admin action is offered on your own account, since the server refuses both. Claude-Session: https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7	2026-09-20 18:23:46 +02:00
Niklas Ye	a4fbd60441	Scope everything to a team, and route alerts by integration key ... The core of #4, and what #1 is for: terdut stops being one shared space. A team owns its incidents, alerts, schedule and integrations; a user sees exactly the teams they are in. Everything that existed moves into one Default team and every existing user becomes an owner of it, so the upgrade is a no-op for the people using it. Ingestion is the load-bearing half. An alert arrives on a team's integration key, and the key is both the credential and the routing: it says that the sender may post, and which team the alerts belong to. That also closes the unauthenticated webhook -- the old path stays for one release, deprecated and routed to the oldest team, so an upgrade does not stop delivering while somebody edits the Alertmanager config. Scoping is enforced in as few places as possible, because the failure mode is silent. serveAs loads the caller's memberships once; list queries carry `team_id = ANY(...)`; and every incident route goes through incidentIDParam, which now parses the id AND checks the team in the same call, so a new handler cannot remember the first half and forget the second. Anything in another team is 404, never 403: whether an incident exists is that team's business. Two bugs this found, both of which would have been silent: * upsertAlerts decided "is this a new occurrence" by looking up the fingerprint alone. Across teams that made team B's first alert look like a re-send of team A's, so it opened no incident at all. The lookups are keyed on (team_id, fingerprint) now, as the index is. * Every uniqueness rule was written for one tenant. Two teams watching two clusters legitimately see the same fingerprint, the same groupKey, and want somebody on call on the same day; all three constraints move to include team_id. Roles inside a team are separate from the system administrator flag: an owner configures the team, a member works its incidents, and an admin is NOT implicitly in every team -- administration is about accounts, not about reading other people's incidents. An admin can still repair a team whose owner has left, which is why requireTeamOwner lets them through. A shift can only be given to somebody in the team. Paging a person who cannot open the incident is worse than paging nobody. The UI is updated only as far as keeping it working: it loads the viewer's teams with the session and uses the first one, since nobody has a second yet. "On call now" shows every team the viewer is in, named only when there is more than one, so the common case reads exactly as before. The team switcher, badges and per-team settings pages are the next step. Breaking for API clients: the schedule endpoints moved under the team, and /api/schedule/current returns an array rather than an object or a 404. terdut-tui will need a version for that. Per-team dead-man configuration is deliberately not here. A heartbeat's incident already opens in the team whose key received it, which is the part that matters for isolation; moving the matchers out of env into per-team rows is a change to how deadman.go is configured rather than to who sees what. Claude-Session: https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7	2026-09-20 13:36:24 +02:00
Niklas Ye	1377d9005b	Add a system administrator role, and gate account management behind it ... Until now every authenticated caller could create and delete users, set anybody's password and mint anybody's API keys -- auth.go said so in a comment. Defensible with one operator and a hand-made account; not once people sign themselves up (#7), and not in a multi-tenant install (#4), where the user list is no longer everybody who works here. users.is_admin is the flag. AdminOnly gates creating and deleting users and granting the flag itself. The endpoints that are self-service for your own account and administration for somebody else's -- password, ntfy topic, API keys -- go through requireSelfOrAdmin instead, because which rule applies depends on the {id} in the path rather than on the route. Minting your own API key stays self-service. A key carries exactly the rights of the user it belongs to, so issuing one is no more than signing in again; requiring an admin for it would mean a responder cannot set up the TUI without somebody else in the room. /api/users stays readable by everybody. The queue's assignment control and the on-call schedule both have to name people, and hiding the roster from the people on it buys nothing. THE MIGRATION MAKES EVERY EXISTING USER AN ADMINISTRATOR. They already hold these powers, so nobody's access changes on upgrade: it names what is already true and leaves demotion as a deliberate act. Promoting only user 1 would silently strip the others, and could leave an install whose only administrator is an account nobody has a password for. Two guards keep an install administrable: the last administrator can be neither deleted nor demoted, and nobody can delete or demote themselves -- the likelier accident, where the only admin clears their own flag while tidying up and locks the door behind them. No UI changes: there are no account-management screens yet. models.User carries is_admin (not omitempty, so a client can tell false from an old server), which is what #5's admin page will render from.	2026-09-20 13:20:04 +02:00
Niklas Ye	bc285799d1	Page the on-call person when an incident opens ... An incident opened, got assigned to whoever held today's schedule entry, and then sat there silently until somebody thought to look. The schedule and the incident model were both built; nothing reached the person holding the pager. Notifications go out through ntfy, over plain HTTP with no new dependencies. Delivery is an outbox rather than an inline call: the pool is limited to a single connection, so a POST made while holding the webhook's transaction would stall every other request behind it. The webhook inserts a row and a notifier goroutine sends it within a tick, retrying with exponential backoff. Only opening an incident has to resolve a topic from scratch. Reminders and all-clears reuse whatever that first notification chose, which keeps configuration out of resolveIfSettled and gives the right rule for free: you only hear that something resolved if you were told it started. Each push carries an Acknowledge button, because the useful thing to do at 3am is stop the pager without unlocking anything. It POSTs to an unauthenticated /api/notify/ack/{token} — a notification body lives on the ntfy server and in the device cache, so a real API key must never appear in one. The token is minted per delivery, scoped to one incident and one action, and expires in a day. Reminders repeat until the incident stops being untouched. The stop conditions are the states that already mean somebody has it: acknowledged, snoozed, resolved, archived. Snooze is the mute button, so there is no separate reminder cap. Notifications sent to the fallback topic carry no Acknowledge button. The topic is shared, and a button on it would let any subscriber acknowledge as somebody else.	2026-08-07 08:51:38 +02:00
Niklas Ye	279ef6cf8b	Turn incoming alerts into incidents ... The alerts row was both Alertmanager's record and the human work queue, and the two have different owners. The webhook upsert rewrites that row on every notification; acknowledgement, comments and archiving were columns on it that the upsert happened not to touch. So an alert that resolved and re-fired days later still read as acknowledged by whoever acked the first occurrence — the ack outlived the thing it referred to. Nothing recorded transitions either: rows are mutated in place, so there was no timeline and no way to compute how long anything took. Alerts are now read-only signal records with two states, and incidents are the work item: triggered, acknowledged or resolved, with an assignee, a snooze, notes and an append-only timeline. Many alerts map to one incident, and a new occurrence opens a new incident, which is what makes a stale ack impossible rather than merely unlikely. Correlation uses Alertmanager's own groupKey. It already grouped the alerts according to the group_by routing tree the operator configured and sends the result on every webhook, where it was being discarded; adopting it means changing group_by in alertmanager.yml changes correlation here, with no second grouping scheme to configure and keep in sync. An incident opens only when an alert transitions into firing — an unseen fingerprint, a newer startsAt, or a resolved alert starting again. The unchanged notifications Alertmanager re-sends every repeat_interval are none of those. That rule is what lets manual resolution be terminal: without it, closing an incident by hand would be undone by the next re-send of an alert that never stopped firing, and the button would be a lie. Snooze covers the "not now" case instead. Incidents otherwise resolve by cascade, once every alert under them has stopped firing, whether by webhook or by expiry. New incidents are assigned to whoever holds today's schedule entry. The schedule table has existed since the first release with nothing reading it. Also here, following from the split: - Incident severity is a high-water mark over its alerts, never lowered. An incident that hit critical was a critical incident, and downgrading a live one would demote it in the queue while the work is still open. - /api/stats/incidents reports MTTA and MTTR, null rather than zero until there is something to average. Neither was computable before. - Alert archiving becomes sweeper-only housekeeping; the archive people interact with is the incident's. Breaking: the alert acknowledge, archive and comment endpoints are gone, and the alert object drops the acknowledgement fields and gains incident_id. The README maps each removed endpoint to its replacement. Migration 008 backfills an incident per existing alert, archived ones included so no comment is orphaned, carrying acknowledgements across and turning comments into timeline notes. Both documented alert contracts are untouched: received_at still advances on every accepted payload, re-sends included, and resolution_source still says how much to trust ends_at. The upsert is byte-for-byte what it was, now running inside the ingest transaction.	2026-07-30 17:02:13 +02:00
Niklas Ye	a602ff3efc	Document received_at and resolution_source as public contract ... The API reference listed endpoints but never the alert object's fields, so two of them were load-bearing for clients while being described nowhere. received_at appeared only in passing, as a stats filter; resolution_source only inside the stale-expiry prose. Both carry meaning a client cannot derive on its own. starts_at comes from Prometheus and never changes for an alert instance, so received_at is the only signal that a firing alert is still being refreshed — it advances on every accepted webhook, including the unchanged notifications Alertmanager re-sends every repeat_interval. resolution_source then says how much to trust ends_at: under 'alertmanager' it is an end time somebody reported, but under 'expiry' nothing ever reported one, so it is either a stale watermark or the sweep timestamp, and only an upper bound. README gains an alert object field table plus a contract section for each, including the nullability rules and the advice to tolerate unrecognised resolution_source values. The field comments in models.Alert now say these are public API rather than ingest details, and the upsert carries a note at the received_at line, which is where a regression would be introduced. Three tests lock the newly documented behaviour, none of which was covered before — the whole suite passed with the received_at bump deleted from the upsert, because the expiry tests only ever set that column via SQL: - a re-send advances received_at and leaves starts_at alone - a discarded out-of-order retry does not count as a heartbeat - an expiry resolve preserves a reported ends_at watermark and stamps sweep time only when none was known	2026-07-30 09:02:44 +02:00
Niklas Ye	42e846f876	Expire stale firing alerts ... A resolved webhook was the only path out of the firing state, so a notification that was dropped, silenced, or lost to a restart pinned an alert as firing forever — Prometheus showed it resolved while terdut-server kept listing it. The archiver only ever touched resolved alerts, and both the list and stats queries compared status with plain equality, so a stale row was indistinguishable from a live one. A sweeper pass now resolves firing alerts on either of two signals: the ends_at watermark Alertmanager sets on outgoing firing notifications has passed (plus a grace period for clock skew), or no webhook has refreshed the alert within TERDUT_STALE_AFTER (default 6h, above Alertmanager's 4h repeat_interval). Such alerts get resolution_source = 'expiry', distinguishing them from a real 'alertmanager' resolve. Two related webhook bugs fixed alongside: - The upsert had no ordering guard, so a retried firing notification arriving after the resolved one resurrected the alert. Payloads for an older alert instance are now discarded: a stale retry carries the same startsAt, a genuine re-fire a newer one. - archived_at was never cleared on re-fire, leaving a re-fired alert archived and invisible in the default list. Stats now exclude archived alerts to match the default list view; this lowers historical firing/resolved totals. The chart exposes both sweeper durations via sweeper.staleAfter and sweeper.archiveAfter.	2026-07-28 11:49:39 +02:00
Niklas Ye	debc4bf78c	Add alert archiving ... Alerts can be manually archived (POST /api/alerts/{id}/archive) or unarchived (DELETE /api/alerts/{id}/archive). A background goroutine auto-archives resolved alerts older than TERDUT_ARCHIVE_AFTER (default 7d). GET /api/alerts hides archived alerts by default; ?archived=true shows them.	2026-05-22 13:22:45 +02:00
Niklas Ye	f8f209dcba	Stage 5: on-call schedule ... - Migration 005: schedule_entries table (date TEXT UNIQUE, one person per day) - POST /api/schedule — assign user to one or more dates in a single transaction; any date conflict rejects the whole request (409) - GET /api/schedule — list all entries ordered by date, optional ?from/?to - GET /api/schedule/current — today's on-call user (UTC date), 404 if none - DELETE /api/schedule/{id} — remove an entry (204)	2026-05-20 22:28:23 +02:00
Niklas Ye	c3348a410a	Stage 4: alert acknowledgement and comments ... - Migration 004: acknowledged_by/acknowledged_at columns on alerts, alert_comments table (FK cascade on delete) - POST /api/alerts/{id}/acknowledge — stamps authed user + timestamp, returns updated alert with acknowledged_by username - DELETE /api/alerts/{id}/acknowledge — clears ack (204) - GET /api/alerts/{id}/comments — list in chronological order - POST /api/alerts/{id}/comments — add comment (returns 201) - DELETE /api/alerts/{id}/comments/{commentID} — own comments only (204) - All alert queries now LEFT JOIN users for ack username	2026-05-20 21:57:11 +02:00
Niklas Ye	9b4ca1482f	Stage 3: Alertmanager webhook ingestion and alert query API ... - Migration 003: alerts table with fingerprint UNIQUE, JSON label/annotation columns, nullable ends_at, and indexed status/name/received_at - POST /api/alertmanager/webhook — upserts each alert by fingerprint; zero endsAt ("0001-01-01") stored as NULL (still firing) - GET /api/alerts — filtered list (?status, ?name, ?from, ?to, ?limit) - GET /api/alerts/{id} — single alert lookup	2026-05-20 21:54:17 +02:00
Niklas Ye	7c3c28b23c	Stage 2: users, API key auth, bootstrap endpoint ... - Migration 002: users and api_keys tables (Unix timestamps, FK cascade) - POST /api/bootstrap — creates first user + key when DB is empty - POST/GET/DELETE /api/users — user CRUD - POST/DELETE /api/users/{id}/api-keys — key issuance and revocation - AuthMiddleware: SHA-256 bearer token lookup, last_used_at tracking - Raw key returned once on creation; only SHA-256 hash stored	2026-05-20 21:51:09 +02:00