Page the next person when nobody answers
Closes #6, and closes the thing this whole line of work was opened for. Until now an unacknowledged incident re-paged the same topic every notify_repeat forever, which is a louder version of the same silence: if the person on call is asleep, out of signal or has left the company, nothing else happened. A team can now configure an ordered ladder. Each level has a timeout and a set of targets; a target is a named person or whoever the team's rota says is on call today. That second kind is the one that keeps working when the rota changes and nobody remembers to edit the policy. When a level's timeout passes with the incident still triggered, the next level is paged; off the end the chain repeats repeat_count times and then the team's fallback topic is paged once. The incident stays open throughout, because running out of people to wake is not somebody answering. Escalation rides the notifier's existing 30-second tick and its outbox rather than adding a second scheduler, and runs before delivery so a level that comes due on a tick is paged on that tick. Each target gets its own outbox row and therefore its own Acknowledge token: the button in a notification must acknowledge as the person holding the phone, not as whoever was paged first. Acknowledging or resolving takes the incident off the ladder. Snoozing pauses it -- a deliberate "not now" holds the ladder where it is and it resumes when the snooze runs out, rather than carrying on without the person who asked for quiet. Reminders and escalation never both run. A team with a ladder gets escalation; a team without keeps today's behaviour exactly. Both would mean two pages for one silence, which is how a tool gets muted. A level whose targets cannot be reached -- no topic, a disabled account, an empty rota -- is entered anyway, recorded as "nobody reachable", and the ladder moves on. Stalling on a rung that cannot ring would be the failure this feature exists to prevent, wearing the feature's clothes. A policy with such a level cannot be created, but an older row could hold one. The API replaces the ladder wholesale rather than patching a rung, because the levels are an order: editing one has to answer what happens to the numbering of the others, and a whole-ladder PUT makes that the client's decision and the edit atomic. Verified against a live server as well as in tests: alice paged, nobody answers, bob paged, nobody answers, the fallback topic paged once and the timeline reading "level 2: bob" then "escalation exhausted: paged terdut-oncall-all" -- and a second incident acknowledged before its timeout, which woke nobody else. No UI yet. The team-settings screens for escalation, integrations and dead man's switches are all still missing, and they are one piece of work rather than three. Claude-Session: https://claude.ai/code/session_01RHPj4ggeFdEjKKfm4SHbD7
This commit is contained in:
@@ -403,12 +403,18 @@ func openIncident(ctx context.Context, q querier, notify NotifyConfig, teamID in
|
||||
}
|
||||
}
|
||||
|
||||
// Queue the page, but do not send it here: this runs inside a transaction on
|
||||
// a single-connection pool, so an HTTP call would hold up every other
|
||||
// request. The notifier picks the row up within a tick.
|
||||
// Queue the page, but do not send it here: this runs inside the webhook's
|
||||
// transaction, and an HTTP call would hold a connection open across a
|
||||
// network round trip. The notifier picks the row up within a tick.
|
||||
if err := enqueueOpened(ctx, q, notify, id, onCall); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// And start the escalation clock, if the team keeps one. In the same
|
||||
// transaction, so an incident is never briefly open with nobody counting.
|
||||
if err := startEscalation(ctx, q, id, teamID); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user