Compare commits
4 Commits
e8d45f9d3d
...
v0.26.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 2b396d22d6 | |||
| 1f1faa437c | |||
| dc92f51cf8 | |||
| d675f8ec9b |
@@ -718,13 +718,14 @@ administrator who is not in the team gets the same `404` as anybody else.
|
|||||||
| `GET` | `/api/teams/{teamID}/members` | member | Who is in the team |
|
| `GET` | `/api/teams/{teamID}/members` | member | Who is in the team |
|
||||||
| `POST` | `/api/teams/{teamID}/members` | **owner** | Add a member, or change their role `{"user_id","role"}` |
|
| `POST` | `/api/teams/{teamID}/members` | **owner** | Add a member, or change their role `{"user_id","role"}` |
|
||||||
| `DELETE` | `/api/teams/{teamID}/members/{userID}` | **owner** | Remove a member. `409` for the last owner |
|
| `DELETE` | `/api/teams/{teamID}/members/{userID}` | **owner** | Remove a member. `409` for the last owner |
|
||||||
| `GET` | `/api/teams/{teamID}/integrations` | member | List integrations. Never returns keys |
|
| `GET` | `/api/teams/{teamID}/integrations` | member | List integrations. Never returns keys. Each carries `status` (`active` if its key posted within 24h, `quiet` if it has but not lately, `never`), `last_used_at` (last webhook, usable or not), `last_alert_at` (when an alert last arrived on it) and `alerts_24h` (distinct alerts it refreshed in the last day). Alerts delivered before the source was recorded (migration 010) have none, so the last two fill in as Alertmanager re-sends them |
|
||||||
|
| `PATCH` | `/api/teams/{teamID}/integrations/{integrationID}` | **owner** | Rename `{"name"}`. The key does not change |
|
||||||
| `POST` | `/api/teams/{teamID}/integrations` | **owner** | Mint an integration `{"name","kind"}` — key and URL shown once |
|
| `POST` | `/api/teams/{teamID}/integrations` | **owner** | Mint an integration `{"name","kind"}` — key and URL shown once |
|
||||||
| `DELETE` | `/api/teams/{teamID}/integrations/{integrationID}` | **owner** | Revoke an integration |
|
| `DELETE` | `/api/teams/{teamID}/integrations/{integrationID}` | **owner** | Revoke an integration. Alerts it delivered stay, unattributed |
|
||||||
| `GET` | `/api/teams/{teamID}/invites` | **owner** | The team's invite links, with their uses and expiry. Never the tokens |
|
| `GET` | `/api/teams/{teamID}/invites` | **owner** | The team's invite links, with their uses and expiry. Never the tokens |
|
||||||
| `POST` | `/api/teams/{teamID}/invites` | **owner** | Mint one `{"role","max_uses"}` — the full URL is returned once |
|
| `POST` | `/api/teams/{teamID}/invites` | **owner** | Mint one `{"role","max_uses"}` — the full URL is returned once |
|
||||||
| `DELETE` | `/api/teams/{teamID}/invites/{inviteID}` | **owner** | Revoke a link before it expires |
|
| `DELETE` | `/api/teams/{teamID}/invites/{inviteID}` | **owner** | Revoke a link before it expires |
|
||||||
| `GET` | `/api/teams/{teamID}/escalation` | member | The team's [escalation ladder](#escalation) `{repeat_count, fallback_topic, levels[]}`. Empty levels means the team has none |
|
| `GET` | `/api/teams/{teamID}/escalation` | member | The team's [escalation ladder](#escalation) `{repeat_count, fallback_topic, levels[], last_escalated_at?, last_escalated_incident_id?}`. Empty levels means the team has none. Each level also carries `status` (`ready`, `escalating` when an unanswered incident has climbed to it, `unreachable` when nobody on it could be woken), `waiting` (ids of the open incidents on it) and, per target, `username` (who it means today — the person on call, for a rota target), `reachable` and `problem`. The extra fields are output only; `PUT` takes the plain shape |
|
||||||
| `PUT` | `/api/teams/{teamID}/escalation` | **owner** | Replace it wholesale. `400` for a level with no targets or no timeout — a rung that pages nobody is a silence with a number on it |
|
| `PUT` | `/api/teams/{teamID}/escalation` | **owner** | Replace it wholesale. `400` for a level with no targets or no timeout — a rung that pages nobody is a silence with a number on it |
|
||||||
| `GET` | `/api/teams/{teamID}/deadman/switches` | member | The team's [dead man's switches](#dead-mans-switch), each `{id, name, matcher, timeout_seconds, severity, status, last_heartbeat_at, last_triggered_at, open_incident_id, sources[]}`. `status` is `healthy`, `dead` or `dormant`; `sources` has one entry per heartbeat fingerprint. Empty when the team watches nothing |
|
| `GET` | `/api/teams/{teamID}/deadman/switches` | member | The team's [dead man's switches](#dead-mans-switch), each `{id, name, matcher, timeout_seconds, severity, status, last_heartbeat_at, last_triggered_at, open_incident_id, sources[]}`. `status` is `healthy`, `dead` or `dormant`; `sources` has one entry per heartbeat fingerprint. Empty when the team watches nothing |
|
||||||
| `POST` | `/api/teams/{teamID}/deadman/switches` | **owner** | Add one: `{name?, matcher, timeout_seconds, severity?}`. `400` when the matcher names no `alertname` or holds several, or the timeout is not positive — a switch that silently watches nothing is the failure this feature exists to prevent |
|
| `POST` | `/api/teams/{teamID}/deadman/switches` | **owner** | Add one: `{name?, matcher, timeout_seconds, severity?}`. `400` when the matcher names no `alertname` or holds several, or the timeout is not positive — a switch that silently watches nothing is the failure this feature exists to prevent |
|
||||||
|
|||||||
@@ -15,5 +15,5 @@ type: application
|
|||||||
# appVersion and image.tag in values.yaml no longer agree, and that is not an oversight:
|
# appVersion and image.tag in values.yaml no longer agree, and that is not an oversight:
|
||||||
# image.tag stays "latest", which is what a local install actually pulls. appVersion is
|
# image.tag stays "latest", which is what a local install actually pulls. appVersion is
|
||||||
# metadata and drives nothing.
|
# metadata and drives nothing.
|
||||||
version: 0.24.0
|
version: 0.26.0
|
||||||
appVersion: "v0.24.0"
|
appVersion: "v0.26.0"
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ type ingested struct {
|
|||||||
// post, and which team the alerts belong to.
|
// post, and which team the alerts belong to.
|
||||||
func handleIntegrationWebhook(db *sql.DB, notify NotifyConfig) http.HandlerFunc {
|
func handleIntegrationWebhook(db *sql.DB, notify NotifyConfig) http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
teamID, err := teamIDForKey(r.Context(), db, chi.URLParam(r, "key"))
|
src, err := sourceForKey(r.Context(), db, chi.URLParam(r, "key"))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, errUnknownIntegration) {
|
if errors.Is(err, errUnknownIntegration) {
|
||||||
// 401 and not 404: the path is real, the key is not, and a
|
// 401 and not 404: the path is real, the key is not, and a
|
||||||
@@ -90,11 +90,12 @@ func handleIntegrationWebhook(db *sql.DB, notify NotifyConfig) http.HandlerFunc
|
|||||||
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
receiveWebhook(w, r, db, notify, teamID)
|
receiveWebhook(w, r, db, notify, src)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func receiveWebhook(w http.ResponseWriter, r *http.Request, db *sql.DB, notify NotifyConfig, teamID int64) {
|
func receiveWebhook(w http.ResponseWriter, r *http.Request, db *sql.DB, notify NotifyConfig, src alertSource) {
|
||||||
|
teamID := src.teamID
|
||||||
var payload amPayload
|
var payload amPayload
|
||||||
if err := decodeJSON(r, &payload); err != nil {
|
if err := decodeJSON(r, &payload); err != nil {
|
||||||
respond(w, http.StatusBadRequest, errResp("invalid payload"))
|
respond(w, http.StatusBadRequest, errResp("invalid payload"))
|
||||||
@@ -104,7 +105,7 @@ func receiveWebhook(w http.ResponseWriter, r *http.Request, db *sql.DB, notify N
|
|||||||
// Alertmanager retries anything that is not 2xx, and a retry of a payload
|
// Alertmanager retries anything that is not 2xx, and a retry of a payload
|
||||||
// we failed to store is more useful than an error it cannot act on — so
|
// we failed to store is more useful than an error it cannot act on — so
|
||||||
// failures are logged, not surfaced.
|
// failures are logged, not surfaced.
|
||||||
if err := ingest(r.Context(), db, notify, teamID, payload); err != nil {
|
if err := ingest(r.Context(), db, notify, src, payload); err != nil {
|
||||||
log.Printf("webhook ingest (team %d, group %q): %v", teamID, payload.GroupKey, err)
|
log.Printf("webhook ingest (team %d, group %q): %v", teamID, payload.GroupKey, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -114,7 +115,8 @@ func receiveWebhook(w http.ResponseWriter, r *http.Request, db *sql.DB, notify N
|
|||||||
// ingest stores a payload's alerts and reconciles the incident for its group.
|
// ingest stores a payload's alerts and reconciles the incident for its group.
|
||||||
// The whole payload is one transaction: an incident that opened but whose alerts
|
// The whole payload is one transaction: an incident that opened but whose alerts
|
||||||
// failed to link would be a work item nobody could act on.
|
// failed to link would be a work item nobody could act on.
|
||||||
func ingest(ctx context.Context, db *sql.DB, notify NotifyConfig, teamID int64, payload amPayload) error {
|
func ingest(ctx context.Context, db *sql.DB, notify NotifyConfig, src alertSource, payload amPayload) error {
|
||||||
|
teamID := src.teamID
|
||||||
tx, err := db.BeginTx(ctx, nil)
|
tx, err := db.BeginTx(ctx, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -129,7 +131,7 @@ func ingest(ctx context.Context, db *sql.DB, notify NotifyConfig, teamID int64,
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
accepted, err := upsertAlerts(ctx, tx, deadman, teamID, payload.Alerts)
|
accepted, err := upsertAlerts(ctx, tx, deadman, src, payload.Alerts)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -186,7 +188,8 @@ func ingest(ctx context.Context, db *sql.DB, notify NotifyConfig, teamID int64,
|
|||||||
|
|
||||||
// upsertAlerts stores each alert of a payload and reports what changed. Payloads
|
// upsertAlerts stores each alert of a payload and reports what changed. Payloads
|
||||||
// the ordering guard rejected are left out entirely.
|
// the ordering guard rejected are left out entirely.
|
||||||
func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman deadmanSet, teamID int64, alerts []amAlert) ([]ingested, error) {
|
func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman deadmanSet, src alertSource, alerts []amAlert) ([]ingested, error) {
|
||||||
|
teamID := src.teamID
|
||||||
now := time.Now().Unix()
|
now := time.Now().Unix()
|
||||||
accepted := make([]ingested, 0, len(alerts))
|
accepted := make([]ingested, 0, len(alerts))
|
||||||
|
|
||||||
@@ -242,8 +245,8 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman deadmanSet, teamID in
|
|||||||
if _, err := tx.ExecContext(ctx, `
|
if _, err := tx.ExecContext(ctx, `
|
||||||
INSERT INTO alerts
|
INSERT INTO alerts
|
||||||
(team_id, fingerprint, name, status, labels, annotations, starts_at, ends_at,
|
(team_id, fingerprint, name, status, labels, annotations, starts_at, ends_at,
|
||||||
generator_url, received_at, resolution_source)
|
generator_url, received_at, resolution_source, integration_id)
|
||||||
VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7, $8, $9, $10, $11)
|
VALUES ($1, $2, $3, $4, $5::jsonb, $6::jsonb, $7, $8, $9, $10, $11, $12)
|
||||||
ON CONFLICT (team_id, fingerprint) DO UPDATE SET
|
ON CONFLICT (team_id, fingerprint) DO UPDATE SET
|
||||||
status = excluded.status,
|
status = excluded.status,
|
||||||
labels = excluded.labels,
|
labels = excluded.labels,
|
||||||
@@ -257,6 +260,8 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman deadmanSet, teamID in
|
|||||||
-- is a breaking API change — see models.Alert.ReceivedAt.
|
-- is a breaking API change — see models.Alert.ReceivedAt.
|
||||||
received_at = excluded.received_at,
|
received_at = excluded.received_at,
|
||||||
resolution_source = excluded.resolution_source,
|
resolution_source = excluded.resolution_source,
|
||||||
|
-- Last sender wins; see migration 010.
|
||||||
|
integration_id = excluded.integration_id,
|
||||||
-- A re-fire makes the alert current again, so it leaves the archive.
|
-- A re-fire makes the alert current again, so it leaves the archive.
|
||||||
archived_at = CASE WHEN excluded.status = 'firing'
|
archived_at = CASE WHEN excluded.status = 'firing'
|
||||||
THEN NULL ELSE alerts.archived_at END
|
THEN NULL ELSE alerts.archived_at END
|
||||||
@@ -267,7 +272,7 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman deadmanSet, teamID in
|
|||||||
teamID, a.Fingerprint, name, a.Status,
|
teamID, a.Fingerprint, name, a.Status,
|
||||||
string(labelsJSON), string(annotationsJSON),
|
string(labelsJSON), string(annotationsJSON),
|
||||||
a.StartsAt.Unix(), endsAtUnix,
|
a.StartsAt.Unix(), endsAtUnix,
|
||||||
a.GeneratorURL, now, resolutionSource,
|
a.GeneratorURL, now, resolutionSource, src.integrationID,
|
||||||
); err != nil {
|
); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|||||||
+185
-1
@@ -346,8 +346,192 @@ func handleGetEscalation(db *sql.DB) http.HandlerFunc {
|
|||||||
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
respond(w, http.StatusOK, escalationResponse(policy, teamID))
|
view, err := escalationStatus(r.Context(), db, teamID, policy)
|
||||||
|
if err != nil {
|
||||||
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
|
return
|
||||||
}
|
}
|
||||||
|
respond(w, http.StatusOK, view)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Level statuses, as the Escalation page colours them.
|
||||||
|
const (
|
||||||
|
levelReady = "ready"
|
||||||
|
levelEscalating = "escalating"
|
||||||
|
levelUnreachable = "unreachable"
|
||||||
|
)
|
||||||
|
|
||||||
|
// escalationTargetView is a target with who it means today and whether that
|
||||||
|
// person can actually be woken. The extra fields are output only: the PUT body
|
||||||
|
// is the plain escalationTargetJSON, and anything else in it is ignored.
|
||||||
|
type escalationTargetView struct {
|
||||||
|
escalationTargetJSON
|
||||||
|
|
||||||
|
// Username is who the target resolves to right now: the named person, or
|
||||||
|
// whoever the rota says is on call today. Empty when nobody is.
|
||||||
|
Username string `json:"username,omitempty"`
|
||||||
|
|
||||||
|
// Reachable is whether a page to this target would go anywhere, and Problem
|
||||||
|
// says why not when it would not — the same conditions pageLevel skips on.
|
||||||
|
Reachable bool `json:"reachable"`
|
||||||
|
Problem string `json:"problem,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type escalationLevelView struct {
|
||||||
|
Position int64 `json:"position"`
|
||||||
|
TimeoutSeconds int64 `json:"timeout_seconds"`
|
||||||
|
Targets []escalationTargetView `json:"targets"`
|
||||||
|
|
||||||
|
// Status is unreachable when no target of the level could be woken — a rung
|
||||||
|
// that looks configured and pages nobody, which is worth seeing before an
|
||||||
|
// incident finds it — escalating when an unanswered incident has climbed to
|
||||||
|
// it, and ready otherwise.
|
||||||
|
Status string `json:"status"`
|
||||||
|
|
||||||
|
// Waiting lists the open, unacknowledged incidents currently on this level.
|
||||||
|
Waiting []int64 `json:"waiting"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type escalationView struct {
|
||||||
|
TeamID int64 `json:"team_id"`
|
||||||
|
RepeatCount int64 `json:"repeat_count"`
|
||||||
|
FallbackTopic string `json:"fallback_topic"`
|
||||||
|
Levels []escalationLevelView `json:"levels"`
|
||||||
|
|
||||||
|
// LastEscalatedAt is when an incident of this team last moved up the ladder,
|
||||||
|
// or ran off the end of it, and LastEscalatedIncidentID which one. Absent
|
||||||
|
// when nothing ever has: a ladder nobody has needed yet.
|
||||||
|
LastEscalatedAt *time.Time `json:"last_escalated_at,omitempty"`
|
||||||
|
LastEscalatedIncidentID *int64 `json:"last_escalated_incident_id,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// escalationStatus is a team's ladder together with what it would do right now
|
||||||
|
// and what it has been doing. The resolution follows pageLevel's rules, so the
|
||||||
|
// page cannot promise a page that the notifier would skip.
|
||||||
|
func escalationStatus(ctx context.Context, db *sql.DB, teamID int64, policy *escalationPolicy) (escalationView, error) {
|
||||||
|
base := escalationResponse(policy, teamID)
|
||||||
|
out := escalationView{
|
||||||
|
TeamID: teamID, RepeatCount: base.RepeatCount, FallbackTopic: base.FallbackTopic,
|
||||||
|
Levels: []escalationLevelView{},
|
||||||
|
}
|
||||||
|
if !policy.configured() {
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
onCall, err := currentOnCall(ctx, db, teamID)
|
||||||
|
if err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
|
||||||
|
type account struct {
|
||||||
|
username string
|
||||||
|
topic bool
|
||||||
|
disabled bool
|
||||||
|
}
|
||||||
|
accounts := map[int64]account{}
|
||||||
|
lookup := func(id int64) (account, error) {
|
||||||
|
if a, ok := accounts[id]; ok {
|
||||||
|
return a, nil
|
||||||
|
}
|
||||||
|
var a account
|
||||||
|
var topic *string
|
||||||
|
var disabledAt *int64
|
||||||
|
if err := db.QueryRowContext(ctx,
|
||||||
|
"SELECT username, ntfy_topic, disabled_at FROM users WHERE id = $1", id).
|
||||||
|
Scan(&a.username, &topic, &disabledAt); err != nil {
|
||||||
|
return a, err
|
||||||
|
}
|
||||||
|
a.topic = topic != nil && *topic != ""
|
||||||
|
a.disabled = disabledAt != nil
|
||||||
|
accounts[id] = a
|
||||||
|
return a, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
waiting := map[int64][]int64{}
|
||||||
|
rows, err := db.QueryContext(ctx, `
|
||||||
|
SELECT id, escalation_level FROM incidents
|
||||||
|
WHERE team_id = $1 AND resolved_at IS NULL AND archived_at IS NULL
|
||||||
|
AND status = 'triggered' AND escalation_level > 0
|
||||||
|
ORDER BY id`, teamID)
|
||||||
|
if err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
for rows.Next() {
|
||||||
|
var id, level int64
|
||||||
|
if err := rows.Scan(&id, &level); err != nil {
|
||||||
|
rows.Close()
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
waiting[level] = append(waiting[level], id)
|
||||||
|
}
|
||||||
|
rows.Close()
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, l := range base.Levels {
|
||||||
|
level := escalationLevelView{
|
||||||
|
Position: l.Position, TimeoutSeconds: l.TimeoutSeconds,
|
||||||
|
Targets: []escalationTargetView{}, Waiting: []int64{},
|
||||||
|
}
|
||||||
|
if w := waiting[l.Position]; w != nil {
|
||||||
|
level.Waiting = w
|
||||||
|
}
|
||||||
|
|
||||||
|
anyReachable := false
|
||||||
|
for _, t := range l.Targets {
|
||||||
|
view := escalationTargetView{escalationTargetJSON: t}
|
||||||
|
userID := t.UserID
|
||||||
|
if t.Kind == "oncall" {
|
||||||
|
userID = onCall
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case userID == nil:
|
||||||
|
view.Problem = "nobody is on call today"
|
||||||
|
default:
|
||||||
|
a, err := lookup(*userID)
|
||||||
|
switch {
|
||||||
|
case err != nil:
|
||||||
|
view.Problem = "account not found"
|
||||||
|
case a.disabled:
|
||||||
|
view.Username, view.Problem = a.username, "account is disabled"
|
||||||
|
case !a.topic:
|
||||||
|
view.Username, view.Problem = a.username, "has no ntfy topic"
|
||||||
|
default:
|
||||||
|
view.Username, view.Reachable = a.username, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
anyReachable = anyReachable || view.Reachable
|
||||||
|
level.Targets = append(level.Targets, view)
|
||||||
|
}
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case !anyReachable:
|
||||||
|
level.Status = levelUnreachable
|
||||||
|
case l.Position >= 2 && len(level.Waiting) > 0:
|
||||||
|
level.Status = levelEscalating
|
||||||
|
default:
|
||||||
|
level.Status = levelReady
|
||||||
|
}
|
||||||
|
out.Levels = append(out.Levels, level)
|
||||||
|
}
|
||||||
|
|
||||||
|
var incidentID, at int64
|
||||||
|
switch err := db.QueryRowContext(ctx, `
|
||||||
|
SELECT e.incident_id, e.created_at
|
||||||
|
FROM incident_events e JOIN incidents i ON i.id = e.incident_id
|
||||||
|
WHERE i.team_id = $1 AND e.type = $2
|
||||||
|
ORDER BY e.created_at DESC, e.id DESC LIMIT 1`, teamID, evEscalated).
|
||||||
|
Scan(&incidentID, &at); {
|
||||||
|
case err == sql.ErrNoRows:
|
||||||
|
case err != nil:
|
||||||
|
return out, err
|
||||||
|
default:
|
||||||
|
t := time.Unix(at, 0).UTC()
|
||||||
|
out.LastEscalatedAt, out.LastEscalatedIncidentID = &t, &incidentID
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
type escalationLevelJSON struct {
|
type escalationLevelJSON struct {
|
||||||
|
|||||||
@@ -383,3 +383,122 @@ func TestEscalation_SkipsUnreachableTargets(t *testing.T) {
|
|||||||
t.Errorf("a target with no topic should page nothing, paged %v", got)
|
t.Errorf("a target with no topic should page nothing, paged %v", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// The ladder as the Escalation page reads it
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
type ladderLevel struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
Waiting []int64 `json:"waiting"`
|
||||||
|
Targets []struct {
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Username string `json:"username"`
|
||||||
|
Reachable bool `json:"reachable"`
|
||||||
|
Problem string `json:"problem"`
|
||||||
|
} `json:"targets"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ladderView struct {
|
||||||
|
Levels []ladderLevel `json:"levels"`
|
||||||
|
LastEscalatedAt *string `json:"last_escalated_at"`
|
||||||
|
LastEscalatedIncidentID *int64 `json:"last_escalated_incident_id"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func readLadder(t *testing.T, s *ts) ladderView {
|
||||||
|
t.Helper()
|
||||||
|
var v ladderView
|
||||||
|
decode(t, s.req(t, http.MethodGet, "/api/teams/"+defaultTeam+"/escalation", nil), &v)
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// Targets say who they mean today, so "whoever is on call" is a name and not a
|
||||||
|
// promise.
|
||||||
|
func TestEscalation_StatusResolvesTargets(t *testing.T) {
|
||||||
|
s, _ := notifyTS(t, api.NotifyConfig{PublicURL: "https://terdut.example.com", RepeatEvery: 15 * time.Minute})
|
||||||
|
second := teamUser(t, s, "second", "terdut-second")
|
||||||
|
ladder(t, s, second, 0, "terdut-fallback")
|
||||||
|
|
||||||
|
v := readLadder(t, s)
|
||||||
|
if len(v.Levels) != 2 {
|
||||||
|
t.Fatalf("expected 2 levels, got %d", len(v.Levels))
|
||||||
|
}
|
||||||
|
if got := v.Levels[0].Targets[0]; got.Kind != "oncall" || got.Username != "admin" || !got.Reachable {
|
||||||
|
t.Errorf("the rota target should resolve to the person on call, got %+v", got)
|
||||||
|
}
|
||||||
|
if got := v.Levels[1].Targets[0]; got.Username != "second" || !got.Reachable {
|
||||||
|
t.Errorf("the named target should be reachable, got %+v", got)
|
||||||
|
}
|
||||||
|
if v.Levels[0].Status != "ready" || v.Levels[1].Status != "ready" || v.LastEscalatedAt != nil {
|
||||||
|
t.Errorf("an idle, healthy ladder is ready and has never escalated, got %+v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A rung that would page nobody is called out before an incident finds it.
|
||||||
|
func TestEscalation_StatusFlagsUnreachableLevels(t *testing.T) {
|
||||||
|
s, _ := notifyTS(t, api.NotifyConfig{PublicURL: "https://terdut.example.com", RepeatEvery: 15 * time.Minute})
|
||||||
|
silent := teamUser(t, s, "silent", "terdut-silent")
|
||||||
|
ladder(t, s, silent, 0, "terdut-fallback")
|
||||||
|
|
||||||
|
// Nobody on call today, and the named person loses their topic.
|
||||||
|
s.exec(t, "DELETE FROM schedule_entries")
|
||||||
|
s.exec(t, "UPDATE users SET ntfy_topic = NULL WHERE id = $1", silent)
|
||||||
|
|
||||||
|
v := readLadder(t, s)
|
||||||
|
if v.Levels[0].Status != "unreachable" || v.Levels[0].Targets[0].Problem != "nobody is on call today" {
|
||||||
|
t.Errorf("an empty rota should make level 1 unreachable, got %+v", v.Levels[0])
|
||||||
|
}
|
||||||
|
if v.Levels[1].Status != "unreachable" || v.Levels[1].Targets[0].Problem != "has no ntfy topic" {
|
||||||
|
t.Errorf("a person with no topic should make level 2 unreachable, got %+v", v.Levels[1])
|
||||||
|
}
|
||||||
|
|
||||||
|
s.exec(t, "UPDATE users SET disabled_at = 1 WHERE id = $1", silent)
|
||||||
|
if p := readLadder(t, s).Levels[1].Targets[0].Problem; p != "account is disabled" {
|
||||||
|
t.Errorf("a disabled account should say so, got %q", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Where unanswered incidents are right now, and when the ladder last did its
|
||||||
|
// job.
|
||||||
|
func TestEscalation_StatusShowsWhoIsWaitingAndLastEscalation(t *testing.T) {
|
||||||
|
s, _ := notifyTS(t, api.NotifyConfig{PublicURL: "https://terdut.example.com", RepeatEvery: 15 * time.Minute})
|
||||||
|
second := teamUser(t, s, "second", "terdut-second")
|
||||||
|
ladder(t, s, second, 0, "terdut-fallback")
|
||||||
|
|
||||||
|
postWebhook(t, s, []map[string]any{
|
||||||
|
amAlert("fp-wait", "DiskFull", "firing", "2026-05-20T10:00:00Z", zeroTime, nil),
|
||||||
|
})
|
||||||
|
s.sweepNotify(t)
|
||||||
|
|
||||||
|
// On level 1 it is waiting, which is normal and not yet an escalation.
|
||||||
|
v := readLadder(t, s)
|
||||||
|
if len(v.Levels[0].Waiting) != 1 || v.Levels[0].Status != "ready" || v.LastEscalatedAt != nil {
|
||||||
|
t.Fatalf("a fresh incident waits on level 1 quietly, got %+v", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
overdue(t, s, 1)
|
||||||
|
s.sweepNotify(t)
|
||||||
|
v = readLadder(t, s)
|
||||||
|
if v.Levels[1].Status != "escalating" || len(v.Levels[1].Waiting) != 1 || v.Levels[1].Waiting[0] != 1 {
|
||||||
|
t.Errorf("level 2 should be escalating with the incident on it, got %+v", v.Levels[1])
|
||||||
|
}
|
||||||
|
if v.LastEscalatedAt == nil || v.LastEscalatedIncidentID == nil || *v.LastEscalatedIncidentID != 1 {
|
||||||
|
t.Errorf("the escalation should be recorded, got %+v", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Somebody answers: nothing is waiting, but the history stays.
|
||||||
|
s.req(t, http.MethodPost, "/api/incidents/1/acknowledge", nil).Body.Close()
|
||||||
|
v = readLadder(t, s)
|
||||||
|
if v.Levels[1].Status != "ready" || len(v.Levels[1].Waiting) != 0 || v.LastEscalatedAt == nil {
|
||||||
|
t.Errorf("an acknowledged incident stops waiting but stays in the history, got %+v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No ladder is a real answer, not an error.
|
||||||
|
func TestEscalation_StatusWithoutALadder(t *testing.T) {
|
||||||
|
s, _ := notifyTS(t, api.NotifyConfig{PublicURL: "https://terdut.example.com", RepeatEvery: 15 * time.Minute})
|
||||||
|
v := readLadder(t, s)
|
||||||
|
if len(v.Levels) != 0 || v.LastEscalatedAt != nil {
|
||||||
|
t.Errorf("a team with no ladder should read as empty, got %+v", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -151,6 +151,7 @@ func NewRouter(db *sql.DB, notify NotifyConfig, cfg config.Config) http.Handler
|
|||||||
// Integrations: where a team's alerts come in, and the key that says so.
|
// Integrations: where a team's alerts come in, and the key that says so.
|
||||||
r.Get("/api/teams/{teamID}/integrations", handleListIntegrations(db))
|
r.Get("/api/teams/{teamID}/integrations", handleListIntegrations(db))
|
||||||
r.Post("/api/teams/{teamID}/integrations", handleCreateIntegration(db, notify.PublicURL))
|
r.Post("/api/teams/{teamID}/integrations", handleCreateIntegration(db, notify.PublicURL))
|
||||||
|
r.Patch("/api/teams/{teamID}/integrations/{integrationID}", handleRenameIntegration(db))
|
||||||
r.Delete("/api/teams/{teamID}/integrations/{integrationID}", handleDeleteIntegration(db))
|
r.Delete("/api/teams/{teamID}/integrations/{integrationID}", handleDeleteIntegration(db))
|
||||||
|
|
||||||
// The rota is per team. /api/schedule/current is the exception: it
|
// The rota is per team. /api/schedule/current is the exception: it
|
||||||
|
|||||||
@@ -0,0 +1,171 @@
|
|||||||
|
package api_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// listSources reads a team's alert sources as the Sources page does.
|
||||||
|
func listSources(t *testing.T, tm teamFixture) []map[string]any {
|
||||||
|
t.Helper()
|
||||||
|
return list(t, tm.call(http.MethodGet, "/api/teams/"+id64(tm.id)+"/integrations", nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
// addSource mints a second source in a team and returns its key.
|
||||||
|
func addSource(t *testing.T, tm teamFixture, name string) string {
|
||||||
|
t.Helper()
|
||||||
|
var out struct {
|
||||||
|
Key string `json:"key"`
|
||||||
|
}
|
||||||
|
decode(t, tm.call(http.MethodPost, "/api/teams/"+id64(tm.id)+"/integrations",
|
||||||
|
map[string]string{"name": name}), &out)
|
||||||
|
return out.Key
|
||||||
|
}
|
||||||
|
|
||||||
|
// A source that has never posted is "never", with nothing to say about alerts.
|
||||||
|
func TestSources_NeverUsedIsBlank(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
|
||||||
|
got := listSources(t, tm)
|
||||||
|
if len(got) != 1 {
|
||||||
|
t.Fatalf("expected 1 source, got %d", len(got))
|
||||||
|
}
|
||||||
|
src := got[0]
|
||||||
|
if src["status"] != "never" || src["last_used_at"] != nil || src["last_alert_at"] != nil {
|
||||||
|
t.Errorf("a source nobody has posted on should be blank, got %v", src)
|
||||||
|
}
|
||||||
|
if src["alerts_24h"].(float64) != 0 {
|
||||||
|
t.Errorf("alerts_24h = %v, want 0", src["alerts_24h"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Each source is credited with what arrived on its own key, and only that.
|
||||||
|
func TestSources_AlertsAreAttributedToTheirSource(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
second := addSource(t, tm, "staging")
|
||||||
|
|
||||||
|
postToIntegration(t, s, tm.key, "fp-1", "DiskFull")
|
||||||
|
postToIntegration(t, s, tm.key, "fp-2", "CPUHot")
|
||||||
|
|
||||||
|
got := listSources(t, tm)
|
||||||
|
first, other := got[0], got[1]
|
||||||
|
if first["status"] != "active" || first["last_used_at"] == nil || first["last_alert_at"] == nil {
|
||||||
|
t.Errorf("the source that posted should be active with timestamps, got %v", first)
|
||||||
|
}
|
||||||
|
if first["alerts_24h"].(float64) != 2 {
|
||||||
|
t.Errorf("alerts_24h = %v, want 2", first["alerts_24h"])
|
||||||
|
}
|
||||||
|
if other["status"] != "never" || other["alerts_24h"].(float64) != 0 {
|
||||||
|
t.Errorf("the other source should be untouched, got %v", other)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-sending the same alert on the other key moves it: last sender wins.
|
||||||
|
postToIntegration(t, s, second, "fp-1", "DiskFull")
|
||||||
|
got = listSources(t, tm)
|
||||||
|
if got[0]["alerts_24h"].(float64) != 1 || got[1]["alerts_24h"].(float64) != 1 {
|
||||||
|
t.Errorf("fp-1 should have moved to the second source, got %v and %v",
|
||||||
|
got[0]["alerts_24h"], got[1]["alerts_24h"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A payload with no alerts in it is a webhook, not an alert: the source was
|
||||||
|
// heard from, and nothing arrived.
|
||||||
|
func TestSources_EmptyPayloadStampsUseButNotAlert(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
|
||||||
|
resp, err := http.Post(s.URL+"/api/integrations/"+tm.key+"/alertmanager",
|
||||||
|
"application/json", bytes.NewReader([]byte(`{"version":"4","status":"firing","alerts":[]}`)))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("post: %v", err)
|
||||||
|
}
|
||||||
|
resp.Body.Close()
|
||||||
|
|
||||||
|
src := listSources(t, tm)[0]
|
||||||
|
if src["status"] != "active" || src["last_alert_at"] != nil {
|
||||||
|
t.Errorf("want active with no alert yet, got %v", src)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Quiet is "has posted, not lately"; the alert counter forgets after a day but
|
||||||
|
// the last alert's timestamp is kept.
|
||||||
|
func TestSources_QuietAfterADay(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
postToIntegration(t, s, tm.key, "fp-1", "DiskFull")
|
||||||
|
|
||||||
|
old := time.Now().Add(-48 * time.Hour).Unix()
|
||||||
|
s.exec(t, "UPDATE integrations SET last_used_at = $1", old)
|
||||||
|
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'fp-1'", old)
|
||||||
|
|
||||||
|
src := listSources(t, tm)[0]
|
||||||
|
if src["status"] != "quiet" {
|
||||||
|
t.Errorf("status = %v, want quiet", src["status"])
|
||||||
|
}
|
||||||
|
if src["alerts_24h"].(float64) != 0 {
|
||||||
|
t.Errorf("alerts_24h = %v, want 0", src["alerts_24h"])
|
||||||
|
}
|
||||||
|
if src["last_alert_at"] == nil {
|
||||||
|
t.Error("last_alert_at should survive the day")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Revoking a source does not take its alerts with it.
|
||||||
|
func TestSources_RevokeKeepsTheAlerts(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
postToIntegration(t, s, tm.key, "fp-1", "DiskFull")
|
||||||
|
|
||||||
|
id := int64(listSources(t, tm)[0]["id"].(float64))
|
||||||
|
resp := tm.call(http.MethodDelete, "/api/teams/"+id64(tm.id)+"/integrations/"+id64(id), nil)
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusNoContent {
|
||||||
|
t.Fatalf("revoke: %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
if got := len(list(t, tm.call(http.MethodGet, "/api/alerts", nil))); got != 1 {
|
||||||
|
t.Errorf("the alert should outlive its source, got %d alerts", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Renaming is an owner's, scoped to the team, and does not touch the key.
|
||||||
|
func TestSources_Rename(t *testing.T) {
|
||||||
|
s := newTS(t)
|
||||||
|
tm := newTeam(t, s, "red")
|
||||||
|
other := newTeam(t, s, "blue")
|
||||||
|
id := int64(listSources(t, tm)[0]["id"].(float64))
|
||||||
|
path := "/api/teams/" + id64(tm.id) + "/integrations/" + id64(id)
|
||||||
|
|
||||||
|
resp := tm.call(http.MethodPatch, path, map[string]string{"name": " prod "})
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusNoContent {
|
||||||
|
t.Fatalf("rename: %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
if name := listSources(t, tm)[0]["name"]; name != "prod" {
|
||||||
|
t.Errorf("name = %q, want it trimmed to prod", name)
|
||||||
|
}
|
||||||
|
postToIntegration(t, s, tm.key, "fp-1", "DiskFull") // the old key still works
|
||||||
|
|
||||||
|
for name, body := range map[string]map[string]string{
|
||||||
|
"empty": {"name": " "},
|
||||||
|
"too long": {"name": strings.Repeat("x", 101)},
|
||||||
|
} {
|
||||||
|
resp := tm.call(http.MethodPatch, path, body)
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusBadRequest {
|
||||||
|
t.Errorf("%s name: expected 400, got %d", name, resp.StatusCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Another team's owner cannot reach it.
|
||||||
|
resp = other.call(http.MethodPatch, "/api/teams/"+id64(other.id)+"/integrations/"+id64(id),
|
||||||
|
map[string]string{"name": "mine now"})
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusNotFound {
|
||||||
|
t.Errorf("renaming another team's source: expected 404, got %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
+126
-19
@@ -355,8 +355,42 @@ func isLastTeamOwner(ctx context.Context, db *sql.DB, teamID, userID int64) (boo
|
|||||||
// Integrations
|
// Integrations
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
// handleListIntegrations lists a team's integrations. Never the keys: those
|
// sourceQuietAfter is how long a source may go without posting before the
|
||||||
// exist in plaintext only in the response that created them.
|
// Sources page calls it quiet rather than active. A day is longer than any
|
||||||
|
// repeat_interval worth having, so an Alertmanager that is up and has anything
|
||||||
|
// firing never crosses it; a source with nothing firing may, and that is a
|
||||||
|
// reason to look, not proof of a fault — which is why this is a colour and not
|
||||||
|
// an alarm. Dead man's switches are where silence pages.
|
||||||
|
const sourceQuietAfter = 24 * time.Hour
|
||||||
|
|
||||||
|
const (
|
||||||
|
sourceActive = "active"
|
||||||
|
sourceQuiet = "quiet"
|
||||||
|
sourceNever = "never"
|
||||||
|
)
|
||||||
|
|
||||||
|
// integrationStatus is an integration as the Sources page shows it.
|
||||||
|
type integrationStatus struct {
|
||||||
|
models.Integration
|
||||||
|
|
||||||
|
// Status is active when the key posted within sourceQuietAfter, quiet when
|
||||||
|
// it has posted but not lately, never when it has not posted at all.
|
||||||
|
Status string `json:"status"`
|
||||||
|
|
||||||
|
// LastAlertAt is when an alert last arrived on this source, which is not the
|
||||||
|
// same as when it last posted: a payload with nothing usable in it stamps
|
||||||
|
// last_used_at and not this. Absent until an alert has arrived since
|
||||||
|
// migration 010 started recording it.
|
||||||
|
LastAlertAt *time.Time `json:"last_alert_at,omitempty"`
|
||||||
|
|
||||||
|
// Alerts24h counts the distinct alerts this source refreshed in the last
|
||||||
|
// day. An alert re-sent every few hours counts once, not once per re-send.
|
||||||
|
Alerts24h int64 `json:"alerts_24h"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleListIntegrations lists a team's integrations with what each has been
|
||||||
|
// delivering. Never the keys: those exist in plaintext only in the response that
|
||||||
|
// created them.
|
||||||
func handleListIntegrations(db *sql.DB) http.HandlerFunc {
|
func handleListIntegrations(db *sql.DB) http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
teamID, ok := teamParam(w, r)
|
teamID, ok := teamParam(w, r)
|
||||||
@@ -367,28 +401,45 @@ func handleListIntegrations(db *sql.DB) http.HandlerFunc {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
now := time.Now()
|
||||||
rows, err := db.QueryContext(r.Context(), `
|
rows, err := db.QueryContext(r.Context(), `
|
||||||
SELECT id, team_id, kind, name, created_at, last_used_at
|
SELECT i.id, i.team_id, i.kind, i.name, i.created_at, i.last_used_at,
|
||||||
FROM integrations
|
-- Scalar subqueries, not a join and GROUP BY: each is a
|
||||||
WHERE team_id = $1
|
-- single range over alerts_integration_idx, where the join
|
||||||
ORDER BY id`, teamID)
|
-- would read every alert a source ever delivered.
|
||||||
|
(SELECT MAX(received_at) FROM alerts WHERE integration_id = i.id),
|
||||||
|
(SELECT COUNT(*) FROM alerts
|
||||||
|
WHERE integration_id = i.id AND received_at >= $2)
|
||||||
|
FROM integrations i
|
||||||
|
WHERE i.team_id = $1
|
||||||
|
ORDER BY i.id`, teamID, now.Add(-sourceQuietAfter).Unix())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
defer rows.Close()
|
defer rows.Close()
|
||||||
|
|
||||||
integrations := []models.Integration{}
|
integrations := []integrationStatus{}
|
||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var i models.Integration
|
var i integrationStatus
|
||||||
var created int64
|
var created int64
|
||||||
var lastUsed *int64
|
var lastUsed, lastAlert *int64
|
||||||
if err := rows.Scan(&i.ID, &i.TeamID, &i.Kind, &i.Name, &created, &lastUsed); err != nil {
|
if err := rows.Scan(&i.ID, &i.TeamID, &i.Kind, &i.Name, &created, &lastUsed,
|
||||||
|
&lastAlert, &i.Alerts24h); err != nil {
|
||||||
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
i.CreatedAt = time.Unix(created, 0).UTC()
|
i.CreatedAt = time.Unix(created, 0).UTC()
|
||||||
i.LastUsedAt = unixPtr(lastUsed)
|
i.LastUsedAt = unixPtr(lastUsed)
|
||||||
|
i.LastAlertAt = unixPtr(lastAlert)
|
||||||
|
switch {
|
||||||
|
case i.LastUsedAt == nil:
|
||||||
|
i.Status = sourceNever
|
||||||
|
case now.Sub(*i.LastUsedAt) > sourceQuietAfter:
|
||||||
|
i.Status = sourceQuiet
|
||||||
|
default:
|
||||||
|
i.Status = sourceActive
|
||||||
|
}
|
||||||
integrations = append(integrations, i)
|
integrations = append(integrations, i)
|
||||||
}
|
}
|
||||||
if err := rows.Err(); err != nil {
|
if err := rows.Err(); err != nil {
|
||||||
@@ -457,6 +508,54 @@ func handleCreateIntegration(db *sql.DB, publicURL string) http.HandlerFunc {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// handleRenameIntegration renames a source. The key is untouched, so nothing
|
||||||
|
// posting with it notices.
|
||||||
|
func handleRenameIntegration(db *sql.DB) http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
teamID, ok := teamParam(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !requireTeamOwner(w, r, teamID) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
id, err := strconv.ParseInt(chi.URLParam(r, "integrationID"), 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
respond(w, http.StatusBadRequest, errResp("invalid integration id"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
var req struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
if err := decodeJSON(r, &req); err != nil {
|
||||||
|
respond(w, http.StatusBadRequest, errResp("invalid request body"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
req.Name = strings.TrimSpace(req.Name)
|
||||||
|
if req.Name == "" {
|
||||||
|
respond(w, http.StatusBadRequest, errResp("name is required"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(req.Name) > 100 {
|
||||||
|
respond(w, http.StatusBadRequest, errResp("name is too long"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
res, err := db.ExecContext(r.Context(),
|
||||||
|
"UPDATE integrations SET name = $1 WHERE id = $2 AND team_id = $3", req.Name, id, teamID)
|
||||||
|
if err != nil {
|
||||||
|
respond(w, http.StatusInternalServerError, errResp("internal error"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if n, _ := res.RowsAffected(); n == 0 {
|
||||||
|
respond(w, http.StatusNotFound, errResp("not found"))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.WriteHeader(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func handleDeleteIntegration(db *sql.DB) http.HandlerFunc {
|
func handleDeleteIntegration(db *sql.DB) http.HandlerFunc {
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
teamID, ok := teamParam(w, r)
|
teamID, ok := teamParam(w, r)
|
||||||
@@ -493,24 +592,32 @@ func integrationPath(key, kind string) string {
|
|||||||
return "/api/integrations/" + key + "/" + kind
|
return "/api/integrations/" + key + "/" + kind
|
||||||
}
|
}
|
||||||
|
|
||||||
// teamIDForKey resolves an integration key to its team, and stamps the key's
|
// alertSource is who an arriving webhook is from: the integration whose key it
|
||||||
|
// used, and the team that integration puts its alerts in.
|
||||||
|
type alertSource struct {
|
||||||
|
integrationID int64
|
||||||
|
teamID int64
|
||||||
|
}
|
||||||
|
|
||||||
|
// sourceForKey resolves an integration key to its source, and stamps the key's
|
||||||
// last use. An unknown key is not an error worth distinguishing: the caller is
|
// last use. An unknown key is not an error worth distinguishing: the caller is
|
||||||
// told nothing beyond "no".
|
// told nothing beyond "no".
|
||||||
func teamIDForKey(ctx context.Context, db *sql.DB, key string) (int64, error) {
|
func sourceForKey(ctx context.Context, db *sql.DB, key string) (alertSource, error) {
|
||||||
var teamID int64
|
var src alertSource
|
||||||
err := db.QueryRowContext(ctx,
|
err := db.QueryRowContext(ctx,
|
||||||
"SELECT team_id FROM integrations WHERE key_hash = $1", hashToken(key)).Scan(&teamID)
|
"SELECT id, team_id FROM integrations WHERE key_hash = $1", hashToken(key)).
|
||||||
|
Scan(&src.integrationID, &src.teamID)
|
||||||
if errors.Is(err, sql.ErrNoRows) {
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
return 0, errUnknownIntegration
|
return alertSource{}, errUnknownIntegration
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, err
|
return alertSource{}, err
|
||||||
}
|
}
|
||||||
// Best effort, like an API key's: a failed stamp must not reject an alert.
|
// Best effort, like an API key's: a failed stamp must not reject an alert.
|
||||||
db.ExecContext(ctx, //nolint:errcheck
|
db.ExecContext(ctx, //nolint:errcheck
|
||||||
"UPDATE integrations SET last_used_at = $1 WHERE key_hash = $2",
|
"UPDATE integrations SET last_used_at = $1 WHERE id = $2",
|
||||||
time.Now().Unix(), hashToken(key))
|
time.Now().Unix(), src.integrationID)
|
||||||
return teamID, nil
|
return src, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
var errUnknownIntegration = errors.New("unknown integration key")
|
var errUnknownIntegration = errors.New("unknown integration key")
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
-- Which alert source an alert last arrived on.
|
||||||
|
--
|
||||||
|
-- Team -> Sources shows when each source last posted, which integrations
|
||||||
|
-- already knew (last_used_at, stamped on every webhook). What it could not say
|
||||||
|
-- was what a source delivered: an alert never recorded the key it came in on, so
|
||||||
|
-- "prod alertmanager" and "staging alertmanager" were indistinguishable once
|
||||||
|
-- inside. This column is that link, and lets the page show each source's last
|
||||||
|
-- alert and how many alerts it has kept fresh over the past day.
|
||||||
|
--
|
||||||
|
-- Last sender wins: every accepted payload restamps it, the way it advances
|
||||||
|
-- received_at. Two sources posting the same fingerprint into one team is
|
||||||
|
-- already one alert, and it is attributed to whichever spoke last.
|
||||||
|
--
|
||||||
|
-- Nullable, and not backfilled. Alerts that arrived before this migration have
|
||||||
|
-- no source, and NULL says so honestly rather than guessing. It heals by itself:
|
||||||
|
-- Alertmanager re-sends every alert each repeat_interval, and each re-send is an
|
||||||
|
-- accepted payload. Deleting a source keeps its alerts, unattributed.
|
||||||
|
ALTER TABLE alerts ADD COLUMN integration_id BIGINT REFERENCES integrations(id) ON DELETE SET NULL;
|
||||||
|
|
||||||
|
CREATE INDEX alerts_integration_idx ON alerts (integration_id, received_at)
|
||||||
|
WHERE integration_id IS NOT NULL;
|
||||||
@@ -363,7 +363,11 @@ input:focus, textarea:focus { outline: none; border-color: var(--accent); box-sh
|
|||||||
.badge.st-dead { background: var(--crit-soft); color: var(--crit); }
|
.badge.st-dead { background: var(--crit-soft); color: var(--crit); }
|
||||||
/* Dormant is the plain badge on purpose: nothing has gone wrong and nothing has
|
/* Dormant is the plain badge on purpose: nothing has gone wrong and nothing has
|
||||||
gone right, which is what the muted default already says. */
|
gone right, which is what the muted default already says. */
|
||||||
.badge.st-dormant { background: var(--surface-2); color: var(--muted); }
|
.badge.st-dormant, .badge.st-never { background: var(--surface-2); color: var(--muted); }
|
||||||
|
.badge.st-active { background: var(--ok-soft); color: var(--ok); }
|
||||||
|
.badge.st-quiet, .badge.st-escalating { background: var(--warn-soft); color: var(--warn); }
|
||||||
|
.badge.st-ready { background: var(--ok-soft); color: var(--ok); }
|
||||||
|
.badge.st-unreachable { background: var(--crit-soft); color: var(--crit); }
|
||||||
.badge.sev-critical { background: var(--crit-soft); color: var(--crit); }
|
.badge.sev-critical { background: var(--crit-soft); color: var(--crit); }
|
||||||
.badge.sev-warning { background: var(--warn-soft); color: var(--warn); }
|
.badge.sev-warning { background: var(--warn-soft); color: var(--warn); }
|
||||||
.badge.sev-info { background: var(--info-soft); color: var(--info); }
|
.badge.sev-info { background: var(--info-soft); color: var(--info); }
|
||||||
@@ -695,16 +699,16 @@ kbd {
|
|||||||
.disabled-row td { opacity: 0.55; }
|
.disabled-row td { opacity: 0.55; }
|
||||||
.btn-sm.danger { color: var(--crit); border-color: var(--crit-soft); }
|
.btn-sm.danger { color: var(--crit); border-color: var(--crit-soft); }
|
||||||
|
|
||||||
/* --- dead man's switches -------------------------------------------------
|
/* --- status lists: alert sources and dead man's switches -----------------
|
||||||
Six columns do not fit a phone, so the table scrolls inside its card rather
|
Six columns do not fit a phone, so the table scrolls inside its card rather
|
||||||
than the page. A heartbeat under a switch with several is indented, the way
|
than the page. A heartbeat under a switch with several is indented, the way
|
||||||
the escalation ladder indents its levels. */
|
the escalation ladder indents its levels. */
|
||||||
.card-head { display: flex; align-items: center; justify-content: space-between; gap: 12px; flex-wrap: wrap; }
|
.card-head { display: flex; align-items: center; justify-content: space-between; gap: 12px; flex-wrap: wrap; }
|
||||||
.table-scroll { overflow-x: auto; margin-top: 12px; }
|
.table-scroll { overflow-x: auto; margin-top: 12px; }
|
||||||
.switch-table th, .switch-table td { white-space: nowrap; }
|
.status-table th, .status-table td { white-space: nowrap; }
|
||||||
.switch-table td:nth-child(2) { white-space: normal; min-width: 12em; }
|
.status-table td.wrap { white-space: normal; min-width: 12em; }
|
||||||
.switch-table .source-row td { border-bottom-style: dashed; }
|
.status-table .source-row td { border-bottom-style: dashed; }
|
||||||
.switch-table .source-row td:first-child { padding-left: 16px; }
|
.status-table .source-row td:first-child { padding-left: 16px; }
|
||||||
.source-labels { display: flex; flex-wrap: wrap; gap: 4px; align-items: center; }
|
.source-labels { display: flex; flex-wrap: wrap; gap: 4px; align-items: center; }
|
||||||
|
|
||||||
.inline-form { display: flex; gap: 8px; margin-top: 12px; }
|
.inline-form { display: flex; gap: 8px; margin-top: 12px; }
|
||||||
@@ -825,6 +829,11 @@ button.rota-day:hover { background: var(--surface-2); }
|
|||||||
.ladder-head { display: flex; align-items: center; gap: 10px; margin-bottom: 6px; }
|
.ladder-head { display: flex; align-items: center; gap: 10px; margin-bottom: 6px; }
|
||||||
.ladder-targets { display: flex; flex-direction: column; gap: 6px; margin-top: 8px; }
|
.ladder-targets { display: flex; flex-direction: column; gap: 6px; margin-top: 8px; }
|
||||||
.target-row { display: flex; gap: 6px; align-items: center; flex-wrap: wrap; }
|
.target-row { display: flex; gap: 6px; align-items: center; flex-wrap: wrap; }
|
||||||
|
.ladder-editor { display: flex; flex-direction: column; gap: 10px; align-items: flex-start; margin-top: 12px; }
|
||||||
|
/* A target that would not wake anybody says why, in place: it is the reason a
|
||||||
|
level is red, and the thing to go and fix. */
|
||||||
|
.target-line { display: flex; gap: 8px; align-items: baseline; flex-wrap: wrap; }
|
||||||
|
.target-problem { color: var(--crit); font-size: 12px; font-weight: 600; }
|
||||||
|
|
||||||
/* An integration key is shown exactly once, so it should look like something
|
/* An integration key is shown exactly once, so it should look like something
|
||||||
to act on rather than another row of text. */
|
to act on rather than another row of text. */
|
||||||
|
|||||||
@@ -133,6 +133,8 @@ export const removeTeamMember = (id, userID) => call('DELETE', `/teams/${id}/mem
|
|||||||
export const integrations = (id) => call('GET', `/teams/${id}/integrations`);
|
export const integrations = (id) => call('GET', `/teams/${id}/integrations`);
|
||||||
export const createIntegration = (id, name) =>
|
export const createIntegration = (id, name) =>
|
||||||
call('POST', `/teams/${id}/integrations`, { body: { name } });
|
call('POST', `/teams/${id}/integrations`, { body: { name } });
|
||||||
|
export const renameIntegration = (id, integrationID, name) =>
|
||||||
|
call('PATCH', `/teams/${id}/integrations/${integrationID}`, { body: { name } });
|
||||||
export const deleteIntegration = (id, integrationID) =>
|
export const deleteIntegration = (id, integrationID) =>
|
||||||
call('DELETE', `/teams/${id}/integrations/${integrationID}`);
|
call('DELETE', `/teams/${id}/integrations/${integrationID}`);
|
||||||
|
|
||||||
|
|||||||
+205
-71
@@ -52,12 +52,10 @@ let freshKey = null; // an integration key, shown once, until the view is left
|
|||||||
export function show(route) {
|
export function show(route) {
|
||||||
const next = route?.tab ?? null;
|
const next = route?.tab ?? null;
|
||||||
// A different sub-section wants different data, so the old answer goes
|
// A different sub-section wants different data, so the old answer goes
|
||||||
// rather than being shown under the new heading until the fetch lands. The
|
// rather than being shown under the new heading until the fetch lands.
|
||||||
// ladder draft goes with it: it is an edit of the page being left.
|
|
||||||
if (next !== tab) {
|
if (next !== tab) {
|
||||||
tab = next;
|
tab = next;
|
||||||
data = null;
|
data = null;
|
||||||
draft = null;
|
|
||||||
}
|
}
|
||||||
if (!data) clear(view(), subnav(), spinner());
|
if (!data) clear(view(), subnav(), spinner());
|
||||||
refresh();
|
refresh();
|
||||||
@@ -185,7 +183,6 @@ function teamPicker() {
|
|||||||
teamID = Number(select.value);
|
teamID = Number(select.value);
|
||||||
data = null;
|
data = null;
|
||||||
freshKey = null;
|
freshKey = null;
|
||||||
draft = null;
|
|
||||||
refresh();
|
refresh();
|
||||||
});
|
});
|
||||||
return h('div', { class: 'card' }, h('h2', { text: 'Team' }), select);
|
return h('div', { class: 'card' }, h('h2', { text: 'Team' }), select);
|
||||||
@@ -457,15 +454,84 @@ function memberSelect(selected) {
|
|||||||
|
|
||||||
// --- escalation ------------------------------------------------------------
|
// --- escalation ------------------------------------------------------------
|
||||||
|
|
||||||
// The ladder is edited as a whole and sent as a whole, because the API replaces
|
const LEVEL_STATUS = {
|
||||||
// it wholesale: the levels are an order, and patching one rung would leave the
|
ready: { label: 'Ready', hint: 'Somebody here can be woken.' },
|
||||||
// numbering of the others undecided.
|
escalating: { label: 'Escalating', hint: 'An unanswered incident has climbed to this level.' },
|
||||||
let draft = null;
|
unreachable: { label: 'Pages nobody', hint: 'Nobody on this level can be woken right now.' },
|
||||||
|
};
|
||||||
|
|
||||||
|
const levelBadge = (status) => statusBadge(LEVEL_STATUS, status, 'ready');
|
||||||
|
|
||||||
|
// One target as the list shows it: who it means today, and why it would not
|
||||||
|
// wake them if it would not.
|
||||||
|
function targetLine(t) {
|
||||||
|
const label = t.kind === 'oncall'
|
||||||
|
? `On call${t.username ? ` · ${t.username}` : ''}`
|
||||||
|
: (t.username || 'Unknown person');
|
||||||
|
return h('div', { class: 'target-line' },
|
||||||
|
h('span', { text: label }),
|
||||||
|
t.problem && h('span', { class: 'target-problem', text: t.problem }));
|
||||||
|
}
|
||||||
|
|
||||||
function escalationCard() {
|
function escalationCard() {
|
||||||
const esc = data.escalation;
|
const esc = data.escalation || {};
|
||||||
if (!draft) {
|
const levels = esc.levels || [];
|
||||||
draft = {
|
|
||||||
|
const rows = levels.map((l) => h('tr', {},
|
||||||
|
h('td', {}, h('strong', { text: `Level ${l.position}` })),
|
||||||
|
h('td', {}, levelBadge(l.status)),
|
||||||
|
h('td', { class: 'wrap' }, ...l.targets.map(targetLine)),
|
||||||
|
h('td', { class: 'muted small', text: duration(l.timeout_seconds * 1000) }),
|
||||||
|
h('td', { class: 'small' }, l.waiting?.length
|
||||||
|
? l.waiting.flatMap((id, i) => [i > 0 && ', ', h('a', { href: `/incidents/${id}`, text: `#${id}` })])
|
||||||
|
: h('span', { class: 'muted', text: '—' })),
|
||||||
|
));
|
||||||
|
|
||||||
|
const facts = [];
|
||||||
|
if (levels.length) {
|
||||||
|
const n = esc.repeat_count || 0;
|
||||||
|
if (n) facts.push(`Then the whole ladder repeats ${n} more ${n === 1 ? 'time' : 'times'}.`);
|
||||||
|
facts.push(esc.fallback_topic
|
||||||
|
? ['Finally the ntfy topic ', h('code', { text: esc.fallback_topic }), ' is paged once.']
|
||||||
|
: 'No fallback topic: after the last level the chain just ends.');
|
||||||
|
facts.push(esc.last_escalated_at
|
||||||
|
? ['Last escalated ',
|
||||||
|
h('span', { title: when(esc.last_escalated_at), text: ago(esc.last_escalated_at) }),
|
||||||
|
' on ', h('a', { href: `/incidents/${esc.last_escalated_incident_id}`, text: `#${esc.last_escalated_incident_id}` }), '.']
|
||||||
|
: 'Nothing has needed to escalate yet.');
|
||||||
|
}
|
||||||
|
|
||||||
|
return h('div', { class: 'card' },
|
||||||
|
h('div', { class: 'card-head' },
|
||||||
|
h('h2', { text: 'Escalation' }),
|
||||||
|
isOwner() && h('button', {
|
||||||
|
class: 'btn', type: 'button', onclick: openLadderEditor,
|
||||||
|
text: levels.length ? 'Edit ladder' : 'Set up ladder',
|
||||||
|
})),
|
||||||
|
h('p', { class: 'muted small' },
|
||||||
|
'When a level’s wait passes and nobody has acknowledged, the next level is ',
|
||||||
|
'paged. Acknowledging or resolving stops it; snoozing pauses it.'),
|
||||||
|
levels.length
|
||||||
|
? h('div', { class: 'table-scroll' },
|
||||||
|
h('table', { class: 'admin-table status-table' },
|
||||||
|
h('thead', {}, h('tr', {},
|
||||||
|
h('th', { text: 'Level' }), h('th', { text: 'Status' }), h('th', { text: 'Pages' }),
|
||||||
|
h('th', { text: 'Then after' }), h('th', { text: 'Waiting now' }))),
|
||||||
|
h('tbody', {}, rows)))
|
||||||
|
: h('p', { class: 'muted' },
|
||||||
|
'No ladder. An unacknowledged incident re-pages the same person every ',
|
||||||
|
'reminder interval and nobody else is woken.'),
|
||||||
|
...facts.map((f) => h('p', { class: 'muted small' }, f)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// The ladder is edited as a whole and sent as a whole, because the API replaces
|
||||||
|
// it wholesale: the levels are an order, and patching one rung would leave the
|
||||||
|
// numbering of the others undecided. The draft lives in the sheet, so a poll of
|
||||||
|
// the page underneath cannot throw away half an edit.
|
||||||
|
function openLadderEditor() {
|
||||||
|
const esc = data.escalation || {};
|
||||||
|
const draft = {
|
||||||
repeat_count: esc.repeat_count || 0,
|
repeat_count: esc.repeat_count || 0,
|
||||||
fallback_topic: esc.fallback_topic || '',
|
fallback_topic: esc.fallback_topic || '',
|
||||||
levels: (esc.levels || []).map((l) => ({
|
levels: (esc.levels || []).map((l) => ({
|
||||||
@@ -473,41 +539,39 @@ function escalationCard() {
|
|||||||
targets: (l.targets || []).map((t) => ({ kind: t.kind, user_id: t.user_id })),
|
targets: (l.targets || []).map((t) => ({ kind: t.kind, user_id: t.user_id })),
|
||||||
})),
|
})),
|
||||||
};
|
};
|
||||||
}
|
|
||||||
|
|
||||||
const body = [];
|
const body = h('div', { class: 'ladder-editor' });
|
||||||
|
const problem = h('p', { class: 'load-error', hidden: true });
|
||||||
|
|
||||||
|
const paint = () => {
|
||||||
|
const parts = [];
|
||||||
if (!draft.levels.length) {
|
if (!draft.levels.length) {
|
||||||
body.push(h('p', { class: 'muted' },
|
parts.push(h('p', { class: 'muted small' }, 'No levels yet. Add the first one.'));
|
||||||
'No ladder. An unacknowledged incident re-pages the same person every ',
|
|
||||||
'reminder interval and nobody else is woken.'));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
draft.levels.forEach((level, i) => {
|
draft.levels.forEach((level, i) => {
|
||||||
body.push(h('div', { class: 'ladder-level' },
|
parts.push(h('div', { class: 'ladder-level' },
|
||||||
h('div', { class: 'ladder-head' },
|
h('div', { class: 'ladder-head' },
|
||||||
h('strong', { text: `Level ${i + 1}` }),
|
h('strong', { text: `Level ${i + 1}` }),
|
||||||
isOwner() && h('button', {
|
h('button', {
|
||||||
class: 'btn-sm danger', type: 'button', text: 'Remove',
|
class: 'btn-sm danger', type: 'button', text: 'Remove',
|
||||||
onclick: () => { draft.levels.splice(i, 1); render(); },
|
onclick: () => { draft.levels.splice(i, 1); paint(); },
|
||||||
})),
|
})),
|
||||||
h('label', {}, 'Wait ', minutesInput(level.timeout_seconds, (secs) => {
|
h('label', {}, 'Wait ', minutesInput(level.timeout_seconds, (secs) => {
|
||||||
level.timeout_seconds = secs;
|
level.timeout_seconds = secs;
|
||||||
}), ' before the next level'),
|
}), ' before the next level'),
|
||||||
h('div', { class: 'ladder-targets' },
|
h('div', { class: 'ladder-targets' },
|
||||||
...level.targets.map((t, ti) => targetRow(level, t, ti)),
|
...level.targets.map((t, ti) => targetRow(level, t, ti, paint)),
|
||||||
isOwner() && h('button', {
|
h('button', {
|
||||||
class: 'btn-sm', type: 'button', text: '+ target',
|
class: 'btn-sm', type: 'button', text: '+ target',
|
||||||
onclick: () => { level.targets.push({ kind: 'oncall' }); render(); },
|
onclick: () => { level.targets.push({ kind: 'oncall' }); paint(); },
|
||||||
})),
|
})),
|
||||||
));
|
));
|
||||||
});
|
});
|
||||||
|
parts.push(h('button', {
|
||||||
if (isOwner()) {
|
|
||||||
body.push(h('button', {
|
|
||||||
class: 'btn-sm', type: 'button', text: '+ level',
|
class: 'btn-sm', type: 'button', text: '+ level',
|
||||||
onclick: () => {
|
onclick: () => {
|
||||||
draft.levels.push({ timeout_seconds: 300, targets: [{ kind: 'oncall' }] });
|
draft.levels.push({ timeout_seconds: 300, targets: [{ kind: 'oncall' }] });
|
||||||
render();
|
paint();
|
||||||
},
|
},
|
||||||
}));
|
}));
|
||||||
|
|
||||||
@@ -520,31 +584,43 @@ function escalationCard() {
|
|||||||
type: 'text', value: draft.fallback_topic, placeholder: 'terdut-oncall-all',
|
type: 'text', value: draft.fallback_topic, placeholder: 'terdut-oncall-all',
|
||||||
oninput: (e) => { draft.fallback_topic = e.target.value; },
|
oninput: (e) => { draft.fallback_topic = e.target.value; },
|
||||||
});
|
});
|
||||||
body.push(h('label', {}, 'Repeat the whole ladder ', repeat, ' more times'));
|
parts.push(h('label', {}, 'Repeat the whole ladder ', repeat, ' more times'));
|
||||||
body.push(h('label', {}, 'Then page this ntfy topic once ', fallback));
|
parts.push(h('label', {}, 'Then page this ntfy topic once ', fallback));
|
||||||
body.push(h('button', {
|
clear(body, ...parts);
|
||||||
class: 'btn', type: 'button', text: 'Save ladder',
|
};
|
||||||
onclick: () => act(() => api.setEscalation(teamID, draft), { resetDraft: true }),
|
paint();
|
||||||
}));
|
|
||||||
}
|
|
||||||
|
|
||||||
return h('div', { class: 'card' },
|
const save = h('button', { class: 'btn btn-primary', type: 'button', text: 'Save ladder' });
|
||||||
h('h2', { text: 'Escalation' }),
|
save.addEventListener('click', async () => {
|
||||||
h('p', { class: 'muted small' },
|
try {
|
||||||
'When a level’s wait passes and nobody has acknowledged, the next level is ',
|
await api.setEscalation(teamID, draft);
|
||||||
'paged. Acknowledging or resolving stops it; snoozing pauses it.'),
|
} catch (err) {
|
||||||
...body,
|
problem.textContent = err.message;
|
||||||
);
|
problem.hidden = false;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
closeSheet(true);
|
||||||
|
refresh();
|
||||||
|
});
|
||||||
|
|
||||||
|
openSheet(() => [
|
||||||
|
h('h2', { class: 'sheet-title', text: 'Edit ladder' }),
|
||||||
|
body,
|
||||||
|
problem,
|
||||||
|
h('div', { class: 'sheet-actions' },
|
||||||
|
h('button', { class: 'btn', type: 'button', text: 'Cancel', onclick: () => closeSheet(false) }),
|
||||||
|
save),
|
||||||
|
]);
|
||||||
}
|
}
|
||||||
|
|
||||||
function targetRow(level, target, index) {
|
function targetRow(level, target, index, repaint) {
|
||||||
const kind = h('select', {},
|
const kind = h('select', {},
|
||||||
h('option', { value: 'oncall', text: 'Whoever is on call', selected: target.kind === 'oncall' }),
|
h('option', { value: 'oncall', text: 'Whoever is on call', selected: target.kind === 'oncall' }),
|
||||||
h('option', { value: 'user', text: 'A specific person', selected: target.kind === 'user' }));
|
h('option', { value: 'user', text: 'A specific person', selected: target.kind === 'user' }));
|
||||||
kind.addEventListener('change', () => {
|
kind.addEventListener('change', () => {
|
||||||
target.kind = kind.value;
|
target.kind = kind.value;
|
||||||
target.user_id = kind.value === 'user' ? (data.members[0] || {}).user_id : undefined;
|
target.user_id = kind.value === 'user' ? (data.members[0] || {}).user_id : undefined;
|
||||||
render();
|
repaint();
|
||||||
});
|
});
|
||||||
|
|
||||||
const who = target.kind === 'user'
|
const who = target.kind === 'user'
|
||||||
@@ -555,10 +631,10 @@ function targetRow(level, target, index) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return h('div', { class: 'target-row' }, kind, who,
|
return h('div', { class: 'target-row' }, kind, who,
|
||||||
isOwner() && h('button', {
|
h('button', {
|
||||||
class: 'btn-sm danger', type: 'button', text: '×',
|
class: 'btn-sm danger', type: 'button', text: '×',
|
||||||
title: 'Remove this target',
|
title: 'Remove this target',
|
||||||
onclick: () => { level.targets.splice(index, 1); render(); },
|
onclick: () => { level.targets.splice(index, 1); repaint(); },
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -576,33 +652,54 @@ function minutesInput(seconds, onChange) {
|
|||||||
function integrationsCard() {
|
function integrationsCard() {
|
||||||
const rows = (data.integrations || []).map((i) =>
|
const rows = (data.integrations || []).map((i) =>
|
||||||
h('tr', {},
|
h('tr', {},
|
||||||
h('td', {}, h('strong', { text: i.name })),
|
h('td', {}, sourceBadge(i.status)),
|
||||||
h('td', { class: 'muted small', text: i.kind }),
|
h('td', { class: 'wrap' },
|
||||||
h('td', { class: 'muted small', text: i.last_used_at ? 'in use' : 'never used' }),
|
h('strong', { text: i.name }),
|
||||||
h('td', {}, isOwner() && h('button', {
|
h('div', { class: 'muted small', text: i.kind })),
|
||||||
|
// When the key last posted, and when an alert last arrived on it. They
|
||||||
|
// differ: a payload with nothing usable in it stamps only the first.
|
||||||
|
h('td', { class: 'muted small' }, timeCell(i.last_used_at)),
|
||||||
|
h('td', { class: 'muted small' }, timeCell(i.last_alert_at)),
|
||||||
|
h('td', { class: 'muted small num', title: 'Distinct alerts refreshed in the last 24 hours',
|
||||||
|
text: String(i.alerts_24h ?? 0) }),
|
||||||
|
h('td', { class: 'muted small' }, h('span', { title: when(i.created_at), text: ago(i.created_at) })),
|
||||||
|
h('td', {}, isOwner() && h('div', { class: 'row-actions' },
|
||||||
|
h('button', {
|
||||||
|
class: 'btn-sm', type: 'button', text: 'Rename', onclick: () => openRenameSource(i),
|
||||||
|
}),
|
||||||
|
h('button', {
|
||||||
class: 'btn-sm danger', type: 'button', text: 'Revoke',
|
class: 'btn-sm danger', type: 'button', text: 'Revoke',
|
||||||
onclick: async () => {
|
onclick: async () => {
|
||||||
if (!(await confirm({
|
if (!(await confirm({
|
||||||
title: `Revoke ${i.name}?`,
|
title: `Revoke ${i.name}?`,
|
||||||
text: 'Anything posting with this key stops delivering immediately.',
|
text: 'Anything posting with this key stops delivering immediately. Alerts it already delivered stay.',
|
||||||
confirmLabel: 'Revoke',
|
confirmLabel: 'Revoke',
|
||||||
danger: true,
|
danger: true,
|
||||||
}))) return;
|
}))) return;
|
||||||
act(() => api.deleteIntegration(teamID, i.id));
|
act(() => api.deleteIntegration(teamID, i.id));
|
||||||
},
|
},
|
||||||
})),
|
}))),
|
||||||
));
|
));
|
||||||
|
|
||||||
return h('div', { class: 'card' },
|
return h('div', { class: 'card' },
|
||||||
|
h('div', { class: 'card-head' },
|
||||||
h('h2', { text: 'Alert sources' }),
|
h('h2', { text: 'Alert sources' }),
|
||||||
|
isOwner() && h('button', {
|
||||||
|
class: 'btn', type: 'button', text: 'New source', onclick: openNewSource,
|
||||||
|
})),
|
||||||
h('p', { class: 'muted small' },
|
h('p', { class: 'muted small' },
|
||||||
'Alerts arrive on an integration key, which says both that the sender may ',
|
'Alerts arrive on an integration key, which says both that the sender may ',
|
||||||
'post and which team the alerts belong to.'),
|
'post and which team the alerts belong to.'),
|
||||||
rows.length
|
|
||||||
? h('table', { class: 'admin-table' }, h('tbody', {}, rows))
|
|
||||||
: h('p', { class: 'muted', text: 'No alert source yet, so nothing can reach this team.' }),
|
|
||||||
freshKey && newKeyPanel(),
|
freshKey && newKeyPanel(),
|
||||||
isOwner() && !freshKey && newIntegrationForm(),
|
rows.length
|
||||||
|
? h('div', { class: 'table-scroll' },
|
||||||
|
h('table', { class: 'admin-table status-table' },
|
||||||
|
h('thead', {}, h('tr', {},
|
||||||
|
h('th', { text: 'Status' }), h('th', { text: 'Source' }),
|
||||||
|
h('th', { text: 'Last webhook' }), h('th', { text: 'Last alert' }),
|
||||||
|
h('th', { class: 'num', text: 'Alerts 24h' }), h('th', { text: 'Created' }), h('th'))),
|
||||||
|
h('tbody', {}, rows)))
|
||||||
|
: h('p', { class: 'muted', text: 'No alert source yet, so nothing can reach this team.' }),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -633,38 +730,77 @@ function newKeyPanel() {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
function newIntegrationForm() {
|
// A sheet with one name field, for adding a source and for renaming one: the two
|
||||||
const name = h('input', { type: 'text', placeholder: 'prod alertmanager', required: true });
|
// differ only in what they call and what they put in the box.
|
||||||
const form = h('form', { class: 'inline-form' }, name,
|
function openNameSheet({ title, submit, value, run }) {
|
||||||
h('button', { class: 'btn', type: 'submit', text: 'Add' }));
|
const name = h('input', {
|
||||||
|
type: 'text', placeholder: 'prod alertmanager', required: true, value, autofocus: true,
|
||||||
|
});
|
||||||
|
const problem = h('p', { class: 'load-error', hidden: true });
|
||||||
|
const form = h('form', { class: 'stacked-form' },
|
||||||
|
h('label', {}, 'Name ', name),
|
||||||
|
problem,
|
||||||
|
h('div', { class: 'sheet-actions' },
|
||||||
|
h('button', { class: 'btn', type: 'button', text: 'Cancel', onclick: () => closeSheet(false) }),
|
||||||
|
h('button', { class: 'btn btn-primary', type: 'submit', text: submit })));
|
||||||
|
|
||||||
form.addEventListener('submit', async (e) => {
|
form.addEventListener('submit', async (e) => {
|
||||||
e.preventDefault();
|
e.preventDefault();
|
||||||
try {
|
try {
|
||||||
freshKey = await api.createIntegration(teamID, name.value.trim());
|
await run(name.value.trim());
|
||||||
await refresh();
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
error = err.message;
|
problem.textContent = err.message;
|
||||||
render();
|
problem.hidden = false;
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
closeSheet(true);
|
||||||
|
refresh();
|
||||||
|
});
|
||||||
|
openSheet(() => [h('h2', { class: 'sheet-title', text: title }), form]);
|
||||||
|
}
|
||||||
|
|
||||||
|
function openNewSource() {
|
||||||
|
openNameSheet({
|
||||||
|
title: 'New source', submit: 'Add source', value: '',
|
||||||
|
// The key comes back once, and the card shows it until dismissed.
|
||||||
|
run: async (name) => { freshKey = await api.createIntegration(teamID, name); },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function openRenameSource(i) {
|
||||||
|
openNameSheet({
|
||||||
|
title: `Rename ${i.name}`, submit: 'Rename', value: i.name,
|
||||||
|
run: (name) => api.renameIntegration(teamID, i.id, name),
|
||||||
});
|
});
|
||||||
return form;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- dead man's switches ---------------------------------------------------
|
// --- dead man's switches ---------------------------------------------------
|
||||||
|
|
||||||
|
// Status badges, shared by the Sources and Switches lists: a table of label and
|
||||||
|
// hint per status, and one function to draw it. Module-level, so the two cards
|
||||||
|
// can be defined in either order.
|
||||||
const SWITCH_STATUS = {
|
const SWITCH_STATUS = {
|
||||||
healthy: { label: 'Healthy', hint: 'Heard from within its timeout.' },
|
healthy: { label: 'Healthy', hint: 'Heard from within its timeout.' },
|
||||||
dead: { label: 'Dead', hint: 'Silent for longer than its timeout.' },
|
dead: { label: 'Dead', hint: 'Silent for longer than its timeout.' },
|
||||||
dormant: { label: 'Dormant', hint: 'Nothing has matched yet, so there is nothing to lose.' },
|
dormant: { label: 'Dormant', hint: 'Nothing has matched yet, so there is nothing to lose.' },
|
||||||
};
|
};
|
||||||
|
|
||||||
function switchBadge(status) {
|
const SOURCE_STATUS = {
|
||||||
const s = SWITCH_STATUS[status] || SWITCH_STATUS.dormant;
|
active: { label: 'Active', hint: 'Posted within the last day.' },
|
||||||
|
quiet: { label: 'Quiet', hint: 'Has posted, but not in the last day. Nothing firing is a fine reason.' },
|
||||||
|
never: { label: 'Never used', hint: 'Nothing has been posted with this key yet.' },
|
||||||
|
};
|
||||||
|
|
||||||
|
function statusBadge(table, status, fallback) {
|
||||||
|
const s = table[status] || table[fallback];
|
||||||
const el = badge(s.label, `st-${status}`);
|
const el = badge(s.label, `st-${status}`);
|
||||||
el.title = s.hint;
|
el.title = s.hint;
|
||||||
return el;
|
return el;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const switchBadge = (status) => statusBadge(SWITCH_STATUS, status, 'dormant');
|
||||||
|
const sourceBadge = (status) => statusBadge(SOURCE_STATUS, status, 'never');
|
||||||
|
|
||||||
const timeCell = (iso) => iso
|
const timeCell = (iso) => iso
|
||||||
? h('span', { title: when(iso), text: ago(iso) })
|
? h('span', { title: when(iso), text: ago(iso) })
|
||||||
: h('span', { class: 'muted', text: 'never' });
|
: h('span', { class: 'muted', text: 'never' });
|
||||||
@@ -681,7 +817,7 @@ const triggeredCell = (iso, incidentID) => {
|
|||||||
function switchRows(sw) {
|
function switchRows(sw) {
|
||||||
const main = h('tr', {},
|
const main = h('tr', {},
|
||||||
h('td', {}, switchBadge(sw.status)),
|
h('td', {}, switchBadge(sw.status)),
|
||||||
h('td', {},
|
h('td', { class: 'wrap' },
|
||||||
h('strong', { text: sw.name }),
|
h('strong', { text: sw.name }),
|
||||||
sw.name !== sw.matcher && h('div', { class: 'muted small' }, h('code', { text: sw.matcher }))),
|
sw.name !== sw.matcher && h('div', { class: 'muted small' }, h('code', { text: sw.matcher }))),
|
||||||
h('td', { class: 'muted small' }, timeCell(sw.last_heartbeat_at)),
|
h('td', { class: 'muted small' }, timeCell(sw.last_heartbeat_at)),
|
||||||
@@ -733,7 +869,7 @@ function deadmanCard() {
|
|||||||
'quiet for longer than the switch’s timeout opens an incident.'),
|
'quiet for longer than the switch’s timeout opens an incident.'),
|
||||||
switches.length
|
switches.length
|
||||||
? h('div', { class: 'table-scroll' },
|
? h('div', { class: 'table-scroll' },
|
||||||
h('table', { class: 'admin-table switch-table' },
|
h('table', { class: 'admin-table status-table' },
|
||||||
h('thead', {}, h('tr', {},
|
h('thead', {}, h('tr', {},
|
||||||
h('th', { text: 'Status' }), h('th', { text: 'Switch' }),
|
h('th', { text: 'Status' }), h('th', { text: 'Switch' }),
|
||||||
h('th', { text: 'Last heartbeat' }), h('th', { text: 'Last triggered' }),
|
h('th', { text: 'Last heartbeat' }), h('th', { text: 'Last triggered' }),
|
||||||
@@ -836,14 +972,12 @@ function membersCard() {
|
|||||||
// act runs a write and reloads. Errors are shown rather than thrown away: a
|
// act runs a write and reloads. Errors are shown rather than thrown away: a
|
||||||
// 409 from the last-owner guard or the schedule's conflict rule is the server
|
// 409 from the last-owner guard or the schedule's conflict rule is the server
|
||||||
// explaining itself, and the reader needs to see it.
|
// explaining itself, and the reader needs to see it.
|
||||||
async function act(fn, { resetDraft = false } = {}) {
|
async function act(fn) {
|
||||||
try {
|
try {
|
||||||
await fn();
|
await fn();
|
||||||
error = null;
|
error = null;
|
||||||
if (resetDraft) draft = null;
|
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
error = err.message;
|
error = err.message;
|
||||||
}
|
}
|
||||||
if (!resetDraft) draft = null;
|
|
||||||
await refresh();
|
await refresh();
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user