Niklas Ye
f344bb707b
Each alert carries a generatorURL pointing at the Prometheus rule that fired it. The server now parses that link, queries the same Prometheus for the rule's expression and draws it under the alert, with the current value beside it. The browser never talks to Prometheus, so the CSP stays at connect-src 'self'. The URL comes out of a webhook payload, so whoever holds an integration key chooses it. Querying it is therefore opt-in and bounded by an allowlist, TERDUT_PROMETHEUS_ALLOWED_HOSTS (names, "*" as one label, a leading "**" as several, IPs, CIDRs). The check runs on the address the dialer is about to use, a name that resolves to loopback or link-local is refused even when it matches, and redirects and proxy variables are ignored. A worker takes a snapshot when an incident opens and another when it closes, so the graph outlives Prometheus' retention. It finds its work in the database rather than being triggered from the webhook, so ingestion is untouched and a restart loses nothing. While an incident is open the metrics endpoint also queries live, cached for 15 seconds. The rule expression keeps its threshold, so Prometheus returns data only while the alert condition holds; "no data" for the current value means the condition cleared. Stripping the threshold needs a PromQL parser and is left out. Claude-Session: https://claude.ai/code/session_016mBLURvJoMuUEr9cB2RpUN
26 lines
1.1 KiB
SQL
26 lines
1.1 KiB
SQL
-- Metrics captured from the Prometheus an alert came from (its generatorURL),
|
|
-- once when the incident opens and once when it closes, so the graph outlives
|
|
-- Prometheus' own retention. One row per incident, alert and kind.
|
|
--
|
|
-- A row exists whether or not the capture worked: a failure is recorded in
|
|
-- error with the attempt count, which is what stops the worker from retrying a
|
|
-- host that is not on the allowlist (or is gone) forever.
|
|
|
|
CREATE TABLE incident_metric_snapshots (
|
|
id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
|
incident_id bigint NOT NULL REFERENCES incidents(id) ON DELETE CASCADE,
|
|
alert_id bigint NOT NULL REFERENCES alerts(id) ON DELETE CASCADE,
|
|
kind text NOT NULL CHECK (kind IN ('opened', 'closed')),
|
|
expr text NOT NULL DEFAULT '',
|
|
window_start bigint NOT NULL,
|
|
window_end bigint NOT NULL,
|
|
step_seconds integer NOT NULL DEFAULT 0,
|
|
series jsonb NOT NULL DEFAULT '[]'::jsonb,
|
|
error text,
|
|
attempts integer NOT NULL DEFAULT 1,
|
|
captured_at bigint NOT NULL,
|
|
UNIQUE (incident_id, alert_id, kind)
|
|
);
|
|
|
|
CREATE INDEX incident_metric_snapshots_alert_id_idx ON incident_metric_snapshots (alert_id);
|