Graph the metric behind each alert on the incident page
Each alert carries a generatorURL pointing at the Prometheus rule that fired it. The server now parses that link, queries the same Prometheus for the rule's expression and draws it under the alert, with the current value beside it. The browser never talks to Prometheus, so the CSP stays at connect-src 'self'. The URL comes out of a webhook payload, so whoever holds an integration key chooses it. Querying it is therefore opt-in and bounded by an allowlist, TERDUT_PROMETHEUS_ALLOWED_HOSTS (names, "*" as one label, a leading "**" as several, IPs, CIDRs). The check runs on the address the dialer is about to use, a name that resolves to loopback or link-local is refused even when it matches, and redirects and proxy variables are ignored. A worker takes a snapshot when an incident opens and another when it closes, so the graph outlives Prometheus' retention. It finds its work in the database rather than being triggered from the webhook, so ingestion is untouched and a restart loses nothing. While an incident is open the metrics endpoint also queries live, cached for 15 seconds. The rule expression keeps its threshold, so Prometheus returns data only while the alert condition holds; "no data" for the current value means the condition cleared. Stripping the threshold needs a PromQL parser and is left out. Claude-Session: https://claude.ai/code/session_016mBLURvJoMuUEr9cB2RpUN
This commit is contained in:
@@ -120,6 +120,14 @@ spec:
|
||||
value: {{ .Values.trustedProxies | quote }}
|
||||
- name: TERDUT_OPERATOR_MODE
|
||||
value: {{ .Values.operatorMode | quote }}
|
||||
{{- if .Values.prometheus.allowedHosts }}
|
||||
- name: TERDUT_PROMETHEUS_ALLOWED_HOSTS
|
||||
value: {{ join "," .Values.prometheus.allowedHosts | quote }}
|
||||
- name: TERDUT_PROMETHEUS_PREROLL
|
||||
value: {{ .Values.prometheus.preroll | quote }}
|
||||
- name: TERDUT_PROMETHEUS_TIMEOUT
|
||||
value: {{ .Values.prometheus.timeout | quote }}
|
||||
{{- end }}
|
||||
{{- if .Values.oidc.enabled }}
|
||||
- name: TERDUT_OIDC_ISSUER
|
||||
value: {{ required "oidc.issuer is required when oidc.enabled" .Values.oidc.issuer | quote }}
|
||||
|
||||
@@ -114,6 +114,18 @@ passwordLogin: true
|
||||
# configuration from here on.
|
||||
operatorMode: false
|
||||
|
||||
# Metrics on incidents: the server queries the Prometheus an alert came from
|
||||
# (the host in its generatorURL) and graphs the alert's expression. The URL is
|
||||
# taken from a webhook payload, so only hosts listed here are ever contacted;
|
||||
# empty turns the feature off. Entries are host names, with "*" as one DNS label
|
||||
# and a leading "**" as one or more (e.g. "*.monitoring.svc.cluster.local",
|
||||
# "prometheus-*.monitoring"), IPs or CIDR ranges, each with an optional ":port".
|
||||
prometheus:
|
||||
allowedHosts: []
|
||||
# How far before an alert started its graph begins.
|
||||
preroll: 1h
|
||||
timeout: 10s
|
||||
|
||||
# Single sign-on through an OpenID Connect provider such as Authentik.
|
||||
#
|
||||
# At the provider, create an OAuth2/OpenID application whose redirect URI is
|
||||
|
||||
Reference in New Issue
Block a user