niklas/terdut-server
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 35 Wiki Activity

Move the database to Postgres, before teams need the schema
CI / chart (pull_request) Successful in 1s
Details
CI / security (pull_request) Successful in 17s
Details
CI / test (pull_request) Successful in 2m5s
Details

Browse Source 
First step of #1, and it goes first for one reason: #4 adds a team_id to
nearly every table, and doing that twice -- once for SQLite, once for
Postgres -- is work nobody gets paid for. The teams migrations now only
have to be written against one database.

The ten SQLite migrations are replaced by a single Postgres baseline
rather than ported one by one. They were incremental in a way that has
no value on a fresh install: 004 adds columns 008 drops again, and 008's
backfill rewrites data a Postgres database never had. The history stays
in git; the schema they add up to is now 001_baseline.sql.

Timestamps stay BIGINT unix seconds and are NOT converted to timestamptz.
Everything in Go already speaks epochs, so converting would have been a
second, larger change riding along inside this one. It is worth doing on
its own. The JSON columns did move to jsonb, because #4 will want to
filter and index on labels.

Most of the port is mechanical -- 170 placeholders from ? to $1 -- but
four things needed more than a search and replace:

  * Dynamically built WHERE clauses cannot keep their numbering straight
    by hand, so they hand out placeholders through sqlArgs instead. A
    filter can now be added or reordered without renumbering anything.

  * SUM(resolved_at IS NULL) was SQLite counting a boolean as 0 or 1.
    Postgres has no sum(boolean), and this was breaking every dead man's
    switch -- silently, since the sweeper only logs. Now COUNT(*) FILTER.

  * unixepoch() became FLOOR(EXTRACT(EPOCH FROM now()))::bigint. The
    FLOOR is load-bearing: a bare cast rounds half up, so a row written
    at .6 of a second claimed a timestamp a second in the future and
    disagreed with the time.Now().Unix() the Go side stamps.

  * The unique-violation check matched SQLite's error text. It matches
    SQLSTATE 23505 now, so a renamed constraint cannot turn a 409 back
    into a 500.

Tests need a real Postgres, because there is no in-memory Postgres the
way there was an in-memory SQLite. Each test gets its own schema on a
shared server -- cheaper than a database each, and still isolated.
TERDUT_TEST_DSN says where it is; `make test-db` starts one locally and
ci.yaml runs one as a service container. An unset DSN fails the suite
rather than skipping it: a run that quietly tests nothing is worse than
one that does not run.

TestMigration_BackfillCarriesAckAndComments is deleted along with the
migrations it replayed. What it protected -- an upgrade not losing
acknowledgements and comments -- now belongs to scripts/sqlite-to-postgres.go,
which is build-tagged so the SQLite driver stays out of the server
binary. Both are meant to be deleted once this install has migrated.

The chart loses the PVC, the data volume and the python backup sidecar,
and requires database.dsnSecret.name: it provisions no database and
cannot guess where the credentials live, so a render without it is meant
to fail. Backups move to where Postgres actually runs. The other half of
that -- the postgresql CR, the k8up pg_dump annotation and the network
policy -- is a change to the wrapper chart in Ryuvia/charts and is not in
here.

Verified rather than assumed: the gate is green with -race against
Postgres 17, govulncheck and gitleaks are clean, and the migration script
was run end to end against a SQLite database built at the old schema and
seeded in every table. Ids survive, so incidents keep their numbers and
every foreign key still points where it did; the identity sequences are
moved past the copied ids, and a webhook after the migration opened
incident 12 rather than colliding at 1.
This commit is contained in:
Niklas Ye
2026-09-20 10:44:12 +02:00
parent 989425e550
commit dc39e3a5d3
44 changed files with 1004 additions and 725 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/api/incidents.go
+28 -35
View File
@@ -17,13 +17,12 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
q := r.URL.Query()
where := []string{}
args := []any{}
args := &sqlArgs{}
// Without an explicit status the queue shows open work, which is what an
// on-call person opens the tool to see.
if status := q.Get("status"); status != "" {
where = append(where, "i.status = ?")
args = append(args, status)
where = append(where, "i.status = "+args.add(status))
} else {
where = append(where, "i.resolved_at IS NULL")
}
@@ -36,33 +35,27 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
// A snooze expires by simply falling into the past; nothing sweeps it.
if q.Get("snoozed") == "true" {
where = append(where, "i.snoozed_until > ?")
args = append(args, time.Now().Unix())
where = append(where, "i.snoozed_until > "+args.add(time.Now().Unix()))
} else {
where = append(where, "(i.snoozed_until IS NULL OR i.snoozed_until <= ?)")
args = append(args, time.Now().Unix())
where = append(where, "(i.snoozed_until IS NULL OR i.snoozed_until <= "+args.add(time.Now().Unix())+")")
}
if severity := q.Get("severity"); severity != "" {
where = append(where, "i.severity = ?")
args = append(args, severity)
where = append(where, "i.severity = "+args.add(severity))
}
if assignee := q.Get("assigned_to"); assignee != "" {
if n, err := strconv.ParseInt(assignee, 10, 64); err == nil {
where = append(where, "i.assigned_to = ?")
args = append(args, n)
where = append(where, "i.assigned_to = "+args.add(n))
}
}
if from := q.Get("from"); from != "" {
if t, err := time.Parse("2006-01-02", from); err == nil {
where = append(where, "i.triggered_at >= ?")
args = append(args, t.UTC().Unix())
where = append(where, "i.triggered_at >= "+args.add(t.UTC().Unix()))
}
}
if to := q.Get("to"); to != "" {
if t, err := time.Parse("2006-01-02", to); err == nil {
where = append(where, "i.triggered_at < ?")
args = append(args, t.UTC().AddDate(0, 0, 1).Unix())
where = append(where, "i.triggered_at < "+args.add(t.UTC().AddDate(0, 0, 1).Unix()))
}
}
@@ -77,12 +70,11 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
if q.Get("sort") == "severity" {
order = severityRankSQL("i.severity") + " DESC, i.triggered_at DESC"
}
args = append(args, limit)
rows, err := db.QueryContext(r.Context(),
fmt.Sprintf("%s WHERE %s ORDER BY %s LIMIT ?",
incidentSelectFrom, strings.Join(where, " AND "), order),
args...)
fmt.Sprintf("%s WHERE %s ORDER BY %s LIMIT %s",
incidentSelectFrom, strings.Join(where, " AND "), order, args.add(limit)),
args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -158,7 +150,7 @@ func handleIncidentTimeline(db *sql.DB) http.HandlerFunc {
e.alert_id, e.detail, e.created_at
FROM incident_events e
LEFT JOIN users u ON u.id = e.user_id
WHERE e.incident_id = ?
WHERE e.incident_id = $1
ORDER BY e.created_at ASC, e.id ASC`, id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -214,7 +206,7 @@ func handleIncidentUnacknowledge(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
`UPDATE incidents SET status = 'triggered', acknowledged_by = NULL, acknowledged_at = NULL
WHERE id = ? AND resolved_at IS NULL`, id) {
WHERE id = $1 AND resolved_at IS NULL`, id) {
return
}
if err := logEvent(r.Context(), db, id, evUnacknowledged, &user.ID, nil, nil); err != nil {
@@ -237,8 +229,8 @@ func handleIncidentResolve(db *sql.DB) http.HandlerFunc {
}
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
`UPDATE incidents SET status = 'resolved', resolved_at = ?, resolution_source = ?
WHERE id = ? AND resolved_at IS NULL`,
`UPDATE incidents SET status = 'resolved', resolved_at = $1, resolution_source = $2
WHERE id = $3 AND resolved_at IS NULL`,
time.Now().Unix(), incidentResolutionManual, id) {
return
}
@@ -269,13 +261,13 @@ func handleIncidentAssign(db *sql.DB) http.HandlerFunc {
}
var exists int
if err := db.QueryRowContext(r.Context(),
"SELECT 1 FROM users WHERE id = ?", req.UserID).Scan(&exists); err != nil {
"SELECT 1 FROM users WHERE id = $1", req.UserID).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("user not found"))
return
}
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET assigned_to = ? WHERE id = ? AND resolved_at IS NULL",
"UPDATE incidents SET assigned_to = $1 WHERE id = $2 AND resolved_at IS NULL",
req.UserID, id) {
return
}
@@ -333,7 +325,7 @@ func handleIncidentSnooze(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET snoozed_until = ? WHERE id = ? AND resolved_at IS NULL",
"UPDATE incidents SET snoozed_until = $1 WHERE id = $2 AND resolved_at IS NULL",
until.Unix(), id) {
return
}
@@ -354,7 +346,7 @@ func handleIncidentUnsnooze(db *sql.DB) http.HandlerFunc {
}
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET snoozed_until = NULL WHERE id = ? AND resolved_at IS NULL", id) {
"UPDATE incidents SET snoozed_until = NULL WHERE id = $1 AND resolved_at IS NULL", id) {
return
}
if err := logEvent(r.Context(), db, id, evUnsnoozed, &user.ID, nil, nil); err != nil {
@@ -372,7 +364,7 @@ func handleIncidentArchive(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"UPDATE incidents SET archived_at = unixepoch() WHERE id = ?", id)
"UPDATE incidents SET archived_at = "+nowEpoch+" WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -392,7 +384,7 @@ func handleIncidentUnarchive(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"UPDATE incidents SET archived_at = NULL WHERE id = ?", id)
"UPDATE incidents SET archived_at = NULL WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -430,14 +422,15 @@ func handleCreateNote(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
now := time.Now()
res, err := db.ExecContext(r.Context(), `
var eventID int64
err := db.QueryRowContext(r.Context(), `
INSERT INTO incident_events (incident_id, type, user_id, detail, created_at)
VALUES (?, ?, ?, ?, ?)`, id, evNote, user.ID, req.Content, now.Unix())
VALUES ($1, $2, $3, $4, $5)
RETURNING id`, id, evNote, user.ID, req.Content, now.Unix()).Scan(&eventID)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
eventID, _ := res.LastInsertId()
respond(w, http.StatusCreated, models.IncidentEvent{
ID: eventID,
@@ -468,7 +461,7 @@ func handleDeleteNote(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
res, err := db.ExecContext(r.Context(), `
DELETE FROM incident_events
WHERE id = ? AND incident_id = ? AND type = ? AND user_id = ?`,
WHERE id = $1 AND incident_id = $2 AND type = $3 AND user_id = $4`,
eventID, id, evNote, user.ID)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -498,7 +491,7 @@ func incidentIDParam(w http.ResponseWriter, r *http.Request) (int64, bool) {
func incidentExists(w http.ResponseWriter, r *http.Request, db *sql.DB, id int64) bool {
var exists int
if err := db.QueryRowContext(r.Context(),
"SELECT 1 FROM incidents WHERE id = ?", id).Scan(&exists); err != nil {
"SELECT 1 FROM incidents WHERE id = $1", id).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("incident not found"))
return false
}
@@ -538,7 +531,7 @@ func respondIncident(w http.ResponseWriter, r *http.Request, db *sql.DB, id int6
func incidentAlerts(r *http.Request, db *sql.DB, id int64) ([]models.Alert, error) {
rows, err := db.QueryContext(r.Context(), alertSelectFrom+`
JOIN incident_alerts m ON m.alert_id = a.id
WHERE m.incident_id = ?
WHERE m.incident_id = $1
ORDER BY a.received_at DESC`, id)
if err != nil {
return nil, err