niklas/terdut-server
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 35 Wiki Activity

Move the database to Postgres, before teams need the schema
CI / chart (pull_request) Successful in 1s
Details
CI / security (pull_request) Successful in 17s
Details
CI / test (pull_request) Successful in 2m5s
Details

Browse Source 
First step of #1, and it goes first for one reason: #4 adds a team_id to
nearly every table, and doing that twice -- once for SQLite, once for
Postgres -- is work nobody gets paid for. The teams migrations now only
have to be written against one database.

The ten SQLite migrations are replaced by a single Postgres baseline
rather than ported one by one. They were incremental in a way that has
no value on a fresh install: 004 adds columns 008 drops again, and 008's
backfill rewrites data a Postgres database never had. The history stays
in git; the schema they add up to is now 001_baseline.sql.

Timestamps stay BIGINT unix seconds and are NOT converted to timestamptz.
Everything in Go already speaks epochs, so converting would have been a
second, larger change riding along inside this one. It is worth doing on
its own. The JSON columns did move to jsonb, because #4 will want to
filter and index on labels.

Most of the port is mechanical -- 170 placeholders from ? to $1 -- but
four things needed more than a search and replace:

  * Dynamically built WHERE clauses cannot keep their numbering straight
    by hand, so they hand out placeholders through sqlArgs instead. A
    filter can now be added or reordered without renumbering anything.

  * SUM(resolved_at IS NULL) was SQLite counting a boolean as 0 or 1.
    Postgres has no sum(boolean), and this was breaking every dead man's
    switch -- silently, since the sweeper only logs. Now COUNT(*) FILTER.

  * unixepoch() became FLOOR(EXTRACT(EPOCH FROM now()))::bigint. The
    FLOOR is load-bearing: a bare cast rounds half up, so a row written
    at .6 of a second claimed a timestamp a second in the future and
    disagreed with the time.Now().Unix() the Go side stamps.

  * The unique-violation check matched SQLite's error text. It matches
    SQLSTATE 23505 now, so a renamed constraint cannot turn a 409 back
    into a 500.

Tests need a real Postgres, because there is no in-memory Postgres the
way there was an in-memory SQLite. Each test gets its own schema on a
shared server -- cheaper than a database each, and still isolated.
TERDUT_TEST_DSN says where it is; `make test-db` starts one locally and
ci.yaml runs one as a service container. An unset DSN fails the suite
rather than skipping it: a run that quietly tests nothing is worse than
one that does not run.

TestMigration_BackfillCarriesAckAndComments is deleted along with the
migrations it replayed. What it protected -- an upgrade not losing
acknowledgements and comments -- now belongs to scripts/sqlite-to-postgres.go,
which is build-tagged so the SQLite driver stays out of the server
binary. Both are meant to be deleted once this install has migrated.

The chart loses the PVC, the data volume and the python backup sidecar,
and requires database.dsnSecret.name: it provisions no database and
cannot guess where the credentials live, so a render without it is meant
to fail. Backups move to where Postgres actually runs. The other half of
that -- the postgresql CR, the k8up pg_dump annotation and the network
policy -- is a change to the wrapper chart in Ryuvia/charts and is not in
here.

Verified rather than assumed: the gate is green with -race against
Postgres 17, govulncheck and gitleaks are clean, and the migration script
was run end to end against a SQLite database built at the old schema and
seeded in every table. Ids survive, so incidents keep their numbers and
every foreign key still points where it did; the identity sequences are
moved past the copied ids, and a webhook after the migration opened
incident 12 rather than colliding at 1.
This commit is contained in:
Niklas Ye
2026-09-20 10:44:12 +02:00
parent 989425e550
commit dc39e3a5d3
44 changed files with 1004 additions and 725 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/api/alertmanager.go
+13 -14
View File
@@ -171,7 +171,7 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman DeadmanConfig, alerts
var prevStartsAt int64
existed := true
switch err := tx.QueryRowContext(ctx,
"SELECT status, starts_at FROM alerts WHERE fingerprint = ?", a.Fingerprint,
"SELECT status, starts_at FROM alerts WHERE fingerprint = $1", a.Fingerprint,
).Scan(&prevStatus, &prevStartsAt); {
case err == sql.ErrNoRows:
existed = false
@@ -212,8 +212,8 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman DeadmanConfig, alerts
INSERT INTO alerts
(fingerprint, name, status, labels, annotations, starts_at, ends_at,
generator_url, received_at, resolution_source)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(fingerprint) DO UPDATE SET
VALUES ($1, $2, $3, $4::jsonb, $5::jsonb, $6, $7, $8, $9, $10)
ON CONFLICT (fingerprint) DO UPDATE SET
status = excluded.status,
labels = excluded.labels,
annotations = excluded.annotations,
@@ -245,7 +245,7 @@ func upsertAlerts(ctx context.Context, tx *sql.Tx, deadman DeadmanConfig, alerts
var curStatus string
var curStartsAt int64
if err := tx.QueryRowContext(ctx,
"SELECT id, status, starts_at FROM alerts WHERE fingerprint = ?", a.Fingerprint,
"SELECT id, status, starts_at FROM alerts WHERE fingerprint = $1", a.Fingerprint,
).Scan(&id, &curStatus, &curStartsAt); err != nil {
return nil, err
}
@@ -315,7 +315,7 @@ func incidentForGroup(ctx context.Context, tx *sql.Tx, notify NotifyConfig, payl
var id int64
switch err := tx.QueryRowContext(ctx,
"SELECT id FROM incidents WHERE group_key = ? AND resolved_at IS NULL", groupKey,
"SELECT id FROM incidents WHERE group_key = $1 AND resolved_at IS NULL", groupKey,
).Scan(&id); {
case err == nil:
return id, nil
@@ -349,15 +349,13 @@ func openIncident(ctx context.Context, q querier, notify NotifyConfig, groupKey,
labelsJSON = []byte("{}")
}
res, err := q.ExecContext(ctx, `
var id int64
err = q.QueryRowContext(ctx, `
INSERT INTO incidents (group_key, title, group_labels, status, severity, triggered_at, assigned_to)
VALUES (?, ?, ?, 'triggered', ?, ?, ?)`,
VALUES ($1, $2, $3::jsonb, 'triggered', $4, $5, $6)
RETURNING id`,
groupKey, title, string(labelsJSON), severity,
time.Now().Unix(), onCall)
if err != nil {
return 0, err
}
id, err := res.LastInsertId()
time.Now().Unix(), onCall).Scan(&id)
if err != nil {
return 0, err
}
@@ -385,8 +383,9 @@ func openIncident(ctx context.Context, q querier, notify NotifyConfig, groupKey,
// first time. Re-sends of an already-linked alert are silent.
func linkAlert(ctx context.Context, tx *sql.Tx, incidentID, alertID int64) error {
res, err := tx.ExecContext(ctx, `
INSERT OR IGNORE INTO incident_alerts (incident_id, alert_id, added_at)
VALUES (?, ?, ?)`, incidentID, alertID, time.Now().Unix())
INSERT INTO incident_alerts (incident_id, alert_id, added_at)
VALUES ($1, $2, $3)
ON CONFLICT (incident_id, alert_id) DO NOTHING`, incidentID, alertID, time.Now().Unix())
if err != nil {
return err
}
internal/api/alerts.go
+9 -15
View File
@@ -36,15 +36,13 @@ func handleListAlerts(db *sql.DB) http.HandlerFunc {
q := r.URL.Query()
where := []string{}
args := []any{}
args := &sqlArgs{}
if status := q.Get("status"); status != "" {
where = append(where, "a.status = ?")
args = append(args, status)
where = append(where, "a.status = "+args.add(status))
}
if name := q.Get("name"); name != "" {
where = append(where, "a.name = ?")
args = append(args, name)
where = append(where, "a.name = "+args.add(name))
}
if archived := q.Get("archived"); archived == "true" {
where = append(where, "a.archived_at IS NOT NULL")
@@ -53,21 +51,18 @@ func handleListAlerts(db *sql.DB) http.HandlerFunc {
}
if incidentID := q.Get("incident_id"); incidentID != "" {
if n, err := strconv.ParseInt(incidentID, 10, 64); err == nil {
where = append(where, "a.id IN (SELECT alert_id FROM incident_alerts WHERE incident_id = ?)")
args = append(args, n)
where = append(where, "a.id IN (SELECT alert_id FROM incident_alerts WHERE incident_id = "+args.add(n)+")")
}
}
if from := q.Get("from"); from != "" {
if t, err := time.Parse("2006-01-02", from); err == nil {
where = append(where, "a.received_at >= ?")
args = append(args, t.UTC().Unix())
where = append(where, "a.received_at >= "+args.add(t.UTC().Unix()))
}
}
if to := q.Get("to"); to != "" {
if t, err := time.Parse("2006-01-02", to); err == nil {
where = append(where, "a.received_at < ?")
args = append(args, t.UTC().AddDate(0, 0, 1).Unix())
where = append(where, "a.received_at < "+args.add(t.UTC().AddDate(0, 0, 1).Unix()))
}
}
@@ -82,11 +77,10 @@ func handleListAlerts(db *sql.DB) http.HandlerFunc {
if len(where) > 0 {
clause = strings.Join(where, " AND ")
}
args = append(args, limit)
rows, err := db.QueryContext(r.Context(),
fmt.Sprintf("%s WHERE %s ORDER BY a.received_at DESC LIMIT ?", alertSelectFrom, clause),
args...)
fmt.Sprintf("%s WHERE %s ORDER BY a.received_at DESC LIMIT %s", alertSelectFrom, clause, args.add(limit)),
args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -128,7 +122,7 @@ func handleGetAlert(db *sql.DB) http.HandlerFunc {
// fetchAlert loads a single alert by ID using the shared query.
func fetchAlert(ctx context.Context, db *sql.DB, id int64) (models.Alert, error) {
return scanAlert(db.QueryRowContext(ctx, alertSelectFrom+" WHERE a.id = ?", id))
return scanAlert(db.QueryRowContext(ctx, alertSelectFrom+" WHERE a.id = $1", id))
}
// scanner is satisfied by both *sql.Row and *sql.Rows.
internal/api/api_test.go
+12 -19
View File
@@ -13,7 +13,6 @@ import (
"time"
"git.ryuvia.com/niklas/terdut-server/internal/api"
"git.ryuvia.com/niklas/terdut-server/internal/db"
)
// ts wraps httptest.Server with a pre-bootstrapped API key. db is exposed so
@@ -26,7 +25,7 @@ type ts struct {
deadman api.DeadmanConfig
}
// newTS builds a server over a fresh in-memory database. Notifications are off
// newTS builds a server over a fresh database. Notifications are off
// unless a NotifyConfig is passed, so tests that predate them are unaffected.
// Dead man's switches are off too — see newDeadmanTS.
func newTS(t *testing.T, notify ...api.NotifyConfig) *ts {
@@ -46,15 +45,9 @@ func newDeadmanTS(t *testing.T, deadman api.DeadmanConfig, notify ...api.NotifyC
cfg = notify[0]
}
database, err := db.Open(":memory:")
if err != nil {
t.Fatalf("open db: %v", err)
}
if err := db.Migrate(database); err != nil {
t.Fatalf("migrate: %v", err)
}
database := newTestDB(t)
srv := httptest.NewServer(api.NewRouter(database, cfg, deadman))
t.Cleanup(func() { srv.Close(); database.Close() })
t.Cleanup(srv.Close)
body, _ := json.Marshal(map[string]string{"username": "admin", "email": "admin@test.com"})
resp, err := http.Post(srv.URL+"/api/bootstrap", "application/json", bytes.NewReader(body))
@@ -84,7 +77,7 @@ func (s *ts) exec(t *testing.T, query string, args ...any) {
func (s *ts) alertRow(t *testing.T, fingerprint string) (status string, source *string, archivedAt *int64) {
t.Helper()
err := s.db.QueryRow(
"SELECT status, resolution_source, archived_at FROM alerts WHERE fingerprint = ?",
"SELECT status, resolution_source, archived_at FROM alerts WHERE fingerprint = $1",
fingerprint).Scan(&status, &source, &archivedAt)
if err != nil {
t.Fatalf("read alert %s: %v", fingerprint, err)
@@ -96,7 +89,7 @@ func (s *ts) alertRow(t *testing.T, fingerprint string) (status string, source *
func (s *ts) alertTimes(t *testing.T, fingerprint string) (startsAt, receivedAt int64) {
t.Helper()
err := s.db.QueryRow(
"SELECT starts_at, received_at FROM alerts WHERE fingerprint = ?",
"SELECT starts_at, received_at FROM alerts WHERE fingerprint = $1",
fingerprint).Scan(&startsAt, &receivedAt)
if err != nil {
t.Fatalf("read alert times %s: %v", fingerprint, err)
@@ -109,7 +102,7 @@ func (s *ts) alertEndsAt(t *testing.T, fingerprint string) *int64 {
t.Helper()
var endsAt *int64
if err := s.db.QueryRow(
"SELECT ends_at FROM alerts WHERE fingerprint = ?", fingerprint).Scan(&endsAt); err != nil {
"SELECT ends_at FROM alerts WHERE fingerprint = $1", fingerprint).Scan(&endsAt); err != nil {
t.Fatalf("read ends_at %s: %v", fingerprint, err)
}
return endsAt
@@ -556,7 +549,7 @@ func TestExpiry_StaleFiringAlert(t *testing.T) {
postAlert(t, s, "stale1", "firing", time.Now().Add(-24*time.Hour).Format(time.RFC3339), zeroTime)
// Age the last-seen timestamp past the staleness window.
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'stale1'",
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'stale1'",
time.Now().Add(-10*time.Hour).Unix())
sweep(t, s, 6*time.Hour)
@@ -653,10 +646,10 @@ func TestWebhook_RefireUnarchivesAndClearsSource(t *testing.T) {
postAlert(t, s, "refire1", "firing", time.Now().Add(-24*time.Hour).Format(time.RFC3339), zeroTime)
// Expire it, then archive it.
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'refire1'",
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'refire1'",
time.Now().Add(-10*time.Hour).Unix())
sweep(t, s, 6*time.Hour)
s.exec(t, "UPDATE alerts SET archived_at = unixepoch() WHERE fingerprint = 'refire1'")
s.exec(t, "UPDATE alerts SET archived_at = FLOOR(EXTRACT(EPOCH FROM now()))::bigint WHERE fingerprint = 'refire1'")
var alerts []map[string]any
decode(t, s.req(t, http.MethodGet, "/api/alerts", nil), &alerts)
@@ -714,7 +707,7 @@ func TestExpiry_EndsAtIsUpperBound(t *testing.T) {
// No watermark: expires on the received_at heartbeat, so the sweeper has
// nothing to go on but its own clock.
postAlert(t, s, "ub-none", "firing", time.Now().Add(-24*time.Hour).Format(time.RFC3339), zeroTime)
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'ub-none'",
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'ub-none'",
time.Now().Add(-10*time.Hour).Unix())
// Stale watermark: expires on the ends_at branch, and that reported time
@@ -768,7 +761,7 @@ func TestWebhook_ResendBumpsReceivedAt(t *testing.T) {
// received_at has one-second granularity, so back-date it to make the bump
// observable instead of sleeping out a second.
aged := time.Now().Add(-2 * time.Hour).Unix()
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'beat1'", aged)
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'beat1'", aged)
// Identical re-send: same fingerprint, same startsAt, still firing.
postAlert(t, s, "beat1", "firing", start, zeroTime)
@@ -796,7 +789,7 @@ func TestWebhook_DiscardedRetryLeavesReceivedAtAlone(t *testing.T) {
postAlert(t, s, "beat2", "resolved", start, time.Now().Format(time.RFC3339))
aged := time.Now().Add(-2 * time.Hour).Unix()
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'beat2'", aged)
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'beat2'", aged)
postAlert(t, s, "beat2", "firing", start, zeroTime) // stale retry, discarded
internal/api/archiver.go
+17 -20
View File
@@ -4,7 +4,7 @@ import (
"context"
"database/sql"
"log"
"strings"
"time"
)
@@ -91,17 +91,18 @@ func expireStale(ctx context.Context, db *sql.DB, staleAfter time.Duration, skip
return
}
args := make([]any, 0, len(ids)+1)
args = append(args, resolutionExpiry)
for _, id := range ids {
args = append(args, id)
args := &sqlArgs{}
source := args.add(resolutionExpiry)
idList := make([]any, len(ids))
for i, id := range ids {
idList[i] = id
}
if _, err := db.ExecContext(ctx, `
UPDATE alerts
SET status = 'resolved',
resolution_source = ?,
ends_at = COALESCE(ends_at, unixepoch())
WHERE id IN (`+placeholders(len(ids))+`)`, args...); err != nil {
resolution_source = `+source+`,
ends_at = COALESCE(ends_at, `+nowEpoch+`)
WHERE id IN (`+args.addList(idList)+`)`, args.all()...); err != nil {
log.Printf("sweeper: expire stale: %v", err)
return
}
@@ -124,14 +125,15 @@ func expireStale(ctx context.Context, db *sql.DB, staleAfter time.Duration, skip
}
// staleAlertIDs reads the ids in one go and closes the cursor before the caller
// writes: the pool is limited to a single connection, so an open read would
// block the update behind it.
// writes. Under SQLite's single connection an open read would have blocked the
// update outright; with a pool it is no longer a deadlock, but reading the set
// first still keeps the write off a cursor the same transaction is walking.
func staleAlertIDs(ctx context.Context, db *sql.DB, now time.Time, staleAfter time.Duration) ([]int64, error) {
rows, err := db.QueryContext(ctx, `
SELECT id FROM alerts
WHERE status = 'firing'
AND archived_at IS NULL
AND ((ends_at IS NOT NULL AND ends_at < ?) OR received_at < ?)`,
AND ((ends_at IS NOT NULL AND ends_at < $1) OR received_at < $2)`,
now.Add(-expiryGrace).Unix(), now.Add(-staleAfter).Unix())
if err != nil {
return nil, err
@@ -206,10 +208,10 @@ func settledIncidentIDs(ctx context.Context, db *sql.DB) ([]int64, error) {
func archiveResolved(ctx context.Context, db *sql.DB, archiveAfter time.Duration) {
cutoff := time.Now().Add(-archiveAfter).Unix()
res, err := db.ExecContext(ctx,
`UPDATE alerts SET archived_at = unixepoch()
`UPDATE alerts SET archived_at = `+nowEpoch+`
WHERE status = 'resolved'
AND archived_at IS NULL
AND COALESCE(ends_at, received_at) < ?`, cutoff)
AND COALESCE(ends_at, received_at) < $1`, cutoff)
if err != nil {
log.Printf("archiver: %v", err)
return
@@ -223,10 +225,10 @@ func archiveResolved(ctx context.Context, db *sql.DB, archiveAfter time.Duration
func archiveResolvedIncidents(ctx context.Context, db *sql.DB, archiveAfter time.Duration) {
cutoff := time.Now().Add(-archiveAfter).Unix()
res, err := db.ExecContext(ctx,
`UPDATE incidents SET archived_at = unixepoch()
`UPDATE incidents SET archived_at = `+nowEpoch+`
WHERE resolved_at IS NOT NULL
AND archived_at IS NULL
AND resolved_at < ?`, cutoff)
AND resolved_at < $1`, cutoff)
if err != nil {
log.Printf("archiver: incidents: %v", err)
return
@@ -235,8 +237,3 @@ func archiveResolvedIncidents(ctx context.Context, db *sql.DB, archiveAfter time
log.Printf("archiver: archived %d resolved incident(s)", n)
}
}
// placeholders builds "?, ?, …" for an IN clause of n values.
func placeholders(n int) string {
return strings.TrimSuffix(strings.Repeat("?, ", n), ", ")
}
internal/api/auth.go
+8 -8
View File
@@ -163,7 +163,7 @@ func handleLogin(db *sql.DB, limiter *loginLimiter, publicURL string) http.Handl
var userID int64
var hash sql.NullString
err := db.QueryRowContext(r.Context(),
"SELECT id, password_hash FROM users WHERE username = ?", username,
"SELECT id, password_hash FROM users WHERE username = $1", username,
).Scan(&userID, &hash)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -190,7 +190,7 @@ func handleLogin(db *sql.DB, limiter *loginLimiter, publicURL string) http.Handl
now := time.Now()
if _, err := db.ExecContext(r.Context(), `
INSERT INTO sessions (token_hash, user_id, created_at, last_seen_at, expires_at, user_agent)
VALUES (?, ?, ?, ?, ?, ?)`,
VALUES ($1, $2, $3, $4, $5, $6)`,
tokenHash, userID, now.Unix(), now.Unix(), now.Add(sessionTTL).Unix(), r.UserAgent()); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -225,7 +225,7 @@ func handleLogout(db *sql.DB, publicURL string) http.HandlerFunc {
return
}
if c, err := r.Cookie(sessionCookie); err == nil && c.Value != "" {
db.ExecContext(r.Context(), "DELETE FROM sessions WHERE token_hash = ?", hashToken(c.Value))
db.ExecContext(r.Context(), "DELETE FROM sessions WHERE token_hash = $1", hashToken(c.Value))
}
http.SetCookie(w, &http.Cookie{
Name: sessionCookie,
@@ -257,7 +257,7 @@ func handleMe(db *sql.DB) http.HandlerFunc {
}
var hash sql.NullString
db.QueryRowContext(r.Context(),
"SELECT password_hash FROM users WHERE id = ?", caller.ID).Scan(&hash)
"SELECT password_hash FROM users WHERE id = $1", caller.ID).Scan(&hash)
respond(w, http.StatusOK, meResponse{User: user, HasPassword: hash.Valid})
}
}
@@ -293,7 +293,7 @@ func handleSetPassword(db *sql.DB) http.HandlerFunc {
var existing sql.NullString
err = db.QueryRowContext(r.Context(),
"SELECT password_hash FROM users WHERE id = ?", id).Scan(&existing)
"SELECT password_hash FROM users WHERE id = $1", id).Scan(&existing)
if errors.Is(err, sql.ErrNoRows) {
respond(w, http.StatusNotFound, errResp("user not found"))
return
@@ -324,13 +324,13 @@ func handleSetPassword(db *sql.DB) http.HandlerFunc {
defer tx.Rollback()
if _, err := tx.ExecContext(r.Context(),
"UPDATE users SET password_hash = ? WHERE id = ?", hash, id); err != nil {
"UPDATE users SET password_hash = $1 WHERE id = $2", hash, id); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
keep, _ := sessionFromContext(r.Context()) // zero when changed with an API key
if _, err := tx.ExecContext(r.Context(),
"DELETE FROM sessions WHERE user_id = ? AND id != ?", id, keep); err != nil {
"DELETE FROM sessions WHERE user_id = $1 AND id != $2", id, keep); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
@@ -345,7 +345,7 @@ func handleSetPassword(db *sql.DB) http.HandlerFunc {
// purgeSessions deletes sessions that have expired, from the sweeper.
func purgeSessions(ctx context.Context, db *sql.DB) {
res, err := db.ExecContext(ctx,
"DELETE FROM sessions WHERE expires_at < ?", time.Now().Unix())
"DELETE FROM sessions WHERE expires_at < $1", time.Now().Unix())
if err != nil {
log.Printf("sweeper: purge sessions: %v", err)
return
internal/api/auth_test.go
+2 -9
View File
@@ -11,7 +11,6 @@ import (
"testing"
"git.ryuvia.com/niklas/terdut-server/internal/api"
"git.ryuvia.com/niklas/terdut-server/internal/db"
)
const adminPassword = "correct horse battery"
@@ -301,15 +300,9 @@ func TestSetPassword_EndsOtherSessionsButNotThisOne(t *testing.T) {
}
func TestBootstrap_WithPassword(t *testing.T) {
database, err := db.Open(":memory:")
if err != nil {
t.Fatal(err)
}
if err := db.Migrate(database); err != nil {
t.Fatal(err)
}
database := newTestDB(t)
srv := httptest.NewServer(api.NewRouter(database, api.NotifyConfig{}, api.DeadmanConfig{}))
t.Cleanup(func() { srv.Close(); database.Close() })
t.Cleanup(srv.Close)
body := `{"username":"admin","email":"a@test.com","password":"` + adminPassword + `"}`
resp, err := http.Post(srv.URL+"/api/bootstrap", "application/json", strings.NewReader(body))
internal/api/deadman.go
+16 -14
View File
@@ -225,19 +225,21 @@ func sweepDeadman(ctx context.Context, db *sql.DB, cfg DeadmanConfig, notify Not
// deadmanAlerts loads every alert row that a matcher claims. The candidate query
// is narrowed by alertname so it rides alerts_name_idx; the rest of the matching
// happens in Go, which keeps one implementation of the rules. The rows are read
// in full before the caller writes, because the pool holds a single connection.
// in full before the caller writes, so the writes do not run against an open
// cursor over the same table.
func deadmanAlerts(ctx context.Context, db *sql.DB, cfg DeadmanConfig) ([]deadmanAlert, error) {
names := cfg.names()
args := make([]any, 0, len(names))
for _, n := range names {
args = append(args, n)
args := &sqlArgs{}
nameList := make([]any, len(names))
for i, n := range names {
nameList[i] = n
}
rows, err := db.QueryContext(ctx, `
SELECT id, fingerprint, labels, status, received_at
FROM alerts
WHERE name IN (`+placeholders(len(names))+`)
AND archived_at IS NULL`, args...)
WHERE name IN (`+args.addList(nameList)+`)
AND archived_at IS NULL`, args.all()...)
if err != nil {
return nil, err
}
@@ -277,8 +279,8 @@ func deadmanDied(ctx context.Context, db *sql.DB, cfg DeadmanConfig, notify Noti
var lastTriggered, open int64
if err := db.QueryRowContext(ctx, `
SELECT COALESCE(MAX(triggered_at), 0),
COALESCE(SUM(resolved_at IS NULL), 0)
FROM incidents WHERE group_key = ?`,
COUNT(*) FILTER (WHERE resolved_at IS NULL)
FROM incidents WHERE group_key = $1`,
sw.groupKey()).Scan(&lastTriggered, &open); err != nil {
return err
}
@@ -299,9 +301,9 @@ func deadmanDied(ctx context.Context, db *sql.DB, cfg DeadmanConfig, notify Noti
if _, err := tx.ExecContext(ctx, `
UPDATE alerts
SET status = 'resolved',
resolution_source = ?,
ends_at = COALESCE(ends_at, unixepoch())
WHERE id = ? AND status = 'firing'`, resolutionDeadman, sw.id); err != nil {
resolution_source = $1,
ends_at = COALESCE(ends_at, `+nowEpoch+`)
WHERE id = $2 AND status = 'firing'`, resolutionDeadman, sw.id); err != nil {
return err
}
}
@@ -341,7 +343,7 @@ func deadmanRecovered(ctx context.Context, db *sql.DB, sw deadmanAlert) error {
var incidentID int64
switch err := db.QueryRowContext(ctx, `
SELECT id FROM incidents
WHERE group_key = ? AND resolved_at IS NULL`, sw.groupKey()).Scan(&incidentID); {
WHERE group_key = $1 AND resolved_at IS NULL`, sw.groupKey()).Scan(&incidentID); {
case err == sql.ErrNoRows:
return nil
case err != nil:
@@ -356,8 +358,8 @@ func deadmanRecovered(ctx context.Context, db *sql.DB, sw deadmanAlert) error {
if _, err := tx.ExecContext(ctx, `
UPDATE incidents
SET status = 'resolved', resolved_at = ?, resolution_source = ?
WHERE id = ? AND resolved_at IS NULL`,
SET status = 'resolved', resolved_at = $1, resolution_source = $2
WHERE id = $3 AND resolved_at IS NULL`,
time.Now().Unix(), incidentResolutionRecovered, incidentID); err != nil {
return err
}
internal/api/deadman_test.go
+7 -4
View File
@@ -51,7 +51,7 @@ func heartbeat(t *testing.T, s *ts, fingerprint string, labels map[string]string
// sweeper reads. There is no fake clock in this package.
func silence(t *testing.T, s *ts, fingerprint string, ago time.Duration) {
t.Helper()
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = ?",
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = $2",
time.Now().Add(-ago).Unix(), fingerprint)
}
@@ -61,9 +61,12 @@ func silence(t *testing.T, s *ts, fingerprint string, ago time.Duration) {
func ageIncidents(t *testing.T, s *ts, ago time.Duration) {
t.Helper()
past := time.Now().Add(-ago).Unix()
// $2 is cast explicitly: with NULL in the other branch Postgres has nothing
// to infer the parameter's type from and defaults it to text, which the
// bigint column then refuses.
s.exec(t, `UPDATE incidents
SET triggered_at = ?,
resolved_at = CASE WHEN resolved_at IS NULL THEN NULL ELSE ? END`,
SET triggered_at = $1,
resolved_at = CASE WHEN resolved_at IS NULL THEN NULL ELSE $2::bigint END`,
past, past)
}
@@ -72,7 +75,7 @@ func incidentByGroup(t *testing.T, s *ts, groupKey string) (id int64, status, se
t.Helper()
err := s.db.QueryRow(`
SELECT id, status, COALESCE(severity, ''), resolution_source
FROM incidents WHERE group_key = ? ORDER BY id DESC LIMIT 1`,
FROM incidents WHERE group_key = $1 ORDER BY id DESC LIMIT 1`,
groupKey).Scan(&id, &status, &severity, &source)
if err != nil {
t.Fatalf("read incident for group %s: %v", groupKey, err)
internal/api/helpers.go
+58
View File
@@ -2,9 +2,67 @@ package api
import (
"encoding/json"
"errors"
"net/http"
"strconv"
"strings"
"github.com/jackc/pgerrcode"
"github.com/jackc/pgx/v5/pgconn"
)
// sqlArgs accumulates query arguments and hands back the placeholder for each.
//
// Postgres numbers its placeholders, so a dynamically assembled WHERE clause has
// to keep its $1, $2, … in step with the order of the values — which SQLite's
// positional `?` did for free. Handing out the placeholder and storing the value
// in one call is what keeps them in step: a filter can be added, removed or
// reordered without renumbering anything by hand.
type sqlArgs struct{ vals []any }
// add stores v and returns the placeholder that refers to it.
func (a *sqlArgs) add(v any) string {
a.vals = append(a.vals, v)
return "$" + strconv.Itoa(len(a.vals))
}
// addList stores every value and returns their placeholders as "$1, $2, …",
// ready to drop into an IN (…) clause. Returns an empty string for no values,
// which no caller should reach: `IN ()` is a syntax error in Postgres as it was
// in SQLite, so callers check for an empty set before building the query.
func (a *sqlArgs) addList(vs []any) string {
parts := make([]string, len(vs))
for i, v := range vs {
parts[i] = a.add(v)
}
return strings.Join(parts, ", ")
}
// all returns the accumulated values, to be passed straight to Query or Exec.
func (a *sqlArgs) all() []any { return a.vals }
// nowEpoch is the SQL expression for "now, as unix seconds", matching how every
// timestamp in this schema is stored. SQLite spelled it unixepoch().
//
// FLOOR, not a bare cast: EXTRACT returns fractional seconds and casting to
// bigint rounds half up, so a row written at .6 of a second would claim a
// timestamp one second in the future — off by one against the time.Now().Unix()
// the Go side stamps, which is what the expiry tests measure.
const nowEpoch = "FLOOR(EXTRACT(EPOCH FROM now()))::bigint"
// isUniqueViolation reports whether err is a broken unique constraint, which
// callers turn into 409 Conflict rather than 500.
//
// Postgres reports it as SQLSTATE 23505 on a typed error; the SQLite driver this
// replaced only put "UNIQUE constraint failed" in the message, which is why the
// check used to be a substring match. Matching the code means a renamed
// constraint or a translated message cannot quietly turn a conflict back into a
// 500.
func isUniqueViolation(err error) bool {
var pgErr *pgconn.PgError
return errors.As(err, &pgErr) && pgErr.Code == pgerrcode.UniqueViolation
}
func respond(w http.ResponseWriter, status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
internal/api/incident_store.go
+14 -14
View File
@@ -95,7 +95,7 @@ func unixPtr(sec *int64) *time.Time {
}
func fetchIncident(ctx context.Context, q querier, id int64) (models.Incident, error) {
return scanIncident(q.QueryRowContext(ctx, incidentSelectFrom+" WHERE i.id = ?", id))
return scanIncident(q.QueryRowContext(ctx, incidentSelectFrom+" WHERE i.id = $1", id))
}
// logEvent appends one entry to an incident's timeline. A nil userID means the
@@ -103,7 +103,7 @@ func fetchIncident(ctx context.Context, q querier, id int64) (models.Incident, e
func logEvent(ctx context.Context, q querier, incidentID int64, evType string, userID, alertID *int64, detail *string) error {
_, err := q.ExecContext(ctx, `
INSERT INTO incident_events (incident_id, type, user_id, alert_id, detail, created_at)
VALUES (?, ?, ?, ?, ?, ?)`,
VALUES ($1, $2, $3, $4, $5, $6)`,
incidentID, evType, userID, alertID, detail, time.Now().Unix())
return err
}
@@ -118,7 +118,7 @@ func todayUTC() string {
func currentOnCall(ctx context.Context, q querier) (*int64, error) {
var userID int64
err := q.QueryRowContext(ctx,
"SELECT user_id FROM schedule_entries WHERE date = ?", todayUTC()).Scan(&userID)
"SELECT user_id FROM schedule_entries WHERE date = $1", todayUTC()).Scan(&userID)
if err == sql.ErrNoRows {
return nil, nil
}
@@ -154,10 +154,10 @@ func severityRank(s string) int {
// in the queue while the work is still open.
func refreshSeverity(ctx context.Context, q querier, incidentID int64) error {
rows, err := q.QueryContext(ctx, `
SELECT json_extract(a.labels, '$.'||?)
SELECT a.labels ->> $1
FROM incident_alerts ia
JOIN alerts a ON a.id = ia.alert_id
WHERE ia.incident_id = ?`, severityLabel, incidentID)
WHERE ia.incident_id = $2`, severityLabel, incidentID)
if err != nil {
return err
}
@@ -185,9 +185,9 @@ func refreshSeverity(ctx context.Context, q querier, incidentID int64) error {
// The comparison lives in SQL so an unrelated concurrent update cannot be
// clobbered by a stale read.
_, err = q.ExecContext(ctx, `
UPDATE incidents SET severity = ?
WHERE id = ?
AND (severity IS NULL OR `+severityRankSQL("severity")+` < ?)`,
UPDATE incidents SET severity = $1
WHERE id = $2
AND (severity IS NULL OR `+severityRankSQL("severity")+` < $3)`,
best, incidentID, severityRank(best))
return err
}
@@ -210,9 +210,9 @@ func resolveIfSettled(ctx context.Context, q querier, incidentID int64) (bool, e
res, err := q.ExecContext(ctx, `
UPDATE incidents
SET status = 'resolved',
resolved_at = ?,
resolution_source = ?
WHERE id = ?
resolved_at = $1,
resolution_source = $2
WHERE id = $3
AND resolved_at IS NULL
-- An incident with no members yet is mid-creation, not settled.
AND EXISTS (SELECT 1 FROM incident_alerts ia WHERE ia.incident_id = incidents.id)
@@ -245,8 +245,8 @@ func resolveIfSettled(ctx context.Context, q querier, incidentID int64) (bool, e
func acknowledgeIncident(ctx context.Context, q querier, incidentID, userID int64) (bool, error) {
res, err := q.ExecContext(ctx, `
UPDATE incidents
SET status = 'acknowledged', acknowledged_by = ?, acknowledged_at = ?
WHERE id = ? AND resolved_at IS NULL`,
SET status = 'acknowledged', acknowledged_by = $1, acknowledged_at = $2
WHERE id = $3 AND resolved_at IS NULL`,
userID, time.Now().Unix(), incidentID)
if err != nil {
return false, err
@@ -266,7 +266,7 @@ func openIncidentForAlert(ctx context.Context, q querier, alertID int64) (int64,
SELECT i.id
FROM incident_alerts ia
JOIN incidents i ON i.id = ia.incident_id
WHERE ia.alert_id = ? AND i.resolved_at IS NULL`, alertID).Scan(&id)
WHERE ia.alert_id = $1 AND i.resolved_at IS NULL`, alertID).Scan(&id)
if err == sql.ErrNoRows {
return 0, nil
}
internal/api/incidents.go
+28 -35
View File
@@ -17,13 +17,12 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
q := r.URL.Query()
where := []string{}
args := []any{}
args := &sqlArgs{}
// Without an explicit status the queue shows open work, which is what an
// on-call person opens the tool to see.
if status := q.Get("status"); status != "" {
where = append(where, "i.status = ?")
args = append(args, status)
where = append(where, "i.status = "+args.add(status))
} else {
where = append(where, "i.resolved_at IS NULL")
}
@@ -36,33 +35,27 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
// A snooze expires by simply falling into the past; nothing sweeps it.
if q.Get("snoozed") == "true" {
where = append(where, "i.snoozed_until > ?")
args = append(args, time.Now().Unix())
where = append(where, "i.snoozed_until > "+args.add(time.Now().Unix()))
} else {
where = append(where, "(i.snoozed_until IS NULL OR i.snoozed_until <= ?)")
args = append(args, time.Now().Unix())
where = append(where, "(i.snoozed_until IS NULL OR i.snoozed_until <= "+args.add(time.Now().Unix())+")")
}
if severity := q.Get("severity"); severity != "" {
where = append(where, "i.severity = ?")
args = append(args, severity)
where = append(where, "i.severity = "+args.add(severity))
}
if assignee := q.Get("assigned_to"); assignee != "" {
if n, err := strconv.ParseInt(assignee, 10, 64); err == nil {
where = append(where, "i.assigned_to = ?")
args = append(args, n)
where = append(where, "i.assigned_to = "+args.add(n))
}
}
if from := q.Get("from"); from != "" {
if t, err := time.Parse("2006-01-02", from); err == nil {
where = append(where, "i.triggered_at >= ?")
args = append(args, t.UTC().Unix())
where = append(where, "i.triggered_at >= "+args.add(t.UTC().Unix()))
}
}
if to := q.Get("to"); to != "" {
if t, err := time.Parse("2006-01-02", to); err == nil {
where = append(where, "i.triggered_at < ?")
args = append(args, t.UTC().AddDate(0, 0, 1).Unix())
where = append(where, "i.triggered_at < "+args.add(t.UTC().AddDate(0, 0, 1).Unix()))
}
}
@@ -77,12 +70,11 @@ func handleListIncidents(db *sql.DB) http.HandlerFunc {
if q.Get("sort") == "severity" {
order = severityRankSQL("i.severity") + " DESC, i.triggered_at DESC"
}
args = append(args, limit)
rows, err := db.QueryContext(r.Context(),
fmt.Sprintf("%s WHERE %s ORDER BY %s LIMIT ?",
incidentSelectFrom, strings.Join(where, " AND "), order),
args...)
fmt.Sprintf("%s WHERE %s ORDER BY %s LIMIT %s",
incidentSelectFrom, strings.Join(where, " AND "), order, args.add(limit)),
args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -158,7 +150,7 @@ func handleIncidentTimeline(db *sql.DB) http.HandlerFunc {
e.alert_id, e.detail, e.created_at
FROM incident_events e
LEFT JOIN users u ON u.id = e.user_id
WHERE e.incident_id = ?
WHERE e.incident_id = $1
ORDER BY e.created_at ASC, e.id ASC`, id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -214,7 +206,7 @@ func handleIncidentUnacknowledge(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
`UPDATE incidents SET status = 'triggered', acknowledged_by = NULL, acknowledged_at = NULL
WHERE id = ? AND resolved_at IS NULL`, id) {
WHERE id = $1 AND resolved_at IS NULL`, id) {
return
}
if err := logEvent(r.Context(), db, id, evUnacknowledged, &user.ID, nil, nil); err != nil {
@@ -237,8 +229,8 @@ func handleIncidentResolve(db *sql.DB) http.HandlerFunc {
}
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
`UPDATE incidents SET status = 'resolved', resolved_at = ?, resolution_source = ?
WHERE id = ? AND resolved_at IS NULL`,
`UPDATE incidents SET status = 'resolved', resolved_at = $1, resolution_source = $2
WHERE id = $3 AND resolved_at IS NULL`,
time.Now().Unix(), incidentResolutionManual, id) {
return
}
@@ -269,13 +261,13 @@ func handleIncidentAssign(db *sql.DB) http.HandlerFunc {
}
var exists int
if err := db.QueryRowContext(r.Context(),
"SELECT 1 FROM users WHERE id = ?", req.UserID).Scan(&exists); err != nil {
"SELECT 1 FROM users WHERE id = $1", req.UserID).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("user not found"))
return
}
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET assigned_to = ? WHERE id = ? AND resolved_at IS NULL",
"UPDATE incidents SET assigned_to = $1 WHERE id = $2 AND resolved_at IS NULL",
req.UserID, id) {
return
}
@@ -333,7 +325,7 @@ func handleIncidentSnooze(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET snoozed_until = ? WHERE id = ? AND resolved_at IS NULL",
"UPDATE incidents SET snoozed_until = $1 WHERE id = $2 AND resolved_at IS NULL",
until.Unix(), id) {
return
}
@@ -354,7 +346,7 @@ func handleIncidentUnsnooze(db *sql.DB) http.HandlerFunc {
}
user, _ := userFromContext(r.Context())
if !updateOpenIncident(w, r, db, id,
"UPDATE incidents SET snoozed_until = NULL WHERE id = ? AND resolved_at IS NULL", id) {
"UPDATE incidents SET snoozed_until = NULL WHERE id = $1 AND resolved_at IS NULL", id) {
return
}
if err := logEvent(r.Context(), db, id, evUnsnoozed, &user.ID, nil, nil); err != nil {
@@ -372,7 +364,7 @@ func handleIncidentArchive(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"UPDATE incidents SET archived_at = unixepoch() WHERE id = ?", id)
"UPDATE incidents SET archived_at = "+nowEpoch+" WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -392,7 +384,7 @@ func handleIncidentUnarchive(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"UPDATE incidents SET archived_at = NULL WHERE id = ?", id)
"UPDATE incidents SET archived_at = NULL WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -430,14 +422,15 @@ func handleCreateNote(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
now := time.Now()
res, err := db.ExecContext(r.Context(), `
var eventID int64
err := db.QueryRowContext(r.Context(), `
INSERT INTO incident_events (incident_id, type, user_id, detail, created_at)
VALUES (?, ?, ?, ?, ?)`, id, evNote, user.ID, req.Content, now.Unix())
VALUES ($1, $2, $3, $4, $5)
RETURNING id`, id, evNote, user.ID, req.Content, now.Unix()).Scan(&eventID)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
eventID, _ := res.LastInsertId()
respond(w, http.StatusCreated, models.IncidentEvent{
ID: eventID,
@@ -468,7 +461,7 @@ func handleDeleteNote(db *sql.DB) http.HandlerFunc {
user, _ := userFromContext(r.Context())
res, err := db.ExecContext(r.Context(), `
DELETE FROM incident_events
WHERE id = ? AND incident_id = ? AND type = ? AND user_id = ?`,
WHERE id = $1 AND incident_id = $2 AND type = $3 AND user_id = $4`,
eventID, id, evNote, user.ID)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -498,7 +491,7 @@ func incidentIDParam(w http.ResponseWriter, r *http.Request) (int64, bool) {
func incidentExists(w http.ResponseWriter, r *http.Request, db *sql.DB, id int64) bool {
var exists int
if err := db.QueryRowContext(r.Context(),
"SELECT 1 FROM incidents WHERE id = ?", id).Scan(&exists); err != nil {
"SELECT 1 FROM incidents WHERE id = $1", id).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("incident not found"))
return false
}
@@ -538,7 +531,7 @@ func respondIncident(w http.ResponseWriter, r *http.Request, db *sql.DB, id int6
func incidentAlerts(r *http.Request, db *sql.DB, id int64) ([]models.Alert, error) {
rows, err := db.QueryContext(r.Context(), alertSelectFrom+`
JOIN incident_alerts m ON m.alert_id = a.id
WHERE m.incident_id = ?
WHERE m.incident_id = $1
ORDER BY a.received_at DESC`, id)
if err != nil {
return nil, err
internal/api/incidents_test.go
+5 -103
View File
@@ -4,14 +4,10 @@ import (
"context"
"fmt"
"net/http"
"os"
"path/filepath"
"sort"
"testing"
"time"
"git.ryuvia.com/niklas/terdut-server/internal/api"
"git.ryuvia.com/niklas/terdut-server/internal/db"
)
// amAlert builds one alert of a webhook payload.
@@ -288,7 +284,7 @@ func TestExpiry_CascadesToIncidentResolution(t *testing.T) {
s := newTS(t)
postAlert(t, s, "fp-exp", "firing", time.Now().Add(-24*time.Hour).Format(time.RFC3339), zeroTime)
s.exec(t, "UPDATE alerts SET received_at = ? WHERE fingerprint = 'fp-exp'",
s.exec(t, "UPDATE alerts SET received_at = $1 WHERE fingerprint = 'fp-exp'",
time.Now().Add(-10*time.Hour).Unix())
sweep(t, s, 6*time.Hour)
@@ -611,7 +607,7 @@ func TestSweeper_ArchivesResolvedIncidents(t *testing.T) {
})
s.req(t, http.MethodPost, "/api/incidents/1/resolve", nil).Body.Close()
s.exec(t, "UPDATE incidents SET resolved_at = ? WHERE id = 1",
s.exec(t, "UPDATE incidents SET resolved_at = $1 WHERE id = 1",
time.Now().Add(-30*24*time.Hour).Unix())
api.Sweep(context.Background(), s.db, 7*24*time.Hour, 6*time.Hour, s.deadman, s.notify)
@@ -654,8 +650,9 @@ func TestStats_Incidents(t *testing.T) {
}
// An empty window is a report of zero, not a failure. SUM over no rows is NULL
// in SQLite, which used to come back as a 500 the moment every incident was
// archived — the state a quiet installation settles into.
// in Postgres as it was in SQLite, and that used to come back as a 500 the
// moment every incident was archived — the state a quiet installation settles
// into.
func TestStats_IncidentsEmptyWindowIsZeroNotAnError(t *testing.T) {
s := newTS(t)
@@ -722,101 +719,6 @@ func TestStats_IncidentsNullMTTAWhenNothingAcknowledged(t *testing.T) {
}
}
// ---------------------------------------------------------------------------
// Migration backfill
// ---------------------------------------------------------------------------
// An upgrade must not drop the acknowledgements and comments people already
// have, so 008 is replayed here over a database left at 007.
func TestMigration_BackfillCarriesAckAndComments(t *testing.T) {
database, err := db.Open(":memory:")
if err != nil {
t.Fatalf("open db: %v", err)
}
t.Cleanup(func() { database.Close() })
files, err := filepath.Glob("../db/migrations/*.sql")
if err != nil || len(files) == 0 {
t.Fatalf("find migrations: %v", err)
}
sort.Strings(files)
var incidentsMigration string
for _, f := range files {
if filepath.Base(f) >= "008" {
incidentsMigration = f
break
}
data, err := os.ReadFile(f)
if err != nil {
t.Fatalf("read %s: %v", f, err)
}
if _, err := database.Exec(string(data)); err != nil {
t.Fatalf("apply %s: %v", f, err)
}
}
if incidentsMigration == "" {
t.Fatal("008 migration not found")
}
// A database as it would look on the old schema: an acknowledged firing
// alert with a comment on it.
if _, err := database.Exec(`
INSERT INTO users (id, username, email) VALUES (1, 'admin', 'admin@test.com');
INSERT INTO alerts (id, fingerprint, name, status, labels, annotations,
starts_at, received_at, acknowledged_by, acknowledged_at)
VALUES (1, 'legacy-fp', 'LegacyAlert', 'firing',
'{"severity":"warning"}', '{}', 1000, 1000, 1, 1500);
INSERT INTO alert_comments (alert_id, user_id, content, created_at)
VALUES (1, 1, 'legacy comment', 1600);`); err != nil {
t.Fatalf("seed pre-008 data: %v", err)
}
data, err := os.ReadFile(incidentsMigration)
if err != nil {
t.Fatalf("read 008: %v", err)
}
if _, err := database.Exec(string(data)); err != nil {
t.Fatalf("apply 008: %v", err)
}
var status, groupKey string
var ackBy int64
var severity string
if err := database.QueryRow(
"SELECT status, group_key, acknowledged_by, severity FROM incidents WHERE id = 1",
).Scan(&status, &groupKey, &ackBy, &severity); err != nil {
t.Fatalf("read backfilled incident: %v", err)
}
if status != "acknowledged" {
t.Errorf("expected the ack to carry over as status, got %q", status)
}
if groupKey != "backfill:legacy-fp" {
t.Errorf("unexpected group_key %q", groupKey)
}
if ackBy != 1 {
t.Errorf("expected acknowledged_by 1, got %d", ackBy)
}
if severity != "warning" {
t.Errorf("expected severity carried from labels, got %q", severity)
}
var notes int
if err := database.QueryRow(
"SELECT COUNT(*) FROM incident_events WHERE type = 'note' AND detail = 'legacy comment'",
).Scan(&notes); err != nil {
t.Fatalf("count notes: %v", err)
}
if notes != 1 {
t.Errorf("expected the comment to become a note, got %d", notes)
}
// And the columns that caused the ack-survives-a-re-fire bug are gone.
if _, err := database.Exec("SELECT acknowledged_by FROM alerts"); err == nil {
t.Error("expected alerts.acknowledged_by to be dropped")
}
}
func contains(haystack []string, needle string) bool {
for _, s := range haystack {
if s == needle {
internal/api/middleware.go
+5 -5
View File
@@ -70,7 +70,7 @@ func AuthMiddleware(db *sql.DB) func(http.Handler) http.Handler {
func apiKeyUser(ctx context.Context, db *sql.DB, token string) (int64, bool) {
var keyID, userID int64
err := db.QueryRowContext(ctx,
"SELECT id, user_id FROM api_keys WHERE key_hash = ?", hashToken(token),
"SELECT id, user_id FROM api_keys WHERE key_hash = $1", hashToken(token),
).Scan(&keyID, &userID)
if err != nil {
return 0, false
@@ -78,7 +78,7 @@ func apiKeyUser(ctx context.Context, db *sql.DB, token string) (int64, bool) {
// best-effort; don't fail the request if this update fails
db.ExecContext(ctx,
"UPDATE api_keys SET last_used_at = ? WHERE id = ?",
"UPDATE api_keys SET last_used_at = $1 WHERE id = $2",
time.Now().Unix(), keyID)
return userID, true
}
@@ -91,7 +91,7 @@ func sessionUser(ctx context.Context, db *sql.DB, token string) (sessionID, user
var lastSeen int64
err := db.QueryRowContext(ctx, `
SELECT id, user_id, last_seen_at FROM sessions
WHERE token_hash = ? AND expires_at > ?`,
WHERE token_hash = $1 AND expires_at > $2`,
hashToken(token), now.Unix()).Scan(&sessionID, &userID, &lastSeen)
if err != nil {
return 0, 0, false
@@ -99,7 +99,7 @@ func sessionUser(ctx context.Context, db *sql.DB, token string) (sessionID, user
if now.Sub(time.Unix(lastSeen, 0)) > sessionTouchEvery {
db.ExecContext(ctx,
"UPDATE sessions SET last_seen_at = ?, expires_at = ? WHERE id = ?",
"UPDATE sessions SET last_seen_at = $1, expires_at = $2 WHERE id = $3",
now.Unix(), now.Add(sessionTTL).Unix(), sessionID)
}
return sessionID, userID, true
@@ -111,7 +111,7 @@ func serveAs(w http.ResponseWriter, r *http.Request, next http.Handler, db *sql.
var u models.User
var createdUnix int64
if err := db.QueryRowContext(r.Context(),
"SELECT id, username, email, created_at FROM users WHERE id = ?", userID,
"SELECT id, username, email, created_at FROM users WHERE id = $1", userID,
).Scan(&u.ID, &u.Username, &u.Email, &createdUnix); err != nil {
respond(w, http.StatusUnauthorized, errResp("unauthorized"))
return
internal/api/notifier.go
+14 -13
View File
@@ -151,19 +151,20 @@ func enqueueReminders(ctx context.Context, db *sql.DB, cfg NotifyConfig) {
JOIN incidents i ON i.id = n.incident_id
WHERE n.id = (SELECT MAX(id) FROM notifications WHERE incident_id = n.incident_id)
AND n.sent_at IS NOT NULL
AND n.created_at <= ?
AND n.created_at <= $1
AND i.resolved_at IS NULL
AND i.archived_at IS NULL
AND i.status = 'triggered'
AND (i.snoozed_until IS NULL OR i.snoozed_until <= ?)`,
AND (i.snoozed_until IS NULL OR i.snoozed_until <= $2)`,
now.Add(-cfg.RepeatEvery).Unix(), now.Unix())
if err != nil {
log.Printf("notifier: find reminders: %v", err)
return
}
// Collected before inserting: the pool is limited to a single connection, so
// an open cursor would block the writes behind it.
// Collected before inserting, rather than written while walking the cursor:
// the inserts below are what this query selects on, and a cursor reading its
// own writes is a hazard whatever the pool size.
var pending []due
for rows.Next() {
var d due
@@ -217,7 +218,7 @@ func deliverPending(ctx context.Context, db *sql.DB, cfg NotifyConfig) {
continue
}
if _, err := db.ExecContext(ctx,
"UPDATE notifications SET sent_at = ?, attempts = attempts + 1, last_error = NULL WHERE id = ?",
"UPDATE notifications SET sent_at = $1, attempts = attempts + 1, last_error = NULL WHERE id = $2",
time.Now().Unix(), n.id); err != nil {
log.Printf("notifier: mark sent %d: %v", n.id, err)
}
@@ -240,10 +241,10 @@ func pendingNotifications(ctx context.Context, db *sql.DB) ([]outboxRow, error)
SELECT id, incident_id, user_id, topic, kind, attempts
FROM notifications
WHERE sent_at IS NULL
AND send_after <= ?
AND attempts < ?
AND send_after <= $1
AND attempts < $2
ORDER BY id
LIMIT ?`, time.Now().Unix(), notifyMaxAttempts, notifyBatch)
LIMIT $3`, time.Now().Unix(), notifyMaxAttempts, notifyBatch)
if err != nil {
return nil, err
}
@@ -269,7 +270,7 @@ func pendingNotifications(ctx context.Context, db *sql.DB) ([]outboxRow, error)
func markFailed(ctx context.Context, db *sql.DB, n outboxRow, cause error) {
next := time.Now().Add(retryDelay(n.attempts)).Unix()
if _, err := db.ExecContext(ctx,
"UPDATE notifications SET attempts = attempts + 1, send_after = ?, last_error = ? WHERE id = ?",
"UPDATE notifications SET attempts = attempts + 1, send_after = $1, last_error = $2 WHERE id = $3",
next, cause.Error(), n.id); err != nil {
log.Printf("notifier: mark failed %d: %v", n.id, err)
}
@@ -307,7 +308,7 @@ func deliver(ctx context.Context, db *sql.DB, cfg NotifyConfig, n outboxRow) err
SELECT COUNT(*)
FROM incident_alerts ia
JOIN alerts a ON a.id = ia.alert_id
WHERE ia.incident_id = ? AND a.status = 'firing'`, n.incidentID).Scan(&firing); err != nil {
WHERE ia.incident_id = $1 AND a.status = 'firing'`, n.incidentID).Scan(&firing); err != nil {
return fmt.Errorf("count firing: %w", err)
}
@@ -475,7 +476,7 @@ func enqueueNotification(ctx context.Context, q querier, incidentID int64, userI
now := time.Now().Unix()
_, err := q.ExecContext(ctx, `
INSERT INTO notifications (incident_id, user_id, topic, kind, created_at, send_after)
VALUES (?, ?, ?, ?, ?, ?)`, incidentID, userID, topic, kind, now, now)
VALUES ($1, $2, $3, $4, $5, $6)`, incidentID, userID, topic, kind, now, now)
return err
}
@@ -489,7 +490,7 @@ func notifyTarget(ctx context.Context, q querier, cfg NotifyConfig, onCall *int6
if onCall != nil {
var t *string
err := q.QueryRowContext(ctx,
"SELECT ntfy_topic FROM users WHERE id = ?", *onCall).Scan(&t)
"SELECT ntfy_topic FROM users WHERE id = $1", *onCall).Scan(&t)
if err == nil && t != nil && *t != "" {
return *t, onCall
}
@@ -522,7 +523,7 @@ func enqueueResolved(ctx context.Context, q querier, incidentID int64) error {
var userID *int64
err := q.QueryRowContext(ctx, `
SELECT topic, user_id FROM notifications
WHERE incident_id = ? ORDER BY id DESC LIMIT 1`, incidentID).Scan(&topic, &userID)
WHERE incident_id = $1 ORDER BY id DESC LIMIT 1`, incidentID).Scan(&topic, &userID)
if err == sql.ErrNoRows {
return nil
}
internal/api/notify_ack.go
+3 -3
View File
@@ -28,7 +28,7 @@ func issueAckToken(ctx context.Context, q querier, incidentID, userID int64) (st
now := time.Now()
if _, err := q.ExecContext(ctx, `
INSERT INTO incident_ack_tokens (token_hash, incident_id, user_id, created_at, expires_at)
VALUES (?, ?, ?, ?, ?)`,
VALUES ($1, $2, $3, $4, $5)`,
hash, incidentID, userID, now.Unix(), now.Add(ackTokenTTL).Unix()); err != nil {
return "", err
}
@@ -51,7 +51,7 @@ func handleNotifyAck(db *sql.DB) http.HandlerFunc {
var incidentID, userID int64
err := db.QueryRowContext(r.Context(), `
SELECT incident_id, user_id FROM incident_ack_tokens
WHERE token_hash = ? AND expires_at > ?`,
WHERE token_hash = $1 AND expires_at > $2`,
hash, time.Now().Unix()).Scan(&incidentID, &userID)
if err != nil {
// Unknown and expired get the same answer, so the endpoint cannot be
@@ -87,7 +87,7 @@ func handleNotifyAck(db *sql.DB) http.HandlerFunc {
// fires in practice.
func purgeAckTokens(ctx context.Context, db *sql.DB) {
res, err := db.ExecContext(ctx,
"DELETE FROM incident_ack_tokens WHERE expires_at < ?", time.Now().Unix())
"DELETE FROM incident_ack_tokens WHERE expires_at < $1", time.Now().Unix())
if err != nil {
log.Printf("sweeper: purge ack tokens: %v", err)
return
internal/api/notify_test.go
+7 -7
View File
@@ -125,7 +125,7 @@ func (s *ts) countNotifications(t *testing.T, kind string) int {
query := "SELECT COUNT(*) FROM notifications"
args := []any{}
if kind != "" {
query += " WHERE kind = ?"
query += " WHERE kind = $1"
args = append(args, kind)
}
if err := s.db.QueryRow(query, args...).Scan(&n); err != nil {
@@ -326,7 +326,7 @@ func TestNotify_ExhaustedRetriesAreRecordedOnce(t *testing.T) {
// One pass per attempt, each made due by clearing the backoff the last one set.
for i := 0; i < 10; i++ {
s.sweepNotify(t)
s.exec(t, "UPDATE notifications SET send_after = ? WHERE sent_at IS NULL",
s.exec(t, "UPDATE notifications SET send_after = $1 WHERE sent_at IS NULL",
time.Now().Add(-time.Second).Unix())
}
@@ -415,7 +415,7 @@ func TestNotify_AckRejectsExpiredToken(t *testing.T) {
// Age the token past its TTL. The token's inputs are wall-clock timestamps,
// so this is the same trick the sweeper tests use.
s.exec(t, "UPDATE incident_ack_tokens SET expires_at = ?", time.Now().Add(-time.Minute).Unix())
s.exec(t, "UPDATE incident_ack_tokens SET expires_at = $1", time.Now().Add(-time.Minute).Unix())
resp, err := http.Post(s.URL+path, "application/json", nil)
if err != nil {
@@ -436,7 +436,7 @@ func TestNotify_SweepPurgesExpiredAckTokens(t *testing.T) {
fireCritical(t, s)
s.sweepNotify(t)
s.exec(t, "UPDATE incident_ack_tokens SET expires_at = ?", time.Now().Add(-time.Minute).Unix())
s.exec(t, "UPDATE incident_ack_tokens SET expires_at = $1", time.Now().Add(-time.Minute).Unix())
api.Sweep(context.Background(), s.db, 168*time.Hour, 6*time.Hour, s.deadman, s.notify)
@@ -457,7 +457,7 @@ func TestNotify_SweepPurgesExpiredAckTokens(t *testing.T) {
// reminder as due.
func (s *ts) ageNotifications(t *testing.T, by time.Duration) {
t.Helper()
s.exec(t, "UPDATE notifications SET created_at = ? WHERE sent_at IS NOT NULL",
s.exec(t, "UPDATE notifications SET created_at = $1 WHERE sent_at IS NOT NULL",
time.Now().Add(-by).Unix())
}
@@ -693,7 +693,7 @@ func TestNotify_FailedDeliveryRetriesWithBackoff(t *testing.T) {
// Once due and once ntfy recovers, it goes out.
f.failWith(http.StatusOK)
s.exec(t, "UPDATE notifications SET send_after = ? WHERE id = 1", time.Now().Add(-time.Second).Unix())
s.exec(t, "UPDATE notifications SET send_after = $1 WHERE id = 1", time.Now().Add(-time.Second).Unix())
s.sweepNotify(t)
if err := s.db.QueryRow("SELECT sent_at FROM notifications WHERE id = 1").Scan(&sentAt); err != nil {
@@ -715,7 +715,7 @@ func TestNotify_UnsentNotificationBlocksReminders(t *testing.T) {
fireCritical(t, s)
s.sweepNotify(t)
s.exec(t, "UPDATE notifications SET created_at = ?", time.Now().Add(-time.Hour).Unix())
s.exec(t, "UPDATE notifications SET created_at = $1", time.Now().Add(-time.Hour).Unix())
s.sweepNotify(t)
if got := s.countNotifications(t, "reminder"); got != 0 {
internal/api/schedule.go
+10 -12
View File
@@ -45,7 +45,7 @@ func handleCreateSchedule(db *sql.DB) http.HandlerFunc {
// Verify the user exists.
var exists int
if err := db.QueryRowContext(r.Context(), "SELECT 1 FROM users WHERE id = ?", req.UserID).Scan(&exists); err != nil {
if err := db.QueryRowContext(r.Context(), "SELECT 1 FROM users WHERE id = $1", req.UserID).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("user not found"))
return
}
@@ -64,14 +64,14 @@ func handleCreateSchedule(db *sql.DB) http.HandlerFunc {
for _, d := range req.Dates {
if req.Replace {
if _, err := tx.ExecContext(r.Context(),
"DELETE FROM schedule_entries WHERE date = ?", d); err != nil {
"DELETE FROM schedule_entries WHERE date = $1", d); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
}
if _, err := tx.ExecContext(r.Context(),
"INSERT INTO schedule_entries (user_id, date) VALUES (?, ?)", req.UserID, d); err != nil {
if strings.Contains(err.Error(), "UNIQUE constraint failed") {
"INSERT INTO schedule_entries (user_id, date) VALUES ($1, $2)", req.UserID, d); err != nil {
if isUniqueViolation(err) {
respond(w, http.StatusConflict,
errResp("date already assigned: "+d+" (pass replace to take it)"))
return
@@ -139,7 +139,7 @@ func handleDeleteSchedule(db *sql.DB) http.HandlerFunc {
respond(w, http.StatusBadRequest, errResp("invalid schedule id"))
return
}
res, err := db.ExecContext(r.Context(), "DELETE FROM schedule_entries WHERE id = ?", id)
res, err := db.ExecContext(r.Context(), "DELETE FROM schedule_entries WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -162,7 +162,7 @@ func handleCurrentSchedule(db *sql.DB) http.HandlerFunc {
SELECT s.id, s.user_id, u.username, s.date, s.created_at
FROM schedule_entries s
JOIN users u ON u.id = s.user_id
WHERE s.date = ?`, today).Scan(&e.ID, &e.UserID, &e.Username, &e.Date, &ts)
WHERE s.date = $1`, today).Scan(&e.ID, &e.UserID, &e.Username, &e.Date, &ts)
if err == sql.ErrNoRows {
respond(w, http.StatusNotFound, errResp("no one is on call today"))
return
@@ -180,14 +180,12 @@ func handleCurrentSchedule(db *sql.DB) http.HandlerFunc {
// from and to are YYYY-MM-DD strings; an empty string means unbounded on that side.
func scheduleRange(ctx context.Context, db *sql.DB, from, to string) ([]models.ScheduleEntry, error) {
where := []string{}
args := []any{}
args := &sqlArgs{}
if from != "" {
where = append(where, "s.date >= ?")
args = append(args, from)
where = append(where, "s.date >= "+args.add(from))
}
if to != "" {
where = append(where, "s.date <= ?")
args = append(args, to)
where = append(where, "s.date <= "+args.add(to))
}
clause := "1=1"
@@ -200,7 +198,7 @@ func scheduleRange(ctx context.Context, db *sql.DB, from, to string) ([]models.S
FROM schedule_entries s
JOIN users u ON u.id = s.user_id
WHERE `+clause+`
ORDER BY s.date ASC`, args...)
ORDER BY s.date ASC`, args.all()...)
if err != nil {
return nil, err
}
internal/api/stats.go
+13 -15
View File
@@ -21,7 +21,7 @@ func handleStatsAlerts(db *sql.DB) http.HandlerFunc {
SELECT COUNT(*),
COALESCE(SUM(CASE WHEN status = 'firing' THEN 1 ELSE 0 END), 0),
COALESCE(SUM(CASE WHEN status = 'resolved' THEN 1 ELSE 0 END), 0)
FROM alerts WHERE %s`, where), args...,
FROM alerts WHERE %s`, where), args.all()...,
).Scan(&total, &firing, &resolved)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -47,15 +47,13 @@ func handleStatsTop(db *sql.DB) http.HandlerFunc {
limit = n
}
}
args = append(args, limit)
rows, err := db.QueryContext(r.Context(), fmt.Sprintf(`
SELECT name, COUNT(*) AS cnt
FROM alerts
WHERE %s
GROUP BY name
ORDER BY cnt DESC
LIMIT ?`, where), args...)
LIMIT %s`, where, args.add(limit)), args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -84,12 +82,12 @@ func handleStatsByHour(db *sql.DB) http.HandlerFunc {
where, args := statsFilter(r.URL.Query(), "received_at")
rows, err := db.QueryContext(r.Context(), fmt.Sprintf(`
SELECT CAST(strftime('%%H', datetime(received_at, 'unixepoch')) AS INTEGER) AS hr,
SELECT EXTRACT(HOUR FROM to_timestamp(received_at) AT TIME ZONE 'UTC')::int AS hr,
COUNT(*) AS cnt
FROM alerts
WHERE %s
GROUP BY hr
ORDER BY hr ASC`, where), args...)
ORDER BY hr ASC`, where), args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -123,14 +121,15 @@ func handleStatsByDay(db *sql.DB) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
where, args := statsFilter(r.URL.Query(), "received_at")
// SQLite strftime('%w') → 0=Sunday … 6=Saturday
// Postgres EXTRACT(DOW …) → 0=Sunday … 6=Saturday, the same numbering
// SQLite's strftime('%w') returned, so the frontend needs no change.
rows, err := db.QueryContext(r.Context(), fmt.Sprintf(`
SELECT CAST(strftime('%%w', datetime(received_at, 'unixepoch')) AS INTEGER) AS dow,
SELECT EXTRACT(DOW FROM to_timestamp(received_at) AT TIME ZONE 'UTC')::int AS dow,
COUNT(*) AS cnt
FROM alerts
WHERE %s
GROUP BY dow
ORDER BY dow ASC`, where), args...)
ORDER BY dow ASC`, where), args.all()...)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -184,7 +183,7 @@ func handleStatsIncidents(db *sql.DB) http.HandlerFunc {
THEN acknowledged_at - triggered_at END),
AVG(CASE WHEN resolved_at IS NOT NULL
THEN resolved_at - triggered_at END)
FROM incidents WHERE %s`, where), args...,
FROM incidents WHERE %s`, where), args.all()...,
).Scan(&total, &triggered, &acknowledged, &resolved, &mtta, &mttr)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
@@ -207,18 +206,17 @@ func handleStatsIncidents(db *sql.DB) http.HandlerFunc {
// statsFilter builds a WHERE clause and args from optional ?from and ?to query
// params, filtering on timeCol. Archived rows are always excluded, matching the
// default list views.
func statsFilter(q url.Values, timeCol string) (where string, args []any) {
func statsFilter(q url.Values, timeCol string) (where string, args *sqlArgs) {
args = &sqlArgs{}
clauses := []string{"archived_at IS NULL"}
if from := q.Get("from"); from != "" {
if t, err := time.Parse("2006-01-02", from); err == nil {
clauses = append(clauses, timeCol+" >= ?")
args = append(args, t.UTC().Unix())
clauses = append(clauses, timeCol+" >= "+args.add(t.UTC().Unix()))
}
}
if to := q.Get("to"); to != "" {
if t, err := time.Parse("2006-01-02", to); err == nil {
clauses = append(clauses, timeCol+" < ?")
args = append(args, t.UTC().AddDate(0, 0, 1).Unix())
clauses = append(clauses, timeCol+" < "+args.add(t.UTC().AddDate(0, 0, 1).Unix()))
}
}
return strings.Join(clauses, " AND "), args
internal/api/testdb_test.go
+100
View File
@@ -0,0 +1,100 @@
package api_test
import (
"database/sql"
"fmt"
"net/url"
"os"
"strings"
"testing"
"git.ryuvia.com/niklas/terdut-server/internal/db"
)
// Tests run against a real Postgres, because the server does. SQLite's
// ":memory:" gave every test a private database for free; Postgres has no
// equivalent, so isolation is bought with a schema per test.
//
// A schema rather than a database: CREATE DATABASE copies a template on disk and
// costs a hundred milliseconds or so each time, while CREATE SCHEMA plus the one
// baseline migration is a few, and the suite runs a few hundred of them. Each
// test's pool is pinned to its own schema through search_path, so two tests
// cannot see each other's rows even though they share a server.
//
// TERDUT_TEST_DSN must point at a database the test role may create schemas in:
//
// postgres://terdut:terdut@localhost:5432/terdut_test?sslmode=disable
//
// `make test-db` starts one locally; ci.yaml runs one as a service container.
// An unset DSN fails rather than skips, deliberately — a suite that quietly
// tests nothing is worse than one that does not run.
const testDSNEnv = "TERDUT_TEST_DSN"
var schemaSeq int
// newTestDB returns a migrated database private to this test, and drops it
// afterwards.
func newTestDB(t *testing.T) *sql.DB {
t.Helper()
dsn := os.Getenv(testDSNEnv)
if dsn == "" {
t.Fatalf("%s is not set: these tests need Postgres.\n"+
"Run `make test-db` for a local one, then\n"+
" export %s=postgres://terdut:terdut@localhost:5432/terdut_test?sslmode=disable",
testDSNEnv, testDSNEnv)
}
schemaSeq++
schema := fmt.Sprintf("test_%d_%d", os.Getpid(), schemaSeq)
admin, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatalf("connect to %s: %v", testDSNEnv, err)
}
defer admin.Close()
if _, err := admin.Exec("CREATE SCHEMA " + schema); err != nil {
t.Fatalf("create schema %s: %v", schema, err)
}
database, err := db.Open(withSearchPath(dsn, schema))
if err != nil {
t.Fatalf("open db: %v", err)
}
if err := db.Migrate(database); err != nil {
t.Fatalf("migrate: %v", err)
}
t.Cleanup(func() {
database.Close()
cleanup, err := sql.Open("pgx", dsn)
if err != nil {
return
}
defer cleanup.Close()
if _, err := cleanup.Exec("DROP SCHEMA " + schema + " CASCADE"); err != nil {
t.Logf("drop schema %s: %v", schema, err)
}
})
return database
}
// withSearchPath pins a DSN to one schema, so every connection the pool opens
// lands there and nothing has to qualify a table name.
//
// Handles both DSN spellings: a postgres:// URL, and libpq's keyword/value form.
func withSearchPath(dsn, schema string) string {
opt := "-csearch_path=" + schema
if strings.HasPrefix(dsn, "postgres://") || strings.HasPrefix(dsn, "postgresql://") {
u, err := url.Parse(dsn)
if err == nil {
q := u.Query()
q.Set("options", opt)
u.RawQuery = q.Encode()
return u.String()
}
}
return dsn + " options='" + opt + "'"
}
internal/api/users.go
+22 -23
View File
@@ -56,27 +56,26 @@ func handleBootstrap(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"INSERT INTO users (username, email, password_hash) VALUES (?, ?, ?)",
req.Username, req.Email, passwordHash)
if err != nil {
var userID int64
if err := db.QueryRowContext(r.Context(),
"INSERT INTO users (username, email, password_hash) VALUES ($1, $2, $3) RETURNING id",
req.Username, req.Email, passwordHash).Scan(&userID); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
userID, _ := res.LastInsertId()
raw, hash, err := randomToken()
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
keyRes, err := db.ExecContext(r.Context(),
"INSERT INTO api_keys (user_id, key_hash, name) VALUES (?, ?, ?)", userID, hash, "bootstrap")
if err != nil {
var keyID int64
if err := db.QueryRowContext(r.Context(),
"INSERT INTO api_keys (user_id, key_hash, name) VALUES ($1, $2, $3) RETURNING id",
userID, hash, "bootstrap").Scan(&keyID); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
keyID, _ := keyRes.LastInsertId()
user, _ := fetchUser(r.Context(), db, userID)
key := models.APIKey{ID: keyID, UserID: userID, Name: "bootstrap", Key: raw, CreatedAt: user.CreatedAt}
@@ -124,17 +123,17 @@ func handleCreateUser(db *sql.DB) http.HandlerFunc {
return
}
res, err := db.ExecContext(r.Context(),
"INSERT INTO users (username, email) VALUES (?, ?)", req.Username, req.Email)
if err != nil {
if strings.Contains(err.Error(), "UNIQUE constraint failed") {
var id int64
if err := db.QueryRowContext(r.Context(),
"INSERT INTO users (username, email) VALUES ($1, $2) RETURNING id",
req.Username, req.Email).Scan(&id); err != nil {
if isUniqueViolation(err) {
respond(w, http.StatusConflict, errResp("username or email already exists"))
return
}
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
id, _ := res.LastInsertId()
user, _ := fetchUser(r.Context(), db, id)
respond(w, http.StatusCreated, user)
}
@@ -165,7 +164,7 @@ func handleSetNotifyTarget(db *sql.DB) http.HandlerFunc {
}
res, err := db.ExecContext(r.Context(),
"UPDATE users SET ntfy_topic = ? WHERE id = ?", topic, id)
"UPDATE users SET ntfy_topic = $1 WHERE id = $2", topic, id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -191,7 +190,7 @@ func handleDeleteUser(db *sql.DB) http.HandlerFunc {
respond(w, http.StatusBadRequest, errResp("invalid user id"))
return
}
res, err := db.ExecContext(r.Context(), "DELETE FROM users WHERE id = ?", id)
res, err := db.ExecContext(r.Context(), "DELETE FROM users WHERE id = $1", id)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -226,7 +225,7 @@ func handleCreateAPIKey(db *sql.DB) http.HandlerFunc {
}
var exists int
if err := db.QueryRowContext(r.Context(), "SELECT 1 FROM users WHERE id = ?", userID).Scan(&exists); err != nil {
if err := db.QueryRowContext(r.Context(), "SELECT 1 FROM users WHERE id = $1", userID).Scan(&exists); err != nil {
respond(w, http.StatusNotFound, errResp("user not found"))
return
}
@@ -236,13 +235,13 @@ func handleCreateAPIKey(db *sql.DB) http.HandlerFunc {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
res, err := db.ExecContext(r.Context(),
"INSERT INTO api_keys (user_id, key_hash, name) VALUES (?, ?, ?)", userID, hash, req.Name)
if err != nil {
var keyID int64
if err := db.QueryRowContext(r.Context(),
"INSERT INTO api_keys (user_id, key_hash, name) VALUES ($1, $2, $3) RETURNING id",
userID, hash, req.Name).Scan(&keyID); err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
}
keyID, _ := res.LastInsertId()
key := models.APIKey{ID: keyID, UserID: userID, Name: req.Name, Key: raw, CreatedAt: time.Now().UTC()}
respond(w, http.StatusCreated, key)
}
@@ -262,7 +261,7 @@ func handleDeleteAPIKey(db *sql.DB) http.HandlerFunc {
}
res, err := db.ExecContext(r.Context(),
"DELETE FROM api_keys WHERE id = ? AND user_id = ?", keyID, userID)
"DELETE FROM api_keys WHERE id = $1 AND user_id = $2", keyID, userID)
if err != nil {
respond(w, http.StatusInternalServerError, errResp("internal error"))
return
@@ -293,7 +292,7 @@ func fetchUser(ctx context.Context, db *sql.DB, id int64) (models.User, error) {
var u models.User
var ts int64
err := db.QueryRowContext(ctx,
"SELECT id, username, email, created_at, ntfy_topic FROM users WHERE id = ?", id).
"SELECT id, username, email, created_at, ntfy_topic FROM users WHERE id = $1", id).
Scan(&u.ID, &u.Username, &u.Email, &ts, &u.NtfyTopic)
if err != nil {
return u, err