diff --git a/README.md b/README.md index 2b98acd..24d991d 100644 --- a/README.md +++ b/README.md @@ -182,19 +182,39 @@ In the Helm chart the two sweeper durations are set via `sweeper.staleAfter` and ## Alertmanager configuration -Add terdut-server as a webhook receiver in your `alertmanager.yml`: +Alerts arrive on a team's **integration key**, which says both that the sender +may post and which team the alerts belong to. Mint one as an owner of the team: + +```bash +curl -X POST https://terdut.example.com/api/teams/1/integrations \ + -H "Authorization: Bearer $TERDUT_API_KEY" \ + -H 'Content-Type: application/json' \ + -d '{"name":"prod alertmanager"}' +``` + +The response carries the key and the full URL **once**; only a SHA-256 hash is +stored. Put it in your `alertmanager.yml`: ```yaml receivers: - name: terdut webhook_configs: - - url: http://terdut-server:8080/api/alertmanager/webhook + - url: http://terdut-server:8080/api/integrations//alertmanager send_resolved: true route: receiver: terdut ``` +The whole URL is a credential, so treat it like one. Alertmanager 0.26 and +later can read it from a file with `url_file:` instead, which keeps it out of +your configuration repository: + +```yaml + - url_file: /etc/alertmanager/secrets/terdut-webhook-url/url + send_resolved: true +``` + The webhook endpoint requires no authentication. If you use the [dead man's switch](#dead-mans-switch) — and the default configuration does — give @@ -447,7 +467,7 @@ of the last heartbeat, and the heartbeat's labels are on the incident's ### Authentication -All endpoints except `/api/bootstrap`, `/api/alertmanager/webhook`, +All endpoints except `/api/bootstrap`, `/api/integrations/{key}/alertmanager`, `/api/notify/ack/{token}`, `/api/login` and `/api/logout` require either an API key: ``` @@ -517,10 +537,10 @@ and the full URL once and stores only a SHA-256 hash. | Method | Path | Description | |---|---|---| | `POST` | `/api/integrations/{key}/alertmanager` | Alertmanager v4 webhook receiver for the key's team. `401` for an unknown key | -| `POST` | `/api/alertmanager/webhook` | **Deprecated, unauthenticated.** The pre-teams receiver, kept for one release so an upgrade does not stop delivering while the Alertmanager config is edited. Routes everything to the oldest team | -The deprecated path is why anything that can reach the port can still open an -incident. Move senders to a key and it goes away. +This is the only way in. The pre-teams `POST /api/alertmanager/webhook` took no +credential at all — anything able to reach the port could open an incident — +and was removed in v0.13.0 once senders had moved onto keys. ### Teams @@ -765,10 +785,11 @@ name on them. What changes, and will need attention: -- **Alert ingestion moved.** `POST /api/alertmanager/webhook` still works but is - deprecated and unauthenticated, and routes everything to the oldest team. Mint - a key with `POST /api/teams/{teamID}/integrations` and point Alertmanager at - the URL it returns. The old path goes away in a later release. +- **Alert ingestion moved.** Mint a key with + `POST /api/teams/{teamID}/integrations` and point Alertmanager at the URL it + returns. In v0.12.0 the old `POST /api/alertmanager/webhook` still worked, + deprecated, routing everything to the oldest team; **v0.13.0 removes it**, so + upgrade straight from v0.11.x to v0.13.0 only after the senders are moved. - **The schedule endpoints moved** under `/api/teams/{teamID}/schedule`, and editing the rota is now an owner's job. `GET /api/schedule/current` stayed where it was but now returns an **array** — one entry per team with somebody @@ -810,30 +831,25 @@ non-administrator now gets `403` where a `200` used to come back. ## Upgrading from SQLite -Versions up to v0.10.2 stored everything in a SQLite file. From the Postgres release onwards -the server needs `TERDUT_DB_DSN` and keeps nothing on disk. +Versions up to v0.10.2 stored everything in a SQLite file. From v0.11.1 the server needs +`TERDUT_DB_DSN` and keeps nothing on disk. -The cutover is ordered — the server must not be running while the copy happens: +The copy was done by `scripts/sqlite-to-postgres.go`, which **was deleted in v0.13.0** along +with the SQLite driver it was the last user of. It is still in the history — check out the +`v0.12.0` tag to get it: ```bash -# 1. Stop the old server, keeping its database file. -# 2. Create an empty Postgres database, then let the new binary build the schema: -TERDUT_DB_DSN='postgres://terdut:secret@localhost:5432/terdut?sslmode=disable' ./terdut & -# ...watch for "listening on", then stop it again. -# 3. Copy the data across: -go run -tags migrate ./scripts/sqlite-to-postgres.go \ - -sqlite /data/terdut.db \ - -dsn 'postgres://terdut:secret@localhost:5432/terdut?sslmode=disable' -# 4. Start the new server for good. +git show v0.12.0:scripts/sqlite-to-postgres.go > sqlite-to-postgres.go ``` +The cutover is ordered, and the server must not be running while the copy happens: stop the +old version, let the new binary build the schema against an empty Postgres, run the script +with `-sqlite` and `-dsn`, then start the new version for good. On Kubernetes step three runs +as a Job with the same image against the PVC before it is removed. + The copy preserves every id, so incidents keep their numbers and the timeline, alert membership, outbox and ack tokens all still point where they did. It refuses a target that -already has rows, so a second run cannot double-insert. On Kubernetes, step 3 runs as a Job -with the same image against the PVC before it is removed. - -The script is deliberately temporary: it is the only thing left that needs the SQLite driver, -and both should be deleted once the installs that need them have migrated. +already has rows, so a second run cannot double-insert. ## Upgrading to incidents diff --git a/go.mod b/go.mod index c0f55df..827ad09 100644 --- a/go.mod +++ b/go.mod @@ -7,22 +7,12 @@ require ( github.com/jackc/pgerrcode v0.0.0-20250907135507-afb5586c32a6 github.com/jackc/pgx/v5 v5.11.0 golang.org/x/crypto v0.55.0 - modernc.org/sqlite v1.50.1 ) require ( - github.com/dustin/go-humanize v1.0.1 // indirect - github.com/google/uuid v1.6.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect - github.com/mattn/go-isatty v0.0.20 // indirect - github.com/ncruces/go-strftime v1.0.0 // indirect - github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect golang.org/x/sync v0.22.0 // indirect - golang.org/x/sys v0.47.0 // indirect golang.org/x/text v0.41.0 // indirect - modernc.org/libc v1.72.3 // indirect - modernc.org/mathutil v1.7.1 // indirect - modernc.org/memory v1.11.0 // indirect ) diff --git a/go.sum b/go.sum index 830fa1b..d2d1bac 100644 --- a/go.sum +++ b/go.sum @@ -1,16 +1,8 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= -github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug= github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0= -github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs= -github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA= -github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= -github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= -github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= github.com/jackc/pgerrcode v0.0.0-20250907135507-afb5586c32a6 h1:D/V0gu4zQ3cL2WKeVNVM4r2gLxGGf6McLwgXzRTo2RQ= github.com/jackc/pgerrcode v0.0.0-20250907135507-afb5586c32a6/go.mod h1:a/s9Lp5W7n/DD0VrVoyJ00FbP2ytTPDVOivvn2bMlds= github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM= @@ -21,14 +13,8 @@ github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg= github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= -github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= -github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= -github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w= -github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE= -github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= @@ -36,46 +22,11 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M= golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis= -golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk= -golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40= golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek= golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0= -golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= -golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8= golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M= -golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE= -golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -modernc.org/cc/v4 v4.28.2 h1:3tQ0lf2ADtoby2EtSP+J7IE2SHwEJdP8ioR59wx7XpY= -modernc.org/cc/v4 v4.28.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI= -modernc.org/ccgo/v4 v4.34.0 h1:yRLPFZieg532OT4rp4JFNIVcquwalMX26G95WQDqwCQ= -modernc.org/ccgo/v4 v4.34.0/go.mod h1:AS5WYMyBakQ+fhsHhtP8mWB82KTGPkNNJDGfGQCe0/A= -modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM= -modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU= -modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI= -modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito= -modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo= -modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY= -modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks= -modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI= -modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU= -modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs= -modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU= -modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg= -modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI= -modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw= -modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg= -modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns= -modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w= -modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE= -modernc.org/sqlite v1.50.1 h1:l+cQvn0sd0zJJtfygGHuQJ5AjlrwXmWPw4KP3ZMwr9w= -modernc.org/sqlite v1.50.1/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM= -modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0= -modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A= -modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y= -modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM= diff --git a/internal/api/alertmanager.go b/internal/api/alertmanager.go index ff669df..2fd1e8e 100644 --- a/internal/api/alertmanager.go +++ b/internal/api/alertmanager.go @@ -94,27 +94,6 @@ func handleIntegrationWebhook(db *sql.DB, notify NotifyConfig) http.HandlerFunc } } -// handleLegacyWebhook is the pre-teams unauthenticated endpoint, kept for one -// release so an upgrade does not silently stop delivering while somebody edits -// the Alertmanager config. It routes to the oldest team, which on an upgraded -// install is the Default team everything was moved into. -// -// It is deprecated and unauthenticated — anything that can reach the port can -// open an incident. Move senders to an integration key and this goes away. -func handleLegacyWebhook(db *sql.DB, notify NotifyConfig) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - teamID, err := defaultTeamID(r.Context(), db) - if err != nil { - log.Printf("legacy webhook: no team to route to: %v", err) - w.WriteHeader(http.StatusOK) - return - } - log.Printf("legacy webhook: unauthenticated payload routed to team %d; "+ - "move this sender to an integration key", teamID) - receiveWebhook(w, r, db, notify, teamID) - } -} - func receiveWebhook(w http.ResponseWriter, r *http.Request, db *sql.DB, notify NotifyConfig, teamID int64) { var payload amPayload if err := decodeJSON(r, &payload); err != nil { diff --git a/internal/api/api_test.go b/internal/api/api_test.go index 7ff6b62..038218f 100644 --- a/internal/api/api_test.go +++ b/internal/api/api_test.go @@ -21,10 +21,14 @@ import ( // tests can age rows directly — the sweeper's inputs are wall-clock timestamps. type ts struct { *httptest.Server - key string - db *sql.DB - notify api.NotifyConfig - deadman api.DeadmanConfig + key string + // ingestKey is an integration key for the default team: the only way in + // since the unauthenticated webhook was removed, so the tests exercise the + // same path production does. + ingestKey string + db *sql.DB + notify api.NotifyConfig + deadman api.DeadmanConfig } // newTS builds a server over a fresh database. Notifications are off @@ -66,6 +70,16 @@ func newDeadmanTS(t *testing.T, deadman api.DeadmanConfig, notify ...api.NotifyC s := &ts{Server: srv, key: key, db: database, notify: cfg, deadman: deadman} + var integration struct { + Key string `json:"key"` + } + decode(t, s.req(t, http.MethodPost, "/api/teams/"+defaultTeam+"/integrations", + map[string]string{"name": "test"}), &integration) + if integration.Key == "" { + t.Fatal("no integration key was returned") + } + s.ingestKey = integration.Key + // Dead man's switches belong to a team now, so a test that wants them // configures the default team the way an owner would. if deadman.Timeout > 0 { @@ -234,7 +248,8 @@ func postWebhook(t *testing.T, s *ts, alerts []map[string]any, groupKey ...strin } } data, _ := json.Marshal(payload) - resp, err := http.Post(s.URL+"/api/alertmanager/webhook", "application/json", bytes.NewReader(data)) + resp, err := http.Post(s.URL+"/api/integrations/"+s.ingestKey+"/alertmanager", + "application/json", bytes.NewReader(data)) if err != nil { t.Fatalf("post webhook: %v", err) } diff --git a/internal/api/router.go b/internal/api/router.go index e448c81..48e072b 100644 --- a/internal/api/router.go +++ b/internal/api/router.go @@ -31,14 +31,13 @@ func NewRouter(db *sql.DB, notify NotifyConfig) http.Handler { // Alert ingestion. The key in the path says both that the sender may post // and which team the alerts belong to, which is why it needs no session. + // + // This is the only way in. The pre-teams /api/alertmanager/webhook, which + // took no credential at all, was removed in v0.13.0 once the cluster's + // Alertmanager had moved onto a key; a sender still posting there gets the + // JSON 404 every unknown /api path gets. r.Post("/api/integrations/{key}/alertmanager", handleIntegrationWebhook(db, notify)) - // DEPRECATED, and unauthenticated: anything that can reach the port can - // open an incident here. Kept for one release so an upgrade does not stop - // delivering while the Alertmanager config is edited; it routes everything - // to the oldest team. Remove it once senders carry a key. - r.Post("/api/alertmanager/webhook", handleLegacyWebhook(db, notify)) - // Signing in to the web UI. Login trades a password for a session cookie, // which AuthMiddleware accepts in place of an API key. r.Post("/api/login", handleLogin(db, newLoginLimiter(), notify.PublicURL)) diff --git a/internal/api/teams.go b/internal/api/teams.go index 1e2bbac..7645063 100644 --- a/internal/api/teams.go +++ b/internal/api/teams.go @@ -463,9 +463,10 @@ func teamParam(w http.ResponseWriter, r *http.Request) (int64, bool) { return id, true } -// defaultTeamID is the team the deprecated unauthenticated webhook routes to: -// the oldest one, which on an upgraded install is the "Default" team every -// pre-teams row was moved into. +// defaultTeamID is the oldest team, which on an upgraded install is the +// "Default" team every pre-teams row was moved into and on a fresh one is the +// team migration 003 creates. Bootstrap puts the first user in it, so somebody +// signing in to a new server lands somewhere rather than in no team at all. func defaultTeamID(ctx context.Context, db *sql.DB) (int64, error) { var id int64 err := db.QueryRowContext(ctx, "SELECT id FROM teams ORDER BY id LIMIT 1").Scan(&id) diff --git a/scripts/sqlite-to-postgres.go b/scripts/sqlite-to-postgres.go deleted file mode 100644 index 9260220..0000000 --- a/scripts/sqlite-to-postgres.go +++ /dev/null @@ -1,198 +0,0 @@ -//go:build migrate - -// Command sqlite-to-postgres copies a terdut SQLite database into a freshly -// migrated Postgres one. It exists for exactly one upgrade — the one that moved -// this server off SQLite — and should be deleted once the installs that need it -// have run it. The modernc.org/sqlite dependency goes with it. -// -// Build-tagged so the dependency stays out of the server binary and out of a -// plain `go build ./...`: -// -// go run -tags migrate ./scripts/sqlite-to-postgres.go \ -// -sqlite /path/to/terdut.db \ -// -dsn 'postgres://terdut:secret@localhost:5432/terdut?sslmode=disable' -// -// The Postgres side must already have the schema: start the new server once -// against an empty database, let it migrate, stop it, then run this. The copy -// refuses to touch a database that already has rows, so a second run cannot -// double-insert. -// -// Ids are preserved, which is what keeps every foreign key — incident_alerts, -// incident_events, notifications, the ack tokens — pointing at the same rows it -// pointed at before. The identity sequences are moved past the copied ids at the -// end, so the first row the server writes afterwards does not collide. -package main - -import ( - "database/sql" - "flag" - "fmt" - "log" - "strings" - - _ "github.com/jackc/pgx/v5/stdlib" - _ "modernc.org/sqlite" -) - -// tables are copied parents first: every foreign key points at a table earlier -// in this list. -var tables = []struct { - name string - columns []string - // jsonb marks columns that were TEXT in SQLite and are jsonb in Postgres, - // so the insert can cast them. - jsonb []string - // sequence is the identity sequence to advance afterwards, empty when the - // table has no generated id. - sequence string -}{ - {name: "users", columns: []string{"id", "username", "email", "created_at", "ntfy_topic", "password_hash"}, sequence: "users_id_seq"}, - {name: "api_keys", columns: []string{"id", "user_id", "key_hash", "name", "created_at", "last_used_at"}, sequence: "api_keys_id_seq"}, - {name: "sessions", columns: []string{"id", "token_hash", "user_id", "created_at", "last_seen_at", "expires_at", "user_agent"}, sequence: "sessions_id_seq"}, - {name: "alerts", columns: []string{"id", "fingerprint", "name", "status", "labels", "annotations", "starts_at", "ends_at", "generator_url", "received_at", "archived_at", "resolution_source"}, jsonb: []string{"labels", "annotations"}, sequence: "alerts_id_seq"}, - {name: "schedule_entries", columns: []string{"id", "user_id", "date", "created_at"}, sequence: "schedule_entries_id_seq"}, - {name: "incidents", columns: []string{"id", "group_key", "title", "group_labels", "status", "severity", "triggered_at", "acknowledged_by", "acknowledged_at", "assigned_to", "snoozed_until", "resolved_at", "resolution_source", "archived_at"}, jsonb: []string{"group_labels"}, sequence: "incidents_id_seq"}, - {name: "incident_alerts", columns: []string{"incident_id", "alert_id", "added_at"}}, - {name: "incident_events", columns: []string{"id", "incident_id", "type", "user_id", "alert_id", "detail", "created_at"}, sequence: "incident_events_id_seq"}, - {name: "notifications", columns: []string{"id", "incident_id", "user_id", "topic", "kind", "created_at", "send_after", "attempts", "sent_at", "last_error"}, sequence: "notifications_id_seq"}, - {name: "incident_ack_tokens", columns: []string{"token_hash", "incident_id", "user_id", "created_at", "expires_at"}}, -} - -func main() { - sqlitePath := flag.String("sqlite", "", "path to the existing terdut SQLite database") - dsn := flag.String("dsn", "", "Postgres DSN of the migrated, empty database") - flag.Parse() - - if *sqlitePath == "" || *dsn == "" { - log.Fatal("both -sqlite and -dsn are required") - } - - src, err := sql.Open("sqlite", *sqlitePath) - if err != nil { - log.Fatalf("open sqlite: %v", err) - } - defer src.Close() - - dst, err := sql.Open("pgx", *dsn) - if err != nil { - log.Fatalf("open postgres: %v", err) - } - defer dst.Close() - - if err := dst.Ping(); err != nil { - log.Fatalf("ping postgres: %v", err) - } - if err := assertEmpty(dst); err != nil { - log.Fatalf("%v", err) - } - - // One transaction for the whole copy: a run that dies half way leaves the - // target as it found it, rather than a partial database someone has to - // recognise as partial. - tx, err := dst.Begin() - if err != nil { - log.Fatalf("begin: %v", err) - } - defer tx.Rollback() //nolint:errcheck - - for _, t := range tables { - n, err := copyTable(src, tx, t.name, t.columns, t.jsonb) - if err != nil { - log.Fatalf("copy %s: %v", t.name, err) - } - log.Printf("%-20s %d row(s)", t.name, n) - } - - for _, t := range tables { - if t.sequence == "" { - continue - } - if err := advanceSequence(tx, t.sequence, t.name); err != nil { - log.Fatalf("advance %s: %v", t.sequence, err) - } - } - - if err := tx.Commit(); err != nil { - log.Fatalf("commit: %v", err) - } - log.Print("done") -} - -// assertEmpty refuses a target that already holds data, so running this twice -// cannot duplicate anything. -func assertEmpty(dst *sql.DB) error { - for _, t := range tables { - var n int64 - if err := dst.QueryRow("SELECT COUNT(*) FROM " + t.name).Scan(&n); err != nil { - return fmt.Errorf("count %s (has the new server migrated this database?): %w", t.name, err) - } - if n > 0 { - return fmt.Errorf("%s already has %d row(s): the target must be empty", t.name, n) - } - } - return nil -} - -func copyTable(src *sql.DB, tx *sql.Tx, table string, columns, jsonb []string) (int64, error) { - rows, err := src.Query("SELECT " + strings.Join(columns, ", ") + " FROM " + table) - if err != nil { - return 0, err - } - defer rows.Close() - - insert := "INSERT INTO " + table + " (" + strings.Join(columns, ", ") + ") VALUES (" + - strings.Join(valuePlaceholders(columns, jsonb), ", ") + ")" - - stmt, err := tx.Prepare(insert) - if err != nil { - return 0, err - } - defer stmt.Close() - - var copied int64 - for rows.Next() { - // Scanning into any lets the SQLite driver decide each column's Go type - // and hands it straight back to pgx, which is all this needs: the column - // types match on both sides, apart from the JSON casts above. - values := make([]any, len(columns)) - targets := make([]any, len(columns)) - for i := range values { - targets[i] = &values[i] - } - if err := rows.Scan(targets...); err != nil { - return copied, err - } - if _, err := stmt.Exec(values...); err != nil { - return copied, fmt.Errorf("insert row %d: %w", copied+1, err) - } - copied++ - } - return copied, rows.Err() -} - -// valuePlaceholders numbers the placeholders, casting the columns that became -// jsonb: pgx sends a Go string as text, and Postgres will not assign text to a -// jsonb column without being told. -func valuePlaceholders(columns, jsonb []string) []string { - isJSON := make(map[string]bool, len(jsonb)) - for _, c := range jsonb { - isJSON[c] = true - } - out := make([]string, len(columns)) - for i, c := range columns { - out[i] = fmt.Sprintf("$%d", i+1) - if isJSON[c] { - out[i] += "::jsonb" - } - } - return out -} - -// advanceSequence puts an identity sequence past the largest copied id. Without -// it the first insert after the migration would reuse id 1. -func advanceSequence(tx *sql.Tx, sequence, table string) error { - _, err := tx.Exec(fmt.Sprintf( - `SELECT setval('%s', COALESCE((SELECT MAX(id) FROM %s), 0) + 1, false)`, - sequence, table)) - return err -}