Niklas Ye
dc39e3a5d3
First step of #1, and it goes first for one reason: #4 adds a team_id to nearly every table, and doing that twice -- once for SQLite, once for Postgres -- is work nobody gets paid for. The teams migrations now only have to be written against one database. The ten SQLite migrations are replaced by a single Postgres baseline rather than ported one by one. They were incremental in a way that has no value on a fresh install: 004 adds columns 008 drops again, and 008's backfill rewrites data a Postgres database never had. The history stays in git; the schema they add up to is now 001_baseline.sql. Timestamps stay BIGINT unix seconds and are NOT converted to timestamptz. Everything in Go already speaks epochs, so converting would have been a second, larger change riding along inside this one. It is worth doing on its own. The JSON columns did move to jsonb, because #4 will want to filter and index on labels. Most of the port is mechanical -- 170 placeholders from ? to $1 -- but four things needed more than a search and replace: * Dynamically built WHERE clauses cannot keep their numbering straight by hand, so they hand out placeholders through sqlArgs instead. A filter can now be added or reordered without renumbering anything. * SUM(resolved_at IS NULL) was SQLite counting a boolean as 0 or 1. Postgres has no sum(boolean), and this was breaking every dead man's switch -- silently, since the sweeper only logs. Now COUNT(*) FILTER. * unixepoch() became FLOOR(EXTRACT(EPOCH FROM now()))::bigint. The FLOOR is load-bearing: a bare cast rounds half up, so a row written at .6 of a second claimed a timestamp a second in the future and disagreed with the time.Now().Unix() the Go side stamps. * The unique-violation check matched SQLite's error text. It matches SQLSTATE 23505 now, so a renamed constraint cannot turn a 409 back into a 500. Tests need a real Postgres, because there is no in-memory Postgres the way there was an in-memory SQLite. Each test gets its own schema on a shared server -- cheaper than a database each, and still isolated. TERDUT_TEST_DSN says where it is; `make test-db` starts one locally and ci.yaml runs one as a service container. An unset DSN fails the suite rather than skipping it: a run that quietly tests nothing is worse than one that does not run. TestMigration_BackfillCarriesAckAndComments is deleted along with the migrations it replayed. What it protected -- an upgrade not losing acknowledgements and comments -- now belongs to scripts/sqlite-to-postgres.go, which is build-tagged so the SQLite driver stays out of the server binary. Both are meant to be deleted once this install has migrated. The chart loses the PVC, the data volume and the python backup sidecar, and requires database.dsnSecret.name: it provisions no database and cannot guess where the credentials live, so a render without it is meant to fail. Backups move to where Postgres actually runs. The other half of that -- the postgresql CR, the k8up pg_dump annotation and the network policy -- is a change to the wrapper chart in Ryuvia/charts and is not in here. Verified rather than assumed: the gate is green with -race against Postgres 17, govulncheck and gitleaks are clean, and the migration script was run end to end against a SQLite database built at the old schema and seeded in every table. Ids survive, so incidents keep their numbers and every foreign key still points where it did; the identity sequences are moved past the copied ids, and a webhook after the migration opened incident 12 rather than colliding at 1.
259 lines
12 KiB
Makefile
259 lines
12 KiB
Makefile
REGISTRY := git.ryuvia.com
|
|
# The personal namespace, not ryuvia — deliberately, and for one reason: Gitea
|
|
# scopes package visibility to the owner with no per-package override, so
|
|
# ryuvia/* is private because the org is. Publishing here keeps the image and
|
|
# chart anonymously pullable, so no pull secret is needed in the cluster and
|
|
# Flux needs no registry credentials. Same choice riksdata and rd-web made.
|
|
OWNER := niklas
|
|
|
|
IMAGE := $(REGISTRY)/$(OWNER)/terdut-server
|
|
HELM_CHART := charts/terdut-server
|
|
HELM_REPO := oci://$(REGISTRY)/$(OWNER)
|
|
|
|
# go.mod pins an exact patch release so nobody builds the shipped binary with a
|
|
# toolchain carrying known stdlib CVEs. Fedora's Go package overrides the
|
|
# upstream GOTOOLCHAIN default to `local`, which turns that pin into a hard
|
|
# failure on a dev box one patch behind, so restore the upstream default here.
|
|
export GOTOOLCHAIN ?= auto
|
|
|
|
.PHONY: help
|
|
help: ## Show this help
|
|
@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | \
|
|
awk 'BEGIN {FS = ":.*?## "}; {printf " \033[36m%-22s\033[0m %s\n", $$1, $$2}'
|
|
|
|
## --- checks ---
|
|
#
|
|
# These three mirror .gitea/workflows/ci.yaml step for step, so a green `make fmt
|
|
# lint test` here means the same thing CI means. The one deliberate difference is
|
|
# -race below. Both need a Postgres to test against; see test-db.
|
|
|
|
# The suite needs a Postgres, because the server does: there is no in-memory
|
|
# Postgres the way there was an in-memory SQLite. TERDUT_TEST_DSN says where, and
|
|
# the tests fail rather than skip without it — a suite that quietly tests nothing
|
|
# is worse than one that does not run. `make test-db` starts a local one;
|
|
# ci.yaml runs the same thing as a service container.
|
|
TEST_DB_CONTAINER ?= terdut-test-db
|
|
TEST_DB_PORT ?= 5433
|
|
TEST_DB_IMAGE ?= docker.io/library/postgres:17-alpine
|
|
export TERDUT_TEST_DSN ?= postgres://terdut:terdut@localhost:$(TEST_DB_PORT)/terdut_test?sslmode=disable
|
|
|
|
.PHONY: test
|
|
test: ## Run the test suite (needs TERDUT_TEST_DSN; see test-db)
|
|
go test -race ./...
|
|
|
|
# podman, with docker as the fallback: this is a dev convenience, not part of the
|
|
# pipeline, where the database arrives as a service container instead.
|
|
.PHONY: test-db
|
|
test-db: ## Start a local Postgres for the tests
|
|
@runtime=$$(command -v podman || command -v docker); \
|
|
if [ -z "$$runtime" ]; then echo "need podman or docker"; exit 1; fi; \
|
|
$$runtime run -d --rm --name $(TEST_DB_CONTAINER) \
|
|
-e POSTGRES_USER=terdut -e POSTGRES_PASSWORD=terdut -e POSTGRES_DB=terdut_test \
|
|
-p $(TEST_DB_PORT):5432 $(TEST_DB_IMAGE) >/dev/null; \
|
|
printf 'waiting for postgres'; \
|
|
for i in $$(seq 1 60); do \
|
|
if $$runtime exec $(TEST_DB_CONTAINER) pg_isready -U terdut -d terdut_test >/dev/null 2>&1; then \
|
|
echo " ready: $(TERDUT_TEST_DSN)"; exit 0; \
|
|
fi; \
|
|
printf '.'; sleep 1; \
|
|
done; \
|
|
echo " timed out"; exit 1
|
|
|
|
.PHONY: test-db-stop
|
|
test-db-stop: ## Stop the local test Postgres
|
|
@runtime=$$(command -v podman || command -v docker); \
|
|
$$runtime rm -f $(TEST_DB_CONTAINER) >/dev/null 2>&1 || true
|
|
|
|
# CI runs a bare `go test ./...`. This is stricter on purpose: the sweeper, the
|
|
# notifier goroutine and the deadman sweep all touch the same single-connection
|
|
# database, and a race there would surface as a flaky production incident rather
|
|
# than a failed build. It passes today; if it ever costs more than it catches,
|
|
# the honest fix is to teach CI -race too, not to quietly drop it here.
|
|
.PHONY: lint
|
|
lint: ## go vet
|
|
go vet ./...
|
|
|
|
# Copied from ci.yaml rather than simplified, because both of gofmt's failure
|
|
# modes need handling and they are not alike. A file that is merely misformatted
|
|
# is listed on stdout with exit 0 — so the failure has to be raised by hand. A
|
|
# file that does not parse is the opposite: nothing on stdout and exit 2, which a
|
|
# naive `[ -n "$$out" ]` reads as success. See 9046f6e.
|
|
.PHONY: fmt
|
|
fmt: ## Report unformatted files
|
|
@if ! unformatted=$$(gofmt -l .); then \
|
|
echo "gofmt could not parse the tree:"; gofmt -l .; exit 1; \
|
|
fi; \
|
|
if [ -n "$$unformatted" ]; then \
|
|
echo "gofmt needed:"; echo "$$unformatted"; gofmt -d .; exit 1; \
|
|
fi
|
|
|
|
# database.dsnSecret.name has no default and the deployment `required`s it: the
|
|
# chart provisions no database and cannot guess where the credentials live, so a
|
|
# render without it is meant to fail. Setting it here keeps the lint honest about
|
|
# what a working install needs.
|
|
HELM_LINT_SET = --set image.tag=v0.0.0 --set database.dsnSecret.name=terdut-db
|
|
|
|
.PHONY: helm-lint
|
|
helm-lint: ## Lint and render the chart
|
|
helm lint $(HELM_CHART) $(HELM_LINT_SET)
|
|
helm template terdut-server $(HELM_CHART) --namespace terdut-server \
|
|
$(HELM_LINT_SET) >/dev/null
|
|
@# networking.listener defaults to "", which attaches the route to every
|
|
@# matching listener including plaintext HTTP. Production sets it, so the
|
|
@# default render proves nothing about the path that actually ships.
|
|
helm template terdut-server $(HELM_CHART) --namespace terdut-server \
|
|
$(HELM_LINT_SET) --set networking.listener=https-terdut >/dev/null
|
|
|
|
## --- release ---
|
|
|
|
# The release process (~/.claude/skills/release) reads these rather than restating them.
|
|
# One definition, so the version that gets tagged, the image that gets pushed and the chart
|
|
# the wrapper pins cannot drift apart in a second copy.
|
|
.PHONY: release-vars
|
|
release-vars: ## Print the variables the release process reads
|
|
@printf 'IMAGE=%s\nHELM_CHART=%s\nHELM_REPO=%s\n' '$(IMAGE)' '$(HELM_CHART)' '$(HELM_REPO)'
|
|
|
|
# There is deliberately no build/push/helm-package/helm-push/release here, unlike
|
|
# riksdata and rd-web. .gitea/workflows/release.yaml owns publishing for this repo,
|
|
# and it does two things a local make cannot: it builds linux/amd64 and linux/arm64
|
|
# through buildx, and it stamps the chart's version and appVersion from the tag. A
|
|
# `docker build && docker push` target would push a single-architecture image over
|
|
# the multi-arch tag, which is both easy to do by accident and invisible afterwards
|
|
# — the tag would still resolve, just not on arm64. Publishing happens by pushing a
|
|
# tag; nothing else.
|
|
|
|
## --- publishing ---
|
|
#
|
|
# These exist so .gitea/workflows/release.yaml can call `make release` instead of
|
|
# restating the build in YAML, the way riksdata and rd-web already do. One definition
|
|
# of how this is built and published, runnable locally, reviewable in a diff.
|
|
#
|
|
# VERSION is the git tag, passed in by the workflow. The guard below is why a stray
|
|
# local `make release` cannot publish: dev is not a version anyone releases.
|
|
|
|
VERSION ?= dev
|
|
|
|
# Helm requires strict SemVer — strip a leading 'v' if present.
|
|
CHART_VERSION := $(shell echo "$(VERSION)" | sed 's/^v//')
|
|
|
|
PLATFORMS ?= linux/amd64,linux/arm64
|
|
BUILDX_BUILDER ?= terdut
|
|
|
|
TRIVY_VERSION := 0.73.0
|
|
GOVULNCHECK_VERSION := v1.1.4
|
|
GITLEAKS_VERSION := v8.30.0
|
|
|
|
# --pull, not --no-cache: refresh the base image without discarding the layer cache.
|
|
DOCKER_BUILD_FLAGS ?= --pull
|
|
|
|
# An isolated repo list. The machine-wide one is not this build's business, and one
|
|
# unreachable entry in it aborts otherwise-fine helm commands — there is a dead
|
|
# TrueCharts repo on this host that does exactly that. HELM_REPOSITORY_CACHE is
|
|
# deliberately NOT overridden alongside it: helm writes a refreshed index to the default
|
|
# cache and then looks for it in the overridden one.
|
|
HELM_ISOLATED = HELM_REPOSITORY_CONFIG=$(CURDIR)/.helm-repos.yaml
|
|
|
|
.PHONY: require-version
|
|
require-version:
|
|
@test "$(VERSION)" != "dev" || \
|
|
(echo "VERSION=dev names no release — pass VERSION=vX.Y.Z (the workflow passes the tag)" && exit 1)
|
|
|
|
.PHONY: build
|
|
build: ## Build the image for this host only, without pushing (local check / CI smoke)
|
|
docker build $(DOCKER_BUILD_FLAGS) \
|
|
--build-arg VERSION=$(VERSION) \
|
|
-t $(IMAGE):$(VERSION) .
|
|
|
|
# Multi-arch, so unlike riksdata and rd-web this cannot be a separate build then push:
|
|
# buildx cannot load a multi-platform result into the local image store, it can only
|
|
# push it. `build` above stays single-platform and local-only for that reason.
|
|
#
|
|
# No QEMU: the Dockerfile's builder stage runs on $$BUILDPLATFORM and cross-compiles from
|
|
# TARGETARCH, so both platforms build natively. The default "docker" driver cannot build
|
|
# more than one platform at a time; the docker-container driver can.
|
|
.PHONY: push
|
|
push: require-version ## Build and publish the multi-arch image
|
|
docker buildx create --name $(BUILDX_BUILDER) --use 2>/dev/null || docker buildx use $(BUILDX_BUILDER)
|
|
docker buildx build \
|
|
--platform $(PLATFORMS) \
|
|
--build-arg "VERSION=$(VERSION)" \
|
|
--tag "$(IMAGE):latest" \
|
|
--tag "$(IMAGE):$(VERSION)" \
|
|
--push .
|
|
|
|
# --version and --app-version come from the tag, so Chart.yaml's own fields decide nothing
|
|
# about what is published. They used to be rewritten in place with sed before packaging;
|
|
# the flags do the same job without mutating the tree mid-build.
|
|
.PHONY: helm-package
|
|
helm-package: require-version ## Package the chart, versioned from the tag
|
|
$(HELM_ISOLATED) helm package $(HELM_CHART) \
|
|
--version $(CHART_VERSION) \
|
|
--app-version $(VERSION) \
|
|
--destination dist
|
|
|
|
.PHONY: helm-push
|
|
helm-push: require-version ## Push the packaged chart to the OCI registry
|
|
$(HELM_ISOLATED) helm push dist/terdut-server-$(CHART_VERSION).tgz $(HELM_REPO)
|
|
|
|
.PHONY: binaries
|
|
binaries: require-version ## Cross-compile the release binaries into dist/
|
|
@mkdir -p dist
|
|
@set -eu; for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64; do \
|
|
GOOS="$${target%/*}"; GOARCH="$${target#*/}"; \
|
|
out="dist/terdut-$(VERSION)-$${GOOS}-$${GOARCH}"; \
|
|
echo "building $$out"; \
|
|
GOOS="$$GOOS" GOARCH="$$GOARCH" go build \
|
|
-ldflags "-w -s -X main.version=$(VERSION)" \
|
|
-o "$$out" ./cmd/terdut; \
|
|
done
|
|
|
|
.PHONY: release
|
|
release: push helm-package helm-push ## Publish image + chart (the workflow's one call)
|
|
|
|
## --- security ---
|
|
|
|
# Symbol-level, not dependency-level: govulncheck reports a vulnerability only when the
|
|
# code can actually reach it. As of 2026-09-02 this repo imports three chi advisories and
|
|
# reports none of them, because all three are middleware.RealIP and router.go uses Logger
|
|
# and Recoverer. That is the useful property rather than a loophole -- adding
|
|
# middleware.RealIP would turn this red, which is exactly when someone should look.
|
|
.PHONY: security-go
|
|
security-go: ## Scan Go deps for known CVEs (govulncheck)
|
|
go run golang.org/x/vuln/cmd/govulncheck@$(GOVULNCHECK_VERSION) ./...
|
|
|
|
# --no-git scans the working tree rather than the history, so this catches a secret on the
|
|
# way in. It is not a history audit and finding nothing here says nothing about what is
|
|
# already committed. --redact because the finding is printed into a CI log.
|
|
#
|
|
# Note when testing it that gitleaks allowlists well-known example credentials -- the AWS
|
|
# key from their own documentation does not trip it. A private key block does.
|
|
.PHONY: security-secrets
|
|
security-secrets: ## Scan the working tree for committed secrets (gitleaks)
|
|
go run github.com/zricethezav/gitleaks/v8@$(GITLEAKS_VERSION) detect --no-git \
|
|
--source . --redact --no-banner --exit-code 1
|
|
|
|
|
|
# Scans the pushed image, not a local one: trivy cannot read a locally built image on the
|
|
# runner -- Talos has no docker socket, and the dind sidecar shares no filesystem with the
|
|
# job -- so it pulls from the registry. Same reason riksdata and rd-web scan after pushing.
|
|
#
|
|
# It cannot gate a deploy, because this pipeline does not deploy. A red scan means: do not
|
|
# bump the wrapper chart in Ryuvia/charts to this version.
|
|
#
|
|
# The image is FROM scratch, so there are no OS packages to scan and trivy sees exactly one
|
|
# target -- the Go binary and its module graph. That also makes scanning a single platform
|
|
# sufficient here: linux/amd64 and linux/arm64 are the same modules built for a different
|
|
# GOARCH, so a CVE in one is a CVE in both. On an image with a base layer that would not
|
|
# hold and both platforms would need scanning.
|
|
#
|
|
# This is the last of the three scans and the only one that needs a published artifact;
|
|
# security-go and security-secrets above run on every push.
|
|
.PHONY: security-image
|
|
security-image: require-version ## Scan the pushed image for CVEs (needs VERSION)
|
|
@# The named volume persists trivy's vulnerability DB between runs; without it every
|
|
@# scan re-downloads the whole database.
|
|
docker run --rm -e TRIVY_USERNAME -e TRIVY_PASSWORD \
|
|
-v trivy-cache:/root/.cache/trivy \
|
|
docker.io/aquasec/trivy:$(TRIVY_VERSION) image --severity HIGH,CRITICAL \
|
|
--ignore-unfixed --exit-code 1 $(IMAGE):$(VERSION)
|