Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 9669b8f477 | |||
| a7871ed7c6 | |||
| 79f5db2636 | |||
| 84146fc903 | |||
| c6f1fe317e |
@@ -71,6 +71,37 @@ jobs:
|
|||||||
- name: Format, vet and test
|
- name: Format, vet and test
|
||||||
run: make fmt lint test
|
run: make fmt lint test
|
||||||
|
|
||||||
|
# Runs on every push and pull request, unlike the image scan, which needs something
|
||||||
|
# published to scan and so lives in release.yaml. Both are needed: govulncheck reads the
|
||||||
|
# source and its module graph, trivy reads the built artifact, and neither sees what the
|
||||||
|
# other does.
|
||||||
|
security:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
container:
|
||||||
|
image: golang:1.26.6-bookworm
|
||||||
|
volumes:
|
||||||
|
- go-mod-cache:/go/pkg/mod
|
||||||
|
- go-build-cache:/root/.cache/go-build
|
||||||
|
- gobin-cache:/go/bin
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
env:
|
||||||
|
REF_NAME: ${{ github.ref_name }}
|
||||||
|
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||||
|
run: |
|
||||||
|
if [ -n "$HEAD_SHA" ]; then
|
||||||
|
git clone "$REPO_URL" .
|
||||||
|
git checkout -q "$HEAD_SHA"
|
||||||
|
else
|
||||||
|
git clone --depth=1 --branch "$REF_NAME" "$REPO_URL" .
|
||||||
|
fi
|
||||||
|
|
||||||
|
- name: Go vulnerability scan (govulncheck)
|
||||||
|
run: make security-go
|
||||||
|
|
||||||
|
- name: Secret scan (gitleaks)
|
||||||
|
run: make security-secrets
|
||||||
|
|
||||||
# Host mode, no `container:`: helm is baked into the runner image, and a container job
|
# Host mode, no `container:`: helm is baked into the runner image, and a container job
|
||||||
# could not install it -- get.helm.sh is unreachable from the dind bridge. Same reason
|
# could not install it -- get.helm.sh is unreachable from the dind bridge. Same reason
|
||||||
# release.yaml's chart job runs on the host.
|
# release.yaml's chart job runs on the host.
|
||||||
|
|||||||
@@ -187,3 +187,37 @@ jobs:
|
|||||||
set -eu
|
set -eu
|
||||||
echo "$TOKEN" | helm registry login "$REGISTRY" -u niklas --password-stdin
|
echo "$TOKEN" | helm registry login "$REGISTRY" -u niklas --password-stdin
|
||||||
make helm-package helm-push VERSION="$REF_NAME"
|
make helm-package helm-push VERSION="$REF_NAME"
|
||||||
|
|
||||||
|
# Host mode, like image and chart: this needs a docker daemon to run trivy in, and a
|
||||||
|
# `container:` job would sit on the dind bridge with none.
|
||||||
|
#
|
||||||
|
# It scans the pushed image rather than a locally built one, because trivy cannot read a
|
||||||
|
# local image on this runner -- Talos has no docker socket and the dind sidecar shares no
|
||||||
|
# filesystem with the job -- so it pulls from the registry. That is also why this runs
|
||||||
|
# after `image` rather than gating it: a red scan does not unpublish anything.
|
||||||
|
#
|
||||||
|
# What a red scan means is therefore not "the release failed" but "do not bump the wrapper
|
||||||
|
# chart in Ryuvia/charts to this version". The image and chart are already published by
|
||||||
|
# the time this runs, and deliberately so -- this pipeline does not deploy.
|
||||||
|
#
|
||||||
|
# riksdata and rd-web have had this since they were set up; terdut-server went without any
|
||||||
|
# image scanning until 2026-09-02, so every release before v0.9.4 was published with no
|
||||||
|
# CVE check at all.
|
||||||
|
scan-image:
|
||||||
|
needs: image
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
env:
|
||||||
|
REF_NAME: ${{ github.ref_name }}
|
||||||
|
run: git clone --depth=1 --branch "$REF_NAME" "$REPO_URL" .
|
||||||
|
|
||||||
|
# Credentials are passed even though these packages are anonymously pullable -- that
|
||||||
|
# is a property of the personal namespace this publishes to, not something a release
|
||||||
|
# should depend on staying true.
|
||||||
|
- name: Scan the pushed image (trivy)
|
||||||
|
env:
|
||||||
|
TRIVY_USERNAME: niklas
|
||||||
|
TRIVY_PASSWORD: ${{ secrets.REGISTRY_TOKEN }}
|
||||||
|
REF_NAME: ${{ github.ref_name }}
|
||||||
|
run: make security-image VERSION="$REF_NAME"
|
||||||
|
|||||||
@@ -16,17 +16,28 @@ Config is `.release.conf` here plus `make release-vars`. The process itself live
|
|||||||
|
|
||||||
Two things about this repo specifically:
|
Two things about this repo specifically:
|
||||||
|
|
||||||
- **The pipeline has no image scan.** `.gitea/workflows/release.yaml` runs `test`,
|
- **The image is scanned after it is published, not before.** `scan-image` runs trivy
|
||||||
`binaries`, `image` and `chart`. A green release run is not evidence the image is
|
against the pushed image, because trivy cannot read a locally built one on this runner.
|
||||||
CVE-clean, and a release note must not imply it is.
|
A red scan therefore unpublishes nothing — it means: do not bump the wrapper chart in
|
||||||
|
`Ryuvia/charts` to this version. Added 2026-09-02; every release up to and including
|
||||||
|
v0.9.3 was published with no CVE check at all.
|
||||||
- **The wrapper chart has two `tag:` lines** — the app image and the python backup sidecar —
|
- **The wrapper chart has two `tag:` lines** — the app image and the python backup sidecar —
|
||||||
so `chart-bump` needs `--image "$IMAGE"` to know which one moves.
|
so `chart-bump` needs `--image "$IMAGE"` to know which one moves.
|
||||||
|
|
||||||
## Checks
|
## Checks
|
||||||
|
|
||||||
`make fmt lint test helm-lint` mirrors `.gitea/workflows/ci.yaml`, so a green gate here means
|
`make fmt lint test helm-lint` **is** what the pipeline runs — `ci.yaml` and `release.yaml`
|
||||||
a green pipeline there. The one deliberate difference is `-race`, which CI does not run; see
|
call these targets rather than restating them, the way riksdata and rd-web do. A green gate
|
||||||
the comment on the `test` target.
|
here and a green pipeline are the same code, not two descriptions of it. `test` adds `-race`,
|
||||||
|
which the workflows do not have to ask for since they call the target; see the comment on it
|
||||||
|
for why.
|
||||||
|
|
||||||
There are deliberately no `build`/`push`/`helm-push` targets — the workflow owns publishing,
|
`make release` (build + push the multi-arch image, package + push the chart) is what
|
||||||
and it builds multi-arch. Publishing happens by pushing a tag.
|
`release.yaml` invokes. Do not run it by hand — it refuses `VERSION=dev` for that reason, and
|
||||||
|
publishing happens by pushing a tag.
|
||||||
|
|
||||||
|
Three scans, and they see different things: `security-go` (govulncheck) reads the source and
|
||||||
|
its module graph and reports only vulnerabilities the code can actually reach;
|
||||||
|
`security-secrets` (gitleaks) reads the working tree, not the history, so it catches a secret
|
||||||
|
on the way in rather than auditing what is already committed; `security-image` (trivy) reads
|
||||||
|
the published artifact and therefore only runs on a tag. The first two gate every push.
|
||||||
|
|||||||
@@ -100,6 +100,10 @@ CHART_VERSION := $(shell echo "$(VERSION)" | sed 's/^v//')
|
|||||||
PLATFORMS ?= linux/amd64,linux/arm64
|
PLATFORMS ?= linux/amd64,linux/arm64
|
||||||
BUILDX_BUILDER ?= terdut
|
BUILDX_BUILDER ?= terdut
|
||||||
|
|
||||||
|
TRIVY_VERSION := 0.73.0
|
||||||
|
GOVULNCHECK_VERSION := v1.1.4
|
||||||
|
GITLEAKS_VERSION := v8.30.0
|
||||||
|
|
||||||
# --pull, not --no-cache: refresh the base image without discarding the layer cache.
|
# --pull, not --no-cache: refresh the base image without discarding the layer cache.
|
||||||
DOCKER_BUILD_FLAGS ?= --pull
|
DOCKER_BUILD_FLAGS ?= --pull
|
||||||
|
|
||||||
@@ -113,7 +117,7 @@ HELM_ISOLATED = HELM_REPOSITORY_CONFIG=$(CURDIR)/.helm-repos.yaml
|
|||||||
.PHONY: require-version
|
.PHONY: require-version
|
||||||
require-version:
|
require-version:
|
||||||
@test "$(VERSION)" != "dev" || \
|
@test "$(VERSION)" != "dev" || \
|
||||||
(echo "refusing to publish VERSION=dev — pass VERSION=vX.Y.Z (the workflow passes the tag)" && exit 1)
|
(echo "VERSION=dev names no release — pass VERSION=vX.Y.Z (the workflow passes the tag)" && exit 1)
|
||||||
|
|
||||||
.PHONY: build
|
.PHONY: build
|
||||||
build: ## Build the image for this host only, without pushing (local check / CI smoke)
|
build: ## Build the image for this host only, without pushing (local check / CI smoke)
|
||||||
@@ -166,3 +170,50 @@ binaries: require-version ## Cross-compile the release binaries into dist/
|
|||||||
|
|
||||||
.PHONY: release
|
.PHONY: release
|
||||||
release: push helm-package helm-push ## Publish image + chart (the workflow's one call)
|
release: push helm-package helm-push ## Publish image + chart (the workflow's one call)
|
||||||
|
|
||||||
|
## --- security ---
|
||||||
|
|
||||||
|
# Symbol-level, not dependency-level: govulncheck reports a vulnerability only when the
|
||||||
|
# code can actually reach it. As of 2026-09-02 this repo imports three chi advisories and
|
||||||
|
# reports none of them, because all three are middleware.RealIP and router.go uses Logger
|
||||||
|
# and Recoverer. That is the useful property rather than a loophole -- adding
|
||||||
|
# middleware.RealIP would turn this red, which is exactly when someone should look.
|
||||||
|
.PHONY: security-go
|
||||||
|
security-go: ## Scan Go deps for known CVEs (govulncheck)
|
||||||
|
go run golang.org/x/vuln/cmd/govulncheck@$(GOVULNCHECK_VERSION) ./...
|
||||||
|
|
||||||
|
# --no-git scans the working tree rather than the history, so this catches a secret on the
|
||||||
|
# way in. It is not a history audit and finding nothing here says nothing about what is
|
||||||
|
# already committed. --redact because the finding is printed into a CI log.
|
||||||
|
#
|
||||||
|
# Note when testing it that gitleaks allowlists well-known example credentials -- the AWS
|
||||||
|
# key from their own documentation does not trip it. A private key block does.
|
||||||
|
.PHONY: security-secrets
|
||||||
|
security-secrets: ## Scan the working tree for committed secrets (gitleaks)
|
||||||
|
go run github.com/zricethezav/gitleaks/v8@$(GITLEAKS_VERSION) detect --no-git \
|
||||||
|
--source . --redact --no-banner --exit-code 1
|
||||||
|
|
||||||
|
|
||||||
|
# Scans the pushed image, not a local one: trivy cannot read a locally built image on the
|
||||||
|
# runner -- Talos has no docker socket, and the dind sidecar shares no filesystem with the
|
||||||
|
# job -- so it pulls from the registry. Same reason riksdata and rd-web scan after pushing.
|
||||||
|
#
|
||||||
|
# It cannot gate a deploy, because this pipeline does not deploy. A red scan means: do not
|
||||||
|
# bump the wrapper chart in Ryuvia/charts to this version.
|
||||||
|
#
|
||||||
|
# The image is FROM scratch, so there are no OS packages to scan and trivy sees exactly one
|
||||||
|
# target -- the Go binary and its module graph. That also makes scanning a single platform
|
||||||
|
# sufficient here: linux/amd64 and linux/arm64 are the same modules built for a different
|
||||||
|
# GOARCH, so a CVE in one is a CVE in both. On an image with a base layer that would not
|
||||||
|
# hold and both platforms would need scanning.
|
||||||
|
#
|
||||||
|
# This is the last of the three scans and the only one that needs a published artifact;
|
||||||
|
# security-go and security-secrets above run on every push.
|
||||||
|
.PHONY: security-image
|
||||||
|
security-image: require-version ## Scan the pushed image for CVEs (needs VERSION)
|
||||||
|
@# The named volume persists trivy's vulnerability DB between runs; without it every
|
||||||
|
@# scan re-downloads the whole database.
|
||||||
|
docker run --rm -e TRIVY_USERNAME -e TRIVY_PASSWORD \
|
||||||
|
-v trivy-cache:/root/.cache/trivy \
|
||||||
|
docker.io/aquasec/trivy:$(TRIVY_VERSION) image --severity HIGH,CRITICAL \
|
||||||
|
--ignore-unfixed --exit-code 1 $(IMAGE):$(VERSION)
|
||||||
|
|||||||
@@ -671,10 +671,13 @@ a flaky incident in production rather than as a red build.
|
|||||||
## Releasing
|
## Releasing
|
||||||
|
|
||||||
```
|
```
|
||||||
push to main → ci.yaml gofmt, go vet, go test
|
push or PR → ci.yaml gofmt, go vet, go test -race
|
||||||
push tag vX.Y.Z → release.yaml same gate, then publish:
|
govulncheck, gitleaks
|
||||||
|
helm lint + render
|
||||||
|
push tag vX.Y.Z → release.yaml the same gate, then publish:
|
||||||
git.ryuvia.com/niklas/terdut-server:vX.Y.Z
|
git.ryuvia.com/niklas/terdut-server:vX.Y.Z
|
||||||
oci://git.ryuvia.com/niklas/terdut-server X.Y.Z
|
oci://git.ryuvia.com/niklas/terdut-server X.Y.Z
|
||||||
|
then trivy-scan the pushed image
|
||||||
PR to Ryuvia/charts → bump the wrapper chart to X.Y.Z; on merge
|
PR to Ryuvia/charts → bump the wrapper chart to X.Y.Z; on merge
|
||||||
Flux reconciles and the release rolls out
|
Flux reconciles and the release rolls out
|
||||||
```
|
```
|
||||||
@@ -695,15 +698,24 @@ different answers — chart 0.9.0 went out reading `appVersion: "latest"` that w
|
|||||||
publisher, triggered by the tag (`766f439`). The cost is that a chart-only change has no
|
publisher, triggered by the tag (`766f439`). The cost is that a chart-only change has no
|
||||||
version of its own and rides the next app tag.
|
version of its own and rides the next app tag.
|
||||||
|
|
||||||
There is deliberately **no** `make build` / `make push`. The workflow builds
|
Both workflows are thin drivers over the Makefile: `ci.yaml` runs `make fmt lint test` and
|
||||||
`linux/amd64,linux/arm64` through buildx; a local single-platform push would land on top of
|
`make helm-lint`, `release.yaml` adds `make binaries`, `make push`, `make helm-package` and
|
||||||
the multi-arch tag and stay invisible, because the tag would still resolve — just not on
|
`make helm-push`. That is deliberate — it is what makes a green local gate and a green
|
||||||
arm64. Publishing happens by pushing a tag.
|
pipeline the same code rather than two descriptions of it, and it is how riksdata and rd-web
|
||||||
|
have always worked.
|
||||||
|
|
||||||
|
`make push` builds and pushes in one step, unlike those two, because the image is
|
||||||
|
`linux/amd64,linux/arm64` and buildx cannot load a multi-platform result into the local image
|
||||||
|
store. `make build` stays single-platform and local-only. Both refuse `VERSION=dev`:
|
||||||
|
publishing is one command, so it is also one command to run by accident. Publishing happens
|
||||||
|
by pushing a tag.
|
||||||
|
|
||||||
Two things the release process needs to know about this repo:
|
Two things the release process needs to know about this repo:
|
||||||
|
|
||||||
- **The pipeline has no image scan**, unlike riksdata and rd-web. A green release run is not
|
- **The image scan runs after publishing**, like riksdata's and rd-web's: trivy cannot read
|
||||||
evidence the image is CVE-clean.
|
a locally built image on this runner, so it pulls the pushed one. A red `scan-image` means
|
||||||
|
do not bump the wrapper chart to that version — it cannot unpublish anything. The image is
|
||||||
|
`FROM scratch`, so trivy sees exactly one target, the Go binary and its module graph.
|
||||||
- **The wrapper chart's `values.yaml` has two `tag:` lines** — the app image and the python
|
- **The wrapper chart's `values.yaml` has two `tag:` lines** — the app image and the python
|
||||||
backup sidecar — so `chart-bump` is given `--image` to say which one moves.
|
backup sidecar — so `chart-bump` is given `--image` to say which one moves.
|
||||||
|
|
||||||
|
|||||||
@@ -15,5 +15,5 @@ type: application
|
|||||||
# appVersion and image.tag in values.yaml no longer agree, and that is not an oversight:
|
# appVersion and image.tag in values.yaml no longer agree, and that is not an oversight:
|
||||||
# image.tag stays "latest", which is what a local install actually pulls. appVersion is
|
# image.tag stays "latest", which is what a local install actually pulls. appVersion is
|
||||||
# metadata and drives nothing.
|
# metadata and drives nothing.
|
||||||
version: 0.9.3
|
version: 0.9.4
|
||||||
appVersion: "v0.9.3"
|
appVersion: "v0.9.4"
|
||||||
|
|||||||
@@ -23,13 +23,23 @@ spec:
|
|||||||
serviceAccountName: {{ include "terdut-server.fullname" . }}-bootstrap
|
serviceAccountName: {{ include "terdut-server.fullname" . }}-bootstrap
|
||||||
containers:
|
containers:
|
||||||
- name: bootstrap
|
- name: bootstrap
|
||||||
image: alpine:3
|
# alpine/curl, not alpine:3 + `apk add curl`. Installing the binary at run time
|
||||||
|
# writes it into the container's writable upper layer, which is exactly the
|
||||||
|
# signature Falco's `Drop and execute new binary in container` (MITRE TA0003)
|
||||||
|
# exists to catch -- this hook emitted two Critical events on every single
|
||||||
|
# upgrade. See Ryuvia/charts#100. It also made `helm upgrade` depend on the
|
||||||
|
# Alpine CDN answering, since this runs as a post-upgrade hook and a failed
|
||||||
|
# hook fails the release.
|
||||||
|
#
|
||||||
|
# Still a full Alpine underneath, so sh, cat, sleep, grep, cut, head and tail
|
||||||
|
# are all present (verified in-cluster 2026-09-04). The image declares
|
||||||
|
# ENTRYPOINT ["/entrypoint.sh"], which `command:` below overrides -- do not
|
||||||
|
# change `command:` to `args:`.
|
||||||
|
image: alpine/curl:8.21.0@sha256:a1c44bab54d88e18ea9a6a4ecefab7f2d230b968567b78960fcaff8d51b7f067
|
||||||
command:
|
command:
|
||||||
- /bin/sh
|
- /bin/sh
|
||||||
- -c
|
- -c
|
||||||
- |
|
- |
|
||||||
apk add --no-cache curl > /dev/null 2>&1
|
|
||||||
|
|
||||||
SERVICE_URL="http://{{ include "terdut-server.fullname" . }}:{{ .Values.service.port }}"
|
SERVICE_URL="http://{{ include "terdut-server.fullname" . }}:{{ .Values.service.port }}"
|
||||||
SECRET_NAME="{{ include "terdut-server.bootstrapSecretName" . }}"
|
SECRET_NAME="{{ include "terdut-server.bootstrapSecretName" . }}"
|
||||||
K8S_API="https://kubernetes.default.svc"
|
K8S_API="https://kubernetes.default.svc"
|
||||||
|
|||||||
Reference in New Issue
Block a user