coulomb/railiance-forge
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implement Gitea Actions runner substrate
Some checks failed
Forge Runner Smoke / compatibility-smoke (push) Has been cancelled
Details

Browse Source
This commit is contained in:
Bernd Worsch
2026-06-08 00:31:06 +02:00
parent 3fb63c9a03
commit 19ee47fe82
15 changed files with 1032 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/ci-runner-actions-gitops-ownership.md
View File

@@ -1,6 +1,6 @@
# CI Runner, Actions, And GitOps Ownership
Last reviewed: 2026-06-05
Last reviewed: 2026-06-07
Status: contract v1. This document defines ownership and handoffs only; it
does not authorize a live runner deployment, credential change, GitOps controller
@@ -161,10 +161,25 @@ should provide:
- representative cluster dry-run evidence for cluster-access labels;
- a rollback or disable path for a bad runner registration.
## Current Implementation Surface
`docs/gitea-actions-runner-substrate.md` defines the first supported Gitea
Actions runner model, the initial haskelseed compatibility labels, the attended
registration path, and recovery steps. `make runner-status` provides the
read-only probe entry point, and `.gitea/workflows/forge-runner-smoke.yaml`
provides the first non-production proof workflow.
The first compatibility labels are `self-hosted` and `haskelseed`, matching the
inter-hub blocker. New workflow consumers should prefer semantic labels such as
`linux`, `container-build`, and `registry-publish` once smoke evidence is
recorded.
## Open Follow-Ups
- `docs/observability-operating-evidence.md` defines the runner health and
artifact evidence signals that consumers may cite.
- `docs/gitea-actions-runner-evidence.md` records the current runner inventory,
smoke evidence, and inter-hub unblock evidence.
- WP-0006-T09 should declare runner substrate, label contracts, and evidence
edges in Railiance Fabric.
- `RAILIANCE-WP-0005-T05` should document app-side dry-run behavior once forge

181
docs/gitea-actions-runner-evidence.md Normal file
View File

@@ -0,0 +1,181 @@
# Gitea Actions Runner Evidence
Last updated: 2026-06-07
Status: initial non-secret inventory for `FORGE-WP-0003`. Haskelseed now has a
running `act_runner` OpenRC service with labels matching the first compatibility
contract. Smoke proof is still pending.
## Workstream
- Workplan: `FORGE-WP-0003-actions-runner-substrate`
- State Hub workstream: `149a0316-64d1-4664-96d0-274577c32e63`
- Immediate consumer blocker: inter-hub `R7` waits on a runner matching
`self-hosted` and `haskelseed`.
## Local Probe Results
Collected from `/home/worsch/railiance-forge` on 2026-06-07.
| Probe | Result | Note |
| --- | --- | --- |
| `curl` | available at `/usr/bin/curl` | Used for public endpoint checks. |
| `ssh` | available at `/usr/bin/ssh` | Direct `haskelseed` alias timed out; ops-bridge path reaches `root@192.168.178.135`. |
| `docker` | available at `/usr/bin/docker` | Local presence only; runner host Docker still pending. |
| `skopeo` | not available | Registry tag inspection pending. |
| local `act_runner` | not available | Haskelseed has `/usr/local/bin/act_runner`; local workstation does not. |
| `kubectl`, `helm`, `sops` | not available in `make check-tools` | Separate operator-tool gap for deploy-capable targets. |
Public endpoint checks from this environment:
| Endpoint | Result | Interpretation |
| --- | --- | --- |
| `https://gitea.coulomb.social/` | HTTP `404` | Public root route is not a useful Gitea web health signal here. |
| `https://gitea.coulomb.social/api/v1/version` | HTTP `404` | Public API version route is not exposed through the current ingress path. |
| `https://gitea.coulomb.social/v2/` | HTTP `401` | OCI registry route responds with an auth challenge. |
| `https://gitea.coulomb.social/api/packages/coulomb/pypi/simple/` | HTTP `404` | Public package route reachable but package/simple root did not return an index. |
| `https://hub.coulomb.social/api/v2/hubs` | HTTP `404` | Confirms inter-hub production still lacks the expected API surface. |
Direct haskelseed alias probe:
```text
ssh -o BatchMode=yes -o ConnectTimeout=5 haskelseed hostname
```
Result:
```text
ssh: connect to host haskelseed port 22: Connection timed out
```
This does not prove the runner host is down; it proves this session does not
currently have the bare SSH alias path needed to inspect it.
Ops-bridge haskelseed path:
```bash
RUNNER_HOST=192.168.178.135 \
RUNNER_SSH_USER=root \
RUNNER_SSH_KEY=/home/worsch/.ssh/id_ops \
make runner-status
```
Observed on 2026-06-07:
| Field | Result |
| --- | --- |
| Hostname | `haskelseed.coulomb.social` |
| Kernel | `Linux 6.18.22-0-virt` on Alpine |
| `act_runner` | `/usr/local/bin/act_runner` |
| `act_runner --version` | `v0.6.1-1-g8e6b3be9` |
| `nix` | `/usr/local/bin/nix`, Determinate Nix `3.18.1`, Nix `2.33.4` |
| Init system | OpenRC (`/sbin/rc-service`) |
| `act_runner` OpenRC service | initially not present; installed and started on 2026-06-07 |
| `gitea-act-runner` OpenRC service | not present |
| live runner process | PID `5911` after activation |
| registration file | `/root/.runner`, mode `0644`, owner `root:root` |
| registration name | `haskelseed` |
| registration address | `http://92.205.130.254:32166` |
| registration labels before activation | `haskelseed:host`, `linux:host`, `x86_64:host` |
| registration labels after activation | `self-hosted:host`, `haskelseed:host`, `linux:host`, `linux_amd64:host`, `x86_64:host`, `container-build:host`, `registry-publish:host` |
| ephemeral | `false` |
| runner backup | `/root/.runner.bak-20260607225905` |
Activation evidence:
```text
rc-service act_runner restart
status: started
act_runner PID: 5911
runner declared successfully with labels:
self-hosted, haskelseed, linux, linux_amd64, x86_64, container-build, registry-publish
```
## Runner Inventory
Known from repo and State Hub:
- Before `FORGE-WP-0003`, this repo had runner ownership contracts but no
runner deployment files, status script, smoke workflow, or runner evidence
file.
- Inter-hub reported that commits intended to trigger deployment did not update
production and that its workflow targets `self-hosted` and `haskelseed`.
- A local registration file exists on haskelseed and `act_runner` is running as
an OpenRC service. Gitea runner admin access has not yet been used to confirm
the runner in the UI, but the daemon log reports successful declaration.
Pending attended checks:
```bash
make runner-status
RUNNER_HOST=192.168.178.135 \
RUNNER_SSH_USER=root \
RUNNER_SSH_KEY=/home/worsch/.ssh/id_ops \
make runner-status
ssh haskelseed 'hostname; command -v act_runner || true'
ssh haskelseed 'systemctl status act_runner --no-pager || systemctl status gitea-act-runner --no-pager || true'
ssh haskelseed 'journalctl -u act_runner -n 200 --no-pager || journalctl -u gitea-act-runner -n 200 --no-pager || true'
```
If Gitea runner admin access is available, verify the `coulomb` organization or
instance runner page for:
- runner name `railiance-haskelseed-build-01`;
- labels `self-hosted`, `haskelseed`, `linux`, `linux_amd64`,
`container-build`, and `registry-publish`;
- online status;
- last contact time;
- workflow ids for the smoke run and later inter-hub run.
## Registry Tag Evidence
Tag inspection remains pending because `skopeo` is unavailable in this
environment.
Run from an operator host with registry access:
```bash
for tag in 91037a4 ae9e497 fa96fb8 7cc3173 latest; do
skopeo inspect --tls-verify=false \
"docker://92.205.130.254:32166/coulomb/inter-hub:${tag}" \
--format "${tag} {{.Name}} {{.Digest}}"
done
```
Record only image names, tags, digests, and status. Do not record registry
tokens.
## Smoke Evidence Slot
No smoke job has passed yet. Current haskelseed registration labels now match
the proposed first compatibility contract.
Expected evidence after `.gitea/workflows/forge-runner-smoke.yaml` passes:
| Field | Value |
| --- | --- |
| Date | `TODO` |
| Runner name | `TODO` |
| Labels used | `TODO` |
| Workflow id/url | `TODO` |
| Repo commit | `TODO` |
| Docker availability | `TODO` |
| Cluster credential absent | `TODO` |
| Registry credential absent in smoke | `TODO` |
## Inter-Hub Unblock Slot
Do not rerun production push probes until the smoke workflow passes.
Expected evidence after inter-hub rerun:
| Field | Value |
| --- | --- |
| Inter-hub commit | `TODO` |
| Workflow id/url | `TODO` |
| Image tag/digest | `TODO` |
| Deployment result | `TODO` |
| `https://hub.coulomb.social/api/v2/hubs` | `TODO` |
| Remaining blocker | `TODO` |

245
docs/gitea-actions-runner-substrate.md Normal file
View File

@@ -0,0 +1,245 @@
# Gitea Actions Runner Substrate
Last reviewed: 2026-06-07
Status: implementation contract v1 and attended operating runbook. This
document does not contain runner registration tokens, package credentials,
kubeconfigs, SSH keys, or decrypted secret values.
## Purpose
Railiance needs a forge-owned Gitea Actions runner substrate so source and app
workflows can build, publish, and verify artifacts without each repo inventing
its own runner assumptions. The immediate blocker is inter-hub: its deployment
workflow currently targets `self-hosted` and `haskelseed`, but no forge-owned
runner deployment or health evidence existed after the first Gitea ownership
move.
This runbook implements the first supported runner model and keeps the
separation clear:
- `railiance-forge` owns runner placement, registration, labels, health, and
recovery.
- Source and app repos own workflow files and app-specific build/deploy logic.
- Cluster deploy authority is not bundled into the build runner unless a
separate human-reviewed approval says so.
Primary upstream reference: https://docs.gitea.com/usage/actions/act-runner
## First Supported Runner
Use one attended Gitea `act_runner` on haskelseed as the compatibility runner
for the inter-hub unblock, unless an operator explicitly chooses a separate
runner host before registration.
Initial contract:
| Field | Value |
| --- | --- |
| Runner name | `railiance-haskelseed-build-01` |
| Runner scope | `coulomb` organization runner, or repository runner if org scope is not approved |
| Initial host | `haskelseed` |
| Capacity | `1` |
| Runtime | Docker-backed or host-backed `act_runner`, confirmed during attended install |
| Cluster deploy authority | Not included in baseline runner approval |
| Registry publish authority | Allowed only through repo/workflow-scoped Gitea secrets |
The first runner exists to prove scheduling, build tooling, and registry publish
readiness. It must not silently become a cluster deployment runner. If a
workflow needs live cluster access, add a separate label and approval note such
as `cluster-deploy` or shift deployment to an app-owned release path.
## Label Contract
Register the first runner with compatibility labels for the inter-hub workflow
and semantic labels for future workflows:
```text
self-hosted:host,haskelseed:host,linux:host,linux_amd64:host,container-build:host,registry-publish:host
```
Rules:
- `self-hosted` and `haskelseed` are compatibility labels for the first unblock.
- New workflows should prefer semantic labels such as `linux`,
`container-build`, and `registry-publish`.
- `registry-publish` means the runner may execute jobs that receive
repo-scoped registry credentials from Gitea secrets.
- Do not add `cluster-read`, `cluster-dry-run`, `cluster-deploy`, or
`s5-release-check` until the cluster/platform credential path is reviewed.
- Keep capacity at `1` until haskelseed resource contention and workload safety
are understood.
Gitea runner labels are supplied at registration with `--labels` and, on modern
Gitea/act_runner versions, can also be adjusted in the generated runner config.
Use the generated config from the installed runner version as the source of
truth for exact YAML keys.
## Secret Boundaries
Runner registration and publish credentials are live secrets. They must be
created and handled outside Git.
Allowed in this repo:
- secret names;
- operator prompts;
- paths such as `/run/secrets/railiance/gitea-act-runner-registration-token`;
- non-secret evidence such as runner name, labels, service status, workflow id,
commit SHA, image tag, and digest.
Forbidden in this repo:
- registration token values;
- Gitea package or registry tokens;
- tokenized URLs;
- SSH private keys;
- kubeconfigs or OpenBao tokens;
- complete environment dumps from runner jobs.
## Attended Install
Run these steps on the selected runner host. Adjust only the installation path
and package manager details for the host; do not change the label contract
without updating this document and State Hub.
1. Prepare the host:
```bash
sudo install -d -o root -g root -m 0755 /etc/act_runner
sudo install -d -o act_runner -g act_runner -m 0750 /var/lib/act_runner
sudo install -d -o act_runner -g act_runner -m 0750 /var/cache/act_runner
```
2. Install `act_runner` using the current Gitea-supported binary or package
source for the host. Record the version, but do not commit downloaded
binaries into this repo.
3. Generate the runner config on the host:
```bash
sudo -u act_runner /usr/local/bin/act_runner generate-config \
| sudo tee /etc/act_runner/config.yaml >/dev/null
sudo chmod 0640 /etc/act_runner/config.yaml
sudo chown root:act_runner /etc/act_runner/config.yaml
```
4. Edit the generated config so capacity is `1` and labels match this document.
If the generated config for the installed version uses a different section
name, keep the generated structure and only change the corresponding values.
5. Place the runner registration token in the approved secret path, for example:
```text
/run/secrets/railiance/gitea-act-runner-registration-token
```
6. Register non-interactively without printing the token:
```bash
sudo -u act_runner sh -lc '
set -eu
token="$(cat /run/secrets/railiance/gitea-act-runner-registration-token)"
exec /usr/local/bin/act_runner --config /etc/act_runner/config.yaml \
register --no-interactive \
--instance https://gitea.coulomb.social/ \
--token "$token" \
--name railiance-haskelseed-build-01 \
--labels self-hosted:host,haskelseed:host,linux:host,linux_amd64:host,container-build:host,registry-publish:host
'
```
7. Install the matching service template, adjusted to the host paths if needed.
For systemd hosts, use `runner/act-runner.service.example`:
```bash
sudo cp runner/act-runner.service.example /etc/systemd/system/act_runner.service
sudo systemctl daemon-reload
sudo systemctl enable --now act_runner
```
For Alpine/OpenRC hosts such as haskelseed, use
`runner/act-runner.openrc.example`:
```bash
sudo install -m 0755 runner/act-runner.openrc.example /etc/init.d/act_runner
sudo rc-update add act_runner default
sudo rc-service act_runner start
```
If reusing the current haskelseed root registration at `/root/.runner`, use
`runner/act-runner-haskelseed.openrc.example` instead. This is less
portable than the dedicated `act_runner` user layout, but it matches the
existing registration state without printing or replacing the token.
8. Inspect without exposing secrets:
```bash
systemctl status act_runner --no-pager
journalctl -u act_runner -n 100 --no-pager
rc-service act_runner status
tail -n 100 /var/log/act_runner.log
```
9. From this repo, run:
```bash
make runner-status
# If using the current ops-bridge haskelseed path:
RUNNER_HOST=192.168.178.135 \
RUNNER_SSH_USER=root \
RUNNER_SSH_KEY=/home/worsch/.ssh/id_ops \
make runner-status
```
## Recovery
Stop or drain:
```bash
sudo systemctl stop act_runner
sudo systemctl disable act_runner
```
Replace a runner:
1. Stop the old service.
2. Revoke or rotate the old registration token in Gitea.
3. Move the old `/var/lib/act_runner/.runner` aside for evidence, not reuse.
4. Register the replacement with the same approved label contract.
5. Run the smoke workflow before re-enabling consumer workflows.
Disable a bad registration:
- Stop the service immediately.
- Remove or disable the runner in Gitea admin/org settings.
- Rotate any repo secrets that were available to failed jobs if exposure is
plausible.
- Record a State Hub note with runner name, labels, time window, and affected
workflows.
## Smoke Workflow
The repo includes `.gitea/workflows/forge-runner-smoke.yaml`. It is deliberately
small: it checks scheduling, basic host context, Docker availability if present,
and verifies that baseline cluster/registry secret environment variables are not
accidentally present.
Do not rerun inter-hub production deploys until this smoke workflow has passed
on the approved runner and the result is recorded in
`docs/gitea-actions-runner-evidence.md`.
## Current Blockers
- Direct non-interactive SSH to the bare `haskelseed` alias timed out from this
environment on 2026-06-07. The current ops-bridge path reaches haskelseed at
`root@192.168.178.135` with `/home/worsch/.ssh/id_ops`.
- Haskelseed has `act_runner v0.6.1-1-g8e6b3be9` installed and `/root/.runner`
registered as `haskelseed`. On 2026-06-07 the OpenRC service was installed
from `runner/act-runner-haskelseed.openrc.example`, labels were updated to the
first compatibility contract, and the runner daemon declared successfully.
- `skopeo` is not installed in this environment, so registry tag inspection must
run from a host with `skopeo` or use an approved equivalent.
- Local `make check-tools` still lacks `kubectl`, `helm`, and `sops`; those are
separate operator-tool prerequisites for deploy-capable Gitea work.

7
docs/observability-operating-evidence.md
View File

@@ -1,6 +1,6 @@
# Forge Observability And Operating Evidence
Last reviewed: 2026-06-05
Last reviewed: 2026-06-07
Status: contract v1. This document defines checks, evidence, and future
monitoring expectations. It does not authorize a live monitoring deployment,
@@ -68,6 +68,9 @@ Git SSH:
Actions and runners:
- Run `make runner-status` for the current read-only runner, public endpoint,
and inter-hub registry probes. The target degrades when optional tools such as
`skopeo` or `act_runner` are unavailable.
- Record runner inventory by semantic label, trust level, and last successful
sample job.
- For privileged labels such as `package-publish`, `registry-publish`,
@@ -75,6 +78,8 @@ Actions and runners:
sample job or release job reference.
- If no runner currently provides a required label, mark the dependent workflow
as blocked on runner prerequisites instead of weakening the workflow.
- The current runner evidence log lives in
`docs/gitea-actions-runner-evidence.md`.
## Storage Growth Checks