reuse-surface/workplans/REUSE-WP-0019-forgejo-automation-and-telemetry.md

---
id: REUSE-WP-0019
type: workplan
title: "Forgejo-native federation automation and reuse telemetry"
domain: infotech
repo: reuse-surface
status: active
owner: claude-code
topic_slug: helix-forge
created: "2026-07-06"
updated: "2026-07-07"
state_hub_workstream_id: "569be717-34f8-4039-bb26-497685f60159"
reuse_check: "new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry"
---

# Forgejo-native federation automation and reuse telemetry

Federation compose is on-demand and the hub serves whatever was last composed;
roster sweeps are manual; reliability evidence is structural (CI exists) rather
than observed (someone reused it and it worked). This workplan makes the
registry **live** (event-driven recompose) and **evidence-backed** (reuse
telemetry feeding the R axis).

**Platform constraint:** the forge is transitioning **Gitea → Forgejo**. All
new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) —
nothing new is built against Gitea. Existing raw URLs
(`https://gitea.coulomb.social/...`) and `.gitea/workflows/` must migrate or
be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly
host/path configuration, but every hardcoded `gitea.` reference is a liability.

**Depends on:** REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01
telemetry schema (shared). Closes SCOPE "not possible yet" item *automatic hub
refresh* and moves reliability evidence beyond structural.

## Design principles

1. **Host-agnostic first** — a single `forge_base_url` configuration
   (env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover
   becomes a one-line change per surface.
2. **Webhook triggers, compose stays pull-based** — the webhook only marks the
   hub's composed index stale and triggers recompose from published raw URLs;
   no push-parsing of payloads into registry state.
3. **Degrade to schedule** — if webhooks are unavailable, a scheduled Forgejo
   Actions job recomposes on an interval; freshness is monitored either way.
4. **Telemetry is append-only and low-ceremony** — reuse events are JSONL
   facts (who consumed what, when, outcome); aggregation derives `reused_by`
   relations and R-axis evidence citations, never hand-edited.

## Dependencies

| Dependency | Owner | Notes |
|---|---|---|
| Forgejo instance + admin | Bernd / infra | webhook config, org-level token, Actions runners |
| Gitea→Forgejo cutover plan | infra | final hostname, raw URL scheme, redirect window |
| Hub deployment (reuse.coulomb.social) | reuse-surface / railiance | new endpoint + config rollout |
| REUSE-WP-0018-T01 | reuse-surface | shared telemetry/outcome schema |
| plan-check adoption | ecosystem | telemetry volume comes from WP-0018-T05 rollout |

---

## Forge Host Abstraction And URL Migration Inventory

```task
id: REUSE-WP-0019-T01
status: done
priority: high
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"
```

**Inventory findings (2026-07-07):** cross-referenced `sources.yaml` (61
entries) against each repo's actual git `origin` remote. Result: 11 repos
already migrated their remote to Forgejo; 50 still on Gitea (correctly, no
action needed). Of the 11 Forgejo-origin repos, 9 already had correct
`sources.yaml` entries (from WP-0017 drafting/registration); **2 were
stale** — `activity-core` and `state-hub` — still pointing at their old
Gitea raw URL despite having migrated. This was real, live debt, not a
hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before
being migrated for real, in both `sources.yaml` and the production hub
registration (`hub update --url`). Verified post-migration against
`GET /v1/federated` — both now show `forgejo.coulomb.social`.

`config-atlas`'s WP-0017-T06 303 was **not** a host-transition symptom —
already diagnosed there as (a) a redirect the current code already follows
fine, and (b) a hub-registration gap unrelated to host. No new finding here.

Implemented:

- `reuse_surface/forge_host.py`: `parse_raw_url`/`derive_raw_url` (handles
  both the legacy `/raw/<branch>/...` form and the canonical
  `/raw/branch/<branch>/...` form both forges serve without a 303
  redirect), `rewrite_url_host`, `forge_base_url` (reads
  `REUSE_SURFACE_FORGE_BASE_URL`, no hard default since migration is
  opt-in per repo), `migrate_source_host` (verifies the new URL resolves
  via HTTP HEAD **before** writing — refuses to point a repo at a host it
  hasn't actually migrated to)
- CLI: `reuse-surface federation migrate-host --repo <slug> [--repo ...]
  --to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify]
  [--update-hub]` — used for real on `activity-core`/`state-hub`
- Fixed two host-agnostic code gaps found while inventorying:
  `registry_update.py`'s `SAFE_EVIDENCE_PREFIXES` and `maintain_llm.py`'s
  git-diff pathspec only recognized `.gitea/workflows/`, missing repos
  already on `.forgejo/workflows/` — both now recognize either
- Fixed stale copy-paste examples in `docs/RegistryFederation.md`
  (state-hub's old Gitea URL, now genuinely wrong post-migration) and a
  pre-existing, unrelated port typo (`8088` vs the real llm-connect
  default `8080`) in `tools/README.md`/`registry/README.md`, discovered
  and confirmed live during REUSE-WP-0018-T03
- 17 new pytest cases (`tests/test_forge_host.py`); 106 total pass
- Recomposed `federated.yaml` post-migration: still 61 capabilities, no
  loss

## Hub Recompose Endpoint And Webhook Receiver

```task
id: REUSE-WP-0019-T02
status: done
priority: high
state_hub_task_id: "691eb32a-6f20-4a2a-b9ff-0ae427b659aa"
```

**Note:** `POST /v1/federated/compose` (token-auth, triggers a real
recompose) already existed from earlier hub work — no separate
`/v1/recompose` route was added; the spec now documents this explicitly
rather than duplicating a route that already does the job.

Implemented in `reuse_surface/hub/`:

- `store.py`: `compose_state` table (`composed_at`, `stale`), with
  `record_compose()`/`mark_stale()`/`get_compose_state()`. `composed_at`
  updates and `stale` clears only on a *forced* recompose (`refresh=true`,
  webhook, or future scheduled fallback) — a plain `GET` still serves
  current best-effort data but never silently reports itself as freshly
  composed
- `webhooks.py`: `verify_signature` (constant-time HMAC-SHA256, fails
  closed on empty secret), `push_touches_registry_index` (path-only
  inspection of the push payload's `added`/`modified`/`removed` lists —
  never parses file content, per design principle 2)
- `app.py`: `POST /v1/webhooks/forgejo` (accepts both `X-Forgejo-Signature`
  and `X-Gitea-Signature`, since repos migrate independently);
  `GET /v1/federated` and `POST /v1/federated/compose` now share an
  `asyncio.Lock` so concurrent recompose triggers (manual, webhook, future
  scheduled) coalesce instead of overlapping
- `specs/FederationHubAPI.md` extended (§5.7-5.9, config table, error
  codes)
- 28 new pytest cases (16 in `test_hub.py`, 12 in `test_webhooks.py`); 128
  total pass
- **Live-verified**: ran the actual hub service locally
  (`reuse-surface serve`), sent a real HMAC-signed webhook payload over
  HTTP — confirmed `composed_at`/`stale` transitions, signature rejection,
  irrelevant-path no-op, and the full webhook-to-recompose path end to end

**Deployed 2026-07-07 (explicit user sign-off "Deploy to live please"):**
built and pushed `gitea.coulomb.social/coulomb/reuse-surface:e3ae22e`,
smoke-tested it locally in a standalone container first, then
`helm upgrade` via `railiance-apps` (`make reuse-deploy`, pinned in
`helm/reuse-surface-values.yaml`, commits `a2c0da1`/`bcb05f5`). Live-verified
against `https://reuse.coulomb.social`: `/v1/federated` (200, 61
capabilities, `composed_at`/`stale` fields present), `/v1/repos` (200), a
real forced recompose via `POST /v1/federated/compose` (token-auth via
`warden access --exec`, token never printed), and the webhook endpoint
correctly failing closed with 503 (`REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET`
not yet configured).

**Webhook secret added 2026-07-07 (explicit user sign-off "Go ahead and
add the webhook secret"):** generated a fresh 32-byte hex secret with
`openssl rand -hex 32`, patched it into the live K8s Secret
`reuse-surface-env` in namespace `reuse`, restarted the deployment (values
injected via `envFrom.secretRef`, not picked up without a restart) — pod
rolled cleanly (`1/1 Ready`, 0 restarts). The raw secret value was never
printed to the transcript: generated and applied in one non-echoing shell
step, and re-fetched only inside an exported env var for the live
signature-verification test below, then unset. Live-verified against
production: a correctly HMAC-signed push payload is now accepted
(`{"accepted": false, "reason": "no registry/indexes/ change"}` for a
payload that doesn't touch `registry/indexes/`, as designed), and a bad
signature is still rejected with 401. `/v1/federated` and `/v1/repos`
unaffected by the restart.

**Found (not fixed) while smoke-testing:** the public ingress's exact-path
`/health` rule 404s at the Traefik edge (shadowed by the catch-all `/`
rule to the landing page) — confirmed ingress-layer only, not a pod/service
problem (`/health` works via direct port-forward; the Deployment's own
readiness/liveness probes pass, pod is `1/1 Ready`). `/v1/*` unaffected.
Flagged to `railiance-apps` via State Hub message and documented in
`railiance-apps` commit `bcb05f5` — not fixed here since it's a shared
production ingress template edit outside this workplan's scope.

## Forgejo Webhook Rollout And Scheduled Fallback

```task
id: REUSE-WP-0019-T03
status: wait
priority: medium
state_hub_task_id: "aa9e9f80-b878-490c-832e-515d8cbbbb60"
```

Blocked on T02 deploy and Forgejo instance availability.

- Org-level Forgejo webhook (single config, all repos) → hub
  `/v1/webhooks/forgejo`, push events only
- Fallback: Forgejo Actions scheduled workflow (cron) in this repo calling
  `POST /v1/recompose`; also serves repos during any Gitea-remnant window
- Migrate this repo's CI `.gitea/workflows/ci.yml` → `.forgejo/workflows/ci.yml`
  (Forgejo Actions; verify runner labels); document the pattern for siblings
- Verify end-to-end: index change in a sibling repo → hub `composed_at`
  advances without manual compose

## Reuse Telemetry Store And Recording

```task
id: REUSE-WP-0019-T04
status: todo
priority: medium
state_hub_task_id: "c8e9064e-5c39-4c84-80e9-8b255f8edaec"
```

- Implement the shared schema from WP-0018-T01: reuse events
  `{ts, consumer_repo, capability_id, verdict, outcome?, source: plan-check|manual|hub}`
- Hub: `POST /v1/reuse-events` (token-auth) + local JSONL fallback when hub
  unreachable; `GET /v1/reuse-events?capability_id=` for aggregation
- `plan-check --record-outcome` (WP-0018) posts here; manual
  `reuse-surface record-reuse` for retroactive facts
- Privacy/scope: repo slugs and capability ids only — no code, no secrets

## Telemetry Aggregation Into R-Axis Evidence

```task
id: REUSE-WP-0019-T05
status: wait
priority: medium
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"
```

Blocked on T04 plus initial event volume.

- `reuse-surface report reuse`: per-capability consumer counts, outcomes,
  last-used; feeds `reused_by` relation suggestions via the WP-0016
  maintain/patch pipeline (evidence-gated, never silent promotion)
- Maturity standard note: what observed-reuse evidence counts toward R2/R3+
  (`specs/CapabilityMaturityStandard.md` amendment)
- Catalog + graph surface consumer counts

## Freshness Monitoring, Docs, SCOPE

```task
id: REUSE-WP-0019-T06
status: todo
priority: low
state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"
```

- `reuse-surface stats`: hub `composed_at` age + stale flag; CI informational
  check warns when the hub index is older than N days
- `docs/RegistryFederation.md` + `docs/deploy/reuse-kubernetes.md`: webhook
  setup, recompose endpoint, Forgejo token handling (route credentials per
  credential-routing rules — no secrets in repo)
- `SCOPE.md`: flip "automatic hub refresh" to possible; update federation
  posture

---

## Acceptance

- [ ] No hardcoded forge host in code or sources.yaml; `migrate-host` tested
- [ ] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place
- [ ] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`)
- [ ] Reuse events recordable via hub API and CLI; `report reuse` aggregates them
- [ ] R-axis evidence rules for observed reuse documented in the maturity standard
- [ ] Hub freshness visible (`composed_at`, stale flag) in API and stats

## Out of scope

- Operating the Forgejo instance or the Gitea→Forgejo data migration itself
- Multi-replica/Postgres hub posture (separate managed-platform track)
- Blocking CI gates on registry freshness in sibling repos
- ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)