reuse-surface/workplans/REUSE-WP-0019-forgejo-automation-and-telemetry.md

id, type, title, domain, repo, status, owner, topic_slug, created, updated, state_hub_workstream_id, reuse_check

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id	reuse_check
REUSE-WP-0019	workplan	Forgejo-native federation automation and reuse telemetry	infotech	reuse-surface	active	claude-code	helix-forge	2026-07-06	2026-07-07	569be717-34f8-4039-bb26-497685f60159	new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry

Forgejo-native federation automation and reuse telemetry

Federation compose is on-demand and the hub serves whatever was last composed; roster sweeps are manual; reliability evidence is structural (CI exists) rather than observed (someone reused it and it worked). This workplan makes the registry live (event-driven recompose) and evidence-backed (reuse telemetry feeding the R axis).

Platform constraint: the forge is transitioning Gitea → Forgejo. All new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) — nothing new is built against Gitea. Existing raw URLs (https://gitea.coulomb.social/...) and .gitea/workflows/ must migrate or be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly host/path configuration, but every hardcoded gitea. reference is a liability.

Depends on: REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01 telemetry schema (shared). Closes SCOPE "not possible yet" item automatic hub refresh and moves reliability evidence beyond structural.

Design principles

1. Host-agnostic first — a single forge_base_url configuration (env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover becomes a one-line change per surface.
2. Webhook triggers, compose stays pull-based — the webhook only marks the hub's composed index stale and triggers recompose from published raw URLs; no push-parsing of payloads into registry state.
3. Degrade to schedule — if webhooks are unavailable, a scheduled Forgejo Actions job recomposes on an interval; freshness is monitored either way.
4. Telemetry is append-only and low-ceremony — reuse events are JSONL facts (who consumed what, when, outcome); aggregation derives reused_by relations and R-axis evidence citations, never hand-edited.

Dependencies

Dependency	Owner	Notes
Forgejo instance + admin	Bernd / infra	webhook config, org-level token, Actions runners
Gitea→Forgejo cutover plan	infra	final hostname, raw URL scheme, redirect window
Hub deployment (reuse.coulomb.social)	reuse-surface / railiance	new endpoint + config rollout
REUSE-WP-0018-T01	reuse-surface	shared telemetry/outcome schema
plan-check adoption	ecosystem	telemetry volume comes from WP-0018-T05 rollout

---

Forge Host Abstraction And URL Migration Inventory

id: REUSE-WP-0019-T01
status: done
priority: high
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"

Inventory findings (2026-07-07): cross-referenced sources.yaml (61 entries) against each repo's actual git origin remote. Result: 11 repos already migrated their remote to Forgejo; 50 still on Gitea (correctly, no action needed). Of the 11 Forgejo-origin repos, 9 already had correct sources.yaml entries (from WP-0017 drafting/registration); 2 were stale — activity-core and state-hub — still pointing at their old Gitea raw URL despite having migrated. This was real, live debt, not a hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before being migrated for real, in both sources.yaml and the production hub registration (hub update --url). Verified post-migration against GET /v1/federated — both now show forgejo.coulomb.social.

config-atlas's WP-0017-T06 303 was not a host-transition symptom — already diagnosed there as (a) a redirect the current code already follows fine, and (b) a hub-registration gap unrelated to host. No new finding here.

Implemented:

• reuse_surface/forge_host.py: parse_raw_url/derive_raw_url (handles both the legacy /raw/<branch>/... form and the canonical /raw/branch/<branch>/... form both forges serve without a 303 redirect), rewrite_url_host, forge_base_url (reads REUSE_SURFACE_FORGE_BASE_URL, no hard default since migration is opt-in per repo), migrate_source_host (verifies the new URL resolves via HTTP HEAD before writing — refuses to point a repo at a host it hasn't actually migrated to)
• CLI: reuse-surface federation migrate-host --repo <slug> [--repo ...] --to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify] [--update-hub] — used for real on activity-core/state-hub
• Fixed two host-agnostic code gaps found while inventorying: registry_update.py's SAFE_EVIDENCE_PREFIXES and maintain_llm.py's git-diff pathspec only recognized .gitea/workflows/, missing repos already on .forgejo/workflows/ — both now recognize either
• Fixed stale copy-paste examples in docs/RegistryFederation.md (state-hub's old Gitea URL, now genuinely wrong post-migration) and a pre-existing, unrelated port typo (8088 vs the real llm-connect default 8080) in tools/README.md/registry/README.md, discovered and confirmed live during REUSE-WP-0018-T03
• 17 new pytest cases (tests/test_forge_host.py); 106 total pass
• Recomposed federated.yaml post-migration: still 61 capabilities, no loss

Hub Recompose Endpoint And Webhook Receiver

id: REUSE-WP-0019-T02
status: done
priority: high
state_hub_task_id: "691eb32a-6f20-4a2a-b9ff-0ae427b659aa"

Note: POST /v1/federated/compose (token-auth, triggers a real recompose) already existed from earlier hub work — no separate /v1/recompose route was added; the spec now documents this explicitly rather than duplicating a route that already does the job.

Implemented in reuse_surface/hub/:

• store.py: compose_state table (composed_at, stale), with record_compose()/mark_stale()/get_compose_state(). composed_at updates and stale clears only on a forced recompose (refresh=true, webhook, or future scheduled fallback) — a plain GET still serves current best-effort data but never silently reports itself as freshly composed
• webhooks.py: verify_signature (constant-time HMAC-SHA256, fails closed on empty secret), push_touches_registry_index (path-only inspection of the push payload's added/modified/removed lists — never parses file content, per design principle 2)
• app.py: POST /v1/webhooks/forgejo (accepts both X-Forgejo-Signature and X-Gitea-Signature, since repos migrate independently); GET /v1/federated and POST /v1/federated/compose now share an asyncio.Lock so concurrent recompose triggers (manual, webhook, future scheduled) coalesce instead of overlapping
• specs/FederationHubAPI.md extended (§5.7-5.9, config table, error codes)
• 28 new pytest cases (16 in test_hub.py, 12 in test_webhooks.py); 128 total pass
• Live-verified: ran the actual hub service locally (reuse-surface serve), sent a real HMAC-signed webhook payload over HTTP — confirmed composed_at/stale transitions, signature rejection, irrelevant-path no-op, and the full webhook-to-recompose path end to end

Deployed 2026-07-07 (explicit user sign-off "Deploy to live please"): built and pushed gitea.coulomb.social/coulomb/reuse-surface:e3ae22e, smoke-tested it locally in a standalone container first, then helm upgrade via railiance-apps (make reuse-deploy, pinned in helm/reuse-surface-values.yaml, commits a2c0da1/bcb05f5). Live-verified against https://reuse.coulomb.social: /v1/federated (200, 61 capabilities, composed_at/stale fields present), /v1/repos (200), a real forced recompose via POST /v1/federated/compose (token-auth via warden access --exec, token never printed), and the webhook endpoint correctly failing closed with 503 (REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET not yet configured).

Webhook secret added 2026-07-07 (explicit user sign-off "Go ahead and add the webhook secret"): generated a fresh 32-byte hex secret with openssl rand -hex 32, patched it into the live K8s Secret reuse-surface-env in namespace reuse, restarted the deployment (values injected via envFrom.secretRef, not picked up without a restart) — pod rolled cleanly (1/1 Ready, 0 restarts). The raw secret value was never printed to the transcript: generated and applied in one non-echoing shell step, and re-fetched only inside an exported env var for the live signature-verification test below, then unset. Live-verified against production: a correctly HMAC-signed push payload is now accepted ({"accepted": false, "reason": "no registry/indexes/ change"} for a payload that doesn't touch registry/indexes/, as designed), and a bad signature is still rejected with 401. /v1/federated and /v1/repos unaffected by the restart.

Found (not fixed) while smoke-testing: the public ingress's exact-path /health rule 404s at the Traefik edge (shadowed by the catch-all / rule to the landing page) — confirmed ingress-layer only, not a pod/service problem (/health works via direct port-forward; the Deployment's own readiness/liveness probes pass, pod is 1/1 Ready). /v1/* unaffected. Flagged to railiance-apps via State Hub message and documented in railiance-apps commit bcb05f5 — not fixed here since it's a shared production ingress template edit outside this workplan's scope.

Forgejo Webhook Rollout And Scheduled Fallback

id: REUSE-WP-0019-T03
status: wait
priority: medium
state_hub_task_id: "aa9e9f80-b878-490c-832e-515d8cbbbb60"

Blocked on T02 deploy and Forgejo instance availability.

• Org-level Forgejo webhook (single config, all repos) → hub /v1/webhooks/forgejo, push events only
• Fallback: Forgejo Actions scheduled workflow (cron) in this repo calling POST /v1/recompose; also serves repos during any Gitea-remnant window
• Migrate this repo's CI .gitea/workflows/ci.yml → .forgejo/workflows/ci.yml (Forgejo Actions; verify runner labels); document the pattern for siblings
• Verify end-to-end: index change in a sibling repo → hub composed_at advances without manual compose

Reuse Telemetry Store And Recording

id: REUSE-WP-0019-T04
status: todo
priority: medium
state_hub_task_id: "c8e9064e-5c39-4c84-80e9-8b255f8edaec"

• Implement the shared schema from WP-0018-T01: reuse events {ts, consumer_repo, capability_id, verdict, outcome?, source: plan-check|manual|hub}
• Hub: POST /v1/reuse-events (token-auth) + local JSONL fallback when hub unreachable; GET /v1/reuse-events?capability_id= for aggregation
• plan-check --record-outcome (WP-0018) posts here; manual reuse-surface record-reuse for retroactive facts
• Privacy/scope: repo slugs and capability ids only — no code, no secrets

Telemetry Aggregation Into R-Axis Evidence

id: REUSE-WP-0019-T05
status: wait
priority: medium
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"

Blocked on T04 plus initial event volume.

• reuse-surface report reuse: per-capability consumer counts, outcomes, last-used; feeds reused_by relation suggestions via the WP-0016 maintain/patch pipeline (evidence-gated, never silent promotion)
• Maturity standard note: what observed-reuse evidence counts toward R2/R3+ (specs/CapabilityMaturityStandard.md amendment)
• Catalog + graph surface consumer counts

Freshness Monitoring, Docs, SCOPE

id: REUSE-WP-0019-T06
status: todo
priority: low
state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"

• reuse-surface stats: hub composed_at age + stale flag; CI informational check warns when the hub index is older than N days
• docs/RegistryFederation.md + docs/deploy/reuse-kubernetes.md: webhook setup, recompose endpoint, Forgejo token handling (route credentials per credential-routing rules — no secrets in repo)
• SCOPE.md: flip "automatic hub refresh" to possible; update federation posture

---

Acceptance

• No hardcoded forge host in code or sources.yaml; migrate-host tested
• Push to a sibling repo's registry/indexes/ recomposes the hub index without manual action (webhook), with scheduled fallback in place
• This repo's CI runs on Forgejo Actions (.forgejo/workflows/)
• Reuse events recordable via hub API and CLI; report reuse aggregates them
• R-axis evidence rules for observed reuse documented in the maturity standard
• Hub freshness visible (composed_at, stale flag) in API and stats

Out of scope

• Operating the Forgejo instance or the Gitea→Forgejo data migration itself
• Multi-replica/Postgres hub posture (separate managed-platform track)
• Blocking CI gates on registry freshness in sibling repos
• ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)