--- id: REUSE-WP-0019 type: workplan title: "Forgejo-native federation automation and reuse telemetry" domain: infotech repo: reuse-surface status: active owner: claude-code topic_slug: helix-forge created: "2026-07-06" updated: "2026-07-07" state_hub_workstream_id: "569be717-34f8-4039-bb26-497685f60159" reuse_check: "new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry" --- # Forgejo-native federation automation and reuse telemetry Federation compose is on-demand and the hub serves whatever was last composed; roster sweeps are manual; reliability evidence is structural (CI exists) rather than observed (someone reused it and it worked). This workplan makes the registry **live** (event-driven recompose) and **evidence-backed** (reuse telemetry feeding the R axis). **Platform constraint:** the forge is transitioning **Gitea → Forgejo**. All new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) — nothing new is built against Gitea. Existing raw URLs (`https://gitea.coulomb.social/...`) and `.gitea/workflows/` must migrate or be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly host/path configuration, but every hardcoded `gitea.` reference is a liability. **Depends on:** REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01 telemetry schema (shared). Closes SCOPE "not possible yet" item *automatic hub refresh* and moves reliability evidence beyond structural. ## Design principles 1. **Host-agnostic first** — a single `forge_base_url` configuration (env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover becomes a one-line change per surface. 2. **Webhook triggers, compose stays pull-based** — the webhook only marks the hub's composed index stale and triggers recompose from published raw URLs; no push-parsing of payloads into registry state. 3. **Degrade to schedule** — if webhooks are unavailable, a scheduled Forgejo Actions job recomposes on an interval; freshness is monitored either way. 4. **Telemetry is append-only and low-ceremony** — reuse events are JSONL facts (who consumed what, when, outcome); aggregation derives `reused_by` relations and R-axis evidence citations, never hand-edited. ## Dependencies | Dependency | Owner | Notes | |---|---|---| | Forgejo instance + admin | Bernd / infra | webhook config, org-level token, Actions runners | | Gitea→Forgejo cutover plan | infra | final hostname, raw URL scheme, redirect window | | Hub deployment (reuse.coulomb.social) | reuse-surface / railiance | new endpoint + config rollout | | REUSE-WP-0018-T01 | reuse-surface | shared telemetry/outcome schema | | plan-check adoption | ecosystem | telemetry volume comes from WP-0018-T05 rollout | --- ## Forge Host Abstraction And URL Migration Inventory ```task id: REUSE-WP-0019-T01 status: done priority: high state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442" ``` **Inventory findings (2026-07-07):** cross-referenced `sources.yaml` (61 entries) against each repo's actual git `origin` remote. Result: 11 repos already migrated their remote to Forgejo; 50 still on Gitea (correctly, no action needed). Of the 11 Forgejo-origin repos, 9 already had correct `sources.yaml` entries (from WP-0017 drafting/registration); **2 were stale** — `activity-core` and `state-hub` — still pointing at their old Gitea raw URL despite having migrated. This was real, live debt, not a hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before being migrated for real, in both `sources.yaml` and the production hub registration (`hub update --url`). Verified post-migration against `GET /v1/federated` — both now show `forgejo.coulomb.social`. `config-atlas`'s WP-0017-T06 303 was **not** a host-transition symptom — already diagnosed there as (a) a redirect the current code already follows fine, and (b) a hub-registration gap unrelated to host. No new finding here. Implemented: - `reuse_surface/forge_host.py`: `parse_raw_url`/`derive_raw_url` (handles both the legacy `/raw//...` form and the canonical `/raw/branch//...` form both forges serve without a 303 redirect), `rewrite_url_host`, `forge_base_url` (reads `REUSE_SURFACE_FORGE_BASE_URL`, no hard default since migration is opt-in per repo), `migrate_source_host` (verifies the new URL resolves via HTTP HEAD **before** writing — refuses to point a repo at a host it hasn't actually migrated to) - CLI: `reuse-surface federation migrate-host --repo [--repo ...] --to [--from ] [--dry-run] [--no-verify] [--update-hub]` — used for real on `activity-core`/`state-hub` - Fixed two host-agnostic code gaps found while inventorying: `registry_update.py`'s `SAFE_EVIDENCE_PREFIXES` and `maintain_llm.py`'s git-diff pathspec only recognized `.gitea/workflows/`, missing repos already on `.forgejo/workflows/` — both now recognize either - Fixed stale copy-paste examples in `docs/RegistryFederation.md` (state-hub's old Gitea URL, now genuinely wrong post-migration) and a pre-existing, unrelated port typo (`8088` vs the real llm-connect default `8080`) in `tools/README.md`/`registry/README.md`, discovered and confirmed live during REUSE-WP-0018-T03 - 17 new pytest cases (`tests/test_forge_host.py`); 106 total pass - Recomposed `federated.yaml` post-migration: still 61 capabilities, no loss ## Hub Recompose Endpoint And Webhook Receiver ```task id: REUSE-WP-0019-T02 status: done priority: high state_hub_task_id: "691eb32a-6f20-4a2a-b9ff-0ae427b659aa" ``` **Note:** `POST /v1/federated/compose` (token-auth, triggers a real recompose) already existed from earlier hub work — no separate `/v1/recompose` route was added; the spec now documents this explicitly rather than duplicating a route that already does the job. Implemented in `reuse_surface/hub/`: - `store.py`: `compose_state` table (`composed_at`, `stale`), with `record_compose()`/`mark_stale()`/`get_compose_state()`. `composed_at` updates and `stale` clears only on a *forced* recompose (`refresh=true`, webhook, or future scheduled fallback) — a plain `GET` still serves current best-effort data but never silently reports itself as freshly composed - `webhooks.py`: `verify_signature` (constant-time HMAC-SHA256, fails closed on empty secret), `push_touches_registry_index` (path-only inspection of the push payload's `added`/`modified`/`removed` lists — never parses file content, per design principle 2) - `app.py`: `POST /v1/webhooks/forgejo` (accepts both `X-Forgejo-Signature` and `X-Gitea-Signature`, since repos migrate independently); `GET /v1/federated` and `POST /v1/federated/compose` now share an `asyncio.Lock` so concurrent recompose triggers (manual, webhook, future scheduled) coalesce instead of overlapping - `specs/FederationHubAPI.md` extended (§5.7-5.9, config table, error codes) - 28 new pytest cases (16 in `test_hub.py`, 12 in `test_webhooks.py`); 128 total pass - **Live-verified**: ran the actual hub service locally (`reuse-surface serve`), sent a real HMAC-signed webhook payload over HTTP — confirmed `composed_at`/`stale` transitions, signature rejection, irrelevant-path no-op, and the full webhook-to-recompose path end to end **Deployment boundary — deliberately not done:** this changes the hub *service source code* in this repo; it does **not** deploy to the live `reuse.coulomb.social` hub. That requires a container rebuild, registry push, and Kubernetes rollout — a production deployment action affecting a shared external service, out of scope for this task without explicit sign-off. See `docs/deploy/reuse-kubernetes.md`. ## Forgejo Webhook Rollout And Scheduled Fallback ```task id: REUSE-WP-0019-T03 status: wait priority: medium state_hub_task_id: "aa9e9f80-b878-490c-832e-515d8cbbbb60" ``` Blocked on T02 deploy and Forgejo instance availability. - Org-level Forgejo webhook (single config, all repos) → hub `/v1/webhooks/forgejo`, push events only - Fallback: Forgejo Actions scheduled workflow (cron) in this repo calling `POST /v1/recompose`; also serves repos during any Gitea-remnant window - Migrate this repo's CI `.gitea/workflows/ci.yml` → `.forgejo/workflows/ci.yml` (Forgejo Actions; verify runner labels); document the pattern for siblings - Verify end-to-end: index change in a sibling repo → hub `composed_at` advances without manual compose ## Reuse Telemetry Store And Recording ```task id: REUSE-WP-0019-T04 status: todo priority: medium state_hub_task_id: "c8e9064e-5c39-4c84-80e9-8b255f8edaec" ``` - Implement the shared schema from WP-0018-T01: reuse events `{ts, consumer_repo, capability_id, verdict, outcome?, source: plan-check|manual|hub}` - Hub: `POST /v1/reuse-events` (token-auth) + local JSONL fallback when hub unreachable; `GET /v1/reuse-events?capability_id=` for aggregation - `plan-check --record-outcome` (WP-0018) posts here; manual `reuse-surface record-reuse` for retroactive facts - Privacy/scope: repo slugs and capability ids only — no code, no secrets ## Telemetry Aggregation Into R-Axis Evidence ```task id: REUSE-WP-0019-T05 status: wait priority: medium state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7" ``` Blocked on T04 plus initial event volume. - `reuse-surface report reuse`: per-capability consumer counts, outcomes, last-used; feeds `reused_by` relation suggestions via the WP-0016 maintain/patch pipeline (evidence-gated, never silent promotion) - Maturity standard note: what observed-reuse evidence counts toward R2/R3+ (`specs/CapabilityMaturityStandard.md` amendment) - Catalog + graph surface consumer counts ## Freshness Monitoring, Docs, SCOPE ```task id: REUSE-WP-0019-T06 status: todo priority: low state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b" ``` - `reuse-surface stats`: hub `composed_at` age + stale flag; CI informational check warns when the hub index is older than N days - `docs/RegistryFederation.md` + `docs/deploy/reuse-kubernetes.md`: webhook setup, recompose endpoint, Forgejo token handling (route credentials per credential-routing rules — no secrets in repo) - `SCOPE.md`: flip "automatic hub refresh" to possible; update federation posture --- ## Acceptance - [ ] No hardcoded forge host in code or sources.yaml; `migrate-host` tested - [ ] Push to a sibling repo's `registry/indexes/` recomposes the hub index without manual action (webhook), with scheduled fallback in place - [ ] This repo's CI runs on Forgejo Actions (`.forgejo/workflows/`) - [ ] Reuse events recordable via hub API and CLI; `report reuse` aggregates them - [ ] R-axis evidence rules for observed reuse documented in the maturity standard - [ ] Hub freshness visible (`composed_at`, stale flag) in API and stats ## Out of scope - Operating the Forgejo instance or the Gitea→Forgejo data migration itself - Multi-replica/Postgres hub posture (separate managed-platform track) - Blocking CI gates on registry freshness in sibling repos - ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)