Files
reuse-surface/workplans/REUSE-WP-0019-forgejo-automation-and-telemetry.md
tegwick 00b7eab154
Some checks failed
ci / validate-registry (push) Has been cancelled
REUSE-WP-0019-T01: forge host abstraction + URL migration inventory
reuse_surface/forge_host.py: parse/derive/rewrite raw index URLs across
Gitea and Forgejo (handles both the legacy /raw/<branch>/... form and the
canonical /raw/branch/<branch>/... form both forges serve without a 303
redirect). migrate_source_host() verifies the new URL resolves via HTTP
HEAD before writing -- refuses to point a repo at a host it hasn't
actually migrated to.

New CLI: reuse-surface federation migrate-host --repo <slug> --to
<base-url> [--from <check>] [--dry-run] [--no-verify] [--update-hub].

Inventory: cross-referenced sources.yaml against each repo's actual git
origin. 11/61 repos already on Forgejo; found 2 with stale sources.yaml
entries (activity-core, state-hub) despite having migrated. Fixed for
real: local sources.yaml + production hub registration (hub update),
verified against GET /v1/federated post-migration. config-atlas's
WP-0017-T06 303 confirmed NOT a host-transition symptom (already diagnosed
there as something else).

Also fixed two host-agnostic gaps found while inventorying:
registry_update.py and maintain_llm.py only recognized .gitea/workflows/,
missing repos already on .forgejo/workflows/. Fixed a stale copy-paste
example (state-hub's now-wrong old URL) in docs/RegistryFederation.md, and
a pre-existing unrelated port typo (8088 vs the real llm-connect default
8080) in tools/README.md and registry/README.md.

17 new pytest cases (tests/test_forge_host.py), 106 total pass.
Recomposed federated.yaml post-migration: still 61 capabilities.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-07 18:14:30 +02:00

9.5 KiB

id, type, title, domain, repo, status, owner, topic_slug, created, updated, state_hub_workstream_id, reuse_check
id type title domain repo status owner topic_slug created updated state_hub_workstream_id reuse_check
REUSE-WP-0019 workplan Forgejo-native federation automation and reuse telemetry infotech reuse-surface active claude-code helix-forge 2026-07-06 2026-07-07 569be717-34f8-4039-bb26-497685f60159 new — dogfooded 2026-07-07 via reuse-surface plan-check against the full 61-capability federated index; no existing capability covers Forgejo webhook automation or reuse telemetry

Forgejo-native federation automation and reuse telemetry

Federation compose is on-demand and the hub serves whatever was last composed; roster sweeps are manual; reliability evidence is structural (CI exists) rather than observed (someone reused it and it worked). This workplan makes the registry live (event-driven recompose) and evidence-backed (reuse telemetry feeding the R axis).

Platform constraint: the forge is transitioning Gitea → Forgejo. All new automation attaches to Forgejo (webhooks, Forgejo Actions, API tokens) — nothing new is built against Gitea. Existing raw URLs (https://gitea.coulomb.social/...) and .gitea/workflows/ must migrate or be made host-agnostic. Forgejo is Gitea-API-compatible, so migration is mostly host/path configuration, but every hardcoded gitea. reference is a liability.

Depends on: REUSE-WP-0017 (content worth refreshing), REUSE-WP-0018-T01 telemetry schema (shared). Closes SCOPE "not possible yet" item automatic hub refresh and moves reliability evidence beyond structural.

Design principles

  1. Host-agnostic first — a single forge_base_url configuration (env/config + hub setting) replaces hardcoded hosts; the Forgejo cutover becomes a one-line change per surface.
  2. Webhook triggers, compose stays pull-based — the webhook only marks the hub's composed index stale and triggers recompose from published raw URLs; no push-parsing of payloads into registry state.
  3. Degrade to schedule — if webhooks are unavailable, a scheduled Forgejo Actions job recomposes on an interval; freshness is monitored either way.
  4. Telemetry is append-only and low-ceremony — reuse events are JSONL facts (who consumed what, when, outcome); aggregation derives reused_by relations and R-axis evidence citations, never hand-edited.

Dependencies

Dependency Owner Notes
Forgejo instance + admin Bernd / infra webhook config, org-level token, Actions runners
Gitea→Forgejo cutover plan infra final hostname, raw URL scheme, redirect window
Hub deployment (reuse.coulomb.social) reuse-surface / railiance new endpoint + config rollout
REUSE-WP-0018-T01 reuse-surface shared telemetry/outcome schema
plan-check adoption ecosystem telemetry volume comes from WP-0018-T05 rollout

Forge Host Abstraction And URL Migration Inventory

id: REUSE-WP-0019-T01
status: done
priority: high
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"

Inventory findings (2026-07-07): cross-referenced sources.yaml (61 entries) against each repo's actual git origin remote. Result: 11 repos already migrated their remote to Forgejo; 50 still on Gitea (correctly, no action needed). Of the 11 Forgejo-origin repos, 9 already had correct sources.yaml entries (from WP-0017 drafting/registration); 2 were staleactivity-core and state-hub — still pointing at their old Gitea raw URL despite having migrated. This was real, live debt, not a hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before being migrated for real, in both sources.yaml and the production hub registration (hub update --url). Verified post-migration against GET /v1/federated — both now show forgejo.coulomb.social.

config-atlas's WP-0017-T06 303 was not a host-transition symptom — already diagnosed there as (a) a redirect the current code already follows fine, and (b) a hub-registration gap unrelated to host. No new finding here.

Implemented:

  • reuse_surface/forge_host.py: parse_raw_url/derive_raw_url (handles both the legacy /raw/<branch>/... form and the canonical /raw/branch/<branch>/... form both forges serve without a 303 redirect), rewrite_url_host, forge_base_url (reads REUSE_SURFACE_FORGE_BASE_URL, no hard default since migration is opt-in per repo), migrate_source_host (verifies the new URL resolves via HTTP HEAD before writing — refuses to point a repo at a host it hasn't actually migrated to)
  • CLI: reuse-surface federation migrate-host --repo <slug> [--repo ...] --to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify] [--update-hub] — used for real on activity-core/state-hub
  • Fixed two host-agnostic code gaps found while inventorying: registry_update.py's SAFE_EVIDENCE_PREFIXES and maintain_llm.py's git-diff pathspec only recognized .gitea/workflows/, missing repos already on .forgejo/workflows/ — both now recognize either
  • Fixed stale copy-paste examples in docs/RegistryFederation.md (state-hub's old Gitea URL, now genuinely wrong post-migration) and a pre-existing, unrelated port typo (8088 vs the real llm-connect default 8080) in tools/README.md/registry/README.md, discovered and confirmed live during REUSE-WP-0018-T03
  • 17 new pytest cases (tests/test_forge_host.py); 106 total pass
  • Recomposed federated.yaml post-migration: still 61 capabilities, no loss

Hub Recompose Endpoint And Webhook Receiver

id: REUSE-WP-0019-T02
status: todo
priority: high
state_hub_task_id: "691eb32a-6f20-4a2a-b9ff-0ae427b659aa"
  • Hub service (reuse_surface/serve): POST /v1/recompose (token-auth) — marks index stale and triggers recompose from registered raw URLs
  • POST /v1/webhooks/forgejo: validates Forgejo webhook signature (X-Forgejo-Signature, secret from env), accepts push events, triggers recompose only when the pushed commits touch registry/indexes/
  • Debounce/coalesce concurrent triggers; GET /v1/federated gains composed_at + stale fields
  • Extend specs/FederationHubAPI.md; pytest with signed fixture payloads

Forgejo Webhook Rollout And Scheduled Fallback

id: REUSE-WP-0019-T03
status: wait
priority: medium
state_hub_task_id: "aa9e9f80-b878-490c-832e-515d8cbbbb60"

Blocked on T02 deploy and Forgejo instance availability.

  • Org-level Forgejo webhook (single config, all repos) → hub /v1/webhooks/forgejo, push events only
  • Fallback: Forgejo Actions scheduled workflow (cron) in this repo calling POST /v1/recompose; also serves repos during any Gitea-remnant window
  • Migrate this repo's CI .gitea/workflows/ci.yml.forgejo/workflows/ci.yml (Forgejo Actions; verify runner labels); document the pattern for siblings
  • Verify end-to-end: index change in a sibling repo → hub composed_at advances without manual compose

Reuse Telemetry Store And Recording

id: REUSE-WP-0019-T04
status: todo
priority: medium
state_hub_task_id: "c8e9064e-5c39-4c84-80e9-8b255f8edaec"
  • Implement the shared schema from WP-0018-T01: reuse events {ts, consumer_repo, capability_id, verdict, outcome?, source: plan-check|manual|hub}
  • Hub: POST /v1/reuse-events (token-auth) + local JSONL fallback when hub unreachable; GET /v1/reuse-events?capability_id= for aggregation
  • plan-check --record-outcome (WP-0018) posts here; manual reuse-surface record-reuse for retroactive facts
  • Privacy/scope: repo slugs and capability ids only — no code, no secrets

Telemetry Aggregation Into R-Axis Evidence

id: REUSE-WP-0019-T05
status: wait
priority: medium
state_hub_task_id: "f0282cfa-0a71-4b46-a558-80b51ef04fa7"

Blocked on T04 plus initial event volume.

  • reuse-surface report reuse: per-capability consumer counts, outcomes, last-used; feeds reused_by relation suggestions via the WP-0016 maintain/patch pipeline (evidence-gated, never silent promotion)
  • Maturity standard note: what observed-reuse evidence counts toward R2/R3+ (specs/CapabilityMaturityStandard.md amendment)
  • Catalog + graph surface consumer counts

Freshness Monitoring, Docs, SCOPE

id: REUSE-WP-0019-T06
status: todo
priority: low
state_hub_task_id: "a9f44d45-91e2-4b43-909f-30a5f906cf3b"
  • reuse-surface stats: hub composed_at age + stale flag; CI informational check warns when the hub index is older than N days
  • docs/RegistryFederation.md + docs/deploy/reuse-kubernetes.md: webhook setup, recompose endpoint, Forgejo token handling (route credentials per credential-routing rules — no secrets in repo)
  • SCOPE.md: flip "automatic hub refresh" to possible; update federation posture

Acceptance

  • No hardcoded forge host in code or sources.yaml; migrate-host tested
  • Push to a sibling repo's registry/indexes/ recomposes the hub index without manual action (webhook), with scheduled fallback in place
  • This repo's CI runs on Forgejo Actions (.forgejo/workflows/)
  • Reuse events recordable via hub API and CLI; report reuse aggregates them
  • R-axis evidence rules for observed reuse documented in the maturity standard
  • Hub freshness visible (composed_at, stale flag) in API and stats

Out of scope

  • Operating the Forgejo instance or the Gitea→Forgejo data migration itself
  • Multi-replica/Postgres hub posture (separate managed-platform track)
  • Blocking CI gates on registry freshness in sibling repos
  • ActivityPub/Forgejo-native federation features (our federation layer stays raw-URL based)