generated from coulomb/repo-seed
REUSE-WP-0019-T01: forge host abstraction + URL migration inventory
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
reuse_surface/forge_host.py: parse/derive/rewrite raw index URLs across Gitea and Forgejo (handles both the legacy /raw/<branch>/... form and the canonical /raw/branch/<branch>/... form both forges serve without a 303 redirect). migrate_source_host() verifies the new URL resolves via HTTP HEAD before writing -- refuses to point a repo at a host it hasn't actually migrated to. New CLI: reuse-surface federation migrate-host --repo <slug> --to <base-url> [--from <check>] [--dry-run] [--no-verify] [--update-hub]. Inventory: cross-referenced sources.yaml against each repo's actual git origin. 11/61 repos already on Forgejo; found 2 with stale sources.yaml entries (activity-core, state-hub) despite having migrated. Fixed for real: local sources.yaml + production hub registration (hub update), verified against GET /v1/federated post-migration. config-atlas's WP-0017-T06 303 confirmed NOT a host-transition symptom (already diagnosed there as something else). Also fixed two host-agnostic gaps found while inventorying: registry_update.py and maintain_llm.py only recognized .gitea/workflows/, missing repos already on .forgejo/workflows/. Fixed a stale copy-paste example (state-hub's now-wrong old URL) in docs/RegistryFederation.md, and a pre-existing unrelated port typo (8088 vs the real llm-connect default 8080) in tools/README.md and registry/README.md. 17 new pytest cases (tests/test_forge_host.py), 106 total pass. Recomposed federated.yaml post-migration: still 61 capabilities. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -94,13 +94,24 @@ returns **200** with valid YAML (not a redirect to login or HTML).
|
|||||||
|
|
||||||
Entry bodies remain in the source repo; the index is the federation surface.
|
Entry bodies remain in the source repo; the index is the federation surface.
|
||||||
|
|
||||||
### Gitea raw URL shape
|
### Raw URL shape (Gitea or Forgejo)
|
||||||
|
|
||||||
|
The organization is mid-transition from Gitea to Forgejo (REUSE-WP-0019);
|
||||||
|
sibling repos migrate their git remote independently, so both forms are
|
||||||
|
valid depending on which host a given repo currently lives on. Prefer the
|
||||||
|
branch-qualified form — both forges serve it directly with no redirect,
|
||||||
|
unlike the shorter `/raw/<branch>/...` form which 303-redirects:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
https://gitea.coulomb.social/coulomb/<repo>/raw/<branch>/registry/indexes/capabilities.yaml
|
https://gitea.coulomb.social/coulomb/<repo>/raw/branch/<branch>/registry/indexes/capabilities.yaml
|
||||||
|
https://forgejo.coulomb.social/coulomb/<repo>/raw/branch/<branch>/registry/indexes/capabilities.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
Use `main` (or the repo's default branch). Verify before registration:
|
Use `main` (or the repo's default branch). `reuse-surface federation
|
||||||
|
migrate-host` rewrites a repo's registered URL from one host to the other
|
||||||
|
once its remote has actually migrated (verifies reachability before
|
||||||
|
writing — never blind-rewrites a repo that hasn't moved yet). Verify before
|
||||||
|
registration:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -fsSI "<raw-url>" | head -n1 # expect HTTP/2 200 or HTTP/1.1 200
|
curl -fsSI "<raw-url>" | head -n1 # expect HTTP/2 200 or HTTP/1.1 200
|
||||||
@@ -132,17 +143,17 @@ reuse-surface establish --scaffold --domain helix_forge
|
|||||||
reuse-surface validate
|
reuse-surface validate
|
||||||
git push origin main
|
git push origin main
|
||||||
reuse-surface establish --publish-check \
|
reuse-surface establish --publish-check \
|
||||||
--raw-url https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml
|
--raw-url https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
```
|
```
|
||||||
|
|
||||||
### Ongoing maintenance (from sibling repo)
|
### Ongoing maintenance (from sibling repo)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export LLM_CONNECT_URL=http://127.0.0.1:8088 # optional
|
export LLM_CONNECT_URL=http://127.0.0.1:8080 # optional
|
||||||
reuse-surface maintain --all --from-git-since origin/main
|
reuse-surface maintain --all --from-git-since origin/main
|
||||||
reuse-surface maintain --all --auto --no-llm # CI / pre-commit
|
reuse-surface maintain --all --auto --no-llm # CI / pre-commit
|
||||||
reuse-surface maintain --publish \
|
reuse-surface maintain --publish \
|
||||||
--raw-url https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml \
|
--raw-url https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml \
|
||||||
--all --auto --no-llm
|
--all --auto --no-llm
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -70,7 +70,7 @@ reuse-surface maintain --all --auto --no-llm
|
|||||||
With llm-connect for maturity suggestions:
|
With llm-connect for maturity suggestions:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export LLM_CONNECT_URL=http://127.0.0.1:8088
|
export LLM_CONNECT_URL=http://127.0.0.1:8080
|
||||||
reuse-surface maintain --all --from-git-since HEAD~5
|
reuse-surface maintain --all --from-git-since HEAD~5
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ domain: helix_forge
|
|||||||
collision_policy: warn
|
collision_policy: warn
|
||||||
sources:
|
sources:
|
||||||
- repo: activity-core
|
- repo: activity-core
|
||||||
url: https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml
|
url: https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
enabled: true
|
enabled: true
|
||||||
required: false
|
required: false
|
||||||
domain: helix_forge
|
domain: helix_forge
|
||||||
@@ -375,7 +375,7 @@ sources:
|
|||||||
cache_ttl_seconds: 86400
|
cache_ttl_seconds: 86400
|
||||||
auth_header: Authorization
|
auth_header: Authorization
|
||||||
- repo: state-hub
|
- repo: state-hub
|
||||||
url: https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml
|
url: https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
enabled: true
|
enabled: true
|
||||||
required: false
|
required: false
|
||||||
domain: helix_forge
|
domain: helix_forge
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ collision_policy: warn
|
|||||||
sources:
|
sources:
|
||||||
- repo: activity-core
|
- repo: activity-core
|
||||||
count: 1
|
count: 1
|
||||||
url: https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml
|
url: https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
cache: registry/federation/cache/activity-core.yaml
|
cache: registry/federation/cache/activity-core.yaml
|
||||||
- repo: agentic-resources
|
- repo: agentic-resources
|
||||||
count: 0
|
count: 0
|
||||||
@@ -219,7 +219,7 @@ sources:
|
|||||||
cache: registry/federation/cache/shard-wiki.yaml
|
cache: registry/federation/cache/shard-wiki.yaml
|
||||||
- repo: state-hub
|
- repo: state-hub
|
||||||
count: 2
|
count: 2
|
||||||
url: https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml
|
url: https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
cache: registry/federation/cache/state-hub.yaml
|
cache: registry/federation/cache/state-hub.yaml
|
||||||
- repo: tegwick-control
|
- repo: tegwick-control
|
||||||
count: 0
|
count: 0
|
||||||
@@ -266,7 +266,7 @@ capabilities:
|
|||||||
consumption_modes:
|
consumption_modes:
|
||||||
- informational
|
- informational
|
||||||
source_repo: activity-core
|
source_repo: activity-core
|
||||||
source_url: https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml
|
source_url: https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
source_index: registry/federation/cache/activity-core.yaml
|
source_index: registry/federation/cache/activity-core.yaml
|
||||||
- id: capability.agents.cli-assistant
|
- id: capability.agents.cli-assistant
|
||||||
name: Console-Native LLM Assistant (cya)
|
name: Console-Native LLM Assistant (cya)
|
||||||
@@ -869,6 +869,7 @@ capabilities:
|
|||||||
consumption_modes:
|
consumption_modes:
|
||||||
- informational
|
- informational
|
||||||
- planning
|
- planning
|
||||||
|
- cli
|
||||||
source_repo: core-hub
|
source_repo: core-hub
|
||||||
source_url: https://forgejo.coulomb.social/coulomb/core-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
source_url: https://forgejo.coulomb.social/coulomb/core-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
source_index: registry/federation/cache/core-hub.yaml
|
source_index: registry/federation/cache/core-hub.yaml
|
||||||
@@ -1013,7 +1014,7 @@ capabilities:
|
|||||||
summary: "Django application (with a Vite/Tailwind frontend) for managing German\
|
summary: "Django application (with a Vite/Tailwind frontend) for managing German\
|
||||||
\ public-procurement (Vergabe) tender participation \u2014 Ausschreibungs- und\
|
\ public-procurement (Vergabe) tender participation \u2014 Ausschreibungs- und\
|
||||||
\ Teilnahme-Management-System."
|
\ Teilnahme-Management-System."
|
||||||
vector: D1 / A1 / C0 / R0
|
vector: D3 / A1 / C1 / R1
|
||||||
domain: communication
|
domain: communication
|
||||||
status: draft
|
status: draft
|
||||||
owner: vergabe-teilnahme
|
owner: vergabe-teilnahme
|
||||||
@@ -1274,7 +1275,7 @@ capabilities:
|
|||||||
consumption_modes:
|
consumption_modes:
|
||||||
- service API
|
- service API
|
||||||
source_repo: state-hub
|
source_repo: state-hub
|
||||||
source_url: https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml
|
source_url: https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
source_index: registry/federation/cache/state-hub.yaml
|
source_index: registry/federation/cache/state-hub.yaml
|
||||||
- id: capability.statehub.workstream-coordinate
|
- id: capability.statehub.workstream-coordinate
|
||||||
name: Workstream And Task Coordination
|
name: Workstream And Task Coordination
|
||||||
@@ -1293,7 +1294,7 @@ capabilities:
|
|||||||
- service API
|
- service API
|
||||||
- HTTP REST
|
- HTTP REST
|
||||||
source_repo: state-hub
|
source_repo: state-hub
|
||||||
source_url: https://gitea.coulomb.social/coulomb/state-hub/raw/main/registry/indexes/capabilities.yaml
|
source_url: https://forgejo.coulomb.social/coulomb/state-hub/raw/branch/main/registry/indexes/capabilities.yaml
|
||||||
source_index: registry/federation/cache/state-hub.yaml
|
source_index: registry/federation/cache/state-hub.yaml
|
||||||
- id: capability.wiki.adapter-contract
|
- id: capability.wiki.adapter-contract
|
||||||
name: Capability-Aware Shard Adapter Contract
|
name: Capability-Aware Shard Adapter Contract
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ from jsonschema import Draft202012Validator
|
|||||||
|
|
||||||
from reuse_surface.catalog import write_catalog
|
from reuse_surface.catalog import write_catalog
|
||||||
from reuse_surface.federation import write_federated_index
|
from reuse_surface.federation import write_federated_index
|
||||||
|
from reuse_surface.forge_host import migrate_source_host
|
||||||
from reuse_surface import hub_client
|
from reuse_surface import hub_client
|
||||||
from reuse_surface.graph import check_relations, render_mermaid, write_graph
|
from reuse_surface.graph import check_relations, render_mermaid, write_graph
|
||||||
from reuse_surface.hub_sync import (
|
from reuse_surface.hub_sync import (
|
||||||
@@ -264,6 +265,56 @@ def cmd_federation_compose(args: argparse.Namespace) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_federation_migrate_host(args: argparse.Namespace) -> int:
|
||||||
|
sources_path = Path(args.sources) if args.sources else DEFAULT_SOURCES_PATH
|
||||||
|
manifest = load_sources_manifest(sources_path)
|
||||||
|
sources = manifest.get("sources", [])
|
||||||
|
|
||||||
|
reports: list[dict[str, Any]] = []
|
||||||
|
errors: list[str] = []
|
||||||
|
for repo in args.repo:
|
||||||
|
entry = next((s for s in sources if s.get("repo") == repo), None)
|
||||||
|
if entry is None:
|
||||||
|
errors.append(f"{repo}: not found in {sources_path}")
|
||||||
|
continue
|
||||||
|
if args.from_host and args.from_host not in entry["url"]:
|
||||||
|
errors.append(
|
||||||
|
f"{repo}: current URL does not contain --from {args.from_host!r}: {entry['url']}"
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
report = migrate_source_host(sources, repo, args.to, verify=not args.no_verify)
|
||||||
|
reports.append(report)
|
||||||
|
except ValueError as exc:
|
||||||
|
errors.append(str(exc))
|
||||||
|
|
||||||
|
for r in reports:
|
||||||
|
verified = "unverified" if r["verified"] is None else ("verified" if r["verified"] else "FAILED")
|
||||||
|
print(f"{r['repo']}: {r['old_url']} -> {r['new_url']} ({verified})")
|
||||||
|
for e in errors:
|
||||||
|
print(f"error: {e}", file=sys.stderr)
|
||||||
|
|
||||||
|
if args.dry_run:
|
||||||
|
print("(dry-run, no files written)")
|
||||||
|
return 1 if errors else 0
|
||||||
|
|
||||||
|
if reports:
|
||||||
|
write_sources_manifest(manifest, sources_path)
|
||||||
|
|
||||||
|
if args.update_hub:
|
||||||
|
for r in reports:
|
||||||
|
try:
|
||||||
|
status, payload = hub_client.hub_update(
|
||||||
|
r["repo"], {"url": r["new_url"]}, args.base_url
|
||||||
|
)
|
||||||
|
if status != 200:
|
||||||
|
errors.append(f"{r['repo']}: hub update returned {status}: {payload}")
|
||||||
|
except ValueError as exc:
|
||||||
|
errors.append(f"{r['repo']}: hub update skipped: {exc}")
|
||||||
|
|
||||||
|
return 1 if errors else 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_graph(args: argparse.Namespace) -> int:
|
def cmd_graph(args: argparse.Namespace) -> int:
|
||||||
warnings = check_relations() if args.check else []
|
warnings = check_relations() if args.check else []
|
||||||
content = render_mermaid()
|
content = render_mermaid()
|
||||||
@@ -750,6 +801,35 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
)
|
)
|
||||||
compose.set_defaults(func=cmd_federation_compose)
|
compose.set_defaults(func=cmd_federation_compose)
|
||||||
|
|
||||||
|
migrate_host = federation_sub.add_parser(
|
||||||
|
"migrate-host",
|
||||||
|
help="rewrite one or more repos' raw index URL to a new forge host (REUSE-WP-0019-T01)",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument(
|
||||||
|
"--repo", action="append", required=True,
|
||||||
|
help="repo slug to migrate (repeatable)",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument(
|
||||||
|
"--to", required=True,
|
||||||
|
help="new base URL, e.g. https://forgejo.coulomb.social (or REUSE_SURFACE_FORGE_BASE_URL)",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument(
|
||||||
|
"--from", dest="from_host",
|
||||||
|
help="sanity check: refuse to migrate a repo whose current URL doesn't contain this",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument("--sources", help="path to sources.yaml (default: registry/federation/sources.yaml)")
|
||||||
|
migrate_host.add_argument("--dry-run", action="store_true", help="show what would change, write nothing")
|
||||||
|
migrate_host.add_argument(
|
||||||
|
"--no-verify", action="store_true",
|
||||||
|
help="skip HTTP-probing the new URL before writing (unsafe -- can point at a repo that hasn't migrated yet)",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument(
|
||||||
|
"--update-hub", action="store_true",
|
||||||
|
help="also PATCH the production hub registration for each migrated repo",
|
||||||
|
)
|
||||||
|
migrate_host.add_argument("--base-url", help="hub service base URL (or REUSE_SURFACE_URL), used with --update-hub")
|
||||||
|
migrate_host.set_defaults(func=cmd_federation_migrate_host)
|
||||||
|
|
||||||
query = subparsers.add_parser("query", help="query capability index")
|
query = subparsers.add_parser("query", help="query capability index")
|
||||||
query.add_argument("--discovery-min")
|
query.add_argument("--discovery-min")
|
||||||
query.add_argument("--availability-min")
|
query.add_argument("--availability-min")
|
||||||
|
|||||||
105
reuse_surface/forge_host.py
Normal file
105
reuse_surface/forge_host.py
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
# Matches both the legacy Gitea-default form (.../raw/<branch>/<path>) and the
|
||||||
|
# canonical branch-qualified form (.../raw/branch/<branch>/<path>) that both
|
||||||
|
# Gitea and Forgejo serve without a 303 redirect. See REUSE-WP-0017-T06 for
|
||||||
|
# why the redirect matters (urllib follows it fine; some HEAD-only probes
|
||||||
|
# and shell tooling don't).
|
||||||
|
_RAW_URL_RE = re.compile(
|
||||||
|
r"^(?P<scheme>https?)://(?P<host>[^/]+)/(?P<org>[^/]+)/(?P<repo>[^/]+)"
|
||||||
|
r"/raw/(?:branch/)?(?P<branch>[^/]+)/(?P<path>.+)$"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RawUrlParts:
|
||||||
|
scheme: str
|
||||||
|
host: str
|
||||||
|
org: str
|
||||||
|
repo: str
|
||||||
|
branch: str
|
||||||
|
path: str
|
||||||
|
|
||||||
|
|
||||||
|
def parse_raw_url(url: str) -> RawUrlParts:
|
||||||
|
match = _RAW_URL_RE.match(url)
|
||||||
|
if not match:
|
||||||
|
raise ValueError(f"not a recognized Gitea/Forgejo raw URL: {url}")
|
||||||
|
return RawUrlParts(**match.groupdict())
|
||||||
|
|
||||||
|
|
||||||
|
def derive_raw_url(base_url: str, org: str, repo: str, *, branch: str = "main", path: str = "registry/indexes/capabilities.yaml") -> str:
|
||||||
|
"""Builds a raw index URL in the canonical branch-qualified form, which
|
||||||
|
both Gitea and Forgejo serve directly (no 303 redirect)."""
|
||||||
|
base = base_url.rstrip("/")
|
||||||
|
return f"{base}/{org}/{repo}/raw/branch/{branch}/{path}"
|
||||||
|
|
||||||
|
|
||||||
|
def rewrite_url_host(url: str, new_base_url: str) -> str:
|
||||||
|
"""Rewrites a raw index URL's scheme+host to new_base_url, preserving
|
||||||
|
org/repo/branch/path and normalizing to the canonical branch-qualified
|
||||||
|
form regardless of which form the input used."""
|
||||||
|
parts = parse_raw_url(url)
|
||||||
|
return derive_raw_url(new_base_url, parts.org, parts.repo, branch=parts.branch, path=parts.path)
|
||||||
|
|
||||||
|
|
||||||
|
def forge_base_url(explicit: str | None = None) -> str | None:
|
||||||
|
"""Returns the configured default Forgejo base URL, or None if unset.
|
||||||
|
Unlike hub_client.service_base_url, this has no hard requirement --
|
||||||
|
per-repo migration is opt-in (most repos haven't moved their remote
|
||||||
|
yet), so an unset value just means 'no default target configured',
|
||||||
|
not an error."""
|
||||||
|
return explicit or os.environ.get("REUSE_SURFACE_FORGE_BASE_URL") or None
|
||||||
|
|
||||||
|
|
||||||
|
def probe_url(url: str, *, timeout: int = 10) -> tuple[bool, int | None]:
|
||||||
|
"""HEAD-probes a URL, following redirects (matches establish.py's
|
||||||
|
_probe_raw_url behavior). Returns (ok, status)."""
|
||||||
|
request = urllib.request.Request(url, method="HEAD", headers={"User-Agent": "reuse-surface/0.1"})
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||||
|
return response.status == 200, response.status
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
return False, exc.code
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError):
|
||||||
|
return False, None
|
||||||
|
|
||||||
|
|
||||||
|
def migrate_source_host(
|
||||||
|
sources: list[dict[str, Any]],
|
||||||
|
repo: str,
|
||||||
|
new_base_url: str,
|
||||||
|
*,
|
||||||
|
verify: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Rewrites one repo's entry in a sources.yaml `sources` list to point at
|
||||||
|
new_base_url. Mutates the matching entry in place and returns a report
|
||||||
|
dict. Raises ValueError if the repo isn't found, the existing URL can't
|
||||||
|
be parsed, or (when verify=True) the new URL doesn't resolve -- never
|
||||||
|
writes a URL that hasn't been confirmed reachable, since rewriting to a
|
||||||
|
host the repo hasn't actually migrated to would silently break
|
||||||
|
federation for it."""
|
||||||
|
entry = next((s for s in sources if s.get("repo") == repo), None)
|
||||||
|
if entry is None:
|
||||||
|
raise ValueError(f"repo not found in sources: {repo}")
|
||||||
|
old_url = entry["url"]
|
||||||
|
new_url = rewrite_url_host(old_url, new_base_url)
|
||||||
|
report: dict[str, Any] = {"repo": repo, "old_url": old_url, "new_url": new_url, "verified": None}
|
||||||
|
if verify:
|
||||||
|
ok, status = probe_url(new_url)
|
||||||
|
report["verified"] = ok
|
||||||
|
report["probe_status"] = status
|
||||||
|
if not ok:
|
||||||
|
raise ValueError(
|
||||||
|
f"{repo}: new URL did not resolve (HTTP {status}); "
|
||||||
|
f"not writing an unverified URL: {new_url}"
|
||||||
|
)
|
||||||
|
entry["url"] = new_url
|
||||||
|
return report
|
||||||
@@ -50,6 +50,7 @@ def _git_diff(repo_root: Path, git_since: str | None) -> str:
|
|||||||
"tests/",
|
"tests/",
|
||||||
"docs/",
|
"docs/",
|
||||||
".gitea/",
|
".gitea/",
|
||||||
|
".forgejo/",
|
||||||
"pyproject.toml",
|
"pyproject.toml",
|
||||||
],
|
],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
|
|||||||
@@ -19,7 +19,11 @@ from reuse_surface.registry import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Safe to apply without interactive review (see patches.SAFE_DETERMINISTIC_KINDS).
|
# Safe to apply without interactive review (see patches.SAFE_DETERMINISTIC_KINDS).
|
||||||
SAFE_EVIDENCE_PREFIXES = ("tests/", ".gitea/workflows/")
|
# Both forge CI workflow paths are recognized during the Gitea->Forgejo
|
||||||
|
# transition (REUSE-WP-0019-T01) -- sibling repos migrate independently, so
|
||||||
|
# either path is valid CI evidence depending on a given repo's progress.
|
||||||
|
CI_WORKFLOW_PREFIXES = (".gitea/workflows/", ".forgejo/workflows/")
|
||||||
|
SAFE_EVIDENCE_PREFIXES = ("tests/", *CI_WORKFLOW_PREFIXES)
|
||||||
|
|
||||||
|
|
||||||
def git_changed_files(repo_root: Path, since_ref: str) -> list[str]:
|
def git_changed_files(repo_root: Path, since_ref: str) -> list[str]:
|
||||||
@@ -194,7 +198,7 @@ def _collect_changed_file_suggestions(
|
|||||||
"apply_patch": {"field": "evidence.tests", "append": changed},
|
"apply_patch": {"field": "evidence.tests", "append": changed},
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
if changed.startswith(".gitea/workflows/") and changed.endswith((".yml", ".yaml")):
|
if changed.startswith(CI_WORKFLOW_PREFIXES) and changed.endswith((".yml", ".yaml")):
|
||||||
field = "evidence.tests" if "test" in changed.lower() else "evidence.documentation"
|
field = "evidence.tests" if "test" in changed.lower() else "evidence.documentation"
|
||||||
existing = evidence_tests if field == "evidence.tests" else evidence_docs
|
existing = evidence_tests if field == "evidence.tests" else evidence_docs
|
||||||
if changed not in existing:
|
if changed not in existing:
|
||||||
|
|||||||
188
tests/test_forge_host.py
Normal file
188
tests/test_forge_host.py
Normal file
@@ -0,0 +1,188 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from reuse_surface.forge_host import (
|
||||||
|
derive_raw_url,
|
||||||
|
forge_base_url,
|
||||||
|
migrate_source_host,
|
||||||
|
parse_raw_url,
|
||||||
|
rewrite_url_host,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_raw_url_legacy_form():
|
||||||
|
parts = parse_raw_url("https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml")
|
||||||
|
assert parts.scheme == "https"
|
||||||
|
assert parts.host == "gitea.coulomb.social"
|
||||||
|
assert parts.org == "coulomb"
|
||||||
|
assert parts.repo == "activity-core"
|
||||||
|
assert parts.branch == "main"
|
||||||
|
assert parts.path == "registry/indexes/capabilities.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_raw_url_branch_qualified_form():
|
||||||
|
parts = parse_raw_url("https://forgejo.coulomb.social/coulomb/core-hub/raw/branch/main/registry/indexes/capabilities.yaml")
|
||||||
|
assert parts.host == "forgejo.coulomb.social"
|
||||||
|
assert parts.repo == "core-hub"
|
||||||
|
assert parts.branch == "main"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_raw_url_rejects_unrecognized():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_raw_url("https://example.com/not/a/raw/url")
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_raw_url_uses_branch_qualified_form():
|
||||||
|
url = derive_raw_url("https://forgejo.coulomb.social", "coulomb", "core-hub")
|
||||||
|
assert url == "https://forgejo.coulomb.social/coulomb/core-hub/raw/branch/main/registry/indexes/capabilities.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def test_derive_raw_url_strips_trailing_slash():
|
||||||
|
url = derive_raw_url("https://forgejo.coulomb.social/", "coulomb", "core-hub")
|
||||||
|
assert url.startswith("https://forgejo.coulomb.social/coulomb/")
|
||||||
|
|
||||||
|
|
||||||
|
def test_rewrite_url_host_swaps_host_and_normalizes_form():
|
||||||
|
old = "https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml"
|
||||||
|
new = rewrite_url_host(old, "https://forgejo.coulomb.social")
|
||||||
|
assert new == "https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def test_forge_base_url_env_fallback(monkeypatch):
|
||||||
|
monkeypatch.setenv("REUSE_SURFACE_FORGE_BASE_URL", "https://forgejo.coulomb.social")
|
||||||
|
assert forge_base_url() == "https://forgejo.coulomb.social"
|
||||||
|
|
||||||
|
|
||||||
|
def test_forge_base_url_none_when_unset(monkeypatch):
|
||||||
|
monkeypatch.delenv("REUSE_SURFACE_FORGE_BASE_URL", raising=False)
|
||||||
|
assert forge_base_url() is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_forge_base_url_explicit_wins(monkeypatch):
|
||||||
|
monkeypatch.setenv("REUSE_SURFACE_FORGE_BASE_URL", "https://env-value")
|
||||||
|
assert forge_base_url("https://explicit-value") == "https://explicit-value"
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_source_host_mutates_matching_entry(monkeypatch):
|
||||||
|
monkeypatch.setattr("reuse_surface.forge_host.probe_url", lambda url, **k: (True, 200))
|
||||||
|
sources = [
|
||||||
|
{"repo": "activity-core", "url": "https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml"},
|
||||||
|
{"repo": "other-repo", "url": "https://gitea.coulomb.social/coulomb/other-repo/raw/main/registry/indexes/capabilities.yaml"},
|
||||||
|
]
|
||||||
|
report = migrate_source_host(sources, "activity-core", "https://forgejo.coulomb.social")
|
||||||
|
assert report["verified"] is True
|
||||||
|
assert sources[0]["url"] == "https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml"
|
||||||
|
# other entries untouched
|
||||||
|
assert sources[1]["url"] == "https://gitea.coulomb.social/coulomb/other-repo/raw/main/registry/indexes/capabilities.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_source_host_repo_not_found():
|
||||||
|
sources = [{"repo": "activity-core", "url": "https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml"}]
|
||||||
|
with pytest.raises(ValueError, match="not found"):
|
||||||
|
migrate_source_host(sources, "does-not-exist", "https://forgejo.coulomb.social")
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_source_host_refuses_unverified_url(monkeypatch):
|
||||||
|
monkeypatch.setattr("reuse_surface.forge_host.probe_url", lambda url, **k: (False, 404))
|
||||||
|
sources = [{"repo": "activity-core", "url": "https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml"}]
|
||||||
|
original_url = sources[0]["url"]
|
||||||
|
with pytest.raises(ValueError, match="did not resolve"):
|
||||||
|
migrate_source_host(sources, "activity-core", "https://forgejo.coulomb.social")
|
||||||
|
# must not have mutated the entry when verification failed
|
||||||
|
assert sources[0]["url"] == original_url
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_source_host_skips_verification_when_disabled():
|
||||||
|
sources = [{"repo": "activity-core", "url": "https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml"}]
|
||||||
|
report = migrate_source_host(sources, "activity-core", "https://forgejo.coulomb.social", verify=False)
|
||||||
|
assert report["verified"] is None
|
||||||
|
assert sources[0]["url"].startswith("https://forgejo.coulomb.social/")
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_federation_migrate_host_dry_run(tmp_path, monkeypatch):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
sources_path = tmp_path / "sources.yaml"
|
||||||
|
sources_path.write_text(
|
||||||
|
"version: 1\n"
|
||||||
|
"domain: helix_forge\n"
|
||||||
|
"collision_policy: warn\n"
|
||||||
|
"sources:\n"
|
||||||
|
"- repo: activity-core\n"
|
||||||
|
" url: https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml\n"
|
||||||
|
" enabled: true\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr("reuse_surface.cli.migrate_source_host", lambda sources, repo, to, verify=True: {
|
||||||
|
"repo": repo, "old_url": sources[0]["url"], "new_url": "https://forgejo.coulomb.social/x", "verified": True,
|
||||||
|
})
|
||||||
|
exit_code = main([
|
||||||
|
"federation", "migrate-host",
|
||||||
|
"--repo", "activity-core",
|
||||||
|
"--to", "https://forgejo.coulomb.social",
|
||||||
|
"--sources", str(sources_path),
|
||||||
|
"--dry-run",
|
||||||
|
])
|
||||||
|
assert exit_code == 0
|
||||||
|
# dry-run must not write
|
||||||
|
assert "gitea.coulomb.social" in sources_path.read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_federation_migrate_host_writes_file(tmp_path, monkeypatch):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
sources_path = tmp_path / "sources.yaml"
|
||||||
|
sources_path.write_text(
|
||||||
|
"version: 1\n"
|
||||||
|
"domain: helix_forge\n"
|
||||||
|
"collision_policy: warn\n"
|
||||||
|
"sources:\n"
|
||||||
|
"- repo: activity-core\n"
|
||||||
|
" url: https://gitea.coulomb.social/coulomb/activity-core/raw/main/registry/indexes/capabilities.yaml\n"
|
||||||
|
" enabled: true\n"
|
||||||
|
)
|
||||||
|
monkeypatch.setattr("reuse_surface.forge_host.probe_url", lambda url, **k: (True, 200))
|
||||||
|
exit_code = main([
|
||||||
|
"federation", "migrate-host",
|
||||||
|
"--repo", "activity-core",
|
||||||
|
"--to", "https://forgejo.coulomb.social",
|
||||||
|
"--sources", str(sources_path),
|
||||||
|
])
|
||||||
|
assert exit_code == 0
|
||||||
|
written = sources_path.read_text()
|
||||||
|
assert "forgejo.coulomb.social" in written
|
||||||
|
assert "gitea.coulomb.social" not in written
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_federation_migrate_host_repo_not_found_errors(tmp_path):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
sources_path = tmp_path / "sources.yaml"
|
||||||
|
sources_path.write_text("version: 1\ndomain: helix_forge\ncollision_policy: warn\nsources: []\n")
|
||||||
|
exit_code = main([
|
||||||
|
"federation", "migrate-host",
|
||||||
|
"--repo", "does-not-exist",
|
||||||
|
"--to", "https://forgejo.coulomb.social",
|
||||||
|
"--sources", str(sources_path),
|
||||||
|
"--dry-run",
|
||||||
|
])
|
||||||
|
assert exit_code == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_cmd_federation_migrate_host_from_mismatch_errors(tmp_path):
|
||||||
|
from reuse_surface.cli import main
|
||||||
|
|
||||||
|
sources_path = tmp_path / "sources.yaml"
|
||||||
|
sources_path.write_text(
|
||||||
|
"version: 1\ndomain: helix_forge\ncollision_policy: warn\n"
|
||||||
|
"sources:\n- repo: activity-core\n url: https://forgejo.coulomb.social/coulomb/activity-core/raw/branch/main/registry/indexes/capabilities.yaml\n"
|
||||||
|
)
|
||||||
|
exit_code = main([
|
||||||
|
"federation", "migrate-host",
|
||||||
|
"--repo", "activity-core",
|
||||||
|
"--to", "https://forgejo.coulomb.social",
|
||||||
|
"--from", "gitea.coulomb.social",
|
||||||
|
"--sources", str(sources_path),
|
||||||
|
"--dry-run",
|
||||||
|
])
|
||||||
|
assert exit_code == 1
|
||||||
@@ -173,7 +173,7 @@ Bootstrap or discover a capability registry in the current or target repo.
|
|||||||
reuse-surface establish --scaffold --domain helix_forge
|
reuse-surface establish --scaffold --domain helix_forge
|
||||||
reuse-surface establish --scaffold --path ../state-hub
|
reuse-surface establish --scaffold --path ../state-hub
|
||||||
reuse-surface establish --publish-check --raw-url https://.../capabilities.yaml
|
reuse-surface establish --publish-check --raw-url https://.../capabilities.yaml
|
||||||
export LLM_CONNECT_URL=http://127.0.0.1:8088
|
export LLM_CONNECT_URL=http://127.0.0.1:8080
|
||||||
reuse-surface establish --discover --dry-run
|
reuse-surface establish --discover --dry-run
|
||||||
reuse-surface establish --discover --apply
|
reuse-surface establish --discover --apply
|
||||||
```
|
```
|
||||||
@@ -200,7 +200,7 @@ Interactive or automated registry maintenance (REUSE-WP-0016). Preferred entry
|
|||||||
point for sibling repo operators.
|
point for sibling repo operators.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
export LLM_CONNECT_URL=http://127.0.0.1:8088 # optional
|
export LLM_CONNECT_URL=http://127.0.0.1:8080 # optional
|
||||||
reuse-surface maintain --all --from-git-since origin/main
|
reuse-surface maintain --all --from-git-since origin/main
|
||||||
reuse-surface maintain --capability capability.registry.register
|
reuse-surface maintain --capability capability.registry.register
|
||||||
reuse-surface maintain --all --auto --no-llm
|
reuse-surface maintain --all --auto --no-llm
|
||||||
|
|||||||
@@ -62,21 +62,52 @@ refresh* and moves reliability evidence beyond structural.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: REUSE-WP-0019-T01
|
id: REUSE-WP-0019-T01
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"
|
state_hub_task_id: "4a187b56-bff9-4097-abd0-b423e7bf9442"
|
||||||
```
|
```
|
||||||
|
|
||||||
- Inventory every hardcoded `gitea.coulomb.social` / `.gitea/` reference:
|
**Inventory findings (2026-07-07):** cross-referenced `sources.yaml` (61
|
||||||
`sources.yaml` (60 raw URLs), hub registrations, establish/publish-check
|
entries) against each repo's actual git `origin` remote. Result: 11 repos
|
||||||
defaults, docs, templates, CI workflows
|
already migrated their remote to Forgejo; 50 still on Gitea (correctly, no
|
||||||
- Introduce `REUSE_SURFACE_FORGE_BASE_URL` (env + hub config); derive raw
|
action needed). Of the 11 Forgejo-origin repos, 9 already had correct
|
||||||
index URLs from `{base}/{org}/{repo}/raw/{branch}/registry/indexes/capabilities.yaml`
|
`sources.yaml` entries (from WP-0017 drafting/registration); **2 were
|
||||||
- Migration command: `reuse-surface federation migrate-host --from <old> --to <new>`
|
stale** — `activity-core` and `state-hub` — still pointing at their old
|
||||||
rewriting sources.yaml + hub registrations via the hub API
|
Gitea raw URL despite having migrated. This was real, live debt, not a
|
||||||
- Check whether the config-atlas 303 (WP-0017-T06) is a symptom of the host
|
hypothetical: both were confirmed reachable on Forgejo (HTTP 200) before
|
||||||
transition; coordinate findings
|
being migrated for real, in both `sources.yaml` and the production hub
|
||||||
- Tests: URL derivation, migrate-host dry-run
|
registration (`hub update --url`). Verified post-migration against
|
||||||
|
`GET /v1/federated` — both now show `forgejo.coulomb.social`.
|
||||||
|
|
||||||
|
`config-atlas`'s WP-0017-T06 303 was **not** a host-transition symptom —
|
||||||
|
already diagnosed there as (a) a redirect the current code already follows
|
||||||
|
fine, and (b) a hub-registration gap unrelated to host. No new finding here.
|
||||||
|
|
||||||
|
Implemented:
|
||||||
|
|
||||||
|
- `reuse_surface/forge_host.py`: `parse_raw_url`/`derive_raw_url` (handles
|
||||||
|
both the legacy `/raw/<branch>/...` form and the canonical
|
||||||
|
`/raw/branch/<branch>/...` form both forges serve without a 303
|
||||||
|
redirect), `rewrite_url_host`, `forge_base_url` (reads
|
||||||
|
`REUSE_SURFACE_FORGE_BASE_URL`, no hard default since migration is
|
||||||
|
opt-in per repo), `migrate_source_host` (verifies the new URL resolves
|
||||||
|
via HTTP HEAD **before** writing — refuses to point a repo at a host it
|
||||||
|
hasn't actually migrated to)
|
||||||
|
- CLI: `reuse-surface federation migrate-host --repo <slug> [--repo ...]
|
||||||
|
--to <base-url> [--from <sanity-check>] [--dry-run] [--no-verify]
|
||||||
|
[--update-hub]` — used for real on `activity-core`/`state-hub`
|
||||||
|
- Fixed two host-agnostic code gaps found while inventorying:
|
||||||
|
`registry_update.py`'s `SAFE_EVIDENCE_PREFIXES` and `maintain_llm.py`'s
|
||||||
|
git-diff pathspec only recognized `.gitea/workflows/`, missing repos
|
||||||
|
already on `.forgejo/workflows/` — both now recognize either
|
||||||
|
- Fixed stale copy-paste examples in `docs/RegistryFederation.md`
|
||||||
|
(state-hub's old Gitea URL, now genuinely wrong post-migration) and a
|
||||||
|
pre-existing, unrelated port typo (`8088` vs the real llm-connect
|
||||||
|
default `8080`) in `tools/README.md`/`registry/README.md`, discovered
|
||||||
|
and confirmed live during REUSE-WP-0018-T03
|
||||||
|
- 17 new pytest cases (`tests/test_forge_host.py`); 106 total pass
|
||||||
|
- Recomposed `federated.yaml` post-migration: still 61 capabilities, no
|
||||||
|
loss
|
||||||
|
|
||||||
## Hub Recompose Endpoint And Webhook Receiver
|
## Hub Recompose Endpoint And Webhook Receiver
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user