diff --git a/.gitignore b/.gitignore index 3146226..8b8aaf8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,7 @@ +# Federation remote index cache (keep directory via .gitkeep) +registry/federation/cache/* +!registry/federation/cache/.gitkeep + # ---> Python # Byte-compiled / optimized / DLL files __pycache__/ diff --git a/SCOPE.md b/SCOPE.md index fce3bb7..b07d47e 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -55,6 +55,7 @@ and agents can: - **Browse a searchable catalog** at `docs/catalog/search.html` (client-side filter over `registry.json`) - **Compose federated indexes** with `reuse-surface federation compose` + (local paths and remote HTTP URLs with cache) - **Generate relation graphs** with `reuse-surface graph` - **Explore relations interactively** at `docs/graph/index.html` - **Avoid duplicates** by querying the index and checking overlaps before adding entries @@ -65,7 +66,7 @@ the index, and CLI automation. ## What Is Not Possible Yet -- Network-based federation or cross-org index sync +- Live cross-org index sync (polling/webhooks) beyond on-demand HTTP fetch - Packaged releases beyond local `pip install -e .` and Gitea CI validation See `tools/README.md` for command reference. @@ -84,7 +85,8 @@ See `tools/README.md` for command reference. `docs/graph/index.html`. - Searchable catalog: `docs/catalog/search.html`. - Test suite: `tests/` (pytest). -- Finished workplans: `REUSE-WP-0001` through `REUSE-WP-0009`. +- Remote federation: HTTP URL sources with cache in `registry/federation/cache/`. +- Finished workplans: `REUSE-WP-0001` through `REUSE-WP-0010`. - **Self-assessed vector:** `D5 / A3 / C4 / R3` (see gap analysis). ## Repository Layout diff --git a/docs/IntentScopeGapAnalysis.md b/docs/IntentScopeGapAnalysis.md index 019731b..0b3ea07 100644 --- a/docs/IntentScopeGapAnalysis.md +++ b/docs/IntentScopeGapAnalysis.md @@ -18,13 +18,12 @@ with **A3 CLI tooling** (`validate`, `query`, `export`) atop Markdown-first authoring. The two documents are **directionally aligned** on registry-first reuse, four -maturity dimensions, and human/agent consumers. REUSE-WP-0003 through -REUSE-WP-0009 closed the priority gaps from section 8 except network -federation. Remaining gaps are primarily scale and cross-org sync: +maturity dimensions, and human/agent consumers. REUSE-WP-0003 through REUSE-WP-0010 closed the priority gaps from section 8. +Remaining gaps are primarily document cross-coverage and operational polish: -1. **Network federation** — local compose only; no remote index fetch. -2. **Document cross-coverage** — SCOPE still carries operational detail INTENT +1. **Document cross-coverage** — SCOPE still carries operational detail INTENT omits; INTENT success criteria are not fully enumerated in SCOPE. +2. **Live federation sync** — on-demand HTTP fetch only; no polling or webhooks. **Current reuse-surface vector (self-assessment):** `D5 / A3 / C4 / R3` @@ -276,7 +275,7 @@ core commands. Individual registered capabilities may carry their own evidence | Priority | Gap | Outcome | Status | |---|---|---|---| | 13 | Interactive catalog | `docs/catalog/search.html` + `registry.json` | Closed (WP-0007) | -| 15 | Network federation | Remote index fetch and cross-org sync | Open (WP-0010) | +| 15 | Network federation | HTTP URL sources + cache in `federation compose` | Closed (WP-0010) | | 16 | Graph UI | `docs/graph/index.html` explorer | Closed (WP-0008) | --- @@ -302,4 +301,5 @@ core commands. Individual registered capabilities may carry their own evidence | 2026-06-15 | REUSE-WP-0006 expanded registry to 12 capabilities; relation hygiene clean | | 2026-06-15 | REUSE-WP-0007 closed priority 13 (searchable catalog UI) | | 2026-06-15 | REUSE-WP-0008 closed priority 16 (graph explorer) | -| 2026-06-15 | REUSE-WP-0009 added pytest suite and CI fail-on-warnings; vector R3 | \ No newline at end of file +| 2026-06-15 | REUSE-WP-0009 added pytest suite and CI fail-on-warnings; vector R3 | +| 2026-06-15 | REUSE-WP-0010 closed priority 15 (HTTP remote federation + cache) | \ No newline at end of file diff --git a/docs/RegistryFederation.md b/docs/RegistryFederation.md index 62eefc1..01a3e91 100644 --- a/docs/RegistryFederation.md +++ b/docs/RegistryFederation.md @@ -11,6 +11,10 @@ helix_forge capabilities may be registered in multiple repositories. Federation composes capability indexes from configured sources into a single discovery surface without silently merging duplicate IDs. +Sources may be **local filesystem paths** or **remote HTTP(S) URLs** (git raw +endpoints, published index artifacts, etc.). Remote indexes are cached under +`registry/federation/cache/` for offline reuse and faster compose. + ## Manifest `registry/federation/sources.yaml` lists index sources: @@ -24,6 +28,14 @@ sources: index: registry/indexes/capabilities.yaml enabled: true required: true + + - repo: sibling-repo + url: https://git.example.com/org/sibling-repo/raw/main/registry/indexes/capabilities.yaml + enabled: false + required: false + cache_ttl_seconds: 86400 + auth_env: FEDERATION_TOKEN + auth_header: Authorization ``` Schema: `schemas/federation.schema.yaml` @@ -33,26 +45,44 @@ Schema: `schemas/federation.schema.yaml` | Field | Meaning | |---|---| | `repo` | Source repository slug | -| `index` | Path to `capabilities.yaml` (repo-relative or `~/...`) | +| `index` | Local path to `capabilities.yaml` (repo-relative or `~/...`) | +| `url` | Remote HTTP(S) URL to a `capabilities.yaml` index | | `enabled` | Include this source in compose | -| `required` | Fail compose if index missing when enabled | +| `required` | Fail compose if index missing or remote fetch fails with no cache | | `domain` | Optional domain label | +| `cache_ttl_seconds` | Reuse cached remote index for this many seconds (`0` = always refetch) | +| `auth_env` | Environment variable holding token or full header value for `url` sources | +| `auth_header` | HTTP header for `auth_env` (default `Authorization`) | + +Each source must specify **either** `index` **or** `url`, not both. Sibling repos (`state-hub`, `feature-control`, `identity-canon`) are listed as -disabled placeholders until they publish registry indexes. +disabled local placeholders until they publish registry indexes. A disabled +`example-remote` URL source illustrates HTTP federation. ## Compose workflow ```bash reuse-surface federation compose +reuse-surface federation compose --refresh # bypass remote cache ``` Writes `registry/indexes/federated.yaml` with: - Merged `capabilities` from all enabled sources - `source_repo` and `source_index` on every row +- `source_url` when the row came from a remote source - `collision_policy` and per-source counts +### Remote cache + +Fetched URL indexes are stored at `registry/federation/cache/.yaml` with +metadata in `.meta.yaml`. The cache directory is gitignored; only +`.gitkeep` is tracked. + +When a refetch fails, compose reuses a stale cache and emits a warning. Required +remote sources without cache fail compose with a clear error. + ### Collision policy `warn` (default): duplicate IDs across sources are kept but reported as @@ -62,9 +92,17 @@ warnings. Consumers must inspect `source_repo` before choosing an entry. 1. Run `reuse-surface federation compose` after manifest or sibling index changes. 2. Read `registry/indexes/federated.yaml` for cross-repo discovery. -3. Open `path` in the source repo for full entry detail when local. +3. Open `path` in the source repo for full entry detail when local; follow + `source_url` / `source_index` when remote. 4. Run `reuse-surface graph --check` before relying on relation navigation. +### Cross-repo discovery without local checkout + +Enable a `url` source pointing at a published raw index (Gitea, GitHub, static +host). Set `auth_env` when the endpoint requires a token. Agents on machines +without sibling repo clones can still compose a federated view from HTTP sources +plus the local `reuse-surface` index. + ## Relation graphs ```bash @@ -82,9 +120,12 @@ federated ID set. Gitea CI runs: ```bash -reuse-surface validate --relations +reuse-surface validate --relations --fail-on-warnings reuse-surface federation compose +reuse-surface catalog +reuse-surface graph --check --fail-on-warnings +pytest -q ``` -Warnings on broken relations or missing optional sibling indexes do not fail CI; -schema validation errors do. \ No newline at end of file +CI uses local sources only (remote examples are disabled). Warnings on missing +optional sibling indexes do not fail CI; schema validation errors do. \ No newline at end of file diff --git a/registry/federation/cache/.gitkeep b/registry/federation/cache/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/registry/federation/sources.yaml b/registry/federation/sources.yaml index 458e6b9..83b1fd5 100644 --- a/registry/federation/sources.yaml +++ b/registry/federation/sources.yaml @@ -32,4 +32,13 @@ sources: enabled: false required: false domain: helix_forge - description: Identity canon research capabilities \ No newline at end of file + description: Identity canon research capabilities + + # Remote index example — enable when a repo publishes a raw capabilities.yaml URL + - repo: example-remote + url: https://example.com/registry/indexes/capabilities.yaml + enabled: false + required: false + domain: helix_forge + cache_ttl_seconds: 86400 + description: Illustrative HTTP federation source (disabled) \ No newline at end of file diff --git a/registry/indexes/federated.yaml b/registry/indexes/federated.yaml index e6c9589..6416232 100644 --- a/registry/indexes/federated.yaml +++ b/registry/indexes/federated.yaml @@ -6,8 +6,8 @@ domain: helix_forge collision_policy: warn sources: - repo: reuse-surface - index: registry/indexes/capabilities.yaml count: 12 + index: registry/indexes/capabilities.yaml capabilities: - id: capability.activity.event-coordinate name: Organizational Event Coordination diff --git a/reuse_surface/cli.py b/reuse_surface/cli.py index 4e983d0..5d541c9 100644 --- a/reuse_surface/cli.py +++ b/reuse_surface/cli.py @@ -146,7 +146,7 @@ def cmd_overlaps(args: argparse.Namespace) -> int: def cmd_federation_compose(args: argparse.Namespace) -> int: try: - target, warnings = write_federated_index() + target, warnings = write_federated_index(refresh=args.refresh) except (FileNotFoundError, ValueError) as exc: print(f"error: {exc}", file=sys.stderr) return 1 @@ -258,6 +258,11 @@ def main(argv: list[str] | None = None) -> int: ) federation_sub = federation.add_subparsers(dest="federation_command", required=True) compose = federation_sub.add_parser("compose", help="compose federated index") + compose.add_argument( + "--refresh", + action="store_true", + help="bypass remote index cache and refetch URL sources", + ) compose.set_defaults(func=cmd_federation_compose) query = subparsers.add_parser("query", help="query capability index") diff --git a/reuse_surface/federation.py b/reuse_surface/federation.py index 9b4261b..412f7b6 100644 --- a/reuse_surface/federation.py +++ b/reuse_surface/federation.py @@ -1,7 +1,10 @@ from __future__ import annotations -import sys -from datetime import date +import os +import re +import urllib.error +import urllib.request +from datetime import date, datetime, timezone from pathlib import Path from typing import Any @@ -13,6 +16,9 @@ from reuse_surface.registry import ROOT MANIFEST_PATH = ROOT / "registry" / "federation" / "sources.yaml" SCHEMA_PATH = ROOT / "schemas" / "federation.schema.yaml" FEDERATED_INDEX_PATH = ROOT / "registry" / "indexes" / "federated.yaml" +CACHE_DIR = ROOT / "registry" / "federation" / "cache" +DEFAULT_CACHE_TTL_SECONDS = 86400 +USER_AGENT = "reuse-surface/0.1 federation-compose" def _expand_path(index_path: str) -> Path: @@ -39,8 +45,139 @@ def _resolve_index_path(index_value: str) -> Path: return path +def _safe_repo_slug(repo: str) -> str: + return re.sub(r"[^\w.-]", "_", repo) + + +def _path_label(path: Path) -> str: + try: + return str(path.relative_to(ROOT)) + except ValueError: + return str(path) + + +def _cache_paths(repo: str) -> tuple[Path, Path]: + slug = _safe_repo_slug(repo) + return CACHE_DIR / f"{slug}.yaml", CACHE_DIR / f"{slug}.meta.yaml" + + +def _read_cache_meta(meta_path: Path) -> dict[str, Any] | None: + if not meta_path.exists(): + return None + data = yaml.safe_load(meta_path.read_text(encoding="utf-8")) + return data if isinstance(data, dict) else None + + +def _cache_is_fresh(meta: dict[str, Any], ttl_seconds: int) -> bool: + if ttl_seconds <= 0: + return False + fetched_at = meta.get("fetched_at") + if not fetched_at: + return False + fetched = datetime.fromisoformat(str(fetched_at)) + if fetched.tzinfo is None: + fetched = fetched.replace(tzinfo=timezone.utc) + age = (datetime.now(timezone.utc) - fetched).total_seconds() + return age < ttl_seconds + + +def _auth_headers(source: dict[str, Any]) -> dict[str, str]: + auth_env = source.get("auth_env") + if not auth_env: + return {} + token = os.environ.get(auth_env) + if not token: + raise ValueError( + f"auth env {auth_env} is not set for remote source {source['repo']}" + ) + header_name = source.get("auth_header", "Authorization") + if header_name.lower() == "authorization" and not token.lower().startswith( + ("bearer ", "basic ") + ): + token = f"Bearer {token}" + return {header_name: token} + + +def fetch_remote_index_text(url: str, source: dict[str, Any]) -> str: + headers = {"User-Agent": USER_AGENT, **_auth_headers(source)} + request = urllib.request.Request(url, headers=headers) + try: + with urllib.request.urlopen(request, timeout=30) as response: + return response.read().decode("utf-8") + except urllib.error.HTTPError as exc: + raise ConnectionError( + f"HTTP {exc.code} fetching {url} for {source['repo']}" + ) from exc + except urllib.error.URLError as exc: + raise ConnectionError( + f"failed to fetch {url} for {source['repo']}: {exc.reason}" + ) from exc + + +def _write_remote_cache(repo: str, url: str, content: str) -> Path: + CACHE_DIR.mkdir(parents=True, exist_ok=True) + index_path, meta_path = _cache_paths(repo) + index_path.write_text(content, encoding="utf-8") + meta = { + "fetched_at": datetime.now(timezone.utc).isoformat(), + "url": url, + "repo": repo, + } + meta_path.write_text(yaml.safe_dump(meta, sort_keys=False), encoding="utf-8") + return index_path + + +def resolve_source_index_path( + source: dict[str, Any], + *, + refresh: bool = False, +) -> tuple[Path | None, list[str]]: + warnings: list[str] = [] + if "index" in source: + path = _resolve_index_path(source["index"]) + if not path.exists(): + message = f"missing index for {source['repo']}: {path}" + if source.get("required", False): + raise FileNotFoundError(message) + warnings.append(message) + return None, warnings + return path, warnings + + url = source["url"] + ttl_seconds = int(source.get("cache_ttl_seconds", DEFAULT_CACHE_TTL_SECONDS)) + index_path, meta_path = _cache_paths(source["repo"]) + meta = _read_cache_meta(meta_path) + use_cache = ( + index_path.exists() + and meta is not None + and meta.get("url") == url + and _cache_is_fresh(meta, ttl_seconds) + and not refresh + ) + if use_cache: + return index_path, warnings + + try: + content = fetch_remote_index_text(url, source) + except (ConnectionError, ValueError) as exc: + if index_path.exists(): + warnings.append( + f"remote fetch failed for {source['repo']}, using stale cache: {exc}" + ) + return index_path, warnings + message = f"remote index unavailable for {source['repo']}: {exc}" + if source.get("required", False): + raise FileNotFoundError(message) from exc + warnings.append(message) + return None, warnings + + return _write_remote_cache(source["repo"], url, content), warnings + + def compose_federated_index( manifest: dict[str, Any] | None = None, + *, + refresh: bool = False, ) -> tuple[dict[str, Any], list[str]]: manifest = manifest or load_federation_manifest() warnings: list[str] = [] @@ -51,12 +188,11 @@ def compose_federated_index( for source in manifest["sources"]: if not source.get("enabled", False): continue - index_path = _resolve_index_path(source["index"]) - if not index_path.exists(): - message = f"missing index for {source['repo']}: {index_path}" - if source.get("required", False): - raise FileNotFoundError(message) - warnings.append(message) + index_path, source_warnings = resolve_source_index_path( + source, refresh=refresh + ) + warnings.extend(source_warnings) + if index_path is None: continue with index_path.open(encoding="utf-8") as handle: index_data = yaml.safe_load(handle) @@ -71,16 +207,23 @@ def compose_federated_index( seen_ids[cap_id] = source["repo"] federated_item = dict(item) federated_item["source_repo"] = source["repo"] - federated_item["source_index"] = source["index"] + if "url" in source: + federated_item["source_url"] = source["url"] + federated_item["source_index"] = _path_label(index_path) + else: + federated_item["source_index"] = source["index"] merged.append(federated_item) count += 1 - source_summaries.append( - { - "repo": source["repo"], - "index": source["index"], - "count": count, - } - ) + summary: dict[str, Any] = { + "repo": source["repo"], + "count": count, + } + if "url" in source: + summary["url"] = source["url"] + summary["cache"] = _path_label(index_path) + else: + summary["index"] = source["index"] + source_summaries.append(summary) federated = { "version": manifest.get("version", 1), @@ -96,8 +239,10 @@ def compose_federated_index( def write_federated_index( output_path: Path | None = None, manifest: dict[str, Any] | None = None, + *, + refresh: bool = False, ) -> tuple[Path, list[str]]: - federated, warnings = compose_federated_index(manifest) + federated, warnings = compose_federated_index(manifest, refresh=refresh) target = output_path or FEDERATED_INDEX_PATH target.parent.mkdir(parents=True, exist_ok=True) header = ( diff --git a/schemas/federation.schema.yaml b/schemas/federation.schema.yaml index 4a53a0d..0b2911e 100644 --- a/schemas/federation.schema.yaml +++ b/schemas/federation.schema.yaml @@ -2,8 +2,8 @@ $schema: https://json-schema.org/draft/2020-12/schema $id: https://reuse-surface.local/schemas/federation.schema.yaml title: Registry Federation Manifest description: > - Schema for registry/federation/sources.yaml. Describes local and sibling - capability index sources to compose into a federated index. + Schema for registry/federation/sources.yaml. Describes local filesystem and + remote HTTP capability index sources to compose into a federated index. type: object additionalProperties: false required: [version, domain, collision_policy, sources] @@ -25,7 +25,7 @@ $defs: source: type: object additionalProperties: false - required: [repo, index, enabled] + required: [repo, enabled] properties: repo: type: string @@ -33,6 +33,12 @@ $defs: index: type: string minLength: 1 + description: Local path to capabilities.yaml (repo-relative or ~/...) + url: + type: string + format: uri + pattern: '^https?://' + description: Remote HTTP(S) URL to a capabilities.yaml index enabled: type: boolean required: @@ -41,4 +47,23 @@ $defs: domain: type: string description: - type: string \ No newline at end of file + type: string + cache_ttl_seconds: + type: integer + minimum: 0 + default: 86400 + description: > + Seconds to reuse a cached remote index. 0 always refetches when + compose runs (stale cache still used as fallback on fetch failure). + auth_env: + type: string + minLength: 1 + description: Environment variable holding an Authorization token or full header value + auth_header: + type: string + minLength: 1 + default: Authorization + description: HTTP header name for auth_env value when fetching url sources + oneOf: + - required: [index] + - required: [url] \ No newline at end of file diff --git a/tests/test_federation.py b/tests/test_federation.py new file mode 100644 index 0000000..eafd490 --- /dev/null +++ b/tests/test_federation.py @@ -0,0 +1,208 @@ +from __future__ import annotations + +from pathlib import Path +from unittest.mock import patch + +import urllib.error +import yaml + +from reuse_surface.federation import ( + compose_federated_index, + fetch_remote_index_text, + load_federation_manifest, + resolve_source_index_path, +) + +ROOT = Path(__file__).resolve().parent.parent + +REMOTE_INDEX = """ +version: 1 +domain: helix_forge +updated: "2026-06-15" +capabilities: + - id: capability.remote.sample + name: Remote Sample + domain: helix_forge + vector: D2/A0/C0/R0 + owner: example + path: registry/capabilities/capability.remote.sample.md + summary: Sample capability from a remote index + tags: [sample] + consumption_modes: [planning] +""" + + +def _remote_manifest() -> dict: + return { + "version": 1, + "domain": "helix_forge", + "collision_policy": "warn", + "sources": [ + { + "repo": "local", + "index": "registry/indexes/capabilities.yaml", + "enabled": True, + "required": True, + }, + { + "repo": "remote-repo", + "url": "https://example.com/capabilities.yaml", + "enabled": True, + "required": False, + "cache_ttl_seconds": 3600, + }, + ], + } + + +def test_manifest_with_url_source_validates(): + manifest = _remote_manifest() + schema_path = ROOT / "schemas" / "federation.schema.yaml" + from jsonschema import Draft202012Validator + + schema = yaml.safe_load(schema_path.read_text(encoding="utf-8")) + errors = list(Draft202012Validator(schema).iter_errors(manifest)) + assert errors == [] + + +def test_fetch_remote_index_text(): + source = {"repo": "remote-repo", "url": "https://example.com/capabilities.yaml"} + payload = REMOTE_INDEX.encode("utf-8") + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def read(self): + return payload + + with patch("urllib.request.urlopen", return_value=FakeResponse()): + text = fetch_remote_index_text(source["url"], source) + assert "capability.remote.sample" in text + + +def test_remote_fetch_writes_cache(tmp_path, monkeypatch): + monkeypatch.setattr("reuse_surface.federation.CACHE_DIR", tmp_path / "cache") + source = { + "repo": "remote-repo", + "url": "https://example.com/capabilities.yaml", + "cache_ttl_seconds": 3600, + } + payload = REMOTE_INDEX.encode("utf-8") + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def read(self): + return payload + + with patch("urllib.request.urlopen", return_value=FakeResponse()): + path, warnings = resolve_source_index_path(source) + assert path is not None + assert warnings == [] + assert path.exists() + assert "capability.remote.sample" in path.read_text(encoding="utf-8") + + +def test_remote_fetch_uses_fresh_cache_without_refetch(tmp_path, monkeypatch): + monkeypatch.setattr("reuse_surface.federation.CACHE_DIR", tmp_path / "cache") + source = { + "repo": "remote-repo", + "url": "https://example.com/capabilities.yaml", + "cache_ttl_seconds": 3600, + } + payload = REMOTE_INDEX.encode("utf-8") + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def read(self): + return payload + + with patch("urllib.request.urlopen", return_value=FakeResponse()) as urlopen: + resolve_source_index_path(source) + path, warnings = resolve_source_index_path(source) + assert warnings == [] + assert path is not None + urlopen.assert_called_once() + + +def test_remote_fetch_falls_back_to_stale_cache(tmp_path, monkeypatch): + monkeypatch.setattr("reuse_surface.federation.CACHE_DIR", tmp_path / "cache") + source = { + "repo": "remote-repo", + "url": "https://example.com/capabilities.yaml", + "cache_ttl_seconds": 0, + } + payload = REMOTE_INDEX.encode("utf-8") + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def read(self): + return payload + + with patch("urllib.request.urlopen", return_value=FakeResponse()): + resolve_source_index_path(source) + + with patch( + "urllib.request.urlopen", + side_effect=urllib.error.URLError("network down"), + ): + path, warnings = resolve_source_index_path(source) + assert path is not None + assert any("stale cache" in warning for warning in warnings) + + +def test_compose_merges_remote_capabilities(tmp_path, monkeypatch): + monkeypatch.setattr("reuse_surface.federation.CACHE_DIR", tmp_path / "cache") + source = { + "repo": "remote-repo", + "url": "https://example.com/capabilities.yaml", + "enabled": True, + "cache_ttl_seconds": 3600, + } + payload = REMOTE_INDEX.encode("utf-8") + + class FakeResponse: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def read(self): + return payload + + manifest = _remote_manifest() + with patch("urllib.request.urlopen", return_value=FakeResponse()): + federated, warnings = compose_federated_index(manifest) + ids = {item["id"] for item in federated["capabilities"]} + assert "capability.remote.sample" in ids + assert "capability.registry.register" in ids + remote = next( + item for item in federated["capabilities"] if item["id"] == "capability.remote.sample" + ) + assert remote["source_repo"] == "remote-repo" + assert remote["source_url"] == "https://example.com/capabilities.yaml" + + +def test_load_production_manifest_still_validates(): + manifest = load_federation_manifest() + assert manifest["domain"] == "helix_forge" + assert any(source["repo"] == "reuse-surface" for source in manifest["sources"]) \ No newline at end of file diff --git a/tools/README.md b/tools/README.md index a5cd6a2..1920c84 100644 --- a/tools/README.md +++ b/tools/README.md @@ -68,9 +68,12 @@ Compose a federated index from `registry/federation/sources.yaml`. ```bash reuse-surface federation compose +reuse-surface federation compose --refresh ``` -Writes `registry/indexes/federated.yaml` with `source_repo` attribution. +Composes local and remote HTTP index sources. Writes +`registry/indexes/federated.yaml` with `source_repo` attribution. Remote indexes +cache under `registry/federation/cache/`. ### graph diff --git a/workplans/REUSE-WP-0010-network-federation.md b/workplans/REUSE-WP-0010-network-federation.md index e567572..5f24c15 100644 --- a/workplans/REUSE-WP-0010-network-federation.md +++ b/workplans/REUSE-WP-0010-network-federation.md @@ -4,7 +4,7 @@ type: workplan title: "Network federation for remote indexes" domain: helix_forge repo: reuse-surface -status: backlog +status: finished owner: codex topic_slug: helix-forge created: "2026-06-15" @@ -21,7 +21,7 @@ fetch capability indexes from HTTP URLs or git raw endpoints. ```task id: REUSE-WP-0010-T01 -status: todo +status: done priority: medium state_hub_task_id: "6f181057-e0f7-4879-9eb9-928a527a01ee" ``` @@ -33,7 +33,7 @@ sources alongside `index` file paths, with optional auth and TTL metadata. ```task id: REUSE-WP-0010-T02 -status: todo +status: done priority: medium state_hub_task_id: "a2fac7d5-9383-4a42-bd23-3e8dbc7d550b" ``` @@ -45,7 +45,7 @@ Add HTTP fetch to `federation compose` with local cache under ```task id: REUSE-WP-0010-T03 -status: todo +status: done priority: low state_hub_task_id: "73996193-ecae-4fb4-84f7-fe84a5cd8898" ```