generated from coulomb/repo-seed
Complete REUSE-WP-0005: registry federation and relation graphs
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
Add federation manifest and schema, federation compose and graph CLI commands, relation cycle/reference checks, federated index and Mermaid graph artifacts, RegistryFederation guide, and CI validation updates.
This commit is contained in:
@@ -22,4 +22,7 @@ jobs:
|
||||
run: python -m pip install -e .
|
||||
|
||||
- name: Validate capability registry
|
||||
run: reuse-surface validate
|
||||
run: reuse-surface validate --relations
|
||||
|
||||
- name: Compose federated index
|
||||
run: reuse-surface federation compose
|
||||
@@ -124,9 +124,11 @@ artifacts.
|
||||
# Registry validation (schema + index drift)
|
||||
.venv/bin/reuse-surface validate
|
||||
|
||||
# Overlap and catalog generation
|
||||
# Overlap, catalog, federation, and graph
|
||||
.venv/bin/reuse-surface overlaps
|
||||
.venv/bin/reuse-surface catalog
|
||||
.venv/bin/reuse-surface federation compose
|
||||
.venv/bin/reuse-surface graph --check
|
||||
|
||||
# Repository hygiene
|
||||
rg --files
|
||||
@@ -182,7 +184,8 @@ implementation reuse.
|
||||
### Orient
|
||||
|
||||
```bash
|
||||
# Fast discovery surface — read this first
|
||||
# Fast discovery surface — read federated index when multi-repo
|
||||
cat registry/indexes/federated.yaml
|
||||
cat registry/indexes/capabilities.yaml
|
||||
|
||||
# CLI discovery and export
|
||||
|
||||
15
SCOPE.md
15
SCOPE.md
@@ -52,6 +52,8 @@ and agents can:
|
||||
- **Export a machine-readable bundle** with `reuse-surface export`
|
||||
- **Detect overlap candidates** with `reuse-surface overlaps`
|
||||
- **Generate a human-readable catalog** with `reuse-surface catalog`
|
||||
- **Compose federated indexes** with `reuse-surface federation compose`
|
||||
- **Generate relation graphs** with `reuse-surface graph`
|
||||
- **Avoid duplicates** by querying the index and checking overlaps before adding entries
|
||||
|
||||
Registry tooling availability is **A3** (CLI). The registry product itself is
|
||||
@@ -61,8 +63,8 @@ the index, and CLI automation.
|
||||
## What Is Not Possible Yet
|
||||
|
||||
- Interactive catalog site with live search beyond static HTML export
|
||||
- Capability graph visualization
|
||||
- Federation across repositories or organizations
|
||||
- Interactive relation graph UI (Mermaid file only)
|
||||
- Network-based federation or cross-org index sync
|
||||
- Packaged releases beyond local `pip install -e .` and Gitea CI validation
|
||||
|
||||
See `tools/README.md` for command reference.
|
||||
@@ -75,9 +77,10 @@ See `tools/README.md` for command reference.
|
||||
`pyproject.toml` and `reuse_surface/`.
|
||||
- `docs/CapabilityRegistryConcept.md` and `docs/IntentScopeGapAnalysis.md`
|
||||
document onboarding and intent-scope tracking.
|
||||
- CI validates the registry on push/PR via `.gitea/workflows/ci.yml`.
|
||||
- Generated catalog: `docs/CapabilityCatalog.md` and `docs/catalog/index.html`.
|
||||
- Finished workplans: `REUSE-WP-0001` through `REUSE-WP-0004`.
|
||||
- CI validates the registry and composes federation on push/PR.
|
||||
- Federated index: `registry/indexes/federated.yaml`.
|
||||
- Relation graph: `docs/graph/capability-graph.mmd`.
|
||||
- Finished workplans: `REUSE-WP-0001` through `REUSE-WP-0005`.
|
||||
- **Self-assessed vector:** `D5 / A3 / C4 / R2` (see gap analysis).
|
||||
|
||||
## Repository Layout
|
||||
@@ -109,6 +112,8 @@ reuse-surface/
|
||||
- Registry index: registry/indexes/capabilities.yaml
|
||||
- Registry guidance: registry/README.md
|
||||
- Generated catalog: docs/CapabilityCatalog.md
|
||||
- Federation guide: docs/RegistryFederation.md
|
||||
- Relation graph: docs/graph/capability-graph.mmd
|
||||
- CLI reference: tools/README.md
|
||||
- Agent instructions: AGENTS.md
|
||||
- Workplans: workplans/
|
||||
@@ -270,13 +270,14 @@ own evidence (e.g. feature-control at R3).
|
||||
| 9 | Catalog site | `reuse-surface catalog` → MD + HTML | Closed (WP-0004) |
|
||||
| 10 | Overlap detection | `reuse-surface overlaps` | Closed (WP-0004) |
|
||||
| 11 | CI validation | `.gitea/workflows/ci.yml` | Closed (WP-0004) |
|
||||
| 12 | Registry federation | Cross-repo capability index composition | Open |
|
||||
| 12 | Registry federation | `federation compose` + federated index | Closed (WP-0005) |
|
||||
| 14 | Graph visualization | `reuse-surface graph` Mermaid output | Closed (WP-0005) |
|
||||
|
||||
| Priority | Gap | Suggested outcome |
|
||||
|---|---|---|
|
||||
| 13 | Interactive catalog | Searchable catalog UI beyond static HTML |
|
||||
| 14 | Graph visualization | Capability relation graphs |
|
||||
| 15 | Federation | Compose indexes across repositories |
|
||||
| 15 | Network federation | Remote index fetch and cross-org sync |
|
||||
| 16 | Graph UI | Interactive relation graph explorer |
|
||||
|
||||
---
|
||||
|
||||
@@ -296,4 +297,5 @@ own evidence (e.g. feature-control at R3).
|
||||
|---|---|
|
||||
| 2026-06-15 | Initial analysis after REUSE-WP-0002 completion |
|
||||
| 2026-06-15 | REUSE-WP-0003 closed priority gaps 1–8; vector updated to D5/A3/C4/R2 |
|
||||
| 2026-06-15 | REUSE-WP-0004 closed priorities 9–11 (catalog, overlaps, CI) |
|
||||
| 2026-06-15 | REUSE-WP-0004 closed priorities 9–11 (catalog, overlaps, CI) |
|
||||
| 2026-06-15 | REUSE-WP-0005 closed priorities 12 and 14 (federation, relation graphs) |
|
||||
90
docs/RegistryFederation.md
Normal file
90
docs/RegistryFederation.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# Registry Federation
|
||||
|
||||
**Repository:** `reuse-surface`
|
||||
**Audience:** Architects and agents composing multi-repo capability indexes
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
helix_forge capabilities may be registered in multiple repositories. Federation
|
||||
composes capability indexes from configured sources into a single discovery
|
||||
surface without silently merging duplicate IDs.
|
||||
|
||||
## Manifest
|
||||
|
||||
`registry/federation/sources.yaml` lists index sources:
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
domain: helix_forge
|
||||
collision_policy: warn
|
||||
sources:
|
||||
- repo: reuse-surface
|
||||
index: registry/indexes/capabilities.yaml
|
||||
enabled: true
|
||||
required: true
|
||||
```
|
||||
|
||||
Schema: `schemas/federation.schema.yaml`
|
||||
|
||||
### Source fields
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `repo` | Source repository slug |
|
||||
| `index` | Path to `capabilities.yaml` (repo-relative or `~/...`) |
|
||||
| `enabled` | Include this source in compose |
|
||||
| `required` | Fail compose if index missing when enabled |
|
||||
| `domain` | Optional domain label |
|
||||
|
||||
Sibling repos (`state-hub`, `feature-control`, `identity-canon`) are listed as
|
||||
disabled placeholders until they publish registry indexes.
|
||||
|
||||
## Compose workflow
|
||||
|
||||
```bash
|
||||
reuse-surface federation compose
|
||||
```
|
||||
|
||||
Writes `registry/indexes/federated.yaml` with:
|
||||
|
||||
- Merged `capabilities` from all enabled sources
|
||||
- `source_repo` and `source_index` on every row
|
||||
- `collision_policy` and per-source counts
|
||||
|
||||
### Collision policy
|
||||
|
||||
`warn` (default): duplicate IDs across sources are kept but reported as
|
||||
warnings. Consumers must inspect `source_repo` before choosing an entry.
|
||||
|
||||
## Agent query pattern
|
||||
|
||||
1. Run `reuse-surface federation compose` after manifest or sibling index changes.
|
||||
2. Read `registry/indexes/federated.yaml` for cross-repo discovery.
|
||||
3. Open `path` in the source repo for full entry detail when local.
|
||||
4. Run `reuse-surface graph --check` before relying on relation navigation.
|
||||
|
||||
## Relation graphs
|
||||
|
||||
```bash
|
||||
reuse-surface graph
|
||||
reuse-surface graph --check
|
||||
reuse-surface graph --stdout
|
||||
```
|
||||
|
||||
Generates `docs/graph/capability-graph.mmd` from local entry `relations`.
|
||||
`--check` reports `depends_on` cycles and broken relation targets against the
|
||||
federated ID set.
|
||||
|
||||
## CI integration
|
||||
|
||||
Gitea CI runs:
|
||||
|
||||
```bash
|
||||
reuse-surface validate --relations
|
||||
reuse-surface federation compose
|
||||
```
|
||||
|
||||
Warnings on broken relations or missing optional sibling indexes do not fail CI;
|
||||
schema validation errors do.
|
||||
24
docs/graph/capability-graph.mmd
Normal file
24
docs/graph/capability-graph.mmd
Normal file
@@ -0,0 +1,24 @@
|
||||
graph LR
|
||||
capability_feature_control_evaluate["capability.feature-control.evaluate<br/>D5 / A4 / C3 / R3"]
|
||||
capability_feature_control_rollout["capability.feature-control.rollout<br/>D4 / A2 / C2 / R1"]
|
||||
capability_identity_subject_resolution["capability.identity.subject-resolution<br/>D3 / A0 / C1 / R0"]
|
||||
capability_identity_vocabulary_canonicalize["capability.identity.vocabulary-canonicalize<br/>D4 / A0 / C2 / R0"]
|
||||
capability_registry_register["capability.registry.register<br/>D3 / A3 / C2 / R2"]
|
||||
capability_statehub_workstream_coordinate["capability.statehub.workstream-coordinate<br/>D4 / A4 / C3 / R2"]
|
||||
capability_registry_register -->|supports| capability_feature_control_evaluate
|
||||
capability_registry_register -->|supports| capability_identity_vocabulary_canonicalize
|
||||
capability_registry_register -->|related_to| capability_registry_validate
|
||||
capability_feature_control_evaluate -->|depends_on| capability_identity_vocabulary_canonicalize
|
||||
capability_feature_control_evaluate -->|supports| capability_registry_register
|
||||
capability_feature_control_evaluate -->|related_to| capability_feature_control_rollout
|
||||
capability_feature_control_evaluate -->|related_to| capability_feature_control_visibility
|
||||
capability_feature_control_rollout -->|depends_on| capability_feature_control_evaluate
|
||||
capability_feature_control_rollout -->|related_to| capability_feature_control_visibility
|
||||
capability_identity_vocabulary_canonicalize -->|supports| capability_feature_control_evaluate
|
||||
capability_identity_vocabulary_canonicalize -->|supports| capability_registry_register
|
||||
capability_identity_vocabulary_canonicalize -->|related_to| capability_identity_subject_resolution
|
||||
capability_identity_subject_resolution -->|depends_on| capability_identity_vocabulary_canonicalize
|
||||
capability_identity_subject_resolution -->|supports| capability_feature_control_evaluate
|
||||
capability_identity_subject_resolution -->|supports| capability_statehub_workstream_coordinate
|
||||
capability_statehub_workstream_coordinate -->|supports| capability_registry_register
|
||||
capability_statehub_workstream_coordinate -->|related_to| capability_statehub_progress_log
|
||||
@@ -31,7 +31,7 @@ registry/
|
||||
- `external_evidence.completeness.level: C0`
|
||||
- `external_evidence.reliability.level: R0`
|
||||
4. Add the entry to `registry/indexes/capabilities.yaml`.
|
||||
5. Run the manual validation checklist below.
|
||||
5. Run `reuse-surface validate --relations` and `reuse-surface federation compose`.
|
||||
|
||||
Missing evidence is acceptable in the MVP when it is explicit rather than hidden.
|
||||
|
||||
@@ -129,6 +129,17 @@ Check for overlap in:
|
||||
When overlap is real, link entries with `relations.related_to`, `specializes`,
|
||||
or `generalizes` rather than creating silent duplicates.
|
||||
|
||||
## Relation graph
|
||||
|
||||
Regenerate the Mermaid graph after relation changes:
|
||||
|
||||
```bash
|
||||
reuse-surface graph
|
||||
reuse-surface graph --check
|
||||
```
|
||||
|
||||
Output: `docs/graph/capability-graph.mmd`
|
||||
|
||||
## Promote a capability
|
||||
|
||||
1. Attach evidence appropriate to the target level in
|
||||
@@ -137,5 +148,6 @@ or `generalizes` rather than creating silent duplicates.
|
||||
3. Append a `promotion_history` record with `date`, `dimension`, `from`, `to`,
|
||||
and `rationale` (optional `author`).
|
||||
4. Update `availability.current_artifacts` when a new consumption mode appears.
|
||||
5. Refresh the index `vector` and run `reuse-surface validate`.
|
||||
5. Refresh the index `vector` and run `reuse-surface validate --relations`.
|
||||
6. Run `reuse-surface federation compose` and `reuse-surface graph`.
|
||||
6. Set `status: reviewed` or `approved` when an assessor validates the entry.
|
||||
35
registry/federation/sources.yaml
Normal file
35
registry/federation/sources.yaml
Normal file
@@ -0,0 +1,35 @@
|
||||
# Federation manifest for helix_forge capability indexes.
|
||||
# Compose with: reuse-surface federation compose
|
||||
version: 1
|
||||
domain: helix_forge
|
||||
collision_policy: warn
|
||||
|
||||
sources:
|
||||
- repo: reuse-surface
|
||||
index: registry/indexes/capabilities.yaml
|
||||
enabled: true
|
||||
required: true
|
||||
domain: helix_forge
|
||||
description: Primary local capability registry
|
||||
|
||||
# Enable when sibling repos publish registry/indexes/capabilities.yaml
|
||||
- repo: state-hub
|
||||
index: ~/state-hub/registry/indexes/capabilities.yaml
|
||||
enabled: false
|
||||
required: false
|
||||
domain: helix_forge
|
||||
description: State Hub coordination capabilities
|
||||
|
||||
- repo: feature-control
|
||||
index: ~/feature-control/registry/indexes/capabilities.yaml
|
||||
enabled: false
|
||||
required: false
|
||||
domain: helix_forge
|
||||
description: Feature control domain capabilities
|
||||
|
||||
- repo: identity-canon
|
||||
index: ~/identity-canon/registry/indexes/capabilities.yaml
|
||||
enabled: false
|
||||
required: false
|
||||
domain: helix_forge
|
||||
description: Identity canon research capabilities
|
||||
118
registry/indexes/federated.yaml
Normal file
118
registry/indexes/federated.yaml
Normal file
@@ -0,0 +1,118 @@
|
||||
# Composed federated capability index. Regenerate with:
|
||||
# reuse-surface federation compose
|
||||
version: 1
|
||||
updated: '2026-06-15'
|
||||
domain: helix_forge
|
||||
collision_policy: warn
|
||||
sources:
|
||||
- repo: reuse-surface
|
||||
index: registry/indexes/capabilities.yaml
|
||||
count: 6
|
||||
capabilities:
|
||||
- id: capability.feature-control.evaluate
|
||||
name: Feature Availability Evaluation
|
||||
summary: Evaluate whether a feature is active, hidden, disabled, or unavailable
|
||||
for a subject in context.
|
||||
vector: D5 / A4 / C3 / R3
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: feature-control
|
||||
path: registry/capabilities/capability.feature-control.evaluate.md
|
||||
tags:
|
||||
- feature-control
|
||||
- evaluation
|
||||
- sdk
|
||||
consumption_modes:
|
||||
- SDK
|
||||
- service API
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
- id: capability.feature-control.rollout
|
||||
name: Feature Rollout Control
|
||||
summary: Gradually expose features to subjects across tenants, domains, groups,
|
||||
or cohorts using rollout rules and staged availability.
|
||||
vector: D4 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: feature-control
|
||||
path: registry/capabilities/capability.feature-control.rollout.md
|
||||
tags:
|
||||
- feature-control
|
||||
- rollout
|
||||
- planning
|
||||
consumption_modes:
|
||||
- source module
|
||||
- SDK
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
- id: capability.identity.subject-resolution
|
||||
name: Identity Subject Resolution
|
||||
summary: Resolve who or what is acting in a context by mapping principals, accounts,
|
||||
actors, and identifiers to a stable subject model.
|
||||
vector: D3 / A0 / C1 / R0
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: identity-canon
|
||||
path: registry/capabilities/capability.identity.subject-resolution.md
|
||||
tags:
|
||||
- identity
|
||||
- subject
|
||||
- architecture
|
||||
consumption_modes:
|
||||
- informational
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
- id: capability.identity.vocabulary-canonicalize
|
||||
name: Identity Vocabulary Canonicalization
|
||||
summary: Define and maintain an implementation-neutral vocabulary for identity-related
|
||||
concepts across overlapping domains.
|
||||
vector: D4 / A0 / C2 / R0
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: identity-canon
|
||||
path: registry/capabilities/capability.identity.vocabulary-canonicalize.md
|
||||
tags:
|
||||
- identity
|
||||
- terminology
|
||||
- research
|
||||
consumption_modes:
|
||||
- informational
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
- id: capability.registry.register
|
||||
name: Capability Registration
|
||||
summary: Register a new capability so it becomes visible for planning and implementation
|
||||
reuse.
|
||||
vector: D3 / A3 / C2 / R2
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: reuse-surface
|
||||
path: registry/capabilities/capability.registry.register.md
|
||||
tags:
|
||||
- registry
|
||||
- governance
|
||||
- meta
|
||||
consumption_modes:
|
||||
- informational
|
||||
- markdown authoring
|
||||
- cli
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
- id: capability.statehub.workstream-coordinate
|
||||
name: Workstream And Task Coordination
|
||||
summary: Track active workstreams, tasks, progress, and consistency across domain
|
||||
repositories through a local-first coordination service.
|
||||
vector: D4 / A4 / C3 / R2
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: state-hub
|
||||
path: registry/capabilities/capability.statehub.workstream-coordinate.md
|
||||
tags:
|
||||
- state-hub
|
||||
- coordination
|
||||
- workplans
|
||||
consumption_modes:
|
||||
- service API
|
||||
- HTTP REST
|
||||
source_repo: reuse-surface
|
||||
source_index: registry/indexes/capabilities.yaml
|
||||
@@ -10,6 +10,8 @@ import yaml
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from reuse_surface.catalog import write_catalog
|
||||
from reuse_surface.federation import write_federated_index
|
||||
from reuse_surface.graph import check_relations, render_mermaid, write_graph
|
||||
from reuse_surface.overlaps import find_overlaps
|
||||
from reuse_surface.registry import (
|
||||
ROOT,
|
||||
@@ -54,6 +56,8 @@ def cmd_validate(args: argparse.Namespace) -> int:
|
||||
if not target:
|
||||
index = load_index()
|
||||
warnings.extend(_check_index_drift(paths, index))
|
||||
if args.relations:
|
||||
warnings.extend(check_relations())
|
||||
|
||||
for warning in warnings:
|
||||
print(f"warning: {warning}", file=sys.stderr)
|
||||
@@ -140,6 +144,35 @@ def cmd_overlaps(args: argparse.Namespace) -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_federation_compose(args: argparse.Namespace) -> int:
|
||||
try:
|
||||
target, warnings = write_federated_index()
|
||||
except (FileNotFoundError, ValueError) as exc:
|
||||
print(f"error: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
for warning in warnings:
|
||||
print(f"warning: {warning}", file=sys.stderr)
|
||||
import yaml
|
||||
|
||||
data = yaml.safe_load(target.read_text(encoding="utf-8"))
|
||||
count = len(data.get("capabilities", []))
|
||||
print(f"ok: wrote {target.relative_to(ROOT)} ({count} capabilities)")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_graph(args: argparse.Namespace) -> int:
|
||||
warnings = check_relations() if args.check else []
|
||||
content = render_mermaid()
|
||||
if args.stdout:
|
||||
print(content, end="")
|
||||
else:
|
||||
path = write_graph()
|
||||
print(f"ok: wrote {path.relative_to(ROOT)}")
|
||||
for warning in warnings:
|
||||
print(f"warning: {warning}", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_catalog(args: argparse.Namespace) -> int:
|
||||
index = load_index()
|
||||
indexed_entries = _load_indexed_entries()
|
||||
@@ -199,8 +232,20 @@ def main(argv: list[str] | None = None) -> int:
|
||||
nargs="?",
|
||||
help="optional capability markdown file; defaults to all entries",
|
||||
)
|
||||
validate.add_argument(
|
||||
"--relations",
|
||||
action="store_true",
|
||||
help="check relation cycles and broken references",
|
||||
)
|
||||
validate.set_defaults(func=cmd_validate)
|
||||
|
||||
federation = subparsers.add_parser(
|
||||
"federation", help="federation index operations"
|
||||
)
|
||||
federation_sub = federation.add_subparsers(dest="federation_command", required=True)
|
||||
compose = federation_sub.add_parser("compose", help="compose federated index")
|
||||
compose.set_defaults(func=cmd_federation_compose)
|
||||
|
||||
query = subparsers.add_parser("query", help="query capability index")
|
||||
query.add_argument("--discovery-min")
|
||||
query.add_argument("--availability-min")
|
||||
@@ -234,6 +279,19 @@ def main(argv: list[str] | None = None) -> int:
|
||||
)
|
||||
catalog.set_defaults(func=cmd_catalog)
|
||||
|
||||
graph = subparsers.add_parser("graph", help="generate relation graph")
|
||||
graph.add_argument(
|
||||
"--stdout",
|
||||
action="store_true",
|
||||
help="print Mermaid to stdout instead of writing docs/graph/",
|
||||
)
|
||||
graph.add_argument(
|
||||
"--check",
|
||||
action="store_true",
|
||||
help="report depends_on cycles and broken relation references",
|
||||
)
|
||||
graph.set_defaults(func=cmd_graph)
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
return args.func(args)
|
||||
|
||||
|
||||
111
reuse_surface/federation.py
Normal file
111
reuse_surface/federation.py
Normal file
@@ -0,0 +1,111 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
from jsonschema import Draft202012Validator
|
||||
|
||||
from reuse_surface.registry import ROOT
|
||||
|
||||
MANIFEST_PATH = ROOT / "registry" / "federation" / "sources.yaml"
|
||||
SCHEMA_PATH = ROOT / "schemas" / "federation.schema.yaml"
|
||||
FEDERATED_INDEX_PATH = ROOT / "registry" / "indexes" / "federated.yaml"
|
||||
|
||||
|
||||
def _expand_path(index_path: str) -> Path:
|
||||
return Path(index_path).expanduser()
|
||||
|
||||
|
||||
def load_federation_manifest(path: Path | None = None) -> dict[str, Any]:
|
||||
manifest_path = path or MANIFEST_PATH
|
||||
with manifest_path.open(encoding="utf-8") as handle:
|
||||
manifest = yaml.safe_load(handle)
|
||||
schema = yaml.safe_load(SCHEMA_PATH.read_text(encoding="utf-8"))
|
||||
validator = Draft202012Validator(schema)
|
||||
errors = sorted(validator.iter_errors(manifest), key=lambda err: err.path)
|
||||
if errors:
|
||||
messages = "; ".join(error.message for error in errors)
|
||||
raise ValueError(f"invalid federation manifest: {messages}")
|
||||
return manifest
|
||||
|
||||
|
||||
def _resolve_index_path(index_value: str) -> Path:
|
||||
path = _expand_path(index_value)
|
||||
if not path.is_absolute():
|
||||
path = (ROOT / path).resolve()
|
||||
return path
|
||||
|
||||
|
||||
def compose_federated_index(
|
||||
manifest: dict[str, Any] | None = None,
|
||||
) -> tuple[dict[str, Any], list[str]]:
|
||||
manifest = manifest or load_federation_manifest()
|
||||
warnings: list[str] = []
|
||||
merged: list[dict[str, Any]] = []
|
||||
seen_ids: dict[str, str] = {}
|
||||
source_summaries: list[dict[str, Any]] = []
|
||||
|
||||
for source in manifest["sources"]:
|
||||
if not source.get("enabled", False):
|
||||
continue
|
||||
index_path = _resolve_index_path(source["index"])
|
||||
if not index_path.exists():
|
||||
message = f"missing index for {source['repo']}: {index_path}"
|
||||
if source.get("required", False):
|
||||
raise FileNotFoundError(message)
|
||||
warnings.append(message)
|
||||
continue
|
||||
with index_path.open(encoding="utf-8") as handle:
|
||||
index_data = yaml.safe_load(handle)
|
||||
count = 0
|
||||
for item in index_data.get("capabilities", []):
|
||||
cap_id = item["id"]
|
||||
if cap_id in seen_ids:
|
||||
warnings.append(
|
||||
f"duplicate id {cap_id}: {seen_ids[cap_id]} and {source['repo']}"
|
||||
)
|
||||
else:
|
||||
seen_ids[cap_id] = source["repo"]
|
||||
federated_item = dict(item)
|
||||
federated_item["source_repo"] = source["repo"]
|
||||
federated_item["source_index"] = source["index"]
|
||||
merged.append(federated_item)
|
||||
count += 1
|
||||
source_summaries.append(
|
||||
{
|
||||
"repo": source["repo"],
|
||||
"index": source["index"],
|
||||
"count": count,
|
||||
}
|
||||
)
|
||||
|
||||
federated = {
|
||||
"version": manifest.get("version", 1),
|
||||
"updated": date.today().isoformat(),
|
||||
"domain": manifest.get("domain"),
|
||||
"collision_policy": manifest.get("collision_policy", "warn"),
|
||||
"sources": source_summaries,
|
||||
"capabilities": sorted(merged, key=lambda item: item["id"]),
|
||||
}
|
||||
return federated, warnings
|
||||
|
||||
|
||||
def write_federated_index(
|
||||
output_path: Path | None = None,
|
||||
manifest: dict[str, Any] | None = None,
|
||||
) -> tuple[Path, list[str]]:
|
||||
federated, warnings = compose_federated_index(manifest)
|
||||
target = output_path or FEDERATED_INDEX_PATH
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
header = (
|
||||
"# Composed federated capability index. Regenerate with:\n"
|
||||
"# reuse-surface federation compose\n"
|
||||
)
|
||||
target.write_text(
|
||||
header + yaml.safe_dump(federated, sort_keys=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return target, warnings
|
||||
153
reuse_surface/graph.py
Normal file
153
reuse_surface/graph.py
Normal file
@@ -0,0 +1,153 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from reuse_surface.federation import FEDERATED_INDEX_PATH, compose_federated_index
|
||||
from reuse_surface.registry import ROOT, load_index, parse_front_matter
|
||||
|
||||
GRAPH_PATH = ROOT / "docs" / "graph" / "capability-graph.mmd"
|
||||
RELATION_TYPES = [
|
||||
"depends_on",
|
||||
"supports",
|
||||
"used_by",
|
||||
"related_to",
|
||||
"specializes",
|
||||
"generalizes",
|
||||
"replaces",
|
||||
"wraps",
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class RelationEdge:
|
||||
source_id: str
|
||||
target_id: str
|
||||
relation_type: str
|
||||
|
||||
|
||||
def _node_id(capability_id: str) -> str:
|
||||
return re.sub(r"[^a-zA-Z0-9_]", "_", capability_id)
|
||||
|
||||
|
||||
def _load_local_relations() -> dict[str, dict[str, list[str]]]:
|
||||
index = load_index()
|
||||
relations_by_id: dict[str, dict[str, list[str]]] = {}
|
||||
for item in index.get("capabilities", []):
|
||||
path = ROOT / item["path"]
|
||||
if not path.exists():
|
||||
continue
|
||||
entry = parse_front_matter(path)
|
||||
relations = entry.get("relations") or {}
|
||||
relations_by_id[entry["id"]] = {
|
||||
relation_type: list(targets)
|
||||
for relation_type, targets in relations.items()
|
||||
if isinstance(targets, list)
|
||||
}
|
||||
return relations_by_id
|
||||
|
||||
|
||||
def _known_ids() -> set[str]:
|
||||
if FEDERATED_INDEX_PATH.exists():
|
||||
import yaml
|
||||
|
||||
data = yaml.safe_load(FEDERATED_INDEX_PATH.read_text(encoding="utf-8"))
|
||||
else:
|
||||
data, _ = compose_federated_index()
|
||||
return {item["id"] for item in data.get("capabilities", [])}
|
||||
|
||||
|
||||
def collect_edges() -> list[RelationEdge]:
|
||||
relations_by_id = _load_local_relations()
|
||||
edges: list[RelationEdge] = []
|
||||
for source_id, relation_map in relations_by_id.items():
|
||||
for relation_type, targets in relation_map.items():
|
||||
for target_id in targets:
|
||||
edges.append(
|
||||
RelationEdge(
|
||||
source_id=source_id,
|
||||
target_id=target_id,
|
||||
relation_type=relation_type,
|
||||
)
|
||||
)
|
||||
return edges
|
||||
|
||||
|
||||
def find_depends_on_cycles() -> list[list[str]]:
|
||||
relations_by_id = _load_local_relations()
|
||||
graph: dict[str, list[str]] = {
|
||||
cap_id: list(relation_map.get("depends_on", []))
|
||||
for cap_id, relation_map in relations_by_id.items()
|
||||
}
|
||||
cycles: list[list[str]] = []
|
||||
visited: set[str] = set()
|
||||
stack: set[str] = set()
|
||||
path: list[str] = []
|
||||
|
||||
def dfs(node: str) -> None:
|
||||
visited.add(node)
|
||||
stack.add(node)
|
||||
path.append(node)
|
||||
for neighbor in graph.get(node, []):
|
||||
if neighbor not in visited:
|
||||
dfs(neighbor)
|
||||
elif neighbor in stack:
|
||||
start = path.index(neighbor)
|
||||
cycles.append(path[start:] + [neighbor])
|
||||
path.pop()
|
||||
stack.remove(node)
|
||||
|
||||
for node in graph:
|
||||
if node not in visited:
|
||||
dfs(node)
|
||||
return cycles
|
||||
|
||||
|
||||
def find_broken_references(known: set[str] | None = None) -> list[str]:
|
||||
known = known or _known_ids()
|
||||
warnings: list[str] = []
|
||||
for edge in collect_edges():
|
||||
if edge.target_id not in known:
|
||||
warnings.append(
|
||||
f"broken relation: {edge.source_id} "
|
||||
f"{edge.relation_type} -> {edge.target_id}"
|
||||
)
|
||||
return warnings
|
||||
|
||||
|
||||
def check_relations() -> list[str]:
|
||||
warnings: list[str] = []
|
||||
for cycle in find_depends_on_cycles():
|
||||
warnings.append(f"depends_on cycle: {' -> '.join(cycle)}")
|
||||
warnings.extend(find_broken_references())
|
||||
return warnings
|
||||
|
||||
|
||||
def _node_labels() -> dict[str, str]:
|
||||
index = load_index()
|
||||
labels: dict[str, str] = {}
|
||||
for item in index.get("capabilities", []):
|
||||
labels[item["id"]] = f"{item['id']}<br/>{item['vector']}"
|
||||
return labels
|
||||
|
||||
|
||||
def render_mermaid() -> str:
|
||||
labels = _node_labels()
|
||||
edges = collect_edges()
|
||||
lines = ["graph LR"]
|
||||
for cap_id, label in sorted(labels.items()):
|
||||
lines.append(f' {_node_id(cap_id)}["{label}"]')
|
||||
for edge in edges:
|
||||
lines.append(
|
||||
f" {_node_id(edge.source_id)} -->|{edge.relation_type}| {_node_id(edge.target_id)}"
|
||||
)
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def write_graph(path: Path | None = None) -> Path:
|
||||
target = path or GRAPH_PATH
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(render_mermaid(), encoding="utf-8")
|
||||
return target
|
||||
44
schemas/federation.schema.yaml
Normal file
44
schemas/federation.schema.yaml
Normal file
@@ -0,0 +1,44 @@
|
||||
$schema: https://json-schema.org/draft/2020-12/schema
|
||||
$id: https://reuse-surface.local/schemas/federation.schema.yaml
|
||||
title: Registry Federation Manifest
|
||||
description: >
|
||||
Schema for registry/federation/sources.yaml. Describes local and sibling
|
||||
capability index sources to compose into a federated index.
|
||||
type: object
|
||||
additionalProperties: false
|
||||
required: [version, domain, collision_policy, sources]
|
||||
properties:
|
||||
version:
|
||||
type: integer
|
||||
minimum: 1
|
||||
domain:
|
||||
type: string
|
||||
collision_policy:
|
||||
type: string
|
||||
enum: [warn, fail]
|
||||
sources:
|
||||
type: array
|
||||
minItems: 1
|
||||
items:
|
||||
$ref: '#/$defs/source'
|
||||
$defs:
|
||||
source:
|
||||
type: object
|
||||
additionalProperties: false
|
||||
required: [repo, index, enabled]
|
||||
properties:
|
||||
repo:
|
||||
type: string
|
||||
minLength: 1
|
||||
index:
|
||||
type: string
|
||||
minLength: 1
|
||||
enabled:
|
||||
type: boolean
|
||||
required:
|
||||
type: boolean
|
||||
default: false
|
||||
domain:
|
||||
type: string
|
||||
description:
|
||||
type: string
|
||||
@@ -61,6 +61,26 @@ reuse-surface catalog
|
||||
|
||||
Writes `docs/CapabilityCatalog.md` and `docs/catalog/index.html`.
|
||||
|
||||
### federation compose
|
||||
|
||||
Compose a federated index from `registry/federation/sources.yaml`.
|
||||
|
||||
```bash
|
||||
reuse-surface federation compose
|
||||
```
|
||||
|
||||
Writes `registry/indexes/federated.yaml` with `source_repo` attribution.
|
||||
|
||||
### graph
|
||||
|
||||
Generate a Mermaid relation graph from capability entry relations.
|
||||
|
||||
```bash
|
||||
reuse-surface graph
|
||||
reuse-surface graph --check
|
||||
reuse-surface graph --stdout
|
||||
```
|
||||
|
||||
## Export format
|
||||
|
||||
The export bundle includes:
|
||||
@@ -80,6 +100,8 @@ Stable IDs and maturity fields are preserved for agent consumption (UC-RS-019).
|
||||
| Export for agents | `reuse-surface export --format json` |
|
||||
| Detect overlap | `reuse-surface overlaps` |
|
||||
| Publish catalog | `reuse-surface catalog` |
|
||||
| Compose federation | `reuse-surface federation compose` |
|
||||
| Relation graph | `reuse-surface graph` |
|
||||
|
||||
## Related use cases
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ type: workplan
|
||||
title: "Registry federation and relation graphs"
|
||||
domain: helix_forge
|
||||
repo: reuse-surface
|
||||
status: ready
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: helix-forge
|
||||
created: "2026-06-15"
|
||||
@@ -34,7 +34,7 @@ from configured sources and generating relation graphs for architects (UC-RS-016
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0005-T01
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "9a9732ea-c546-49fe-bf61-0f3bdf94406a"
|
||||
```
|
||||
@@ -55,7 +55,7 @@ Include commented placeholders for `state-hub`, `feature-control`, and
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0005-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "9539c609-ac44-40a7-90bc-6f294fe085b9"
|
||||
```
|
||||
@@ -73,7 +73,7 @@ and writes `registry/indexes/federated.yaml`. Requirements:
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0005-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "4f7e85a5-c73a-49b6-a044-ba1fabc54d8a"
|
||||
```
|
||||
@@ -90,7 +90,7 @@ entry relations. Requirements:
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0005-T04
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "6f7e2913-9634-4ebb-841f-9024b35961ef"
|
||||
```
|
||||
@@ -105,7 +105,7 @@ Extend validation or graph generation to report:
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0005-T05
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "8d69121b-9961-41c2-8802-d9c5b5d94c69"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user