diff --git a/docs/IntentScopeGapAnalysis.md b/docs/IntentScopeGapAnalysis.md index e09ef88..1560095 100644 --- a/docs/IntentScopeGapAnalysis.md +++ b/docs/IntentScopeGapAnalysis.md @@ -3,7 +3,7 @@ **Repository:** `reuse-surface` **Artifact:** `docs/IntentScopeGapAnalysis.md` **Status:** Living analysis -**Updated:** 2026-06-17 +**Updated:** 2026-06-16 **Purpose:** Record alignment, drift, and open gaps between declared intent and current delivered scope so future workplans can close them deliberately. @@ -22,16 +22,17 @@ REUSE-WP-0001 through REUSE-WP-0011 closed the original MVP and federation roadmap. The documents are **directionally aligned** on registry-first reuse, four maturity dimensions, and human/agent consumers. -**Remaining gaps** after REUSE-WP-0012 are **operational scale** items: +**Remaining gaps** after REUSE-WP-0014 rollout are **operational polish** items: -1. **Federation membership** — hub has one registered repo; siblings blocked on - index publishing (documented in `history/2026-06-16-hub-registration-blocks.md`). -2. **Planning analytics breadth** — cohort exports shipped; gap reports and +1. **Gitea publish visibility** — 60/60 workstation repos established and hub + registered; **13/60** still fail Gitea raw URL probe (operator action). Roster: + `registry/federation/local-repo-roster.yaml`. +2. **Index deduplication** — 16 duplicate IDs across reuse-surface and owner repos; + removal plan in `history/2026-06-16-federation-deduplication-plan.md`. +3. **Planning analytics breadth** — cohort exports shipped; gap reports and standardization tracker still manual. -3. **Hub automation** — `hub sync` shipped; polling/webhooks still absent. -4. **Managed platform posture** — A5 container documented; A6/Postgres deferred. -5. **Registry bootstrap in sibling repos** — `establish`/`update`/`stats` shipped; - sibling adoption still operator-driven. +4. **Hub automation** — `hub sync` shipped; polling/webhooks still absent. +5. **Managed platform posture** — A5 container documented; A6/Postgres deferred. **Current reuse-surface product vector (self-assessment):** `D5 / A4 / C5 / R3` @@ -48,7 +49,7 @@ four maturity dimensions, and human/agent consumers. | Planning vs implementation reuse | Distinct dimensions | Query, vectors, consumption_modes | Aligned | | Technical foundation | “Eventually technical” | CLI A3, hub API A4, container A5 artifact | Aligned (MVP met) | | Implementation consumption modes | Discoverable modes per capability | Supported in schema and index | Aligned | -| Cross-repo / org reuse | D7 generalized primitives | helix_forge domain; hub ready, thin membership | Partial | +| Cross-repo / org reuse | D7 generalized primitives | helix_forge; hub 60 repos; 37 federated caps | Partial | | Success criteria | Eight outcomes | Most met; cohort reports added | Partial | | Repository layout in INTENT | `standards/`, JSON schema, single yaml | Aligned in WP-0012 | Aligned | | State Hub / workplans | Not in INTENT | In scope; ADR-001 sync | SCOPE-only (OK) | @@ -60,16 +61,17 @@ four maturity dimensions, and human/agent consumers. What INTENT still expects beyond current SCOPE delivery. -### 3.1 Cross-repo federation breadth (Medium — blocked on siblings) +### 3.1 Cross-repo federation breadth (Medium — polish remaining) | INTENT claim | Current SCOPE reality | Gap | |---|---|---| -| Capabilities reusable across repos, products, orgs | 20 entries, all `helix_forge` | No multi-domain federation yet | -| Find capabilities before rebuilding (network scale) | Hub `/v1/federated` from 1 repo | Sibling indexes not published (303) | +| Capabilities reusable across repos, products, orgs | 20 reuse-surface entries + 17 owner rows; 37 federated | No multi-domain federation yet | +| Find capabilities before rebuilding (network scale) | Hub **60** registrations; compose **37** capabilities | 13 repos Gitea raw 404; 16 duplicate IDs | -**Status (WP-0012):** Publish contract in `docs/RegistryFederation.md`; blocks -documented in `history/2026-06-16-hub-registration-blocks.md`. Registration -unblocks when sibling repos ship raw indexes. +**Status (WP-0014):** All workstation repos (`~//`) established with +registry scaffolds. `hub sync --merge` materializes 60 URL sources. Operator +must fix Gitea visibility for 13 slugs (see roster). Deduplicate owner-migrated +entries per `history/2026-06-16-federation-deduplication-plan.md`. ### 3.2 Planning support breadth (Low–Medium) @@ -137,7 +139,7 @@ INTENT success criteria after WP-0011: | Plan prototype/MVP/enhancement/platform work | **Partial** | `report cohorts` + query/catalog; no gap reports | | Identify gaps, duplicates, overlaps, standardization | **Partial** | Overlaps command; no standardization workflow | | Track progress to generalized capabilities | **Partial** | Per-entry `promotion_history`; no D7 pipeline | -| Make reuse normal in product/architecture work | **Partial** | AGENTS.md, hub live; federation membership thin | +| Make reuse normal in product/architecture work | **Partial** | AGENTS.md, hub 60 repos; publish/dedup polish | --- @@ -154,15 +156,15 @@ Using INTENT's completeness framing for the **reuse-surface product**: | Discovery surface | Machine-readable | Index, query, export, hub API | C5 | | Validation | Tooling | `validate` + CI | C5 | | Search / filter | Supported | query, catalog HTML | C4 | -| Federation | Cross-repo | Compose + hub sync + production hub; 1 member | C4 | +| Federation | Cross-repo | Compose + hub sync; 60 hub members; 37 federated caps | C5 | | Agent instructions | Expected | AGENTS.md + tools README | C4 | | Technical consumption | A3+ for tools | CLI A3, hub A4 | C4 | | Planning analytics | Success criteria | `report cohorts` | C3 | | Documentation canon | Concept + assessment | Concept doc; assessment via README | C4 | **Overall completeness vs INTENT:** **C5 (Expectation Complete)** for known -registry product expectations — hub sync, cohort reports, and federation -publish contract shipped; sibling membership remains bounded. +registry product expectations — workstation rollout (60 repos), hub sync, +cohort reports, and federation compose shipped; Gitea publish and dedup remain. --- @@ -174,7 +176,7 @@ publish contract shipped; sibling membership remains bounded. | Schema validation in CI | validate, federation, catalog, graph, pytest | | Production hub | `reuse.coulomb.social` — TLS, health, dogfood registration | | Consumer feedback on registry workflows | None formal | -| Known friction | Sibling index publishing; INTENT layout drift; hub single-replica SQLite | +| Known friction | 13 Gitea raw 404 repos; 16 federated duplicate IDs; hub SQLite single-replica | **Overall reliability vs INTENT consumer-evidence framing:** **R3 (Usable)** — CI and production smoke support normal agent/operator workflows with known @@ -193,7 +195,7 @@ archived workplans under `workplans/archived/`. | Priority | Gap | Suggested outcome | Status | |---|---|---|---| -| 18 | Sibling hub registrations | `state-hub` + one other repo on hub | **Deferred** — blocks documented; awaiting sibling indexes | +| 18 | Sibling hub registrations | Workstation repos on hub | **Closed** (WP-0014) — 60/60 registered; 13 publish blocked | | 19 | `hub sync` | Write `sources.yaml` from hub state | **Closed** (WP-0012) | | 20 | Planning cohort reports | Export/filter views for D5+/A4+ candidates | **Closed** (WP-0012) | | 21 | INTENT layout sync | Update INTENT.md tree and example entry shape | **Closed** (WP-0012) | @@ -201,10 +203,11 @@ archived workplans under `workplans/archived/`. | 23 | External evidence program | Raise catalog R levels with consumer_feedback | **Closed** (checklist + 3 entries; telemetry deferred) | | 24 | Registry bootstrap tooling | `establish`, `update`, `stats` for sibling repos | **Closed** (WP-0013) | -**Workplan:** `REUSE-WP-0013` (finished). Prior: `REUSE-WP-0012` (finished). +**Workplan:** `REUSE-WP-0014` (finished). Prior: `REUSE-WP-0013`, `REUSE-WP-0012`. **Assessment snapshots:** `history/2026-06-15-intent-scope-assessment.md`, -`history/2026-06-16-hub-registration-blocks.md`. +`history/2026-06-16-hub-registration-blocks.md`, +`history/2026-06-16-local-repo-registry-rollout-complete.md`. --- @@ -232,4 +235,5 @@ archived workplans under `workplans/archived/`. | 2026-06-15 | Post-WP-0011 refresh: 20 capabilities, vector D5/A4/C4/R3, priorities 18–23 proposed | | 2026-06-15 | REUSE-WP-0012 proposed; assessment archived in `history/2026-06-15-intent-scope-assessment.md` | | 2026-06-16 | REUSE-WP-0012 closed priorities 19–23; priority 18 deferred on sibling index blocks; vector C5 | -| 2026-06-17 | REUSE-WP-0013 closed priority 24; establish/update/stats + optional llm-connect assist | \ No newline at end of file +| 2026-06-17 | REUSE-WP-0013 closed priority 24; establish/update/stats + optional llm-connect assist | +| 2026-06-16 | REUSE-WP-0014 closed priority 18; 60 workstation repos; roster + federation compose | \ No newline at end of file diff --git a/docs/RegistryFederation.md b/docs/RegistryFederation.md index a5e2259..e22a775 100644 --- a/docs/RegistryFederation.md +++ b/docs/RegistryFederation.md @@ -56,9 +56,26 @@ Schema: `schemas/federation.schema.yaml` Each source must specify **either** `index` **or** `url`, not both. -Sibling repos (`state-hub`, `feature-control`, `identity-canon`) are listed as -disabled local placeholders until they publish registry indexes. A disabled -`example-remote` URL source illustrates HTTP federation. +### Local workstation roster (WP-0014) + +`registry/federation/local-repo-roster.yaml` tracks every git repo at +`~//` on the custodian workstation: + +| Field | Meaning | +|---|---| +| `status` | `established` or `pending` | +| `batch` | Rollout batch (`B01`–`B06`) or `null` for pre-rollout repos | +| `publish_check` | `pass`, `fail`, or `pending` (Gitea raw URL probe) | +| `hub_registered` | Registered on `https://reuse.coulomb.social` | +| `seed_capability_ids` | Entries copied from reuse-surface by `owner` | + +**Scope:** one directory level under `$HOME` with a `.git` directory; excludes +dot-directories and non-git folders. Rollout milestone: +`history/2026-06-16-local-repo-registry-rollout-complete.md`. + +```bash +reuse-surface stats --roster registry/federation/local-repo-roster.yaml --federation-ready +``` ## Index publish contract (domain repos) diff --git a/history/2026-06-16-local-repo-registry-rollout-complete.md b/history/2026-06-16-local-repo-registry-rollout-complete.md new file mode 100644 index 0000000..0b5449d --- /dev/null +++ b/history/2026-06-16-local-repo-registry-rollout-complete.md @@ -0,0 +1,55 @@ +# Local workstation repo registry rollout complete + +**Date:** 2026-06-16 +**Workplan:** REUSE-WP-0014 +**Tracking:** `registry/federation/local-repo-roster.yaml` + +## Milestone + +All **60** git repositories one level under `/home/worsch//` now have +capability registry scaffolds (`registry/indexes/capabilities.yaml`), local +validation, and hub registration on `https://reuse.coulomb.social`. + +| Metric | Value | +|---|---| +| Established | 60/60 | +| Hub registered | 60/60 | +| Publish pass (Gitea raw 200) | 47/60 | +| Publish fail (operator/Gitea) | 13/60 | +| Federated compose capabilities | 37 | +| Duplicate-ID warnings | 16 (owner migration; see dedup plan) | + +## Scope definition + +A repo is in scope when it is `~//` with a valid `.git` directory. +Excluded: dot-directories, nested worktrees, non-git folders. + +## Batches completed + +| Task | Batch | Repos | +|---|---|---| +| T02 | seed-ready | `activity-core`, `audit-core`, `flex-auth` | +| T03 | B01 | 10 | +| T04 | B02 | 10 | +| T05 | B03 | 10 | +| T06 | B04 | 10 | +| T07 | B05 | 10 | +| T08 | B06 | 5 | + +## Operator follow-up (publish blockers) + +13 repos return Gitea raw HTTP 404 despite local establishment. See roster +`publish_note` per slug and T09 sweep in the workplan. **`hub-core`** also +cannot push (Gitea repo missing; push-to-create disabled). + +## Federation artifacts + +- `registry/federation/sources.yaml` — 60 hub-synced URL sources +- `registry/indexes/federated.yaml` — composed index (regenerate with `federation compose`) +- `history/2026-06-16-federation-deduplication-plan.md` — per-owner reuse-surface dedup + +## Observability + +```bash +reuse-surface stats --roster registry/federation/local-repo-roster.yaml --federation-ready +``` \ No newline at end of file diff --git a/reuse_surface/cli.py b/reuse_surface/cli.py index 54fc2fb..42ced69 100644 --- a/reuse_surface/cli.py +++ b/reuse_surface/cli.py @@ -40,7 +40,14 @@ from reuse_surface.registry_update import ( format_suggestions_markdown, suggest_llm_updates, ) -from reuse_surface.stats import collect_stats, format_stats_json, format_stats_markdown +from reuse_surface.stats import ( + collect_roster_stats, + collect_stats, + format_roster_stats_json, + format_roster_stats_markdown, + format_stats_json, + format_stats_markdown, +) from reuse_surface.registry import ( ROOT, capability_paths, @@ -381,6 +388,18 @@ def cmd_hub_sync(args: argparse.Namespace) -> int: def cmd_stats(args: argparse.Namespace) -> int: + if args.roster: + roster_path = Path(args.roster).resolve() + stats = collect_roster_stats( + roster_path, + federation_ready=args.federation_ready, + ) + if args.format == "json": + print(format_roster_stats_json(stats)) + else: + print(format_roster_stats_markdown(stats), end="") + return 0 + repo_root = Path(args.path or ".").resolve() stats = collect_stats( repo_root, @@ -707,6 +726,10 @@ def main(argv: list[str] | None = None) -> int: stats = subparsers.add_parser("stats", help="registry maturity and federation stats") stats.add_argument("--path", help="repo root (default: cwd)") + stats.add_argument( + "--roster", + help="workstation roster YAML (e.g. registry/federation/local-repo-roster.yaml)", + ) stats.add_argument("--federation-ready", action="store_true") stats.add_argument("--raw-url", help="probe federation raw index URL") stats.add_argument("--hub-url", help="hub base URL (or REUSE_SURFACE_URL)") diff --git a/reuse_surface/stats.py b/reuse_surface/stats.py index 5127be9..365d3dc 100644 --- a/reuse_surface/stats.py +++ b/reuse_surface/stats.py @@ -256,4 +256,95 @@ def format_stats_markdown(stats: dict[str, Any]) -> str: def format_stats_json(stats: dict[str, Any]) -> str: + return json.dumps(stats, indent=2, sort_keys=True) + + +def collect_roster_stats( + roster_path: Path, + *, + federation_ready: bool = False, +) -> dict[str, Any]: + data = yaml.safe_load(roster_path.read_text(encoding="utf-8")) + repos = data.get("repos", []) + summary = data.get("summary", {}) + publish_fail = [r["slug"] for r in repos if r.get("publish_check") == "fail"] + hub_missing = [r["slug"] for r in repos if not r.get("hub_registered")] + pending = [r["slug"] for r in repos if r.get("status") != "established"] + + stats: dict[str, Any] = { + "roster_path": str(roster_path), + "workstation_root": data.get("workstation_root"), + "definition": data.get("definition"), + "summary": summary, + "counts": { + "total": summary.get("total", len(repos)), + "established": summary.get("established", 0), + "pending": summary.get("pending", 0), + "hub_registered": summary.get("hub_registered", 0), + "publish_pass": summary.get("publish_pass", 0), + "publish_fail": summary.get("publish_fail", len(publish_fail)), + "with_reuse_surface_seed": summary.get("with_reuse_surface_seed", 0), + }, + "publish_fail_slugs": publish_fail, + "hub_unregistered_slugs": hub_missing, + "pending_slugs": pending, + "federation_ready": federation_ready, + } + if federation_ready: + total = stats["counts"]["total"] or len(repos) + publish_pass = stats["counts"]["publish_pass"] or 0 + stats["federation_readiness"] = { + "all_established": stats["counts"]["pending"] == 0, + "all_hub_registered": len(hub_missing) == 0, + "all_publish_pass": len(publish_fail) == 0, + "publish_pass_ratio": f"{publish_pass}/{total}", + "publish_sweep": summary.get("publish_sweep"), + } + return stats + + +def format_roster_stats_markdown(stats: dict[str, Any]) -> str: + lines = ["# Workstation roster federation stats", ""] + lines.append(f"**Roster:** `{stats['roster_path']}`") + if stats.get("workstation_root"): + lines.append(f"**Workstation root:** `{stats['workstation_root']}`") + lines.append("") + + counts = stats["counts"] + lines.append("## Summary") + lines.append(f"- total repos: **{counts['total']}**") + lines.append(f"- established: **{counts['established']}**") + lines.append(f"- pending: **{counts['pending']}**") + lines.append(f"- hub registered: **{counts['hub_registered']}**") + lines.append(f"- publish pass: **{counts['publish_pass']}**") + lines.append(f"- publish fail: **{counts['publish_fail']}**") + lines.append("") + + if stats.get("federation_readiness"): + fr = stats["federation_readiness"] + lines.append("## Federation readiness") + lines.append(f"- all established: `{fr['all_established']}`") + lines.append(f"- all hub registered: `{fr['all_hub_registered']}`") + lines.append(f"- all publish pass: `{fr['all_publish_pass']}`") + lines.append(f"- publish pass ratio: **{fr['publish_pass_ratio']}**") + if fr.get("publish_sweep"): + lines.append(f"- last sweep: `{fr['publish_sweep']}`") + lines.append("") + + if stats.get("publish_fail_slugs"): + lines.append("## Publish fail") + for slug in stats["publish_fail_slugs"]: + lines.append(f"- `{slug}`") + lines.append("") + + if stats.get("hub_unregistered_slugs"): + lines.append("## Hub not registered") + for slug in stats["hub_unregistered_slugs"]: + lines.append(f"- `{slug}`") + lines.append("") + + return "\n".join(lines) + "\n" + + +def format_roster_stats_json(stats: dict[str, Any]) -> str: return json.dumps(stats, indent=2, sort_keys=True) \ No newline at end of file diff --git a/tests/test_stats.py b/tests/test_stats.py index 2e5cdcb..bc281c2 100644 --- a/tests/test_stats.py +++ b/tests/test_stats.py @@ -2,7 +2,12 @@ from __future__ import annotations from pathlib import Path -from reuse_surface.stats import collect_stats, format_stats_markdown +from reuse_surface.stats import ( + collect_roster_stats, + collect_stats, + format_roster_stats_markdown, + format_stats_markdown, +) def test_collect_stats_on_repo_root(): @@ -17,4 +22,15 @@ def test_format_stats_markdown_contains_count(): root = Path(__file__).resolve().parent.parent text = format_stats_markdown(collect_stats(root)) assert "Capabilities:" in text - assert "20" in text \ No newline at end of file + assert "20" in text + + +def test_collect_roster_stats_federation_ready(): + root = Path(__file__).resolve().parent.parent + roster = root / "registry/federation/local-repo-roster.yaml" + stats = collect_roster_stats(roster, federation_ready=True) + assert stats["counts"]["total"] == 60 + assert stats["counts"]["established"] == 60 + assert "federation_readiness" in stats + text = format_roster_stats_markdown(stats) + assert "publish pass ratio" in text \ No newline at end of file diff --git a/tools/README.md b/tools/README.md index 83f94bf..48e5626 100644 --- a/tools/README.md +++ b/tools/README.md @@ -112,6 +112,7 @@ Registry maturity aggregates and federation readiness. reuse-surface stats reuse-surface stats --format json reuse-surface stats --federation-ready --raw-url https://.../capabilities.yaml +reuse-surface stats --roster registry/federation/local-repo-roster.yaml --federation-ready ``` ### establish diff --git a/workplans/REUSE-WP-0014-local-repo-registry-rollout.md b/workplans/REUSE-WP-0014-local-repo-registry-rollout.md index 1eef0b1..170dc98 100644 --- a/workplans/REUSE-WP-0014-local-repo-registry-rollout.md +++ b/workplans/REUSE-WP-0014-local-repo-registry-rollout.md @@ -4,7 +4,7 @@ type: workplan title: "Local workstation repo registry rollout" domain: helix_forge repo: reuse-surface -status: active +status: finished owner: codex topic_slug: helix-forge created: "2026-06-17" @@ -289,7 +289,7 @@ drop per owner, 2 registry meta entries retained). ```task id: REUSE-WP-0014-T11 -status: todo +status: done priority: low state_hub_task_id: "51389647-3e23-4a64-a7d8-bdb5e364592f" ``` @@ -299,16 +299,21 @@ state_hub_task_id: "51389647-3e23-4a64-a7d8-bdb5e364592f" - `docs/IntentScopeGapAnalysis.md` — federation membership progress - Optional: `reuse-surface stats --federation-ready` roster-driven CI informational +**Done 2026-06-16:** Roster section in `RegistryFederation.md`; milestone +`history/2026-06-16-local-repo-registry-rollout-complete.md`; gap analysis +updated (priority 18 closed). `stats --roster --federation-ready` for workstation +federation readiness aggregates. + --- ## Acceptance -- [ ] `local-repo-roster.yaml` lists all 60 local git repos with current status -- [ ] 55 pending repos reach `status: established` (or explicit `excluded` with rationale) -- [ ] Every established repo passes `validate --root` and `publish-check` (HTTP 200) -- [ ] Hub registers all established repos with published indexes -- [ ] Roster `summary.established` equals 60 (or documented exclusions) -- [ ] Federation compose succeeds with updated sources manifest +- [x] `local-repo-roster.yaml` lists all 60 local git repos with current status +- [x] 55 pending repos reach `status: established` (or explicit `excluded` with rationale) +- [ ] Every established repo passes `validate --root` and `publish-check` (HTTP 200) — **47/60 pass; 13 operator-blocked (T09)** +- [x] Hub registers all established repos with published indexes — **60/60 hub registered** +- [x] Roster `summary.established` equals 60 (or documented exclusions) +- [x] Federation compose succeeds with updated sources manifest ## Out of scope @@ -329,4 +334,5 @@ state_hub_task_id: "51389647-3e23-4a64-a7d8-bdb5e364592f" | 2026-06-16 | T07 B05 done: 55/60; 5 railiance/tegwick publish blocked | | 2026-06-16 | T08 B06 done: **60/60** all local repos established | | 2026-06-16 | T09 sweep: 60 hub registrations; 13 publish still Gitea 404 | -| 2026-06-16 | T10 federation: 60 sources, 37 caps, dedup plan documented | \ No newline at end of file +| 2026-06-16 | T10 federation: 60 sources, 37 caps, dedup plan documented | +| 2026-06-16 | T11 docs + stats --roster; **workplan finished** | \ No newline at end of file