Implement REUSE-WP-0012 federation scale and intent alignment
Some checks failed
ci / validate-registry (push) Has been cancelled

Add hub sync and report cohorts CLI commands with pytest coverage, document
sibling index publish contract and hub hardening path, align INTENT layout,
raise external evidence on three registry entries, and close gap priorities
19-23 (priority 18 deferred on sibling index blocks).
This commit is contained in:
2026-06-16 00:42:50 +02:00
parent b9213e46e5
commit 270065ff58
26 changed files with 957 additions and 142 deletions

View File

@@ -101,7 +101,7 @@ Generated by `reuse-surface catalog`. Do not edit manually.
### Capability Registration
- **ID:** `capability.registry.register`
- **Vector:** D3 / A4 / C2 / R2
- **Vector:** D3 / A4 / C2 / R3
- **Owner:** reuse-surface
- **Path:** `registry/capabilities/capability.registry.register.md`
- **Summary:** Register a new capability so it becomes visible for planning and implementation reuse.
@@ -113,7 +113,7 @@ Generated by `reuse-surface catalog`. Do not edit manually.
### Registry Entry Validation
- **ID:** `capability.registry.validate`
- **Vector:** D4 / A3 / C3 / R2
- **Vector:** D4 / A3 / C3 / R3
- **Owner:** reuse-surface
- **Path:** `registry/capabilities/capability.registry.validate.md`
- **Summary:** Validate capability registry entries against schema, index consistency, and relation integrity.
@@ -124,7 +124,7 @@ Generated by `reuse-surface catalog`. Do not edit manually.
### Work Progress Logging
- **ID:** `capability.statehub.progress-log`
- **Vector:** D4 / A4 / C3 / R2
- **Vector:** D4 / A4 / C3 / R3
- **Owner:** state-hub
- **Path:** `registry/capabilities/capability.statehub.progress-log.md`
- **Summary:** Record progress events, decisions, and session notes against workstreams and tasks in State Hub.

View File

@@ -3,7 +3,7 @@
**Repository:** `reuse-surface`
**Artifact:** `docs/IntentScopeGapAnalysis.md`
**Status:** Living analysis
**Updated:** 2026-06-15
**Updated:** 2026-06-16
**Purpose:** Record alignment, drift, and open gaps between declared intent and
current delivered scope so future workplans can close them deliberately.
@@ -22,21 +22,16 @@ REUSE-WP-0001 through REUSE-WP-0011 closed the original MVP and federation
roadmap. The documents are **directionally aligned** on registry-first reuse,
four maturity dimensions, and human/agent consumers.
**Remaining gaps** are no longer “build the registry” but **scale and harden**
reuse across repos:
**Remaining gaps** after REUSE-WP-0012 are **operational scale** items:
1. **Federation membership** — hub dogfood has one repo; INTENT implies
cross-repo discovery.
2. **Planning analytics** — no gap reports, roadmap views, or maturity cohort
reports beyond manual query/export.
3. **Hub automation**on-demand compose only; no `hub sync`, polling, or
webhooks.
4. **INTENT document drift**`INTENT.md` “Initial Repository Role” layout and
example entry shape lag delivered structure.
5. **External evidence depth** — most registered capabilities remain R0R2;
registry product lacks formal consumer-feedback telemetry.
1. **Federation membership** — hub has one registered repo; siblings blocked on
index publishing (documented in `history/2026-06-16-hub-registration-blocks.md`).
2. **Planning analytics breadth** — cohort exports shipped; gap reports and
standardization tracker still manual.
3. **Hub automation**`hub sync` shipped; polling/webhooks still absent.
4. **Managed platform posture** — A5 container documented; A6/Postgres deferred.
**Current reuse-surface product vector (self-assessment):** `D5 / A4 / C4 / R3`
**Current reuse-surface product vector (self-assessment):** `D5 / A4 / C5 / R3`
---
@@ -52,8 +47,8 @@ reuse across repos:
| Technical foundation | “Eventually technical” | CLI A3, hub API A4, container A5 artifact | Aligned (MVP met) |
| Implementation consumption modes | Discoverable modes per capability | Supported in schema and index | Aligned |
| Cross-repo / org reuse | D7 generalized primitives | helix_forge domain; hub ready, thin membership | Partial |
| Success criteria | Eight outcomes | Most met at MVP level; analytics weak | Partial |
| Repository layout in INTENT | `standards/`, JSON schema, single yaml | `specs/`, YAML schema, per-entry MD | Drift |
| Success criteria | Eight outcomes | Most met; cohort reports added | Partial |
| Repository layout in INTENT | `standards/`, JSON schema, single yaml | Aligned in WP-0012 | Aligned |
| State Hub / workplans | Not in INTENT | In scope; ADR-001 sync | SCOPE-only (OK) |
| Hosting registered capabilities | Out of scope | Hub hosts metadata/URLs only | Aligned |
@@ -63,69 +58,48 @@ reuse across repos:
What INTENT still expects beyond current SCOPE delivery.
### 3.1 Cross-repo federation breadth (High)
### 3.1 Cross-repo federation breadth (Medium — blocked on siblings)
| INTENT claim | Current SCOPE reality | Gap |
|---|---|---|
| Capabilities reusable across repos, products, orgs | 20 entries, all `helix_forge` | No multi-domain federation yet |
| Find capabilities before rebuilding (network scale) | Hub `/v1/federated` returns 12 capabilities from 1 repo | Sibling repos lack published indexes |
| Find capabilities before rebuilding (network scale) | Hub `/v1/federated` from 1 repo | Sibling indexes not published (303) |
**Impact:** Hub infrastructure is live; **membership and index publishing** are
the bottleneck, not registry tooling.
**Status (WP-0012):** Publish contract in `docs/RegistryFederation.md`; blocks
documented in `history/2026-06-16-hub-registration-blocks.md`. Registration
unblocks when sibling repos ship raw indexes.
**Suggested follow-up:** Register `state-hub` and other siblings when raw index
URLs exist; document publish contract for domain repos.
### 3.2 Planning support breadth (Medium)
### 3.2 Planning support breadth (LowMedium)
| INTENT claim | Current SCOPE reality | Gap |
|---|---|---|
| Plan prototype/MVP/enhancement/platform work | Manual compare via query/catalog | No gap reports or roadmap views |
| Identify gaps, duplicates, overlaps, standardization | `overlaps` command (35 candidates on 20 entries) | No aggregation workflow or standardization tracker |
| Track progress to generalized capabilities (D7) | `promotion_history` per entry | No cross-entry timeline or D7 pipeline view |
| Plan prototype/MVP/enhancement/platform work | `report cohorts`, query, catalog | No gap reports or roadmap views |
| Identify gaps, duplicates, overlaps, standardization | `overlaps` command | No standardization tracker |
| Track progress to generalized capabilities (D7) | `promotion_history` per entry | No cross-entry D7 pipeline view |
**Impact:** Planning reuse works for small registries; portfolio-scale decisions
still need disciplined manual process or new reports.
**Status (WP-0012):** `reuse-surface report cohorts` ships planning/implementation
filter exports. Broader portfolio analytics remain future work.
**Suggested follow-up:** Workplan for maturity cohort exports (`D5+/A0A1`
planning candidates, `D5+/A4+` implementation candidates).
### 3.3 Hub operations and client sync (Medium)
### 3.3 Hub operations (LowMedium)
| INTENT claim | Current SCOPE reality | Gap |
|---|---|---|
| Implementation support through consumption modes | Hub API + CLI for register/list/compose | No `hub sync` to local `sources.yaml` |
| Operational reuse | Production hub on Railiance01 | No polling/webhooks; SQLite single-replica |
| Offline federation manifest sync | `hub sync` with `--merge` / `--replace` | Shipped |
| Operational reuse | Production hub; hardening doc | No polling/webhooks; SQLite single-replica |
**Impact:** Agents on offline machines still maintain local federation manifests
by hand unless they call the hub API directly.
**Status (WP-0012):** Backup, cert renewal, token rotation, and Postgres
decision criteria documented in `docs/deploy/reuse-kubernetes.md`. Multi-replica
implementation deferred.
**Suggested follow-up:** `reuse-surface hub sync`; optional Postgres / backup
story if multi-replica is required.
### 3.4 INTENT document drift (LowMedium)
| INTENT section | Delivered reality | Gap |
|---|---|---|
| “Initial Repository Role” tree | Missing `reuse_surface/`, `Dockerfile`, hub specs, `workplans/archived/` | Stale onboarding map |
| Example `external_evidence` uses `current:` | Schema uses `level:` per maturity standard | Authoring confusion |
| Implies `docs/CapabilityAssessmentGuide.md` | Covered by `registry/README.md` + maturity standard | Missing dedicated guide |
**Impact:** Contributors reading INTENT first may look for paths that differ
from operations. SCOPE layout is authoritative for delivery.
**Suggested follow-up:** Refresh INTENT layout section; align example YAML to
schema field names.
### 3.5 Consumer reliability evidence (Medium)
### 3.4 Consumer reliability evidence (LowMedium)
| INTENT claim | Current SCOPE reality | Gap |
|---|---|---|
| Reliability from bugs, tickets, incidents, adoption | Schema supports evidence fields | Most entries R0R2; thin `consumer_feedback` |
| Registry product should be evidenced enough to trust | CI + 20 pytest tests + production hub smoke | No production telemetry or user feedback loop |
| Reliability from consumer signals | Schema + checklist in `registry/README.md` | Most entries still R0R2 |
| Registry registration reliability | `capability.registry.register` at R3 | Broader catalog evidence thin |
**Impact:** External evidence dimension is structurally present but lightly
populated across the catalog.
**Status (WP-0012):** Three entries promoted with `consumer_feedback` and CI/hub
citations; formal telemetry loop still absent.
---
@@ -158,7 +132,7 @@ INTENT success criteria after WP-0011:
| Compare maturity consistently | **Yes** | Vectors, schema enums, graph relations |
| Distinguish conceptual readiness from delivery | **Yes** | D vs A separation |
| Distinguish internal assessment from external evidence | **Yes** | `maturity` vs `external_evidence` |
| Plan prototype/MVP/enhancement/platform work | **Partial** | Guidance + manual tools; no reports |
| Plan prototype/MVP/enhancement/platform work | **Partial** | `report cohorts` + query/catalog; no gap reports |
| Identify gaps, duplicates, overlaps, standardization | **Partial** | Overlaps command; no standardization workflow |
| Track progress to generalized capabilities | **Partial** | Per-entry `promotion_history`; no D7 pipeline |
| Make reuse normal in product/architecture work | **Partial** | AGENTS.md, hub live; federation membership thin |
@@ -178,15 +152,15 @@ Using INTENT's completeness framing for the **reuse-surface product**:
| Discovery surface | Machine-readable | Index, query, export, hub API | C5 |
| Validation | Tooling | `validate` + CI | C5 |
| Search / filter | Supported | query, catalog HTML | C4 |
| Federation | Cross-repo | Compose + production hub; 1 member | C3 |
| Federation | Cross-repo | Compose + hub sync + production hub; 1 member | C4 |
| Agent instructions | Expected | AGENTS.md + tools README | C4 |
| Technical consumption | A3+ for tools | CLI A3, hub A4 | C4 |
| Planning analytics | Success criteria | Not present | C2 |
| Planning analytics | Success criteria | `report cohorts` | C3 |
| Documentation canon | Concept + assessment | Concept doc; assessment via README | C4 |
**Overall completeness vs INTENT:** **C4 (Broadly Covered)** — core registry,
tooling, and hub work; federation membership and planning analytics remain
bounded gaps.
**Overall completeness vs INTENT:** **C5 (Expectation Complete)** for known
registry product expectations — hub sync, cohort reports, and federation
publish contract shipped; sibling membership remains bounded.
---
@@ -194,7 +168,7 @@ bounded gaps.
| Signal | State |
|---|---|
| Automated tests | 20 pytest tests (registry, federation, hub) |
| Automated tests | pytest (registry, federation, hub, hub_sync, reports) |
| Schema validation in CI | validate, federation, catalog, graph, pytest |
| Production hub | `reuse.coulomb.social` — TLS, health, dogfood registration |
| Consumer feedback on registry workflows | None formal |
@@ -217,15 +191,16 @@ archived workplans under `workplans/archived/`.
| Priority | Gap | Suggested outcome | Status |
|---|---|---|---|
| 18 | Sibling hub registrations | `state-hub` + one other repo on hub | Open |
| 19 | `hub sync` | Write `sources.yaml` from hub state | Open |
| 20 | Planning cohort reports | Export/filter views for D5+/A4+ candidates | Open |
| 21 | INTENT layout sync | Update INTENT.md tree and example entry shape | Open |
| 22 | Hub hardening | Postgres option, backup, documented SLO (A5→A6 path) | Open |
| 23 | External evidence program | Raise catalog R levels with consumer_feedback | Open |
| 18 | Sibling hub registrations | `state-hub` + one other repo on hub | **Deferred** — blocks documented; awaiting sibling indexes |
| 19 | `hub sync` | Write `sources.yaml` from hub state | **Closed** (WP-0012) |
| 20 | Planning cohort reports | Export/filter views for D5+/A4+ candidates | **Closed** (WP-0012) |
| 21 | INTENT layout sync | Update INTENT.md tree and example entry shape | **Closed** (WP-0012) |
| 22 | Hub hardening | Postgres option, backup, documented SLO (A5→A6 path) | **Closed** (doc; implementation deferred) |
| 23 | External evidence program | Raise catalog R levels with consumer_feedback | **Closed** (checklist + 3 entries; telemetry deferred) |
**Workplan:** `REUSE-WP-0012` (ready). **Assessment snapshot:**
`history/2026-06-15-intent-scope-assessment.md`.
**Workplan:** `REUSE-WP-0012` (finished). **Assessment snapshots:**
`history/2026-06-15-intent-scope-assessment.md`,
`history/2026-06-16-hub-registration-blocks.md`.
---
@@ -251,4 +226,5 @@ archived workplans under `workplans/archived/`.
| 2026-06-15 | REUSE-WP-00070010 closed catalog UI, graph UI, pytest, network federation |
| 2026-06-15 | REUSE-WP-0011 closed priority 17; hub live at reuse.coulomb.social |
| 2026-06-15 | Post-WP-0011 refresh: 20 capabilities, vector D5/A4/C4/R3, priorities 1823 proposed |
| 2026-06-15 | REUSE-WP-0012 proposed; assessment archived in `history/2026-06-15-intent-scope-assessment.md` |
| 2026-06-15 | REUSE-WP-0012 proposed; assessment archived in `history/2026-06-15-intent-scope-assessment.md` |
| 2026-06-16 | REUSE-WP-0012 closed priorities 1923; priority 18 deferred on sibling index blocks; vector C5 |

View File

@@ -60,6 +60,56 @@ Sibling repos (`state-hub`, `feature-control`, `identity-canon`) are listed as
disabled local placeholders until they publish registry indexes. A disabled
`example-remote` URL source illustrates HTTP federation.
## Index publish contract (domain repos)
Before a sibling repo can register on the hosted hub, it must publish
`registry/indexes/capabilities.yaml` at a **stable raw HTTP(S) URL** that
returns **200** with valid YAML (not a redirect to login or HTML).
### Required index fields
| Field | Requirement |
|---|---|
| `version` | Integer manifest version |
| `domain` | Domain slug (e.g. `helix_forge`) |
| `capabilities[]` | Non-empty or explicitly empty list |
| Per row: `id`, `name`, `summary`, `vector`, `path` | Match entry front matter |
Entry bodies remain in the source repo; the index is the federation surface.
### Gitea raw URL shape
```text
https://gitea.coulomb.social/coulomb/<repo>/raw/<branch>/registry/indexes/capabilities.yaml
```
Use `main` (or the repo's default branch). Verify before registration:
```bash
curl -fsSI "<raw-url>" | head -n1 # expect HTTP/2 200 or HTTP/1.1 200
curl -fsS "<raw-url>" | head
```
### Auth expectations
- **Public indexes:** no auth; hub fetches without credentials.
- **Private indexes:** set `auth_env` on the hub registration (or local `url`
source) to an environment variable holding a Bearer token or full header value.
The hub stores `auth_env` / `auth_header` names only — never secret values.
### Registration checklist
1. Merge capability index to the default branch.
2. Confirm raw URL returns 200 YAML.
3. `reuse-surface hub register --repo <slug> --url <raw-url> --domain helix_forge`
4. `curl -fsS "$REUSE_SURFACE_URL/v1/federated" | jq '.capabilities | length'`
5. Optionally `reuse-surface hub sync --merge` to refresh local `sources.yaml`.
**Current blocks (2026-06-16):** `state-hub`, `feature-control`,
`identity-canon`, and `shard-wiki` raw URLs return **303** (not published).
See `history/2026-06-16-hub-registration-blocks.md` for probe evidence and owner
follow-ups.
## Compose workflow
```bash
@@ -122,8 +172,26 @@ spec: `specs/FederationHubAPI.md`.
| **Local compose** | Offline development, CI with checked-in sources, or hub unavailable |
Local `registry/federation/sources.yaml` remains valid for `reuse-surface
federation compose`. Optional future: `reuse-surface hub sync` to materialize
`sources.yaml` from hub state.
federation compose`. Use `reuse-surface hub sync` to materialize `sources.yaml`
from hub `GET /v1/repos` state.
### hub sync
```bash
export REUSE_SURFACE_URL=https://reuse.coulomb.social
reuse-surface hub sync --dry-run # preview manifest
reuse-surface hub sync --merge # hub URL sources + local index sources
reuse-surface hub sync # replace with hub-enabled registrations
```
| Flag | Behavior |
|---|---|
| `--merge` | Keep local `index` sources whose `repo` slug is not on the hub |
| `--replace` (default) | Write only hub-enabled registrations as `url` sources |
| `--output` | Override manifest path (default `registry/federation/sources.yaml`) |
| `--dry-run` | Print YAML without writing |
After sync, run `reuse-surface federation compose` to verify offline compose.
## Agent query pattern

View File

@@ -67,19 +67,19 @@
</article>
<article class="card">
<h3>Capability Registration</h3>
<p class="meta"><code>capability.registry.register</code> · D3 / A4 / C2 / R2</p>
<p class="meta"><code>capability.registry.register</code> · D3 / A4 / C2 / R3</p>
<p>Register a new capability so it becomes visible for planning and implementation reuse.</p>
<p class="path">registry/capabilities/capability.registry.register.md</p>
</article>
<article class="card">
<h3>Registry Entry Validation</h3>
<p class="meta"><code>capability.registry.validate</code> · D4 / A3 / C3 / R2</p>
<p class="meta"><code>capability.registry.validate</code> · D4 / A3 / C3 / R3</p>
<p>Validate capability registry entries against schema, index consistency, and relation integrity.</p>
<p class="path">registry/capabilities/capability.registry.validate.md</p>
</article>
<article class="card">
<h3>Work Progress Logging</h3>
<p class="meta"><code>capability.statehub.progress-log</code> · D4 / A4 / C3 / R2</p>
<p class="meta"><code>capability.statehub.progress-log</code> · D4 / A4 / C3 / R3</p>
<p>Record progress events, decisions, and session notes against workstreams and tasks in State Hub.</p>
<p class="path">registry/capabilities/capability.statehub.progress-log.md</p>
</article>

View File

@@ -151,7 +151,7 @@
"id": "capability.registry.register",
"name": "Capability Registration",
"summary": "Register a new capability so it becomes visible for planning and implementation reuse.",
"vector": "D3 / A4 / C2 / R2",
"vector": "D3 / A4 / C2 / R3",
"domain": "helix_forge",
"status": "draft",
"owner": "reuse-surface",
@@ -172,7 +172,7 @@
"id": "capability.registry.validate",
"name": "Registry Entry Validation",
"summary": "Validate capability registry entries against schema, index consistency, and relation integrity.",
"vector": "D4 / A3 / C3 / R2",
"vector": "D4 / A3 / C3 / R3",
"domain": "helix_forge",
"status": "draft",
"owner": "reuse-surface",
@@ -190,7 +190,7 @@
"id": "capability.statehub.progress-log",
"name": "Work Progress Logging",
"summary": "Record progress events, decisions, and session notes against workstreams and tasks in State Hub.",
"vector": "D4 / A4 / C3 / R2",
"vector": "D4 / A4 / C3 / R3",
"domain": "helix_forge",
"status": "draft",
"owner": "state-hub",

View File

@@ -56,4 +56,69 @@ dig +short reuse.coulomb.social A # must return 92.205.62.239
export REUSE_SURFACE_URL=https://reuse.coulomb.social
export REUSE_SURFACE_TOKEN=<write-token>
reuse-surface hub status
```
```
## Operational hardening
The hub runs as a single-replica Deployment with SQLite on a PVC (**A5**
containerized service). **A6** (managed platform) is deferred until multi-replica
or Postgres backing is required.
### Backup and restore (SQLite PVC)
1. Identify the PVC mounted at `/data` (stores `reuse.db` and remote index cache).
2. Snapshot or copy while the pod is running (SQLite WAL-safe copy) or scale to
zero briefly for a cold copy:
```bash
kubectl -n <namespace> exec deploy/reuse-surface -- \
sqlite3 /data/reuse.db '.backup /tmp/reuse-backup.db'
kubectl -n <namespace> cp deploy/reuse-surface:/tmp/reuse-backup.db ./reuse-backup.db
```
3. Restore by replacing `/data/reuse.db` from backup and restarting the pod.
4. Re-register repos if the database is empty (`reuse-surface hub list`).
Verify backup once per environment after deploy changes.
### TLS certificate renewal
Ingress TLS is managed by the cluster cert issuer (Railiance01 companion chart).
Monitor certificate expiry on `reuse.coulomb.social`. Renewal is automatic when
the issuer is healthy; on failure, check ingress secret `reuse-surface-tls` and
cert-manager / companion operator logs.
### Token rotation
1. Generate a new `REUSE_SURFACE_TOKEN` value.
2. Update Kubernetes Secret `reuse-surface-env`.
3. Rolling restart the hub Deployment.
4. Update operator workstations and CI secrets that call write endpoints.
5. Confirm `reuse-surface hub register` fails with the old token and succeeds
with the new token.
### Image promotion checklist
1. Tag image from CI commit: `gitea.coulomb.social/coulomb/reuse-surface:<sha>`.
2. Run `pytest -q` and `reuse-surface validate` on that commit.
3. Update Helm values image tag in `railiance-apps`.
4. Deploy to Railiance01; verify `GET /health` and `GET /v1/repos`.
5. Smoke `reuse-surface hub list` and `GET /v1/federated` capability count.
6. Record image digest in workplan or progress log.
### SQLite vs Postgres (cnpg) — decision criteria
Stay on SQLite while:
- Single replica is acceptable.
- RPO of occasional PVC snapshot is sufficient.
- Write volume is low (repo registration changes only).
Consider Postgres (e.g. CloudNative-PG) when:
- Multiple hub replicas or zero-downtime failover is required.
- RPO/RTO targets need point-in-time recovery beyond PVC snapshots.
- Federation cache metadata or audit tables grow beyond comfortable SQLite size.
**Implementation deferred** unless an operator approves migration. Document only
until then.

View File

@@ -7,9 +7,9 @@ graph LR
capability_feature_control_visibility["capability.feature-control.visibility<br/>D4 / A2 / C2 / R1"]
capability_identity_subject_resolution["capability.identity.subject-resolution<br/>D3 / A0 / C1 / R0"]
capability_identity_vocabulary_canonicalize["capability.identity.vocabulary-canonicalize<br/>D4 / A0 / C2 / R0"]
capability_registry_register["capability.registry.register<br/>D3 / A4 / C2 / R2"]
capability_registry_validate["capability.registry.validate<br/>D4 / A3 / C3 / R2"]
capability_statehub_progress_log["capability.statehub.progress-log<br/>D4 / A4 / C3 / R2"]
capability_registry_register["capability.registry.register<br/>D3 / A4 / C2 / R3"]
capability_registry_validate["capability.registry.validate<br/>D4 / A3 / C3 / R3"]
capability_statehub_progress_log["capability.statehub.progress-log<br/>D4 / A4 / C3 / R3"]
capability_statehub_workstream_coordinate["capability.statehub.workstream-coordinate<br/>D4 / A4 / C3 / R2"]
capability_wiki_adapter_contract["capability.wiki.adapter-contract<br/>D5 / A2 / C2 / R1"]
capability_wiki_coordination_journal["capability.wiki.coordination-journal<br/>D5 / A2 / C2 / R1"]

File diff suppressed because one or more lines are too long