REUSE-WP-0018 T01/T02/T04/T06: plan-check deterministic matching + State Hub bridge
Some checks failed
ci / validate-registry (push) Has been cancelled

T01: specs/PlanCheck.md design doc, plan-check-result.schema.json and
reuse-event.schema.json (the latter shared with WP-0019's reuse telemetry).

T02: reuse_surface/plan_check.py + 'reuse-surface plan-check' CLI command.
Deterministic token-Jaccard matching against registry/indexes/federated.yaml
(reuses overlaps.py's TOKEN_RE rather than a second scoring method), with
reuse/extend/new verdicts, markdown and --format json output, staleness
warning, and --record-outcome JSONL telemetry.

T04: reuse_surface/statehub_bridge.py bridges plan-check 'new' verdicts to
State Hub capability requests (--file-request) and surfaces open requests
with no matching capability (report gaps --check-capability-requests, opt-in
to stay offline-safe). Verified against the live local State Hub API;
status field (not catalog_entry_id presence) is the correct open/closed
signal, and the list endpoint needs a longer timeout than the health check
(~7s observed with 5 rows).

T06: docs (tools/README.md, RegistryFederation.md, SCOPE.md,
IntentScopeGapAnalysis.md priority 29) and an informational CI smoke step.

T03 (LLM rerank) not started -- llm-connect isn't running on this
workstation. T05 (ecosystem rollout) remains blocked: WP-0017 has drafted
entries for all 61 repos but they're still local-only pending its own T05
push/publish pass, so the federated index isn't yet worth rolling out
plan-check as ecosystem convention.

16 new tests, all mocked -- no network calls in the default test run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-07 00:57:18 +02:00
parent 2a6818d4fe
commit e72966a658
15 changed files with 1179 additions and 29 deletions

169
specs/PlanCheck.md Normal file
View File

@@ -0,0 +1,169 @@
# Plan Check
**Repository:** `reuse-surface`
**Artifact:** `specs/PlanCheck.md`
**Status:** Draft 0.1 (REUSE-WP-0018-T01)
**Schema:** `schemas/plan-check-result.schema.json`
---
## 1. Purpose
`plan-check` closes the consumption loop the registry has lacked since
inception: nothing today nudges an agent planning work in a sibling repo to
query the federated capability index before building. The registry has been
write-mostly. `plan-check` is the query-before-build step — advisory, not a
gate — that matches a draft workplan or a free-text intent against the
federated index and returns a verdict: reuse it, extend it, or it's genuinely
new.
This spec covers the deterministic matching core (T02) and its interfaces
with the optional LLM rerank (T03) and the State Hub capability-request
bridge (T04). It does not cover the ecosystem rollout (T05) — the
session-protocol convention is designed once this spec is implemented and
dogfooded in this repo.
## 2. Design principles
1. **Deterministic core, LLM assist optional.** Keyword/scope/relation
matching against `registry/indexes/federated.yaml` works with no external
dependency. `LLM_CONNECT_URL` unlocks a semantic rerank pass; its absence
degrades gracefully, never blocks.
2. **Advisory, not blocking.** `plan-check` never fails a workplan into
existence or refuses to let one be created. Adoption comes from the tool
being useful and from convention (REUSE-WP-0018-T05), not from a gate.
3. **Deterministic matches always rank first.** Whatever the LLM rerank
proposes, it is listed after — and clearly labeled apart from — the
deterministic candidates, so a human or agent can trust the base result
even with `--no-llm`.
4. **Every invocation is a data point.** `--record-outcome` writes an
append-only fact. This is the raw material for REUSE-WP-0019's reuse
telemetry — `plan-check` and telemetry share one event schema from day
one so nothing needs migrating later.
## 3. Input model
Two input shapes, mutually exclusive:
| Input | How it's read |
|---|---|
| Workplan file | Parsed like `registry_update.py`'s git-diff signal collector: YAML frontmatter (`---`-delimited) plus the Markdown body. `title`, the one-liner in the intro paragraph, and any `## Problem statement` / `## Core Idea` heading content feed the match blob. |
| `--intent "free text"` | Used verbatim as the match blob. |
Both normalize to the same internal `MatchQuery { text: str, tokens: set[str] }`
before matching — the matcher does not care which input shape it came from.
## 4. Matching pipeline
Reuses the token-Jaccard approach already proven in `overlaps.py`
(`TOKEN_RE`, `_tokens`) rather than inventing a second scoring method in the
same codebase:
1. **Tokenize** the query blob with the existing `TOKEN_RE`.
2. **Score** against every federated capability's blob (`name` + `summary` +
`tags` + `discovery.intent` + `discovery.includes`, mirroring
`overlaps._entry_blob`) via Jaccard similarity.
3. **Rank signal, not just tie-break:** among candidates within
`--tie-window` (default 0.05) of the top score, prefer the entry with the
higher discovery/availability vector (a more mature capability is a safer
reuse bet at equal textual match).
4. **Relation expansion:** if a top candidate has `relations.supports` or
`relations.related_to` entries, surface them as secondary candidates
labeled `related` rather than silently dropped.
## 5. Verdict model
| Verdict | Condition | Meaning |
|---|---|---|
| `reuse` | top score ≥ `--reuse-threshold` (default 0.45) | An existing capability already covers this need — link it, don't rebuild it. |
| `extend` | top score in `[--extend-threshold, --reuse-threshold)` (default 0.220.45) | Scope overlaps a capability closely enough that extending it is very likely cheaper than a new one — surfaced with an explicit "consider extending" framing. |
| `new` | top score < `--extend-threshold`, or no federated capabilities exist | No good match. Proceed; optionally file a capability request (T04). |
Thresholds are CLI flags, not hardcoded, because the right cutoff will drift
as coverage and entry quality improve (see REUSE-WP-0017). Defaults come from
the same threshold reasoning as `overlaps.py`'s `--threshold 0.28` default,
shifted since plan-check matches short intent text against short summaries
rather than long entry-to-entry blobs.
## 6. Output
### Markdown (TTY default)
```text
# Plan check: reuse | extend | new
**Query:** <workplan title or intent text>
## Top match
- `capability.infotech.issue-tracking` (score 0.52, D4/A2/C2/R1) — issue-core
> Unified Python/CLI interface for issue tracking across Gitea, GitHub, and GitLab...
## Other candidates
- ...
## Related (via relations.*)
- ...
Verdict: REUSE — link this capability instead of building new.
```
### JSON (`--format json`, agent-consumable)
Validated against `schemas/plan-check-result.schema.json`:
```json
{
"query": {"source": "workplan|intent", "text": "...", "workplan_id": "..."},
"verdict": "reuse|extend|new",
"top_score": 0.52,
"matches": [
{"id": "capability.infotech.issue-tracking", "score": 0.52, "vector": "D4/A2/C2/R1", "owner": "issue-core", "kind": "deterministic|llm|related"}
],
"federated_index_updated": "2026-07-06",
"federated_index_stale_warning": null
}
```
`federated_index_stale_warning` is set when `federated.yaml`'s `updated`
field is older than 14 days — a cheap freshness signal until
REUSE-WP-0019's automatic recompose lands.
## 7. Outcome recording (`--record-outcome`)
Append-only JSONL under `registry/telemetry/plan-check-events.jsonl`
(reuse-surface's own copy; sibling repos get their own via the same schema
when they adopt T05). One line per invocation:
```json
{"ts": "2026-07-07T10:00:00Z", "consumer_repo": "reuse-surface", "capability_id": "capability.infotech.issue-tracking", "verdict": "reuse", "outcome": "reused|extended|new|skipped", "source": "plan-check"}
```
This schema is deliberately identical to the reuse-event schema
REUSE-WP-0019-T04 will implement server-side — `plan-check`'s local JSONL
file is the fallback path when the hub is unreachable, and the same record
shape posts to `POST /v1/reuse-events` once T04/WP-0019 exist. No local
telemetry write ever blocks the primary command.
## 8. Relationship to T03 (LLM rerank) and T04 (State Hub bridge)
- **T03** adds an optional post-pass: send the deterministic top-N candidates
plus the query text to `llm-connect` for a semantic confidence score and
possible reordering *within* the deterministic candidate set. It does not
invent new candidates outside what deterministic matching already found —
keeps the "deterministic matches always rank first" guarantee simple to
reason about. Schema-constrained JSON response, mirrors the
`maintain_llm.py` pattern (graceful skip on missing `LLM_CONNECT_URL`,
reject malformed responses rather than guess).
- **T04** wires `new` verdicts to `POST /messages/` (State Hub
`request_capability`-equivalent) and reads back open capability requests
for `report gaps` to list unmatched. Deferred to its own task since it
depends on State Hub API shapes this spec does not need to pin down yet.
## 9. Non-goals
- Blocking or gating workplan creation (§2.2).
- Embedding-based / vector-similarity matching — token-Jaccard is the
deterministic core for consistency with `overlaps.py`; embeddings are an
LLM-rerank concern (T03), not a second deterministic path.
- Editing sibling repos' rules files directly — that's the template
propagation mechanism's job (T05), not this tool's.