generated from coulomb/repo-seed
REUSE-WP-0018 T01/T02/T04/T06: plan-check deterministic matching + State Hub bridge
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
T01: specs/PlanCheck.md design doc, plan-check-result.schema.json and reuse-event.schema.json (the latter shared with WP-0019's reuse telemetry). T02: reuse_surface/plan_check.py + 'reuse-surface plan-check' CLI command. Deterministic token-Jaccard matching against registry/indexes/federated.yaml (reuses overlaps.py's TOKEN_RE rather than a second scoring method), with reuse/extend/new verdicts, markdown and --format json output, staleness warning, and --record-outcome JSONL telemetry. T04: reuse_surface/statehub_bridge.py bridges plan-check 'new' verdicts to State Hub capability requests (--file-request) and surfaces open requests with no matching capability (report gaps --check-capability-requests, opt-in to stay offline-safe). Verified against the live local State Hub API; status field (not catalog_entry_id presence) is the correct open/closed signal, and the list endpoint needs a longer timeout than the health check (~7s observed with 5 rows). T06: docs (tools/README.md, RegistryFederation.md, SCOPE.md, IntentScopeGapAnalysis.md priority 29) and an informational CI smoke step. T03 (LLM rerank) not started -- llm-connect isn't running on this workstation. T05 (ecosystem rollout) remains blocked: WP-0017 has drafted entries for all 61 repos but they're still local-only pending its own T05 push/publish pass, so the federated index isn't yet worth rolling out plan-check as ecosystem convention. 16 new tests, all mocked -- no network calls in the default test run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
169
specs/PlanCheck.md
Normal file
169
specs/PlanCheck.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# Plan Check
|
||||
|
||||
**Repository:** `reuse-surface`
|
||||
**Artifact:** `specs/PlanCheck.md`
|
||||
**Status:** Draft 0.1 (REUSE-WP-0018-T01)
|
||||
**Schema:** `schemas/plan-check-result.schema.json`
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
`plan-check` closes the consumption loop the registry has lacked since
|
||||
inception: nothing today nudges an agent planning work in a sibling repo to
|
||||
query the federated capability index before building. The registry has been
|
||||
write-mostly. `plan-check` is the query-before-build step — advisory, not a
|
||||
gate — that matches a draft workplan or a free-text intent against the
|
||||
federated index and returns a verdict: reuse it, extend it, or it's genuinely
|
||||
new.
|
||||
|
||||
This spec covers the deterministic matching core (T02) and its interfaces
|
||||
with the optional LLM rerank (T03) and the State Hub capability-request
|
||||
bridge (T04). It does not cover the ecosystem rollout (T05) — the
|
||||
session-protocol convention is designed once this spec is implemented and
|
||||
dogfooded in this repo.
|
||||
|
||||
## 2. Design principles
|
||||
|
||||
1. **Deterministic core, LLM assist optional.** Keyword/scope/relation
|
||||
matching against `registry/indexes/federated.yaml` works with no external
|
||||
dependency. `LLM_CONNECT_URL` unlocks a semantic rerank pass; its absence
|
||||
degrades gracefully, never blocks.
|
||||
2. **Advisory, not blocking.** `plan-check` never fails a workplan into
|
||||
existence or refuses to let one be created. Adoption comes from the tool
|
||||
being useful and from convention (REUSE-WP-0018-T05), not from a gate.
|
||||
3. **Deterministic matches always rank first.** Whatever the LLM rerank
|
||||
proposes, it is listed after — and clearly labeled apart from — the
|
||||
deterministic candidates, so a human or agent can trust the base result
|
||||
even with `--no-llm`.
|
||||
4. **Every invocation is a data point.** `--record-outcome` writes an
|
||||
append-only fact. This is the raw material for REUSE-WP-0019's reuse
|
||||
telemetry — `plan-check` and telemetry share one event schema from day
|
||||
one so nothing needs migrating later.
|
||||
|
||||
## 3. Input model
|
||||
|
||||
Two input shapes, mutually exclusive:
|
||||
|
||||
| Input | How it's read |
|
||||
|---|---|
|
||||
| Workplan file | Parsed like `registry_update.py`'s git-diff signal collector: YAML frontmatter (`---`-delimited) plus the Markdown body. `title`, the one-liner in the intro paragraph, and any `## Problem statement` / `## Core Idea` heading content feed the match blob. |
|
||||
| `--intent "free text"` | Used verbatim as the match blob. |
|
||||
|
||||
Both normalize to the same internal `MatchQuery { text: str, tokens: set[str] }`
|
||||
before matching — the matcher does not care which input shape it came from.
|
||||
|
||||
## 4. Matching pipeline
|
||||
|
||||
Reuses the token-Jaccard approach already proven in `overlaps.py`
|
||||
(`TOKEN_RE`, `_tokens`) rather than inventing a second scoring method in the
|
||||
same codebase:
|
||||
|
||||
1. **Tokenize** the query blob with the existing `TOKEN_RE`.
|
||||
2. **Score** against every federated capability's blob (`name` + `summary` +
|
||||
`tags` + `discovery.intent` + `discovery.includes`, mirroring
|
||||
`overlaps._entry_blob`) via Jaccard similarity.
|
||||
3. **Rank signal, not just tie-break:** among candidates within
|
||||
`--tie-window` (default 0.05) of the top score, prefer the entry with the
|
||||
higher discovery/availability vector (a more mature capability is a safer
|
||||
reuse bet at equal textual match).
|
||||
4. **Relation expansion:** if a top candidate has `relations.supports` or
|
||||
`relations.related_to` entries, surface them as secondary candidates
|
||||
labeled `related` rather than silently dropped.
|
||||
|
||||
## 5. Verdict model
|
||||
|
||||
| Verdict | Condition | Meaning |
|
||||
|---|---|---|
|
||||
| `reuse` | top score ≥ `--reuse-threshold` (default 0.45) | An existing capability already covers this need — link it, don't rebuild it. |
|
||||
| `extend` | top score in `[--extend-threshold, --reuse-threshold)` (default 0.22–0.45) | Scope overlaps a capability closely enough that extending it is very likely cheaper than a new one — surfaced with an explicit "consider extending" framing. |
|
||||
| `new` | top score < `--extend-threshold`, or no federated capabilities exist | No good match. Proceed; optionally file a capability request (T04). |
|
||||
|
||||
Thresholds are CLI flags, not hardcoded, because the right cutoff will drift
|
||||
as coverage and entry quality improve (see REUSE-WP-0017). Defaults come from
|
||||
the same threshold reasoning as `overlaps.py`'s `--threshold 0.28` default,
|
||||
shifted since plan-check matches short intent text against short summaries
|
||||
rather than long entry-to-entry blobs.
|
||||
|
||||
## 6. Output
|
||||
|
||||
### Markdown (TTY default)
|
||||
|
||||
```text
|
||||
# Plan check: reuse | extend | new
|
||||
|
||||
**Query:** <workplan title or intent text>
|
||||
|
||||
## Top match
|
||||
- `capability.infotech.issue-tracking` (score 0.52, D4/A2/C2/R1) — issue-core
|
||||
> Unified Python/CLI interface for issue tracking across Gitea, GitHub, and GitLab...
|
||||
|
||||
## Other candidates
|
||||
- ...
|
||||
|
||||
## Related (via relations.*)
|
||||
- ...
|
||||
|
||||
Verdict: REUSE — link this capability instead of building new.
|
||||
```
|
||||
|
||||
### JSON (`--format json`, agent-consumable)
|
||||
|
||||
Validated against `schemas/plan-check-result.schema.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"query": {"source": "workplan|intent", "text": "...", "workplan_id": "..."},
|
||||
"verdict": "reuse|extend|new",
|
||||
"top_score": 0.52,
|
||||
"matches": [
|
||||
{"id": "capability.infotech.issue-tracking", "score": 0.52, "vector": "D4/A2/C2/R1", "owner": "issue-core", "kind": "deterministic|llm|related"}
|
||||
],
|
||||
"federated_index_updated": "2026-07-06",
|
||||
"federated_index_stale_warning": null
|
||||
}
|
||||
```
|
||||
|
||||
`federated_index_stale_warning` is set when `federated.yaml`'s `updated`
|
||||
field is older than 14 days — a cheap freshness signal until
|
||||
REUSE-WP-0019's automatic recompose lands.
|
||||
|
||||
## 7. Outcome recording (`--record-outcome`)
|
||||
|
||||
Append-only JSONL under `registry/telemetry/plan-check-events.jsonl`
|
||||
(reuse-surface's own copy; sibling repos get their own via the same schema
|
||||
when they adopt T05). One line per invocation:
|
||||
|
||||
```json
|
||||
{"ts": "2026-07-07T10:00:00Z", "consumer_repo": "reuse-surface", "capability_id": "capability.infotech.issue-tracking", "verdict": "reuse", "outcome": "reused|extended|new|skipped", "source": "plan-check"}
|
||||
```
|
||||
|
||||
This schema is deliberately identical to the reuse-event schema
|
||||
REUSE-WP-0019-T04 will implement server-side — `plan-check`'s local JSONL
|
||||
file is the fallback path when the hub is unreachable, and the same record
|
||||
shape posts to `POST /v1/reuse-events` once T04/WP-0019 exist. No local
|
||||
telemetry write ever blocks the primary command.
|
||||
|
||||
## 8. Relationship to T03 (LLM rerank) and T04 (State Hub bridge)
|
||||
|
||||
- **T03** adds an optional post-pass: send the deterministic top-N candidates
|
||||
plus the query text to `llm-connect` for a semantic confidence score and
|
||||
possible reordering *within* the deterministic candidate set. It does not
|
||||
invent new candidates outside what deterministic matching already found —
|
||||
keeps the "deterministic matches always rank first" guarantee simple to
|
||||
reason about. Schema-constrained JSON response, mirrors the
|
||||
`maintain_llm.py` pattern (graceful skip on missing `LLM_CONNECT_URL`,
|
||||
reject malformed responses rather than guess).
|
||||
- **T04** wires `new` verdicts to `POST /messages/` (State Hub
|
||||
`request_capability`-equivalent) and reads back open capability requests
|
||||
for `report gaps` to list unmatched. Deferred to its own task since it
|
||||
depends on State Hub API shapes this spec does not need to pin down yet.
|
||||
|
||||
## 9. Non-goals
|
||||
|
||||
- Blocking or gating workplan creation (§2.2).
|
||||
- Embedding-based / vector-similarity matching — token-Jaccard is the
|
||||
deterministic core for consistency with `overlaps.py`; embeddings are an
|
||||
LLM-rerank concern (T03), not a second deterministic path.
|
||||
- Editing sibling repos' rules files directly — that's the template
|
||||
propagation mechanism's job (T05), not this tool's.
|
||||
Reference in New Issue
Block a user