REUSE-WP-0018-T03: LLM semantic rerank for plan-check
Some checks failed
ci / validate-registry (push) Has been cancelled

llm-connect came up locally (mock provider, 127.0.0.1:8080), unblocking
this task.

reuse_surface/plan_check.py: build_rerank_prompt/request_rerank/apply_rerank,
reusing llm_bridge.execute_prompt/extract_json_object (same pattern as
maintain_llm.py's request_maintain_patches). New schema
schemas/plan-check-rerank.schema.json rejects malformed responses (missing
fields, non-JSON, invented candidate ids outside the input set) rather than
guessing.

Per design principle 3 (deterministic matches always rank first),
apply_rerank appends LLM-scored entries after the deterministic list
(kind: 'llm') instead of reordering it -- the trusted base result is
identical with or without the rerank pass. Graceful skip via a 'notes'
field when LLM_CONNECT_URL is unset or the response is malformed, mirroring
maintain.py's no_llm/skip pattern. New --llm-url/--no-llm CLI flags.

Also extended plan-check-result.schema.json for 'notes' and the
(pre-existing, previously unschema'd) 'filed_capability_request' field.

9 new pytest cases; 89 total pass. Live-verified against the running
llm-connect instance: it correctly rejected the mock provider's non-JSON
response and surfaced the skip note, with the deterministic verdict and
match order completely unaffected.

REUSE-WP-0018 is now fully done (T01-T06). Updated
docs/IntentScopeGapAnalysis.md priority 29 to Closed and the workplan's
own frontmatter status to finished.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-07 17:27:39 +02:00
parent a1b3425c84
commit 0c76ccb28b
9 changed files with 335 additions and 17 deletions

View File

@@ -4,7 +4,7 @@ type: workplan
title: "plan-check: close the consumption loop for capability reuse"
domain: infotech
repo: reuse-surface
status: active
status: finished
owner: claude-code
topic_slug: helix-forge
created: "2026-07-06"
@@ -126,20 +126,38 @@ Implemented in `reuse_surface/plan_check.py` + `plan-check` CLI command:
```task
id: REUSE-WP-0018-T03
status: todo
status: done
priority: medium
state_hub_task_id: "9040505a-238e-4815-ae1a-8c5f8c12afa9"
```
**Not started**`llm-connect` is not running on this workstation
(same gap noted in REUSE-WP-0017-T04's drafting cohorts). Deterministic
`plan-check` (T02) is fully usable without it; this task adds the optional
rerank on top when a backend is available.
Unblocked 2026-07-07`llm-connect` running locally (mock provider,
`127.0.0.1:8080`). Implemented in `reuse_surface/plan_check.py`:
- Optional rerank/expansion stage via `LLM_CONNECT_URL` (reuse
`llm_bridge.py`); schema-constrained JSON, graceful skip when unset
- Confidence surfaced per match; deterministic matches always listed first
- Pytest with mocked llm-connect (valid + malformed responses)
- `build_rerank_prompt`/`request_rerank`/`apply_rerank`, reusing
`llm_bridge.execute_prompt`/`extract_json_object` (same pattern as
`maintain_llm.py`'s `request_maintain_patches`)
- New schema `schemas/plan-check-rerank.schema.json`; malformed responses
(missing fields, non-JSON, invented candidate ids not in the input set)
are rejected via schema validation + an id-membership filter, never
guessed at
- `apply_rerank` **appends** LLM-scored entries after the deterministic
list (`kind: "llm"`) rather than reordering it — deterministic matches
keep their original order/scores untouched, per design principle 3
- Graceful skip (`ValueError` → a `notes` field in the result, mirroring
`maintain.py`'s `no_llm`/skip pattern) when `LLM_CONNECT_URL` is unset
or the LLM response is malformed; `--no-llm` skips without attempting a
call at all
- New CLI flags: `--llm-url`, `--no-llm`
- `schemas/plan-check-result.schema.json` extended for `notes` and the
(pre-existing, previously unschema'd) `filed_capability_request` field
- 9 new pytest cases (valid, malformed, non-JSON, invented-id-filtering,
append-not-reorder, graceful skip, `--no-llm` short-circuit, successful
integration) — 89 total pass
- **Live-verified**, not just mocked: real HTTP round-trip against the
running mock `llm-connect` instance correctly rejected its non-JSON mock
response and surfaced the skip note, while the deterministic verdict and
match order stayed completely unaffected
## Bridge State Hub Capability Requests
@@ -221,7 +239,7 @@ state_hub_task_id: "ff0a91eb-cc41-487b-b726-8c2f84860399"
## Acceptance
- [x] `plan-check` returns reuse/extend/new verdicts for workplan files and intent text (without llm-connect T03 rerank not built, not available on this workstation)
- [x] `plan-check` returns reuse/extend/new verdicts for workplan files and intent text, with and without llm-connect (T03 shipped 2026-07-07, live-verified against a running instance)
- [x] JSON output validates against the published schema
- [x] `new` verdicts can file State Hub capability requests; `report gaps` lists unmatched open requests
- [x] Session-protocol convention drafted; propagation *proposed* to custodian-agent