generated from coulomb/repo-seed
REUSE-WP-0018-T03: LLM semantic rerank for plan-check
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
llm-connect came up locally (mock provider, 127.0.0.1:8080), unblocking this task. reuse_surface/plan_check.py: build_rerank_prompt/request_rerank/apply_rerank, reusing llm_bridge.execute_prompt/extract_json_object (same pattern as maintain_llm.py's request_maintain_patches). New schema schemas/plan-check-rerank.schema.json rejects malformed responses (missing fields, non-JSON, invented candidate ids outside the input set) rather than guessing. Per design principle 3 (deterministic matches always rank first), apply_rerank appends LLM-scored entries after the deterministic list (kind: 'llm') instead of reordering it -- the trusted base result is identical with or without the rerank pass. Graceful skip via a 'notes' field when LLM_CONNECT_URL is unset or the response is malformed, mirroring maintain.py's no_llm/skip pattern. New --llm-url/--no-llm CLI flags. Also extended plan-check-result.schema.json for 'notes' and the (pre-existing, previously unschema'd) 'filed_capability_request' field. 9 new pytest cases; 89 total pass. Live-verified against the running llm-connect instance: it correctly rejected the mock provider's non-JSON response and surfaced the skip note, with the deterministic verdict and match order completely unaffected. REUSE-WP-0018 is now fully done (T01-T06). Updated docs/IntentScopeGapAnalysis.md priority 29 to Closed and the workplan's own frontmatter status to finished. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -4,7 +4,7 @@ type: workplan
|
||||
title: "plan-check: close the consumption loop for capability reuse"
|
||||
domain: infotech
|
||||
repo: reuse-surface
|
||||
status: active
|
||||
status: finished
|
||||
owner: claude-code
|
||||
topic_slug: helix-forge
|
||||
created: "2026-07-06"
|
||||
@@ -126,20 +126,38 @@ Implemented in `reuse_surface/plan_check.py` + `plan-check` CLI command:
|
||||
|
||||
```task
|
||||
id: REUSE-WP-0018-T03
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "9040505a-238e-4815-ae1a-8c5f8c12afa9"
|
||||
```
|
||||
|
||||
**Not started** — `llm-connect` is not running on this workstation
|
||||
(same gap noted in REUSE-WP-0017-T04's drafting cohorts). Deterministic
|
||||
`plan-check` (T02) is fully usable without it; this task adds the optional
|
||||
rerank on top when a backend is available.
|
||||
Unblocked 2026-07-07 — `llm-connect` running locally (mock provider,
|
||||
`127.0.0.1:8080`). Implemented in `reuse_surface/plan_check.py`:
|
||||
|
||||
- Optional rerank/expansion stage via `LLM_CONNECT_URL` (reuse
|
||||
`llm_bridge.py`); schema-constrained JSON, graceful skip when unset
|
||||
- Confidence surfaced per match; deterministic matches always listed first
|
||||
- Pytest with mocked llm-connect (valid + malformed responses)
|
||||
- `build_rerank_prompt`/`request_rerank`/`apply_rerank`, reusing
|
||||
`llm_bridge.execute_prompt`/`extract_json_object` (same pattern as
|
||||
`maintain_llm.py`'s `request_maintain_patches`)
|
||||
- New schema `schemas/plan-check-rerank.schema.json`; malformed responses
|
||||
(missing fields, non-JSON, invented candidate ids not in the input set)
|
||||
are rejected via schema validation + an id-membership filter, never
|
||||
guessed at
|
||||
- `apply_rerank` **appends** LLM-scored entries after the deterministic
|
||||
list (`kind: "llm"`) rather than reordering it — deterministic matches
|
||||
keep their original order/scores untouched, per design principle 3
|
||||
- Graceful skip (`ValueError` → a `notes` field in the result, mirroring
|
||||
`maintain.py`'s `no_llm`/skip pattern) when `LLM_CONNECT_URL` is unset
|
||||
or the LLM response is malformed; `--no-llm` skips without attempting a
|
||||
call at all
|
||||
- New CLI flags: `--llm-url`, `--no-llm`
|
||||
- `schemas/plan-check-result.schema.json` extended for `notes` and the
|
||||
(pre-existing, previously unschema'd) `filed_capability_request` field
|
||||
- 9 new pytest cases (valid, malformed, non-JSON, invented-id-filtering,
|
||||
append-not-reorder, graceful skip, `--no-llm` short-circuit, successful
|
||||
integration) — 89 total pass
|
||||
- **Live-verified**, not just mocked: real HTTP round-trip against the
|
||||
running mock `llm-connect` instance correctly rejected its non-JSON mock
|
||||
response and surfaced the skip note, while the deterministic verdict and
|
||||
match order stayed completely unaffected
|
||||
|
||||
## Bridge State Hub Capability Requests
|
||||
|
||||
@@ -221,7 +239,7 @@ state_hub_task_id: "ff0a91eb-cc41-487b-b726-8c2f84860399"
|
||||
|
||||
## Acceptance
|
||||
|
||||
- [x] `plan-check` returns reuse/extend/new verdicts for workplan files and intent text (without llm-connect — T03 rerank not built, not available on this workstation)
|
||||
- [x] `plan-check` returns reuse/extend/new verdicts for workplan files and intent text, with and without llm-connect (T03 shipped 2026-07-07, live-verified against a running instance)
|
||||
- [x] JSON output validates against the published schema
|
||||
- [x] `new` verdicts can file State Hub capability requests; `report gaps` lists unmatched open requests
|
||||
- [x] Session-protocol convention drafted; propagation *proposed* to custodian-agent
|
||||
|
||||
Reference in New Issue
Block a user