generated from coulomb/repo-seed
REUSE-WP-0018-T03: LLM semantic rerank for plan-check
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
llm-connect came up locally (mock provider, 127.0.0.1:8080), unblocking this task. reuse_surface/plan_check.py: build_rerank_prompt/request_rerank/apply_rerank, reusing llm_bridge.execute_prompt/extract_json_object (same pattern as maintain_llm.py's request_maintain_patches). New schema schemas/plan-check-rerank.schema.json rejects malformed responses (missing fields, non-JSON, invented candidate ids outside the input set) rather than guessing. Per design principle 3 (deterministic matches always rank first), apply_rerank appends LLM-scored entries after the deterministic list (kind: 'llm') instead of reordering it -- the trusted base result is identical with or without the rerank pass. Graceful skip via a 'notes' field when LLM_CONNECT_URL is unset or the response is malformed, mirroring maintain.py's no_llm/skip pattern. New --llm-url/--no-llm CLI flags. Also extended plan-check-result.schema.json for 'notes' and the (pre-existing, previously unschema'd) 'filed_capability_request' field. 9 new pytest cases; 89 total pass. Live-verified against the running llm-connect instance: it correctly rejected the mock provider's non-JSON response and surfaced the skip note, with the deterministic verdict and match order completely unaffected. REUSE-WP-0018 is now fully done (T01-T06). Updated docs/IntentScopeGapAnalysis.md priority 29 to Closed and the workplan's own frontmatter status to finished. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -64,6 +64,9 @@ reuse-surface plan-check --intent "parse invoices and file evidence"
|
||||
reuse-surface plan-check --intent "..." --format json
|
||||
reuse-surface plan-check --intent "..." --record-outcome reused
|
||||
reuse-surface plan-check --intent "..." --file-request --requesting-domain infotech
|
||||
export LLM_CONNECT_URL=http://127.0.0.1:8080 # optional, enables semantic rerank
|
||||
reuse-surface plan-check --intent "..." --no-llm # skip the rerank pass
|
||||
reuse-surface plan-check --intent "..." --llm-url http://127.0.0.1:8080
|
||||
```
|
||||
|
||||
`reuse|extend|new` verdict from `--reuse-threshold`/`--extend-threshold`
|
||||
@@ -73,6 +76,14 @@ REUSE-WP-0019's reuse telemetry). `--file-request` files a State Hub
|
||||
capability request on a `new` verdict (requires the hub reachable at
|
||||
`127.0.0.1:8000`; degrades gracefully offline).
|
||||
|
||||
When `LLM_CONNECT_URL` is set, an optional rerank pass sends the top
|
||||
deterministic candidates to llm-connect for a semantic confidence score.
|
||||
Per design, this **never reorders or replaces** the deterministic result —
|
||||
LLM-scored entries are appended after as separately-labeled `[llm]` matches,
|
||||
so the trusted base result is identical whether or not the rerank runs.
|
||||
Malformed/non-JSON LLM responses are rejected and reported as a note, never
|
||||
silently guessed at; missing `LLM_CONNECT_URL` degrades the same way.
|
||||
|
||||
### catalog
|
||||
|
||||
Generate human-readable catalog artifacts (UC-RS-018).
|
||||
|
||||
Reference in New Issue
Block a user