generated from coulomb/repo-seed
36 lines
1.6 KiB
Markdown
36 lines
1.6 KiB
Markdown
# Self-Scoping Assessment Artifacts
|
|
|
|
This directory contains repo-scoping's own baseline and assessment artifacts.
|
|
These files are meant to make scoping-engine changes comparable across releases
|
|
instead of relying on memory or screenshots.
|
|
|
|
## Artifact Types
|
|
|
|
- `golden/repo-scoping-golden-profile.v1.json` is the curated target profile for
|
|
repo-scoping itself.
|
|
- `assessments/repo-scoping-known-bad-2026-05-15-run-39.json` captures the
|
|
known-bad self-analysis that promoted LLM-provider vocabulary into native
|
|
repo-scoping capability truth.
|
|
- `../schemas/self-scoping-assessment.schema.json` defines the immutable
|
|
assessment-run artifact shape.
|
|
|
|
## Release Binding
|
|
|
|
Comparable assessment artifacts must bind generated results to the repo-scoping
|
|
engine release that produced them. A complete binding records package version,
|
|
engine git commit or release tag, dirty state, scanner version, candidate
|
|
generator version, quality criteria version, and prompt version when applicable.
|
|
|
|
The current known-bad artifact is marked `historical_incomplete` because the
|
|
original database run did not record the engine commit. It remains useful as a
|
|
negative regression seed, but future challenger artifacts should be fully bound
|
|
before they are accepted as comparable baselines.
|
|
|
|
## Review Use
|
|
|
|
When the engine changes, run repo-scoping against itself and export a challenger
|
|
assessment. Compare the challenger to the golden profile and to the negative
|
|
seed. Reviewers should be able to choose whether the old result, new result, or
|
|
neither is better, then store that judgement as a new assessment outcome.
|
|
|