fix(example): skip prompt writes when output exists, add quality rubrics

INFRA-TASKS #5 — process_chapters.py now skips writing *-prompt.md files
when the corresponding output file already exists on disk. DB-only rebuilds
no longer dirty the working tree with unchanged prompt content.

INFRA-TASKS #8 — Added '## Quality Metrics' section to the entity and VSM
mapping schemas, defining the five evaluation dimensions (Definition Precision,
Source Grounding, Domain Placement, VSM Relevance, Explanatory Value) with
1–5 rubrics used by the evaluate-entity template.

Also updated INFRA-TASKS.md to reflect current resolution status for tasks
4–19 across S2 and S3.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 06:04:09 +01:00
parent dfab3d598b
commit fa27572f43
4 changed files with 258 additions and 39 deletions

View File

@@ -39,6 +39,45 @@ this entity. Must be enclosed in quotation marks with chapter reference.
How this entity is understood in modern economic theory, including
any evolution in meaning since Smith's time.
## Quality Metrics
Used by the `evaluate-entity` prompt template to score each entity on five
dimensions. Each dimension is scored 15, where 1 = very poor and 5 = excellent.
### Definition Precision (1-5)
Is the definition specific, non-circular, and clearly distinguishable from
neighbouring concepts? A score of 5 means the definition uniquely identifies
the concept without relying on terms that are themselves undefined within the
infospace. A score of 1 means the definition is vague, tautological, or
indistinguishable from another entity.
### Source Grounding (1-5)
Is the entity grounded in a specific, verifiable passage from the source text?
A score of 5 means a citation is present, the cited chapter exists, and the
definition accurately reflects the cited passage. A score of 1 means no
citation is given or the definition contradicts the source.
### Domain Placement (1-5)
Is the economic domain assignment correct and specific? A score of 5 means
the assigned domain (e.g., Production, Distribution) is the most precise
fit and would not be improved by a different choice. A score of 1 means the
domain is wrong, or "General Theory" is used when a more specific domain
applies.
### VSM Relevance (1-5)
Does this entity connect meaningfully to at least one VSM system (S1S5,
recursion, variety, algedonic signals)? A score of 5 means the entity is
directly mappable to a VSM concept with a clear structural rationale. A
score of 1 means the entity has no discernible VSM connection and may be
too granular or peripheral to the system model.
### Explanatory Value (1-5)
Does this entity contribute to explaining the economic system as a whole, or
is it a restatement of another concept? A score of 5 means removing this
entity would leave a meaningful gap in the infospace. A score of 1 means
another entity already covers this ground, or the entity adds no
explanatory power.
## Validation Rules
1. The document MUST contain an H1 heading with the entity name.

View File

@@ -33,6 +33,25 @@ might not fit the VSM concept perfectly.
Other VSM concepts this entity could plausibly map to,
with brief rationale for each alternative.
## Quality Metrics
Used by the `evaluate-entity` prompt template when assessing mapping quality.
Each dimension is scored 15, where 1 = very poor and 5 = excellent.
### Rationale Rigour (1-5)
Is the mapping justified with reference to Beer's VSM definitions, not just
surface-level analogy? A score of 5 means the rationale cites specific VSM
properties (e.g., "S2 attenuates variety between S1 units") and shows how
the economic entity fulfils that role. A score of 1 means the rationale is
a loose metaphor with no structural grounding.
### Strength Calibration (1-5)
Is the declared Mapping Strength (Strong, Moderate, Weak) consistent with
the rationale given? A score of 5 means the declared strength matches the
depth of correspondence described. A score of 1 means the strength is
overclaimed (e.g., "Strong" for a tangential analogy) or underclaimed
(e.g., "Weak" for a direct structural match).
## Validation Rules
1. The document MUST contain an H1 heading in the format "Entity Name -> VSM Concept Name".