fix(example): skip prompt writes when output exists, add quality rubrics
INFRA-TASKS #5 — process_chapters.py now skips writing *-prompt.md files when the corresponding output file already exists on disk. DB-only rebuilds no longer dirty the working tree with unchanged prompt content. INFRA-TASKS #8 — Added '## Quality Metrics' section to the entity and VSM mapping schemas, defining the five evaluation dimensions (Definition Precision, Source Grounding, Domain Placement, VSM Relevance, Explanatory Value) with 1–5 rubrics used by the evaluate-entity template. Also updated INFRA-TASKS.md to reflect current resolution status for tasks 4–19 across S2 and S3. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -39,6 +39,45 @@ this entity. Must be enclosed in quotation marks with chapter reference.
|
||||
How this entity is understood in modern economic theory, including
|
||||
any evolution in meaning since Smith's time.
|
||||
|
||||
## Quality Metrics
|
||||
|
||||
Used by the `evaluate-entity` prompt template to score each entity on five
|
||||
dimensions. Each dimension is scored 1–5, where 1 = very poor and 5 = excellent.
|
||||
|
||||
### Definition Precision (1-5)
|
||||
Is the definition specific, non-circular, and clearly distinguishable from
|
||||
neighbouring concepts? A score of 5 means the definition uniquely identifies
|
||||
the concept without relying on terms that are themselves undefined within the
|
||||
infospace. A score of 1 means the definition is vague, tautological, or
|
||||
indistinguishable from another entity.
|
||||
|
||||
### Source Grounding (1-5)
|
||||
Is the entity grounded in a specific, verifiable passage from the source text?
|
||||
A score of 5 means a citation is present, the cited chapter exists, and the
|
||||
definition accurately reflects the cited passage. A score of 1 means no
|
||||
citation is given or the definition contradicts the source.
|
||||
|
||||
### Domain Placement (1-5)
|
||||
Is the economic domain assignment correct and specific? A score of 5 means
|
||||
the assigned domain (e.g., Production, Distribution) is the most precise
|
||||
fit and would not be improved by a different choice. A score of 1 means the
|
||||
domain is wrong, or "General Theory" is used when a more specific domain
|
||||
applies.
|
||||
|
||||
### VSM Relevance (1-5)
|
||||
Does this entity connect meaningfully to at least one VSM system (S1–S5,
|
||||
recursion, variety, algedonic signals)? A score of 5 means the entity is
|
||||
directly mappable to a VSM concept with a clear structural rationale. A
|
||||
score of 1 means the entity has no discernible VSM connection and may be
|
||||
too granular or peripheral to the system model.
|
||||
|
||||
### Explanatory Value (1-5)
|
||||
Does this entity contribute to explaining the economic system as a whole, or
|
||||
is it a restatement of another concept? A score of 5 means removing this
|
||||
entity would leave a meaningful gap in the infospace. A score of 1 means
|
||||
another entity already covers this ground, or the entity adds no
|
||||
explanatory power.
|
||||
|
||||
## Validation Rules
|
||||
|
||||
1. The document MUST contain an H1 heading with the entity name.
|
||||
|
||||
@@ -33,6 +33,25 @@ might not fit the VSM concept perfectly.
|
||||
Other VSM concepts this entity could plausibly map to,
|
||||
with brief rationale for each alternative.
|
||||
|
||||
## Quality Metrics
|
||||
|
||||
Used by the `evaluate-entity` prompt template when assessing mapping quality.
|
||||
Each dimension is scored 1–5, where 1 = very poor and 5 = excellent.
|
||||
|
||||
### Rationale Rigour (1-5)
|
||||
Is the mapping justified with reference to Beer's VSM definitions, not just
|
||||
surface-level analogy? A score of 5 means the rationale cites specific VSM
|
||||
properties (e.g., "S2 attenuates variety between S1 units") and shows how
|
||||
the economic entity fulfils that role. A score of 1 means the rationale is
|
||||
a loose metaphor with no structural grounding.
|
||||
|
||||
### Strength Calibration (1-5)
|
||||
Is the declared Mapping Strength (Strong, Moderate, Weak) consistent with
|
||||
the rationale given? A score of 5 means the declared strength matches the
|
||||
depth of correspondence described. A score of 1 means the strength is
|
||||
overclaimed (e.g., "Strong" for a tangential analogy) or underclaimed
|
||||
(e.g., "Weak" for a direct structural match).
|
||||
|
||||
## Validation Rules
|
||||
|
||||
1. The document MUST contain an H1 heading in the format "Entity Name -> VSM Concept Name".
|
||||
|
||||
Reference in New Issue
Block a user