Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

4.0 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

engrossers_and_forestallers

null

2026-02-23T05:23:09.639039

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes between two related but distinct medieval practices - forestalling (buying before market) and engrossing (accumulating stocks) - and explains their common purpose of market manipulation. The definition is precise and non-circular, though it could benefit from slightly more specificity about the mechanisms involved.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is well-grounded in Smith's actual text, particularly Book IV where he extensively discusses these medieval commercial regulations as part of his critique of the "ancient policy of Europe." Smith explicitly uses these terms and analyzes how such restrictions hindered agricultural improvement and efficient commerce.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Regulation" domain assignment is perfectly appropriate, as these are specific regulatory mechanisms that Smith analyzes as counterproductive government interventions in markets. This fits squarely within Smith's broader examination of how regulations can distort natural market processes.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	This entity has moderate VSM relevance, primarily mapping to S2 (coordination) as these regulations attempted to control market oscillations and price variations, albeit counterproductively. It also touches on S3 (internal regulation) as a failed regulatory mechanism, though the mapping is not as natural as more structural economic concepts.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity provides strong explanatory value by illustrating a specific historical mechanism through which well-intentioned regulations can obstruct efficient market development. It demonstrates Smith's broader argument about unintended consequences of market intervention and helps explain the evolution from medieval to modern commercial practices.

Evaluation: Engrossers And Forestallers

definition_precision — 4.0 / 5.0

The definition clearly distinguishes between two related but distinct medieval practices - forestalling (buying before market) and engrossing (accumulating stocks) - and explains their common purpose of market manipulation. The definition is precise and non-circular, though it could benefit from slightly more specificity about the mechanisms involved.

source_grounding — 5.0 / 5.0

This entity is well-grounded in Smith's actual text, particularly Book IV where he extensively discusses these medieval commercial regulations as part of his critique of the "ancient policy of Europe." Smith explicitly uses these terms and analyzes how such restrictions hindered agricultural improvement and efficient commerce.

domain_placement — 5.0 / 5.0

The "Regulation" domain assignment is perfectly appropriate, as these are specific regulatory mechanisms that Smith analyzes as counterproductive government interventions in markets. This fits squarely within Smith's broader examination of how regulations can distort natural market processes.

vsm_relevance — 3.0 / 5.0

This entity has moderate VSM relevance, primarily mapping to S2 (coordination) as these regulations attempted to control market oscillations and price variations, albeit counterproductively. It also touches on S3 (internal regulation) as a failed regulatory mechanism, though the mapping is not as natural as more structural economic concepts.

explanatory_value — 4.0 / 5.0

This entity provides strong explanatory value by illustrating a specific historical mechanism through which well-intentioned regulations can obstruct efficient market development. It demonstrates Smith's broader argument about unintended consequences of market intervention and helps explain the evolution from medieval to modern commercial practices.

4.0 KiB Raw Blame History