Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_principles

null

2026-02-23T05:19:59.413197

2.0

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is quite vague and circular, essentially defining principles as "fundamental concepts, rules, and theoretical foundations" without specifying what makes them fundamental or how they differ from general economic concepts. It functions more as an umbrella term than a precise analytical concept.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does discuss different economic systems in Book IV, the entity appears to extrapolate a meta-theoretical concept that Smith doesn't explicitly articulate. The attribution to "Chapter 0" is suspicious and suggests this may be an interpretive overlay rather than a direct textual concept.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the abstract nature of the concept, though the entity is so broad it could arguably belong in multiple domains. The placement reflects the meta-theoretical character but doesn't clearly distinguish it from other theoretical constructs.

name	value	max_value	rationale
vsm_relevance	1.0	5.0	This entity is far too abstract and meta-theoretical to map meaningfully to any specific VSM system. It operates at a level above the VSM framework, describing theoretical foundations rather than operational or regulatory mechanisms that the VSM is designed to model.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity provides minimal explanatory power, functioning more as a general label than illuminating specific mechanisms or relationships. It doesn't help us understand how economic systems actually work or what distinguishes one system from another in operational terms.

Evaluation: Economic System Principles

definition_precision — 2.0 / 5.0

The definition is quite vague and circular, essentially defining principles as "fundamental concepts, rules, and theoretical foundations" without specifying what makes them fundamental or how they differ from general economic concepts. It functions more as an umbrella term than a precise analytical concept.

source_grounding — 2.0 / 5.0

While Smith does discuss different economic systems in Book IV, the entity appears to extrapolate a meta-theoretical concept that Smith doesn't explicitly articulate. The attribution to "Chapter 0" is suspicious and suggests this may be an interpretive overlay rather than a direct textual concept.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the abstract nature of the concept, though the entity is so broad it could arguably belong in multiple domains. The placement reflects the meta-theoretical character but doesn't clearly distinguish it from other theoretical constructs.

vsm_relevance — 1.0 / 5.0

This entity is far too abstract and meta-theoretical to map meaningfully to any specific VSM system. It operates at a level above the VSM framework, describing theoretical foundations rather than operational or regulatory mechanisms that the VSM is designed to model.

explanatory_value — 2.0 / 5.0

The entity provides minimal explanatory power, functioning more as a general label than illuminating specific mechanisms or relationships. It doesn't help us understand how economic systems actually work or what distinguishes one system from another in operational terms.

3.5 KiB Raw Blame History