Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.8 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_framework

null

2026-02-23T05:16:13.249386

2.0

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and circular, essentially defining an "economic system framework" as a "conceptual structure and theoretical foundation" without clearly distinguishing what makes this a distinct concept. It reads more like a meta-description of analytical frameworks in general rather than identifying a specific theoretical construct.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does discuss different economic systems and their principles in Book IV, the entity appears to impose a modern analytical framework concept onto Smith's work rather than extracting a concept Smith himself articulates. The attribution to "Book IV, Chapter 0" is also problematic as this chapter designation doesn't align with standard divisions of the text.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is an appropriate domain for a foundational conceptual framework, though the entity is so abstract it could arguably belong in methodology or philosophy of economics. The domain assignment is reasonable but not definitively correct given the entity's vagueness.

name	value	max_value	rationale
vsm_relevance	1.0	5.0	This entity is far too abstract and meta-theoretical to map to any specific VSM system—it's describing the conceptual apparatus for understanding systems rather than being a functional component of a viable system itself. It operates at a level above the VSM framework entirely.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity provides minimal explanatory value as it merely labels the existence of theoretical frameworks without illuminating specific mechanisms, relationships, or structural features that Smith identifies. It's more of a taxonomic placeholder than a substantive analytical concept.

Evaluation: Economic System Framework

definition_precision — 2.0 / 5.0

The definition is overly broad and circular, essentially defining an "economic system framework" as a "conceptual structure and theoretical foundation" without clearly distinguishing what makes this a distinct concept. It reads more like a meta-description of analytical frameworks in general rather than identifying a specific theoretical construct.

source_grounding — 2.0 / 5.0

While Smith does discuss different economic systems and their principles in Book IV, the entity appears to impose a modern analytical framework concept onto Smith's work rather than extracting a concept Smith himself articulates. The attribution to "Book IV, Chapter 0" is also problematic as this chapter designation doesn't align with standard divisions of the text.

domain_placement — 3.0 / 5.0

"General Theory" is an appropriate domain for a foundational conceptual framework, though the entity is so abstract it could arguably belong in methodology or philosophy of economics. The domain assignment is reasonable but not definitively correct given the entity's vagueness.

vsm_relevance — 1.0 / 5.0

This entity is far too abstract and meta-theoretical to map to any specific VSM system—it's describing the conceptual apparatus for understanding systems rather than being a functional component of a viable system itself. It operates at a level above the VSM framework entirely.

explanatory_value — 2.0 / 5.0

The entity provides minimal explanatory value as it merely labels the existence of theoretical frameworks without illuminating specific mechanisms, relationships, or structural features that Smith identifies. It's more of a taxonomic placeholder than a substantive analytical concept.

3.8 KiB Raw Blame History