Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.7 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_standard

null

2026-02-23T05:20:54.653181

1.8

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and vague, essentially describing any evaluative framework for economic systems without identifying specific characteristics or boundaries. It reads more like a generic description of "standards in general" rather than capturing a distinct, well-defined concept.

name	value	max_value	rationale
source_grounding	1.0	5.0	The entity claims to derive from "Book IV, Chapter 0" which doesn't exist (Book IV begins with Chapter 1), and the definition appears to impose modern framework thinking onto Smith's work rather than emerging from his actual text. Smith discusses specific economic arrangements and policies but doesn't articulate a meta-theory of "economic system standards" as defined here.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the abstract nature of the concept, though the entity is so broadly defined that it could arguably belong in multiple domains. The domain assignment is reasonable but doesn't add much clarity given the entity's vagueness.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	While standards and evaluation criteria could theoretically relate to S3 (internal regulation/audit) or S5 (identity/policy), this entity is too abstract and poorly defined to map meaningfully to any specific VSM system. It lacks the operational specificity needed for effective VSM integration.

name	value	max_value	rationale
explanatory_value	1.0	5.0	The entity provides no genuine explanatory power about how economic systems actually function or what mechanisms drive their effectiveness. It merely labels the general concept of having standards without illuminating any particular economic relationships or structures that Smith identified.

Evaluation: Economic System Standard

definition_precision — 2.0 / 5.0

The definition is overly broad and vague, essentially describing any evaluative framework for economic systems without identifying specific characteristics or boundaries. It reads more like a generic description of "standards in general" rather than capturing a distinct, well-defined concept.

source_grounding — 1.0 / 5.0

The entity claims to derive from "Book IV, Chapter 0" which doesn't exist (Book IV begins with Chapter 1), and the definition appears to impose modern framework thinking onto Smith's work rather than emerging from his actual text. Smith discusses specific economic arrangements and policies but doesn't articulate a meta-theory of "economic system standards" as defined here.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the abstract nature of the concept, though the entity is so broadly defined that it could arguably belong in multiple domains. The domain assignment is reasonable but doesn't add much clarity given the entity's vagueness.

vsm_relevance — 2.0 / 5.0

While standards and evaluation criteria could theoretically relate to S3 (internal regulation/audit) or S5 (identity/policy), this entity is too abstract and poorly defined to map meaningfully to any specific VSM system. It lacks the operational specificity needed for effective VSM integration.

explanatory_value — 1.0 / 5.0

The entity provides no genuine explanatory power about how economic systems actually function or what mechanisms drive their effectiveness. It merely labels the general concept of having standards without illuminating any particular economic relationships or structures that Smith identified.

3.7 KiB Raw Blame History