feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,66 @@
---
entity_slug: economic_system_framework
evaluator: null
evaluated_at: '2026-02-23T05:16:13.249386'
overall_score: 2.0
scores:
- name: definition_precision
value: 2.0
max_value: 5.0
rationale: The definition is overly broad and circular, essentially defining an
"economic system framework" as a "conceptual structure and theoretical foundation"
without clearly distinguishing what makes this a distinct concept. It reads more
like a meta-description of analytical frameworks in general rather than identifying
a specific theoretical construct.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith does discuss different economic systems and their principles
in Book IV, the entity appears to impose a modern analytical framework concept
onto Smith's work rather than extracting a concept Smith himself articulates.
The attribution to "Book IV, Chapter 0" is also problematic as this chapter designation
doesn't align with standard divisions of the text.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: '"General Theory" is an appropriate domain for a foundational conceptual
framework, though the entity is so abstract it could arguably belong in methodology
or philosophy of economics. The domain assignment is reasonable but not definitively
correct given the entity''s vagueness.'
- name: vsm_relevance
value: 1.0
max_value: 5.0
rationale: "This entity is far too abstract and meta-theoretical to map to any specific\
\ VSM system\u2014it's describing the conceptual apparatus for understanding systems\
\ rather than being a functional component of a viable system itself. It operates\
\ at a level above the VSM framework entirely."
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: The entity provides minimal explanatory value as it merely labels the
existence of theoretical frameworks without illuminating specific mechanisms,
relationships, or structural features that Smith identifies. It's more of a taxonomic
placeholder than a substantive analytical concept.
---
# Evaluation: Economic System Framework
## definition_precision — 2.0 / 5.0
The definition is overly broad and circular, essentially defining an "economic system framework" as a "conceptual structure and theoretical foundation" without clearly distinguishing what makes this a distinct concept. It reads more like a meta-description of analytical frameworks in general rather than identifying a specific theoretical construct.
## source_grounding — 2.0 / 5.0
While Smith does discuss different economic systems and their principles in Book IV, the entity appears to impose a modern analytical framework concept onto Smith's work rather than extracting a concept Smith himself articulates. The attribution to "Book IV, Chapter 0" is also problematic as this chapter designation doesn't align with standard divisions of the text.
## domain_placement — 3.0 / 5.0
"General Theory" is an appropriate domain for a foundational conceptual framework, though the entity is so abstract it could arguably belong in methodology or philosophy of economics. The domain assignment is reasonable but not definitively correct given the entity's vagueness.
## vsm_relevance — 1.0 / 5.0
This entity is far too abstract and meta-theoretical to map to any specific VSM system—it's describing the conceptual apparatus for understanding systems rather than being a functional component of a viable system itself. It operates at a level above the VSM framework entirely.
## explanatory_value — 2.0 / 5.0
The entity provides minimal explanatory value as it merely labels the existence of theoretical frameworks without illuminating specific mechanisms, relationships, or structural features that Smith identifies. It's more of a taxonomic placeholder than a substantive analytical concept.