feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,64 @@
---
entity_slug: economic_system_analysis
evaluator: null
evaluated_at: '2026-02-23T05:12:49.904979'
overall_score: 2.2
scores:
- name: definition_precision
value: 2.0
max_value: 5.0
rationale: The definition is overly broad and circular, essentially defining economic
system analysis as "examining economic arrangements to understand how they function."
It lacks specificity about what distinguishes this from general economic inquiry
and reads more like a generic description than a precise conceptual definition.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith does analyze different economic systems in Book IV, the entity
appears to abstract this into a meta-concept that Smith himself doesn't explicitly
articulate. The source chapter reference "Book IV, Chapter 0" is problematic since
Book IV doesn't have a Chapter 0, suggesting weak textual grounding.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: '"General Theory" is appropriate given the broad, methodological nature
of the concept, though the entity is so abstract it could arguably fit in multiple
domains. The placement isn''t incorrect but reflects the entity''s lack of conceptual
specificity.'
- name: vsm_relevance
value: 2.0
max_value: 5.0
rationale: This entity is too abstract and meta-analytical to map meaningfully to
any specific VSM system. It describes a general analytical approach rather than
a functional component of an economic system, making it largely VSM-neutral but
not in a useful way.
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: The entity adds little explanatory power beyond naming what economists
generally do when studying systems. It doesn't illuminate specific mechanisms,
structural relations, or provide insights that wouldn't be captured by more concrete
analytical concepts Smith actually discusses.
---
# Evaluation: Economic System Analysis
## definition_precision — 2.0 / 5.0
The definition is overly broad and circular, essentially defining economic system analysis as "examining economic arrangements to understand how they function." It lacks specificity about what distinguishes this from general economic inquiry and reads more like a generic description than a precise conceptual definition.
## source_grounding — 2.0 / 5.0
While Smith does analyze different economic systems in Book IV, the entity appears to abstract this into a meta-concept that Smith himself doesn't explicitly articulate. The source chapter reference "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, suggesting weak textual grounding.
## domain_placement — 3.0 / 5.0
"General Theory" is appropriate given the broad, methodological nature of the concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't incorrect but reflects the entity's lack of conceptual specificity.
## vsm_relevance — 2.0 / 5.0
This entity is too abstract and meta-analytical to map meaningfully to any specific VSM system. It describes a general analytical approach rather than a functional component of an economic system, making it largely VSM-neutral but not in a useful way.
## explanatory_value — 2.0 / 5.0
The entity adds little explanatory power beyond naming what economists generally do when studying systems. It doesn't illuminate specific mechanisms, structural relations, or provide insights that wouldn't be captured by more concrete analytical concepts Smith actually discusses.