feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,63 @@
---
entity_slug: economic_system_mechanisms
evaluator: null
evaluated_at: '2026-02-23T05:18:49.254418'
overall_score: 2.0
scores:
- name: definition_precision
value: 2.0
max_value: 5.0
rationale: The definition is overly broad and umbrella-like, essentially describing
"how economic systems work" rather than identifying a specific, distinct concept.
It lists multiple disparate elements (processes, institutions, procedures, commerce,
trade, production, distribution) without clear boundaries or precision.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith does discuss various economic mechanisms throughout his work,
this entity appears to be a modern analytical abstraction rather than a concept
Smith himself articulated. The reference to "Book IV, Chapter 0" is also problematic
as Book IV doesn't have a Chapter 0.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: '"General Theory" is appropriate given the broad, meta-level nature of
this concept, though the entity is so abstract it could arguably fit in multiple
domains. The placement isn''t wrong but reflects the entity''s lack of specificity.'
- name: vsm_relevance
value: 1.0
max_value: 5.0
rationale: "This entity is far too abstract and all-encompassing to map meaningfully\
\ to any specific VSM system\u2014it essentially describes the entire viable system\
\ rather than any particular component or function. It lacks the specificity needed\
\ for VSM analysis."
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: While understanding economic mechanisms is important, this entity merely
names a category without illuminating specific causal relationships or structural
dynamics. It provides a label rather than genuine analytical insight into how
economic systems actually function.
---
# Evaluation: Economic System Mechanisms
## definition_precision — 2.0 / 5.0
The definition is overly broad and umbrella-like, essentially describing "how economic systems work" rather than identifying a specific, distinct concept. It lists multiple disparate elements (processes, institutions, procedures, commerce, trade, production, distribution) without clear boundaries or precision.
## source_grounding — 2.0 / 5.0
While Smith does discuss various economic mechanisms throughout his work, this entity appears to be a modern analytical abstraction rather than a concept Smith himself articulated. The reference to "Book IV, Chapter 0" is also problematic as Book IV doesn't have a Chapter 0.
## domain_placement — 3.0 / 5.0
"General Theory" is appropriate given the broad, meta-level nature of this concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't wrong but reflects the entity's lack of specificity.
## vsm_relevance — 1.0 / 5.0
This entity is far too abstract and all-encompassing to map meaningfully to any specific VSM system—it essentially describes the entire viable system rather than any particular component or function. It lacks the specificity needed for VSM analysis.
## explanatory_value — 2.0 / 5.0
While understanding economic mechanisms is important, this entity merely names a category without illuminating specific causal relationships or structural dynamics. It provides a label rather than genuine analytical insight into how economic systems actually function.