feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,63 @@
---
entity_slug: economic_system_actor
evaluator: null
evaluated_at: '2026-02-23T05:12:14.020874'
overall_score: 2.0
scores:
- name: definition_precision
value: 2.0
max_value: 5.0
rationale: The definition is overly broad and umbrella-like, encompassing "individuals,
organizations, and entities" without clear boundaries or distinguishing characteristics.
It reads more like a general category than a precise concept that captures a distinct
analytical unit.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith certainly discusses various economic actors throughout The
Wealth of Nations, this entity appears to be a modern analytical construct rather
than a concept Smith explicitly develops. The attribution to "Book IV, Chapter
0" is problematic since Book IV doesn't have a Chapter 0, and the definition seems
to impose contemporary frameworks on Smith's text.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: '"General Theory" is appropriate given the broad nature of this entity,
though the vagueness of the concept makes any domain placement somewhat arbitrary.
The entity is too abstract to clearly belong to a more specific economic domain.'
- name: vsm_relevance
value: 1.0
max_value: 5.0
rationale: This entity is far too abstract and general to map meaningfully to any
specific VSM system - it could theoretically encompass actors across all systems
S1-S5. The concept lacks the structural specificity needed for useful VSM mapping.
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: The entity provides minimal explanatory power beyond simply naming the
obvious fact that economic systems have participants. It doesn't illuminate specific
mechanisms, relationships, or structural dynamics that would advance understanding
of Smith's political economy.
---
# Evaluation: Economic System Actor
## definition_precision — 2.0 / 5.0
The definition is overly broad and umbrella-like, encompassing "individuals, organizations, and entities" without clear boundaries or distinguishing characteristics. It reads more like a general category than a precise concept that captures a distinct analytical unit.
## source_grounding — 2.0 / 5.0
While Smith certainly discusses various economic actors throughout The Wealth of Nations, this entity appears to be a modern analytical construct rather than a concept Smith explicitly develops. The attribution to "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, and the definition seems to impose contemporary frameworks on Smith's text.
## domain_placement — 3.0 / 5.0
"General Theory" is appropriate given the broad nature of this entity, though the vagueness of the concept makes any domain placement somewhat arbitrary. The entity is too abstract to clearly belong to a more specific economic domain.
## vsm_relevance — 1.0 / 5.0
This entity is far too abstract and general to map meaningfully to any specific VSM system - it could theoretically encompass actors across all systems S1-S5. The concept lacks the structural specificity needed for useful VSM mapping.
## explanatory_value — 2.0 / 5.0
The entity provides minimal explanatory power beyond simply naming the obvious fact that economic systems have participants. It doesn't illuminate specific mechanisms, relationships, or structural dynamics that would advance understanding of Smith's political economy.