feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,64 @@
---
entity_slug: economic_system_knowledge_transfer
evaluator: null
evaluated_at: '2026-02-23T05:17:58.384706'
overall_score: 2.6
scores:
- name: definition_precision
value: 2.0
max_value: 5.0
rationale: The definition is overly broad and vague, essentially describing any
form of knowledge sharing about economic matters. It lacks specificity about what
constitutes "economic system knowledge" versus general economic information, and
uses circular language like "understanding about economic arrangements."
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith does discuss different economic systems in Book IV, the entity
extrapolates far beyond what Smith explicitly addresses about knowledge transfer
mechanisms. Smith's focus is on critiquing mercantilism and advocating for natural
liberty, not on theorizing knowledge dissemination processes.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: '"General Theory" is appropriate given the broad conceptual nature, but
the entity is so abstract it could arguably fit in multiple domains. The placement
is reasonable but not particularly illuminating about the entity''s specific theoretical
contribution.'
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: This entity maps well to S4 (intelligence/environmental adaptation) as
it concerns how economic systems learn from and adapt based on external information
and experiences. It could also relate to S2 (coordination) in terms of harmonizing
practices across contexts.
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: The entity names a general phenomenon but provides little insight into
specific mechanisms or structural relations that would enhance understanding of
Smith's economic theory. It remains at a surface level without illuminating deeper
economic principles or causal relationships.
---
# Evaluation: Economic System Knowledge Transfer
## definition_precision — 2.0 / 5.0
The definition is overly broad and vague, essentially describing any form of knowledge sharing about economic matters. It lacks specificity about what constitutes "economic system knowledge" versus general economic information, and uses circular language like "understanding about economic arrangements."
## source_grounding — 2.0 / 5.0
While Smith does discuss different economic systems in Book IV, the entity extrapolates far beyond what Smith explicitly addresses about knowledge transfer mechanisms. Smith's focus is on critiquing mercantilism and advocating for natural liberty, not on theorizing knowledge dissemination processes.
## domain_placement — 3.0 / 5.0
"General Theory" is appropriate given the broad conceptual nature, but the entity is so abstract it could arguably fit in multiple domains. The placement is reasonable but not particularly illuminating about the entity's specific theoretical contribution.
## vsm_relevance — 4.0 / 5.0
This entity maps well to S4 (intelligence/environmental adaptation) as it concerns how economic systems learn from and adapt based on external information and experiences. It could also relate to S2 (coordination) in terms of harmonizing practices across contexts.
## explanatory_value — 2.0 / 5.0
The entity names a general phenomenon but provides little insight into specific mechanisms or structural relations that would enhance understanding of Smith's economic theory. It remains at a surface level without illuminating deeper economic principles or causal relationships.