feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
---
|
||||
entity_slug: bank_economic_contribution
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T00:38:25.785555'
|
||||
overall_score: 2.6
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: The definition is overly broad and umbrella-like, essentially describing
|
||||
"all the ways banks help the economy" rather than capturing a distinct, bounded
|
||||
concept. It lacks precision by lumping together disparate mechanisms (capital
|
||||
allocation, transaction facilitation, risk management) under a vague "overall
|
||||
impact" framing.
|
||||
- name: source_grounding
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: While Smith does discuss banking's economic benefits in Book II, Chapter
|
||||
2, this entity appears to be a modern analytical construct that aggregates his
|
||||
observations rather than reflecting a specific concept he articulated. Smith examines
|
||||
particular banking functions and their effects, but doesn't present a unified
|
||||
theory of "bank economic contribution" as such.
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The "Accumulation" domain placement is appropriate since Smith's discussion
|
||||
of banking in Book II, Chapter 2 focuses on how banks facilitate capital formation
|
||||
and productive investment. Banking's role in channeling savings into productive
|
||||
uses aligns well with the accumulation theme.
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is too abstract and aggregated to map meaningfully to specific
|
||||
VSM systems, as it encompasses functions that would span multiple systems (S1
|
||||
operations, S2 coordination, S4 intelligence). The broad "overall impact" framing
|
||||
makes it VSM-neutral rather than illuminating particular systemic functions.
|
||||
- name: explanatory_value
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity provides little explanatory power beyond naming the general
|
||||
fact that banks contribute to economic development. It doesn't illuminate specific
|
||||
mechanisms or structural relations that would help understand how banking actually
|
||||
works within Smith's economic framework.
|
||||
---
|
||||
|
||||
# Evaluation: Bank Economic Contribution
|
||||
|
||||
## definition_precision — 2.0 / 5.0
|
||||
|
||||
The definition is overly broad and umbrella-like, essentially describing "all the ways banks help the economy" rather than capturing a distinct, bounded concept. It lacks precision by lumping together disparate mechanisms (capital allocation, transaction facilitation, risk management) under a vague "overall impact" framing.
|
||||
|
||||
## source_grounding — 3.0 / 5.0
|
||||
|
||||
While Smith does discuss banking's economic benefits in Book II, Chapter 2, this entity appears to be a modern analytical construct that aggregates his observations rather than reflecting a specific concept he articulated. Smith examines particular banking functions and their effects, but doesn't present a unified theory of "bank economic contribution" as such.
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
The "Accumulation" domain placement is appropriate since Smith's discussion of banking in Book II, Chapter 2 focuses on how banks facilitate capital formation and productive investment. Banking's role in channeling savings into productive uses aligns well with the accumulation theme.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
This entity is too abstract and aggregated to map meaningfully to specific VSM systems, as it encompasses functions that would span multiple systems (S1 operations, S2 coordination, S4 intelligence). The broad "overall impact" framing makes it VSM-neutral rather than illuminating particular systemic functions.
|
||||
|
||||
## explanatory_value — 2.0 / 5.0
|
||||
|
||||
This entity provides little explanatory power beyond naming the general fact that banks contribute to economic development. It doesn't illuminate specific mechanisms or structural relations that would help understand how banking actually works within Smith's economic framework.
|
||||
Reference in New Issue
Block a user