feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,63 @@
---
entity_slug: bank_regulatory_effectiveness
evaluator: null
evaluated_at: '2026-02-23T00:49:12.640425'
overall_score: 4.0
scores:
- name: definition_precision
value: 3.0
max_value: 5.0
rationale: The definition captures a meaningful concept about balancing stability
and efficiency in banking regulation, but "degree to which regulations achieve
their intended objectives" is somewhat circular and could apply to any regulatory
domain. The balance between stability and efficiency is more specific and useful.
- name: source_grounding
value: 4.0
max_value: 5.0
rationale: Smith does indeed analyze banking regulations in Book II, Chapter 2,
examining how different regulatory approaches affect both stability and operational
efficiency. The concept of regulatory effectiveness as balancing these competing
objectives aligns well with Smith's analytical framework in this section.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: '"Regulation" is the correct domain placement, as this entity specifically
concerns the effectiveness of regulatory mechanisms rather than pure banking operations
or market dynamics. It sits squarely within regulatory theory and practice.'
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: This entity maps well to S3 (internal regulation/audit) as it concerns
the effectiveness of regulatory mechanisms in maintaining system stability and
performance. It could also relate to S2 (coordination/anti-oscillation) in terms
of preventing banking system instabilities.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The entity illuminates an important structural mechanism - how regulatory
design affects the fundamental trade-off between stability and efficiency in banking
systems. This provides genuine insight into why some regulatory approaches succeed
while others fail to achieve their objectives.
---
# Evaluation: Bank Regulatory Effectiveness
## definition_precision — 3.0 / 5.0
The definition captures a meaningful concept about balancing stability and efficiency in banking regulation, but "degree to which regulations achieve their intended objectives" is somewhat circular and could apply to any regulatory domain. The balance between stability and efficiency is more specific and useful.
## source_grounding — 4.0 / 5.0
Smith does indeed analyze banking regulations in Book II, Chapter 2, examining how different regulatory approaches affect both stability and operational efficiency. The concept of regulatory effectiveness as balancing these competing objectives aligns well with Smith's analytical framework in this section.
## domain_placement — 5.0 / 5.0
"Regulation" is the correct domain placement, as this entity specifically concerns the effectiveness of regulatory mechanisms rather than pure banking operations or market dynamics. It sits squarely within regulatory theory and practice.
## vsm_relevance — 4.0 / 5.0
This entity maps well to S3 (internal regulation/audit) as it concerns the effectiveness of regulatory mechanisms in maintaining system stability and performance. It could also relate to S2 (coordination/anti-oscillation) in terms of preventing banking system instabilities.
## explanatory_value — 4.0 / 5.0
The entity illuminates an important structural mechanism - how regulatory design affects the fundamental trade-off between stability and efficiency in banking systems. This provides genuine insight into why some regulatory approaches succeed while others fail to achieve their objectives.