Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_regulatory_effectiveness

null

2026-02-23T00:49:12.640425

4.0

name	value	max_value	rationale
definition_precision	3.0	5.0	The definition captures a meaningful concept about balancing stability and efficiency in banking regulation, but "degree to which regulations achieve their intended objectives" is somewhat circular and could apply to any regulatory domain. The balance between stability and efficiency is more specific and useful.

name	value	max_value	rationale
source_grounding	4.0	5.0	Smith does indeed analyze banking regulations in Book II, Chapter 2, examining how different regulatory approaches affect both stability and operational efficiency. The concept of regulatory effectiveness as balancing these competing objectives aligns well with Smith's analytical framework in this section.

name	value	max_value	rationale
domain_placement	5.0	5.0	"Regulation" is the correct domain placement, as this entity specifically concerns the effectiveness of regulatory mechanisms rather than pure banking operations or market dynamics. It sits squarely within regulatory theory and practice.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S3 (internal regulation/audit) as it concerns the effectiveness of regulatory mechanisms in maintaining system stability and performance. It could also relate to S2 (coordination/anti-oscillation) in terms of preventing banking system instabilities.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity illuminates an important structural mechanism - how regulatory design affects the fundamental trade-off between stability and efficiency in banking systems. This provides genuine insight into why some regulatory approaches succeed while others fail to achieve their objectives.

Evaluation: Bank Regulatory Effectiveness

definition_precision — 3.0 / 5.0

The definition captures a meaningful concept about balancing stability and efficiency in banking regulation, but "degree to which regulations achieve their intended objectives" is somewhat circular and could apply to any regulatory domain. The balance between stability and efficiency is more specific and useful.

source_grounding — 4.0 / 5.0

Smith does indeed analyze banking regulations in Book II, Chapter 2, examining how different regulatory approaches affect both stability and operational efficiency. The concept of regulatory effectiveness as balancing these competing objectives aligns well with Smith's analytical framework in this section.

domain_placement — 5.0 / 5.0

"Regulation" is the correct domain placement, as this entity specifically concerns the effectiveness of regulatory mechanisms rather than pure banking operations or market dynamics. It sits squarely within regulatory theory and practice.

vsm_relevance — 4.0 / 5.0

This entity maps well to S3 (internal regulation/audit) as it concerns the effectiveness of regulatory mechanisms in maintaining system stability and performance. It could also relate to S2 (coordination/anti-oscillation) in terms of preventing banking system instabilities.

explanatory_value — 4.0 / 5.0

The entity illuminates an important structural mechanism - how regulatory design affects the fundamental trade-off between stability and efficiency in banking systems. This provides genuine insight into why some regulatory approaches succeed while others fail to achieve their objectives.

3.5 KiB Raw Blame History