Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_systemic_risk

null

2026-02-23T04:37:10.567719

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly captures the concept of contagion risk in banking systems - how problems spread from individual institutions to the entire system. It's precise and non-circular, though it could be slightly more specific about the mechanisms of spread.

name	value	max_value	rationale
source_grounding	3.0	5.0	While Smith does discuss banking problems and their broader economic effects in Book II, Chapter 2, the modern term "systemic risk" and its technical framing may impose contemporary financial theory concepts that weren't explicitly articulated in Smith's original analysis. Smith focuses more on individual bank failures and their local effects.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Regulation" domain is perfectly appropriate for systemic risk, as this concept is fundamentally about the need for oversight and management of system-wide vulnerabilities. This is precisely the type of concern that drives regulatory frameworks.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S3 (internal regulation/audit) as it represents a control concern that requires monitoring and management of system-wide stability. It also has relevance to S4 (intelligence) as it involves detecting and responding to emerging threats to system viability.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The concept provides genuine explanatory power by illuminating how individual bank problems can cascade through interconnected financial networks, explaining a key structural vulnerability in banking systems. It goes beyond surface description to identify an important systemic mechanism.

Evaluation: Bank Systemic Risk

definition_precision — 4.0 / 5.0

The definition clearly captures the concept of contagion risk in banking systems - how problems spread from individual institutions to the entire system. It's precise and non-circular, though it could be slightly more specific about the mechanisms of spread.

source_grounding — 3.0 / 5.0

While Smith does discuss banking problems and their broader economic effects in Book II, Chapter 2, the modern term "systemic risk" and its technical framing may impose contemporary financial theory concepts that weren't explicitly articulated in Smith's original analysis. Smith focuses more on individual bank failures and their local effects.

domain_placement — 5.0 / 5.0

The "Regulation" domain is perfectly appropriate for systemic risk, as this concept is fundamentally about the need for oversight and management of system-wide vulnerabilities. This is precisely the type of concern that drives regulatory frameworks.

vsm_relevance — 4.0 / 5.0

This entity maps well to S3 (internal regulation/audit) as it represents a control concern that requires monitoring and management of system-wide stability. It also has relevance to S4 (intelligence) as it involves detecting and responding to emerging threats to system viability.

explanatory_value — 4.0 / 5.0

The concept provides genuine explanatory power by illuminating how individual bank problems can cascade through interconnected financial networks, explaining a key structural vulnerability in banking systems. It goes beyond surface description to identify an important systemic mechanism.

3.5 KiB Raw Blame History