Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

4.1 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_systemic_stability

null

2026-02-23T04:37:20.705432

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes systemic stability from individual bank stability, specifying that it involves the banking system's ability to continue providing essential services during stress. The mention of "appropriate regulation and market discipline" as requirements adds specificity, though it could be slightly more precise about what constitutes "essential financial services."

name	value	max_value	rationale
source_grounding	3.0	5.0	While Smith does discuss banking regulation and the conditions for stable banking in Book II, Chapter 2, the modern concept of "systemic stability" as a distinct analytical category may be reading contemporary banking theory back into Smith's text. Smith focuses more on individual bank prudence and basic regulatory principles than on system-wide stability as a separate concern.

name	value	max_value	rationale
domain_placement	5.0	5.0	The placement in the "Regulation" domain is entirely appropriate, as systemic banking stability is fundamentally a regulatory concern requiring oversight and policy intervention. This aligns perfectly with Smith's discussion of banking regulation in the cited chapter.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S3 (internal regulation/audit) as it concerns the regulatory mechanisms needed to maintain system integrity, and also connects to S2 (coordination/anti-oscillation) as systemic stability involves preventing destabilizing oscillations across the banking network. The regulatory focus gives it clear VSM relevance.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The concept provides genuine explanatory power by distinguishing between individual bank health and system-wide resilience, illuminating how banking regulation must consider network effects and systemic risks. It captures an important structural relationship between individual institutions and overall system performance.

Evaluation: Bank Systemic Stability

definition_precision — 4.0 / 5.0

The definition clearly distinguishes systemic stability from individual bank stability, specifying that it involves the banking system's ability to continue providing essential services during stress. The mention of "appropriate regulation and market discipline" as requirements adds specificity, though it could be slightly more precise about what constitutes "essential financial services."

source_grounding — 3.0 / 5.0

While Smith does discuss banking regulation and the conditions for stable banking in Book II, Chapter 2, the modern concept of "systemic stability" as a distinct analytical category may be reading contemporary banking theory back into Smith's text. Smith focuses more on individual bank prudence and basic regulatory principles than on system-wide stability as a separate concern.

domain_placement — 5.0 / 5.0

The placement in the "Regulation" domain is entirely appropriate, as systemic banking stability is fundamentally a regulatory concern requiring oversight and policy intervention. This aligns perfectly with Smith's discussion of banking regulation in the cited chapter.

vsm_relevance — 4.0 / 5.0

This entity maps well to S3 (internal regulation/audit) as it concerns the regulatory mechanisms needed to maintain system integrity, and also connects to S2 (coordination/anti-oscillation) as systemic stability involves preventing destabilizing oscillations across the banking network. The regulatory focus gives it clear VSM relevance.

explanatory_value — 4.0 / 5.0

The concept provides genuine explanatory power by distinguishing between individual bank health and system-wide resilience, illuminating how banking regulation must consider network effects and systemic risks. It captures an important structural relationship between individual institutions and overall system performance.

4.1 KiB Raw Blame History