Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_operational_risk

null

2026-02-23T00:48:46.771310

3.6

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is precise and captures a distinct concept with clear categories (internal processes, people, systems, external events). It avoids circularity and provides specific boundaries for what constitutes operational risk in banking.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith discusses banking risks and stability in Book II, Chapter 2, the modern concept of "operational risk" as a formal risk management category is anachronistic and not clearly articulated in Smith's text. Smith's analysis focuses more on broader banking practices and their economic effects rather than this specific risk taxonomy.

name	value	max_value	rationale
domain_placement	4.0	5.0	The "Regulation" domain is appropriate since operational risk management is fundamentally about internal controls and regulatory compliance. This aligns well with Smith's concerns about banking stability and the need for sound banking practices.

name	value	max_value	rationale
vsm_relevance	5.0	5.0	This entity maps excellently to S3 (internal regulation/audit) as operational risk management is precisely about internal control systems, monitoring, and regulatory compliance. It also has clear connections to S2 (coordination) for managing internal processes.

name	value	max_value	rationale
explanatory_value	3.0	5.0	While the concept provides a useful framework for understanding banking stability concerns that Smith discusses, it primarily categorizes risks rather than illuminating the underlying economic mechanisms. It offers organizational value but limited explanatory power about how these risks actually affect economic systems.

Evaluation: Bank Operational Risk

definition_precision — 4.0 / 5.0

The definition is precise and captures a distinct concept with clear categories (internal processes, people, systems, external events). It avoids circularity and provides specific boundaries for what constitutes operational risk in banking.

source_grounding — 2.0 / 5.0

While Smith discusses banking risks and stability in Book II, Chapter 2, the modern concept of "operational risk" as a formal risk management category is anachronistic and not clearly articulated in Smith's text. Smith's analysis focuses more on broader banking practices and their economic effects rather than this specific risk taxonomy.

domain_placement — 4.0 / 5.0

The "Regulation" domain is appropriate since operational risk management is fundamentally about internal controls and regulatory compliance. This aligns well with Smith's concerns about banking stability and the need for sound banking practices.

vsm_relevance — 5.0 / 5.0

This entity maps excellently to S3 (internal regulation/audit) as operational risk management is precisely about internal control systems, monitoring, and regulatory compliance. It also has clear connections to S2 (coordination) for managing internal processes.

explanatory_value — 3.0 / 5.0

While the concept provides a useful framework for understanding banking stability concerns that Smith discusses, it primarily categorizes risks rather than illuminating the underlying economic mechanisms. It offers organizational value but limited explanatory power about how these risks actually affect economic systems.

3.5 KiB Raw Blame History