feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,65 @@
---
entity_slug: bank_operational_risk
evaluator: null
evaluated_at: '2026-02-23T00:48:46.771310'
overall_score: 3.6
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition is precise and captures a distinct concept with clear
categories (internal processes, people, systems, external events). It avoids circularity
and provides specific boundaries for what constitutes operational risk in banking.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Smith discusses banking risks and stability in Book II, Chapter
2, the modern concept of "operational risk" as a formal risk management category
is anachronistic and not clearly articulated in Smith's text. Smith's analysis
focuses more on broader banking practices and their economic effects rather than
this specific risk taxonomy.
- name: domain_placement
value: 4.0
max_value: 5.0
rationale: The "Regulation" domain is appropriate since operational risk management
is fundamentally about internal controls and regulatory compliance. This aligns
well with Smith's concerns about banking stability and the need for sound banking
practices.
- name: vsm_relevance
value: 5.0
max_value: 5.0
rationale: This entity maps excellently to S3 (internal regulation/audit) as operational
risk management is precisely about internal control systems, monitoring, and regulatory
compliance. It also has clear connections to S2 (coordination) for managing internal
processes.
- name: explanatory_value
value: 3.0
max_value: 5.0
rationale: While the concept provides a useful framework for understanding banking
stability concerns that Smith discusses, it primarily categorizes risks rather
than illuminating the underlying economic mechanisms. It offers organizational
value but limited explanatory power about how these risks actually affect economic
systems.
---
# Evaluation: Bank Operational Risk
## definition_precision — 4.0 / 5.0
The definition is precise and captures a distinct concept with clear categories (internal processes, people, systems, external events). It avoids circularity and provides specific boundaries for what constitutes operational risk in banking.
## source_grounding — 2.0 / 5.0
While Smith discusses banking risks and stability in Book II, Chapter 2, the modern concept of "operational risk" as a formal risk management category is anachronistic and not clearly articulated in Smith's text. Smith's analysis focuses more on broader banking practices and their economic effects rather than this specific risk taxonomy.
## domain_placement — 4.0 / 5.0
The "Regulation" domain is appropriate since operational risk management is fundamentally about internal controls and regulatory compliance. This aligns well with Smith's concerns about banking stability and the need for sound banking practices.
## vsm_relevance — 5.0 / 5.0
This entity maps excellently to S3 (internal regulation/audit) as operational risk management is precisely about internal control systems, monitoring, and regulatory compliance. It also has clear connections to S2 (coordination) for managing internal processes.
## explanatory_value — 3.0 / 5.0
While the concept provides a useful framework for understanding banking stability concerns that Smith discusses, it primarily categorizes risks rather than illuminating the underlying economic mechanisms. It offers organizational value but limited explanatory power about how these risks actually affect economic systems.