coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
examples/infospace-with-history/output/evaluations/bank_systemic_risk.md Normal file
View File

@@ -0,0 +1,65 @@
---
entity_slug: bank_systemic_risk
evaluator: null
evaluated_at: '2026-02-23T04:37:10.567719'
overall_score: 4.0
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly captures the concept of contagion risk in banking
systems - how problems spread from individual institutions to the entire system.
It's precise and non-circular, though it could be slightly more specific about
the mechanisms of spread.
- name: source_grounding
value: 3.0
max_value: 5.0
rationale: While Smith does discuss banking problems and their broader economic
effects in Book II, Chapter 2, the modern term "systemic risk" and its technical
framing may impose contemporary financial theory concepts that weren't explicitly
articulated in Smith's original analysis. Smith focuses more on individual bank
failures and their local effects.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: The "Regulation" domain is perfectly appropriate for systemic risk, as
this concept is fundamentally about the need for oversight and management of system-wide
vulnerabilities. This is precisely the type of concern that drives regulatory
frameworks.
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: This entity maps well to S3 (internal regulation/audit) as it represents
a control concern that requires monitoring and management of system-wide stability.
It also has relevance to S4 (intelligence) as it involves detecting and responding
to emerging threats to system viability.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The concept provides genuine explanatory power by illuminating how individual
bank problems can cascade through interconnected financial networks, explaining
a key structural vulnerability in banking systems. It goes beyond surface description
to identify an important systemic mechanism.
---
# Evaluation: Bank Systemic Risk
## definition_precision — 4.0 / 5.0
The definition clearly captures the concept of contagion risk in banking systems - how problems spread from individual institutions to the entire system. It's precise and non-circular, though it could be slightly more specific about the mechanisms of spread.
## source_grounding — 3.0 / 5.0
While Smith does discuss banking problems and their broader economic effects in Book II, Chapter 2, the modern term "systemic risk" and its technical framing may impose contemporary financial theory concepts that weren't explicitly articulated in Smith's original analysis. Smith focuses more on individual bank failures and their local effects.
## domain_placement — 5.0 / 5.0
The "Regulation" domain is perfectly appropriate for systemic risk, as this concept is fundamentally about the need for oversight and management of system-wide vulnerabilities. This is precisely the type of concern that drives regulatory frameworks.
## vsm_relevance — 4.0 / 5.0
This entity maps well to S3 (internal regulation/audit) as it represents a control concern that requires monitoring and management of system-wide stability. It also has relevance to S4 (intelligence) as it involves detecting and responding to emerging threats to system viability.
## explanatory_value — 4.0 / 5.0
The concept provides genuine explanatory power by illuminating how individual bank problems can cascade through interconnected financial networks, explaining a key structural vulnerability in banking systems. It goes beyond surface description to identify an important systemic mechanism.