feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,63 @@
|
||||
---
|
||||
entity_slug: bank_financial_innovation_metrics
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T00:41:06.932134'
|
||||
overall_score: 1.8
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: The definition is vague and circular, essentially defining metrics as
|
||||
"measures used to assess" without specifying what these metrics actually are or
|
||||
how they differ from general performance indicators. It reads more like a modern
|
||||
business concept than a distinct analytical tool.
|
||||
- name: source_grounding
|
||||
value: 1.0
|
||||
max_value: 5.0
|
||||
rationale: This entity appears to impose modern financial analytics terminology
|
||||
onto Smith's text, which does not discuss formal "innovation metrics" or systematic
|
||||
measurement frameworks for banking innovations. The language and conceptual framework
|
||||
are anachronistic to Smith's 18th-century analysis.
|
||||
- name: domain_placement
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: While "Accumulation" is a reasonable domain for banking-related concepts,
|
||||
this particular entity feels more like a meta-analytical tool than a substantive
|
||||
economic mechanism that Smith would have discussed in the context of capital accumulation.
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: The entity could theoretically relate to S3 (monitoring/audit) or S4
|
||||
(intelligence gathering), but it's too abstract and modern in conception to meaningfully
|
||||
map to VSM systems in the context of Smith's economic analysis. It lacks the structural
|
||||
specificity needed for clear VSM placement.
|
||||
- name: explanatory_value
|
||||
value: 1.0
|
||||
max_value: 5.0
|
||||
rationale: This entity adds no genuine explanatory power about economic mechanisms
|
||||
or relationships that Smith identified. It merely labels a modern analytical practice
|
||||
without illuminating any underlying economic structure or causal relationship
|
||||
from the source material.
|
||||
---
|
||||
|
||||
# Evaluation: Bank Financial Innovation Metrics
|
||||
|
||||
## definition_precision — 2.0 / 5.0
|
||||
|
||||
The definition is vague and circular, essentially defining metrics as "measures used to assess" without specifying what these metrics actually are or how they differ from general performance indicators. It reads more like a modern business concept than a distinct analytical tool.
|
||||
|
||||
## source_grounding — 1.0 / 5.0
|
||||
|
||||
This entity appears to impose modern financial analytics terminology onto Smith's text, which does not discuss formal "innovation metrics" or systematic measurement frameworks for banking innovations. The language and conceptual framework are anachronistic to Smith's 18th-century analysis.
|
||||
|
||||
## domain_placement — 3.0 / 5.0
|
||||
|
||||
While "Accumulation" is a reasonable domain for banking-related concepts, this particular entity feels more like a meta-analytical tool than a substantive economic mechanism that Smith would have discussed in the context of capital accumulation.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
The entity could theoretically relate to S3 (monitoring/audit) or S4 (intelligence gathering), but it's too abstract and modern in conception to meaningfully map to VSM systems in the context of Smith's economic analysis. It lacks the structural specificity needed for clear VSM placement.
|
||||
|
||||
## explanatory_value — 1.0 / 5.0
|
||||
|
||||
This entity adds no genuine explanatory power about economic mechanisms or relationships that Smith identified. It merely labels a modern analytical practice without illuminating any underlying economic structure or causal relationship from the source material.
|
||||
Reference in New Issue
Block a user