Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_financial_innovation_metrics

null

2026-02-23T00:41:06.932134

1.8

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is vague and circular, essentially defining metrics as "measures used to assess" without specifying what these metrics actually are or how they differ from general performance indicators. It reads more like a modern business concept than a distinct analytical tool.

name	value	max_value	rationale
source_grounding	1.0	5.0	This entity appears to impose modern financial analytics terminology onto Smith's text, which does not discuss formal "innovation metrics" or systematic measurement frameworks for banking innovations. The language and conceptual framework are anachronistic to Smith's 18th-century analysis.

name	value	max_value	rationale
domain_placement	3.0	5.0	While "Accumulation" is a reasonable domain for banking-related concepts, this particular entity feels more like a meta-analytical tool than a substantive economic mechanism that Smith would have discussed in the context of capital accumulation.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	The entity could theoretically relate to S3 (monitoring/audit) or S4 (intelligence gathering), but it's too abstract and modern in conception to meaningfully map to VSM systems in the context of Smith's economic analysis. It lacks the structural specificity needed for clear VSM placement.

name	value	max_value	rationale
explanatory_value	1.0	5.0	This entity adds no genuine explanatory power about economic mechanisms or relationships that Smith identified. It merely labels a modern analytical practice without illuminating any underlying economic structure or causal relationship from the source material.

Evaluation: Bank Financial Innovation Metrics

definition_precision — 2.0 / 5.0

The definition is vague and circular, essentially defining metrics as "measures used to assess" without specifying what these metrics actually are or how they differ from general performance indicators. It reads more like a modern business concept than a distinct analytical tool.

source_grounding — 1.0 / 5.0

This entity appears to impose modern financial analytics terminology onto Smith's text, which does not discuss formal "innovation metrics" or systematic measurement frameworks for banking innovations. The language and conceptual framework are anachronistic to Smith's 18th-century analysis.

domain_placement — 3.0 / 5.0

While "Accumulation" is a reasonable domain for banking-related concepts, this particular entity feels more like a meta-analytical tool than a substantive economic mechanism that Smith would have discussed in the context of capital accumulation.

vsm_relevance — 2.0 / 5.0

The entity could theoretically relate to S3 (monitoring/audit) or S4 (intelligence gathering), but it's too abstract and modern in conception to meaningfully map to VSM systems in the context of Smith's economic analysis. It lacks the structural specificity needed for clear VSM placement.

explanatory_value — 1.0 / 5.0

This entity adds no genuine explanatory power about economic mechanisms or relationships that Smith identified. It merely labels a modern analytical practice without illuminating any underlying economic structure or causal relationship from the source material.

3.4 KiB Raw Blame History