Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.3 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_explanation

null

2026-02-23T05:15:54.297748

2.6

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and circular, essentially defining "economic system explanation" as "explaining economic systems." It lacks specificity about what constitutes such an explanation versus other forms of economic discourse or analysis.

name	value	max_value	rationale
source_grounding	3.0	5.0	While Smith does indicate his intention to explain commercial and agricultural systems in Book IV, the entity extrapolates this into a general concept of "economic system explanation" that goes beyond what the source text specifically discusses. The grounding is present but stretched.

name	value	max_value	rationale
domain_placement	4.0	5.0	"General Theory" is an appropriate domain placement since this concerns Smith's methodological approach to explaining economic systems rather than specific operational mechanisms. The domain assignment correctly captures the meta-theoretical nature of the concept.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity is too abstract and meta-analytical to map naturally to any specific VSM system. It describes the act of explanation itself rather than functional economic mechanisms that would correspond to S1-S5 operations.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity provides minimal explanatory power, essentially naming the fact that Smith explains economic systems without illuminating any specific mechanisms or structural relations. It captures a methodological observation rather than substantive economic insight.

Evaluation: Economic System Explanation

definition_precision — 2.0 / 5.0

The definition is overly broad and circular, essentially defining "economic system explanation" as "explaining economic systems." It lacks specificity about what constitutes such an explanation versus other forms of economic discourse or analysis.

source_grounding — 3.0 / 5.0

While Smith does indicate his intention to explain commercial and agricultural systems in Book IV, the entity extrapolates this into a general concept of "economic system explanation" that goes beyond what the source text specifically discusses. The grounding is present but stretched.

domain_placement — 4.0 / 5.0

"General Theory" is an appropriate domain placement since this concerns Smith's methodological approach to explaining economic systems rather than specific operational mechanisms. The domain assignment correctly captures the meta-theoretical nature of the concept.

vsm_relevance — 2.0 / 5.0

This entity is too abstract and meta-analytical to map naturally to any specific VSM system. It describes the act of explanation itself rather than functional economic mechanisms that would correspond to S1-S5 operations.

explanatory_value — 2.0 / 5.0

The entity provides minimal explanatory power, essentially naming the fact that Smith explains economic systems without illuminating any specific mechanisms or structural relations. It captures a methodological observation rather than substantive economic insight.

3.3 KiB Raw Blame History