Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_knowledge

null

2026-02-23T05:18:07.197443

2.6

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and circular, essentially defining economic knowledge as "understanding economic principles" without specifying what constitutes this knowledge or its boundaries. It functions more as a vague umbrella term than a precise concept with clear analytical utility.

name	value	max_value	rationale
source_grounding	3.0	5.0	While Smith does discuss political economy as a science requiring understanding by governors and legislators, this entity extrapolates beyond what the source explicitly states by creating a general category of "economic system knowledge." The source grounding exists but is stretched thin to support this broad conceptual framework.

name	value	max_value	rationale
domain_placement	4.0	5.0	The placement in "General Theory" domain is appropriate given the abstract, foundational nature of this concept. It correctly identifies this as a meta-level concern about economic understanding rather than a specific mechanism or policy.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity is too abstract and general to map meaningfully to specific VSM systems, though it might loosely relate to S4 (intelligence) or S5 (policy) functions. Its broad scope makes it VSM-neutral rather than providing clear structural insight into system operations.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity adds little explanatory power beyond naming the obvious fact that economic understanding is important for economic actors. It doesn't illuminate specific mechanisms, relationships, or structural dynamics that would enhance analytical understanding of Smith's system.

Evaluation: Economic System Knowledge

definition_precision — 2.0 / 5.0

The definition is overly broad and circular, essentially defining economic knowledge as "understanding economic principles" without specifying what constitutes this knowledge or its boundaries. It functions more as a vague umbrella term than a precise concept with clear analytical utility.

source_grounding — 3.0 / 5.0

While Smith does discuss political economy as a science requiring understanding by governors and legislators, this entity extrapolates beyond what the source explicitly states by creating a general category of "economic system knowledge." The source grounding exists but is stretched thin to support this broad conceptual framework.

domain_placement — 4.0 / 5.0

The placement in "General Theory" domain is appropriate given the abstract, foundational nature of this concept. It correctly identifies this as a meta-level concern about economic understanding rather than a specific mechanism or policy.

vsm_relevance — 2.0 / 5.0

This entity is too abstract and general to map meaningfully to specific VSM systems, though it might loosely relate to S4 (intelligence) or S5 (policy) functions. Its broad scope makes it VSM-neutral rather than providing clear structural insight into system operations.

explanatory_value — 2.0 / 5.0

The entity adds little explanatory power beyond naming the obvious fact that economic understanding is important for economic actors. It doesn't illuminate specific mechanisms, relationships, or structural dynamics that would enhance analytical understanding of Smith's system.

3.5 KiB Raw Blame History