Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_consequence

null

2026-02-23T05:14:00.285904

2.2

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and umbrella-like, encompassing "outcomes and results" across multiple domains (wealth, distribution, social organization) without clear boundaries. It reads more like a general category than a precise concept that could be operationally distinguished from related phenomena.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does analyze consequences of different economic systems, this entity appears to abstract away from his specific discussions into a generic meta-concept. The attribution to "Book IV, Chapter 0" is suspicious since Chapter 0 typically doesn't exist, and the definition doesn't reflect Smith's particular analytical framework.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the broad, abstract nature of this concept, though the entity might be better placed in a more specific domain related to comparative systems analysis. The current placement accurately reflects its high level of generality.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity is too abstract and outcome-focused to map naturally to any specific VSM system, as it describes results rather than organizational functions or processes. It lacks the structural specificity needed for meaningful VSM integration.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity merely names a surface phenomenon (that economic systems have consequences) without illuminating specific mechanisms or structural relations that Smith identifies. It adds little explanatory power beyond stating the obvious fact that systems produce outcomes.

Evaluation: Economic System Consequence

definition_precision — 2.0 / 5.0

The definition is overly broad and umbrella-like, encompassing "outcomes and results" across multiple domains (wealth, distribution, social organization) without clear boundaries. It reads more like a general category than a precise concept that could be operationally distinguished from related phenomena.

source_grounding — 2.0 / 5.0

While Smith does analyze consequences of different economic systems, this entity appears to abstract away from his specific discussions into a generic meta-concept. The attribution to "Book IV, Chapter 0" is suspicious since Chapter 0 typically doesn't exist, and the definition doesn't reflect Smith's particular analytical framework.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the broad, abstract nature of this concept, though the entity might be better placed in a more specific domain related to comparative systems analysis. The current placement accurately reflects its high level of generality.

vsm_relevance — 2.0 / 5.0

This entity is too abstract and outcome-focused to map naturally to any specific VSM system, as it describes results rather than organizational functions or processes. It lacks the structural specificity needed for meaningful VSM integration.

explanatory_value — 2.0 / 5.0

The entity merely names a surface phenomenon (that economic systems have consequences) without illuminating specific mechanisms or structural relations that Smith identifies. It adds little explanatory power beyond stating the obvious fact that systems produce outcomes.

3.5 KiB Raw Blame History