Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_mechanisms

null

2026-02-23T05:18:49.254418

2.0

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and umbrella-like, essentially describing "how economic systems work" rather than identifying a specific, distinct concept. It lists multiple disparate elements (processes, institutions, procedures, commerce, trade, production, distribution) without clear boundaries or precision.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does discuss various economic mechanisms throughout his work, this entity appears to be a modern analytical abstraction rather than a concept Smith himself articulated. The reference to "Book IV, Chapter 0" is also problematic as Book IV doesn't have a Chapter 0.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the broad, meta-level nature of this concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't wrong but reflects the entity's lack of specificity.

name	value	max_value	rationale
vsm_relevance	1.0	5.0	This entity is far too abstract and all-encompassing to map meaningfully to any specific VSM system—it essentially describes the entire viable system rather than any particular component or function. It lacks the specificity needed for VSM analysis.

name	value	max_value	rationale
explanatory_value	2.0	5.0	While understanding economic mechanisms is important, this entity merely names a category without illuminating specific causal relationships or structural dynamics. It provides a label rather than genuine analytical insight into how economic systems actually function.

Evaluation: Economic System Mechanisms

definition_precision — 2.0 / 5.0

The definition is overly broad and umbrella-like, essentially describing "how economic systems work" rather than identifying a specific, distinct concept. It lists multiple disparate elements (processes, institutions, procedures, commerce, trade, production, distribution) without clear boundaries or precision.

source_grounding — 2.0 / 5.0

While Smith does discuss various economic mechanisms throughout his work, this entity appears to be a modern analytical abstraction rather than a concept Smith himself articulated. The reference to "Book IV, Chapter 0" is also problematic as Book IV doesn't have a Chapter 0.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the broad, meta-level nature of this concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't wrong but reflects the entity's lack of specificity.

vsm_relevance — 1.0 / 5.0

This entity is far too abstract and all-encompassing to map meaningfully to any specific VSM system—it essentially describes the entire viable system rather than any particular component or function. It lacks the specificity needed for VSM analysis.

explanatory_value — 2.0 / 5.0

While understanding economic mechanisms is important, this entity merely names a category without illuminating specific causal relationships or structural dynamics. It provides a label rather than genuine analytical insight into how economic systems actually function.

3.4 KiB Raw Blame History