Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_actor

null

2026-02-23T05:12:14.020874

2.0

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and umbrella-like, encompassing "individuals, organizations, and entities" without clear boundaries or distinguishing characteristics. It reads more like a general category than a precise concept that captures a distinct analytical unit.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith certainly discusses various economic actors throughout The Wealth of Nations, this entity appears to be a modern analytical construct rather than a concept Smith explicitly develops. The attribution to "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, and the definition seems to impose contemporary frameworks on Smith's text.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the broad nature of this entity, though the vagueness of the concept makes any domain placement somewhat arbitrary. The entity is too abstract to clearly belong to a more specific economic domain.

name	value	max_value	rationale
vsm_relevance	1.0	5.0	This entity is far too abstract and general to map meaningfully to any specific VSM system - it could theoretically encompass actors across all systems S1-S5. The concept lacks the structural specificity needed for useful VSM mapping.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity provides minimal explanatory power beyond simply naming the obvious fact that economic systems have participants. It doesn't illuminate specific mechanisms, relationships, or structural dynamics that would advance understanding of Smith's political economy.

Evaluation: Economic System Actor

definition_precision — 2.0 / 5.0

The definition is overly broad and umbrella-like, encompassing "individuals, organizations, and entities" without clear boundaries or distinguishing characteristics. It reads more like a general category than a precise concept that captures a distinct analytical unit.

source_grounding — 2.0 / 5.0

While Smith certainly discusses various economic actors throughout The Wealth of Nations, this entity appears to be a modern analytical construct rather than a concept Smith explicitly develops. The attribution to "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, and the definition seems to impose contemporary frameworks on Smith's text.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the broad nature of this entity, though the vagueness of the concept makes any domain placement somewhat arbitrary. The entity is too abstract to clearly belong to a more specific economic domain.

vsm_relevance — 1.0 / 5.0

This entity is far too abstract and general to map meaningfully to any specific VSM system - it could theoretically encompass actors across all systems S1-S5. The concept lacks the structural specificity needed for useful VSM mapping.

explanatory_value — 2.0 / 5.0

The entity provides minimal explanatory power beyond simply naming the obvious fact that economic systems have participants. It doesn't illuminate specific mechanisms, relationships, or structural dynamics that would advance understanding of Smith's political economy.

3.4 KiB Raw Blame History