Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.0 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

wool_grower

null

2026-02-23T06:39:45.453405

4.4

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is clear and precise, identifying a specific agricultural role in the wool production chain. It avoids circularity and captures a distinct occupational category rather than a vague concept.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's text from Book I, Chapter 1, where he explicitly discusses wool growers as part of the division of labor in woollen manufacturing. The entity accurately reflects Smith's actual example without adding external concepts.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Production" domain assignment is entirely appropriate, as wool growers are primary producers in the agricultural sector who create raw materials for manufacturing. This placement correctly categorizes their economic function.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps clearly to S1 (primary operations) as wool growers represent fundamental productive activities that generate raw materials for the economic system. The mapping is natural and meaningful within the VSM framework.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity effectively illustrates Smith's key insight about how division of labor extends beyond manufacturing into agricultural specialization. It demonstrates the structural relationship between primary production and downstream manufacturing processes.

Evaluation: Wool Grower

definition_precision — 4.0 / 5.0

The definition is clear and precise, identifying a specific agricultural role in the wool production chain. It avoids circularity and captures a distinct occupational category rather than a vague concept.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's text from Book I, Chapter 1, where he explicitly discusses wool growers as part of the division of labor in woollen manufacturing. The entity accurately reflects Smith's actual example without adding external concepts.

domain_placement — 5.0 / 5.0

The "Production" domain assignment is entirely appropriate, as wool growers are primary producers in the agricultural sector who create raw materials for manufacturing. This placement correctly categorizes their economic function.

vsm_relevance — 4.0 / 5.0

This entity maps clearly to S1 (primary operations) as wool growers represent fundamental productive activities that generate raw materials for the economic system. The mapping is natural and meaningful within the VSM framework.

explanatory_value — 4.0 / 5.0

The entity effectively illustrates Smith's key insight about how division of labor extends beyond manufacturing into agricultural specialization. It demonstrates the structural relationship between primary production and downstream manufacturing processes.

3.0 KiB Raw Blame History