Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.7 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

farmer

null

2026-02-23T05:27:33.407362

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly identifies farmers as agricultural producers and specifies their key characteristic - performing multiple interconnected tasks that resist complete specialization. This captures a distinct concept with clear boundaries, though it could be slightly more precise about what constitutes "multiple interconnected tasks."

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's text from Book I, Chapter 1, where he explicitly contrasts agricultural work with manufacturing to illustrate limits of division of labour. The definition accurately reflects Smith's observation about seasonal variation and task interconnectedness in farming.

name	value	max_value	rationale
domain_placement	5.0	5.0	"Production" is the correct domain placement, as farmers are primary producers in Smith's economic framework. The entity represents a fundamental production unit that demonstrates important principles about how division of labour operates differently across economic sectors.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	Farmers map most naturally to S1 (primary operations) as they represent basic productive units, but the entity's significance lies more in illustrating economic principles about specialization rather than organizational cybernetics. The VSM mapping is possible but not particularly illuminating for this concept.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity provides genuine explanatory power by illustrating the structural limits of division of labour and why agricultural production differs from manufacturing. It helps explain why certain economic sectors resist the efficiency gains possible through extreme specialization.

Evaluation: Farmer

definition_precision — 4.0 / 5.0

The definition clearly identifies farmers as agricultural producers and specifies their key characteristic - performing multiple interconnected tasks that resist complete specialization. This captures a distinct concept with clear boundaries, though it could be slightly more precise about what constitutes "multiple interconnected tasks."

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's text from Book I, Chapter 1, where he explicitly contrasts agricultural work with manufacturing to illustrate limits of division of labour. The definition accurately reflects Smith's observation about seasonal variation and task interconnectedness in farming.

domain_placement — 5.0 / 5.0

"Production" is the correct domain placement, as farmers are primary producers in Smith's economic framework. The entity represents a fundamental production unit that demonstrates important principles about how division of labour operates differently across economic sectors.

vsm_relevance — 3.0 / 5.0

Farmers map most naturally to S1 (primary operations) as they represent basic productive units, but the entity's significance lies more in illustrating economic principles about specialization rather than organizational cybernetics. The VSM mapping is possible but not particularly illuminating for this concept.

explanatory_value — 4.0 / 5.0

This entity provides genuine explanatory power by illustrating the structural limits of division of labour and why agricultural production differs from manufacturing. It helps explain why certain economic sectors resist the efficiency gains possible through extreme specialization.

3.7 KiB Raw Blame History