Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.2 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

truck

null

2026-02-23T06:34:31.324746

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes "truck" as a specific form of exchange (bartering goods/commodities) and positions it within Smith's tripartite classification. While brief, it avoids circularity and captures a distinct concept rather than being vague.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's actual text from Book I, Chapter 2, where he explicitly discusses the human propensity to "truck, barter and exchange" as foundational to the division of labour. The definition accurately reflects Smith's usage of the term.

name	value	max_value	rationale
domain_placement	5.0	5.0	Placement in the "Exchange" domain is entirely appropriate, as truck represents a fundamental mechanism of exchange that underlies economic organization. This is precisely where such a concept belongs thematically.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	Truck is a basic operational activity that might loosely relate to S1 (primary operations), but it's more of a foundational economic behavior than a distinct organizational system. It lacks the structural complexity to map meaningfully to VSM systems.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity provides genuine explanatory value by identifying one of the three fundamental mechanisms that Smith argues enables division of labour and economic specialization. It illuminates a basic structural relation in economic organization rather than merely naming a surface phenomenon.

Evaluation: Truck

definition_precision — 4.0 / 5.0

The definition clearly distinguishes "truck" as a specific form of exchange (bartering goods/commodities) and positions it within Smith's tripartite classification. While brief, it avoids circularity and captures a distinct concept rather than being vague.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's actual text from Book I, Chapter 2, where he explicitly discusses the human propensity to "truck, barter and exchange" as foundational to the division of labour. The definition accurately reflects Smith's usage of the term.

domain_placement — 5.0 / 5.0

Placement in the "Exchange" domain is entirely appropriate, as truck represents a fundamental mechanism of exchange that underlies economic organization. This is precisely where such a concept belongs thematically.

vsm_relevance — 2.0 / 5.0

Truck is a basic operational activity that might loosely relate to S1 (primary operations), but it's more of a foundational economic behavior than a distinct organizational system. It lacks the structural complexity to map meaningfully to VSM systems.

explanatory_value — 4.0 / 5.0

This entity provides genuine explanatory value by identifying one of the three fundamental mechanisms that Smith argues enables division of labour and economic specialization. It illuminates a basic structural relation in economic organization rather than merely naming a surface phenomenon.

3.2 KiB Raw Blame History