Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.2 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

coal_heaver

null

2026-02-23T04:43:29.357053

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is precise and specific, clearly identifying coal-heavers as laborers who unload coal from ships and noting their specific wage premium (4-5 times common labor). It captures a distinct occupational category rather than a vague concept.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's text from Book I, Chapter 10, where he explicitly discusses coal-heavers as an example of how disagreeable and irregular work commands higher wages. The wage multiple cited appears to come directly from Smith's analysis.

name	value	max_value	rationale
domain_placement	4.0	5.0	The "Distribution" domain is appropriate since this relates to wage determination and how labor compensation varies based on working conditions. However, it could arguably fit in a labor market or wage theory subdomain if one existed.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	Coal-heavers represent a specific type of worker rather than a systemic function, making them largely VSM-neutral. They don't naturally map to any particular VSM system (S1-S5) as they are an empirical example rather than a structural element.

name	value	max_value	rationale
explanatory_value	5.0	5.0	This entity provides excellent explanatory value by illustrating the concrete mechanism of how multiple negative job characteristics (arduous, dirty, irregular) combine to drive wages significantly above market averages. It demonstrates Smith's theory of compensating wage differentials in action.

Evaluation: Coal Heaver

definition_precision — 4.0 / 5.0

The definition is precise and specific, clearly identifying coal-heavers as laborers who unload coal from ships and noting their specific wage premium (4-5 times common labor). It captures a distinct occupational category rather than a vague concept.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's text from Book I, Chapter 10, where he explicitly discusses coal-heavers as an example of how disagreeable and irregular work commands higher wages. The wage multiple cited appears to come directly from Smith's analysis.

domain_placement — 4.0 / 5.0

The "Distribution" domain is appropriate since this relates to wage determination and how labor compensation varies based on working conditions. However, it could arguably fit in a labor market or wage theory subdomain if one existed.

vsm_relevance — 2.0 / 5.0

Coal-heavers represent a specific type of worker rather than a systemic function, making them largely VSM-neutral. They don't naturally map to any particular VSM system (S1-S5) as they are an empirical example rather than a structural element.

explanatory_value — 5.0 / 5.0

This entity provides excellent explanatory value by illustrating the concrete mechanism of how multiple negative job characteristics (arduous, dirty, irregular) combine to drive wages significantly above market averages. It demonstrates Smith's theory of compensating wage differentials in action.

3.2 KiB Raw Blame History