Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.3 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

manufacturer

null

2026-02-23T05:42:01.735560

4.6

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes manufacturers from other types of workers by emphasizing specialization in transforming raw materials and performing only one aspect of production. It's precise and non-circular, though it could be slightly more specific about the industrial context.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's famous pin factory example from Book I, Chapter 1, where he explicitly discusses how manufacturers become specialized workers performing single operations. The contrast with multifunctional farmers is also authentic to Smith's text.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Production" domain is perfectly appropriate for manufacturers, as they are fundamentally concerned with the transformation of materials into goods. This placement aligns with Smith's focus on productive processes and division of labor.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	Manufacturers map naturally to S1 (primary operations) as they perform the core productive work of transforming inputs into outputs. They represent the operational level where actual value creation occurs within the economic system.

name	value	max_value	rationale
explanatory_value	5.0	5.0	This entity illuminates a crucial mechanism in Smith's theory—how division of labor creates specialized roles that increase productivity. It's not merely descriptive but explains the structural transformation from generalist to specialist production that drives economic efficiency.

Evaluation: Manufacturer

definition_precision — 4.0 / 5.0

The definition clearly distinguishes manufacturers from other types of workers by emphasizing specialization in transforming raw materials and performing only one aspect of production. It's precise and non-circular, though it could be slightly more specific about the industrial context.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's famous pin factory example from Book I, Chapter 1, where he explicitly discusses how manufacturers become specialized workers performing single operations. The contrast with multifunctional farmers is also authentic to Smith's text.

domain_placement — 5.0 / 5.0

The "Production" domain is perfectly appropriate for manufacturers, as they are fundamentally concerned with the transformation of materials into goods. This placement aligns with Smith's focus on productive processes and division of labor.

vsm_relevance — 4.0 / 5.0

Manufacturers map naturally to S1 (primary operations) as they perform the core productive work of transforming inputs into outputs. They represent the operational level where actual value creation occurs within the economic system.

explanatory_value — 5.0 / 5.0

This entity illuminates a crucial mechanism in Smith's theory—how division of labor creates specialized roles that increase productivity. It's not merely descriptive but explains the structural transformation from generalist to specialist production that drives economic efficiency.

3.3 KiB Raw Blame History