Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

domestic_industry_protection

null

2026-02-23T05:07:14.327479

4.4

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes domestic industry protection as government policies using specific mechanisms (import restrictions, tariffs, prohibitions) for a defined purpose (shielding domestic producers). It avoids circularity and captures a distinct policy concept with concrete examples.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's analysis in Book IV, Chapter 2, with specific examples mentioned (cattle imports, corn duties, woollen goods restrictions) that Smith actually discusses. The concept reflects Smith's own examination of protective trade policies.

name	value	max_value	rationale
domain_placement	5.0	5.0	"Regulation" is the correct domain assignment, as domestic industry protection represents government regulatory intervention in markets through trade policy mechanisms. This clearly falls under regulatory rather than market or production domains.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S3 (internal regulation) as it represents regulatory mechanisms that control the economic system's internal operations by managing external inputs. It also has some S4 relevance as protection policies respond to perceived environmental threats from foreign competition.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity illuminates an important structural mechanism in Smith's analysis—how government intervention shapes market dynamics and resource allocation. It explains the regulatory apparatus behind trade protection rather than merely naming a surface phenomenon.

Evaluation: Domestic Industry Protection

definition_precision — 4.0 / 5.0

The definition clearly distinguishes domestic industry protection as government policies using specific mechanisms (import restrictions, tariffs, prohibitions) for a defined purpose (shielding domestic producers). It avoids circularity and captures a distinct policy concept with concrete examples.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's analysis in Book IV, Chapter 2, with specific examples mentioned (cattle imports, corn duties, woollen goods restrictions) that Smith actually discusses. The concept reflects Smith's own examination of protective trade policies.

domain_placement — 5.0 / 5.0

"Regulation" is the correct domain assignment, as domestic industry protection represents government regulatory intervention in markets through trade policy mechanisms. This clearly falls under regulatory rather than market or production domains.

vsm_relevance — 4.0 / 5.0

This entity maps well to S3 (internal regulation) as it represents regulatory mechanisms that control the economic system's internal operations by managing external inputs. It also has some S4 relevance as protection policies respond to perceived environmental threats from foreign competition.

explanatory_value — 4.0 / 5.0

The entity illuminates an important structural mechanism in Smith's analysis—how government intervention shapes market dynamics and resource allocation. It explains the regulatory apparatus behind trade protection rather than merely naming a surface phenomenon.

3.5 KiB Raw Blame History