feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: domestic_industry_protection
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:07:14.327479'
|
||||
overall_score: 4.4
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly distinguishes domestic industry protection as
|
||||
government policies using specific mechanisms (import restrictions, tariffs, prohibitions)
|
||||
for a defined purpose (shielding domestic producers). It avoids circularity and
|
||||
captures a distinct policy concept with concrete examples.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is directly grounded in Smith's analysis in Book IV, Chapter
|
||||
2, with specific examples mentioned (cattle imports, corn duties, woollen goods
|
||||
restrictions) that Smith actually discusses. The concept reflects Smith's own
|
||||
examination of protective trade policies.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: '"Regulation" is the correct domain assignment, as domestic industry
|
||||
protection represents government regulatory intervention in markets through trade
|
||||
policy mechanisms. This clearly falls under regulatory rather than market or production
|
||||
domains.'
|
||||
- name: vsm_relevance
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps well to S3 (internal regulation) as it represents regulatory
|
||||
mechanisms that control the economic system's internal operations by managing
|
||||
external inputs. It also has some S4 relevance as protection policies respond
|
||||
to perceived environmental threats from foreign competition.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: "The entity illuminates an important structural mechanism in Smith's\
|
||||
\ analysis\u2014how government intervention shapes market dynamics and resource\
|
||||
\ allocation. It explains the regulatory apparatus behind trade protection rather\
|
||||
\ than merely naming a surface phenomenon."
|
||||
---
|
||||
|
||||
# Evaluation: Domestic Industry Protection
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition clearly distinguishes domestic industry protection as government policies using specific mechanisms (import restrictions, tariffs, prohibitions) for a defined purpose (shielding domestic producers). It avoids circularity and captures a distinct policy concept with concrete examples.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This entity is directly grounded in Smith's analysis in Book IV, Chapter 2, with specific examples mentioned (cattle imports, corn duties, woollen goods restrictions) that Smith actually discusses. The concept reflects Smith's own examination of protective trade policies.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
"Regulation" is the correct domain assignment, as domestic industry protection represents government regulatory intervention in markets through trade policy mechanisms. This clearly falls under regulatory rather than market or production domains.
|
||||
|
||||
## vsm_relevance — 4.0 / 5.0
|
||||
|
||||
This entity maps well to S3 (internal regulation) as it represents regulatory mechanisms that control the economic system's internal operations by managing external inputs. It also has some S4 relevance as protection policies respond to perceived environmental threats from foreign competition.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
The entity illuminates an important structural mechanism in Smith's analysis—how government intervention shapes market dynamics and resource allocation. It explains the regulatory apparatus behind trade protection rather than merely naming a surface phenomenon.
|
||||
Reference in New Issue
Block a user