Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.6 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

drawbacks

null

2026-02-23T05:07:57.291223

4.8

name	value	max_value	rationale
definition_precision	5.0	5.0	The definition is highly precise and non-circular, clearly explaining drawbacks as a specific tax refund mechanism for exporters that recovers excise or inland duties. It distinctly captures the concept's purpose (maintaining export competitiveness) and mechanism (duty recovery) without vagueness.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is thoroughly grounded in Book IV, Chapter 4, where Smith explicitly discusses drawbacks as a form of export encouragement with specific examples (tobacco, sugar, wine). The context accurately reflects Smith's analysis of drawbacks as the most reasonable export promotion mechanism.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Regulation" domain placement is perfectly appropriate, as drawbacks represent a specific regulatory mechanism involving tax policy and trade controls. This fits squarely within governmental regulatory frameworks rather than pure market operations or theoretical concepts.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S3 (internal regulation) as it represents a specific regulatory control mechanism that manages the relationship between domestic taxation and export competitiveness. It could also relate to S2 (coordination) in balancing revenue collection with trade promotion goals.

name	value	max_value	rationale
explanatory_value	5.0	5.0	The entity provides excellent explanatory power by illuminating the specific mechanism through which governments can promote exports without artificial subsidies—simply removing the disadvantage created by domestic duties. This reveals an important structural relationship between taxation policy and trade competitiveness.

Evaluation: Drawbacks

definition_precision — 5.0 / 5.0

The definition is highly precise and non-circular, clearly explaining drawbacks as a specific tax refund mechanism for exporters that recovers excise or inland duties. It distinctly captures the concept's purpose (maintaining export competitiveness) and mechanism (duty recovery) without vagueness.

source_grounding — 5.0 / 5.0

This entity is thoroughly grounded in Book IV, Chapter 4, where Smith explicitly discusses drawbacks as a form of export encouragement with specific examples (tobacco, sugar, wine). The context accurately reflects Smith's analysis of drawbacks as the most reasonable export promotion mechanism.

domain_placement — 5.0 / 5.0

The "Regulation" domain placement is perfectly appropriate, as drawbacks represent a specific regulatory mechanism involving tax policy and trade controls. This fits squarely within governmental regulatory frameworks rather than pure market operations or theoretical concepts.

vsm_relevance — 4.0 / 5.0

This entity maps well to S3 (internal regulation) as it represents a specific regulatory control mechanism that manages the relationship between domestic taxation and export competitiveness. It could also relate to S2 (coordination) in balancing revenue collection with trade promotion goals.

explanatory_value — 5.0 / 5.0

The entity provides excellent explanatory power by illuminating the specific mechanism through which governments can promote exports without artificial subsidies—simply removing the disadvantage created by domestic duties. This reveals an important structural relationship between taxation policy and trade competitiveness.

3.6 KiB Raw Blame History