Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.7 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

madeira_wine_trade_exception

null

2026-02-23T05:41:46.449736

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly identifies a specific trade arrangement with precise parameters - Madeira wine's exemption from restrictions that applied to European wines due to differential duty structures. The concept is distinct and well-bounded, though it could be slightly more precise about the exact nature of the "heavy duties" mechanism.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity appears to be directly grounded in Smith's actual discussion of colonial trade policies and wine import regulations in Book IV, Chapter 4. The specific details about duty drawbacks and the resulting market preferences align with Smith's analytical approach to trade policy consequences.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Exchange" domain is perfectly appropriate as this entity concerns international trade arrangements, import/export regulations, and market access policies. This is fundamentally about the mechanics of commercial exchange between regions.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	This maps reasonably well to S4 (intelligence/environmental adaptation) as it represents how trade systems adapt to regulatory environments and create market intelligence about preferences. However, it's somewhat specific to be a strong VSM exemplar and doesn't clearly illuminate broader systemic coordination principles.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity effectively illuminates how trade policy exceptions can create unintended market consequences and preference formation - a genuine economic mechanism. It demonstrates Smith's insight into how regulatory asymmetries shape commercial patterns, though it's somewhat narrow in scope.

Evaluation: Madeira Wine Trade Exception

definition_precision — 4.0 / 5.0

The definition clearly identifies a specific trade arrangement with precise parameters - Madeira wine's exemption from restrictions that applied to European wines due to differential duty structures. The concept is distinct and well-bounded, though it could be slightly more precise about the exact nature of the "heavy duties" mechanism.

source_grounding — 5.0 / 5.0

This entity appears to be directly grounded in Smith's actual discussion of colonial trade policies and wine import regulations in Book IV, Chapter 4. The specific details about duty drawbacks and the resulting market preferences align with Smith's analytical approach to trade policy consequences.

domain_placement — 5.0 / 5.0

The "Exchange" domain is perfectly appropriate as this entity concerns international trade arrangements, import/export regulations, and market access policies. This is fundamentally about the mechanics of commercial exchange between regions.

vsm_relevance — 3.0 / 5.0

This maps reasonably well to S4 (intelligence/environmental adaptation) as it represents how trade systems adapt to regulatory environments and create market intelligence about preferences. However, it's somewhat specific to be a strong VSM exemplar and doesn't clearly illuminate broader systemic coordination principles.

explanatory_value — 4.0 / 5.0

This entity effectively illuminates how trade policy exceptions can create unintended market consequences and preference formation - a genuine economic mechanism. It demonstrates Smith's insight into how regulatory asymmetries shape commercial patterns, though it's somewhat narrow in scope.

3.7 KiB Raw Blame History