Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.2 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

agricultural_price_transmission

null

2026-02-23T00:30:36.442165

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly describes a specific economic mechanism - how price changes propagate between agricultural markets through trade and transportation. It avoids circularity and captures a distinct concept of market interconnectedness.

name	value	max_value	rationale
source_grounding	4.0	5.0	Smith extensively discusses agricultural markets, transportation improvements, and regional price variations in Book I, Chapter 11, making this concept well-grounded in the source text. The entity accurately reflects Smith's analysis of market integration effects.

name	value	max_value	rationale
domain_placement	5.0	5.0	"Exchange" is the perfect domain placement since price transmission is fundamentally about how markets communicate and coordinate through trading relationships. This captures the core mechanism of market exchange that Smith analyzes.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps naturally to S2 (coordination/anti-oscillation) as it describes how markets coordinate to reduce price disparities and dampen regional oscillations. It also has elements of S4 (intelligence) as markets transmit information about supply/demand conditions.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity illuminates a key structural mechanism in Smith's economic theory - how markets self-organize and coordinate through price signals. It explains how local supply/demand shocks get transmitted and absorbed across the broader economic system.

Evaluation: Agricultural Price Transmission

definition_precision — 4.0 / 5.0

The definition clearly describes a specific economic mechanism - how price changes propagate between agricultural markets through trade and transportation. It avoids circularity and captures a distinct concept of market interconnectedness.

source_grounding — 4.0 / 5.0

Smith extensively discusses agricultural markets, transportation improvements, and regional price variations in Book I, Chapter 11, making this concept well-grounded in the source text. The entity accurately reflects Smith's analysis of market integration effects.

domain_placement — 5.0 / 5.0

"Exchange" is the perfect domain placement since price transmission is fundamentally about how markets communicate and coordinate through trading relationships. This captures the core mechanism of market exchange that Smith analyzes.

vsm_relevance — 4.0 / 5.0

This entity maps naturally to S2 (coordination/anti-oscillation) as it describes how markets coordinate to reduce price disparities and dampen regional oscillations. It also has elements of S4 (intelligence) as markets transmit information about supply/demand conditions.

explanatory_value — 4.0 / 5.0

The entity illuminates a key structural mechanism in Smith's economic theory - how markets self-organize and coordinate through price signals. It explains how local supply/demand shocks get transmitted and absorbed across the broader economic system.

3.2 KiB Raw Blame History