Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

annual_plate_addition_estimation

null

2026-02-23T00:33:49.053654

4.4

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is quite precise, clearly distinguishing between new plate made from fresh silver versus recycled old plate, and explicitly connects this to determining true silver import demand. The concept is distinct and well-bounded rather than vague.

name	value	max_value	rationale
source_grounding	5.0	5.0	This is directly grounded in Smith's actual analysis in Book IV, Chapter 6, where he makes this specific argument about plate recycling to challenge assumptions about silver import needs. The entity accurately reflects Smith's reasoning without introducing foreign concepts.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Exchange" domain is perfectly appropriate since this estimation directly relates to understanding the balance of trade and the true demand for imported silver. It's fundamentally about international exchange dynamics rather than production or consumption per se.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This maps well to S4 (intelligence/environmental adaptation) as it represents analytical intelligence gathering about trade patterns and market realities that inform policy decisions. It could also relate to S3 as an audit function challenging conventional assumptions about trade balances.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity illuminates an important mechanism in Smith's argument against mercantilism - showing how recycling dynamics affect import demand calculations. It reveals structural relations between domestic consumption patterns and international trade flows rather than just naming a surface phenomenon.

Evaluation: Annual Plate Addition Estimation

definition_precision — 4.0 / 5.0

The definition is quite precise, clearly distinguishing between new plate made from fresh silver versus recycled old plate, and explicitly connects this to determining true silver import demand. The concept is distinct and well-bounded rather than vague.

source_grounding — 5.0 / 5.0

This is directly grounded in Smith's actual analysis in Book IV, Chapter 6, where he makes this specific argument about plate recycling to challenge assumptions about silver import needs. The entity accurately reflects Smith's reasoning without introducing foreign concepts.

domain_placement — 5.0 / 5.0

The "Exchange" domain is perfectly appropriate since this estimation directly relates to understanding the balance of trade and the true demand for imported silver. It's fundamentally about international exchange dynamics rather than production or consumption per se.

vsm_relevance — 4.0 / 5.0

This maps well to S4 (intelligence/environmental adaptation) as it represents analytical intelligence gathering about trade patterns and market realities that inform policy decisions. It could also relate to S3 as an audit function challenging conventional assumptions about trade balances.

explanatory_value — 4.0 / 5.0

This entity illuminates an important mechanism in Smith's argument against mercantilism - showing how recycling dynamics affect import demand calculations. It reveals structural relations between domestic consumption patterns and international trade flows rather than just naming a surface phenomenon.

3.5 KiB Raw Blame History