Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.6 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

frozen_ocean_barrier

null

2026-02-23T05:31:52.980730

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is clear and specific, identifying a distinct physical phenomenon (frozen waterways) and its economic consequences (barriers to trade and market development). It avoids circularity and captures a concrete concept rather than a vague abstraction.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's text, specifically his discussion of "the frozen ocean of Tartary" as a barrier that "admits of no navigation" and contributes to the uncivilized state of certain regions. The concept faithfully represents Smith's actual argument about geographical barriers to commerce.

name	value	max_value	rationale
domain_placement	5.0	5.0	The placement in the "Exchange" domain is entirely appropriate, as this entity directly concerns barriers to trade and commerce. Smith uses this example specifically to illustrate how geographical constraints limit market development and economic exchange.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	This entity has moderate VSM relevance, primarily relating to S4 (intelligence/environmental adaptation) as it represents an environmental constraint that economic systems must recognize and adapt to. However, it's more of an external constraint than an active system component.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity provides genuine explanatory power by illustrating how geographical barriers create structural limitations on market development and specialization. It demonstrates a concrete mechanism by which physical environment shapes economic possibilities, supporting Smith's broader theory about the relationship between navigation, trade, and civilization.

Evaluation: Frozen Ocean Barrier

definition_precision — 4.0 / 5.0

The definition is clear and specific, identifying a distinct physical phenomenon (frozen waterways) and its economic consequences (barriers to trade and market development). It avoids circularity and captures a concrete concept rather than a vague abstraction.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's text, specifically his discussion of "the frozen ocean of Tartary" as a barrier that "admits of no navigation" and contributes to the uncivilized state of certain regions. The concept faithfully represents Smith's actual argument about geographical barriers to commerce.

domain_placement — 5.0 / 5.0

The placement in the "Exchange" domain is entirely appropriate, as this entity directly concerns barriers to trade and commerce. Smith uses this example specifically to illustrate how geographical constraints limit market development and economic exchange.

vsm_relevance — 3.0 / 5.0

This entity has moderate VSM relevance, primarily relating to S4 (intelligence/environmental adaptation) as it represents an environmental constraint that economic systems must recognize and adapt to. However, it's more of an external constraint than an active system component.

explanatory_value — 4.0 / 5.0

The entity provides genuine explanatory power by illustrating how geographical barriers create structural limitations on market development and specialization. It demonstrates a concrete mechanism by which physical environment shapes economic possibilities, supporting Smith's broader theory about the relationship between navigation, trade, and civilization.

3.6 KiB Raw Blame History