Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

barbarous_nations_barrier

null

2026-02-23T04:37:38.728725

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly identifies a specific type of barrier to trade - political and security obstacles in regions with hostile populations that increase transportation costs and risks. It avoids circularity and captures a distinct concept beyond general trade barriers.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's specific example of the difficulty of transporting goods "through the territories of so many barbarous nations" between London and Calcutta in Book I, Chapter 3. The concept emerges naturally from Smith's own language and illustration.

name	value	max_value	rationale
domain_placement	4.0	5.0	"Regulation" is appropriate since this involves political and institutional factors that govern trade safety and feasibility. While it could potentially fit under "Geography" or "Trade," the regulatory/institutional nature of political barriers makes this placement sound.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	This maps reasonably well to S4 (intelligence/environmental adaptation) as it represents external environmental threats that a trading system must monitor and adapt to. However, it's somewhat abstract and could also relate to S3 (internal regulation) regarding risk management.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity illuminates an important mechanism by which political instability and security concerns limit market extent, even when natural transportation advantages exist. It helps explain why some potentially profitable trade routes remain underdeveloped due to non-economic factors.

Evaluation: Barbarous Nations Barrier

definition_precision — 4.0 / 5.0

The definition clearly identifies a specific type of barrier to trade - political and security obstacles in regions with hostile populations that increase transportation costs and risks. It avoids circularity and captures a distinct concept beyond general trade barriers.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's specific example of the difficulty of transporting goods "through the territories of so many barbarous nations" between London and Calcutta in Book I, Chapter 3. The concept emerges naturally from Smith's own language and illustration.

domain_placement — 4.0 / 5.0

"Regulation" is appropriate since this involves political and institutional factors that govern trade safety and feasibility. While it could potentially fit under "Geography" or "Trade," the regulatory/institutional nature of political barriers makes this placement sound.

vsm_relevance — 3.0 / 5.0

This maps reasonably well to S4 (intelligence/environmental adaptation) as it represents external environmental threats that a trading system must monitor and adapt to. However, it's somewhat abstract and could also relate to S3 (internal regulation) regarding risk management.

explanatory_value — 4.0 / 5.0

This entity illuminates an important mechanism by which political instability and security concerns limit market extent, even when natural transportation advantages exist. It helps explain why some potentially profitable trade routes remain underdeveloped due to non-economic factors.

3.5 KiB Raw Blame History