coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/barbarous_nations_barrier.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: barbarous_nations_barrier
evaluator: null
evaluated_at: '2026-02-23T04:37:38.728725'
overall_score: 4.0
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly identifies a specific type of barrier to trade
- political and security obstacles in regions with hostile populations that increase
transportation costs and risks. It avoids circularity and captures a distinct
concept beyond general trade barriers.
- name: source_grounding
value: 5.0
max_value: 5.0
rationale: This entity is directly grounded in Smith's specific example of the difficulty
of transporting goods "through the territories of so many barbarous nations" between
London and Calcutta in Book I, Chapter 3. The concept emerges naturally from Smith's
own language and illustration.
- name: domain_placement
value: 4.0
max_value: 5.0
rationale: '"Regulation" is appropriate since this involves political and institutional
factors that govern trade safety and feasibility. While it could potentially fit
under "Geography" or "Trade," the regulatory/institutional nature of political
barriers makes this placement sound.'
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: This maps reasonably well to S4 (intelligence/environmental adaptation)
as it represents external environmental threats that a trading system must monitor
and adapt to. However, it's somewhat abstract and could also relate to S3 (internal
regulation) regarding risk management.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: This entity illuminates an important mechanism by which political instability
and security concerns limit market extent, even when natural transportation advantages
exist. It helps explain why some potentially profitable trade routes remain underdeveloped
due to non-economic factors.
---
# Evaluation: Barbarous Nations Barrier
## definition_precision — 4.0 / 5.0
The definition clearly identifies a specific type of barrier to trade - political and security obstacles in regions with hostile populations that increase transportation costs and risks. It avoids circularity and captures a distinct concept beyond general trade barriers.
## source_grounding — 5.0 / 5.0
This entity is directly grounded in Smith's specific example of the difficulty of transporting goods "through the territories of so many barbarous nations" between London and Calcutta in Book I, Chapter 3. The concept emerges naturally from Smith's own language and illustration.
## domain_placement — 4.0 / 5.0
"Regulation" is appropriate since this involves political and institutional factors that govern trade safety and feasibility. While it could potentially fit under "Geography" or "Trade," the regulatory/institutional nature of political barriers makes this placement sound.
## vsm_relevance — 3.0 / 5.0
This maps reasonably well to S4 (intelligence/environmental adaptation) as it represents external environmental threats that a trading system must monitor and adapt to. However, it's somewhat abstract and could also relate to S3 (internal regulation) regarding risk management.
## explanatory_value — 4.0 / 5.0
This entity illuminates an important mechanism by which political instability and security concerns limit market extent, even when natural transportation advantages exist. It helps explain why some potentially profitable trade routes remain underdeveloped due to non-economic factors.