coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/agricultural_price_volatility.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: agricultural_price_volatility
evaluator: null
evaluated_at: '2026-02-23T00:30:45.056374'
overall_score: 4.0
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly captures the concept of price fluctuations in
agricultural markets and identifies specific causal factors (supply, demand, weather,
production changes). It avoids circularity and distinguishes this from general
price movements by focusing on the volatility aspect.
- name: source_grounding
value: 4.0
max_value: 5.0
rationale: Smith does extensively discuss agricultural price fluctuations in Book
I, Chapter 11, examining how corn prices vary with harvests and market conditions.
The entity accurately reflects his analysis of both short-term disruptions and
long-term price trends in agricultural markets.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: '"Exchange" is the correct domain placement since agricultural price
volatility is fundamentally about how prices fluctuate in market transactions.
This belongs squarely within exchange mechanisms rather than production, distribution,
or consumption domains.'
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: This entity has moderate VSM relevance, primarily mapping to S2 (coordination/anti-oscillation)
as price volatility represents oscillatory behavior that markets must manage.
It also touches on S4 (environmental adaptation) as markets respond to external
shocks like weather.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The entity provides good explanatory value by illuminating how agricultural
markets function differently from other markets due to production uncertainties
and seasonal factors. It helps explain market dynamics and the challenges of economic
planning in agriculture.
---
# Evaluation: Agricultural Price Volatility
## definition_precision — 4.0 / 5.0
The definition clearly captures the concept of price fluctuations in agricultural markets and identifies specific causal factors (supply, demand, weather, production changes). It avoids circularity and distinguishes this from general price movements by focusing on the volatility aspect.
## source_grounding — 4.0 / 5.0
Smith does extensively discuss agricultural price fluctuations in Book I, Chapter 11, examining how corn prices vary with harvests and market conditions. The entity accurately reflects his analysis of both short-term disruptions and long-term price trends in agricultural markets.
## domain_placement — 5.0 / 5.0
"Exchange" is the correct domain placement since agricultural price volatility is fundamentally about how prices fluctuate in market transactions. This belongs squarely within exchange mechanisms rather than production, distribution, or consumption domains.
## vsm_relevance — 3.0 / 5.0
This entity has moderate VSM relevance, primarily mapping to S2 (coordination/anti-oscillation) as price volatility represents oscillatory behavior that markets must manage. It also touches on S4 (environmental adaptation) as markets respond to external shocks like weather.
## explanatory_value — 4.0 / 5.0
The entity provides good explanatory value by illuminating how agricultural markets function differently from other markets due to production uncertainties and seasonal factors. It helps explain market dynamics and the challenges of economic planning in agriculture.