coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
examples/infospace-with-history/output/evaluations/agricultural_price_discovery.md Normal file
View File

@@ -0,0 +1,65 @@
---
entity_slug: agricultural_price_discovery
evaluator: null
evaluated_at: '2026-02-23T00:29:25.484525'
overall_score: 4.2
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly describes a specific market mechanism - how prices
are established through buyer-seller interactions that incorporate supply, demand,
and cost information. It avoids circularity and distinguishes this process from
general price-setting by emphasizing the information aggregation function.
- name: source_grounding
value: 4.0
max_value: 5.0
rationale: Smith extensively discusses agricultural markets and price formation
in Book I, Chapter 11, including how natural market processes establish prices
and how government interference disrupts these mechanisms. The concept of price
discovery as an information-processing function aligns well with Smith's analysis
of market coordination.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: '"Exchange" is the correct domain placement since price discovery is
fundamentally about the exchange process itself - how trading interactions generate
price information. This is distinct from production or consumption domains.'
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: This entity maps well to S2 (coordination/anti-oscillation) as price
discovery serves as a coordination mechanism that prevents market oscillations
by efficiently aggregating and transmitting information. It also has some S4 (intelligence)
aspects in how it processes environmental information about supply and demand
conditions.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The entity illuminates a crucial market mechanism - how decentralized
trading generates the price signals that coordinate economic activity. It explains
the structural process by which individual transactions aggregate into market-wide
information, which is fundamental to understanding market efficiency.
---
# Evaluation: Agricultural Price Discovery
## definition_precision — 4.0 / 5.0
The definition clearly describes a specific market mechanism - how prices are established through buyer-seller interactions that incorporate supply, demand, and cost information. It avoids circularity and distinguishes this process from general price-setting by emphasizing the information aggregation function.
## source_grounding — 4.0 / 5.0
Smith extensively discusses agricultural markets and price formation in Book I, Chapter 11, including how natural market processes establish prices and how government interference disrupts these mechanisms. The concept of price discovery as an information-processing function aligns well with Smith's analysis of market coordination.
## domain_placement — 5.0 / 5.0
"Exchange" is the correct domain placement since price discovery is fundamentally about the exchange process itself - how trading interactions generate price information. This is distinct from production or consumption domains.
## vsm_relevance — 4.0 / 5.0
This entity maps well to S2 (coordination/anti-oscillation) as price discovery serves as a coordination mechanism that prevents market oscillations by efficiently aggregating and transmitting information. It also has some S4 (intelligence) aspects in how it processes environmental information about supply and demand conditions.
## explanatory_value — 4.0 / 5.0
The entity illuminates a crucial market mechanism - how decentralized trading generates the price signals that coordinate economic activity. It explains the structural process by which individual transactions aggregate into market-wide information, which is fundamental to understanding market efficiency.