coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
examples/infospace-with-history/output/evaluations/exportation_bounty.md Normal file
View File

@@ -0,0 +1,65 @@
---
entity_slug: exportation_bounty
evaluator: null
evaluated_at: '2026-02-23T05:26:04.039244'
overall_score: 4.2
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly identifies exportation bounty as a specific government
subsidy mechanism for grain exports with distinct characteristics (supporting
prices, encouraging production through export profitability). The definition is
precise and non-circular, though it could be slightly more specific about the
payment structure.
- name: source_grounding
value: 5.0
max_value: 5.0
rationale: This entity is directly grounded in Smith's detailed analysis in Book
I, Chapter 11, where he extensively examines the bounty on corn exportation and
its economic effects. Smith provides substantial discussion of this specific policy
mechanism and its consequences.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: The "Regulation" domain assignment is entirely appropriate, as exportation
bounties represent a clear form of government market intervention through regulatory
policy. This fits perfectly within the broader category of economic regulation
that Smith analyzes.
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: This entity maps reasonably well to S3 (internal regulation) as a government
control mechanism, and potentially to S4 (intelligence/adaptation) as a policy
response to market conditions. However, it's primarily a specific policy tool
rather than a fundamental system component.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The entity provides strong explanatory value by illuminating a specific
mechanism of government market intervention and its unintended consequences (raising
domestic prices, potentially discouraging production). It reveals important structural
relationships between policy tools and market outcomes that Smith analyzes.
---
# Evaluation: Exportation Bounty
## definition_precision — 4.0 / 5.0
The definition clearly identifies exportation bounty as a specific government subsidy mechanism for grain exports with distinct characteristics (supporting prices, encouraging production through export profitability). The definition is precise and non-circular, though it could be slightly more specific about the payment structure.
## source_grounding — 5.0 / 5.0
This entity is directly grounded in Smith's detailed analysis in Book I, Chapter 11, where he extensively examines the bounty on corn exportation and its economic effects. Smith provides substantial discussion of this specific policy mechanism and its consequences.
## domain_placement — 5.0 / 5.0
The "Regulation" domain assignment is entirely appropriate, as exportation bounties represent a clear form of government market intervention through regulatory policy. This fits perfectly within the broader category of economic regulation that Smith analyzes.
## vsm_relevance — 3.0 / 5.0
This entity maps reasonably well to S3 (internal regulation) as a government control mechanism, and potentially to S4 (intelligence/adaptation) as a policy response to market conditions. However, it's primarily a specific policy tool rather than a fundamental system component.
## explanatory_value — 4.0 / 5.0
The entity provides strong explanatory value by illuminating a specific mechanism of government market intervention and its unintended consequences (raising domestic prices, potentially discouraging production). It reveals important structural relationships between policy tools and market outcomes that Smith analyzes.