coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/crown_lands_revenue.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: crown_lands_revenue
evaluator: null
evaluated_at: '2026-02-23T05:04:18.835824'
overall_score: 4.2
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly specifies crown lands revenue as income from state-owned
lands through rent and produce, distinguishing it from other revenue sources.
It's precise and non-circular, though could be slightly more specific about the
mechanisms of revenue generation.
- name: source_grounding
value: 5.0
max_value: 5.0
rationale: This entity is well-grounded in Smith's actual text, as Book V extensively
discusses various sources of public revenue including crown lands. Smith specifically
analyzes how different nations have relied on crown lands as a principal revenue
source and examines their management.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: The placement in "General Theory" domain is appropriate as this represents
a fundamental category of public finance that Smith analyzes systematically. It
fits naturally within his broader theoretical framework of public revenue sources
rather than being a specific policy or operational detail.
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: This entity has moderate VSM relevance, primarily mapping to S1 (primary
operations) as a basic revenue-generating function of the state. It could also
relate to S3 (internal regulation) regarding land management, but it's not as
clearly systemic as other VSM-relevant concepts.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: The entity provides good explanatory value by identifying a specific
mechanism of public finance and its historical importance. It illuminates how
states can generate revenue through direct ownership rather than taxation, contributing
to understanding of fiscal structure and state capacity.
---
# Evaluation: Crown Lands Revenue
## definition_precision — 4.0 / 5.0
The definition clearly specifies crown lands revenue as income from state-owned lands through rent and produce, distinguishing it from other revenue sources. It's precise and non-circular, though could be slightly more specific about the mechanisms of revenue generation.
## source_grounding — 5.0 / 5.0
This entity is well-grounded in Smith's actual text, as Book V extensively discusses various sources of public revenue including crown lands. Smith specifically analyzes how different nations have relied on crown lands as a principal revenue source and examines their management.
## domain_placement — 5.0 / 5.0
The placement in "General Theory" domain is appropriate as this represents a fundamental category of public finance that Smith analyzes systematically. It fits naturally within his broader theoretical framework of public revenue sources rather than being a specific policy or operational detail.
## vsm_relevance — 3.0 / 5.0
This entity has moderate VSM relevance, primarily mapping to S1 (primary operations) as a basic revenue-generating function of the state. It could also relate to S3 (internal regulation) regarding land management, but it's not as clearly systemic as other VSM-relevant concepts.
## explanatory_value — 4.0 / 5.0
The entity provides good explanatory value by identifying a specific mechanism of public finance and its historical importance. It illuminates how states can generate revenue through direct ownership rather than taxation, contributing to understanding of fiscal structure and state capacity.