feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: post_office_revenue
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T06:07:36.208090'
|
||||
overall_score: 4.0
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition is clear and specific, identifying post-office revenue
|
||||
as income from postal fees charged by the sovereign. It avoids circularity and
|
||||
captures a distinct financial concept with measurable characteristics.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is directly grounded in Smith's text, where he explicitly
|
||||
discusses the post-office as a successful mercantile project managed by governments.
|
||||
The definition accurately reflects Smith's analysis of postal operations as revenue-generating
|
||||
state enterprises.
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: '"General Theory" is appropriate as this concept relates to Smith''s
|
||||
broader theoretical framework about state revenue sources and public finance.
|
||||
It could potentially fit in a more specific domain like "Public Finance" but the
|
||||
current placement is defensible.'
|
||||
- name: vsm_relevance
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: This entity has moderate VSM relevance, primarily mapping to S1 (primary
|
||||
operations) as a specific revenue-generating activity of the state. However, it's
|
||||
more of a concrete operational outcome than a systemic function, making the VSM
|
||||
connection somewhat indirect.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The entity provides good explanatory value by illustrating Smith's argument
|
||||
about viable state enterprises that can generate revenue while serving public
|
||||
purposes. It demonstrates the mechanism by which governments can successfully
|
||||
engage in commercial activities without the typical inefficiencies Smith associates
|
||||
with state enterprise.
|
||||
---
|
||||
|
||||
# Evaluation: Post Office Revenue
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition is clear and specific, identifying post-office revenue as income from postal fees charged by the sovereign. It avoids circularity and captures a distinct financial concept with measurable characteristics.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This entity is directly grounded in Smith's text, where he explicitly discusses the post-office as a successful mercantile project managed by governments. The definition accurately reflects Smith's analysis of postal operations as revenue-generating state enterprises.
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
"General Theory" is appropriate as this concept relates to Smith's broader theoretical framework about state revenue sources and public finance. It could potentially fit in a more specific domain like "Public Finance" but the current placement is defensible.
|
||||
|
||||
## vsm_relevance — 3.0 / 5.0
|
||||
|
||||
This entity has moderate VSM relevance, primarily mapping to S1 (primary operations) as a specific revenue-generating activity of the state. However, it's more of a concrete operational outcome than a systemic function, making the VSM connection somewhat indirect.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
The entity provides good explanatory value by illustrating Smith's argument about viable state enterprises that can generate revenue while serving public purposes. It demonstrates the mechanism by which governments can successfully engage in commercial activities without the typical inefficiencies Smith associates with state enterprise.
|
||||
Reference in New Issue
Block a user