feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: agricultural_price_regulation
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T00:30:17.513975'
|
||||
overall_score: 4.2
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly distinguishes agricultural price regulation from
|
||||
other forms of market intervention by specifying the mechanisms (price floors,
|
||||
ceilings, stabilization schemes) and target sector. It avoids circularity and
|
||||
captures a distinct policy category with measurable characteristics.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is directly grounded in Smith's text, with specific references
|
||||
to historical examples he analyzes (Statute of Labourers, Assize of Bread and
|
||||
Ale) and his explicit critique of such interventions. The context accurately reflects
|
||||
Smith's position on these regulatory mechanisms.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: '"Regulation" is the appropriate domain for this entity, as it deals
|
||||
specifically with government intervention mechanisms rather than pure market phenomena
|
||||
or theoretical principles. The regulatory focus distinguishes it from broader
|
||||
agricultural or pricing concepts.'
|
||||
- name: vsm_relevance
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps primarily to S3 (internal regulation) as a control mechanism,
|
||||
but also touches S4 (policy intelligence) regarding market intervention decisions.
|
||||
While it has VSM relevance, the mapping is not as clear-cut as entities that represent
|
||||
pure operational or coordination functions.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The entity illuminates important structural relationships between government
|
||||
intervention and market efficiency that Smith analyzes extensively. It explains
|
||||
a concrete mechanism of economic control rather than merely naming a surface phenomenon,
|
||||
contributing to understanding of regulatory dynamics.
|
||||
---
|
||||
|
||||
# Evaluation: Agricultural Price Regulation
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition clearly distinguishes agricultural price regulation from other forms of market intervention by specifying the mechanisms (price floors, ceilings, stabilization schemes) and target sector. It avoids circularity and captures a distinct policy category with measurable characteristics.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This entity is directly grounded in Smith's text, with specific references to historical examples he analyzes (Statute of Labourers, Assize of Bread and Ale) and his explicit critique of such interventions. The context accurately reflects Smith's position on these regulatory mechanisms.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
"Regulation" is the appropriate domain for this entity, as it deals specifically with government intervention mechanisms rather than pure market phenomena or theoretical principles. The regulatory focus distinguishes it from broader agricultural or pricing concepts.
|
||||
|
||||
## vsm_relevance — 3.0 / 5.0
|
||||
|
||||
This entity maps primarily to S3 (internal regulation) as a control mechanism, but also touches S4 (policy intelligence) regarding market intervention decisions. While it has VSM relevance, the mapping is not as clear-cut as entities that represent pure operational or coordination functions.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
The entity illuminates important structural relationships between government intervention and market efficiency that Smith analyzes extensively. It explains a concrete mechanism of economic control rather than merely naming a surface phenomenon, contributing to understanding of regulatory dynamics.
|
||||
Reference in New Issue
Block a user