feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
---
|
||||
entity_slug: economic_system_purpose
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:20:10.382815'
|
||||
overall_score: 4.6
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly identifies two specific, measurable purposes (providing
|
||||
subsistence and generating public revenue) rather than using vague language. While
|
||||
it could be slightly more precise about the relationship between these dual purposes,
|
||||
it avoids circularity and captures a distinct conceptual framework.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity directly reflects Smith's explicit identification of political
|
||||
economy's dual purposes as stated in the source text. The concept is not an interpretation
|
||||
or extrapolation but represents Smith's own articulated framework for understanding
|
||||
economic system objectives.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: '"General Theory" is the appropriate domain placement since this represents
|
||||
Smith''s foundational theoretical framework for understanding what economic systems
|
||||
should accomplish. This is clearly a meta-theoretical concept rather than belonging
|
||||
to a specific economic mechanism or policy area.'
|
||||
- name: vsm_relevance
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps directly to VSM System 5 (identity/policy) as it defines
|
||||
the fundamental identity and purpose that should guide all economic system operations.
|
||||
It establishes the overarching policy framework that would govern how all other
|
||||
systems (S1-S4) should function.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity provides significant explanatory power by establishing the
|
||||
evaluative criteria for judging economic arrangements and policies throughout
|
||||
Smith's work. While it doesn't explain specific mechanisms, it illuminates the
|
||||
structural logic underlying Smith's entire analytical framework for political
|
||||
economy.
|
||||
---
|
||||
|
||||
# Evaluation: Economic System Purpose
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition clearly identifies two specific, measurable purposes (providing subsistence and generating public revenue) rather than using vague language. While it could be slightly more precise about the relationship between these dual purposes, it avoids circularity and captures a distinct conceptual framework.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This entity directly reflects Smith's explicit identification of political economy's dual purposes as stated in the source text. The concept is not an interpretation or extrapolation but represents Smith's own articulated framework for understanding economic system objectives.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
"General Theory" is the appropriate domain placement since this represents Smith's foundational theoretical framework for understanding what economic systems should accomplish. This is clearly a meta-theoretical concept rather than belonging to a specific economic mechanism or policy area.
|
||||
|
||||
## vsm_relevance — 5.0 / 5.0
|
||||
|
||||
This entity maps directly to VSM System 5 (identity/policy) as it defines the fundamental identity and purpose that should guide all economic system operations. It establishes the overarching policy framework that would govern how all other systems (S1-S4) should function.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
This entity provides significant explanatory power by establishing the evaluative criteria for judging economic arrangements and policies throughout Smith's work. While it doesn't explain specific mechanisms, it illuminates the structural logic underlying Smith's entire analytical framework for political economy.
|
||||
Reference in New Issue
Block a user