feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
---
|
||||
entity_slug: economic_system_effectiveness
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:15:03.351419'
|
||||
overall_score: 3.2
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: The definition captures a meaningful concept about measuring economic
|
||||
system performance against objectives, but it's somewhat broad and could benefit
|
||||
from more specific criteria. The phrase "intended objectives" is vague and could
|
||||
apply to almost any economic arrangement.
|
||||
- name: source_grounding
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: While Smith does discuss the objectives of political economy (enriching
|
||||
people and sovereign), the specific concept of "economic system effectiveness"
|
||||
as a formal evaluative framework is more of an analytical overlay than something
|
||||
Smith explicitly develops. The source chapter reference "Book IV, Chapter 0" appears
|
||||
incorrect as Book IV doesn't have a Chapter 0.
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: '"General Theory" is appropriate for this meta-analytical concept that
|
||||
cuts across different economic systems and approaches. This entity represents
|
||||
an evaluative framework rather than belonging to a specific economic doctrine
|
||||
or mechanism.'
|
||||
- name: vsm_relevance
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps well to S3 (internal regulation/audit) as it represents
|
||||
a performance measurement and evaluation function that assesses how well the economic
|
||||
system is achieving its stated objectives. It could also relate to S5 (identity/policy)
|
||||
in terms of defining success criteria.
|
||||
- name: explanatory_value
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: The entity provides a useful analytical lens for comparing different
|
||||
economic approaches Smith discusses, but it doesn't reveal specific mechanisms
|
||||
or structural relations. It's more of an evaluative meta-concept than an explanatory
|
||||
mechanism that illuminates how economic systems actually function.
|
||||
---
|
||||
|
||||
# Evaluation: Economic System Effectiveness
|
||||
|
||||
## definition_precision — 3.0 / 5.0
|
||||
|
||||
The definition captures a meaningful concept about measuring economic system performance against objectives, but it's somewhat broad and could benefit from more specific criteria. The phrase "intended objectives" is vague and could apply to almost any economic arrangement.
|
||||
|
||||
## source_grounding — 2.0 / 5.0
|
||||
|
||||
While Smith does discuss the objectives of political economy (enriching people and sovereign), the specific concept of "economic system effectiveness" as a formal evaluative framework is more of an analytical overlay than something Smith explicitly develops. The source chapter reference "Book IV, Chapter 0" appears incorrect as Book IV doesn't have a Chapter 0.
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
"General Theory" is appropriate for this meta-analytical concept that cuts across different economic systems and approaches. This entity represents an evaluative framework rather than belonging to a specific economic doctrine or mechanism.
|
||||
|
||||
## vsm_relevance — 4.0 / 5.0
|
||||
|
||||
This entity maps well to S3 (internal regulation/audit) as it represents a performance measurement and evaluation function that assesses how well the economic system is achieving its stated objectives. It could also relate to S5 (identity/policy) in terms of defining success criteria.
|
||||
|
||||
## explanatory_value — 3.0 / 5.0
|
||||
|
||||
The entity provides a useful analytical lens for comparing different economic approaches Smith discusses, but it doesn't reveal specific mechanisms or structural relations. It's more of an evaluative meta-concept than an explanatory mechanism that illuminates how economic systems actually function.
|
||||
Reference in New Issue
Block a user