feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,65 @@
|
||||
---
|
||||
entity_slug: economic_system_best_practices
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:13:25.628492'
|
||||
overall_score: 2.6
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: The definition is vague and umbrella-like, listing broad categories ("free
|
||||
trade, sound monetary policy, limited government intervention") without clearly
|
||||
delineating what constitutes a distinct concept. It reads more like a summary
|
||||
of policy recommendations rather than a precise conceptual definition.
|
||||
- name: source_grounding
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: While Smith does advocate for free trade and limited government intervention
|
||||
throughout his work, the framing as "best practices" with "proven effectiveness"
|
||||
imposes a modern analytical framework that doesn't reflect Smith's actual argumentative
|
||||
approach. Smith argues from first principles about natural liberty rather than
|
||||
from empirical "best practices."
|
||||
- name: domain_placement
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: '"General Theory" is appropriate since this entity attempts to synthesize
|
||||
Smith''s broader policy recommendations across multiple economic domains. The
|
||||
placement correctly recognizes this as a theoretical synthesis rather than a specific
|
||||
mechanism.'
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is too abstract and policy-oriented to map naturally to specific
|
||||
VSM systems. It spans multiple systems (S4 intelligence about what works, S5 policy
|
||||
identity) without clearly belonging to any particular one, making it VSM-neutral
|
||||
at best.
|
||||
- name: explanatory_value
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity merely aggregates Smith's policy positions without illuminating
|
||||
underlying mechanisms or structural relations. It names surface-level policy recommendations
|
||||
rather than explaining why these practices emerge from Smith's theoretical framework
|
||||
about natural liberty and market coordination.
|
||||
---
|
||||
|
||||
# Evaluation: Economic System Best Practices
|
||||
|
||||
## definition_precision — 2.0 / 5.0
|
||||
|
||||
The definition is vague and umbrella-like, listing broad categories ("free trade, sound monetary policy, limited government intervention") without clearly delineating what constitutes a distinct concept. It reads more like a summary of policy recommendations rather than a precise conceptual definition.
|
||||
|
||||
## source_grounding — 3.0 / 5.0
|
||||
|
||||
While Smith does advocate for free trade and limited government intervention throughout his work, the framing as "best practices" with "proven effectiveness" imposes a modern analytical framework that doesn't reflect Smith's actual argumentative approach. Smith argues from first principles about natural liberty rather than from empirical "best practices."
|
||||
|
||||
## domain_placement — 4.0 / 5.0
|
||||
|
||||
"General Theory" is appropriate since this entity attempts to synthesize Smith's broader policy recommendations across multiple economic domains. The placement correctly recognizes this as a theoretical synthesis rather than a specific mechanism.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
This entity is too abstract and policy-oriented to map naturally to specific VSM systems. It spans multiple systems (S4 intelligence about what works, S5 policy identity) without clearly belonging to any particular one, making it VSM-neutral at best.
|
||||
|
||||
## explanatory_value — 2.0 / 5.0
|
||||
|
||||
This entity merely aggregates Smith's policy positions without illuminating underlying mechanisms or structural relations. It names surface-level policy recommendations rather than explaining why these practices emerge from Smith's theoretical framework about natural liberty and market coordination.
|
||||
Reference in New Issue
Block a user