feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,63 @@
|
||||
---
|
||||
entity_slug: merchant_capital_employment_choices
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:51:07.424320'
|
||||
overall_score: 1.8
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 1.0
|
||||
max_value: 5.0
|
||||
rationale: There is no definition provided at all, making this entity completely
|
||||
imprecise. Without any definitional content, it's impossible to assess whether
|
||||
the concept captures something distinct or is merely a vague umbrella term.
|
||||
- name: source_grounding
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: While Smith does discuss merchant capital and employment choices in "The
|
||||
Wealth of Nations," the lack of definition and context makes it unclear whether
|
||||
this entity accurately reflects Smith's specific analysis. The slug suggests relevance
|
||||
but provides no evidence of actual grounding in the source text.
|
||||
- name: domain_placement
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: The concept appears to relate to economic decision-making and capital
|
||||
allocation, which would be appropriate for an economic domain. However, without
|
||||
a definition or specified domain, it's difficult to assess whether the placement
|
||||
is optimal or if it belongs in a more specific conceptual category.
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: Merchant capital employment choices could potentially map to S1 (operational
|
||||
decisions) or S4 (strategic adaptation to market conditions), but the complete
|
||||
lack of definition makes VSM placement speculative. The entity is too undefined
|
||||
to establish clear VSM relevance.
|
||||
- name: explanatory_value
|
||||
value: 1.0
|
||||
max_value: 5.0
|
||||
rationale: With no definition, context, or elaboration, this entity provides zero
|
||||
explanatory power. It merely names a potential phenomenon without illuminating
|
||||
any mechanisms, relationships, or structural insights that would enhance understanding
|
||||
of Smith's economic theory.
|
||||
---
|
||||
|
||||
# Evaluation: Merchant Capital Employment Choices
|
||||
|
||||
## definition_precision — 1.0 / 5.0
|
||||
|
||||
There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether the concept captures something distinct or is merely a vague umbrella term.
|
||||
|
||||
## source_grounding — 2.0 / 5.0
|
||||
|
||||
While Smith does discuss merchant capital and employment choices in "The Wealth of Nations," the lack of definition and context makes it unclear whether this entity accurately reflects Smith's specific analysis. The slug suggests relevance but provides no evidence of actual grounding in the source text.
|
||||
|
||||
## domain_placement — 3.0 / 5.0
|
||||
|
||||
The concept appears to relate to economic decision-making and capital allocation, which would be appropriate for an economic domain. However, without a definition or specified domain, it's difficult to assess whether the placement is optimal or if it belongs in a more specific conceptual category.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
Merchant capital employment choices could potentially map to S1 (operational decisions) or S4 (strategic adaptation to market conditions), but the complete lack of definition makes VSM placement speculative. The entity is too undefined to establish clear VSM relevance.
|
||||
|
||||
## explanatory_value — 1.0 / 5.0
|
||||
|
||||
With no definition, context, or elaboration, this entity provides zero explanatory power. It merely names a potential phenomenon without illuminating any mechanisms, relationships, or structural insights that would enhance understanding of Smith's economic theory.
|
||||
Reference in New Issue
Block a user