feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: dwelling_house_distinction
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:08:22.502713'
|
||||
overall_score: 4.2
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly distinguishes between two uses of the same physical
|
||||
asset based on revenue generation, creating a precise economic categorization.
|
||||
The distinction between capital use (revenue-generating) and consumption use (non-revenue-generating)
|
||||
is well-defined and non-circular.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This concept is directly grounded in Smith's text from Book II, Chapter
|
||||
1, where he explicitly discusses how dwelling houses function differently as capital
|
||||
versus consumption goods. Smith uses this example to illustrate his broader theory
|
||||
of capital classification.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: The placement in "General Theory" is appropriate as this distinction
|
||||
represents a fundamental principle of how Smith categorizes capital versus consumption
|
||||
goods. It's a core theoretical concept that underpins his broader economic framework
|
||||
rather than belonging to a specific application area.
|
||||
- name: vsm_relevance
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: This entity has moderate VSM relevance, potentially mapping to S1 (as
|
||||
operational capital) or S4 (as intelligence about asset utilization). However,
|
||||
it's primarily a classificatory distinction rather than a dynamic system component,
|
||||
making it somewhat abstract for direct VSM placement.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity provides significant explanatory value by illuminating how
|
||||
the same physical asset can serve different economic functions depending on its
|
||||
use. It demonstrates Smith's sophisticated understanding of how economic categorization
|
||||
depends on function rather than just physical properties.
|
||||
---
|
||||
|
||||
# Evaluation: Dwelling House Distinction
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition clearly distinguishes between two uses of the same physical asset based on revenue generation, creating a precise economic categorization. The distinction between capital use (revenue-generating) and consumption use (non-revenue-generating) is well-defined and non-circular.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This concept is directly grounded in Smith's text from Book II, Chapter 1, where he explicitly discusses how dwelling houses function differently as capital versus consumption goods. Smith uses this example to illustrate his broader theory of capital classification.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
The placement in "General Theory" is appropriate as this distinction represents a fundamental principle of how Smith categorizes capital versus consumption goods. It's a core theoretical concept that underpins his broader economic framework rather than belonging to a specific application area.
|
||||
|
||||
## vsm_relevance — 3.0 / 5.0
|
||||
|
||||
This entity has moderate VSM relevance, potentially mapping to S1 (as operational capital) or S4 (as intelligence about asset utilization). However, it's primarily a classificatory distinction rather than a dynamic system component, making it somewhat abstract for direct VSM placement.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
This entity provides significant explanatory value by illuminating how the same physical asset can serve different economic functions depending on its use. It demonstrates Smith's sophisticated understanding of how economic categorization depends on function rather than just physical properties.
|
||||
Reference in New Issue
Block a user