feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: public_mourning_effects
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T06:13:38.731333'
|
||||
overall_score: 4.0
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: "The definition is quite precise, clearly specifying the mechanism (temporary\
|
||||
\ demand increase \u2192 price above natural price \u2192 higher profits) and\
|
||||
\ the specific context (black cloth/mourning goods during national bereavement).\
|
||||
\ It captures a distinct market phenomenon rather than a vague concept."
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This is directly grounded in Smith's text from Book I, Chapter 10, where
|
||||
he explicitly uses public mourning as an example of how extraordinary demand temporarily
|
||||
raises prices above natural levels. The entity accurately reflects Smith's actual
|
||||
argument about demand variations and profit inequalities.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: The "Exchange" domain placement is correct, as this phenomenon directly
|
||||
concerns market price formation, demand-supply dynamics, and the temporary deviation
|
||||
of market prices from natural prices. This is fundamentally about exchange mechanisms
|
||||
rather than production or distribution.
|
||||
- name: vsm_relevance
|
||||
value: 2.0
|
||||
max_value: 5.0
|
||||
rationale: This entity describes a market response to external events but doesn't
|
||||
naturally map to specific VSM systems - it's more of a market phenomenon than
|
||||
an organizational function. While it might loosely relate to S4 (environmental
|
||||
response), the connection is forced rather than natural.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: This entity illuminates an important economic mechanism about how external
|
||||
social events create temporary market distortions and profit opportunities. It
|
||||
demonstrates Smith's broader theoretical point about the relationship between
|
||||
demand variations and profit inequalities across different employments.
|
||||
---
|
||||
|
||||
# Evaluation: Public Mourning Effects
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition is quite precise, clearly specifying the mechanism (temporary demand increase → price above natural price → higher profits) and the specific context (black cloth/mourning goods during national bereavement). It captures a distinct market phenomenon rather than a vague concept.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This is directly grounded in Smith's text from Book I, Chapter 10, where he explicitly uses public mourning as an example of how extraordinary demand temporarily raises prices above natural levels. The entity accurately reflects Smith's actual argument about demand variations and profit inequalities.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
The "Exchange" domain placement is correct, as this phenomenon directly concerns market price formation, demand-supply dynamics, and the temporary deviation of market prices from natural prices. This is fundamentally about exchange mechanisms rather than production or distribution.
|
||||
|
||||
## vsm_relevance — 2.0 / 5.0
|
||||
|
||||
This entity describes a market response to external events but doesn't naturally map to specific VSM systems - it's more of a market phenomenon than an organizational function. While it might loosely relate to S4 (environmental response), the connection is forced rather than natural.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
This entity illuminates an important economic mechanism about how external social events create temporary market distortions and profit opportunities. It demonstrates Smith's broader theoretical point about the relationship between demand variations and profit inequalities across different employments.
|
||||
Reference in New Issue
Block a user