feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on first pass; 3 failed (network errors). eval-summary --update-metrics written with per_entity_mean=3.9556. Viability dashboard: 6/6 PASS redundancy_ratio 0.0061 (max 0.10) coverage_ratio 0.6190 (min 0.40) coherence_comps 0.0000 (max 3) consistency_cycles 0.0000 (max 0) granularity_entropy 2.6748 (min 1.0) per_entity_mean 3.9556 (min 3.5) Dimension breakdown (mean across 985 entities): definition_precision 3.62 source_grounding 4.36 domain_placement 4.56 vsm_relevance 3.31 explanatory_value 3.94 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
---
|
||||
entity_slug: mutual_good_offices
|
||||
evaluator: null
|
||||
evaluated_at: '2026-02-23T05:56:25.014930'
|
||||
overall_score: 4.2
|
||||
scores:
|
||||
- name: definition_precision
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The definition clearly captures a distinct concept - the reciprocal benefits
|
||||
and services exchanged between individuals in economic transactions. It avoids
|
||||
circularity and specifies that these constitute the primary way people meet needs
|
||||
in civilized society.
|
||||
- name: source_grounding
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: This entity is directly grounded in Smith's text from Book I, Chapter
|
||||
2, where he explicitly discusses how mutual good offices are obtained through
|
||||
"treaty, barter, and purchase." The concept reflects Smith's actual language and
|
||||
reasoning about exchange mechanisms.
|
||||
- name: domain_placement
|
||||
value: 5.0
|
||||
max_value: 5.0
|
||||
rationale: The placement in the "Exchange" domain is precisely correct, as mutual
|
||||
good offices represent the fundamental reciprocal nature of economic exchange.
|
||||
This is exactly where Smith discusses this concept in his analysis of how exchange
|
||||
works.
|
||||
- name: vsm_relevance
|
||||
value: 3.0
|
||||
max_value: 5.0
|
||||
rationale: This entity maps most naturally to S1 (primary operations) as it describes
|
||||
the basic operational exchanges that constitute economic activity. However, it's
|
||||
somewhat abstract and could also relate to S2 (coordination) since it involves
|
||||
reciprocal relationships between actors.
|
||||
- name: explanatory_value
|
||||
value: 4.0
|
||||
max_value: 5.0
|
||||
rationale: The entity provides genuine explanatory power by illuminating the reciprocal
|
||||
mechanism underlying exchange relationships - it explains why people engage in
|
||||
trade and how mutual benefit is achieved. It goes beyond merely naming a phenomenon
|
||||
to reveal the structural logic of exchange.
|
||||
---
|
||||
|
||||
# Evaluation: Mutual Good Offices
|
||||
|
||||
## definition_precision — 4.0 / 5.0
|
||||
|
||||
The definition clearly captures a distinct concept - the reciprocal benefits and services exchanged between individuals in economic transactions. It avoids circularity and specifies that these constitute the primary way people meet needs in civilized society.
|
||||
|
||||
## source_grounding — 5.0 / 5.0
|
||||
|
||||
This entity is directly grounded in Smith's text from Book I, Chapter 2, where he explicitly discusses how mutual good offices are obtained through "treaty, barter, and purchase." The concept reflects Smith's actual language and reasoning about exchange mechanisms.
|
||||
|
||||
## domain_placement — 5.0 / 5.0
|
||||
|
||||
The placement in the "Exchange" domain is precisely correct, as mutual good offices represent the fundamental reciprocal nature of economic exchange. This is exactly where Smith discusses this concept in his analysis of how exchange works.
|
||||
|
||||
## vsm_relevance — 3.0 / 5.0
|
||||
|
||||
This entity maps most naturally to S1 (primary operations) as it describes the basic operational exchanges that constitute economic activity. However, it's somewhat abstract and could also relate to S2 (coordination) since it involves reciprocal relationships between actors.
|
||||
|
||||
## explanatory_value — 4.0 / 5.0
|
||||
|
||||
The entity provides genuine explanatory power by illuminating the reciprocal mechanism underlying exchange relationships - it explains why people engage in trade and how mutual benefit is achieved. It goes beyond merely naming a phenomenon to reveal the structural logic of exchange.
|
||||
Reference in New Issue
Block a user