Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.3 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

mutual_good_offices

null

2026-02-23T05:56:25.014930

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly captures a distinct concept - the reciprocal benefits and services exchanged between individuals in economic transactions. It avoids circularity and specifies that these constitute the primary way people meet needs in civilized society.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's text from Book I, Chapter 2, where he explicitly discusses how mutual good offices are obtained through "treaty, barter, and purchase." The concept reflects Smith's actual language and reasoning about exchange mechanisms.

name	value	max_value	rationale
domain_placement	5.0	5.0	The placement in the "Exchange" domain is precisely correct, as mutual good offices represent the fundamental reciprocal nature of economic exchange. This is exactly where Smith discusses this concept in his analysis of how exchange works.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	This entity maps most naturally to S1 (primary operations) as it describes the basic operational exchanges that constitute economic activity. However, it's somewhat abstract and could also relate to S2 (coordination) since it involves reciprocal relationships between actors.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity provides genuine explanatory power by illuminating the reciprocal mechanism underlying exchange relationships - it explains why people engage in trade and how mutual benefit is achieved. It goes beyond merely naming a phenomenon to reveal the structural logic of exchange.

Evaluation: Mutual Good Offices

definition_precision — 4.0 / 5.0

The definition clearly captures a distinct concept - the reciprocal benefits and services exchanged between individuals in economic transactions. It avoids circularity and specifies that these constitute the primary way people meet needs in civilized society.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's text from Book I, Chapter 2, where he explicitly discusses how mutual good offices are obtained through "treaty, barter, and purchase." The concept reflects Smith's actual language and reasoning about exchange mechanisms.

domain_placement — 5.0 / 5.0

The placement in the "Exchange" domain is precisely correct, as mutual good offices represent the fundamental reciprocal nature of economic exchange. This is exactly where Smith discusses this concept in his analysis of how exchange works.

vsm_relevance — 3.0 / 5.0

This entity maps most naturally to S1 (primary operations) as it describes the basic operational exchanges that constitute economic activity. However, it's somewhat abstract and could also relate to S2 (coordination) since it involves reciprocal relationships between actors.

explanatory_value — 4.0 / 5.0

The entity provides genuine explanatory power by illuminating the reciprocal mechanism underlying exchange relationships - it explains why people engage in trade and how mutual benefit is achieved. It goes beyond merely naming a phenomenon to reveal the structural logic of exchange.

3.3 KiB Raw Blame History