Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.6 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

merchant

null

2026-02-23T05:51:43.450582

4.2

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is clear and precise, identifying a merchant as someone who engages in buying and selling goods, with the important insight that everyone becomes a merchant to some degree in commercial society. The definition avoids circularity and captures a distinct economic role while acknowledging its universal nature under division of labor.

name	value	max_value	rationale
source_grounding	5.0	5.0	This concept is directly grounded in Smith's text from Book I, Chapter 4, where he explicitly discusses how division of labor necessitates exchange and makes every person a merchant in some measure. The entity accurately reflects Smith's actual argument about the fundamental role of commerce in society.

name	value	max_value	rationale
domain_placement	5.0	5.0	The placement in the "Exchange" domain is perfectly appropriate, as merchants are the primary agents facilitating exchange in Smith's commercial society. This is the natural conceptual home for entities involved in buying and selling activities.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	Merchants primarily operate in S1 (primary operations) as they execute the fundamental exchange activities of the economic system. However, they also have some S2 coordination functions in facilitating market transactions, making the VSM mapping somewhat distributed but still meaningful.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity provides significant explanatory value by illuminating how division of labor transforms social relations, making exchange and merchant activity universal rather than specialized. It reveals a key structural mechanism in Smith's theory of how commercial society functions at the individual level.

Evaluation: Merchant

definition_precision — 4.0 / 5.0

The definition is clear and precise, identifying a merchant as someone who engages in buying and selling goods, with the important insight that everyone becomes a merchant to some degree in commercial society. The definition avoids circularity and captures a distinct economic role while acknowledging its universal nature under division of labor.

source_grounding — 5.0 / 5.0

This concept is directly grounded in Smith's text from Book I, Chapter 4, where he explicitly discusses how division of labor necessitates exchange and makes every person a merchant in some measure. The entity accurately reflects Smith's actual argument about the fundamental role of commerce in society.

domain_placement — 5.0 / 5.0

The placement in the "Exchange" domain is perfectly appropriate, as merchants are the primary agents facilitating exchange in Smith's commercial society. This is the natural conceptual home for entities involved in buying and selling activities.

vsm_relevance — 3.0 / 5.0

Merchants primarily operate in S1 (primary operations) as they execute the fundamental exchange activities of the economic system. However, they also have some S2 coordination functions in facilitating market transactions, making the VSM mapping somewhat distributed but still meaningful.

explanatory_value — 4.0 / 5.0

This entity provides significant explanatory value by illuminating how division of labor transforms social relations, making exchange and merchant activity universal rather than specialized. It reveals a key structural mechanism in Smith's theory of how commercial society functions at the individual level.

3.6 KiB Raw Blame History