Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

treaty

null

2026-02-23T06:34:23.388943

3.4

name	value	max_value	rationale
definition_precision	3.0	5.0	The definition correctly identifies treaty as one of three mechanisms for exchange, but lacks precision about what distinguishes treaties from other formal agreements. The definition could be more specific about the formal, negotiated nature that differentiates treaties from simpler barter or purchase arrangements.

name	value	max_value	rationale
source_grounding	4.0	5.0	This entity is well-grounded in Smith's actual text from Book I, Chapter 2, where he explicitly lists "treaty, barter, and purchase" as the three means of obtaining mutual good offices. The concept directly reflects Smith's categorization rather than introducing external interpretations.

name	value	max_value	rationale
domain_placement	5.0	5.0	The placement in the "Exchange" domain is entirely appropriate, as treaties represent one of the fundamental mechanisms Smith identifies for economic exchange and obtaining mutual benefits in civilized society.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	Treaties as formal exchange mechanisms don't map clearly to any specific VSM system - they could potentially relate to S1 (operations), S2 (coordination), or S3 (regulation) depending on context. The concept is more about exchange methodology than organizational cybernetics.

name	value	max_value	rationale
explanatory_value	3.0	5.0	While the entity identifies an important mechanism in Smith's framework of exchange, it provides limited explanatory power beyond categorization. It names a phenomenon but doesn't deeply illuminate the structural relations or mechanisms that make treaties effective for mutual benefit.

Evaluation: Treaty

definition_precision — 3.0 / 5.0

The definition correctly identifies treaty as one of three mechanisms for exchange, but lacks precision about what distinguishes treaties from other formal agreements. The definition could be more specific about the formal, negotiated nature that differentiates treaties from simpler barter or purchase arrangements.

source_grounding — 4.0 / 5.0

This entity is well-grounded in Smith's actual text from Book I, Chapter 2, where he explicitly lists "treaty, barter, and purchase" as the three means of obtaining mutual good offices. The concept directly reflects Smith's categorization rather than introducing external interpretations.

domain_placement — 5.0 / 5.0

The placement in the "Exchange" domain is entirely appropriate, as treaties represent one of the fundamental mechanisms Smith identifies for economic exchange and obtaining mutual benefits in civilized society.

vsm_relevance — 2.0 / 5.0

Treaties as formal exchange mechanisms don't map clearly to any specific VSM system - they could potentially relate to S1 (operations), S2 (coordination), or S3 (regulation) depending on context. The concept is more about exchange methodology than organizational cybernetics.

explanatory_value — 3.0 / 5.0

While the entity identifies an important mechanism in Smith's framework of exchange, it provides limited explanatory power beyond categorization. It names a phenomenon but doesn't deeply illuminate the structural relations or mechanisms that make treaties effective for mutual benefit.

3.4 KiB Raw Blame History