Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.6 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

public_mourning_effects

null

2026-02-23T06:13:38.731333

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is quite precise, clearly specifying the mechanism (temporary demand increase → price above natural price → higher profits) and the specific context (black cloth/mourning goods during national bereavement). It captures a distinct market phenomenon rather than a vague concept.

name	value	max_value	rationale
source_grounding	5.0	5.0	This is directly grounded in Smith's text from Book I, Chapter 10, where he explicitly uses public mourning as an example of how extraordinary demand temporarily raises prices above natural levels. The entity accurately reflects Smith's actual argument about demand variations and profit inequalities.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Exchange" domain placement is correct, as this phenomenon directly concerns market price formation, demand-supply dynamics, and the temporary deviation of market prices from natural prices. This is fundamentally about exchange mechanisms rather than production or distribution.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity describes a market response to external events but doesn't naturally map to specific VSM systems - it's more of a market phenomenon than an organizational function. While it might loosely relate to S4 (environmental response), the connection is forced rather than natural.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity illuminates an important economic mechanism about how external social events create temporary market distortions and profit opportunities. It demonstrates Smith's broader theoretical point about the relationship between demand variations and profit inequalities across different employments.

Evaluation: Public Mourning Effects

definition_precision — 4.0 / 5.0

The definition is quite precise, clearly specifying the mechanism (temporary demand increase → price above natural price → higher profits) and the specific context (black cloth/mourning goods during national bereavement). It captures a distinct market phenomenon rather than a vague concept.

source_grounding — 5.0 / 5.0

This is directly grounded in Smith's text from Book I, Chapter 10, where he explicitly uses public mourning as an example of how extraordinary demand temporarily raises prices above natural levels. The entity accurately reflects Smith's actual argument about demand variations and profit inequalities.

domain_placement — 5.0 / 5.0

The "Exchange" domain placement is correct, as this phenomenon directly concerns market price formation, demand-supply dynamics, and the temporary deviation of market prices from natural prices. This is fundamentally about exchange mechanisms rather than production or distribution.

vsm_relevance — 2.0 / 5.0

This entity describes a market response to external events but doesn't naturally map to specific VSM systems - it's more of a market phenomenon than an organizational function. While it might loosely relate to S4 (environmental response), the connection is forced rather than natural.

explanatory_value — 4.0 / 5.0

This entity illuminates an important economic mechanism about how external social events create temporary market distortions and profit opportunities. It demonstrates Smith's broader theoretical point about the relationship between demand variations and profit inequalities across different employments.

3.6 KiB Raw Blame History