Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

4.1 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

public_lottery

null

2026-02-23T06:13:29.706363

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly distinguishes public lottery as a government-sponsored gambling scheme and precisely captures Smith's analytical use of it as an illustration of systematic cognitive bias regarding risk assessment. The definition avoids circularity and identifies the specific behavioral economics insight Smith derives from this example.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's actual text from Book I, Chapter 10, where he explicitly uses public lotteries as an analytical tool to explain how people misjudge probabilities. The behavioral insight about overvaluing gains and undervaluing losses accurately reflects Smith's argument without introducing external concepts.

name	value	max_value	rationale
domain_placement	4.0	5.0	"General Theory" is appropriate as this concept illustrates Smith's broader theoretical insights about human psychology and market behavior rather than belonging to a specific economic sector. The lottery example serves as a foundational building block for understanding labor market dynamics and risk assessment across multiple domains.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity represents a cognitive/behavioral phenomenon rather than an organizational system or function, making it largely VSM-neutral. While it might tangentially relate to S4 (intelligence/environmental adaptation) in terms of how individuals process information about opportunities, it doesn't naturally map to VSM system categories.

name	value	max_value	rationale
explanatory_value	5.0	5.0	This entity provides significant explanatory power by illuminating the psychological mechanism underlying market inefficiencies and career choices, explaining why certain professions remain overcrowded despite poor average outcomes. It reveals a fundamental structural relationship between human psychology and economic behavior that Smith uses to explain broader market phenomena.

Evaluation: Public Lottery

definition_precision — 4.0 / 5.0

The definition clearly distinguishes public lottery as a government-sponsored gambling scheme and precisely captures Smith's analytical use of it as an illustration of systematic cognitive bias regarding risk assessment. The definition avoids circularity and identifies the specific behavioral economics insight Smith derives from this example.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's actual text from Book I, Chapter 10, where he explicitly uses public lotteries as an analytical tool to explain how people misjudge probabilities. The behavioral insight about overvaluing gains and undervaluing losses accurately reflects Smith's argument without introducing external concepts.

domain_placement — 4.0 / 5.0

"General Theory" is appropriate as this concept illustrates Smith's broader theoretical insights about human psychology and market behavior rather than belonging to a specific economic sector. The lottery example serves as a foundational building block for understanding labor market dynamics and risk assessment across multiple domains.

vsm_relevance — 2.0 / 5.0

This entity represents a cognitive/behavioral phenomenon rather than an organizational system or function, making it largely VSM-neutral. While it might tangentially relate to S4 (intelligence/environmental adaptation) in terms of how individuals process information about opportunities, it doesn't naturally map to VSM system categories.

explanatory_value — 5.0 / 5.0

This entity provides significant explanatory power by illuminating the psychological mechanism underlying market inefficiencies and career choices, explaining why certain professions remain overcrowded despite poor average outcomes. It reveals a fundamental structural relationship between human psychology and economic behavior that Smith uses to explain broader market phenomena.

4.1 KiB Raw Blame History