Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.6 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

adulteration_of_metals

null

2026-02-23T00:19:45.942275

4.4

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is precise and captures a distinct fraudulent practice with clear mechanics (mixing cheaper materials with precious metals to deceive). It avoids circularity and specifies the key elements: deception, material composition, and difficulty of detection without assaying.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is well-grounded in Smith's actual discussion of currency problems in Book I, Chapter 4, where he explicitly addresses how unstamped metals create opportunities for fraud through adulteration. The concept directly reflects Smith's concerns about metal purity verification.

name	value	max_value	rationale
domain_placement	5.0	5.0	The "Regulation" domain assignment is correct, as adulteration of metals represents a market failure that Smith identifies as requiring institutional solutions (like official stamping/certification). This fits squarely within regulatory concerns about market integrity and fraud prevention.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps naturally to S3 (internal regulation/audit) as it represents a control problem requiring verification and quality assurance mechanisms. It also touches on S2 (coordination) since adulteration creates market oscillations and trust breakdowns that need systematic dampening.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity illuminates an important structural mechanism in Smith's monetary theory—how the absence of certification systems enables fraud and undermines market function. It explains a specific causal relationship between institutional gaps and market failures rather than merely naming a surface phenomenon.

Evaluation: Adulteration Of Metals

definition_precision — 4.0 / 5.0

The definition is precise and captures a distinct fraudulent practice with clear mechanics (mixing cheaper materials with precious metals to deceive). It avoids circularity and specifies the key elements: deception, material composition, and difficulty of detection without assaying.

source_grounding — 5.0 / 5.0

This entity is well-grounded in Smith's actual discussion of currency problems in Book I, Chapter 4, where he explicitly addresses how unstamped metals create opportunities for fraud through adulteration. The concept directly reflects Smith's concerns about metal purity verification.

domain_placement — 5.0 / 5.0

The "Regulation" domain assignment is correct, as adulteration of metals represents a market failure that Smith identifies as requiring institutional solutions (like official stamping/certification). This fits squarely within regulatory concerns about market integrity and fraud prevention.

vsm_relevance — 4.0 / 5.0

This entity maps naturally to S3 (internal regulation/audit) as it represents a control problem requiring verification and quality assurance mechanisms. It also touches on S2 (coordination) since adulteration creates market oscillations and trust breakdowns that need systematic dampening.

explanatory_value — 4.0 / 5.0

The entity illuminates an important structural mechanism in Smith's monetary theory—how the absence of certification systems enables fraud and undermines market function. It explains a specific causal relationship between institutional gaps and market failures rather than merely naming a surface phenomenon.

3.6 KiB Raw Blame History