Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.3 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

retailers

null

2026-02-23T06:18:00.744541

2.8

name	value	max_value	rationale
definition_precision	1.0	5.0	There is no definition provided at all, making this entity completely imprecise. Without any definition, it's impossible to determine what specific concept of "retailers" is being captured or whether it's distinct from related commercial entities.

name	value	max_value	rationale
source_grounding	3.0	5.0	While Smith does discuss retail trade and merchants who sell to consumers in "The Wealth of Nations," the lack of definition and context makes it unclear whether this entity accurately reflects Smith's specific treatment of retailers. The term itself is plausible but needs proper grounding in the source text.

name	value	max_value	rationale
domain_placement	4.0	5.0	Retailers would naturally belong in a commercial/trade domain within economic analysis, which seems appropriate for Smith's work. However, without a specified domain or definition, it's difficult to confirm the exact placement is optimal.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	Retailers would map well to S1 (primary operations) as they represent operational units that interface directly with the environment (consumers) and perform the essential function of distribution. This is a natural VSM placement for commercial entities.

name	value	max_value	rationale
explanatory_value	2.0	5.0	Without definition or context, this entity currently provides minimal explanatory value beyond being a label. To be valuable, it would need to illuminate Smith's insights about retail mechanisms, their role in market coordination, or their structural relationships within the economic system.

Evaluation: Retailers

definition_precision — 1.0 / 5.0

There is no definition provided at all, making this entity completely imprecise. Without any definition, it's impossible to determine what specific concept of "retailers" is being captured or whether it's distinct from related commercial entities.

source_grounding — 3.0 / 5.0

While Smith does discuss retail trade and merchants who sell to consumers in "The Wealth of Nations," the lack of definition and context makes it unclear whether this entity accurately reflects Smith's specific treatment of retailers. The term itself is plausible but needs proper grounding in the source text.

domain_placement — 4.0 / 5.0

Retailers would naturally belong in a commercial/trade domain within economic analysis, which seems appropriate for Smith's work. However, without a specified domain or definition, it's difficult to confirm the exact placement is optimal.

vsm_relevance — 4.0 / 5.0

Retailers would map well to S1 (primary operations) as they represent operational units that interface directly with the environment (consumers) and perform the essential function of distribution. This is a natural VSM placement for commercial entities.

explanatory_value — 2.0 / 5.0

Without definition or context, this entity currently provides minimal explanatory value beyond being a label. To be valuable, it would need to illuminate Smith's insights about retail mechanisms, their role in market coordination, or their structural relationships within the economic system.

3.3 KiB Raw Blame History