Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_institution

null

2026-02-23T05:17:31.297712

1.4

name	value	max_value	rationale
definition_precision	1.0	5.0	There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether this captures a distinct concept or represents a vague umbrella term.

name	value	max_value	rationale
source_grounding	1.0	5.0	With no definition, context, or source chapter specified, there's no evidence this entity is grounded in Smith's actual text. The generic term "Economic System Institution" could refer to anything and provides no clear connection to specific passages or concepts from "The Wealth of Nations."

name	value	max_value	rationale
domain_placement	2.0	5.0	While "Economic System Institution" sounds like it belongs in an economic domain, the unspecified domain and lack of definition make it impossible to verify correct placement. The term is so broad it could encompass multiple conceptual categories without clear boundaries.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	The term "institution" suggests some structural/regulatory function that might map to VSM systems (potentially S2-S5), but without definition or context, it's too abstract and undefined to place meaningfully within the VSM framework. It could theoretically relate to any system depending on what type of institution is meant.

name	value	max_value	rationale
explanatory_value	1.0	5.0	This entity provides zero explanatory value in its current form, as it merely offers a generic label without any mechanism, relation, or substantive content. It names nothing specific and illuminates no particular economic phenomenon or structural relationship.

Evaluation: Economic System Institution

definition_precision — 1.0 / 5.0

There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether this captures a distinct concept or represents a vague umbrella term.

source_grounding — 1.0 / 5.0

With no definition, context, or source chapter specified, there's no evidence this entity is grounded in Smith's actual text. The generic term "Economic System Institution" could refer to anything and provides no clear connection to specific passages or concepts from "The Wealth of Nations."

domain_placement — 2.0 / 5.0

While "Economic System Institution" sounds like it belongs in an economic domain, the unspecified domain and lack of definition make it impossible to verify correct placement. The term is so broad it could encompass multiple conceptual categories without clear boundaries.

vsm_relevance — 2.0 / 5.0

The term "institution" suggests some structural/regulatory function that might map to VSM systems (potentially S2-S5), but without definition or context, it's too abstract and undefined to place meaningfully within the VSM framework. It could theoretically relate to any system depending on what type of institution is meant.

explanatory_value — 1.0 / 5.0

This entity provides zero explanatory value in its current form, as it merely offers a generic label without any mechanism, relation, or substantive content. It names nothing specific and illuminates no particular economic phenomenon or structural relationship.

3.4 KiB Raw Blame History