Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_analysis

null

2026-02-23T05:12:49.904979

2.2

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and circular, essentially defining economic system analysis as "examining economic arrangements to understand how they function." It lacks specificity about what distinguishes this from general economic inquiry and reads more like a generic description than a precise conceptual definition.

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does analyze different economic systems in Book IV, the entity appears to abstract this into a meta-concept that Smith himself doesn't explicitly articulate. The source chapter reference "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, suggesting weak textual grounding.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the broad, methodological nature of the concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't incorrect but reflects the entity's lack of conceptual specificity.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	This entity is too abstract and meta-analytical to map meaningfully to any specific VSM system. It describes a general analytical approach rather than a functional component of an economic system, making it largely VSM-neutral but not in a useful way.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity adds little explanatory power beyond naming what economists generally do when studying systems. It doesn't illuminate specific mechanisms, structural relations, or provide insights that wouldn't be captured by more concrete analytical concepts Smith actually discusses.

Evaluation: Economic System Analysis

definition_precision — 2.0 / 5.0

The definition is overly broad and circular, essentially defining economic system analysis as "examining economic arrangements to understand how they function." It lacks specificity about what distinguishes this from general economic inquiry and reads more like a generic description than a precise conceptual definition.

source_grounding — 2.0 / 5.0

While Smith does analyze different economic systems in Book IV, the entity appears to abstract this into a meta-concept that Smith himself doesn't explicitly articulate. The source chapter reference "Book IV, Chapter 0" is problematic since Book IV doesn't have a Chapter 0, suggesting weak textual grounding.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the broad, methodological nature of the concept, though the entity is so abstract it could arguably fit in multiple domains. The placement isn't incorrect but reflects the entity's lack of conceptual specificity.

vsm_relevance — 2.0 / 5.0

This entity is too abstract and meta-analytical to map meaningfully to any specific VSM system. It describes a general analytical approach rather than a functional component of an economic system, making it largely VSM-neutral but not in a useful way.

explanatory_value — 2.0 / 5.0

The entity adds little explanatory power beyond naming what economists generally do when studying systems. It doesn't illuminate specific mechanisms, structural relations, or provide insights that wouldn't be captured by more concrete analytical concepts Smith actually discusses.

3.5 KiB Raw Blame History