Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

annual_consumption_of_goods

null

2026-02-23T00:33:14.144865

1.8

name	value	max_value	rationale
definition_precision	1.0	5.0	There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether the concept captures something distinct or is merely a vague umbrella term.

name	value	max_value	rationale
source_grounding	2.0	5.0	While "annual consumption of goods" uses terminology that would appear in The Wealth of Nations, the lack of source chapter specification and definition suggests this may not be grounded in Smith's actual conceptual framework. The entity appears to be a modern economic term imposed on the text rather than derived from it.

name	value	max_value	rationale
domain_placement	3.0	5.0	The concept of annual consumption would logically fit within economic domains related to demand, markets, or national accounting. However, without domain specification or definition, it's unclear whether this placement aligns with Smith's actual treatment of consumption patterns.

name	value	max_value	rationale
vsm_relevance	2.0	5.0	Annual consumption could potentially relate to S1 (operational flows) or S4 (environmental intelligence about demand patterns), but without proper definition it remains too abstract to meaningfully map to VSM systems. The entity lacks the structural specificity needed for VSM integration.

name	value	max_value	rationale
explanatory_value	1.0	5.0	With no definition, context, or source grounding, this entity provides zero explanatory value. It merely names a surface-level economic phenomenon without illuminating any mechanisms, relationships, or structural insights from Smith's work.

Evaluation: Annual Consumption Of Goods

definition_precision — 1.0 / 5.0

There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether the concept captures something distinct or is merely a vague umbrella term.

source_grounding — 2.0 / 5.0

While "annual consumption of goods" uses terminology that would appear in The Wealth of Nations, the lack of source chapter specification and definition suggests this may not be grounded in Smith's actual conceptual framework. The entity appears to be a modern economic term imposed on the text rather than derived from it.

domain_placement — 3.0 / 5.0

The concept of annual consumption would logically fit within economic domains related to demand, markets, or national accounting. However, without domain specification or definition, it's unclear whether this placement aligns with Smith's actual treatment of consumption patterns.

vsm_relevance — 2.0 / 5.0

Annual consumption could potentially relate to S1 (operational flows) or S4 (environmental intelligence about demand patterns), but without proper definition it remains too abstract to meaningfully map to VSM systems. The entity lacks the structural specificity needed for VSM integration.

explanatory_value — 1.0 / 5.0

With no definition, context, or source grounding, this entity provides zero explanatory value. It merely names a surface-level economic phenomenon without illuminating any mechanisms, relationships, or structural insights from Smith's work.

3.4 KiB Raw Blame History