feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions

View File

@@ -0,0 +1,64 @@
---
entity_slug: annual_consumption_of_goods
evaluator: null
evaluated_at: '2026-02-23T00:33:14.144865'
overall_score: 1.8
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making this entity completely
imprecise. Without any definitional content, it's impossible to assess whether
the concept captures something distinct or is merely a vague umbrella term.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While "annual consumption of goods" uses terminology that would appear
in The Wealth of Nations, the lack of source chapter specification and definition
suggests this may not be grounded in Smith's actual conceptual framework. The
entity appears to be a modern economic term imposed on the text rather than derived
from it.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: The concept of annual consumption would logically fit within economic
domains related to demand, markets, or national accounting. However, without domain
specification or definition, it's unclear whether this placement aligns with Smith's
actual treatment of consumption patterns.
- name: vsm_relevance
value: 2.0
max_value: 5.0
rationale: Annual consumption could potentially relate to S1 (operational flows)
or S4 (environmental intelligence about demand patterns), but without proper definition
it remains too abstract to meaningfully map to VSM systems. The entity lacks the
structural specificity needed for VSM integration.
- name: explanatory_value
value: 1.0
max_value: 5.0
rationale: With no definition, context, or source grounding, this entity provides
zero explanatory value. It merely names a surface-level economic phenomenon without
illuminating any mechanisms, relationships, or structural insights from Smith's
work.
---
# Evaluation: Annual Consumption Of Goods
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether the concept captures something distinct or is merely a vague umbrella term.
## source_grounding — 2.0 / 5.0
While "annual consumption of goods" uses terminology that would appear in The Wealth of Nations, the lack of source chapter specification and definition suggests this may not be grounded in Smith's actual conceptual framework. The entity appears to be a modern economic term imposed on the text rather than derived from it.
## domain_placement — 3.0 / 5.0
The concept of annual consumption would logically fit within economic domains related to demand, markets, or national accounting. However, without domain specification or definition, it's unclear whether this placement aligns with Smith's actual treatment of consumption patterns.
## vsm_relevance — 2.0 / 5.0
Annual consumption could potentially relate to S1 (operational flows) or S4 (environmental intelligence about demand patterns), but without proper definition it remains too abstract to meaningfully map to VSM systems. The entity lacks the structural specificity needed for VSM integration.
## explanatory_value — 1.0 / 5.0
With no definition, context, or source grounding, this entity provides zero explanatory value. It merely names a surface-level economic phenomenon without illuminating any mechanisms, relationships, or structural insights from Smith's work.