Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

materials_and_subsistence

null

2026-02-23T05:49:49.770479

1.8

name	value	max_value	rationale
definition_precision	1.0	5.0	There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to determine what specific concept this is meant to capture or whether it's distinct from other related terms.

name	value	max_value	rationale
source_grounding	2.0	5.0	While "materials" and "subsistence" are terms that appear in The Wealth of Nations, combining them into a single entity without definition or context makes it unclear whether this reflects a specific concept Smith actually discusses. The lack of source chapter specification further undermines confidence in its grounding.

name	value	max_value	rationale
domain_placement	2.0	5.0	The unspecified domain placement is problematic, though the terms suggest this could relate to production factors or basic economic needs. Without a clear definition, it's impossible to assess whether the domain assignment (or lack thereof) is appropriate.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	The concept could potentially map to S1 (primary operations) if it refers to basic production inputs, but the lack of definition makes VSM placement speculative. The terms suggest operational-level concerns but without clarity on the specific relationship or mechanism involved.

name	value	max_value	rationale
explanatory_value	1.0	5.0	With no definition, context, or clear conceptual boundaries, this entity provides no explanatory power. It appears to be merely a label without illuminating any particular economic mechanism or structural relationship that Smith describes.

Evaluation: Materials And Subsistence

definition_precision — 1.0 / 5.0

There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to determine what specific concept this is meant to capture or whether it's distinct from other related terms.

source_grounding — 2.0 / 5.0

While "materials" and "subsistence" are terms that appear in The Wealth of Nations, combining them into a single entity without definition or context makes it unclear whether this reflects a specific concept Smith actually discusses. The lack of source chapter specification further undermines confidence in its grounding.

domain_placement — 2.0 / 5.0

The unspecified domain placement is problematic, though the terms suggest this could relate to production factors or basic economic needs. Without a clear definition, it's impossible to assess whether the domain assignment (or lack thereof) is appropriate.

vsm_relevance — 3.0 / 5.0

The concept could potentially map to S1 (primary operations) if it refers to basic production inputs, but the lack of definition makes VSM placement speculative. The terms suggest operational-level concerns but without clarity on the specific relationship or mechanism involved.

explanatory_value — 1.0 / 5.0

With no definition, context, or clear conceptual boundaries, this entity provides no explanatory power. It appears to be merely a label without illuminating any particular economic mechanism or structural relationship that Smith describes.

3.4 KiB Raw Blame History