Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.8 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_implementation_barrier

null

2026-02-23T05:16:38.185885

3.0

name	value	max_value	rationale
definition_precision	3.0	5.0	The definition captures a coherent concept about obstacles to economic policy implementation, but it's somewhat broad and umbrella-like, listing multiple types of barriers without sharp conceptual boundaries. While not circular, it could be more precise about what distinguishes implementation barriers from general economic challenges.

name	value	max_value	rationale
source_grounding	2.0	5.0	The entity claims to derive from Smith's "discussion of different systems and their application" but Book IV, Chapter 0 doesn't exist in The Wealth of Nations, and the context description is vague about specific textual evidence. This appears to be an interpretive overlay rather than a concept Smith explicitly develops.

name	value	max_value	rationale
domain_placement	4.0	5.0	The "Regulation" domain assignment is appropriate since implementation barriers are fundamentally about the gap between policy design and regulatory execution. This concept naturally fits within discussions of how economic arrangements are put into practice through institutional mechanisms.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to multiple VSM systems - S3 (internal regulation challenges), S4 (environmental adaptation obstacles), and S5 (policy implementation gaps). Implementation barriers are precisely the kind of systemic dysfunction that VSM is designed to identify and address.

name	value	max_value	rationale
explanatory_value	2.0	5.0	While the concept of implementation barriers is practically important, this entity primarily names a general phenomenon rather than illuminating specific mechanisms or structural relations. It lacks the analytical depth to explain why barriers arise or how they function within economic systems.

Evaluation: Economic System Implementation Barrier

definition_precision — 3.0 / 5.0

The definition captures a coherent concept about obstacles to economic policy implementation, but it's somewhat broad and umbrella-like, listing multiple types of barriers without sharp conceptual boundaries. While not circular, it could be more precise about what distinguishes implementation barriers from general economic challenges.

source_grounding — 2.0 / 5.0

The entity claims to derive from Smith's "discussion of different systems and their application" but Book IV, Chapter 0 doesn't exist in The Wealth of Nations, and the context description is vague about specific textual evidence. This appears to be an interpretive overlay rather than a concept Smith explicitly develops.

domain_placement — 4.0 / 5.0

The "Regulation" domain assignment is appropriate since implementation barriers are fundamentally about the gap between policy design and regulatory execution. This concept naturally fits within discussions of how economic arrangements are put into practice through institutional mechanisms.

vsm_relevance — 4.0 / 5.0

This entity maps well to multiple VSM systems - S3 (internal regulation challenges), S4 (environmental adaptation obstacles), and S5 (policy implementation gaps). Implementation barriers are precisely the kind of systemic dysfunction that VSM is designed to identify and address.

explanatory_value — 2.0 / 5.0

While the concept of implementation barriers is practically important, this entity primarily names a general phenomenon rather than illuminating specific mechanisms or structural relations. It lacks the analytical depth to explain why barriers arise or how they function within economic systems.

3.8 KiB Raw Blame History