Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.4 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

public_warehouse_system

null

2026-02-23T06:14:31.436924

4.4

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition clearly describes a specific institutional mechanism for customs collection with distinct operational features (government warehouses, deferred duty payment, anti-smuggling benefits). It avoids circularity and captures a concrete administrative system rather than a vague concept.

name	value	max_value	rationale
source_grounding	5.0	5.0	This entity is directly grounded in Smith's actual proposals in Book V, Chapter 2, where he explicitly discusses warehouse systems as a customs reform measure. The description accurately reflects Smith's arguments about efficiency gains and smuggling prevention.

name	value	max_value	rationale
domain_placement	5.0	5.0	"General Theory" is the appropriate domain placement as this represents Smith's theoretical proposal for institutional reform of tax collection systems. It fits naturally within his broader analysis of public finance and administrative efficiency.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to VSM System 3 (internal regulation/audit) as it represents an operational control mechanism for government revenue collection, with clear monitoring and regulatory functions. It also has some S1 characteristics as a primary operational system.

name	value	max_value	rationale
explanatory_value	4.0	5.0	The entity illuminates an important structural mechanism in Smith's thinking about how institutional design can solve coordination problems (smuggling, collection costs) in public finance. It demonstrates Smith's attention to implementation details rather than just abstract principles.

Evaluation: Public Warehouse System

definition_precision — 4.0 / 5.0

The definition clearly describes a specific institutional mechanism for customs collection with distinct operational features (government warehouses, deferred duty payment, anti-smuggling benefits). It avoids circularity and captures a concrete administrative system rather than a vague concept.

source_grounding — 5.0 / 5.0

This entity is directly grounded in Smith's actual proposals in Book V, Chapter 2, where he explicitly discusses warehouse systems as a customs reform measure. The description accurately reflects Smith's arguments about efficiency gains and smuggling prevention.

domain_placement — 5.0 / 5.0

"General Theory" is the appropriate domain placement as this represents Smith's theoretical proposal for institutional reform of tax collection systems. It fits naturally within his broader analysis of public finance and administrative efficiency.

vsm_relevance — 4.0 / 5.0

This entity maps well to VSM System 3 (internal regulation/audit) as it represents an operational control mechanism for government revenue collection, with clear monitoring and regulatory functions. It also has some S1 characteristics as a primary operational system.

explanatory_value — 4.0 / 5.0

The entity illuminates an important structural mechanism in Smith's thinking about how institutional design can solve coordination problems (smuggling, collection costs) in public finance. It demonstrates Smith's attention to implementation details rather than just abstract principles.

3.4 KiB Raw Blame History