Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.5 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

economic_system_knowledge_transfer

null

2026-02-23T05:17:58.384706

2.6

name	value	max_value	rationale
definition_precision	2.0	5.0	The definition is overly broad and vague, essentially describing any form of knowledge sharing about economic matters. It lacks specificity about what constitutes "economic system knowledge" versus general economic information, and uses circular language like "understanding about economic arrangements."

name	value	max_value	rationale
source_grounding	2.0	5.0	While Smith does discuss different economic systems in Book IV, the entity extrapolates far beyond what Smith explicitly addresses about knowledge transfer mechanisms. Smith's focus is on critiquing mercantilism and advocating for natural liberty, not on theorizing knowledge dissemination processes.

name	value	max_value	rationale
domain_placement	3.0	5.0	"General Theory" is appropriate given the broad conceptual nature, but the entity is so abstract it could arguably fit in multiple domains. The placement is reasonable but not particularly illuminating about the entity's specific theoretical contribution.

name	value	max_value	rationale
vsm_relevance	4.0	5.0	This entity maps well to S4 (intelligence/environmental adaptation) as it concerns how economic systems learn from and adapt based on external information and experiences. It could also relate to S2 (coordination) in terms of harmonizing practices across contexts.

name	value	max_value	rationale
explanatory_value	2.0	5.0	The entity names a general phenomenon but provides little insight into specific mechanisms or structural relations that would enhance understanding of Smith's economic theory. It remains at a surface level without illuminating deeper economic principles or causal relationships.

Evaluation: Economic System Knowledge Transfer

definition_precision — 2.0 / 5.0

The definition is overly broad and vague, essentially describing any form of knowledge sharing about economic matters. It lacks specificity about what constitutes "economic system knowledge" versus general economic information, and uses circular language like "understanding about economic arrangements."

source_grounding — 2.0 / 5.0

While Smith does discuss different economic systems in Book IV, the entity extrapolates far beyond what Smith explicitly addresses about knowledge transfer mechanisms. Smith's focus is on critiquing mercantilism and advocating for natural liberty, not on theorizing knowledge dissemination processes.

domain_placement — 3.0 / 5.0

"General Theory" is appropriate given the broad conceptual nature, but the entity is so abstract it could arguably fit in multiple domains. The placement is reasonable but not particularly illuminating about the entity's specific theoretical contribution.

vsm_relevance — 4.0 / 5.0

This entity maps well to S4 (intelligence/environmental adaptation) as it concerns how economic systems learn from and adapt based on external information and experiences. It could also relate to S2 (coordination) in terms of harmonizing practices across contexts.

explanatory_value — 2.0 / 5.0

The entity names a general phenomenon but provides little insight into specific mechanisms or structural relations that would enhance understanding of Smith's economic theory. It remains at a surface level without illuminating deeper economic principles or causal relationships.

3.5 KiB Raw Blame History