Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

2.9 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

bank_financial_system_integration

null

2026-02-23T00:42:07.875554

1.0

name	value	max_value	rationale
definition_precision	1.0	5.0	There is no definition provided whatsoever, making it impossible to assess what this entity actually represents. Without any definitional content, this fails completely on precision and distinctness.

name	value	max_value	rationale
source_grounding	1.0	5.0	With no source chapter specified and no definition or context provided, there is no evidence this entity is grounded in Smith's actual text. The term "Bank Financial System Integration" uses modern financial terminology that would be anachronistic for Smith's era.

name	value	max_value	rationale
domain_placement	1.0	5.0	The domain is listed as "unspecified," providing no basis for evaluating whether the conceptual categorization is appropriate. Without knowing what this entity is supposed to represent, domain placement cannot be assessed.

name	value	max_value	rationale
vsm_relevance	1.0	5.0	Without any definition or context, it's impossible to determine which VSM system this might relate to or whether it has any VSM relevance at all. The entity provides no information to enable VSM mapping.

name	value	max_value	rationale
explanatory_value	1.0	5.0	An entity with no definition, context, or source grounding cannot provide any explanatory power about economic mechanisms or structural relations. It adds no value to understanding Smith's work or economic theory.

Evaluation: Bank Financial System Integration

definition_precision — 1.0 / 5.0

There is no definition provided whatsoever, making it impossible to assess what this entity actually represents. Without any definitional content, this fails completely on precision and distinctness.

source_grounding — 1.0 / 5.0

With no source chapter specified and no definition or context provided, there is no evidence this entity is grounded in Smith's actual text. The term "Bank Financial System Integration" uses modern financial terminology that would be anachronistic for Smith's era.

domain_placement — 1.0 / 5.0

The domain is listed as "unspecified," providing no basis for evaluating whether the conceptual categorization is appropriate. Without knowing what this entity is supposed to represent, domain placement cannot be assessed.

vsm_relevance — 1.0 / 5.0

Without any definition or context, it's impossible to determine which VSM system this might relate to or whether it has any VSM relevance at all. The entity provides no information to enable VSM mapping.

explanatory_value — 1.0 / 5.0

An entity with no definition, context, or source grounding cannot provide any explanatory power about economic mechanisms or structural relations. It adds no value to understanding Smith's work or economic theory.

2.9 KiB Raw Blame History