coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

62
examples/infospace-with-history/output/evaluations/artificer_planter_independence.md Normal file
View File

@@ -0,0 +1,62 @@
---
entity_slug: artificer_planter_independence
evaluator: null
evaluated_at: '2026-02-23T00:34:47.874089'
overall_score: 1.2
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making it impossible to assess
precision or distinctness. The entity name suggests a relationship between artificers,
planters, and independence, but without definition, it remains completely vague.
- name: source_grounding
value: 1.0
max_value: 5.0
rationale: With no definition, context, or source chapter specified, there's no
evidence this entity is grounded in Smith's actual text. The combination of "artificer,"
"planter," and "independence" could relate to colonial economic themes in Smith,
but this is pure speculation without textual anchoring.
- name: domain_placement
value: 1.0
max_value: 5.0
rationale: The domain is listed as unspecified, and without a definition or context,
it's impossible to determine what economic or thematic category this entity should
occupy. The terms suggest possible connections to manufacturing, agriculture,
or colonial economics, but placement cannot be assessed.
- name: vsm_relevance
value: 2.0
max_value: 5.0
rationale: The concept of "independence" might relate to autonomous operations (S1)
or identity/policy (S5) in VSM terms, but without knowing what specific independence
is being referenced or how it relates to artificers and planters, meaningful VSM
mapping is impossible.
- name: explanatory_value
value: 1.0
max_value: 5.0
rationale: An undefined entity with no context cannot provide any explanatory power
about economic mechanisms or structural relations. It currently functions as an
empty label rather than an illuminating concept.
---
# Evaluation: Artificer Planter Independence
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making it impossible to assess precision or distinctness. The entity name suggests a relationship between artificers, planters, and independence, but without definition, it remains completely vague.
## source_grounding — 1.0 / 5.0
With no definition, context, or source chapter specified, there's no evidence this entity is grounded in Smith's actual text. The combination of "artificer," "planter," and "independence" could relate to colonial economic themes in Smith, but this is pure speculation without textual anchoring.
## domain_placement — 1.0 / 5.0
The domain is listed as unspecified, and without a definition or context, it's impossible to determine what economic or thematic category this entity should occupy. The terms suggest possible connections to manufacturing, agriculture, or colonial economics, but placement cannot be assessed.
## vsm_relevance — 2.0 / 5.0
The concept of "independence" might relate to autonomous operations (S1) or identity/policy (S5) in VSM terms, but without knowing what specific independence is being referenced or how it relates to artificers and planters, meaningful VSM mapping is impossible.
## explanatory_value — 1.0 / 5.0
An undefined entity with no context cannot provide any explanatory power about economic mechanisms or structural relations. It currently functions as an empty label rather than an illuminating concept.