coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions 1 Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/materials_and_subsistence.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: materials_and_subsistence
evaluator: null
evaluated_at: '2026-02-23T05:49:49.770479'
overall_score: 1.8
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making this entity completely
imprecise. Without any definitional content, it's impossible to determine what
specific concept this is meant to capture or whether it's distinct from other
related terms.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While "materials" and "subsistence" are terms that appear in The Wealth
of Nations, combining them into a single entity without definition or context
makes it unclear whether this reflects a specific concept Smith actually discusses.
The lack of source chapter specification further undermines confidence in its
grounding.
- name: domain_placement
value: 2.0
max_value: 5.0
rationale: The unspecified domain placement is problematic, though the terms suggest
this could relate to production factors or basic economic needs. Without a clear
definition, it's impossible to assess whether the domain assignment (or lack thereof)
is appropriate.
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: The concept could potentially map to S1 (primary operations) if it refers
to basic production inputs, but the lack of definition makes VSM placement speculative.
The terms suggest operational-level concerns but without clarity on the specific
relationship or mechanism involved.
- name: explanatory_value
value: 1.0
max_value: 5.0
rationale: With no definition, context, or clear conceptual boundaries, this entity
provides no explanatory power. It appears to be merely a label without illuminating
any particular economic mechanism or structural relationship that Smith describes.
---
# Evaluation: Materials And Subsistence
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to determine what specific concept this is meant to capture or whether it's distinct from other related terms.
## source_grounding — 2.0 / 5.0
While "materials" and "subsistence" are terms that appear in The Wealth of Nations, combining them into a single entity without definition or context makes it unclear whether this reflects a specific concept Smith actually discusses. The lack of source chapter specification further undermines confidence in its grounding.
## domain_placement — 2.0 / 5.0
The unspecified domain placement is problematic, though the terms suggest this could relate to production factors or basic economic needs. Without a clear definition, it's impossible to assess whether the domain assignment (or lack thereof) is appropriate.
## vsm_relevance — 3.0 / 5.0
The concept could potentially map to S1 (primary operations) if it refers to basic production inputs, but the lack of definition makes VSM placement speculative. The terms suggest operational-level concerns but without clarity on the specific relationship or mechanism involved.
## explanatory_value — 1.0 / 5.0
With no definition, context, or clear conceptual boundaries, this entity provides no explanatory power. It appears to be merely a label without illuminating any particular economic mechanism or structural relationship that Smith describes.