coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
examples/infospace-with-history/output/evaluations/market_separation.md Normal file
View File

@@ -0,0 +1,60 @@
---
entity_slug: market_separation
evaluator: null
evaluated_at: '2026-02-23T05:48:14.418521'
overall_score: 1.0
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making it impossible to assess
precision or conceptual distinctness. Without any definitional content, this entity
fails to establish what "market separation" even means.
- name: source_grounding
value: 1.0
max_value: 5.0
rationale: With no definition, context, or source chapter specified, there is no
evidence this entity is grounded in Smith's actual text. The term "market separation"
is not a standard concept from The Wealth of Nations and appears to be an unsupported
abstraction.
- name: domain_placement
value: 1.0
max_value: 5.0
rationale: The domain is listed as "unspecified," providing no basis for evaluating
whether the conceptual categorization is appropriate. Without knowing what market
separation refers to, proper domain placement is impossible.
- name: vsm_relevance
value: 1.0
max_value: 5.0
rationale: Without any definition or context, it's impossible to determine whether
this entity maps to any VSM system or has organizational relevance. The complete
lack of content makes VSM placement assessment meaningless.
- name: explanatory_value
value: 1.0
max_value: 5.0
rationale: An entity with no definition, context, or source grounding provides zero
explanatory value. It neither illuminates mechanisms nor explains phenomena, functioning
merely as an empty label.
---
# Evaluation: Market Separation
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making it impossible to assess precision or conceptual distinctness. Without any definitional content, this entity fails to establish what "market separation" even means.
## source_grounding — 1.0 / 5.0
With no definition, context, or source chapter specified, there is no evidence this entity is grounded in Smith's actual text. The term "market separation" is not a standard concept from The Wealth of Nations and appears to be an unsupported abstraction.
## domain_placement — 1.0 / 5.0
The domain is listed as "unspecified," providing no basis for evaluating whether the conceptual categorization is appropriate. Without knowing what market separation refers to, proper domain placement is impossible.
## vsm_relevance — 1.0 / 5.0
Without any definition or context, it's impossible to determine whether this entity maps to any VSM system or has organizational relevance. The complete lack of content makes VSM placement assessment meaningless.
## explanatory_value — 1.0 / 5.0
An entity with no definition, context, or source grounding provides zero explanatory value. It neither illuminates mechanisms nor explains phenomena, functioning merely as an empty label.