coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/economic_system_institution.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: economic_system_institution
evaluator: null
evaluated_at: '2026-02-23T05:17:31.297712'
overall_score: 1.4
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making this entity completely
imprecise. Without any definitional content, it's impossible to assess whether
this captures a distinct concept or represents a vague umbrella term.
- name: source_grounding
value: 1.0
max_value: 5.0
rationale: With no definition, context, or source chapter specified, there's no
evidence this entity is grounded in Smith's actual text. The generic term "Economic
System Institution" could refer to anything and provides no clear connection to
specific passages or concepts from "The Wealth of Nations."
- name: domain_placement
value: 2.0
max_value: 5.0
rationale: While "Economic System Institution" sounds like it belongs in an economic
domain, the unspecified domain and lack of definition make it impossible to verify
correct placement. The term is so broad it could encompass multiple conceptual
categories without clear boundaries.
- name: vsm_relevance
value: 2.0
max_value: 5.0
rationale: The term "institution" suggests some structural/regulatory function that
might map to VSM systems (potentially S2-S5), but without definition or context,
it's too abstract and undefined to place meaningfully within the VSM framework.
It could theoretically relate to any system depending on what type of institution
is meant.
- name: explanatory_value
value: 1.0
max_value: 5.0
rationale: This entity provides zero explanatory value in its current form, as it
merely offers a generic label without any mechanism, relation, or substantive
content. It names nothing specific and illuminates no particular economic phenomenon
or structural relationship.
---
# Evaluation: Economic System Institution
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making this entity completely imprecise. Without any definitional content, it's impossible to assess whether this captures a distinct concept or represents a vague umbrella term.
## source_grounding — 1.0 / 5.0
With no definition, context, or source chapter specified, there's no evidence this entity is grounded in Smith's actual text. The generic term "Economic System Institution" could refer to anything and provides no clear connection to specific passages or concepts from "The Wealth of Nations."
## domain_placement — 2.0 / 5.0
While "Economic System Institution" sounds like it belongs in an economic domain, the unspecified domain and lack of definition make it impossible to verify correct placement. The term is so broad it could encompass multiple conceptual categories without clear boundaries.
## vsm_relevance — 2.0 / 5.0
The term "institution" suggests some structural/regulatory function that might map to VSM systems (potentially S2-S5), but without definition or context, it's too abstract and undefined to place meaningfully within the VSM framework. It could theoretically relate to any system depending on what type of institution is meant.
## explanatory_value — 1.0 / 5.0
This entity provides zero explanatory value in its current form, as it merely offers a generic label without any mechanism, relation, or substantive content. It names nothing specific and illuminates no particular economic phenomenon or structural relationship.