coulomb/markitect-main
2
0
Fork 0
Code Issues 133 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
examples/infospace-with-history/output/evaluations/manufacturing_process_subdivision.md Normal file
View File

@@ -0,0 +1,66 @@
---
entity_slug: manufacturing_process_subdivision
evaluator: null
evaluated_at: '2026-02-23T05:42:28.009436'
overall_score: 2.4
scores:
- name: definition_precision
value: 1.0
max_value: 5.0
rationale: There is no definition provided at all, making it impossible to assess
precision or distinctness. The entity name suggests it relates to breaking down
manufacturing processes, but without a definition, the concept remains completely
undefined.
- name: source_grounding
value: 2.0
max_value: 5.0
rationale: While Adam Smith does discuss the division of labor in manufacturing
(famously with the pin factory example), the specific term "manufacturing process
subdivision" is not clearly established as grounded in his actual text. Without
source chapter information or context, it's unclear if this represents Smith's
concepts or modern interpretation.
- name: domain_placement
value: 3.0
max_value: 5.0
rationale: The concept appears to belong in the economic domain related to production
and labor organization, which would be appropriate for "The Wealth of Nations."
However, without a specified domain or definition, this assessment is based purely
on the entity name.
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: Manufacturing process subdivision would naturally map to S1 (primary
operations) in the VSM, as it deals with how operational work is organized and
divided. This represents a clear operational-level concept that fits well within
the VSM framework.
- name: explanatory_value
value: 2.0
max_value: 5.0
rationale: While the subdivision of manufacturing processes is potentially important
for understanding productivity and efficiency mechanisms in Smith's work, the
complete lack of definition and context means this entity currently provides no
explanatory value. It names a potentially important concept but fails to illuminate
any mechanisms or relations.
---
# Evaluation: Manufacturing Process Subdivision
## definition_precision — 1.0 / 5.0
There is no definition provided at all, making it impossible to assess precision or distinctness. The entity name suggests it relates to breaking down manufacturing processes, but without a definition, the concept remains completely undefined.
## source_grounding — 2.0 / 5.0
While Adam Smith does discuss the division of labor in manufacturing (famously with the pin factory example), the specific term "manufacturing process subdivision" is not clearly established as grounded in his actual text. Without source chapter information or context, it's unclear if this represents Smith's concepts or modern interpretation.
## domain_placement — 3.0 / 5.0
The concept appears to belong in the economic domain related to production and labor organization, which would be appropriate for "The Wealth of Nations." However, without a specified domain or definition, this assessment is based purely on the entity name.
## vsm_relevance — 4.0 / 5.0
Manufacturing process subdivision would naturally map to S1 (primary operations) in the VSM, as it deals with how operational work is organized and divided. This represents a clear operational-level concept that fits well within the VSM framework.
## explanatory_value — 2.0 / 5.0
While the subdivision of manufacturing processes is potentially important for understanding productivity and efficiency mechanisms in Smith's work, the complete lack of definition and context means this entity currently provides no explanatory value. It names a potentially important concept but fails to illuminate any mechanisms or relations.