coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
examples/infospace-with-history/output/evaluations/annual_consumption_of_metals.md Normal file
View File

@@ -0,0 +1,63 @@
---
entity_slug: annual_consumption_of_metals
evaluator: null
evaluated_at: '2026-02-23T00:33:23.768836'
overall_score: 4.2
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition clearly distinguishes between metals that are consumed/removed
from circulation versus those that remain in use, creating a precise economic
concept. It avoids circularity and identifies specific mechanisms (wear, damage,
industrial processes) that constitute consumption.
- name: source_grounding
value: 5.0
max_value: 5.0
rationale: Smith explicitly discusses the annual consumption of precious metals
in Book I, Chapter 11, analyzing how consumption must be balanced by new supply
from mines to maintain stable quantities. This is a core concept in his analysis
of precious metal economics, not an imposed modern interpretation.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: '"Consumption" is the correct domain placement as this entity specifically
measures the rate at which metals are used up or destroyed. This is fundamentally
about consumption patterns rather than production, trade, or monetary policy.'
- name: vsm_relevance
value: 3.0
max_value: 5.0
rationale: This entity maps reasonably well to S4 (intelligence/environmental adaptation)
as it represents environmental monitoring of resource depletion rates that inform
economic decisions. However, it could also be viewed as S1 operational data, making
the VSM placement somewhat ambiguous.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: This entity illuminates a crucial mechanism in precious metal economics
- the balance between consumption and production that determines long-term availability
and value. It explains structural dynamics rather than merely labeling a surface
phenomenon, though it's somewhat descriptive rather than deeply mechanistic.
---
# Evaluation: Annual Consumption Of Metals
## definition_precision — 4.0 / 5.0
The definition clearly distinguishes between metals that are consumed/removed from circulation versus those that remain in use, creating a precise economic concept. It avoids circularity and identifies specific mechanisms (wear, damage, industrial processes) that constitute consumption.
## source_grounding — 5.0 / 5.0
Smith explicitly discusses the annual consumption of precious metals in Book I, Chapter 11, analyzing how consumption must be balanced by new supply from mines to maintain stable quantities. This is a core concept in his analysis of precious metal economics, not an imposed modern interpretation.
## domain_placement — 5.0 / 5.0
"Consumption" is the correct domain placement as this entity specifically measures the rate at which metals are used up or destroyed. This is fundamentally about consumption patterns rather than production, trade, or monetary policy.
## vsm_relevance — 3.0 / 5.0
This entity maps reasonably well to S4 (intelligence/environmental adaptation) as it represents environmental monitoring of resource depletion rates that inform economic decisions. However, it could also be viewed as S1 operational data, making the VSM placement somewhat ambiguous.
## explanatory_value — 4.0 / 5.0
This entity illuminates a crucial mechanism in precious metal economics - the balance between consumption and production that determines long-term availability and value. It explains structural dynamics rather than merely labeling a surface phenomenon, though it's somewhat descriptive rather than deeply mechanistic.