coulomb/markitect-main
Archived
2
0
Fork 0
Code Issues 135 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Browse Source 
Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-23 09:36:46 +01:00
parent 81a4c8796a
commit a9ca0adfcf
986 changed files with 63216 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
examples/infospace-with-history/output/evaluations/annual_plate_addition_estimation.md Normal file
View File

@@ -0,0 +1,64 @@
---
entity_slug: annual_plate_addition_estimation
evaluator: null
evaluated_at: '2026-02-23T00:33:49.053654'
overall_score: 4.4
scores:
- name: definition_precision
value: 4.0
max_value: 5.0
rationale: The definition is quite precise, clearly distinguishing between new plate
made from fresh silver versus recycled old plate, and explicitly connects this
to determining true silver import demand. The concept is distinct and well-bounded
rather than vague.
- name: source_grounding
value: 5.0
max_value: 5.0
rationale: This is directly grounded in Smith's actual analysis in Book IV, Chapter
6, where he makes this specific argument about plate recycling to challenge assumptions
about silver import needs. The entity accurately reflects Smith's reasoning without
introducing foreign concepts.
- name: domain_placement
value: 5.0
max_value: 5.0
rationale: The "Exchange" domain is perfectly appropriate since this estimation
directly relates to understanding the balance of trade and the true demand for
imported silver. It's fundamentally about international exchange dynamics rather
than production or consumption per se.
- name: vsm_relevance
value: 4.0
max_value: 5.0
rationale: This maps well to S4 (intelligence/environmental adaptation) as it represents
analytical intelligence gathering about trade patterns and market realities that
inform policy decisions. It could also relate to S3 as an audit function challenging
conventional assumptions about trade balances.
- name: explanatory_value
value: 4.0
max_value: 5.0
rationale: This entity illuminates an important mechanism in Smith's argument against
mercantilism - showing how recycling dynamics affect import demand calculations.
It reveals structural relations between domestic consumption patterns and international
trade flows rather than just naming a surface phenomenon.
---
# Evaluation: Annual Plate Addition Estimation
## definition_precision — 4.0 / 5.0
The definition is quite precise, clearly distinguishing between new plate made from fresh silver versus recycled old plate, and explicitly connects this to determining true silver import demand. The concept is distinct and well-bounded rather than vague.
## source_grounding — 5.0 / 5.0
This is directly grounded in Smith's actual analysis in Book IV, Chapter 6, where he makes this specific argument about plate recycling to challenge assumptions about silver import needs. The entity accurately reflects Smith's reasoning without introducing foreign concepts.
## domain_placement — 5.0 / 5.0
The "Exchange" domain is perfectly appropriate since this estimation directly relates to understanding the balance of trade and the true demand for imported silver. It's fundamentally about international exchange dynamics rather than production or consumption per se.
## vsm_relevance — 4.0 / 5.0
This maps well to S4 (intelligence/environmental adaptation) as it represents analytical intelligence gathering about trade patterns and market realities that inform policy decisions. It could also relate to S3 as an audit function challenging conventional assumptions about trade balances.
## explanatory_value — 4.0 / 5.0
This entity illuminates an important mechanism in Smith's argument against mercantilism - showing how recycling dynamics affect import demand calculations. It reveals structural relations between domestic consumption patterns and international trade flows rather than just naming a surface phenomenon.