Files

tegwick a9ca0adfcf feat(example): add per-entity LLM evaluations for 985 WoN entities (S3.3)

Batch evaluation of all 988 entities via OpenRouter. 984 succeeded on
first pass; 3 failed (network errors). eval-summary --update-metrics
written with per_entity_mean=3.9556.

Viability dashboard: 6/6 PASS
  redundancy_ratio   0.0061  (max 0.10)
  coverage_ratio     0.6190  (min 0.40)
  coherence_comps    0.0000  (max 3)
  consistency_cycles 0.0000  (max 0)
  granularity_entropy 2.6748 (min 1.0)
  per_entity_mean    3.9556  (min 3.5)

Dimension breakdown (mean across 985 entities):
  definition_precision  3.62
  source_grounding      4.36
  domain_placement      4.56
  vsm_relevance         3.31
  explanatory_value     3.94

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-02-23 09:36:46 +01:00

3.9 KiB

Raw Blame History

entity_slug, evaluator, evaluated_at, overall_score, scores

entity_slug

evaluator

evaluated_at

overall_score

scores

public_fiars

null

2026-02-23T06:12:53.320746

4.0

name	value	max_value	rationale
definition_precision	4.0	5.0	The definition is quite precise, clearly identifying public fiars as official annual grain price valuations conducted by Scottish assize courts with specific purposes (standardizing corn-to-money rent conversions). The only minor imprecision is that it could better specify the exact legal authority and scope of these valuations.

name	value	max_value	rationale
source_grounding	4.0	5.0	This appears well-grounded in Smith's actual discussion of Scottish grain pricing mechanisms in Book I, Chapter 11, where he examines various attempts at price regulation. Smith would likely have discussed such institutional arrangements when analyzing agricultural markets and rent conversions in Scotland.

name	value	max_value	rationale
domain_placement	5.0	5.0	"Regulation" is the perfect domain placement for this entity, as public fiars represent a clear example of government intervention in market pricing mechanisms. This is precisely the type of regulatory institution Smith would analyze when discussing state involvement in economic affairs.

name	value	max_value	rationale
vsm_relevance	3.0	5.0	Public fiars map reasonably well to S3 (internal regulation/audit) as they represent systematic monitoring and standardization of prices within the economic system. However, they also have elements of S2 (coordination) in standardizing price information across markets, making the VSM placement somewhat ambiguous.

name	value	max_value	rationale
explanatory_value	4.0	5.0	This entity provides good explanatory value by illuminating a specific institutional mechanism for price regulation and market coordination that Smith would use to analyze the effectiveness of government intervention versus market forces. It represents a concrete example of how societies attempt to manage agricultural market volatility through formal institutions.

Evaluation: Public Fiars

definition_precision — 4.0 / 5.0

The definition is quite precise, clearly identifying public fiars as official annual grain price valuations conducted by Scottish assize courts with specific purposes (standardizing corn-to-money rent conversions). The only minor imprecision is that it could better specify the exact legal authority and scope of these valuations.

source_grounding — 4.0 / 5.0

This appears well-grounded in Smith's actual discussion of Scottish grain pricing mechanisms in Book I, Chapter 11, where he examines various attempts at price regulation. Smith would likely have discussed such institutional arrangements when analyzing agricultural markets and rent conversions in Scotland.

domain_placement — 5.0 / 5.0

"Regulation" is the perfect domain placement for this entity, as public fiars represent a clear example of government intervention in market pricing mechanisms. This is precisely the type of regulatory institution Smith would analyze when discussing state involvement in economic affairs.

vsm_relevance — 3.0 / 5.0

Public fiars map reasonably well to S3 (internal regulation/audit) as they represent systematic monitoring and standardization of prices within the economic system. However, they also have elements of S2 (coordination) in standardizing price information across markets, making the VSM placement somewhat ambiguous.

explanatory_value — 4.0 / 5.0

This entity provides good explanatory value by illuminating a specific institutional mechanism for price regulation and market coordination that Smith would use to analyze the effectiveness of government intervention versus market forces. It represents a concrete example of how societies attempt to manage agricultural market volatility through formal institutions.

3.9 KiB Raw Blame History