tegwick
fecc2fd4fa
Implements markitect/llm/ package with concrete LLMAdapter implementations:
- OpenRouterAdapter: HTTP via urllib with retry/backoff on 429/5xx
- ClaudeCodeAdapter: subprocess-based Claude CLI with stdin piping
- Factory pattern: create_adapter("openrouter") or create_adapter("claude-code")
- API key resolution chain: constructor > env var > project-root key file
- 42 unit tests, 2 integration tests (gated on API key / CLI availability)
Also adds the infospace-with-history example with Wealth of Nations VSM
analysis pipeline, templates, schemas, source chapters, and processed
output for chapters 1-2. process_chapters.py now supports --provider
and --model flags for automatic LLM-driven processing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1.6 KiB
1.6 KiB
Assess Completeness & Consistency Metrics
You are a quality assurance analyst evaluating the completeness and consistency of a growing information space that maps classical economics to the Viable System Model.
All Chapter Analyses
@{all_analyses}
VSM Framework Reference
@{vsm_framework}
Instructions
- Review all chapter analyses produced so far.
- Compute the following completeness metrics:
- VSM Concept Coverage: Percentage of VSM concepts (S1, S2, S3, S3*, S4, S5, plus key concepts like recursion, variety, requisite variety, autonomy, viability, attenuation/amplification, algedonic signals) that have at least one mapping. List covered and uncovered concepts.
- Chapter Coverage: Percentage of source chapters processed.
- Entity Count: Total number of distinct economic entities extracted.
- Mapping Count: Total number of entity-to-VSM mappings created.
- Compute the following consistency metrics:
- Terminology Consistency (score 0.0-1.0): Whether the same economic concept is named consistently across all documents. Flag inconsistencies.
- Cross-reference Integrity (score 0.0-1.0): Whether all entity references in mappings point to existing entities. Flag broken references.
- Schema Compliance (percentage): Percentage of output documents that contain all required sections per their respective schemas.
- Produce a metrics report summarizing the current state of the information space.
Output Format
Output a single markdown document with sections for each metric category: Completeness Metrics, Consistency Metrics, and Recommendations.