tegwick
fecc2fd4fa
Implements markitect/llm/ package with concrete LLMAdapter implementations:
- OpenRouterAdapter: HTTP via urllib with retry/backoff on 429/5xx
- ClaudeCodeAdapter: subprocess-based Claude CLI with stdin piping
- Factory pattern: create_adapter("openrouter") or create_adapter("claude-code")
- API key resolution chain: constructor > env var > project-root key file
- 42 unit tests, 2 integration tests (gated on API key / CLI availability)
Also adds the infospace-with-history example with Wealth of Nations VSM
analysis pipeline, templates, schemas, source chapters, and processed
output for chapters 1-2. process_chapters.py now supports --provider
and --model flags for automatic LLM-driven processing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
39 lines
1.6 KiB
Markdown
39 lines
1.6 KiB
Markdown
# Assess Completeness & Consistency Metrics
|
|
|
|
You are a quality assurance analyst evaluating the completeness and
|
|
consistency of a growing information space that maps classical economics
|
|
to the Viable System Model.
|
|
|
|
## All Chapter Analyses
|
|
|
|
@{all_analyses}
|
|
|
|
## VSM Framework Reference
|
|
|
|
@{vsm_framework}
|
|
|
|
## Instructions
|
|
|
|
1. Review all chapter analyses produced so far.
|
|
2. Compute the following completeness metrics:
|
|
- **VSM Concept Coverage**: Percentage of VSM concepts (S1, S2, S3, S3*, S4, S5,
|
|
plus key concepts like recursion, variety, requisite variety, autonomy,
|
|
viability, attenuation/amplification, algedonic signals) that have at
|
|
least one mapping. List covered and uncovered concepts.
|
|
- **Chapter Coverage**: Percentage of source chapters processed.
|
|
- **Entity Count**: Total number of distinct economic entities extracted.
|
|
- **Mapping Count**: Total number of entity-to-VSM mappings created.
|
|
3. Compute the following consistency metrics:
|
|
- **Terminology Consistency** (score 0.0-1.0): Whether the same economic
|
|
concept is named consistently across all documents. Flag inconsistencies.
|
|
- **Cross-reference Integrity** (score 0.0-1.0): Whether all entity references
|
|
in mappings point to existing entities. Flag broken references.
|
|
- **Schema Compliance** (percentage): Percentage of output documents that
|
|
contain all required sections per their respective schemas.
|
|
4. Produce a metrics report summarizing the current state of the information space.
|
|
|
|
## Output Format
|
|
|
|
Output a single markdown document with sections for each metric category:
|
|
Completeness Metrics, Consistency Metrics, and Recommendations.
|