Commit Graph

42 Commits

Author SHA1 Message Date
7f1eecbdb2 feat(infospace): add eval-summary command and improve evaluate pipeline (S3.3)
- Fix evaluate dimensions to match template file:
  definition_precision, source_grounding, domain_placement,
  vsm_relevance, explanatory_value (was domain_relevance,
  discipline_alignment, conceptual_clarity)
- Add VSM background context to evaluation prompt so LLM can
  score vsm_relevance without macro injection
- Fix model_name bug: was sending literal "default" to API (HTTP 400)
- Refactor run_entity_evaluation to write files incrementally via
  callback rather than all at once after the batch — long runs are
  now resumable if interrupted
- Add incremental skip in CLI: entities with existing eval files
  are skipped automatically on re-run (acts as resume)
- Add eval-summary command: reads all eval files, shows per-dimension
  means, optionally writes per_entity_mean to metrics.yaml
- Fix record_check_results to merge rather than overwrite metrics.yaml
  so per_entity_mean survives subsequent check runs
- Add per_entity_mean viability threshold (min: 3.5) to infospace.yaml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-23 01:26:45 +01:00
0f54f094e4 chore(example): final metrics snapshot — all 35 chapters processed
1021 entities extracted across all Books 1-5 of The Wealth of Nations.
Final metrics: coverage=0.4424, granularity=2.9533, redundancy=0.0059.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 22:54:54 +01:00
4a15a50337 infospace: process book-5-chapter-03
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:54:40 +01:00
92dfe367c7 infospace: process book-5-chapter-02
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:46:32 +01:00
23c397e46a infospace: process book-5-chapter-01
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:36:06 +01:00
e695ddfbbd infospace: process book-4-chapter-09
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:32:07 +01:00
5245dbbfc8 infospace: process book-4-chapter-08
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:25:52 +01:00
4319d2a32b infospace: process book-4-chapter-07
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:14:18 +01:00
efdaa884c8 infospace: process book-4-chapter-06
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 22:01:44 +01:00
2804de3d24 infospace: process book-4-chapter-05
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:47:52 +01:00
3e96ac7b8d infospace: process book-4-chapter-04
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:36:17 +01:00
a687e508f3 infospace: process book-4-chapter-03
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:31:40 +01:00
da9c5fce80 infospace: process book-4-chapter-02
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:19:39 +01:00
cd87ebfdc0 infospace: process book-4-chapter-01
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:13:08 +01:00
666f78d1ba infospace: process book-4-introduction
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 21:02:00 +01:00
579e02989b infospace: process book-3-chapter-04
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 20:46:20 +01:00
8401c69ff2 infospace: process book-3-chapter-03
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 20:40:35 +01:00
06e904ccf5 infospace: process book-3-chapter-02
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 20:30:22 +01:00
59d42b1665 infospace: process book-3-chapter-01
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 20:18:15 +01:00
8c11e13fef infospace: process book-2-chapter-05
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 20:03:11 +01:00
ac4e508aff infospace: process book-2-chapter-04
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 19:57:59 +01:00
8e1943afdb infospace: process book-2-chapter-03
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 19:50:53 +01:00
05711e541d infospace: process book-2-chapter-02
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 19:43:19 +01:00
8cb9ee6f6e infospace: process book-2-chapter-01
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 19:26:57 +01:00
db129fde6b infospace: process book-1-chapter-11
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 19:19:20 +01:00
6d9ec4e34b infospace: process book-1-chapter-10
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 18:59:36 +01:00
368571905a infospace: process book-1-chapter-09
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:58:08 +01:00
9c95912d68 infospace: process book-1-chapter-08
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:47:12 +01:00
0828581269 infospace: process book-1-chapter-07
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:40:24 +01:00
283abac378 infospace: process book-1-chapter-06
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:29:59 +01:00
098b781f92 infospace: process book-1-chapter-05
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:20:35 +01:00
eea397a380 infospace: process book-1-chapter-04
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:12:54 +01:00
7615beb139 chore(example): update metrics after chapter-03 collection check
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 15:06:03 +01:00
c2e06c15d7 infospace: process book-1-chapter-03
Extract entities, map to VSM, and synthesize analysis.
2026-02-19 15:04:57 +01:00
715ef19d1c infospace: remove example output — will replay chapter by chapter
This commit clears the tangled example output so each chapter
can be re-committed cleanly via S3.2.
2026-02-19 09:22:55 +01:00
3ac8447c10 feat(example): add baseline metrics snapshot from collection checks run
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Initial metrics from S2.4 checks on 85 entities (7 of 35 chapters):
coverage_ratio=0.361, redundancy=0.0, coherence_components=0.0,
consistency_cycles=0.0, granularity_entropy=2.69

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-19 07:44:01 +01:00
41773f1320 feat(llm): add OpenAI adapter, entity archive policy, process chapters 5-7
Add OpenAIAdapter for the OpenAI chat completions API (apikey-chatgpt.txt
or OPENAI_API_KEY). Set default model to arcee-ai/trinity-large-preview:free
for the infospace pipeline and increase max_tokens from 4096 to 8192.

Reprocess chapter 05 with Trinity Large (was Gemini: 1 truncated entity,
now 19 complete entities). Process chapters 06 (Aurora Alpha, 10 entities)
and 07 (Trinity Large, 15 entities including regenerated violent-policy.md).
Canonical set now at 85 unique entities.

Add entity archive policy: entities are never silently deleted. Retired
entities move to output/entities/archive/ with a dated reason header.
New CLI option: --archive-entity <slug> --reason "...". The --list
output shows the archive count alongside the canonical set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 23:39:44 +01:00
880c1d1374 feat(llm): add Gemini adapter and process book-1-chapter-05
Add GeminiAdapter calling Google's Generative Language REST API
(default model: gemini-2.5-flash). Register "gemini" as third
provider in the factory and CLI. Add rate-limit retry with
exponential backoff to the pipeline's _call_llm helper. Increase
default max_tokens from 2000 to 4096.

Process book-1-chapter-05 via Gemini free tier — 1 new entity
extracted (necessaries-conveniencies-and-amusements-of-life),
41 existing entities correctly skipped by dedup. Canonical set
now at 42 unique entities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 22:54:37 +01:00
2d1282a61e feat(infospace): flat canonical entity set with cross-chapter deduplication
Restructure entity storage from per-chapter subdirectories to a flat
canonical set in output/entities/. Each entity exists as a single file;
duplicates across chapters are detected by slug collision and skipped
(first occurrence wins). Chapter views use {{ include }} transclusion
to reference shared entity files.

Add @{existing_entities} macro to extract-entities template so the LLM
knows which entities already exist and focuses on genuinely new ones.
Refactor _call_llm() from _execute_llm() for callers that handle their
own file I/O. 41 unique entities from 4 chapters (2 duplicates removed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 22:24:20 +01:00
ad84dd3a41 infospace: process book-1-chapter-04 via OpenRouter
All 3 stages (entities, mappings, analysis) auto-generated.
1m53s wall time, 9,478 tokens (real), ~$0.07 est. cost.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 01:42:05 +01:00
e806a701ca infospace: process book-1-chapter-03 with LLM integration
Auto-generated mappings and analysis via Claude Code CLI adapter.
Entities were already present from a previous session.

Stats: 5m04s wall time, ~51K estimated tokens, ~$0.35 estimated cost.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 01:32:24 +01:00
fecc2fd4fa feat(llm): add LLM integration module with OpenRouter and Claude Code adapters
Implements markitect/llm/ package with concrete LLMAdapter implementations:
- OpenRouterAdapter: HTTP via urllib with retry/backoff on 429/5xx
- ClaudeCodeAdapter: subprocess-based Claude CLI with stdin piping
- Factory pattern: create_adapter("openrouter") or create_adapter("claude-code")
- API key resolution chain: constructor > env var > project-root key file
- 42 unit tests, 2 integration tests (gated on API key / CLI availability)

Also adds the infospace-with-history example with Wealth of Nations VSM
analysis pipeline, templates, schemas, source chapters, and processed
output for chapters 1-2. process_chapters.py now supports --provider
and --model flags for automatic LLM-driven processing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 01:17:58 +01:00