feat(llm): add LLM integration module with OpenRouter and Claude Code adapters
Implements markitect/llm/ package with concrete LLMAdapter implementations:
- OpenRouterAdapter: HTTP via urllib with retry/backoff on 429/5xx
- ClaudeCodeAdapter: subprocess-based Claude CLI with stdin piping
- Factory pattern: create_adapter("openrouter") or create_adapter("claude-code")
- API key resolution chain: constructor > env var > project-root key file
- 42 unit tests, 2 integration tests (gated on API key / CLI availability)
Also adds the infospace-with-history example with Wealth of Nations VSM
analysis pipeline, templates, schemas, source chapters, and processed
output for chapters 1-2. process_chapters.py now supports --provider
and --model flags for automatic LLM-driven processing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
16
markitect/llm/_token_estimator.py
Normal file
16
markitect/llm/_token_estimator.py
Normal file
@@ -0,0 +1,16 @@
|
||||
"""
|
||||
Rough token estimation for backends that don't return usage data.
|
||||
|
||||
Uses the ~4 characters per token heuristic common across English LLM tokenizers.
|
||||
"""
|
||||
|
||||
|
||||
def estimate_tokens(text: str) -> int:
|
||||
"""Estimate the number of tokens in *text*.
|
||||
|
||||
This is intentionally coarse — it is only used by the Claude Code CLI
|
||||
adapter where real token counts are unavailable.
|
||||
"""
|
||||
if not text:
|
||||
return 0
|
||||
return max(1, len(text) // 4)
|
||||
Reference in New Issue
Block a user