diff --git a/roadmap/llm-shared-library/PLAN.md b/roadmap/llm-shared-library/PLAN.md new file mode 100644 index 00000000..256c0502 --- /dev/null +++ b/roadmap/llm-shared-library/PLAN.md @@ -0,0 +1,214 @@ +# LLM Adapter Layer — Extract as Shared Library + +## Vision + +The `markitect.llm` module is a clean, stdlib-only adapter layer for calling +LLMs via OpenRouter, Gemini, OpenAI, and the Claude Code CLI. It implements a +uniform interface, a 7-layer TOML config chain, embedding support with caching, +and typed exceptions. It should be usable by all projects in the Bernd Worsch +ecosystem without pulling in all of markitect. + +This roadmap tracks extracting it into a standalone installable library. + +--- + +## Current State + +The module lives at `markitect/llm/` (~16 files, ~1500 LOC, stdlib-only) and +provides: +- **4 text adapters**: OpenRouter, Gemini, OpenAI, Claude Code CLI +- **2 embedding adapters**: OpenAI-compatible (OpenAI + OpenRouter) +- **Embedding cache**: JSON-backed, content-digest validated +- **Similarity utilities**: pure-Python cosine similarity, matrix, pair-finding +- **7-layer TOML config chain**: CLI > env > user/dir preference/default > hardcoded +- **Typed exceptions**: LLMError hierarchy +- **HTTP wrapper**: urllib-only, typed exception translation + +### Two Coupling Issues Blocking Clean Extraction + +| Issue | Location | Severity | +|-------|----------|----------| +| `RunConfig` and `LLMResponse` are defined in `markitect.prompts.execution.models`, not in `markitect.llm` | `markitect/prompts/execution/models.py` | High — creates cross-module import for all consumers | +| TOML config chain hardcodes `"markitect"` as app name (paths: `~/.config/markitect/`, env prefix `MARKITECT_`, files: `.markitect.toml`) | `markitect/llm/toml_config.py` | Medium — consumers either accept markitect config or can't use the chain | + +--- + +## Terminology + +- **adapter**: concrete implementation of `LLMAdapter` for a single provider +- **factory**: `create_adapter()` / `create_embedding_adapter()` — provider-agnostic entry points +- **config chain**: 7-layer resolution of provider + model (CLI → env → TOML → hardcoded) +- **standalone library**: a Python package installable with `pip install` from a git URL or local path, without PyPI +- **consumer**: any project that imports and uses the library (markitect itself, custodian, railiance, etc.) + +--- + +## Packaging Decision (Pending) + +Before Phase 2 starts, one architectural decision must be resolved: + +> **D1: Where does the extracted library live?** +> +> **Option A — Standalone repo** (`~/bw-llm` or similar): +> - Clean separation, versioned independently, installable via `pip install git+file:///...` or git URL +> - Adds a repo to maintain; changes require bumping version in dependents +> +> **Option B — Subfolder of markitect with own `pyproject.toml`** (monorepo-lite): +> - Stays co-located with the main codebase that will use it most +> - Less friction for iteration; single git history +> - Slightly unorthodox but valid for personal infrastructure +> +> **Option C — Just `pip install markitect` in other projects**: +> - Zero extraction work; reuse today +> - Pulls all of markitect (prompts, infospace, CLI, etc.) as transitive deps +> - Acceptable short-term if other projects are small + +--- + +## Stages + +### Stage 1 — Decouple (within markitect) + +Prepare the module for extraction without changing its public API. + +#### S1.1 — Move RunConfig + LLMResponse into markitect.llm + +`RunConfig` and `LLMResponse` are currently in `markitect.prompts.execution.models`. +The LLM adapters import from there, creating a hard dependency on the prompt system. + +**Work:** +- Move both dataclasses to `markitect/llm/models.py` +- Update all imports in `markitect.llm` and `markitect.prompts` +- Keep a re-export shim in `markitect.prompts.execution.models` for backwards compat + +**Acceptance:** `markitect/llm/` has zero imports from `markitect.prompts.*` + +#### S1.2 — Parameterize the TOML config chain + +Replace the hardcoded `"markitect"` app name with a configurable `app_name` parameter. + +**Work:** +- Add `app_name: str = "markitect"` parameter to `resolve_llm()` and the config + path helpers in `toml_config.py` +- Derive config file path (`~/.config/{app_name}/config.toml`), env prefix + (`{APP_NAME}_HELPER_MODEL`), and local config file (`.{app_name}.toml`) from it +- All existing behaviour is preserved when `app_name="markitect"` (default) + +**Acceptance:** A consumer can call `resolve_llm(app_name="railiance")` and get +config from `~/.config/railiance/config.toml` and `RAILIANCE_HELPER_MODEL`. + +#### S1.3 — Isolation tests + +Write a test file that imports only from `markitect.llm.*` and verifies no +accidental coupling remains. + +**Acceptance:** `pytest tests/test_llm_isolation.py` passes; no import of +`markitect.prompts` or `markitect.infospace` in the LLM module tree. + +--- + +### Stage 2 — Extract + +#### S2.1 — Resolve D1: packaging location + +Record the decision and create the package scaffold. + +**Acceptance:** D1 resolved, `pyproject.toml` for the library exists at the +chosen location with name, version `0.1.0`, and declared dependencies. + +#### S2.2 — Create standalone package + +Move (or symlink) the llm module into the new package structure. Wire up +the `pyproject.toml` entry points. Verify `pip install -e ` works. + +**Files to carry over:** +``` +llm/ + __init__.py # re-exports: create_adapter, create_embedding_adapter, + # LLMAdapter, EmbeddingAdapter, LLMConfig, exceptions + models.py # RunConfig, LLMResponse (moved from S1.1) + config.py # load_config, resolve_api_key + toml_config.py # resolve_llm (parameterized from S1.2) + factory.py # create_adapter + exceptions.py # LLM exception hierarchy + openrouter.py + claude_code.py + gemini.py + openai.py + embedding_adapter.py + embedding_openai.py + embedding_factory.py # create_embedding_adapter + embedding_cache.py + similarity.py + _http.py + _token_estimator.py +``` + +**Acceptance:** `python -c "from bw_llm import create_adapter; print('ok')"` works +in a fresh venv with only the new package installed. + +#### S2.3 — Update markitect to depend on extracted package + +Replace `markitect/llm/` with an import alias pointing to the new package, or +add the package as a path dependency in markitect's `pyproject.toml`. + +**Acceptance:** All markitect tests pass; `markitect/llm/__init__.py` is either +removed or becomes a thin re-export of `bw_llm`. + +#### S2.4 — Integration smoke test + +Run the full markitect infospace pipeline (entity extraction + evaluation) end-to-end +against a small fixture to confirm nothing broke. + +**Acceptance:** `markitect infospace evaluate --dry-run` succeeds on a 3-entity fixture. + +--- + +### Stage 3 — Adopt in First Consumer + +#### S3.1 — Integrate in one other project + +Pick the first real consumer (likely the custodian state-hub, for LLM-assisted +state summaries or decision rationale generation) and wire up the library. + +**Work:** +- Add `bw-llm` (or equivalent) as a dependency +- Write a small usage example (e.g., `llm_helper.py`) +- Confirm config chain works with the consumer's own app name + +#### S3.2 — Usage guide + +Write `README.md` for the library covering: +- Installation (local path / git URL) +- Supported providers and env vars +- TOML config file locations and format +- `create_adapter()` / `create_embedding_adapter()` quick-start +- Error handling + +**Acceptance:** Another developer (or agent) can follow the README to use the library +in a new project without reading source code. + +--- + +## Stage Summary + +| Stage | Description | Key Deliverable | Blocks | +|-------|-------------|-----------------|--------| +| S1.1 | Move RunConfig/LLMResponse to llm | Zero cross-module deps | S2.2 | +| S1.2 | Parameterize app name | Configurable config chain | S2.2 | +| S1.3 | Isolation tests | Green test suite | S2.1 | +| S2.1 | Resolve packaging decision (D1) | pyproject.toml scaffold | S2.2 | +| S2.2 | Create standalone package | `pip install` works | S2.3 | +| S2.3 | Update markitect | markitect uses extracted lib | S2.4 | +| S2.4 | Integration smoke test | Full pipeline passes | S3.1 | +| S3.1 | First consumer integration | Library used in real project | S3.2 | +| S3.2 | Usage guide | README published | — | + +--- + +## Out of Scope + +- Publishing to PyPI (unnecessary for personal infrastructure; git/local installs suffice) +- Adding new LLM providers (separate concern) +- Porting the helper CLI to the library (the CLI is markitect-specific) +- Async adapters (current sync interface is sufficient; can be added later)