markitect-main/roadmap/llm-shared-library/PLAN.md

LLM Adapter Layer — Extract as Shared Library

Vision

The markitect.llm module is a clean, stdlib-only adapter layer for calling LLMs via OpenRouter, Gemini, OpenAI, and the Claude Code CLI. It implements a uniform interface, a 7-layer TOML config chain, embedding support with caching, and typed exceptions. It should be usable by all projects in the Bernd Worsch ecosystem without pulling in all of markitect.

This roadmap tracks extracting it into a standalone installable library.

---

Current State

The module lives at markitect/llm/ (~16 files, ~1500 LOC, stdlib-only) and provides:

• 4 text adapters: OpenRouter, Gemini, OpenAI, Claude Code CLI
• 2 embedding adapters: OpenAI-compatible (OpenAI + OpenRouter)
• Embedding cache: JSON-backed, content-digest validated
• Similarity utilities: pure-Python cosine similarity, matrix, pair-finding
• 7-layer TOML config chain: CLI > env > user/dir preference/default > hardcoded
• Typed exceptions: LLMError hierarchy
• HTTP wrapper: urllib-only, typed exception translation

Two Coupling Issues Blocking Clean Extraction

Issue	Location	Severity
RunConfig and LLMResponse are defined in markitect.prompts.execution.models, not in markitect.llm	markitect/prompts/execution/models.py	High — creates cross-module import for all consumers
TOML config chain hardcodes "markitect" as app name (paths: ~/.config/markitect/, env prefix MARKITECT_, files: .markitect.toml)	markitect/llm/toml_config.py	Medium — consumers either accept markitect config or can't use the chain

---

Terminology

• adapter: concrete implementation of LLMAdapter for a single provider
• factory: create_adapter() / create_embedding_adapter() — provider-agnostic entry points
• config chain: 7-layer resolution of provider + model (CLI → env → TOML → hardcoded)
• standalone library: a Python package installable with pip install from a git URL or local path, without PyPI
• consumer: any project that imports and uses the library (markitect itself, custodian, railiance, etc.)

---

Packaging Decision (Pending)

Before Phase 2 starts, one architectural decision must be resolved:

D1: Where does the extracted library live?

Option A — Standalone repo (~/bw-llm or similar):

• Clean separation, versioned independently, installable via pip install git+file:///... or git URL
• Adds a repo to maintain; changes require bumping version in dependents

Option B — Subfolder of markitect with own pyproject.toml (monorepo-lite):

• Stays co-located with the main codebase that will use it most
• Less friction for iteration; single git history
• Slightly unorthodox but valid for personal infrastructure

Option C — Just pip install markitect in other projects:

• Zero extraction work; reuse today
• Pulls all of markitect (prompts, infospace, CLI, etc.) as transitive deps
• Acceptable short-term if other projects are small

---

Stages

Stage 1 — Decouple (within markitect)

Prepare the module for extraction without changing its public API.

S1.1 — Move RunConfig + LLMResponse into markitect.llm

RunConfig and LLMResponse are currently in markitect.prompts.execution.models. The LLM adapters import from there, creating a hard dependency on the prompt system.

Work:

• Move both dataclasses to markitect/llm/models.py
• Update all imports in markitect.llm and markitect.prompts
• Keep a re-export shim in markitect.prompts.execution.models for backwards compat

Acceptance: markitect/llm/ has zero imports from markitect.prompts.*

S1.2 — Parameterize the TOML config chain

Replace the hardcoded "markitect" app name with a configurable app_name parameter.

Work:

• Add app_name: str = "markitect" parameter to resolve_llm() and the config path helpers in toml_config.py
• Derive config file path (~/.config/{app_name}/config.toml), env prefix ({APP_NAME}_HELPER_MODEL), and local config file (.{app_name}.toml) from it
• All existing behaviour is preserved when app_name="markitect" (default)

Acceptance: A consumer can call resolve_llm(app_name="railiance") and get config from ~/.config/railiance/config.toml and RAILIANCE_HELPER_MODEL.

S1.3 — Isolation tests

Write a test file that imports only from markitect.llm.* and verifies no accidental coupling remains.

Acceptance: pytest tests/test_llm_isolation.py passes; no import of markitect.prompts or markitect.infospace in the LLM module tree.

---

Stage 2 — Extract

S2.1 — Resolve D1: packaging location

Record the decision and create the package scaffold.

Acceptance: D1 resolved, pyproject.toml for the library exists at the chosen location with name, version 0.1.0, and declared dependencies.

S2.2 — Create standalone package

Move (or symlink) the llm module into the new package structure. Wire up the pyproject.toml entry points. Verify pip install -e <path> works.

Files to carry over:

llm/
  __init__.py          # re-exports: create_adapter, create_embedding_adapter,
                       #   LLMAdapter, EmbeddingAdapter, LLMConfig, exceptions
  models.py            # RunConfig, LLMResponse (moved from S1.1)
  config.py            # load_config, resolve_api_key
  toml_config.py       # resolve_llm (parameterized from S1.2)
  factory.py           # create_adapter
  exceptions.py        # LLM exception hierarchy
  openrouter.py
  claude_code.py
  gemini.py
  openai.py
  embedding_adapter.py
  embedding_openai.py
  embedding_factory.py # create_embedding_adapter
  embedding_cache.py
  similarity.py
  _http.py
  _token_estimator.py

Acceptance: python -c "from bw_llm import create_adapter; print('ok')" works in a fresh venv with only the new package installed.

S2.3 — Update markitect to depend on extracted package

Replace markitect/llm/ with an import alias pointing to the new package, or add the package as a path dependency in markitect's pyproject.toml.

Acceptance: All markitect tests pass; markitect/llm/__init__.py is either removed or becomes a thin re-export of bw_llm.

S2.4 — Integration smoke test

Run the full markitect infospace pipeline (entity extraction + evaluation) end-to-end against a small fixture to confirm nothing broke.

Acceptance: markitect infospace evaluate --dry-run succeeds on a 3-entity fixture.

---

Stage 3 — Adopt in First Consumer

S3.1 — Integrate in one other project

Pick the first real consumer (likely the custodian state-hub, for LLM-assisted state summaries or decision rationale generation) and wire up the library.

Work:

• Add bw-llm (or equivalent) as a dependency
• Write a small usage example (e.g., llm_helper.py)
• Confirm config chain works with the consumer's own app name

S3.2 — Usage guide

Write README.md for the library covering:

• Installation (local path / git URL)
• Supported providers and env vars
• TOML config file locations and format
• create_adapter() / create_embedding_adapter() quick-start
• Error handling

Acceptance: Another developer (or agent) can follow the README to use the library in a new project without reading source code.

---

Stage Summary

Stage	Description	Key Deliverable	Blocks
S1.1	Move RunConfig/LLMResponse to llm	Zero cross-module deps	S2.2
S1.2	Parameterize app name	Configurable config chain	S2.2
S1.3	Isolation tests	Green test suite	S2.1
S2.1	Resolve packaging decision (D1)	pyproject.toml scaffold	S2.2
S2.2	Create standalone package	pip install works	S2.3
S2.3	Update markitect	markitect uses extracted lib	S2.4
S2.4	Integration smoke test	Full pipeline passes	S3.1
S3.1	First consumer integration	Library used in real project	S3.2
S3.2	Usage guide	README published	—

---

Out of Scope

• Publishing to PyPI (unnecessary for personal infrastructure; git/local installs suffice)
• Adding new LLM providers (separate concern)
• Porting the helper CLI to the library (the CLI is markitect-specific)
• Async adapters (current sync interface is sufficient; can be added later)