generated from coulomb/repo-seed
tegwick
019f6e7dc7
Wire LLMConnectAdapter behind the existing LLMAdapter seam with config-driven selection, graceful degradation, --offline mode, and bounded session context. Add unit tests, integration docs, and update README/SCOPE/AGENTS.
100 lines
3.3 KiB
Markdown
100 lines
3.3 KiB
Markdown
# llm-connect Integration (CYA-WP-0008)
|
|
|
|
## Mapping: cya ↔ llm-connect
|
|
|
|
| cya (`AssistanceRequest`) | llm-connect |
|
|
|---------------------------|-------------|
|
|
| `user_request` | User message body (after context framing) |
|
|
| `context` (envelope + memory + `session_turns`) | Serialized into the prompt via `cya.llm.prompt.build_assistance_prompt` |
|
|
| `hints` (`model`, `temperature`, `max_tokens`) | `RunConfig` fields for `execute_prompt` |
|
|
| `AssistanceResponse.suggestion` | `LLMResponse.content` |
|
|
| `AssistanceResponse.metadata` | `LLMResponse.model`, `usage`, `finish_reason` |
|
|
|
|
llm-connect owns provider clients (`create_adapter`), API key resolution, retries, and
|
|
token usage. `cya` never imports vendor SDKs directly.
|
|
|
|
## Configuration
|
|
|
|
User config: `~/.config/cya/config.toml`
|
|
|
|
```toml
|
|
[llm]
|
|
adapter = "connect" # "connect" | "fake" (default: fake when absent)
|
|
backend = "openrouter" # openrouter | openai | gemini | claude-code | mock
|
|
model = "anthropic/claude-sonnet-4"
|
|
temperature = 0.3
|
|
max_tokens = 2000
|
|
api_key_env = "OPENROUTER_API_KEY" # optional override
|
|
# system_prompt = "..." # optional; uses cya default when omitted
|
|
```
|
|
|
|
Optional project override: `.cya.toml` (same `[llm]` section; merged over user config).
|
|
|
|
Environment overrides:
|
|
|
|
| Variable | Purpose |
|
|
|----------|---------|
|
|
| `CYA_LLM_ADAPTER` | `connect` or `fake` |
|
|
| `CYA_LLM_BACKEND` / `CYA_LLM_PROVIDER` | Provider name |
|
|
| `CYA_LLM_MODEL` | Model id |
|
|
|
|
CLI: `cya --offline "..."` forces `FakeLLMAdapter`.
|
|
|
|
## Session context budget (multi-turn / `cya shell`)
|
|
|
|
Recent turns are passed in `AssistanceRequest.context["session_turns"]` as
|
|
`{"user": "...", "assistant": "..."}` records.
|
|
|
|
Bounds (see `cya.config`):
|
|
|
|
- **Max turns:** 10
|
|
- **Max characters:** 4000 (total across included turns)
|
|
|
|
Older or oversized history is dropped from the prompt automatically.
|
|
|
|
## Credential routing
|
|
|
|
Do **not** commit API keys. Before requesting secrets, route custody:
|
|
|
|
```bash
|
|
warden route find "OpenRouter API key" --json
|
|
warden route show <catalog-id> --json
|
|
```
|
|
|
|
Typical ownership:
|
|
|
|
| Need | Owner | ops-warden executes? |
|
|
|------|-------|----------------------|
|
|
| `OPENROUTER_API_KEY` | OpenBao (`railiance-platform`) | No — route only |
|
|
| `OPENAI_API_KEY` | OpenBao | No — route only |
|
|
| `GEMINI_API_KEY` | OpenBao | No — route only |
|
|
|
|
llm-connect resolves keys via `resolve_api_key()` (explicit arg → env var → project key file).
|
|
|
|
## Adapter selection
|
|
|
|
`cya.llm.factory.get_adapter()` is the single factory for one-shot and shell paths:
|
|
|
|
1. `--offline` or `CYA_LLM_ADAPTER=fake` → `FakeLLMAdapter`
|
|
2. `adapter = "connect"` in config/env → `LLMConnectAdapter` (graceful degrade on failure)
|
|
3. Otherwise → `FakeLLMAdapter` (current default)
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
make dev-install
|
|
pip install -e ~/llm-connect # sibling checkout
|
|
```
|
|
|
|
Optional extra group (placeholder for packaging): `pip install -e ".[llm]"`.
|
|
|
|
## Tests
|
|
|
|
- Default CI: `make test` — mocks llm-connect; no network.
|
|
- Manual live check: `pytest -m llm_live` (requires configured API key).
|
|
|
|
## Known gaps
|
|
|
|
- Structured JSON output schema not enforced yet (free-form model text).
|
|
- `claude-code` backend does not require an API key; other backends do.
|
|
- Per-directory `.cya.toml` overrides user config but does not yet mirror llm-connect's full 7-layer resolution. |