Add activity-core LLM endpoint support
Some checks failed
CI / test (3.10) (push) Has been cancelled
CI / test (3.11) (push) Has been cancelled
CI / test (3.12) (push) Has been cancelled

This commit is contained in:
2026-06-07 19:24:45 +02:00
parent 1d9fc107ed
commit 14ba47c129
25 changed files with 2082 additions and 18 deletions

View File

@@ -62,7 +62,51 @@ Execute a prompt through the configured adapter.
|------|-----------|
| 400 | Missing `prompt` field or invalid JSON body |
| 404 | Unknown path |
| 500 | Adapter raised an exception |
| 429 | Provider rate limit |
| 500 | Configuration or adapter failure |
| 502 | Provider API / transport failure |
| 504 | Provider timeout |
Server error bodies are structured and must not expose provider credentials:
```json
{
"error": "provider_api_error",
"message": "HTTP 500 from https://provider.example/v1?key=<redacted>",
"type": "LLMAPIError",
"provider_status": 500
}
```
Known error codes include `unknown_profile`, `configuration_error`,
`provider_api_error`, `provider_rate_limited`, `provider_timeout`,
`budget_exceeded`, `llm_error`, and `internal_error`.
## Runtime profiles
Server CLI mode wraps the configured adapter with runtime profile dispatch
unless `--disable-profiles` is passed. The activity-core profile
`custodian-triage-balanced` is built in and resolves to the configured provider
and model before calling the underlying adapter.
Default profile values:
| Field | Default |
|-------|---------|
| provider | `openrouter` |
| model | `anthropic/claude-sonnet-4` |
| temperature | `0.2` |
| max_tokens | `1800` |
| max_depth | `2` |
| timeout_seconds | `300` |
| model_params.reasoning_effort | `medium` |
Profile provider/model and default call values can be overridden with
environment variables such as `LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER`,
`LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL`, and
`LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_TOKENS`. Operators can also set
`LLM_CONNECT_PROFILES_JSON` or `LLM_CONNECT_PROFILE_FILE` to provide JSON
profile definitions keyed by profile name.
## Implementation notes
@@ -75,10 +119,12 @@ Execute a prompt through the configured adapter.
## CLI
```
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL]
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL] [--disable-profiles] [--strict-profiles]
```
Default provider: `mock`. All registered providers from `create_adapter` are valid.
CLI defaults can also be supplied with `LLM_CONNECT_HOST`, `LLM_CONNECT_PORT`,
`LLM_CONNECT_PROVIDER`, and `LLM_CONNECT_MODEL`. Default provider: `mock`. All
registered providers from `create_adapter` are valid.
## Known consumers