generated from coulomb/repo-seed
Add activity-core LLM endpoint support
This commit is contained in:
@@ -62,7 +62,51 @@ Execute a prompt through the configured adapter.
|
||||
|------|-----------|
|
||||
| 400 | Missing `prompt` field or invalid JSON body |
|
||||
| 404 | Unknown path |
|
||||
| 500 | Adapter raised an exception |
|
||||
| 429 | Provider rate limit |
|
||||
| 500 | Configuration or adapter failure |
|
||||
| 502 | Provider API / transport failure |
|
||||
| 504 | Provider timeout |
|
||||
|
||||
Server error bodies are structured and must not expose provider credentials:
|
||||
|
||||
```json
|
||||
{
|
||||
"error": "provider_api_error",
|
||||
"message": "HTTP 500 from https://provider.example/v1?key=<redacted>",
|
||||
"type": "LLMAPIError",
|
||||
"provider_status": 500
|
||||
}
|
||||
```
|
||||
|
||||
Known error codes include `unknown_profile`, `configuration_error`,
|
||||
`provider_api_error`, `provider_rate_limited`, `provider_timeout`,
|
||||
`budget_exceeded`, `llm_error`, and `internal_error`.
|
||||
|
||||
## Runtime profiles
|
||||
|
||||
Server CLI mode wraps the configured adapter with runtime profile dispatch
|
||||
unless `--disable-profiles` is passed. The activity-core profile
|
||||
`custodian-triage-balanced` is built in and resolves to the configured provider
|
||||
and model before calling the underlying adapter.
|
||||
|
||||
Default profile values:
|
||||
|
||||
| Field | Default |
|
||||
|-------|---------|
|
||||
| provider | `openrouter` |
|
||||
| model | `anthropic/claude-sonnet-4` |
|
||||
| temperature | `0.2` |
|
||||
| max_tokens | `1800` |
|
||||
| max_depth | `2` |
|
||||
| timeout_seconds | `300` |
|
||||
| model_params.reasoning_effort | `medium` |
|
||||
|
||||
Profile provider/model and default call values can be overridden with
|
||||
environment variables such as `LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER`,
|
||||
`LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL`, and
|
||||
`LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_TOKENS`. Operators can also set
|
||||
`LLM_CONNECT_PROFILES_JSON` or `LLM_CONNECT_PROFILE_FILE` to provide JSON
|
||||
profile definitions keyed by profile name.
|
||||
|
||||
## Implementation notes
|
||||
|
||||
@@ -75,10 +119,12 @@ Execute a prompt through the configured adapter.
|
||||
## CLI
|
||||
|
||||
```
|
||||
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL]
|
||||
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL] [--disable-profiles] [--strict-profiles]
|
||||
```
|
||||
|
||||
Default provider: `mock`. All registered providers from `create_adapter` are valid.
|
||||
CLI defaults can also be supplied with `LLM_CONNECT_HOST`, `LLM_CONNECT_PORT`,
|
||||
`LLM_CONNECT_PROVIDER`, and `LLM_CONNECT_MODEL`. Default provider: `mock`. All
|
||||
registered providers from `create_adapter` are valid.
|
||||
|
||||
## Known consumers
|
||||
|
||||
|
||||
Reference in New Issue
Block a user