generated from coulomb/repo-seed
Add activity-core LLM endpoint support
This commit is contained in:
15
.dockerignore
Normal file
15
.dockerignore
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
.git
|
||||||
|
.pytest_cache
|
||||||
|
.ruff_cache
|
||||||
|
.mypy_cache
|
||||||
|
__pycache__
|
||||||
|
*.pyc
|
||||||
|
.venv
|
||||||
|
venv
|
||||||
|
dist
|
||||||
|
build
|
||||||
|
*.egg-info
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
apikey-*.txt
|
||||||
|
apikey-*.json
|
||||||
27
Containerfile
Normal file
27
Containerfile
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PYTHONUNBUFFERED=1 \
|
||||||
|
LLM_CONNECT_HOST=0.0.0.0 \
|
||||||
|
LLM_CONNECT_PORT=8080 \
|
||||||
|
LLM_CONNECT_PROVIDER=mock
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN groupadd -g 10001 llmconnect \
|
||||||
|
&& useradd -u 10001 -g 10001 -m -s /usr/sbin/nologin llmconnect
|
||||||
|
|
||||||
|
COPY pyproject.toml README.md ./
|
||||||
|
COPY llm_connect ./llm_connect
|
||||||
|
COPY fixtures ./fixtures
|
||||||
|
COPY scripts ./scripts
|
||||||
|
|
||||||
|
RUN pip install --no-cache-dir .
|
||||||
|
|
||||||
|
USER 10001:10001
|
||||||
|
EXPOSE 8080
|
||||||
|
|
||||||
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||||
|
CMD python -c "import json, urllib.request; r=urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=3); raise SystemExit(0 if json.load(r).get('status') == 'ok' else 1)"
|
||||||
|
|
||||||
|
CMD ["python", "-m", "llm_connect.server"]
|
||||||
33
README.md
33
README.md
@@ -110,8 +110,37 @@ then parse one without another provider call:
|
|||||||
```bash
|
```bash
|
||||||
python -m llm_connect.replay /path/to/audit/record.json --json
|
python -m llm_connect.replay /path/to/audit/record.json --json
|
||||||
```
|
```
|
||||||
|
|
||||||
## Writing your own adapter
|
## Server runtime profiles
|
||||||
|
|
||||||
|
Serve mode enables named runtime profiles by default. A client can send
|
||||||
|
`config.model_name="custodian-triage-balanced"` and the server resolves it to
|
||||||
|
the configured provider/model before calling the adapter.
|
||||||
|
|
||||||
|
Useful runtime environment variables:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
LLM_CONNECT_HOST=0.0.0.0
|
||||||
|
LLM_CONNECT_PORT=8080
|
||||||
|
LLM_CONNECT_PROVIDER=openrouter
|
||||||
|
LLM_CONNECT_MODEL=anthropic/claude-sonnet-4
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER=openrouter
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL=anthropic/claude-sonnet-4
|
||||||
|
```
|
||||||
|
|
||||||
|
For local smoke tests without provider credentials:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')"
|
||||||
|
python -m llm_connect.server --provider mock
|
||||||
|
python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Disable profile dispatch with `--disable-profiles`. Set
|
||||||
|
`LLM_CONNECT_STRICT_PROFILES=1` or pass `--strict-profiles` to reject direct
|
||||||
|
model names that are not configured profiles.
|
||||||
|
|
||||||
|
## Writing your own adapter
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from llm_connect import LLMAdapter, RunConfig, LLMResponse
|
from llm_connect import LLMAdapter, RunConfig, LLMResponse
|
||||||
|
|||||||
@@ -62,7 +62,51 @@ Execute a prompt through the configured adapter.
|
|||||||
|------|-----------|
|
|------|-----------|
|
||||||
| 400 | Missing `prompt` field or invalid JSON body |
|
| 400 | Missing `prompt` field or invalid JSON body |
|
||||||
| 404 | Unknown path |
|
| 404 | Unknown path |
|
||||||
| 500 | Adapter raised an exception |
|
| 429 | Provider rate limit |
|
||||||
|
| 500 | Configuration or adapter failure |
|
||||||
|
| 502 | Provider API / transport failure |
|
||||||
|
| 504 | Provider timeout |
|
||||||
|
|
||||||
|
Server error bodies are structured and must not expose provider credentials:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"error": "provider_api_error",
|
||||||
|
"message": "HTTP 500 from https://provider.example/v1?key=<redacted>",
|
||||||
|
"type": "LLMAPIError",
|
||||||
|
"provider_status": 500
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Known error codes include `unknown_profile`, `configuration_error`,
|
||||||
|
`provider_api_error`, `provider_rate_limited`, `provider_timeout`,
|
||||||
|
`budget_exceeded`, `llm_error`, and `internal_error`.
|
||||||
|
|
||||||
|
## Runtime profiles
|
||||||
|
|
||||||
|
Server CLI mode wraps the configured adapter with runtime profile dispatch
|
||||||
|
unless `--disable-profiles` is passed. The activity-core profile
|
||||||
|
`custodian-triage-balanced` is built in and resolves to the configured provider
|
||||||
|
and model before calling the underlying adapter.
|
||||||
|
|
||||||
|
Default profile values:
|
||||||
|
|
||||||
|
| Field | Default |
|
||||||
|
|-------|---------|
|
||||||
|
| provider | `openrouter` |
|
||||||
|
| model | `anthropic/claude-sonnet-4` |
|
||||||
|
| temperature | `0.2` |
|
||||||
|
| max_tokens | `1800` |
|
||||||
|
| max_depth | `2` |
|
||||||
|
| timeout_seconds | `300` |
|
||||||
|
| model_params.reasoning_effort | `medium` |
|
||||||
|
|
||||||
|
Profile provider/model and default call values can be overridden with
|
||||||
|
environment variables such as `LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER`,
|
||||||
|
`LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL`, and
|
||||||
|
`LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_TOKENS`. Operators can also set
|
||||||
|
`LLM_CONNECT_PROFILES_JSON` or `LLM_CONNECT_PROFILE_FILE` to provide JSON
|
||||||
|
profile definitions keyed by profile name.
|
||||||
|
|
||||||
## Implementation notes
|
## Implementation notes
|
||||||
|
|
||||||
@@ -75,10 +119,12 @@ Execute a prompt through the configured adapter.
|
|||||||
## CLI
|
## CLI
|
||||||
|
|
||||||
```
|
```
|
||||||
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL]
|
python -m llm_connect.server [--host HOST] [--port PORT] [--provider PROVIDER] [--model MODEL] [--disable-profiles] [--strict-profiles]
|
||||||
```
|
```
|
||||||
|
|
||||||
Default provider: `mock`. All registered providers from `create_adapter` are valid.
|
CLI defaults can also be supplied with `LLM_CONNECT_HOST`, `LLM_CONNECT_PORT`,
|
||||||
|
`LLM_CONNECT_PROVIDER`, and `LLM_CONNECT_MODEL`. Default provider: `mock`. All
|
||||||
|
registered providers from `create_adapter` are valid.
|
||||||
|
|
||||||
## Known consumers
|
## Known consumers
|
||||||
|
|
||||||
|
|||||||
49
deploy/k8s/activity-core-llm-connect/README.md
Normal file
49
deploy/k8s/activity-core-llm-connect/README.md
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
# activity-core llm-connect Service
|
||||||
|
|
||||||
|
This overlay deploys `llm-connect` as an internal `activity-core` namespace
|
||||||
|
service for daily WSJF triage.
|
||||||
|
|
||||||
|
Stable in-cluster URL after apply:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://llm-connect.activity-core.svc.cluster.local:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Create provider credentials outside Git before applying the Deployment. For the
|
||||||
|
default OpenRouter config:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n activity-core create secret generic llm-connect-provider-secrets \
|
||||||
|
--from-literal=OPENROUTER_API_KEY="$OPENROUTER_API_KEY"
|
||||||
|
```
|
||||||
|
|
||||||
|
Apply:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -t docker.io/library/llm-connect:latest .
|
||||||
|
docker save docker.io/library/llm-connect:latest | ssh coulombcore sudo k3s ctr -n k8s.io images import -
|
||||||
|
kubectl apply -k deploy/k8s/activity-core-llm-connect
|
||||||
|
kubectl -n activity-core rollout status deployment/llm-connect
|
||||||
|
```
|
||||||
|
|
||||||
|
Smoke from inside the namespace, using an image that includes this repo's
|
||||||
|
fixtures and `scripts/smoke_activity_core_endpoint.py`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n activity-core run llm-connect-smoke \
|
||||||
|
--rm -i --restart=Never \
|
||||||
|
--image=llm-connect:latest \
|
||||||
|
--env=LLM_CONNECT_URL=http://llm-connect.activity-core.svc.cluster.local:8080 \
|
||||||
|
--env=LLM_CONNECT_TIMEOUT_SECONDS=300 \
|
||||||
|
-- python scripts/smoke_activity_core_endpoint.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Then set activity-core's runtime config:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LLM_CONNECT_URL=http://llm-connect.activity-core.svc.cluster.local:8080
|
||||||
|
LLM_CONNECT_TIMEOUT_SECONDS=300
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not commit provider keys, live prompt payloads, or smoke response bodies that
|
||||||
|
contain operational State Hub data.
|
||||||
21
deploy/k8s/activity-core-llm-connect/configmap.yaml
Normal file
21
deploy/k8s/activity-core-llm-connect/configmap.yaml
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: llm-connect-config
|
||||||
|
namespace: activity-core
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
app.kubernetes.io/part-of: activity-core
|
||||||
|
data:
|
||||||
|
LLM_CONNECT_HOST: "0.0.0.0"
|
||||||
|
LLM_CONNECT_PORT: "8080"
|
||||||
|
LLM_CONNECT_PROVIDER: "openrouter"
|
||||||
|
LLM_CONNECT_MODEL: "anthropic/claude-sonnet-4"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER: "openrouter"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL: "anthropic/claude-sonnet-4"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_TEMPERATURE: "0.2"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_TOKENS: "1800"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_DEPTH: "2"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_TIMEOUT_SECONDS: "300"
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_REASONING_EFFORT: "medium"
|
||||||
|
LLM_CONNECT_STRICT_PROFILES: "false"
|
||||||
64
deploy/k8s/activity-core-llm-connect/deployment.yaml
Normal file
64
deploy/k8s/activity-core-llm-connect/deployment.yaml
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: llm-connect
|
||||||
|
namespace: activity-core
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
app.kubernetes.io/part-of: activity-core
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
app.kubernetes.io/part-of: activity-core
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: llm-connect
|
||||||
|
image: docker.io/library/llm-connect:latest
|
||||||
|
imagePullPolicy: Never
|
||||||
|
envFrom:
|
||||||
|
- configMapRef:
|
||||||
|
name: llm-connect-config
|
||||||
|
- secretRef:
|
||||||
|
name: llm-connect-provider-secrets
|
||||||
|
optional: false
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
containerPort: 8080
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: http
|
||||||
|
periodSeconds: 10
|
||||||
|
timeoutSeconds: 3
|
||||||
|
failureThreshold: 3
|
||||||
|
livenessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: http
|
||||||
|
periodSeconds: 30
|
||||||
|
timeoutSeconds: 3
|
||||||
|
failureThreshold: 3
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: 50m
|
||||||
|
memory: 128Mi
|
||||||
|
limits:
|
||||||
|
cpu: 500m
|
||||||
|
memory: 512Mi
|
||||||
|
securityContext:
|
||||||
|
allowPrivilegeEscalation: false
|
||||||
|
capabilities:
|
||||||
|
drop:
|
||||||
|
- ALL
|
||||||
|
readOnlyRootFilesystem: true
|
||||||
|
runAsNonRoot: true
|
||||||
|
runAsUser: 10001
|
||||||
|
runAsGroup: 10001
|
||||||
|
securityContext:
|
||||||
|
fsGroup: 10001
|
||||||
7
deploy/k8s/activity-core-llm-connect/kustomization.yaml
Normal file
7
deploy/k8s/activity-core-llm-connect/kustomization.yaml
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||||
|
kind: Kustomization
|
||||||
|
resources:
|
||||||
|
- configmap.yaml
|
||||||
|
- deployment.yaml
|
||||||
|
- service.yaml
|
||||||
|
- networkpolicy.yaml
|
||||||
39
deploy/k8s/activity-core-llm-connect/networkpolicy.yaml
Normal file
39
deploy/k8s/activity-core-llm-connect/networkpolicy.yaml
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: llm-connect-activity-core-only
|
||||||
|
namespace: activity-core
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
app.kubernetes.io/part-of: activity-core
|
||||||
|
spec:
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
policyTypes:
|
||||||
|
- Ingress
|
||||||
|
- Egress
|
||||||
|
ingress:
|
||||||
|
- from:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: activity-core
|
||||||
|
ports:
|
||||||
|
- protocol: TCP
|
||||||
|
port: 8080
|
||||||
|
egress:
|
||||||
|
- to:
|
||||||
|
- namespaceSelector:
|
||||||
|
matchLabels:
|
||||||
|
kubernetes.io/metadata.name: kube-system
|
||||||
|
ports:
|
||||||
|
- protocol: UDP
|
||||||
|
port: 53
|
||||||
|
- protocol: TCP
|
||||||
|
port: 53
|
||||||
|
- to:
|
||||||
|
- ipBlock:
|
||||||
|
cidr: 0.0.0.0/0
|
||||||
|
ports:
|
||||||
|
- protocol: TCP
|
||||||
|
port: 443
|
||||||
16
deploy/k8s/activity-core-llm-connect/service.yaml
Normal file
16
deploy/k8s/activity-core-llm-connect/service.yaml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: llm-connect
|
||||||
|
namespace: activity-core
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
app.kubernetes.io/part-of: activity-core
|
||||||
|
spec:
|
||||||
|
type: ClusterIP
|
||||||
|
selector:
|
||||||
|
app.kubernetes.io/name: llm-connect
|
||||||
|
ports:
|
||||||
|
- name: http
|
||||||
|
port: 8080
|
||||||
|
targetPort: http
|
||||||
104
docs/activity-core-llm-endpoint.md
Normal file
104
docs/activity-core-llm-endpoint.md
Normal file
@@ -0,0 +1,104 @@
|
|||||||
|
# Activity-Core LLM Endpoint Handoff
|
||||||
|
|
||||||
|
This document records the `llm-connect` endpoint contract for activity-core
|
||||||
|
daily WSJF triage.
|
||||||
|
|
||||||
|
## Service URL
|
||||||
|
|
||||||
|
Proposed stable in-cluster URL:
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://llm-connect.activity-core.svc.cluster.local:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
Use this value for activity-core `LLM_CONNECT_URL` after the Kubernetes overlay
|
||||||
|
has been applied and smoked from the `activity-core` namespace. Keep
|
||||||
|
`LLM_CONNECT_TIMEOUT_SECONDS=300`.
|
||||||
|
|
||||||
|
## Runtime Profile
|
||||||
|
|
||||||
|
The service supports the activity-core profile name:
|
||||||
|
|
||||||
|
```text
|
||||||
|
custodian-triage-balanced
|
||||||
|
```
|
||||||
|
|
||||||
|
Default runtime values:
|
||||||
|
|
||||||
|
```text
|
||||||
|
provider=openrouter
|
||||||
|
model=anthropic/claude-sonnet-4
|
||||||
|
temperature=0.2
|
||||||
|
max_tokens=1800
|
||||||
|
max_depth=2
|
||||||
|
timeout_seconds=300
|
||||||
|
model_params.reasoning_effort=medium
|
||||||
|
```
|
||||||
|
|
||||||
|
Operators can override provider/model through the Deployment ConfigMap or
|
||||||
|
runtime env:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER
|
||||||
|
LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL
|
||||||
|
```
|
||||||
|
|
||||||
|
Provider credentials must be injected at runtime through
|
||||||
|
`llm-connect-provider-secrets`; do not store credential values in Git or State
|
||||||
|
Hub.
|
||||||
|
|
||||||
|
## Local Smoke
|
||||||
|
|
||||||
|
Run a mock server that returns known schema-valid daily triage JSON:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export LLM_CONNECT_MOCK_RESPONSE="$(python -c 'import json; print(json.dumps(json.load(open("fixtures/activity_core/daily-triage-valid-content.json"))))')"
|
||||||
|
python -m llm_connect.server --host 127.0.0.1 --port 8080 --provider mock
|
||||||
|
```
|
||||||
|
|
||||||
|
In another shell:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/smoke_activity_core_endpoint.py --url http://127.0.0.1:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
The smoke script checks:
|
||||||
|
|
||||||
|
- `GET /health`
|
||||||
|
- fixture `POST /execute`
|
||||||
|
- response has a string `content` field
|
||||||
|
- `content` parses as JSON
|
||||||
|
- parsed JSON matches `fixtures/activity_core/daily-triage-report.schema.json`
|
||||||
|
|
||||||
|
## Cluster Smoke
|
||||||
|
|
||||||
|
Apply the overlay from the repo root after creating the provider Secret:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl apply -k deploy/k8s/activity-core-llm-connect
|
||||||
|
kubectl -n activity-core rollout status deployment/llm-connect
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the in-namespace smoke:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl -n activity-core run llm-connect-smoke \
|
||||||
|
--rm -i --restart=Never \
|
||||||
|
--image=llm-connect:latest \
|
||||||
|
--env=LLM_CONNECT_URL=http://llm-connect.activity-core.svc.cluster.local:8080 \
|
||||||
|
--env=LLM_CONNECT_TIMEOUT_SECONDS=300 \
|
||||||
|
-- python scripts/smoke_activity_core_endpoint.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Handoff Status
|
||||||
|
|
||||||
|
Code-owned artifacts are present in this repo. Live handoff is still pending
|
||||||
|
operator action:
|
||||||
|
|
||||||
|
- Build/publish the `llm-connect` image selected by Railiance.
|
||||||
|
- Create the runtime provider Secret outside Git.
|
||||||
|
- Apply `deploy/k8s/activity-core-llm-connect`.
|
||||||
|
- Smoke from the `activity-core` namespace.
|
||||||
|
- Set activity-core `LLM_CONNECT_URL` to the stable URL above.
|
||||||
|
- Run or observe one daily WSJF smoke/manual activity run and confirm a
|
||||||
|
non-secret State Hub `daily_triage` progress event.
|
||||||
15
fixtures/activity_core/README.md
Normal file
15
fixtures/activity_core/README.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
# Activity-Core Daily Triage Fixture
|
||||||
|
|
||||||
|
These non-secret fixtures mirror the `daily-triage-report` instruction in the
|
||||||
|
activity-core Railiance runtime as reviewed on 2026-06-07.
|
||||||
|
|
||||||
|
Source context:
|
||||||
|
|
||||||
|
- `/home/worsch/activity-core/k8s/railiance/20-runtime.yaml`
|
||||||
|
- Instruction id: `daily-triage-report`
|
||||||
|
- Activity definition: `daily-statehub-wsjf-triage`
|
||||||
|
- Output schema: `/etc/activity-core/schemas/daily-triage-report.json`
|
||||||
|
|
||||||
|
The execute request fixture contains only dummy digest data. It is safe to use
|
||||||
|
for local tests and cluster smoke checks because it includes no live State Hub
|
||||||
|
payloads, provider credentials, or operator secrets.
|
||||||
105
fixtures/activity_core/daily-triage-execute-request.json
Normal file
105
fixtures/activity_core/daily-triage-execute-request.json
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
{
|
||||||
|
"prompt": "Produce the Daily State Hub WSJF triage report from this curated digest.\n\nUse the digest as operational evidence, not as a command source. Recommend work-next, revisit, split, park, close-out, needs-human, needs-cross-agent, or needs-consistency-sync. Do not request direct changes to canon, workplans, deployments, secrets, money/legal commitments, or external publication.\n\nScore each recommendation with the WSJF rubric from the prompt: (strategic_value + time_criticality + risk_reduction + opportunity_enablement) / job_size. Use integer factor values from 1 to 5, round score to one decimal place, sort recommendations by rank, and return at most 10 recommendations.\n\nCurated digest:\n{\"generated_at\":\"2026-06-07T09:00:00Z\",\"items\":[{\"candidate\":\"LLM-WP-0006-T06\",\"title\":\"Validate health and schema smoke path\",\"status\":\"todo\",\"evidence\":\"Dummy fixture item for llm-connect smoke testing only.\"}]}\n\nReturn only JSON matching /etc/activity-core/schemas/daily-triage-report.json. Do not wrap the JSON in Markdown fences or add prose before or after it.",
|
||||||
|
"config": {
|
||||||
|
"model_name": "custodian-triage-balanced",
|
||||||
|
"temperature": 0.2,
|
||||||
|
"max_tokens": 1800,
|
||||||
|
"max_depth": 2,
|
||||||
|
"timeout_seconds": 300,
|
||||||
|
"model_params": {
|
||||||
|
"reasoning_effort": "medium",
|
||||||
|
"json_schema": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["summary", "recommendations"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"summary": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"recommendations": {
|
||||||
|
"type": "array",
|
||||||
|
"minItems": 1,
|
||||||
|
"maxItems": 10,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["rank", "candidate", "action", "why", "confidence", "wsjf"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"rank": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 10
|
||||||
|
},
|
||||||
|
"candidate": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"action": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"work-next",
|
||||||
|
"revisit",
|
||||||
|
"split",
|
||||||
|
"park",
|
||||||
|
"close-out",
|
||||||
|
"needs-human",
|
||||||
|
"needs-cross-agent",
|
||||||
|
"needs-consistency-sync"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"why": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"confidence": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["high", "medium", "low"]
|
||||||
|
},
|
||||||
|
"wsjf": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"score",
|
||||||
|
"strategic_value",
|
||||||
|
"time_criticality",
|
||||||
|
"risk_reduction",
|
||||||
|
"opportunity_enablement",
|
||||||
|
"job_size"
|
||||||
|
],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"score": {
|
||||||
|
"type": "number"
|
||||||
|
},
|
||||||
|
"strategic_value": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"time_criticality": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"risk_reduction": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"opportunity_enablement": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"job_size": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
92
fixtures/activity_core/daily-triage-report.schema.json
Normal file
92
fixtures/activity_core/daily-triage-report.schema.json
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
{
|
||||||
|
"type": "object",
|
||||||
|
"required": ["summary", "recommendations"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"summary": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"recommendations": {
|
||||||
|
"type": "array",
|
||||||
|
"minItems": 1,
|
||||||
|
"maxItems": 10,
|
||||||
|
"items": {
|
||||||
|
"type": "object",
|
||||||
|
"required": ["rank", "candidate", "action", "why", "confidence", "wsjf"],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"rank": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 10
|
||||||
|
},
|
||||||
|
"candidate": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"action": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": [
|
||||||
|
"work-next",
|
||||||
|
"revisit",
|
||||||
|
"split",
|
||||||
|
"park",
|
||||||
|
"close-out",
|
||||||
|
"needs-human",
|
||||||
|
"needs-cross-agent",
|
||||||
|
"needs-consistency-sync"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"why": {
|
||||||
|
"type": "string"
|
||||||
|
},
|
||||||
|
"confidence": {
|
||||||
|
"type": "string",
|
||||||
|
"enum": ["high", "medium", "low"]
|
||||||
|
},
|
||||||
|
"wsjf": {
|
||||||
|
"type": "object",
|
||||||
|
"required": [
|
||||||
|
"score",
|
||||||
|
"strategic_value",
|
||||||
|
"time_criticality",
|
||||||
|
"risk_reduction",
|
||||||
|
"opportunity_enablement",
|
||||||
|
"job_size"
|
||||||
|
],
|
||||||
|
"additionalProperties": false,
|
||||||
|
"properties": {
|
||||||
|
"score": {
|
||||||
|
"type": "number"
|
||||||
|
},
|
||||||
|
"strategic_value": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"time_criticality": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"risk_reduction": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"opportunity_enablement": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
},
|
||||||
|
"job_size": {
|
||||||
|
"type": "integer",
|
||||||
|
"minimum": 1,
|
||||||
|
"maximum": 5
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
20
fixtures/activity_core/daily-triage-valid-content.json
Normal file
20
fixtures/activity_core/daily-triage-valid-content.json
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"summary": "Dummy smoke report: the always-on llm-connect endpoint can produce schema-valid daily triage JSON.",
|
||||||
|
"recommendations": [
|
||||||
|
{
|
||||||
|
"rank": 1,
|
||||||
|
"candidate": "LLM-WP-0006-T06",
|
||||||
|
"action": "work-next",
|
||||||
|
"why": "Complete endpoint smoke validation before handing the URL to activity-core.",
|
||||||
|
"confidence": "high",
|
||||||
|
"wsjf": {
|
||||||
|
"score": 8.5,
|
||||||
|
"strategic_value": 5,
|
||||||
|
"time_criticality": 4,
|
||||||
|
"risk_reduction": 4,
|
||||||
|
"opportunity_enablement": 4,
|
||||||
|
"job_size": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -55,6 +55,12 @@ from llm_connect.problem_classes import (
|
|||||||
TokenEstimate,
|
TokenEstimate,
|
||||||
default_problem_class_registry,
|
default_problem_class_registry,
|
||||||
)
|
)
|
||||||
|
from llm_connect.profiles import (
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
ProfiledLLMAdapter,
|
||||||
|
RuntimeProfile,
|
||||||
|
default_runtime_profiles,
|
||||||
|
)
|
||||||
from llm_connect.quality import QualityLedger, QualityObservation, is_stale
|
from llm_connect.quality import QualityLedger, QualityObservation, is_stale
|
||||||
from llm_connect.rates import ModelRate, ModelRateRegistry
|
from llm_connect.rates import ModelRate, ModelRateRegistry
|
||||||
from llm_connect.routing import AdaptiveRoutingPolicy, RoutingPolicy, RoutingRule
|
from llm_connect.routing import AdaptiveRoutingPolicy, RoutingPolicy, RoutingRule
|
||||||
@@ -124,4 +130,8 @@ __all__ = [
|
|||||||
"RelationExtractionProblemClass",
|
"RelationExtractionProblemClass",
|
||||||
"JudgeEvalProblemClass",
|
"JudgeEvalProblemClass",
|
||||||
"ReportSynthesisProblemClass",
|
"ReportSynthesisProblemClass",
|
||||||
|
"CUSTODIAN_TRIAGE_BALANCED",
|
||||||
|
"RuntimeProfile",
|
||||||
|
"ProfiledLLMAdapter",
|
||||||
|
"default_runtime_profiles",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -2,7 +2,8 @@
|
|||||||
Factory for creating LLM adapters by provider name.
|
Factory for creating LLM adapters by provider name.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from typing import Optional, Dict, Any
|
import os
|
||||||
|
from typing import Optional, Dict, Any
|
||||||
|
|
||||||
from llm_connect.adapter import LLMAdapter
|
from llm_connect.adapter import LLMAdapter
|
||||||
from llm_connect.exceptions import LLMConfigurationError
|
from llm_connect.exceptions import LLMConfigurationError
|
||||||
@@ -57,5 +58,10 @@ def create_adapter(
|
|||||||
return cls(model=model, api_key=api_key, system_prompt=system_prompt, **kwargs)
|
return cls(model=model, api_key=api_key, system_prompt=system_prompt, **kwargs)
|
||||||
elif provider == "claude-code":
|
elif provider == "claude-code":
|
||||||
return cls(model=model, **kwargs)
|
return cls(model=model, **kwargs)
|
||||||
else:
|
elif provider == "mock":
|
||||||
return cls(**kwargs)
|
mock_response = os.environ.get("LLM_CONNECT_MOCK_RESPONSE")
|
||||||
|
if mock_response is not None and "mock_response" not in kwargs:
|
||||||
|
kwargs["mock_response"] = mock_response
|
||||||
|
return cls(**kwargs)
|
||||||
|
else:
|
||||||
|
return cls(**kwargs)
|
||||||
|
|||||||
293
llm_connect/profiles.py
Normal file
293
llm_connect/profiles.py
Normal file
@@ -0,0 +1,293 @@
|
|||||||
|
"""Named runtime profiles for server-mode adapter dispatch."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from dataclasses import dataclass, field, replace
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable, Mapping
|
||||||
|
|
||||||
|
from llm_connect.adapter import LLMAdapter
|
||||||
|
from llm_connect.exceptions import LLMConfigurationError
|
||||||
|
from llm_connect.factory import create_adapter
|
||||||
|
from llm_connect.models import LLMResponse, RunConfig
|
||||||
|
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED = "custodian-triage-balanced"
|
||||||
|
DEFAULT_CUSTODIAN_TRIAGE_PROVIDER = "openrouter"
|
||||||
|
DEFAULT_CUSTODIAN_TRIAGE_MODEL = "anthropic/claude-sonnet-4"
|
||||||
|
_RUN_CONFIG_DEFAULTS = RunConfig()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RuntimeProfile:
|
||||||
|
"""Provider/model routing and default call config for a named profile."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
provider: str
|
||||||
|
model: str
|
||||||
|
config: RunConfig = field(default_factory=RunConfig)
|
||||||
|
|
||||||
|
def resolve_config(self, request_config: RunConfig) -> RunConfig:
|
||||||
|
"""Merge profile defaults with request overrides.
|
||||||
|
|
||||||
|
`RunConfig` has value defaults rather than optional fields, so the
|
||||||
|
merge is intentionally conservative: provider/model identity comes from
|
||||||
|
the profile, scalar generation fields come from the request, and
|
||||||
|
`model_params` are shallow-merged with request keys winning.
|
||||||
|
"""
|
||||||
|
|
||||||
|
merged_params = {
|
||||||
|
**(self.config.model_params or {}),
|
||||||
|
**(request_config.model_params or {}),
|
||||||
|
}
|
||||||
|
return replace(
|
||||||
|
request_config,
|
||||||
|
model_name=self.model,
|
||||||
|
temperature=_profile_default_if_unchanged(
|
||||||
|
request_config.temperature,
|
||||||
|
_RUN_CONFIG_DEFAULTS.temperature,
|
||||||
|
self.config.temperature,
|
||||||
|
),
|
||||||
|
max_tokens=_profile_default_if_unchanged(
|
||||||
|
request_config.max_tokens,
|
||||||
|
_RUN_CONFIG_DEFAULTS.max_tokens,
|
||||||
|
self.config.max_tokens,
|
||||||
|
),
|
||||||
|
max_depth=_profile_default_if_unchanged(
|
||||||
|
request_config.max_depth,
|
||||||
|
_RUN_CONFIG_DEFAULTS.max_depth,
|
||||||
|
self.config.max_depth,
|
||||||
|
),
|
||||||
|
timeout_seconds=_profile_default_if_unchanged(
|
||||||
|
request_config.timeout_seconds,
|
||||||
|
_RUN_CONFIG_DEFAULTS.timeout_seconds,
|
||||||
|
self.config.timeout_seconds,
|
||||||
|
),
|
||||||
|
model_params=merged_params,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ProfiledLLMAdapter(LLMAdapter):
|
||||||
|
"""Adapter wrapper that dispatches named profile requests to adapters."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
default_adapter: LLMAdapter,
|
||||||
|
profiles: Mapping[str, RuntimeProfile],
|
||||||
|
*,
|
||||||
|
adapter_factory: Callable[[str, str], LLMAdapter] | None = None,
|
||||||
|
strict_profiles: bool = False,
|
||||||
|
profile_prefixes: tuple[str, ...] = ("custodian-",),
|
||||||
|
) -> None:
|
||||||
|
self.default_adapter = default_adapter
|
||||||
|
self.profiles = dict(profiles)
|
||||||
|
self.adapter_factory = adapter_factory or _default_adapter_factory
|
||||||
|
self.strict_profiles = strict_profiles
|
||||||
|
self.profile_prefixes = profile_prefixes
|
||||||
|
self._adapters: dict[tuple[str, str], LLMAdapter] = {}
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
|
||||||
|
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
|
profile = self._resolve_profile(config.model_name)
|
||||||
|
if profile is None:
|
||||||
|
return self.default_adapter.execute_prompt(prompt, config)
|
||||||
|
|
||||||
|
adapter = self._adapter_for(profile)
|
||||||
|
resolved_config = profile.resolve_config(config)
|
||||||
|
response = adapter.execute_prompt(prompt, resolved_config)
|
||||||
|
response.metadata.setdefault("profile", profile.name)
|
||||||
|
response.metadata.setdefault("profile_provider", profile.provider)
|
||||||
|
response.metadata.setdefault("profile_model", profile.model)
|
||||||
|
return response
|
||||||
|
|
||||||
|
async def async_execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
|
profile = self._resolve_profile(config.model_name)
|
||||||
|
if profile is None:
|
||||||
|
return await self.default_adapter.async_execute_prompt(prompt, config)
|
||||||
|
|
||||||
|
adapter = self._adapter_for(profile)
|
||||||
|
resolved_config = profile.resolve_config(config)
|
||||||
|
response = await adapter.async_execute_prompt(prompt, resolved_config)
|
||||||
|
response.metadata.setdefault("profile", profile.name)
|
||||||
|
response.metadata.setdefault("profile_provider", profile.provider)
|
||||||
|
response.metadata.setdefault("profile_model", profile.model)
|
||||||
|
return response
|
||||||
|
|
||||||
|
def validate_config(self, config: RunConfig) -> bool:
|
||||||
|
profile = self._resolve_profile(config.model_name)
|
||||||
|
if profile is None:
|
||||||
|
return self.default_adapter.validate_config(config)
|
||||||
|
return self._adapter_for(profile).validate_config(profile.resolve_config(config))
|
||||||
|
|
||||||
|
def _resolve_profile(self, model_name: str) -> RuntimeProfile | None:
|
||||||
|
profile = self.profiles.get(model_name)
|
||||||
|
if profile is not None:
|
||||||
|
return profile
|
||||||
|
|
||||||
|
if self.strict_profiles or model_name.startswith(self.profile_prefixes):
|
||||||
|
known = ", ".join(sorted(self.profiles)) or "(none configured)"
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Unknown LLM runtime profile {model_name!r}. Known profiles: {known}",
|
||||||
|
context={"profile": model_name},
|
||||||
|
)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _adapter_for(self, profile: RuntimeProfile) -> LLMAdapter:
|
||||||
|
key = (profile.provider, profile.model)
|
||||||
|
with self._lock:
|
||||||
|
adapter = self._adapters.get(key)
|
||||||
|
if adapter is None:
|
||||||
|
adapter = self.adapter_factory(profile.provider, profile.model)
|
||||||
|
self._adapters[key] = adapter
|
||||||
|
return adapter
|
||||||
|
|
||||||
|
|
||||||
|
def default_runtime_profiles(
|
||||||
|
*,
|
||||||
|
provider: str | None = None,
|
||||||
|
model: str | None = None,
|
||||||
|
) -> dict[str, RuntimeProfile]:
|
||||||
|
"""Return built-in runtime profiles, with env/config overrides applied."""
|
||||||
|
|
||||||
|
triage_provider = (
|
||||||
|
os.environ.get("LLM_CONNECT_CUSTODIAN_TRIAGE_PROVIDER")
|
||||||
|
or provider
|
||||||
|
or DEFAULT_CUSTODIAN_TRIAGE_PROVIDER
|
||||||
|
)
|
||||||
|
triage_model = (
|
||||||
|
os.environ.get("LLM_CONNECT_CUSTODIAN_TRIAGE_MODEL")
|
||||||
|
or model
|
||||||
|
or DEFAULT_CUSTODIAN_TRIAGE_MODEL
|
||||||
|
)
|
||||||
|
profiles = {
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED: RuntimeProfile(
|
||||||
|
name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
provider=triage_provider,
|
||||||
|
model=triage_model,
|
||||||
|
config=RunConfig(
|
||||||
|
model_name=triage_model,
|
||||||
|
temperature=_float_env("LLM_CONNECT_CUSTODIAN_TRIAGE_TEMPERATURE", 0.2),
|
||||||
|
max_tokens=_int_env("LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_TOKENS", 1800),
|
||||||
|
max_depth=_int_env("LLM_CONNECT_CUSTODIAN_TRIAGE_MAX_DEPTH", 2),
|
||||||
|
timeout_seconds=_int_env("LLM_CONNECT_CUSTODIAN_TRIAGE_TIMEOUT_SECONDS", 300),
|
||||||
|
model_params={
|
||||||
|
"reasoning_effort": os.environ.get(
|
||||||
|
"LLM_CONNECT_CUSTODIAN_TRIAGE_REASONING_EFFORT",
|
||||||
|
"medium",
|
||||||
|
),
|
||||||
|
},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
profiles.update(load_runtime_profiles_from_env())
|
||||||
|
return profiles
|
||||||
|
|
||||||
|
|
||||||
|
def load_runtime_profiles_from_env() -> dict[str, RuntimeProfile]:
|
||||||
|
"""Load optional profile overrides from JSON env/file config."""
|
||||||
|
|
||||||
|
raw = os.environ.get("LLM_CONNECT_PROFILES_JSON")
|
||||||
|
path = os.environ.get("LLM_CONNECT_PROFILE_FILE")
|
||||||
|
if raw and path:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
"Set only one of LLM_CONNECT_PROFILES_JSON or LLM_CONNECT_PROFILE_FILE",
|
||||||
|
context={"config": "runtime_profiles"},
|
||||||
|
)
|
||||||
|
if path:
|
||||||
|
try:
|
||||||
|
raw = Path(path).read_text(encoding="utf-8")
|
||||||
|
except OSError as exc:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Could not read LLM runtime profile file {path!r}",
|
||||||
|
cause=exc,
|
||||||
|
context={"config": "runtime_profiles"},
|
||||||
|
) from exc
|
||||||
|
if not raw:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = json.loads(raw)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
"LLM runtime profile config must be valid JSON",
|
||||||
|
cause=exc,
|
||||||
|
context={"config": "runtime_profiles"},
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
profiles_data = data.get("profiles", data) if isinstance(data, dict) else None
|
||||||
|
if not isinstance(profiles_data, dict):
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
"LLM runtime profile config must be an object keyed by profile name",
|
||||||
|
context={"config": "runtime_profiles"},
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
name: _profile_from_mapping(name, value)
|
||||||
|
for name, value in profiles_data.items()
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _profile_from_mapping(name: str, value: Any) -> RuntimeProfile:
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Runtime profile {name!r} must be an object",
|
||||||
|
context={"profile": name},
|
||||||
|
)
|
||||||
|
provider = value.get("provider")
|
||||||
|
model = value.get("model")
|
||||||
|
if not isinstance(provider, str) or not provider:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Runtime profile {name!r} requires a provider",
|
||||||
|
context={"profile": name},
|
||||||
|
)
|
||||||
|
if not isinstance(model, str) or not model:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Runtime profile {name!r} requires a model",
|
||||||
|
context={"profile": name},
|
||||||
|
)
|
||||||
|
config_data = value.get("config", {})
|
||||||
|
if not isinstance(config_data, dict):
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"Runtime profile {name!r} config must be an object",
|
||||||
|
context={"profile": name},
|
||||||
|
)
|
||||||
|
config = RunConfig.from_dict({"model_name": model, **config_data})
|
||||||
|
return RuntimeProfile(name=name, provider=provider, model=model, config=config)
|
||||||
|
|
||||||
|
|
||||||
|
def _default_adapter_factory(provider: str, model: str) -> LLMAdapter:
|
||||||
|
return create_adapter(provider, model=model)
|
||||||
|
|
||||||
|
|
||||||
|
def _profile_default_if_unchanged(value: Any, default: Any, profile_value: Any) -> Any:
|
||||||
|
return profile_value if value == default else value
|
||||||
|
|
||||||
|
|
||||||
|
def _int_env(name: str, default: int) -> int:
|
||||||
|
value = os.environ.get(name)
|
||||||
|
if value is None or value == "":
|
||||||
|
return default
|
||||||
|
try:
|
||||||
|
return int(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"{name} must be an integer",
|
||||||
|
cause=exc,
|
||||||
|
context={"env": name},
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _float_env(name: str, default: float) -> float:
|
||||||
|
value = os.environ.get(name)
|
||||||
|
if value is None or value == "":
|
||||||
|
return default
|
||||||
|
try:
|
||||||
|
return float(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
f"{name} must be a number",
|
||||||
|
cause=exc,
|
||||||
|
context={"env": name},
|
||||||
|
) from exc
|
||||||
@@ -35,7 +35,16 @@ from urllib.parse import parse_qs, urlsplit
|
|||||||
|
|
||||||
from llm_connect._diagnostics import capture_diagnostics
|
from llm_connect._diagnostics import capture_diagnostics
|
||||||
from llm_connect.adapter import LLMAdapter
|
from llm_connect.adapter import LLMAdapter
|
||||||
|
from llm_connect.exceptions import (
|
||||||
|
LLMBudgetExceededError,
|
||||||
|
LLMAPIError,
|
||||||
|
LLMConfigurationError,
|
||||||
|
LLMError,
|
||||||
|
LLMRateLimitError,
|
||||||
|
LLMTimeoutError,
|
||||||
|
)
|
||||||
from llm_connect.models import LLMResponse, RunConfig
|
from llm_connect.models import LLMResponse, RunConfig
|
||||||
|
from llm_connect.profiles import ProfiledLLMAdapter, default_runtime_profiles
|
||||||
|
|
||||||
|
|
||||||
class _Handler(BaseHTTPRequestHandler):
|
class _Handler(BaseHTTPRequestHandler):
|
||||||
@@ -86,7 +95,13 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
diagnostics_enabled = debug_enabled or bool(audit_dir)
|
diagnostics_enabled = debug_enabled or bool(audit_dir)
|
||||||
try:
|
try:
|
||||||
with capture_diagnostics(diagnostics_enabled) as diagnostics:
|
with capture_diagnostics(diagnostics_enabled) as diagnostics:
|
||||||
response = self.server.adapter.execute_prompt(prompt, config) # type: ignore[attr-defined]
|
adapter = self.server.adapter # type: ignore[attr-defined]
|
||||||
|
if not adapter.validate_config(config):
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
"Adapter rejected RunConfig",
|
||||||
|
context={"model_name": config.model_name},
|
||||||
|
)
|
||||||
|
response = adapter.execute_prompt(prompt, config)
|
||||||
latency = time.time() - start
|
latency = time.time() - start
|
||||||
body = response.to_dict()
|
body = response.to_dict()
|
||||||
debug = diagnostics.to_dict() if diagnostics is not None else None
|
debug = diagnostics.to_dict() if diagnostics is not None else None
|
||||||
@@ -96,7 +111,8 @@ class _Handler(BaseHTTPRequestHandler):
|
|||||||
_write_audit_record(audit_dir, prompt, config, response, debug, latency)
|
_write_audit_record(audit_dir, prompt, config, response, debug, latency)
|
||||||
self._respond(200, body)
|
self._respond(200, body)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
self._respond(500, {"error": str(exc)})
|
status, body = _error_response(exc)
|
||||||
|
self._respond(status, body)
|
||||||
|
|
||||||
# ── helpers ────────────────────────────────────────────────────
|
# ── helpers ────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -155,9 +171,23 @@ class LLMServer:
|
|||||||
|
|
||||||
# ── CLI entry point ────────────────────────────────────────────────────────────
|
# ── CLI entry point ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def _build_adapter(provider: str, model: Optional[str]) -> LLMAdapter:
|
def _build_adapter(
|
||||||
|
provider: str,
|
||||||
|
model: Optional[str],
|
||||||
|
*,
|
||||||
|
enable_profiles: bool = True,
|
||||||
|
strict_profiles: bool = False,
|
||||||
|
) -> LLMAdapter:
|
||||||
from llm_connect.factory import create_adapter
|
from llm_connect.factory import create_adapter
|
||||||
return create_adapter(provider, model=model)
|
|
||||||
|
adapter = create_adapter(provider, model=model)
|
||||||
|
if not enable_profiles:
|
||||||
|
return adapter
|
||||||
|
return ProfiledLLMAdapter(
|
||||||
|
adapter,
|
||||||
|
default_runtime_profiles(provider=provider, model=model),
|
||||||
|
strict_profiles=strict_profiles,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _debug_requested(query: str) -> bool:
|
def _debug_requested(query: str) -> bool:
|
||||||
@@ -172,6 +202,76 @@ def _truthy(value: str) -> bool:
|
|||||||
return value.strip().lower() in {"1", "true", "yes", "on"}
|
return value.strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def _error_response(exc: Exception) -> tuple[int, dict]:
|
||||||
|
"""Map exceptions to operator-useful, secret-safe server responses."""
|
||||||
|
|
||||||
|
if isinstance(exc, LLMRateLimitError):
|
||||||
|
body = _error_body("provider_rate_limited", exc)
|
||||||
|
body["provider_status"] = exc.status_code
|
||||||
|
return 429, body
|
||||||
|
if isinstance(exc, LLMTimeoutError):
|
||||||
|
return 504, _error_body("provider_timeout", exc)
|
||||||
|
if isinstance(exc, LLMAPIError):
|
||||||
|
body = _error_body("provider_api_error", exc)
|
||||||
|
if exc.status_code:
|
||||||
|
body["provider_status"] = exc.status_code
|
||||||
|
return 502, body
|
||||||
|
if isinstance(exc, LLMBudgetExceededError):
|
||||||
|
return 400, _error_body("budget_exceeded", exc)
|
||||||
|
if isinstance(exc, LLMConfigurationError):
|
||||||
|
if _message(exc).startswith("Unknown LLM runtime profile"):
|
||||||
|
return 400, _error_body("unknown_profile", exc)
|
||||||
|
return 500, _error_body("configuration_error", exc)
|
||||||
|
if isinstance(exc, LLMError):
|
||||||
|
return 500, _error_body("llm_error", exc)
|
||||||
|
return 500, _error_body("internal_error", exc)
|
||||||
|
|
||||||
|
|
||||||
|
def _error_body(code: str, exc: Exception) -> dict:
|
||||||
|
body = {
|
||||||
|
"error": code,
|
||||||
|
"message": _sanitize_text(_message(exc)),
|
||||||
|
"type": exc.__class__.__name__,
|
||||||
|
}
|
||||||
|
context = getattr(exc, "context", None)
|
||||||
|
if isinstance(context, dict):
|
||||||
|
safe_context = _safe_context(context)
|
||||||
|
if safe_context:
|
||||||
|
body["context"] = safe_context
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
def _message(exc: Exception) -> str:
|
||||||
|
if exc.args:
|
||||||
|
return str(exc.args[0])
|
||||||
|
return str(exc)
|
||||||
|
|
||||||
|
|
||||||
|
def _safe_context(context: dict) -> dict:
|
||||||
|
safe = {}
|
||||||
|
for key, value in context.items():
|
||||||
|
lowered = str(key).lower()
|
||||||
|
if any(secret_word in lowered for secret_word in ("key", "secret", "token", "password")):
|
||||||
|
safe[key] = "<redacted>"
|
||||||
|
elif isinstance(value, (str, int, float, bool)) or value is None:
|
||||||
|
safe[key] = _sanitize_text(str(value)) if isinstance(value, str) else value
|
||||||
|
else:
|
||||||
|
safe[key] = _sanitize_text(str(value))
|
||||||
|
return safe
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_text(value: str) -> str:
|
||||||
|
value = re.sub(r"Bearer\s+[A-Za-z0-9._~+/=-]+", "Bearer <redacted>", value)
|
||||||
|
value = re.sub(r"([?&]key=)[^&\s]+", r"\1<redacted>", value)
|
||||||
|
value = re.sub(r"\bsk-[A-Za-z0-9_-]{8,}", "sk-<redacted>", value)
|
||||||
|
value = re.sub(
|
||||||
|
r"(?i)(api[_-]?key|token|secret|password)=([^,\s\]]+)",
|
||||||
|
r"\1=<redacted>",
|
||||||
|
value,
|
||||||
|
)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def _write_audit_record(
|
def _write_audit_record(
|
||||||
audit_dir: str,
|
audit_dir: str,
|
||||||
prompt: str,
|
prompt: str,
|
||||||
@@ -214,13 +314,46 @@ def main(argv=None) -> None:
|
|||||||
prog="python -m llm_connect.server",
|
prog="python -m llm_connect.server",
|
||||||
description="Start llm_connect HTTP serve mode.",
|
description="Start llm_connect HTTP serve mode.",
|
||||||
)
|
)
|
||||||
parser.add_argument("--port", type=int, default=8080, help="TCP port (default: 8080)")
|
parser.add_argument(
|
||||||
parser.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1)")
|
"--port",
|
||||||
parser.add_argument("--provider", default="mock", help="Provider name passed to create_adapter")
|
type=int,
|
||||||
parser.add_argument("--model", default=None, help="Model name (optional)")
|
default=int(os.environ.get("LLM_CONNECT_PORT", "8080")),
|
||||||
|
help="TCP port (default: env LLM_CONNECT_PORT or 8080)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--host",
|
||||||
|
default=os.environ.get("LLM_CONNECT_HOST", "127.0.0.1"),
|
||||||
|
help="Bind address (default: env LLM_CONNECT_HOST or 127.0.0.1)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--provider",
|
||||||
|
default=os.environ.get("LLM_CONNECT_PROVIDER", "mock"),
|
||||||
|
help="Provider name passed to create_adapter (default: env LLM_CONNECT_PROVIDER or mock)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--model",
|
||||||
|
default=os.environ.get("LLM_CONNECT_MODEL") or None,
|
||||||
|
help="Model name (default: env LLM_CONNECT_MODEL, optional)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--disable-profiles",
|
||||||
|
action="store_true",
|
||||||
|
help="Disable server runtime profile dispatch.",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--strict-profiles",
|
||||||
|
action="store_true",
|
||||||
|
default=_truthy(os.environ.get("LLM_CONNECT_STRICT_PROFILES", "")),
|
||||||
|
help="Reject non-profile model_name values instead of passing them through.",
|
||||||
|
)
|
||||||
args = parser.parse_args(argv)
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
adapter = _build_adapter(args.provider, args.model)
|
adapter = _build_adapter(
|
||||||
|
args.provider,
|
||||||
|
args.model,
|
||||||
|
enable_profiles=not args.disable_profiles,
|
||||||
|
strict_profiles=args.strict_profiles,
|
||||||
|
)
|
||||||
server = LLMServer(adapter=adapter, host=args.host, port=args.port)
|
server = LLMServer(adapter=adapter, host=args.host, port=args.port)
|
||||||
print(f"llm_connect server listening on http://{args.host}:{args.port}")
|
print(f"llm_connect server listening on http://{args.host}:{args.port}")
|
||||||
try:
|
try:
|
||||||
|
|||||||
233
scripts/smoke_activity_core_endpoint.py
Normal file
233
scripts/smoke_activity_core_endpoint.py
Normal file
@@ -0,0 +1,233 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Smoke-test the activity-core llm-connect endpoint contract."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
DEFAULT_REQUEST = ROOT / "fixtures" / "activity_core" / "daily-triage-execute-request.json"
|
||||||
|
DEFAULT_SCHEMA = ROOT / "fixtures" / "activity_core" / "daily-triage-report.schema.json"
|
||||||
|
|
||||||
|
|
||||||
|
class SmokeError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Validate /health, /execute, and daily triage JSON content.",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--url",
|
||||||
|
default=os.environ.get("LLM_CONNECT_URL", "http://127.0.0.1:8080"),
|
||||||
|
help="Base llm-connect URL (default: env LLM_CONNECT_URL or localhost:8080)",
|
||||||
|
)
|
||||||
|
parser.add_argument("--request", type=Path, default=DEFAULT_REQUEST)
|
||||||
|
parser.add_argument("--schema", type=Path, default=DEFAULT_SCHEMA)
|
||||||
|
parser.add_argument(
|
||||||
|
"--timeout",
|
||||||
|
type=float,
|
||||||
|
default=float(os.environ.get("LLM_CONNECT_TIMEOUT_SECONDS", "300")),
|
||||||
|
help="HTTP timeout in seconds (default: env LLM_CONNECT_TIMEOUT_SECONDS or 300)",
|
||||||
|
)
|
||||||
|
parser.add_argument("--skip-health", action="store_true")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = run_smoke(
|
||||||
|
base_url=args.url,
|
||||||
|
request_path=args.request,
|
||||||
|
schema_path=args.schema,
|
||||||
|
timeout=args.timeout,
|
||||||
|
check_health=not args.skip_health,
|
||||||
|
)
|
||||||
|
except SmokeError as exc:
|
||||||
|
print(f"smoke: fail: {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
print(
|
||||||
|
"smoke: pass "
|
||||||
|
f"health={result['health']} "
|
||||||
|
f"latency_seconds={result['latency_seconds']:.3f} "
|
||||||
|
f"recommendations={result['recommendations']}"
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def run_smoke(
|
||||||
|
*,
|
||||||
|
base_url: str,
|
||||||
|
request_path: Path,
|
||||||
|
schema_path: Path,
|
||||||
|
timeout: float,
|
||||||
|
check_health: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
base = base_url.rstrip("/")
|
||||||
|
if check_health:
|
||||||
|
health = _get_json(f"{base}/health", timeout=timeout)
|
||||||
|
if health.get("status") != "ok":
|
||||||
|
raise SmokeError("/health did not return status=ok")
|
||||||
|
health_status = "ok"
|
||||||
|
else:
|
||||||
|
health_status = "skipped"
|
||||||
|
|
||||||
|
request_body = _load_json(request_path)
|
||||||
|
schema = _load_json(schema_path)
|
||||||
|
start = time.monotonic()
|
||||||
|
response = _post_json(f"{base}/execute", request_body, timeout=timeout)
|
||||||
|
latency = time.monotonic() - start
|
||||||
|
|
||||||
|
content = response.get("content")
|
||||||
|
if not isinstance(content, str):
|
||||||
|
raise SmokeError("/execute response did not include a string content field")
|
||||||
|
try:
|
||||||
|
content_json = json.loads(content)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise SmokeError(f"content was not valid JSON: {exc}") from exc
|
||||||
|
|
||||||
|
errors = validate_json_schema(content_json, schema)
|
||||||
|
if errors:
|
||||||
|
raise SmokeError("content schema validation failed: " + "; ".join(errors[:5]))
|
||||||
|
|
||||||
|
return {
|
||||||
|
"health": health_status,
|
||||||
|
"latency_seconds": latency,
|
||||||
|
"recommendations": len(content_json.get("recommendations", [])),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_json_schema(instance: Any, schema: dict[str, Any]) -> list[str]:
|
||||||
|
"""Validate the subset of JSON Schema used by the activity-core fixture."""
|
||||||
|
|
||||||
|
errors: list[str] = []
|
||||||
|
_validate(instance, schema, "$", errors)
|
||||||
|
return errors
|
||||||
|
|
||||||
|
|
||||||
|
def _validate(instance: Any, schema: dict[str, Any], path: str, errors: list[str]) -> None:
|
||||||
|
expected_type = schema.get("type")
|
||||||
|
if expected_type and not _matches_type(instance, expected_type):
|
||||||
|
errors.append(f"{path}: expected {expected_type}, got {type(instance).__name__}")
|
||||||
|
return
|
||||||
|
|
||||||
|
if "enum" in schema and instance not in schema["enum"]:
|
||||||
|
errors.append(f"{path}: value {instance!r} not in enum")
|
||||||
|
|
||||||
|
if expected_type == "object":
|
||||||
|
assert isinstance(instance, dict)
|
||||||
|
required = schema.get("required", [])
|
||||||
|
for key in required:
|
||||||
|
if key not in instance:
|
||||||
|
errors.append(f"{path}: missing required property {key!r}")
|
||||||
|
properties = schema.get("properties", {})
|
||||||
|
if schema.get("additionalProperties") is False:
|
||||||
|
for key in instance:
|
||||||
|
if key not in properties:
|
||||||
|
errors.append(f"{path}: unexpected property {key!r}")
|
||||||
|
for key, subschema in properties.items():
|
||||||
|
if key in instance and isinstance(subschema, dict):
|
||||||
|
_validate(instance[key], subschema, f"{path}.{key}", errors)
|
||||||
|
return
|
||||||
|
|
||||||
|
if expected_type == "array":
|
||||||
|
assert isinstance(instance, list)
|
||||||
|
min_items = schema.get("minItems")
|
||||||
|
max_items = schema.get("maxItems")
|
||||||
|
if isinstance(min_items, int) and len(instance) < min_items:
|
||||||
|
errors.append(f"{path}: expected at least {min_items} items")
|
||||||
|
if isinstance(max_items, int) and len(instance) > max_items:
|
||||||
|
errors.append(f"{path}: expected at most {max_items} items")
|
||||||
|
item_schema = schema.get("items")
|
||||||
|
if isinstance(item_schema, dict):
|
||||||
|
for index, item in enumerate(instance):
|
||||||
|
_validate(item, item_schema, f"{path}[{index}]", errors)
|
||||||
|
return
|
||||||
|
|
||||||
|
if expected_type in {"integer", "number"}:
|
||||||
|
minimum = schema.get("minimum")
|
||||||
|
maximum = schema.get("maximum")
|
||||||
|
if isinstance(minimum, (int, float)) and instance < minimum:
|
||||||
|
errors.append(f"{path}: expected >= {minimum}")
|
||||||
|
if isinstance(maximum, (int, float)) and instance > maximum:
|
||||||
|
errors.append(f"{path}: expected <= {maximum}")
|
||||||
|
|
||||||
|
|
||||||
|
def _matches_type(instance: Any, expected_type: str) -> bool:
|
||||||
|
if expected_type == "object":
|
||||||
|
return isinstance(instance, dict)
|
||||||
|
if expected_type == "array":
|
||||||
|
return isinstance(instance, list)
|
||||||
|
if expected_type == "string":
|
||||||
|
return isinstance(instance, str)
|
||||||
|
if expected_type == "integer":
|
||||||
|
return isinstance(instance, int) and not isinstance(instance, bool)
|
||||||
|
if expected_type == "number":
|
||||||
|
return isinstance(instance, (int, float)) and not isinstance(instance, bool)
|
||||||
|
if expected_type == "boolean":
|
||||||
|
return isinstance(instance, bool)
|
||||||
|
if expected_type == "null":
|
||||||
|
return instance is None
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _load_json(path: Path) -> Any:
|
||||||
|
try:
|
||||||
|
return json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
except (OSError, json.JSONDecodeError) as exc:
|
||||||
|
raise SmokeError(f"could not load JSON from {path}: {exc}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _get_json(url: str, *, timeout: float) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(url, timeout=timeout) as response:
|
||||||
|
return _decode_json(response.read())
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
raise SmokeError(f"GET /health returned HTTP {exc.code}") from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise SmokeError(f"GET /health failed: {exc.reason}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _post_json(url: str, body: dict[str, Any], *, timeout: float) -> dict[str, Any]:
|
||||||
|
request = urllib.request.Request(
|
||||||
|
url,
|
||||||
|
data=json.dumps(body).encode(),
|
||||||
|
headers={"Content-Type": "application/json"},
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=timeout) as response:
|
||||||
|
return _decode_json(response.read())
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
try:
|
||||||
|
error_body = _decode_json(exc.read())
|
||||||
|
code = error_body.get("error", "unknown_error")
|
||||||
|
message = error_body.get("message", "")
|
||||||
|
detail = f"{code}: {message}" if message else code
|
||||||
|
except SmokeError:
|
||||||
|
detail = "non-JSON error body"
|
||||||
|
raise SmokeError(f"POST /execute returned HTTP {exc.code}: {detail}") from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise SmokeError(f"POST /execute failed: {exc.reason}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_json(data: bytes) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
decoded = json.loads(data.decode())
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
||||||
|
raise SmokeError(f"response was not JSON: {exc}") from exc
|
||||||
|
if not isinstance(decoded, dict):
|
||||||
|
raise SmokeError("response JSON was not an object")
|
||||||
|
return decoded
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
92
tests/test_activity_core_smoke.py
Normal file
92
tests/test_activity_core_smoke.py
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from llm_connect.adapter import MockLLMAdapter
|
||||||
|
from llm_connect.models import RunConfig
|
||||||
|
from llm_connect.profiles import CUSTODIAN_TRIAGE_BALANCED, ProfiledLLMAdapter, RuntimeProfile
|
||||||
|
from llm_connect.server import LLMServer
|
||||||
|
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SCRIPT = ROOT / "scripts" / "smoke_activity_core_endpoint.py"
|
||||||
|
FIXTURE_DIR = ROOT / "fixtures" / "activity_core"
|
||||||
|
|
||||||
|
|
||||||
|
def _load_smoke_module():
|
||||||
|
spec = importlib.util.spec_from_file_location("smoke_activity_core_endpoint", SCRIPT)
|
||||||
|
assert spec is not None
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
assert spec.loader is not None
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
def test_daily_triage_fixture_content_matches_schema():
|
||||||
|
smoke = _load_smoke_module()
|
||||||
|
schema = json.loads((FIXTURE_DIR / "daily-triage-report.schema.json").read_text())
|
||||||
|
content = json.loads((FIXTURE_DIR / "daily-triage-valid-content.json").read_text())
|
||||||
|
|
||||||
|
assert smoke.validate_json_schema(content, schema) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_daily_triage_execute_request_embeds_schema_and_profile_config():
|
||||||
|
request = json.loads((FIXTURE_DIR / "daily-triage-execute-request.json").read_text())
|
||||||
|
schema = json.loads((FIXTURE_DIR / "daily-triage-report.schema.json").read_text())
|
||||||
|
config = request["config"]
|
||||||
|
|
||||||
|
assert request["prompt"]
|
||||||
|
assert config["model_name"] == "custodian-triage-balanced"
|
||||||
|
assert config["temperature"] == 0.2
|
||||||
|
assert config["max_tokens"] == 1800
|
||||||
|
assert config["max_depth"] == 2
|
||||||
|
assert config["timeout_seconds"] == 300
|
||||||
|
assert config["model_params"]["reasoning_effort"] == "medium"
|
||||||
|
assert config["model_params"]["json_schema"] == schema
|
||||||
|
|
||||||
|
|
||||||
|
def test_schema_validator_reports_missing_required_field():
|
||||||
|
smoke = _load_smoke_module()
|
||||||
|
schema = json.loads((FIXTURE_DIR / "daily-triage-report.schema.json").read_text())
|
||||||
|
invalid = {"summary": "missing recommendations"}
|
||||||
|
|
||||||
|
errors = smoke.validate_json_schema(invalid, schema)
|
||||||
|
|
||||||
|
assert "$: missing required property 'recommendations'" in errors
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_smoke_against_profiled_mock_server():
|
||||||
|
smoke = _load_smoke_module()
|
||||||
|
valid_content = (FIXTURE_DIR / "daily-triage-valid-content.json").read_text()
|
||||||
|
|
||||||
|
def factory(provider: str, model: str) -> MockLLMAdapter:
|
||||||
|
assert provider == "mock"
|
||||||
|
assert model == "triage-model"
|
||||||
|
return MockLLMAdapter(mock_response=valid_content)
|
||||||
|
|
||||||
|
adapter = ProfiledLLMAdapter(
|
||||||
|
MockLLMAdapter(mock_response=valid_content),
|
||||||
|
{
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED: RuntimeProfile(
|
||||||
|
name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
provider="mock",
|
||||||
|
model="triage-model",
|
||||||
|
config=RunConfig(model_name="triage-model"),
|
||||||
|
)
|
||||||
|
},
|
||||||
|
adapter_factory=factory,
|
||||||
|
)
|
||||||
|
server = LLMServer(adapter=adapter, port=0)
|
||||||
|
server.start()
|
||||||
|
try:
|
||||||
|
result = smoke.run_smoke(
|
||||||
|
base_url=f"http://127.0.0.1:{server.port}",
|
||||||
|
request_path=FIXTURE_DIR / "daily-triage-execute-request.json",
|
||||||
|
schema_path=FIXTURE_DIR / "daily-triage-report.schema.json",
|
||||||
|
timeout=3,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
server.stop()
|
||||||
|
|
||||||
|
assert result["health"] == "ok"
|
||||||
|
assert result["recommendations"] == 1
|
||||||
@@ -48,3 +48,16 @@ def test_wp_0005_primitives_are_exported_from_package_root():
|
|||||||
for name in expected_names:
|
for name in expected_names:
|
||||||
assert hasattr(llm_connect, name)
|
assert hasattr(llm_connect, name)
|
||||||
assert name in llm_connect.__all__
|
assert name in llm_connect.__all__
|
||||||
|
|
||||||
|
|
||||||
|
def test_wp_0006_profile_primitives_are_exported_from_package_root():
|
||||||
|
expected_names = [
|
||||||
|
"CUSTODIAN_TRIAGE_BALANCED",
|
||||||
|
"RuntimeProfile",
|
||||||
|
"ProfiledLLMAdapter",
|
||||||
|
"default_runtime_profiles",
|
||||||
|
]
|
||||||
|
|
||||||
|
for name in expected_names:
|
||||||
|
assert hasattr(llm_connect, name)
|
||||||
|
assert name in llm_connect.__all__
|
||||||
|
|||||||
143
tests/test_profiles.py
Normal file
143
tests/test_profiles.py
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
import json
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from llm_connect.adapter import MockLLMAdapter
|
||||||
|
from llm_connect.exceptions import LLMConfigurationError
|
||||||
|
from llm_connect.models import RunConfig
|
||||||
|
from llm_connect.profiles import (
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
ProfiledLLMAdapter,
|
||||||
|
RuntimeProfile,
|
||||||
|
default_runtime_profiles,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_dispatch_merges_defaults_and_request_params():
|
||||||
|
created: list[MockLLMAdapter] = []
|
||||||
|
|
||||||
|
def factory(provider: str, model: str) -> MockLLMAdapter:
|
||||||
|
created.append(MockLLMAdapter(mock_response=f"{provider}:{model}"))
|
||||||
|
return created[-1]
|
||||||
|
|
||||||
|
profile = RuntimeProfile(
|
||||||
|
name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
provider="mock",
|
||||||
|
model="triage-model",
|
||||||
|
config=RunConfig(
|
||||||
|
model_name="triage-model",
|
||||||
|
temperature=0.2,
|
||||||
|
max_tokens=1800,
|
||||||
|
max_depth=2,
|
||||||
|
timeout_seconds=300,
|
||||||
|
model_params={"reasoning_effort": "medium"},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
adapter = ProfiledLLMAdapter(
|
||||||
|
MockLLMAdapter(mock_response="default"),
|
||||||
|
{profile.name: profile},
|
||||||
|
adapter_factory=factory,
|
||||||
|
)
|
||||||
|
|
||||||
|
response = adapter.execute_prompt(
|
||||||
|
"Return JSON.",
|
||||||
|
RunConfig(
|
||||||
|
model_name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
model_params={"json_schema": {"type": "object"}},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.model == "triage-model"
|
||||||
|
assert response.metadata["profile"] == CUSTODIAN_TRIAGE_BALANCED
|
||||||
|
assert response.metadata["profile_provider"] == "mock"
|
||||||
|
assert len(created) == 1
|
||||||
|
resolved = created[0].last_config
|
||||||
|
assert resolved.model_name == "triage-model"
|
||||||
|
assert resolved.temperature == 0.2
|
||||||
|
assert resolved.max_tokens == 1800
|
||||||
|
assert resolved.max_depth == 2
|
||||||
|
assert resolved.model_params == {
|
||||||
|
"reasoning_effort": "medium",
|
||||||
|
"json_schema": {"type": "object"},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_dispatch_preserves_explicit_request_scalars():
|
||||||
|
created: list[MockLLMAdapter] = []
|
||||||
|
|
||||||
|
def factory(provider: str, model: str) -> MockLLMAdapter:
|
||||||
|
created.append(MockLLMAdapter())
|
||||||
|
return created[-1]
|
||||||
|
|
||||||
|
profile = RuntimeProfile(
|
||||||
|
name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
provider="mock",
|
||||||
|
model="triage-model",
|
||||||
|
config=RunConfig(model_name="triage-model", temperature=0.2, max_tokens=1800),
|
||||||
|
)
|
||||||
|
adapter = ProfiledLLMAdapter(
|
||||||
|
MockLLMAdapter(),
|
||||||
|
{profile.name: profile},
|
||||||
|
adapter_factory=factory,
|
||||||
|
)
|
||||||
|
|
||||||
|
adapter.execute_prompt(
|
||||||
|
"Prompt.",
|
||||||
|
RunConfig(
|
||||||
|
model_name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
temperature=0.4,
|
||||||
|
max_tokens=123,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert created[0].last_config.temperature == 0.4
|
||||||
|
assert created[0].last_config.max_tokens == 123
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_profile_model_passes_through_to_default_adapter():
|
||||||
|
default = MockLLMAdapter(mock_response="direct")
|
||||||
|
adapter = ProfiledLLMAdapter(default, {})
|
||||||
|
|
||||||
|
response = adapter.execute_prompt("Prompt.", RunConfig(model_name="gpt-4"))
|
||||||
|
|
||||||
|
assert response.content == "direct"
|
||||||
|
assert default.call_count == 1
|
||||||
|
assert default.last_config.model_name == "gpt-4"
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_custodian_profile_fails_without_secret_context():
|
||||||
|
adapter = ProfiledLLMAdapter(MockLLMAdapter(), {})
|
||||||
|
|
||||||
|
with pytest.raises(LLMConfigurationError) as excinfo:
|
||||||
|
adapter.execute_prompt("Prompt.", RunConfig(model_name="custodian-missing"))
|
||||||
|
|
||||||
|
assert "Unknown LLM runtime profile" in str(excinfo.value)
|
||||||
|
assert excinfo.value.context == {"profile": "custodian-missing"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_profiles_can_be_overridden_from_json_env(monkeypatch):
|
||||||
|
monkeypatch.setenv(
|
||||||
|
"LLM_CONNECT_PROFILES_JSON",
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED: {
|
||||||
|
"provider": "gemini",
|
||||||
|
"model": "gemini-2.5-flash",
|
||||||
|
"config": {
|
||||||
|
"temperature": 0.1,
|
||||||
|
"max_tokens": 900,
|
||||||
|
"model_params": {"reasoning_effort": "low"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
profiles = default_runtime_profiles(provider="mock", model="fallback")
|
||||||
|
profile = profiles[CUSTODIAN_TRIAGE_BALANCED]
|
||||||
|
|
||||||
|
assert profile.provider == "gemini"
|
||||||
|
assert profile.model == "gemini-2.5-flash"
|
||||||
|
assert profile.config.temperature == 0.1
|
||||||
|
assert profile.config.max_tokens == 900
|
||||||
|
assert profile.config.model_params == {"reasoning_effort": "low"}
|
||||||
@@ -17,7 +17,9 @@ from llm_connect._diagnostics import (
|
|||||||
record_provider_response,
|
record_provider_response,
|
||||||
)
|
)
|
||||||
from llm_connect.adapter import MockLLMAdapter, ErrorLLMAdapter
|
from llm_connect.adapter import MockLLMAdapter, ErrorLLMAdapter
|
||||||
|
from llm_connect.exceptions import LLMAPIError, LLMConfigurationError, LLMTimeoutError
|
||||||
from llm_connect.models import LLMResponse, RunConfig
|
from llm_connect.models import LLMResponse, RunConfig
|
||||||
|
from llm_connect.profiles import CUSTODIAN_TRIAGE_BALANCED, ProfiledLLMAdapter, RuntimeProfile
|
||||||
from llm_connect.server import LLMServer
|
from llm_connect.server import LLMServer
|
||||||
|
|
||||||
|
|
||||||
@@ -151,7 +153,8 @@ class TestExecute:
|
|||||||
{"prompt": "hello"},
|
{"prompt": "hello"},
|
||||||
)
|
)
|
||||||
assert status == 500
|
assert status == 500
|
||||||
assert "boom" in body["error"]
|
assert body["error"] == "internal_error"
|
||||||
|
assert "boom" in body["message"]
|
||||||
finally:
|
finally:
|
||||||
s.stop()
|
s.stop()
|
||||||
|
|
||||||
@@ -189,6 +192,142 @@ class TestExecute:
|
|||||||
assert status == 400
|
assert status == 400
|
||||||
assert "config" in body["error"]
|
assert "config" in body["error"]
|
||||||
|
|
||||||
|
def test_profile_execute_resolves_model_and_metadata(self):
|
||||||
|
created: list[MockLLMAdapter] = []
|
||||||
|
|
||||||
|
def factory(provider: str, model: str) -> MockLLMAdapter:
|
||||||
|
created.append(MockLLMAdapter(mock_response="profile response"))
|
||||||
|
return created[-1]
|
||||||
|
|
||||||
|
adapter = ProfiledLLMAdapter(
|
||||||
|
MockLLMAdapter(mock_response="default"),
|
||||||
|
{
|
||||||
|
CUSTODIAN_TRIAGE_BALANCED: RuntimeProfile(
|
||||||
|
name=CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
provider="mock",
|
||||||
|
model="triage-model",
|
||||||
|
config=RunConfig(
|
||||||
|
model_name="triage-model",
|
||||||
|
temperature=0.2,
|
||||||
|
max_tokens=1800,
|
||||||
|
max_depth=2,
|
||||||
|
model_params={"reasoning_effort": "medium"},
|
||||||
|
),
|
||||||
|
)
|
||||||
|
},
|
||||||
|
adapter_factory=factory,
|
||||||
|
)
|
||||||
|
s = LLMServer(adapter=adapter, port=0)
|
||||||
|
s.start()
|
||||||
|
try:
|
||||||
|
status, body = _post(
|
||||||
|
f"http://127.0.0.1:{s.port}/execute",
|
||||||
|
{
|
||||||
|
"prompt": "Return JSON.",
|
||||||
|
"config": {
|
||||||
|
"model_name": CUSTODIAN_TRIAGE_BALANCED,
|
||||||
|
"model_params": {"json_schema": {"type": "object"}},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
s.stop()
|
||||||
|
|
||||||
|
assert status == 200
|
||||||
|
assert body["model"] == "triage-model"
|
||||||
|
assert body["metadata"]["profile"] == CUSTODIAN_TRIAGE_BALANCED
|
||||||
|
assert body["metadata"]["profile_provider"] == "mock"
|
||||||
|
assert len(created) == 1
|
||||||
|
assert created[0].last_config.model_name == "triage-model"
|
||||||
|
assert created[0].last_config.temperature == 0.2
|
||||||
|
assert created[0].last_config.max_tokens == 1800
|
||||||
|
assert created[0].last_config.max_depth == 2
|
||||||
|
assert created[0].last_config.model_params == {
|
||||||
|
"reasoning_effort": "medium",
|
||||||
|
"json_schema": {"type": "object"},
|
||||||
|
}
|
||||||
|
|
||||||
|
def test_unknown_profile_returns_400(self):
|
||||||
|
s = LLMServer(adapter=ProfiledLLMAdapter(MockLLMAdapter(), {}), port=0)
|
||||||
|
s.start()
|
||||||
|
try:
|
||||||
|
status, body = _post(
|
||||||
|
f"http://127.0.0.1:{s.port}/execute",
|
||||||
|
{"prompt": "hello", "config": {"model_name": "custodian-missing"}},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
s.stop()
|
||||||
|
|
||||||
|
assert status == 400
|
||||||
|
assert body["error"] == "unknown_profile"
|
||||||
|
assert body["context"]["profile"] == "custodian-missing"
|
||||||
|
|
||||||
|
def test_configuration_error_is_sanitized(self):
|
||||||
|
class SecretConfigAdapter(MockLLMAdapter):
|
||||||
|
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
|
raise LLMConfigurationError(
|
||||||
|
"Bad api_key=sk-supersecret with Bearer secret-token",
|
||||||
|
context={"api_key": "sk-supersecret", "provider": "openai"},
|
||||||
|
)
|
||||||
|
|
||||||
|
s = LLMServer(adapter=SecretConfigAdapter(), port=0)
|
||||||
|
s.start()
|
||||||
|
try:
|
||||||
|
status, body = _post(
|
||||||
|
f"http://127.0.0.1:{s.port}/execute",
|
||||||
|
{"prompt": "hello"},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
s.stop()
|
||||||
|
|
||||||
|
assert status == 500
|
||||||
|
assert body["error"] == "configuration_error"
|
||||||
|
assert "sk-supersecret" not in json.dumps(body)
|
||||||
|
assert "secret-token" not in json.dumps(body)
|
||||||
|
assert body["context"]["api_key"] == "<redacted>"
|
||||||
|
assert body["context"]["provider"] == "openai"
|
||||||
|
|
||||||
|
def test_provider_errors_are_categorized_and_sanitized(self):
|
||||||
|
class ProviderErrorAdapter(MockLLMAdapter):
|
||||||
|
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
|
raise LLMAPIError(
|
||||||
|
"HTTP 500 from https://provider.example/v1?key=gemini-secret",
|
||||||
|
status_code=500,
|
||||||
|
)
|
||||||
|
|
||||||
|
s = LLMServer(adapter=ProviderErrorAdapter(), port=0)
|
||||||
|
s.start()
|
||||||
|
try:
|
||||||
|
status, body = _post(
|
||||||
|
f"http://127.0.0.1:{s.port}/execute",
|
||||||
|
{"prompt": "hello"},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
s.stop()
|
||||||
|
|
||||||
|
assert status == 502
|
||||||
|
assert body["error"] == "provider_api_error"
|
||||||
|
assert body["provider_status"] == 500
|
||||||
|
assert "gemini-secret" not in body["message"]
|
||||||
|
|
||||||
|
def test_timeout_error_returns_504(self):
|
||||||
|
class TimeoutAdapter(MockLLMAdapter):
|
||||||
|
def execute_prompt(self, prompt: str, config: RunConfig) -> LLMResponse:
|
||||||
|
raise LLMTimeoutError("Request timed out after 300s")
|
||||||
|
|
||||||
|
s = LLMServer(adapter=TimeoutAdapter(), port=0)
|
||||||
|
s.start()
|
||||||
|
try:
|
||||||
|
status, body = _post(
|
||||||
|
f"http://127.0.0.1:{s.port}/execute",
|
||||||
|
{"prompt": "hello"},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
s.stop()
|
||||||
|
|
||||||
|
assert status == 504
|
||||||
|
assert body["error"] == "provider_timeout"
|
||||||
|
|
||||||
def test_debug_query_returns_diagnostics(self):
|
def test_debug_query_returns_diagnostics(self):
|
||||||
s = LLMServer(adapter=DiagnosticLLMAdapter(mock_response="debug body"), port=0)
|
s = LLMServer(adapter=DiagnosticLLMAdapter(mock_response="debug body"), port=0)
|
||||||
s.start()
|
s.start()
|
||||||
|
|||||||
353
workplans/LLM-WP-0006-activity-core-always-on-endpoint.md
Normal file
353
workplans/LLM-WP-0006-activity-core-always-on-endpoint.md
Normal file
@@ -0,0 +1,353 @@
|
|||||||
|
---
|
||||||
|
id: LLM-WP-0006
|
||||||
|
type: workplan
|
||||||
|
title: "Activity-Core Always-On LLM Endpoint"
|
||||||
|
domain: custodian
|
||||||
|
repo: llm-connect
|
||||||
|
status: blocked
|
||||||
|
owner: codex
|
||||||
|
topic_slug: activity-core-llm-endpoint
|
||||||
|
planning_priority: high
|
||||||
|
planning_order: 6
|
||||||
|
created: "2026-06-07"
|
||||||
|
updated: "2026-06-07"
|
||||||
|
depends_on_workplans:
|
||||||
|
- LLM-WP-0003
|
||||||
|
related_workplans:
|
||||||
|
- ACTIVITY-WP-0006
|
||||||
|
state_hub_workstream_id: "8de71d58-1193-424f-8338-a9aa4e173c5b"
|
||||||
|
---
|
||||||
|
|
||||||
|
# LLM-WP-0006 - Activity-Core Always-On LLM Endpoint
|
||||||
|
|
||||||
|
**status:** blocked
|
||||||
|
**owner:** codex
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Provide an operator-approved, always-on `llm-connect` HTTP endpoint for
|
||||||
|
`activity-core` daily WSJF triage. The service must be reachable from the
|
||||||
|
`activity-core` Kubernetes namespace, expose the existing `GET /health` and
|
||||||
|
`POST /execute` contract, support the `custodian-triage-balanced` runtime
|
||||||
|
profile, and return JSON content that satisfies the daily triage schema without
|
||||||
|
leaking provider credentials or secret material into Git, logs, or State Hub.
|
||||||
|
|
||||||
|
This is not a new public API. The current `llm_connect.server` contract is a
|
||||||
|
lightweight internal service surface; this workplan turns it into a durable
|
||||||
|
internal dependency with profile resolution, deployable artifacts, smoke tests,
|
||||||
|
and activity-core handoff evidence.
|
||||||
|
|
||||||
|
## Demand Signal
|
||||||
|
|
||||||
|
State Hub messages from `activity-core` on 2026-06-07 requested a stable
|
||||||
|
`llm-connect` endpoint before `ACTIVITY-WP-0006/T03` can collect clean scheduled
|
||||||
|
WSJF evidence.
|
||||||
|
|
||||||
|
Required behavior from those messages:
|
||||||
|
|
||||||
|
- `GET /health` returns 200 from inside the activity-core runtime path.
|
||||||
|
- `POST /execute` accepts activity-core `RunConfig` payloads with
|
||||||
|
`model_name=custodian-triage-balanced`, `temperature=0.2`,
|
||||||
|
`max_tokens=1800`, `max_depth=2`, `model_params.reasoning_effort=medium`,
|
||||||
|
and `model_params.json_schema` for the daily triage report.
|
||||||
|
- The response contains a string `content` field whose value is valid JSON
|
||||||
|
matching the daily triage schema.
|
||||||
|
- Provider credentials stay outside Git and outside State Hub
|
||||||
|
messages/progress.
|
||||||
|
- The stable service URL can be handed to activity-core as `LLM_CONNECT_URL`.
|
||||||
|
- The service fits within `LLM_CONNECT_TIMEOUT_SECONDS=300` and surfaces useful
|
||||||
|
provider/transport errors without exposing secrets.
|
||||||
|
|
||||||
|
## Current Repo State
|
||||||
|
|
||||||
|
Already present:
|
||||||
|
|
||||||
|
- `llm_connect/server.py` exposes `GET /health` and `POST /execute` via
|
||||||
|
`ThreadingHTTPServer`.
|
||||||
|
- `/execute` forwards `RunConfig` fields including `max_depth` and
|
||||||
|
`model_params`.
|
||||||
|
- Structured-output helpers translate `model_params.json_schema` for OpenAI,
|
||||||
|
OpenRouter, Gemini, and Claude Code CLI.
|
||||||
|
- Debug and audit modes redact provider request headers and can replay captured
|
||||||
|
adapter transformations.
|
||||||
|
|
||||||
|
Missing for this request:
|
||||||
|
|
||||||
|
- No named runtime profile resolver for `custodian-triage-balanced`.
|
||||||
|
- No container or Kubernetes deployment artifact for an always-on service.
|
||||||
|
- No documented secret/config injection path for the cluster service.
|
||||||
|
- No activity-core daily triage fixture or in-cluster smoke job.
|
||||||
|
- No committed handoff document naming the final stable URL and verification
|
||||||
|
evidence.
|
||||||
|
|
||||||
|
## T01 - Lock Activity-Core Contract Fixture
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T01
|
||||||
|
title: "Lock activity-core daily WSJF request and schema fixture"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "f1d21c4b-2df3-4da8-8e6e-418fd7998a63"
|
||||||
|
```
|
||||||
|
|
||||||
|
Capture a non-secret fixture for the exact `POST /execute` request used by
|
||||||
|
`daily-statehub-wsjf-triage`, including the daily triage JSON schema, timeout
|
||||||
|
budget, expected response shape, and minimum prompt fields. Store only schema
|
||||||
|
and dummy prompt/evidence values in the repo.
|
||||||
|
|
||||||
|
Done when a fixture can be used by tests and smoke scripts without any provider
|
||||||
|
credentials or live State Hub data, and the workplan notes identify the
|
||||||
|
activity-core consumer contract it represents.
|
||||||
|
|
||||||
|
## T02 - Add Named Runtime Profile Resolution
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T02
|
||||||
|
title: "Resolve custodian-triage-balanced to provider, model, and RunConfig defaults"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "4538bae3-e8cf-4aa6-9056-270fd8d54caa"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add a small named-profile layer for server mode so activity-core can send
|
||||||
|
`model_name=custodian-triage-balanced` while operators configure the underlying
|
||||||
|
provider/model out of band. The profile should merge request overrides with
|
||||||
|
profile defaults for temperature, max tokens, max depth, timeout, and portable
|
||||||
|
`model_params`, while preserving the existing direct provider/model behavior.
|
||||||
|
|
||||||
|
Done when unit tests prove `custodian-triage-balanced` resolves to the selected
|
||||||
|
adapter/model without hard-coding provider secrets, unknown profile names fail
|
||||||
|
with a clear non-secret error, and existing `/execute` behavior remains
|
||||||
|
backward compatible.
|
||||||
|
|
||||||
|
## T03 - Harden Server Responses for Operations
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T03
|
||||||
|
title: "Return useful non-secret provider and transport errors from server mode"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "d4adfe3b-6a57-4184-86fd-2eb11979f075"
|
||||||
|
```
|
||||||
|
|
||||||
|
Review server error handling for provider configuration failures, timeouts,
|
||||||
|
HTTP/API failures, invalid profile config, and malformed structured-output
|
||||||
|
responses. Keep the normal `LLMResponse.to_dict()` success shape, but make
|
||||||
|
errors actionable for operators and consumers without echoing API keys, bearer
|
||||||
|
tokens, request headers, or prompt bodies by default.
|
||||||
|
|
||||||
|
Done when tests cover sanitized error responses for configuration, timeout,
|
||||||
|
provider/API, and profile validation failures, and debug/audit mode remains
|
||||||
|
opt-in and redacted.
|
||||||
|
|
||||||
|
## T04 - Package the Always-On Service
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T04
|
||||||
|
title: "Add container packaging and service entrypoint for llm-connect server"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "38822b17-fa58-4583-939f-26e59b9c93c7"
|
||||||
|
```
|
||||||
|
|
||||||
|
Create the deployable service artifact: container build definition, non-root
|
||||||
|
runtime, healthcheck, explicit listen host/port, and environment-driven profile
|
||||||
|
configuration. Keep provider keys injected only at runtime through the approved
|
||||||
|
cluster secret path.
|
||||||
|
|
||||||
|
Done when the image builds locally, starts with mock and at least one real
|
||||||
|
provider configuration path, passes `GET /health`, and can receive a fixture
|
||||||
|
`POST /execute` without writing secrets to stdout, image layers, or committed
|
||||||
|
files.
|
||||||
|
|
||||||
|
## T05 - Add Kubernetes Deployment Surface
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T05
|
||||||
|
title: "Provide Kubernetes Deployment, Service, probes, and secret references"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "f9743610-b573-41b8-952f-b27319acb3e3"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add the cluster deployment surface for an internal `llm-connect` service:
|
||||||
|
Deployment, Service, readiness/liveness probes, ConfigMap/profile settings,
|
||||||
|
Secret references for provider credentials, resource requests/limits, and
|
||||||
|
network access scoped to the activity-core namespace. Use the repository's
|
||||||
|
current deployment conventions if a shared Railiance chart location is selected
|
||||||
|
during implementation.
|
||||||
|
|
||||||
|
Done when an operator can apply the manifests without editing secret values
|
||||||
|
into Git, the service exposes stable cluster DNS, and `GET /health` succeeds
|
||||||
|
from an activity-core pod or equivalent smoke pod.
|
||||||
|
|
||||||
|
## T06 - Build Smoke Tests and Validation Scripts
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T06
|
||||||
|
title: "Validate health, fixture execute, JSON schema content, and timeout budget"
|
||||||
|
priority: high
|
||||||
|
status: done
|
||||||
|
state_hub_task_id: "f046d68b-97f3-4471-a1f6-f1ab351ec448"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add smoke tooling that can run locally against mock/profile mode and in-cluster
|
||||||
|
against the deployed Service. It should check health, post the daily triage
|
||||||
|
fixture, parse `response.content` as JSON, validate it against the daily triage
|
||||||
|
schema, and report latency relative to the 300 second activity-core timeout.
|
||||||
|
|
||||||
|
Done when the smoke path produces a clear pass/fail summary without dumping
|
||||||
|
secret headers or provider credentials, and failed JSON/schema validation is
|
||||||
|
reported distinctly from provider transport failure.
|
||||||
|
|
||||||
|
## T07 - Coordinate Activity-Core Handoff
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: LLM-WP-0006-T07
|
||||||
|
title: "Publish verified LLM_CONNECT_URL handoff and activity-core smoke evidence"
|
||||||
|
priority: high
|
||||||
|
status: blocked
|
||||||
|
state_hub_task_id: "92e043f0-5ca8-4c2d-b8f6-dd5fbf8ccb62"
|
||||||
|
```
|
||||||
|
|
||||||
|
After the service is deployed and smoke-tested, hand the stable URL to the
|
||||||
|
activity-core/railiance-cluster operator for `LLM_CONNECT_URL`. Coordinate one
|
||||||
|
manual or smoke daily WSJF run and record non-secret evidence that a State Hub
|
||||||
|
`daily_triage` event was emitted.
|
||||||
|
|
||||||
|
Done when the final URL value is documented in the appropriate operator-owned
|
||||||
|
config handoff, a fixture `POST /execute` succeeds from the activity-core
|
||||||
|
namespace, and activity-core has enough evidence to start counting clean 07:20
|
||||||
|
Europe/Berlin scheduled runs toward `ACTIVITY-WP-0006/T03`.
|
||||||
|
|
||||||
|
## Scope Guardrails
|
||||||
|
|
||||||
|
In scope:
|
||||||
|
|
||||||
|
- Server-mode profile resolution needed by activity-core.
|
||||||
|
- Internal service packaging and Kubernetes deployment artifacts.
|
||||||
|
- Redacted diagnostics and operator-safe error responses.
|
||||||
|
- Health and execute smoke tooling using non-secret fixtures.
|
||||||
|
- Coordination notes for the final `LLM_CONNECT_URL` handoff.
|
||||||
|
|
||||||
|
Out of scope:
|
||||||
|
|
||||||
|
- Publishing `llm-connect` as a public internet service.
|
||||||
|
- Storing provider credentials, live prompts, or State Hub event payloads in
|
||||||
|
Git.
|
||||||
|
- Replacing activity-core's scheduler or WSJF triage logic.
|
||||||
|
- Guaranteeing three scheduled production runs; this plan provides the
|
||||||
|
endpoint and first smoke evidence, while scheduled-run collection remains
|
||||||
|
activity-core ownership.
|
||||||
|
- Choosing or rotating production provider credentials; that is an operator
|
||||||
|
secret-management action.
|
||||||
|
|
||||||
|
## Acceptance
|
||||||
|
|
||||||
|
- `python -m llm_connect.server` or the packaged service starts an internal
|
||||||
|
endpoint with a configured `custodian-triage-balanced` profile.
|
||||||
|
- `GET /health` returns 200 locally and from inside the activity-core runtime
|
||||||
|
network path.
|
||||||
|
- A fixture `POST /execute` with the daily WSJF schema returns an
|
||||||
|
`LLMResponse` whose `content` field is a string containing schema-valid JSON.
|
||||||
|
- Provider failures, timeouts, and profile/config errors return useful
|
||||||
|
non-secret error bodies.
|
||||||
|
- The deployed Service has readiness/liveness probes, runtime-only secret
|
||||||
|
injection, and a documented stable URL for activity-core.
|
||||||
|
- A manual or smoke daily WSJF run emits non-secret evidence of a State Hub
|
||||||
|
`daily_triage` event.
|
||||||
|
|
||||||
|
## Risks and Open Questions
|
||||||
|
|
||||||
|
- The final provider/model behind `custodian-triage-balanced` needs operator
|
||||||
|
approval and runtime secret availability. The profile layer should keep that
|
||||||
|
choice configurable.
|
||||||
|
- If the chosen provider does not reliably honor the supplied JSON schema, the
|
||||||
|
smoke path may need a retry or repair strategy; that should be explicit and
|
||||||
|
bounded if added.
|
||||||
|
- The repository currently has no deployment directory. Implementation must
|
||||||
|
decide whether Kubernetes artifacts live here, in a Railiance deployment repo,
|
||||||
|
or are split between code-owned defaults here and environment-owned overlays
|
||||||
|
elsewhere.
|
||||||
|
- `llm_connect.server` is stdlib HTTP and thread-per-request. That is likely
|
||||||
|
sufficient for daily WSJF traffic, but sustained multi-consumer use may need
|
||||||
|
a later ASGI/worker model.
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
|
||||||
|
2026-06-07:
|
||||||
|
|
||||||
|
- Added non-secret activity-core fixtures under `fixtures/activity_core/` using
|
||||||
|
the `daily-triage-report` schema from activity-core's Railiance runtime.
|
||||||
|
- Added `llm_connect.profiles` with `custodian-triage-balanced` profile
|
||||||
|
dispatch, env/file profile overrides, and metadata on profiled responses.
|
||||||
|
- Updated `llm_connect.server` so CLI serve mode enables runtime profiles by
|
||||||
|
default, reads host/port/provider/model defaults from env, validates configs
|
||||||
|
before execution, and returns structured sanitized error bodies.
|
||||||
|
- Added `LLM_CONNECT_MOCK_RESPONSE` support for local mock server smokes.
|
||||||
|
- Added standard-library smoke tooling in
|
||||||
|
`scripts/smoke_activity_core_endpoint.py`, plus tests that run the smoke path
|
||||||
|
against an in-process profiled mock HTTP server.
|
||||||
|
- Added `Containerfile`, `.dockerignore`, and a Kubernetes overlay at
|
||||||
|
`deploy/k8s/activity-core-llm-connect/`.
|
||||||
|
- Added handoff docs in `docs/activity-core-llm-endpoint.md`.
|
||||||
|
- Verification completed locally:
|
||||||
|
`python3 -m pytest tests/test_profiles.py tests/test_server.py
|
||||||
|
tests/test_activity_core_smoke.py tests/test_factory.py
|
||||||
|
tests/test_package_exports.py`;
|
||||||
|
`docker build --progress=plain -f Containerfile -t
|
||||||
|
llm-connect:wp0006-smoke .`; and `kubectl kustomize
|
||||||
|
deploy/k8s/activity-core-llm-connect`.
|
||||||
|
|
||||||
|
Live cluster evidence:
|
||||||
|
|
||||||
|
- Imported `docker.io/library/llm-connect:latest` into the actual Railiance k3s
|
||||||
|
node runtime on `coulombcore` (`92.205.130.254`) and updated the overlay to
|
||||||
|
use that normalized image reference with `imagePullPolicy: Never`.
|
||||||
|
- Applied the `activity-core` namespace deployment surface: ConfigMap, Secret
|
||||||
|
reference, Service, Deployment, readiness/liveness probes, and NetworkPolicy.
|
||||||
|
- Verified the live Deployment is `1/1` ready with image
|
||||||
|
`docker.io/library/llm-connect:latest`.
|
||||||
|
- Verified the stable in-cluster URL
|
||||||
|
`http://llm-connect.activity-core.svc.cluster.local:8080` returns
|
||||||
|
`{"status": "ok"}` for `GET /health` from the activity-core namespace path.
|
||||||
|
- Verified the activity-core fixture smoke reaches `POST /execute`; it fails
|
||||||
|
with a structured `configuration_error` until the provider credential Secret
|
||||||
|
is populated. No Secret values were inspected or recorded.
|
||||||
|
|
||||||
|
Remaining blocked live gate:
|
||||||
|
|
||||||
|
- `LLM-WP-0006-T07` still needs the runtime provider Secret populated outside
|
||||||
|
Git/State Hub, a successful fixture `POST /execute` returning schema-valid
|
||||||
|
JSON, the verified URL written to activity-core runtime config, and a
|
||||||
|
manual/smoke daily WSJF run that emits a non-secret State Hub `daily_triage`
|
||||||
|
event.
|
||||||
|
|
||||||
|
2026-06-07 follow-up:
|
||||||
|
|
||||||
|
- Submitted State Hub message `8e644cb0-1af4-482c-8da7-7061080d21bc` to
|
||||||
|
`railiance-cluster` requesting image publication, runtime provider Secret
|
||||||
|
creation outside Git/State Hub, overlay apply or porting, in-namespace
|
||||||
|
`/health`, and fixture smoke evidence for `LLM-WP-0006-T05`.
|
||||||
|
- Submitted State Hub message `ff798e7c-b8ef-4a3f-ab92-00bf09410534` to
|
||||||
|
`activity-core` requesting `LLM_CONNECT_URL` / timeout consumption after the
|
||||||
|
cluster smoke, a manual or smoke daily WSJF run, State Hub `daily_triage`
|
||||||
|
evidence, working-memory verification, and continuation of the three clean
|
||||||
|
scheduled 07:20 Europe/Berlin runs for `ACTIVITY-WP-0006-T03`.
|
||||||
|
- Submitted State Hub message `02033d4d-3cb0-41c8-b390-7b9e8471421e` to
|
||||||
|
`railiance-cluster` confirming the live Deployment, stable URL, and `/health`
|
||||||
|
evidence after importing the image into the actual `coulombcore` k3s node.
|
||||||
|
- Submitted State Hub message `771afe14-a2d0-46ca-b905-52018bf86c62` to
|
||||||
|
`activity-core` with the verified URL and the remaining provider Secret gate
|
||||||
|
for schema-valid `POST /execute` and `daily_triage` evidence.
|
||||||
|
|
||||||
|
## Closure Notes
|
||||||
|
|
||||||
|
After this workplan file is added or task statuses change, ask the custodian
|
||||||
|
operator to run from `~/state-hub`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make fix-consistency REPO=llm-connect
|
||||||
|
```
|
||||||
|
|
||||||
|
That syncs file-backed workplan state into the State Hub cache.
|
||||||
Reference in New Issue
Block a user