llm-connect/workplans/ADHOC-2026-06-02.md

---
id: ADHOC-2026-06-02
type: workplan
title: "Ad hoc — llm-connect lessons from CUST-WP-0045 canary"
domain: custodian
repo: llm-connect
status: finished
owner: custodian
topic_slug: custodian
created: "2026-06-02"
updated: "2026-06-03"
state_hub_workstream_id: "1c936c91-79c7-427d-ab37-9052e8a61cda"
---

# ADHOC-2026-06-02 — llm-connect lessons from CUST-WP-0045 canary

Captured at the close of CUST-WP-0045 T06. The daily-triage canary uncovered
a sequence of bugs that took the better part of a day to diagnose, mostly
because llm-connect has no way to see what the underlying provider returned
or what payload the adapter sent. Five commits landed today:

- `9de0f49` — Claude Code adapter passes `--output-format json` and unwraps
- `435da49` — envelope unwrap prefers JSON-bearing fields over prose
- `cd4551c` — OpenRouter adapter translates `json_schema` → `response_format`
  and drops Claude-specific keys
- `583ab57` — OpenRouter `response_format.json_schema.strict` defaults to `False`
- `1b01f0e` — OpenRouter honours explicit `--model` even when it equals the
  hardcoded default

This adhoc captures the structural improvements that would have collapsed the
diagnosis loop from half a day to minutes. Each task is sized as an opportunistic
improvement; the larger ones explicitly call out when to promote into a
workplan.

## Tasks

### T01 - Debug envelope mode for /execute responses

```task
id: ADHOC-2026-06-02-T01
status: done
priority: medium
state_hub_task_id: "69626e9e-29f1-40f6-8cd2-d38a7e802293"
```

When the `LLM_CONNECT_DEBUG=1` env var is set (or `?debug=1` is added to the
`/execute` request), include the raw provider response and the adapter's
constructed payload alongside the normal `content` field. Today's diagnosis
required two iterations on the Claude Code envelope shape and one on the
OpenRouter adapter routing — each required modifying source, restarting
llm-connect, re-running, observing, modifying again. A debug envelope mode
would have given the diagnostic data in one curl.

Proposed response shape under debug mode:

```json
{
  "content": "<unchanged>",
  "model": "<unchanged>",
  "usage": {...},
  "metadata": {...},
  "debug": {
    "provider_request": {"url": "...", "payload": {...}, "headers_redacted": {...}},
    "provider_response": {"status": 200, "body": {...}},
    "adapter_transformations": [
      {"step": "merge_model_params", "before": {...}, "after": {...}},
      {"step": "unwrap_cli_envelope", "before": "...", "after": "..."}
    ]
  }
}
```

Done when `LLM_CONNECT_DEBUG=1 curl /execute …` returns the raw round-trip in
the response without changing the default response shape, and a test pins the
debug field is omitted in normal mode.

### T02 - Replace stdlib HTTPServer with ThreadingHTTPServer

```task
id: ADHOC-2026-06-02-T02
status: done
priority: low
state_hub_task_id: "e2b1be30-71f7-4497-9b10-b0f24d37beba"
```

`llm_connect/server.py` uses `http.server.HTTPServer`, which is single-threaded.
Concurrent requests queue behind whatever subprocess or HTTPS call is in flight.
That made today's diagnosis worse: a smoke probe sent while the canary's real
LLM call was running just hung until the canary finished. With Claude CLI calls
taking 60-90s each, queuing is painful.

`http.server.ThreadingHTTPServer` is a drop-in replacement (subclass of
`HTTPServer` + `ThreadingMixIn`). One-line change. Document that adapters must
be thread-safe (the subprocess and HTTPS adapters already are).

Done when concurrent `/execute` calls run in parallel and a regression test
asserts two requests with measurable backend latency complete in roughly the
max of their individual latencies, not the sum.

### T03 - Per-call audit log for replay

```task
id: ADHOC-2026-06-02-T03
status: done
priority: medium
state_hub_task_id: "da4821f0-a876-44ce-9dc3-f3fc67732d0f"
```

When `LLM_CONNECT_AUDIT_DIR=/path` is set, record every `/execute` call as a
JSON file named `{timestamp}-{response_id}.json` containing `{prompt, config,
provider_request, provider_response, parsed_content, latency_seconds}`. A
sibling `python -m llm_connect.replay <file>` command re-runs the parser /
unwrapper / validation against the recorded provider response without making a
new provider call.

Saves quota during diagnosis (today we burned several Sonnet calls just to
re-observe the envelope shape) and gives downstream consumers like
activity-core's instruction executor a "what did the model actually return"
artifact when validation fails.

Promote to a workplan if anyone picks it up — schema for the audit record
needs a small design pass (retention, redaction of API keys in
`provider_request.headers`, file size cap), and the CLI side has its own
ergonomics.

### T04 - Apply param-translation contract to OpenAI and Gemini adapters

```task
id: ADHOC-2026-06-02-T04
status: done
priority: medium
state_hub_task_id: "f8a033e6-22ac-4700-b8d2-43a5d76a3751"
```

Today's `cd4551c` added a `_merge_model_params` helper in the OpenRouter
adapter that whitelists OpenAI Chat Completions fields, translates
`json_schema` to `response_format`, and drops Claude/llm-connect-specific
keys. The same naive `payload.update(config.model_params)` is latent in
`llm_connect/openai.py` and `llm_connect/gemini.py` — neither has been
exercised against an activity-core-style payload yet, but they will 400
the moment they are.

Extract `_merge_model_params` to a shared module (`llm_connect/_payload.py`?)
and use it from all three adapters. Gemini's accepted top-level fields differ
(`generationConfig` wrapper, `safetySettings`, etc.), so the helper either
needs adapter-specific whitelists or a per-adapter translation table. The
shape choice is small but real.

Done when a parametrised test sends the activity-core daily-triage
`model_params` shape (`reasoning_effort`, `max_depth`, `json_schema`) through
each adapter and asserts the resulting payload is provider-accepted (no
forbidden top-level fields, schema in the right wrapper).

### T05 - Provider-agnostic structured-output smoke test in CI

```task
id: ADHOC-2026-06-02-T05
status: done
priority: medium
state_hub_task_id: "5d53dbb4-b374-45fe-b81c-ff0b222ca74f"
```

CI today does not exercise the round-trip from `model_params.json_schema`
through any real provider. The `1b01f0e` model-routing bug — where
`--model anthropic/claude-sonnet-4` silently became `gpt-4` because of an
inverted comparison — only surfaced through the canary, because none of the
adapter tests check the actual `payload["model"]` value end-to-end.

Add a test (gated on each provider's API key being present in CI env) that:

1. Starts the server with `--provider {p} --model {m}` for each registered
   adapter.
2. POSTs a fixed prompt with `model_params.json_schema` requiring
   `{summary, recommendations}` (mirroring activity-core's canary shape).
3. Asserts the response parses cleanly against the schema.
4. Asserts `response.model` matches what was configured at startup, not the
   `RunConfig.model_name` default.

Without API keys present, the test mocks the provider HTTP layer and asserts
the *payload* matches expectations. That catches routing/translation bugs
without provider cost.

Done when a smoke probe shaped like today's CUST-WP-0045 canary would have
failed CI on `1b01f0e` (the routing bug) and on `cd4551c` (the translation
bug) before either was merged.

### T06 - Document required model_params translation in adapter README

```task
id: ADHOC-2026-06-02-T06
status: done
priority: low
state_hub_task_id: "33fcb951-d7ab-4d3c-8d67-9eebd986c711"
```

Adapter authors and callers both need to know:

- which `model_params` keys are pass-through to the underlying provider
- which are translated (`json_schema` → `response_format` for OpenAI-shaped
  providers, → tool-use for native Anthropic, etc.)
- which are dropped (Claude-only `reasoning_effort`, llm-connect's
  `max_depth`)
- the strict-mode tradeoff and how to opt back in

Today's bug chain happened partly because the activity-core team assumed
`model_params` was a pass-through and the llm-connect team assumed callers
would only send OpenAI-valid fields. Codify the contract in
`docs/adapter-model-params.md` (new file) or extend the existing
`ARCHITECTURE-LAYERS.md`.

Done when a new adapter author can read the doc and know what their
`_merge_model_params` implementation must support.

## Implementation Notes

Completed on 2026-06-03:

- Added opt-in `/execute` debug envelopes via `LLM_CONNECT_DEBUG=1` or
  `?debug=1`, with redacted provider request/response capture and adapter
  transformation records.
- Switched serve mode to `ThreadingHTTPServer` and added a concurrency
  regression test.
- Added `LLM_CONNECT_AUDIT_DIR` per-call audit records plus
  `python -m llm_connect.replay` for parser/unwrapper replay.
- Extracted shared OpenAI-compatible and Gemini payload translation helpers
  and wired OpenRouter, OpenAI, and Gemini through them.
- Added CI-safe structured-output smoke tests that mock provider HTTP calls
  and assert model routing plus payload shape.
- Documented the adapter `model_params` contract in
  `docs/adapter-model-params.md`.