generated from coulomb/repo-seed
Implement llm-connect ADHOC diagnostics
This commit is contained in:
38
README.md
38
README.md
@@ -73,15 +73,15 @@ config = RunConfig(
|
||||
)
|
||||
```
|
||||
|
||||
| Field | Default | Description |
|
||||
|---|---|---|
|
||||
| `model_name` | `"gpt-4"` | Model identifier (adapter may override) |
|
||||
| `temperature` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `2000` | Maximum output tokens |
|
||||
| `model_params` | `{}` | Extra provider-specific parameters |
|
||||
| `max_depth` | `3` | Max nesting depth for recursive calls |
|
||||
| `skip_if_exists` | `True` | Skip if identical input hash already processed |
|
||||
| `timeout_seconds` | `300` | Request timeout |
|
||||
| Field | Default | Description |
|
||||
|---|---|---|
|
||||
| `model_name` | `"gpt-4"` | Model identifier (adapter may override) |
|
||||
| `temperature` | `0.7` | Sampling temperature |
|
||||
| `max_tokens` | `2000` | Maximum output tokens |
|
||||
| `model_params` | `{}` | Portable extras translated by each adapter; see `docs/adapter-model-params.md` |
|
||||
| `max_depth` | `3` | Max nesting depth for recursive calls |
|
||||
| `skip_if_exists` | `True` | Skip if identical input hash already processed |
|
||||
| `timeout_seconds` | `300` | Request timeout |
|
||||
|
||||
### `LLMResponse`
|
||||
|
||||
@@ -92,8 +92,24 @@ response = adapter.execute_prompt(prompt, config)
|
||||
print(response.content) # generated text
|
||||
print(response.model) # model actually used
|
||||
print(response.usage) # {"prompt_tokens": …, "completion_tokens": …, "total_tokens": …}
|
||||
print(response.finish_reason) # "stop", "length", etc.
|
||||
```
|
||||
print(response.finish_reason) # "stop", "length", etc.
|
||||
```
|
||||
|
||||
## Server diagnostics
|
||||
|
||||
Serve mode can include a debug envelope without changing normal responses:
|
||||
|
||||
```bash
|
||||
LLM_CONNECT_DEBUG=1 python -m llm_connect.server --provider openrouter
|
||||
curl 'http://127.0.0.1:8080/execute?debug=1' -d '{"prompt":"hi"}'
|
||||
```
|
||||
|
||||
Set `LLM_CONNECT_AUDIT_DIR=/path/to/audit` to write per-call replay records,
|
||||
then parse one without another provider call:
|
||||
|
||||
```bash
|
||||
python -m llm_connect.replay /path/to/audit/record.json --json
|
||||
```
|
||||
|
||||
## Writing your own adapter
|
||||
|
||||
|
||||
Reference in New Issue
Block a user