# Credential Lane Lifecycle Runbook

Status: active (RAILIANCE-WP-0009-T07 / RAILIANCE-WP-0010-T07)
Date: 2026-07-02

Covers deactivation, rotation, and compromise response for the workload KV
lanes established by `CCR-2026-0002` (issue-core) and `CCR-2026-0003`
(llm-connect). The **canonical, always-current procedure** is generated from
the CCR itself — this runbook adds only the lane-specific consumer facts the
generator cannot know.

```bash
scripts/credential-change.py lifecycle-plan <CCR-ID> --action {deactivate|rotate|compromise}
# then execute the rendered steps and record:
scripts/credential-change.py lifecycle-event <CCR-ID> --action <action> \
  --actor <operator> --reason "<non-secret>" --detail "<non-secret>" --record-state-hub
```

All three actions share the same invariants: the front door goes
non-resolvable *first*, OpenBao metadata changes use approved operator or
delegated-applier authority (never `platform-admin` handoffs), audit
evidence is preserved (never delete the audit device or its entries), and no
secret value ever appears in Git, State Hub, chat, prompts, or shell history.

## Lane: issue-core runtime ingestion (`CCR-2026-0002`)

| Item | Value |
| --- | --- |
| KV path | `platform/workloads/issue-core/issue-core/issue-core-runtime` |
| Fields | `ISSUE_CORE_API_KEY`, `GITEA_BACKEND_TOKEN` |
| Policy / auth role | `workload-kv-read-issue-core-runtime` / `auth/kubernetes/role/external-secrets-issue-core` |
| Primary consumer | ExternalSecret `issue-core/issue-core-runtime` (CoulombCore cluster, 1h refresh) |
| ops-warden catalog | `issue-core-ingestion-api-key` |

**Consumer facts the generated plan does not cover:**

- Deactivating the policy/role stops the ExternalSecret from *refreshing*,
  but the materialized Kubernetes Secret **persists** with the last value —
  a real deactivation or compromise response must also delete
  `secret/issue-core-runtime` in the `issue-core` namespace (ESO will not
  recreate it while the lane is down) and restart the issue-core Deployment.
- **`ISSUE_CORE_API_KEY` has a second consumer**: railiance01's
  `activity-core/actcore-runtime-secret` holds an operator-injected copy
  (2026-07-02, ISSUE-WP-0003-T06). Rotation and compromise response MUST
  re-inject the new value there (stdin-only pipe from OpenBao) and restart
  `deploy/actcore-worker`, or activity-core emission silently starts failing
  with 401s on the next run.
- `GITEA_BACKEND_TOKEN` is a scoped Gitea token for service user
  `issue-core-svc`; rotating it means minting a new token in Gitea first,
  then updating OpenBao — order matters, or ingestion breaks between steps.

## Lane: llm-connect OpenRouter provider key (`CCR-2026-0003`)

| Item | Value |
| --- | --- |
| KV path | `platform/workloads/activity-core/llm-connect/llm-connect-provider-secrets` |
| Field | `OPENROUTER_API_KEY` |
| Policy / auth role | `workload-kv-read-llm-connect-provider-secrets` / `auth/kubernetes/role/external-secrets-activity-core` |
| Primary consumer | ExternalSecret `activity-core/llm-connect-provider-secrets` (CoulombCore cluster, 1h refresh) |
| ops-warden catalog | `openrouter-llm-connect` |

**Consumer facts the generated plan does not cover:**

- llm-connect consumes the Secret via `envFrom`, so a rotated value reaches
  the runtime only after `kubectl -n activity-core rollout restart
  deploy/llm-connect` (CoulombCore). Wait for the ExternalSecret refresh (or
  `force-sync` annotate) *before* restarting.
- **The railiance01 llm-connect instance is out of scope of this lane**: it
  uses a bootstrap-provisioned Secret from
  `activity-core/k8s/railiance/bootstrap-secrets.sh`. Rotating the OpenRouter
  key upstream (at OpenRouter) invalidates *both* copies — a provider-side
  rotation therefore always requires the railiance01 manual update too, or
  the daily triage runs start failing with provider auth errors.
- Compromise response for a provider key has an extra step the plan cannot
  render: **revoke the key at OpenRouter itself** (provider console) before
  or immediately after disabling the front door; OpenBao custody actions
  alone do not stop a leaked provider key from working.

## Verification after rotate

Return the lane to `active` only with fresh positive + negative evidence,
same shape as activation (2026-07-02 precedent):

- positive: ExternalSecret `SecretSynced=True` with a new refresh timestamp,
  consumer pod healthy after restart;
- negative: a `default`-policy token denied on the KV data path, matched in
  the file audit device by path and timestamp;
- record via `lifecycle-event ... --record-state-hub` and notify ops-warden
  to flip the catalog entry back to active.