feat: close WP-0009/WP-0013 production integration stewardship strand

Ship flex-auth policy gate registry and smoke evidence, archive WP-0009 through WP-0013, and add integration docs: ops-bridge cert_command migration playbook, operator OpenBao token hygiene, principals drift check script, and 2026-06-24 INTENT/SCOPE gap analysis.
2026-06-24 12:44:32 +02:00
parent 1778b169da
commit 90007c2cda
24 changed files with 2192 additions and 121 deletions
--- a/wiki/OpsWardenConfig.md
+++ b/wiki/OpsWardenConfig.md
@@ -128,6 +128,9 @@ vault login
 `VAULT_TOKEN`). OpenBao uses the same header; you do not need a separate
 `BAO_TOKEN` unless you configure `token_env` that way.

+See `wiki/playbooks/operator-openbao-token-hygiene.md` for scoped `warden-sign`
+tokens, OIDC routing, and HTTP 403 recovery.
+
 On failure, `warden sign` suggests falling back to `--backend local` only for
 lab recovery — not as a production substitute.

@@ -272,4 +275,5 @@ tunnels:

 `ops-bridge` runs `cert_command` before each SSH launch, captures stdout as the cert,
 and passes it alongside the private key via `ssh -i <key> -i <cert>`.
-See `wiki/CertCommandInterface.md` for the full contract.
+See `wiki/CertCommandInterface.md` for the full contract and
+`wiki/playbooks/ops-bridge-tunnel-cert.md` for static-key → cert_command migration.
--- a/wiki/PolicyGatedSigning.md
+++ b/wiki/PolicyGatedSigning.md
@@ -1,7 +1,7 @@
 # Policy-Gated SSH Signing

-Date: 2026-06-17  
-Status: **implemented (opt-in)** — WARDEN-WP-0007
+Date: 2026-06-23  
+Status: **implemented (opt-in)** — WARDEN-WP-0007; policy package confirmed FLEX-WP-0006

 By default `warden sign` authorizes via **inventory allow-list** and TTL policy
 only. When `policy.enabled: true` in `warden.yaml`, ops-warden calls flex-auth
@@ -104,12 +104,129 @@ defines **what the actor is allowed to request**.

 ---

+## flex-auth policy package (FLEX-WP-0006)
+
+flex-auth owns the `ssh-certificate` / `sign` policy package. ops-warden consumes
+it via `POST /v1/check` when `policy.enabled: true`.
+
+**Handoff (canonical):** `~/flex-auth/docs/ops-warden-policy-gate-handoff.md`
+
+| Asset | flex-auth path |
+| --- | --- |
+| Policy package | `examples/ops-warden/policy_package.md` |
+| Allow/deny fixtures | `examples/ops-warden/policy_fixtures.yaml` |
+| Registry snapshot | `examples/ops-warden/registry_snapshot.json` |
+| Subject manifest | `examples/ops-warden/subject_manifest.yaml` |
+| Resource manifest | `examples/ops-warden/resource_manifest.yaml` |
+
+### Tenant and subject bindings
+
+| Field | Value |
+| --- | --- |
+| Tenant | `tenant:platform` (`policy.tenant`) |
+| Resource system | `ops-warden` (`policy.system`) |
+| Resource type | `ssh-certificate` |
+| Action | `sign` |
+| Resource id | `ssh-cert:actor/<actor-name>` |
+
+| Actor type | Example flex-auth subject | ops-warden inventory name pattern |
+| --- | --- | --- |
+| `adm` | `platform-steward` | `adm-*` |
+| `agt` | `ci-deploy-agent` | `agt-*` |
+| `atm` | `backup-automation` | `atm-*` |
+
+**Subject id sent to flex-auth:** `WARDEN_POLICY_SUBJECT` when set, otherwise the
+inventory actor name. flex-auth may also allow `iam:<actor-name>` when listed in
+`allowed_subjects` on the resource.
+
+**Principals and TTL:** Taken from the sign request (inventory defaults). flex-auth
+denies when principals are empty/disallowed or TTL exceeds `max_ttl_hours` on the
+registered resource.
+
+### Fixture coverage (flex-auth)
+
+Allow: `fixture:ops-warden-adm-sign-allow`, `fixture:ops-warden-agt-sign-allow`,
+`fixture:ops-warden-atm-sign-allow`.
+
+Deny: `fixture:ops-warden-unknown-subject-deny`,
+`fixture:ops-warden-actor-type-mismatch-deny`, `fixture:ops-warden-ttl-above-max-deny`,
+`fixture:ops-warden-disallowed-principal-deny`,
+`fixture:ops-warden-missing-fingerprint-deny`.
+
+### Local smoke
+
+```bash
+# flex-auth (from ~/flex-auth)
+flex-auth serve --addr 127.0.0.1:8080 \
+  --registry examples/ops-warden/registry_snapshot.json \
+  --policy examples/ops-warden/policy_package.md \
+  --log /tmp/flex-auth-ops-warden-decisions.jsonl
+
+# warden.yaml — policy.enabled: true, flex_auth_url pointing at flex-auth
+# Use an actor registered in the flex-auth registry (example fixtures use
+# template names; production needs a registry slice for real inventory actors).
+```
+
+Local end-to-end evidence: `history/2026-06-23-flex-auth-policy-gate-local-smoke.md`.
+
+### Production registry from inventory
+
+Build a flex-auth registry snapshot that mirrors `inventory.yaml` actors:
+
+```bash
+python scripts/build_flex_auth_registry.py ~/.config/warden/inventory.yaml \
+  -o registry/flex-auth/production_registry_snapshot.json
+flex-auth load-registry --file registry/flex-auth/production_registry_snapshot.json
+```
+
+Re-run after adding or changing actors. Deploy the snapshot to the production
+flex-auth runtime together with `~/flex-auth/examples/ops-warden/policy_package.md`.
+
+Smoke (non-secret):
+
+```bash
+./scripts/policy_gate_production_smoke.sh
+# OpenBao-backed when VAULT_TOKEN is valid:
+SMOKE_VAULT=1 ./scripts/policy_gate_production_smoke.sh
+```
+
+Evidence: `history/2026-06-23-flex-auth-policy-gate-production-smoke.md`.
+
+---
+
 ## Production rollout

-1. Deploy flex-auth policies for resource type `ssh-certificate`.
-2. Enable `policy.enabled: true` in production `warden.yaml`.
-3. Keep `fail_closed: true` unless an explicit break-glass procedure exists.
-4. Verify `signatures.log` entries include `policy_decision_id`.
+**Keep `policy.enabled: false` until flex-auth is reachable** at `policy.flex_auth_url`
+with `fail_closed: true`, unreachable flex-auth blocks all signs.
+
+### Operator checklist
+
+| Step | Owner | Action |
+| --- | --- | --- |
+| 1 | flex-auth | Deploy runtime; confirm `curl <flex_auth_url>/healthz` → 200 (**FLEX-WP-0007**) |
+| 2 | flex-auth | Load production registry + policy package (`~/flex-auth/examples/ops-warden/`) |
+| 3 | ops-warden | Regenerate registry from inventory: `scripts/build_flex_auth_registry.py` |
+| 4 | ops-warden | Local smoke: `./scripts/policy_gate_production_smoke.sh` |
+| 5 | operator | Vault smoke: `SMOKE_VAULT=1 ./scripts/policy_gate_production_smoke.sh` (valid `VAULT_TOKEN`) |
+| 6 | operator | Set `policy.flex_auth_url` in `~/.config/warden/warden.yaml` |
+| 7 | operator | Set `policy.enabled: true`; keep `fail_closed: true` |
+| 8 | operator | Allow smoke: `warden sign <actor>` — `signatures.log` has `policy_decision_id` |
+| 9 | operator | Deny smoke: e.g. `--ttl` above max — CLI shows flex-auth `reason`, no cert |
+
+Cross-repo references:
+
+- `~/flex-auth/workplans/FLEX-WP-0007-ops-warden-policy-gate-production-deployment.md`
+- `history/2026-06-23-flex-auth-production-pickup-suggestion.md`
+- `history/2026-06-23-flex-auth-policy-gate-production-smoke.md`
+
+### Summary
+
+1. Deploy the flex-auth registry and policy package to the production flex-auth
+   runtime — **not** only the example fixtures.
+2. Set `policy.flex_auth_url` to the production flex-auth base URL.
+3. Enable `policy.enabled: true` only after steps 1–5 pass.
+4. Keep `fail_closed: true` unless an explicit break-glass procedure exists.
+5. Smoke allow and deny paths; preserve non-secret evidence only.

 ---

@@ -117,5 +234,6 @@ defines **what the actor is allowed to request**.

 - `wiki/OpsWardenConfig.md` — full config reference
 - `wiki/CredentialRouting.md`
+- `~/flex-auth/docs/ops-warden-policy-gate-handoff.md` — flex-auth handoff
 - `flex-auth/INTENT.md`
 - `net-kingdom/docs/platform-identity-security-architecture.md`
--- a/wiki/playbooks/operator-openbao-token-hygiene.md
+++ b/wiki/playbooks/operator-openbao-token-hygiene.md
@@ -0,0 +1,105 @@
+# Operator OpenBao Token Hygiene
+
+Date: 2026-06-24  
+Workplan: WARDEN-WP-0013 T4
+
+Daily `warden sign` against production OpenBao requires a **scoped** API token in
+`VAULT_TOKEN` — not the cluster root token.
+
+---
+
+## Rules
+
+| Rule | Rationale |
+| --- | --- |
+| Never commit `VAULT_TOKEN` | Tokens are secrets |
+| Never paste tokens in chat, State Hub, or workplans | Same |
+| Do not use root token for daily `warden sign` | Break-glass only |
+| Prefer short-lived tokens | Limit blast radius |
+| Refresh on HTTP 403 | Token expired or policy mismatch |
+
+---
+
+## Scoped token for warden
+
+Production signing needs permission to call the SSH engine sign endpoint for the
+roles mapped in `warden.yaml` (`adm-role`, `agt-role`, `atm-role`).
+
+Illustrative policy shape (create in OpenBao policy admin — adjust names to match
+your cluster):
+
+```hcl
+# warden-sign — least privilege for ops-warden CLI
+path "ssh/sign/agt-role" {
+  capabilities = ["create", "update"]
+}
+path "ssh/sign/adm-role" {
+  capabilities = ["create", "update"]
+}
+path "ssh/sign/atm-role" {
+  capabilities = ["create", "update"]
+}
+```
+
+Issue a token bound to `warden-sign` (operator procedure in `railiance-platform` /
+OpenBao admin runbooks).
+
+---
+
+## Session pattern
+
+```bash
+# Set for current shell only — do not add to ~/.bashrc with a literal token
+export VAULT_TOKEN="<scoped-token>"
+
+warden status agt-state-hub-bridge
+warden sign agt-state-hub-bridge --pubkey ~/.ssh/agt-state-hub-bridge_ed25519.pub
+```
+
+`warden` reads the env var named in `vault.token_env` (default `VAULT_TOKEN`).
+
+---
+
+## OIDC / interactive login
+
+For human operators, prefer platform OIDC login that yields a short-lived OpenBao
+token instead of copying long-lived secrets.
+
+| Need | Route to |
+| --- | --- |
+| Interactive login, OIDC, MFA | key-cape / Keycloak — `warden route show key-cape-oidc-login` |
+
+ops-warden does not implement login; it documents the route only.
+
+---
+
+## Troubleshooting
+
+| Symptom | Likely cause | Action |
+| --- | --- | --- |
+| `Vault token not found` | `VAULT_TOKEN` unset | Export scoped token |
+| `HTTP 403` / `permission denied` | Expired token or insufficient policy | Re-issue `warden-sign` token |
+| `Signing failed` + connection error | Wrong `vault.addr` or network | Check `warden.yaml`, tunnel/VPN |
+| Suggest `--backend local` | OpenBao unreachable | Fix connectivity; local is lab-only |
+
+After fixing token issues, re-run:
+
+```bash
+warden sign <actor> --pubkey <path>
+```
+
+---
+
+## Root token (break-glass only)
+
+Cluster root tokens bypass all policy. Use only for one-time engine setup
+(`wiki/OpenBaoSshEngineChecklist.md` § One-time SSH engine setup), then revoke
+from daily shell profile.
+
+---
+
+## See also
+
+- `wiki/OpenBaoSshEngineChecklist.md`
+- `wiki/OpsWardenConfig.md` — Authentication section
+- `examples/warden.production.example.yaml`
--- a/wiki/playbooks/ops-bridge-tunnel-cert.md
+++ b/wiki/playbooks/ops-bridge-tunnel-cert.md
@@ -0,0 +1,121 @@
+# ops-bridge Tunnel — cert_command Migration
+
+Date: 2026-06-24  
+Workplan: WARDEN-WP-0013 T3  
+Catalog: `ops-bridge-tunnel`
+
+Migrate an ops-bridge tunnel from **static SSH keys** to **short-lived warden-signed
+certificates** via the `cert_command` contract (`wiki/CertCommandInterface.md`).
+
+ops-warden documents the migration; **ops-bridge** owns tunnel config changes.
+
+---
+
+## Prerequisites
+
+- [ ] Actor registered in `~/.config/warden/inventory.yaml` (see `wiki/ActorInventoryPatterns.md`)
+- [ ] Actor keypair on disk (`ssh_key` private, `.pub` for signing)
+- [ ] Production `warden.yaml` with `backend: vault` and valid scoped `VAULT_TOKEN`
+- [ ] Host trusts warden/OpenBao CA (`railiance-infra` `bootstrap-ssh-ca`)
+- [ ] Host principal allows the actor's principals (`railiance-infra` `ssh_principals.yaml`)
+
+---
+
+## Pilot tunnel: `agt-state-hub-bridge`
+
+| Field | Value |
+| --- | --- |
+| Actor | `agt-state-hub-bridge` |
+| Type | `agt` |
+| Principals | `agt-task-bridge` |
+| TTL | 24 h |
+| Private key | `~/.ssh/agt-state-hub-bridge_ed25519` |
+| Public key | `~/.ssh/agt-state-hub-bridge_ed25519.pub` |
+| cert_command | `warden sign agt-state-hub-bridge --pubkey ~/.ssh/agt-state-hub-bridge_ed25519.pub` |
+
+### Pre-migration smoke (operator workstation)
+
+```bash
+export VAULT_TOKEN="<scoped-warden-sign-token>"   # never commit or paste in chat
+warden status agt-state-hub-bridge
+warden sign agt-state-hub-bridge --pubkey ~/.ssh/agt-state-hub-bridge_ed25519.pub | head -1
+```
+
+Confirm exit 0 and cert line starts with `ssh-ed25519-cert-v01@openssh.com`.
+
+---
+
+## Migration checklist
+
+### 1. Inventory and signing path
+
+- [ ] Actor exists: `warden inventory list` shows `agt-state-hub-bridge`
+- [ ] `warden sign` succeeds with production OpenBao backend
+- [ ] `signatures.log` records the sign (`~/.local/state/warden/signatures.log`)
+
+### 2. ops-bridge tunnel config
+
+Edit `~/.config/bridge/tunnels.yaml` (ops-bridge repo owns schema; example below):
+
+```yaml
+tunnels:
+  state-hub-coulombcore:
+    host: coulombcore
+    remote_port: 8001
+    local_port: 8000
+    ssh_user: agt-state-hub-bridge
+    ssh_key: ~/.ssh/agt-state-hub-bridge_ed25519
+    actor: agt-state-hub-bridge
+    cert_command: "warden sign agt-state-hub-bridge --pubkey ~/.ssh/agt-state-hub-bridge_ed25519.pub"
+```
+
+- [ ] `cert_command` uses the **public** key path (warden reads pubkey, writes cert to stdout)
+- [ ] `ssh_user` matches the certificate identity / host expectation
+- [ ] Remove or disable static-key-only fallback once cert path is verified
+
+### 3. Host-side verification
+
+- [ ] Principal `agt-task-bridge` present in `railiance-infra` `ssh_principals.yaml` for target host
+- [ ] Run `scripts/check_principals_drift.py` if inventory `hosts` section documents allowed principals
+
+### 4. Tunnel smoke
+
+```bash
+# ops-bridge (from ops-bridge repo)
+bridge status state-hub-coulombcore
+bridge up state-hub-coulombcore
+```
+
+- [ ] Tunnel establishes without static cert file on disk
+- [ ] Re-run `bridge up` after cert TTL expires — `cert_command` re-issues automatically
+
+### 5. Policy gate (optional, after FLEX-WP-0007)
+
+When `policy.enabled: true`, confirm `signatures.log` includes `policy_decision_id`
+on tunnel-driven signs. See `wiki/PolicyGatedSigning.md`.
+
+---
+
+## Rollback
+
+Keep the static key path until cert_command smoke passes. To roll back:
+
+1. Remove `cert_command` from tunnel config
+2. Restore prior static-key or `CertificateFile` workflow
+3. Document rollback in ops-bridge session notes (not in git secrets)
+
+---
+
+## Static-key tunnels (legacy)
+
+Tunnels using `agt-claude-*` or other long-lived keys are **out of scope** for this
+pilot. Migrate per-tunnel when ops-bridge owner prioritizes them.
+
+---
+
+## See also
+
+- `wiki/CertCommandInterface.md`
+- `wiki/OpsWardenConfig.md` — cert_command example
+- `wiki/playbooks/operator-openbao-token-hygiene.md`
+- `warden route show ops-bridge-tunnel --json`