ops-warden/history/2026-06-17-openbao-production-verify.md

# OpenBao Production Verification — 2026-06-17

**Workplan:** WARDEN-WP-0007-T01  
**Endpoint:** `https://bao.coulomb.social`  
**Operator:** codex (automated probe, no secrets recorded)

---

## Health probe

```bash
curl -s "https://bao.coulomb.social/v1/sys/health" | python3 -m json.tool
```

**Result (2026-06-17):**

| Field | Value |
| --- | --- |
| `initialized` | `true` |
| `sealed` | `false` |
| `standby` | `false` |
| `version` | `2.5.4` |
| `cluster_name` | `vault-cluster-ebe7da39` |
| `replication_performance_mode` | `primary` |

OpenBao is **reachable, initialized, and unsealed**. Suitable as the production
platform secrets endpoint for ops-warden `backend: vault`.

---

## Authenticated API (blocked without token)

```bash
curl -s -o /dev/null -w "%{http_code}" "https://bao.coulomb.social/v1/sys/mounts"
```

**Result:** HTTP `403` (expected without `X-Vault-Token`).

Full SSH engine verification (`bao secrets list`, role TTL alignment, live
`warden sign`) requires a **scoped operator token** with permission to:

1. List mounts and confirm `ssh/` engine is enabled
2. Read `ssh/roles/{adm,agt,atm}-role` TTL limits
3. Call `POST /v1/ssh/sign/<role>` for each actor type

See `wiki/OpenBaoSshEngineChecklist.md` for the step-by-step checklist.

---

## Operator session (2026-06-17) — WP-0008 T2

| Check | Result |
| --- | --- |
| `warden.yaml` + `inventory.yaml` on workstation | Done (operator) |
| Test keypair `agt-state-hub-bridge_ed25519` | Done (operator) |
| OpenBao UI login | `netkingdom` / `platform-admin` — OK |
| **`ssh/` secrets engine** | **Not enabled** — confirmed by operator |
| Legacy SSH | Predates OpenBao and ops-warden (file/static-key era) |

**Conclusion:** T2 cannot complete until the OpenBao SSH engine is bootstrapped
and host trust is planned (see migration paths below). Token and warden config
are not the blocker.

---

## Blockers for end-to-end `warden sign`

| Blocker | Owner | Status |
| --- | --- | --- |
| SSH secrets engine not mounted | `railiance-platform` / operator | **Confirmed missing** |
| Host `TrustedUserCAKeys` for OpenBao SSH CA | `railiance-infra` | Not started (legacy CA on hosts today) |
| Workstation `warden.yaml` | Operator | Done |
| Scoped `VAULT_TOKEN` in shell | Operator | UI login OK; CLI `bao login` still needed for `warden` |
| flex-auth `ssh-certificate` policies | `flex-auth` | Future (T5) |

---

## Migration paths (legacy SSH → OpenBao SSH engine)

| Path | When | Host impact |
| --- | --- | --- |
| **A — New OpenBao CA** | Greenfield or willing to rotate trust | OpenBao generates new CA; distribute new `.pub` via `railiance-infra` |
| **B — Dual trust** | Gradual migration | Hosts trust legacy CA **and** OpenBao SSH CA during transition |
| **C — Import legacy CA** | Keep same host trust file | Import existing CA private key into SSH engine (custody ceremony) |
| **D — Defer** | Prove warden only | `backend: local` + legacy `ca_key` until platform ready |

ops-warden signs either way; **hosts only accept certs from CAs they trust**.

---

## NET-WP-0020 T5 artifacts (2026-06-18)

Automation is implemented; live cluster apply is the remaining gate.

| Artifact | Repo | Status |
| --- | --- | --- |
| `openbao/ssh/roles-spec.yaml` | railiance-platform | Ready |
| `openbao/policies/warden-sign.hcl` | railiance-platform | Ready |
| `scripts/openbao-apply-ssh-engine.sh` | railiance-platform | Ready (`--dry-run` OK) |
| `scripts/openbao-verify-ssh-engine.sh` | railiance-platform | Ready |
| `make openbao-configure-ssh` / `openbao-verify-ssh` | railiance-platform | Ready |
| `ansible/roles/ssh_ca_host` + `bootstrap-ssh-ca.yaml` | railiance-infra | Ready |
| `ansible/inventory/ssh_principals.yaml` | railiance-infra | Ready (synced with warden principals) |
| `make bootstrap-ssh-ca` | railiance-infra | Ready |

Live cluster check (2026-06-18): OpenBao initialized and unsealed; `ssh/` mount,
roles, and `warden-sign` policy **not yet applied** (no operator token in session).

---

## Live apply + sign smoke (2026-06-18)

| Step | Result |
| --- | --- |
| `ssh/` engine enabled | Pass |
| Default SSH CA issuer (`ed25519`) | Pass — fingerprint `sha256:23bc9636bdd9109e040028953c14b75668bd72de68b8b8ff08e85513b8ea028f` |
| Roles `adm-role`, `agt-role`, `atm-role` | Pass |
| Policy `warden-sign` | Pass |
| `openbao-verify-ssh` | Pass |
| `bootstrap-ssh-ca` on CoulombCore + Railiance01 | Pass |
| `warden sign agt-state-hub-bridge` | Pass — principal `agt-task-bridge`, TTL 24h, backend `vault` |
| `warden status agt-state-hub-bridge` | Pass — remaining ~26h at sign time |

**Note:** OpenBao 2.5.x requires explicit `ssh/config/ca` issuer generation before
`public_key` export; roles need `allow_user_key_ids=true` for ops-warden `key_id`
embedding. Script fixes committed to `railiance-platform`.

**WP-0008:** closed 2026-06-18 — production sign path verified. flex-auth production
enablement continues in WP-0009.

---

## Recommended next operator steps

1. ~~Create production `warden.yaml`~~ — done on workstation.
2. ~~Apply SSH engine automation~~ — done 2026-06-18.
3. ~~Deploy host CA trust~~ — done on CoulombCore + Railiance01 (path A).
4. ~~`warden sign` smoke test~~ — done; use scoped `warden-sign` tokens for daily work (not root).
5. Enable `policy.enabled: true` only after flex-auth policies exist.
6. Rotate/revoke bootstrap root token if still in shell profile — use OIDC + `warden-sign` tokens.

---

## Cross-repo assessment

Full bootstrap + custody + SSH gap navigation map:
`net-kingdom/history/2026-06-17-openbao-ssh-custody-and-bootstrap-assessment.md`

---

## See also

- `wiki/OpsWardenConfig.md` — production config examples
- `wiki/OpenBaoSshEngineChecklist.md` — SSH engine validation
- `wiki/PolicyGatedSigning.md` — opt-in flex-auth gate (implemented WP-0007)