# Phoenix Machine Drill Runbook (CUST-WP-0054-T08)

Date: 2026-07-08

Composes existing bootstrap pieces into one rehearsable machine-scale phoenix
path. Live wipe/rebuild proof requires an operator-approved disposable target.

## Entry Points

```bash
# Non-destructive prerequisite check
/home/worsch/the-custodian/tools/phoenix-drill.sh check

# Ordered drill plan (no execution)
/home/worsch/the-custodian/tools/phoenix-drill.sh plan
```

## Composed Pieces

| Layer | Source | Command / artifact |
| --- | --- | --- |
| Node bootstrap | `railiance-cluster` | `make bootstrap` → `ansible/bootstrap.yml` |
| Secret custody | `railiance-platform` / NET-WP-0020 | `make openbao-init-unseal` |
| Workload promotion | RAIL-BS-WP-0006 | staged-promotion overlay + `railiance/app.toml` |
| Data restore | `railiance-infra` | `tools/forgejo-restore-drill.sh` |
| Ops evidence | `core-hub` | `make operator-cli CLI_ARGS="deployed-smoke ..."` |
| Hub coordination | `state-hub` | `make dev-hub` for local rebuild from files |

## Drill Sequence

1. **Custody gate** — operator holds unseal/init tokens; no secrets in Git.
2. **Greenfield OS** — ansible bootstrap on disposable target.
3. **OpenBao + identity** — NET-WP-0020 greenfield proof chain.
4. **k3s join** — cluster membership per `railiance-cluster` inventory.
5. **Staged promotion** — one non-critical workload overlay first (glas-harness probe).
6. **Restore drill** — CNPG backup/restore for one database workload.
7. **Fleet mesh** — `atm-` tunnels; verify no workstation relay in path.
8. **Evidence** — readiness-summary JSON, non-secret smoke ids, rollback commands.

## Rollback

Each step records the inverse command before proceeding. Machine phoenix for
coulombcore → railiance02 (T09) uses this same drill after Waves 1–7 complete.

## References

- `docs/workstation-independence-fleet-architecture.md`
- `docs/coulombcore-drain-placement-plan.md`
- `history/2026-07-02-openbao-greenfield-init-unseal-proof.md` (railiance-platform)
- `workplans/CUST-WP-0054-workstation-independence-and-fleet-realignment.md` T08/T09