net-kingdom/canon/standards/credential-management_v0.2.md

# Credential Management Standard — net-kingdom
**Version:** 0.2  **Status:** current with OpenBao runtime refinement  **Supersedes:** v0.1 (retired with NK-WP-0004)

---

## 1. Purpose

Define how service credentials are generated, stored, rotated, handed off
to runtime secret authority, and recovered in the net-kingdom SSO/MFA
platform. This standard governs operational security of all secrets used
by the Authelia + LLDAP + KeyCape + privacyIDEA stack, its PostgreSQL
backend, and the OpenBao runtime secret authority used by the platform
control plane.

---

## 2. Trust Hierarchy

```
age private key  ─────────────►  encrypts secrets.enc/ in git
      │
      └──► ops bundle (encrypted snapshot of all secrets)
                └──► stored offsite; decrypt with age key

break-glass passwords  ────────►  direct service access if cluster/auth is down
      │
      └──► stored in human's personal password manager

K8s Secrets  ──────────────────►  bootstrap/delivery state for running services
      │
      └──► bootstrap/delivery mechanism; created by create-secrets.sh scripts

OpenBao runtime authority ─────►  scoped workload secrets and dynamic credentials
      │
      ├──► leases, revocation, and audit records
      └──► direct client, External Secrets Operator, or CSI delivery
```

**KeePassXC is NOT in the operational path.** If you choose to import the
emergency bundle into KeePassXC for personal use, that is your business — it
is not required or assumed by any tooling in this repo.

The age private key and SOPS/age-encrypted git files are the bootstrap
credential store. The ops bundle is the bootstrap backup. The emergency
bundle is the human's break-glass key ring. OpenBao is the runtime secret
authority once the platform control plane is alive.

---

## 3. Credential Lifecycle

### Phase 0 — Bootstrap (run once, agent-driven)

```
make creds-agent-init
```

This single command runs the full bootstrap end-to-end:
1. Verifies prerequisites (age, kubectl, openssl, cluster)
2. Generates or verifies the age keypair at `~/.config/sops/age/keys.txt`
3. Generates all service secrets via `gen-secrets.sh`
4. Encrypts them to `secrets.enc/` with age and commits
5. Injects them into the cluster via `creds-apply.sh`
6. Verifies all K8s Secrets exist
7. Waits for privacyIDEA to be Ready, then runs enckey bootstrap + admin creation
8. Applies KeyCape secrets (requires pi-admin)
9. Hands off runtime secret authority to OpenBao when the Railiance
   platform layer is present and verified
10. Creates the ops bundle (age-encrypted snapshot)
11. Delivers the emergency bundle to the terminal for human storage
    ← **only human touchpoint**
12. Shreds all plaintext and marks `bootstrap_complete: true`

The script resumes from where it left off if interrupted — each phase is
tracked in `creds-state.yaml`.

### Phase 1 — Normal operation

No human credential management is needed after bootstrap. All secrets live in:
- `secrets.enc/` — encrypted in git (decrypt with age key)
- K8s Secrets — bootstrap or delivery state (updated by `creds-apply.sh`,
  External Secrets Operator, CSI, or other approved delivery paths)
- OpenBao — runtime secret authority for platform services, workload
  secrets, dynamic credentials, leases, revocation, and audit

Once OpenBao is ready, new long-lived workload secret authority should be
introduced there rather than by expanding the bootstrap SOPS/age surface.

### Phase 2 — Rotation

```
make creds-rotate SECRET=<name>            # guided interactive mode
SECRET=<name> bash sso-mfa/bootstrap/creds-rotate.sh --non-interactive  # agent mode
```

Rotation is handled per-secret with appropriate atomicity guarantees.
See Section 6 for details.

### Phase 3 — Recovery

Use the emergency bundle stored in your personal password manager.
See Section 4 — Emergency Bundle.

---

## 4. Emergency Bundle

### Contents

| Item | Purpose |
|------|---------|
| age private key | Decrypt any `secrets.enc/*.age` file from git |
| privacyIDEA admin password | Direct access to privacyIDEA if cluster/auth is down |
| LLDAP admin password | Direct LDAP directory access |
| PostgreSQL root password | Direct database access |
| break-glass user password | Emergency login if Authelia/KeyCape is down |
| ops bundle location + decrypt command | Point-in-time snapshot of all secrets |
| OpenBao unseal/recovery material, if used | Platform-control-plane break-glass only |

### Delivery

The emergency bundle is displayed once in the terminal at the end of
`make creds-agent-init`. The operator must copy it into their personal
password manager before pressing Enter to continue.

The confirmation gate is a **deliberate security control** — bootstrap
does not complete until the human confirms receipt.

### Re-delivery

If the emergency bundle is lost or stolen:

```
make creds-emergency-reprint
```

This reprints the bundle from the current age key and secrets.
It does **not** rotate any secrets — if you suspect compromise, rotate
the affected secrets separately with `make creds-rotate`.

### Storage recommendations

Store the emergency bundle in one or more of:
- 1Password, Bitwarden, KeePassXC, or equivalent
- Encrypted offline storage
- Printed and physically secured (air-gapped scenarios)

The agent does not care which — it only cares that you confirm receipt.

### Recovery procedure

1. Restore `~/.config/sops/age/keys.txt` from the emergency bundle
2. Clone net-kingdom repo
3. Run: `make creds-apply` — re-injects all secrets from `secrets.enc/`
4. Use break-glass passwords for direct service access if needed

---

## 5. OpenBao Runtime Authority

OpenBao is the canonical runtime secret authority for the platform
control plane once deployed and verified. It stores, issues, leases,
audits, and revokes runtime secret material; it does not replace identity
or authorization.

Required boundaries:

- SOPS/age remains the bootstrap and Git-at-rest protection mechanism.
- OpenBao root tokens, unseal keys, recovery keys, platform mounts, and
  global auth methods are platform-root material.
- Tenant administrators may manage tenant-scoped secret paths only
  through approved policies; they must not receive OpenBao platform-root
  authority.
- Workloads should use scoped OpenBao auth roles, External Secrets
  Operator, CSI-mounted secrets, or another approved delivery mechanism.
- flex-auth decides whether a secret or dynamic credential request is
  allowed when the request is authorization-sensitive; OpenBao performs
  storage, issuance, lease, revocation, and audit.
- OpenBao audit logs must be shipped to durable storage and included in
  restore and break-glass drills.

OpenBao may issue dynamic credentials for databases, object storage, or
other systems where provider support and policy make that safer than
static secret distribution. Provider-native STS remains valid where it
gives better-scoped temporary credentials; OpenBao can still broker,
store, or audit the handoff where appropriate.

---

## 6. Secret Rotation

### Rotatable secrets

| Secret | Blast radius | Notes |
|--------|-------------|-------|
| OpenBao workload secret | Variable | Prefer lease expiry or dynamic regeneration |
| OpenBao auth role/policy | Variable | Requires policy review and audit check |
| `PI_SECRET_KEY` | Low — invalidates PI sessions | Safe to rotate anytime |
| `PI_DB_PASSWORD` | Medium — DB + pod | Atomic update required |
| `LLDAP_JWT_SECRET` | Low — invalidates LLDAP sessions | Safe to rotate anytime |
| `LLDAP_LDAP_USER_PASS` | High — 3-way coordinated | Authelia + KeyCape + LLDAP web UI |
| `AUTHELIA_SESSION_SECRET` | Low — invalidates sessions | Safe to rotate anytime |
| `AUTHELIA_KEYCAPE_CLIENT_SECRET` | Medium — 2-way atomic | Authelia + KeyCape together |
| `KEYCAPE_RSA_KEY` | High — invalidates all JWTs | Causes brief auth outage |
| `BREAKGLASS_PASSWORD` | Minimal | Rotate freely |

### Non-rotatable

| Secret | Reason |
|--------|--------|
| `PI_PEPPER` | Rotating requires re-hashing all PI user passwords |

### Age key rotation

Rotating the age private key is a special case:
1. Generate a new age key
2. Re-encrypt all `secrets.enc/` files with the new public key (`make sops-rotate`)
3. Commit
4. A **new emergency bundle must be delivered** before the old key is revoked
   (see Section 4)

### OpenBao root and recovery material

OpenBao root tokens, unseal keys, and recovery keys are not routine
rotation targets. Treat them as platform-root break-glass material:

1. Prefer short-lived or revoked root tokens after initial setup.
2. Use scoped policies and auth methods for normal operations.
3. Rotate recovery or unseal material only with an explicit maintenance
   window, backup verification, and post-rotation emergency-bundle update.

---

## 7. Ops Bundle

The ops bundle is an age-encrypted tar archive of all plaintext secrets at a
point in time. It is created automatically during bootstrap and can be
refreshed with:

```
make creds-bundle
```

Store the bundle offsite (cloud, external drive, second location). Decrypt:

```
age -d -i ~/.config/sops/age/keys.txt ops-bundle-<date>.tar.age | tar xf -
```

---

## 8. Prohibited Patterns

The following are permanently prohibited:

1. **Committing `secrets/` (plaintext) to git** — the pre-commit hook blocks
   this; do not bypass with `--no-verify`

2. **Storing secrets in KeePassXC as the primary credential store** — KeePassXC
   is for optional personal backup of the emergency bundle only

3. **Skipping the `emergency_bundle_delivered` confirmation gate** — even in
   non-interactive or automated runs, the agent MUST NOT mark bootstrap
   complete until the human confirms receipt. This gate exists to ensure
   break-glass access is always available.

4. **Hardcoding secrets in manifests or ConfigMaps** — all service secrets
   must flow through K8s Secrets created by `create-secrets.sh` scripts

5. **Storing the age private key in the repo** — the key lives outside the
   repo at `~/.config/sops/age/keys.txt`

6. **Using OpenBao as a policy decision point** — OpenBao stores, leases,
   audits, and revokes secret material; identity comes from the IAM
   Profile and authorization decisions come from flex-auth where a
   decision boundary is needed

7. **Giving tenants OpenBao platform-root authority** — tenants may only
   receive scoped access paths approved for their tenant resources

---

## Appendix A — KeePassXC Group Structure (optional)

If you choose to import the emergency bundle into KeePassXC, a suggested
group structure for the break-glass passwords is:

```
net-kingdom/
  Break-glass/
    privacyIDEA admin    username=pi-admin        password=<PI_ADMIN_PASSWORD>
    LLDAP admin          username=admin           password=<LLDAP_LDAP_USER_PASS>
    PostgreSQL root      username=postgres        password=<PG_ROOT_PASSWORD>
    break-glass user     username=break-glass     password=<BREAKGLASS_PASSWORD>
  age key               (attach keys.txt as binary attachment)
  ops bundle decrypt    (store path + decrypt command as a note)
```

This is entirely optional — the agent does not read from or write to KeePassXC.