generated from coulomb/repo-seed
tegwick
b59b9a565c
Add RTO-RPO matrix, backup pickup queue tracker, and TestEvidence with Forgejo drill import. Mark T03, T04, T05 done.
96 lines
3.5 KiB
Markdown
96 lines
3.5 KiB
Markdown
# Restore Drill: Forgejo production
|
||
|
||
**Date:** 2026-07-04
|
||
**System:** `backup-forgejo-production` (Forgejo git + packages + `forgejo-db`)
|
||
**Operator:** Bernd (via grok session, `railiance-infra`)
|
||
**Result:** success
|
||
**Workplan:** RAIL-HO-WP-0005-T09
|
||
|
||
`no_secret_material_recorded: true`
|
||
|
||
## Summary
|
||
|
||
Proved that a production `forgejo dump` can be restored into an isolated
|
||
namespace and serve repository metadata without touching production Forgejo or
|
||
Gitea. This validates **recovery mechanics** — not operational backup coverage.
|
||
|
||
## Backup artifact
|
||
|
||
| Field | Value |
|
||
| --- | --- |
|
||
| Method | `forgejo dump` from production pod (manual) |
|
||
| Production pod | `forgejo-gitea-64c5b57684-ph9vt` (namespace `forgejo`) |
|
||
| Archive path | `/tmp/forgejo-drill/forgejo-drill-backup.zip` (workstation) |
|
||
| Archive size | 12,284,847 bytes (~11.7 MiB) |
|
||
| Archive timestamp | 2026-07-04 11:20 +0200 |
|
||
| Contents | `repos/`, `data/`, `forgejo-db.sql`, `app.ini` |
|
||
| Repos in dump | `forgejo-actions-probe`, `glas-harness`, `key-cape` (`coulomb/`) |
|
||
| Automated / offsite | **No** — workstation `/tmp` only; not from Nextcloud |
|
||
|
||
## Restore target
|
||
|
||
| Field | Value |
|
||
| --- | --- |
|
||
| Namespace | `forgejo-restore-drill` |
|
||
| Database | CNPG `forgejo-db-restore` (isolated, 1 instance) |
|
||
| App data PVC | `forgejo-restore-data` (`local-path`, 10Gi) |
|
||
| Helm release | `forgejo-restore` (`gitea-charts/gitea` 12.5.0) |
|
||
| Orchestration | `railiance-infra/tools/forgejo-restore-drill.sh` |
|
||
|
||
Restore steps (Forgejo 11.0.3 — no `forgejo restore` CLI):
|
||
|
||
1. Unzip dump into import pod staging area.
|
||
2. Copy `repos/` → `/data/git/gitea-repositories/`.
|
||
3. Copy `data/` → `/data/` (packages, attachments, avatars).
|
||
4. Import `forgejo-db.sql` via `psql` into `forgejo-db-restore`.
|
||
5. Deploy isolated Helm release bound to restored PVC + restore DB host.
|
||
|
||
## Validation checks
|
||
|
||
Port-forward: `svc/forgejo-restore-gitea-http` → `127.0.0.1:13000`
|
||
|
||
| Check | Result |
|
||
| --- | --- |
|
||
| `GET /` health | HTTP 200 |
|
||
| `GET /api/v1/repos/coulomb/glas-harness` | `full_name=coulomb/glas-harness`, `default_branch=main` |
|
||
| `GET /api/v1/repos/coulomb/key-cape` | `full_name=coulomb/key-cape`, `default_branch=main` |
|
||
| `GET /api/v1/orgs/coulomb/repos` | 3 repos visible |
|
||
|
||
Script exit marker: `restore-drill-complete`
|
||
|
||
## RPO / RTO measured
|
||
|
||
| Metric | Target (adopted) | Achieved |
|
||
| --- | --- | --- |
|
||
| RPO | 24h | Point-in-time of manual dump — **no scheduled backup** |
|
||
| RTO | 4h | ~3–5 min (isolated namespace on railiance01) |
|
||
| Production impact | None | Read-only dump; separate namespace |
|
||
|
||
Posture: RTO target met. RPO target **not operational** until daily automated
|
||
backups run (see `BackupPickupQueue.md` items 3–4).
|
||
|
||
## Issues found
|
||
|
||
- No scheduled backups; CNPG `Backup` CRs not configured.
|
||
- Dump stored on workstation only — no encryption-at-rest on approved offsite target.
|
||
- SQL import not idempotent — re-runs need `DRILL_CLEAN=1`.
|
||
|
||
## Follow-up actions
|
||
|
||
| Action | Owner | Queue / workplan |
|
||
| --- | --- | --- |
|
||
| Wire daily `forgejo dump` + Nextcloud upload | `railiance-platform` | Pickup #3 |
|
||
| Restore platform `railiance-backup` tool | `railiance-platform` | Pickup #4 |
|
||
| Re-drill from Nextcloud artifact | `railiance-infra` | Promotion gate (`BackupPolicy.md` §7) |
|
||
| 7 consecutive daily backup successes | `railiance-platform` | Promotion gate |
|
||
|
||
## Canonical evidence
|
||
|
||
Full drill record (owner repo):
|
||
|
||
`railiance-infra/docs/forgejo-restore-drill-evidence.md`
|
||
|
||
Supporting artifacts:
|
||
|
||
- `railiance-infra/tools/forgejo-restore-drill.sh`
|
||
- `railiance-infra/infra/forgejo-restore-drill/` |