Add Core Hub staging deployment profile

2026-06-27 21:11:46 +02:00
parent f280f9b0f8
commit b895d1f772
12 changed files with 519 additions and 2 deletions
--- a/.dockerignore
+++ b/.dockerignore
@@ -0,0 +1,15 @@
+.git
+.venv
+.pytest_cache
+.ruff_cache
+.mypy_cache
+.local
+__pycache__
+*.pyc
+*.pyo
+*.db
+*.sqlite
+*.sqlite3
+.env
+.env.*
+!.env.example
--- a/31
+++ b/31
@@ -0,0 +1,31 @@
+# Stage 1: install production dependencies into an isolated virtualenv.
+FROM python:3.12-slim AS builder
+
+RUN pip install uv --no-cache-dir
+WORKDIR /app
+
+COPY pyproject.toml uv.lock README.md ./
+COPY src/ ./src/
+COPY alembic.ini ./
+COPY migrations/ ./migrations/
+COPY scripts/ ./scripts/
+
+RUN uv sync --no-dev --frozen
+
+# Stage 2: runtime image for API and migration job commands.
+FROM python:3.12-slim AS runtime
+
+WORKDIR /app
+COPY --from=builder /app/.venv /app/.venv
+COPY --from=builder /app/src /app/src
+COPY --from=builder /app/alembic.ini /app/alembic.ini
+COPY --from=builder /app/migrations /app/migrations
+COPY --from=builder /app/scripts /app/scripts
+
+ENV PATH="/app/.venv/bin:$PATH"
+ENV PYTHONPATH="/app/src"
+ENV CORE_HUB_ENV="staging"
+ENV CORE_HUB_AUTO_CREATE_TABLES="0"
+
+EXPOSE 8010
+CMD ["uvicorn", "core_hub.app:app", "--host", "0.0.0.0", "--port", "8010"]
--- a/10
+++ b/10
@@ -1,7 +1,9 @@
-.PHONY: install test lint run openapi migrate-validate deployed-smoke playwright-install visual-check
+.PHONY: install test lint run openapi migrate-validate deployed-smoke staging-profile-check container-build playwright-install visual-check

 UV ?= /home/worsch/.local/bin/uv
 PYTHONPATH ?= src
+IMAGE_REPOSITORY ?= gitea.coulomb.social/coulomb/core-hub
+IMAGE_TAG ?= dev

 install:
 	$(UV) sync --extra dev
@@ -25,6 +27,12 @@ migrate-validate:
 deployed-smoke:
 	PYTHONPATH=$(PYTHONPATH) $(UV) run --extra dev python scripts/core_hub_deployed_smoke.py

+staging-profile-check:
+	PYTHONPATH=$(PYTHONPATH) $(UV) run --extra dev python scripts/check_staging_profile.py
+
+container-build:
+	docker build -t $(IMAGE_REPOSITORY):$(IMAGE_TAG) .
+
 playwright-install:
 	$(UV) run --extra dev playwright install chromium

--- a/docs/deployment/staging-profile.md
+++ b/docs/deployment/staging-profile.md
@@ -0,0 +1,153 @@
+# Core Hub Staging Deployment Profile
+
+Status: service-repo profile for `CORE-WP-0008-T04`.
+
+This profile turns the Core Hub replacement work into a repeatable staging
+shape that can run the deployed API smoke (`CORE-WP-0008-T02`), the
+activity-core evidence sink proof (`CORE-WP-0008-T03`), and the staging import
+for `CORE-WP-0005-T02` without reconstructing shell state from memory.
+
+## Ownership Boundary
+
+Core Hub owns the application image, runtime environment contract, migrations,
+smokes, and API behavior. Railiance application deployment ownership should
+move to `railiance-apps` once the permanent Helm/chart path is created. Until
+then, `k8s/railiance-staging/` is the checked-in service-repo staging profile.
+
+## Runtime Configuration
+
+The staging API runs as `core-hub-api` in namespace `core-hub-staging` on port
+`8010`.
+
+Non-secret runtime config lives in `core-hub-staging-runtime`:
+
+| Variable | Staging value | Notes |
+| --- | --- | --- |
+| `CORE_HUB_ENV` | `staging` | Appears in process config only. |
+| `CORE_HUB_AUTO_CREATE_TABLES` | `0` | Alembic is the only staging schema path. |
+
+Secret runtime config is referenced through `core-hub-staging-env`; it is not
+committed:
+
+| Secret key | Owner / custody path | Notes |
+| --- | --- | --- |
+| `CORE_HUB_DATABASE_URL` | OpenBao / railiance-platform database custody | Async SQLAlchemy URL for the staging Postgres database. |
+| `CORE_HUB_API_TOKEN` | OpenBao / railiance-platform app-secret custody | Operator token for protected API smokes and bootstrap operations. |
+
+The runtime Secret may later also provide an activity-core runtime token or
+widget mapping if the operator chooses to run activity-core from the cluster
+against this staging API. Keep those in the consuming service's Secret, not in
+Core Hub manifests, unless Core Hub itself needs them.
+
+## Database And Migrations
+
+Use a dedicated staging database, for example database `core_hub_staging` and
+least-privilege app user `core_hub_staging`. The actual password and URL are
+operator-owned secrets.
+
+Local validation before deploy:
+
+```bash
+PYTHONPATH=src uv run --extra dev alembic upgrade head
+```
+
+Cluster migration path:
+
+```bash
+kubectl -n core-hub-staging delete job core-hub-staging-migrate --ignore-not-found
+kubectl apply -f k8s/railiance-staging/20-runtime.yaml
+kubectl -n core-hub-staging set image job/core-hub-staging-migrate \
+  migrate=gitea.coulomb.social/coulomb/core-hub:<tag>
+kubectl -n core-hub-staging wait --for=condition=complete \
+  job/core-hub-staging-migrate --timeout=180s
+```
+
+Do not set `CORE_HUB_AUTO_CREATE_TABLES=1` in staging. That flag is for local
+smoke/test bootstraps only.
+
+## Build And Release Commands
+
+Near-term service-repo image build:
+
+```bash
+IMAGE_REPOSITORY=gitea.coulomb.social/coulomb/core-hub \
+IMAGE_TAG=$(git rev-parse --short HEAD) \
+make container-build
+
+docker push gitea.coulomb.social/coulomb/core-hub:$(git rev-parse --short HEAD)
+```
+
+Railiance-style deploy skeleton:
+
+```bash
+kubectl apply -f k8s/railiance-staging/00-namespace.yaml
+kubectl -n core-hub-staging get secret core-hub-staging-env
+kubectl apply -f k8s/railiance-staging/20-runtime.yaml
+kubectl -n core-hub-staging set image deploy/core-hub-api \
+  api=gitea.coulomb.social/coulomb/core-hub:<tag>
+kubectl -n core-hub-staging rollout status deploy/core-hub-api --timeout=180s
+```
+
+Longer term, promote this into `railiance-apps` with explicit targets analogous
+to `inter-hub-dry-run`, `inter-hub-server-dry-run`, `inter-hub-deploy`, and
+`inter-hub-status`. Production or shared staging deploys must use explicit image
+tags, not mutable `latest`.
+
+## Health, Readiness, And Smokes
+
+Cluster-local checks:
+
+```bash
+kubectl -n core-hub-staging exec deploy/core-hub-api -- \
+  python -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8010/healthz').read().decode())"
+
+kubectl -n core-hub-staging exec deploy/core-hub-api -- \
+  python -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8010/readyz').read().decode())"
+```
+
+Operator smoke after routing or port-forward is available:
+
+```bash
+CORE_HUB_BASE_URL=https://core-hub-staging.example.invalid \
+CORE_HUB_OPERATOR_TOKEN_FILE=/secure/operator/path/core-hub-staging-operator-token \
+CORE_HUB_SMOKE_OUTPUT=.local/smoke/core-hub-staging.json \
+make deployed-smoke
+```
+
+activity-core can use the direct sink from `ACTIVITY-WP-0017` once it has:
+
+- `CORE_HUB_BASE_URL` for staging;
+- `CORE_HUB_RUNTIME_TOKEN_FILE` or a named token environment variable;
+- `CORE_HUB_WIDGET_ID` or `CORE_HUB_WIDGET_MAPPING` for the target widget.
+
+## Staging Import Relationship
+
+`CORE-WP-0005-T02` starts only after this profile is deployed, migrations have
+completed, and the target staging database is known. Import sequence:
+
+```bash
+PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py validate \
+  /secure/operator/path/interhub-export.bundle.json
+
+CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
+PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
+  /secure/operator/path/interhub-export.bundle.json --dry-run
+
+CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
+PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
+  /secure/operator/path/interhub-export.bundle.json
+```
+
+Record the generated row counts and diagnostics as non-secret evidence. The
+bundle path and database URL shown above are placeholders; use approved operator
+paths and secret references.
+
+## Rollback Notes
+
+- Keep Haskell Inter-Hub live as compatibility and rollback until Core Hub
+  staging import, deployed smokes, dual-run, and operator approval complete.
+- For staging-only failures, scale the API to zero or roll the Deployment image
+  back to the last known good tag.
+- For migration failures, snapshot/reset the staging database and replay the
+  same approved bundle after fixing mapper/schema defects.
+- Never rotate or delete legacy Inter-Hub keys during staging rollback.
--- a/docs/specs/testing-release-and-migration.md
+++ b/docs/specs/testing-release-and-migration.md
@@ -49,6 +49,15 @@ harness creates a runtime key for the smoke API consumer, set
 value into the approved custody system immediately. Do not commit `.local/`
 reports or runtime token files.

+## Staging Deployment Profile
+
+The staging deployment profile lives in `docs/deployment/staging-profile.md`
+and `k8s/railiance-staging/`. Validate it with `make
+staging-profile-check`. The profile defines the non-secret Kubernetes
+runtime shape, secret references, Alembic migration job, image build
+command, health/readiness checks, rollback notes, and the handoff to
+`CORE-WP-0005-T02` staging import.
+
 ## Release Gates

 A Core Hub release cannot replace Inter-Hub until:
--- a/k8s/railiance-staging/00-namespace.yaml
+++ b/k8s/railiance-staging/00-namespace.yaml
@@ -0,0 +1,8 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: core-hub-staging
+  labels:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/part-of: core-hub
+    environment: staging
--- a/k8s/railiance-staging/20-runtime.yaml
+++ b/k8s/railiance-staging/20-runtime.yaml
@@ -0,0 +1,113 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: core-hub-staging-runtime
+  namespace: core-hub-staging
+  labels:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/part-of: core-hub
+    environment: staging
+data:
+  CORE_HUB_ENV: staging
+  CORE_HUB_AUTO_CREATE_TABLES: "0"
+---
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: core-hub-staging-migrate
+  namespace: core-hub-staging
+  labels:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/component: migration
+    app.kubernetes.io/part-of: core-hub
+    environment: staging
+spec:
+  backoffLimit: 1
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: core-hub
+        app.kubernetes.io/component: migration
+        app.kubernetes.io/part-of: core-hub
+        environment: staging
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: migrate
+          image: gitea.coulomb.social/coulomb/core-hub:staging-placeholder
+          imagePullPolicy: IfNotPresent
+          command: ["alembic", "upgrade", "head"]
+          envFrom:
+            - configMapRef:
+                name: core-hub-staging-runtime
+            - secretRef:
+                name: core-hub-staging-env
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: core-hub-api
+  namespace: core-hub-staging
+  labels:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/component: api
+    app.kubernetes.io/part-of: core-hub
+    environment: staging
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app.kubernetes.io/name: core-hub
+      app.kubernetes.io/component: api
+  template:
+    metadata:
+      labels:
+        app.kubernetes.io/name: core-hub
+        app.kubernetes.io/component: api
+        app.kubernetes.io/part-of: core-hub
+        environment: staging
+    spec:
+      containers:
+        - name: api
+          image: gitea.coulomb.social/coulomb/core-hub:staging-placeholder
+          imagePullPolicy: IfNotPresent
+          ports:
+            - name: http
+              containerPort: 8010
+          envFrom:
+            - configMapRef:
+                name: core-hub-staging-runtime
+            - secretRef:
+                name: core-hub-staging-env
+          readinessProbe:
+            httpGet:
+              path: /readyz
+              port: http
+            initialDelaySeconds: 5
+            periodSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: http
+            initialDelaySeconds: 15
+            periodSeconds: 20
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: core-hub-api
+  namespace: core-hub-staging
+  labels:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/component: api
+    app.kubernetes.io/part-of: core-hub
+    environment: staging
+spec:
+  type: ClusterIP
+  selector:
+    app.kubernetes.io/name: core-hub
+    app.kubernetes.io/component: api
+  ports:
+    - name: http
+      port: 8010
+      targetPort: http
--- a/k8s/railiance-staging/README.md
+++ b/k8s/railiance-staging/README.md
@@ -0,0 +1,53 @@
+# Core Hub Railiance Staging Profile
+
+This directory contains the service-repo staging profile for Core Hub. It is a
+handoffable profile, not the final Railiance app chart. When `railiance-apps`
+gets the permanent chart/Helm ownership, keep these contracts intact or record
+an explicit supersede decision.
+
+## Runtime Shape
+
+- Namespace: `core-hub-staging`
+- API deployment: `deploy/core-hub-api`
+- Service: `svc/core-hub-api` on port `8010`
+- Migration job: `job/core-hub-staging-migrate`
+- Non-secret ConfigMap: `core-hub-staging-runtime`
+- Secret reference: `core-hub-staging-env`
+
+`core-hub-staging-env` must be provisioned by the approved custody path before
+running the migration job or API deployment. It must contain:
+
+- `CORE_HUB_DATABASE_URL`: async SQLAlchemy URL, normally
+  `postgresql+asyncpg://...` for the staging database.
+- `CORE_HUB_API_TOKEN`: operator token used by deployed smokes and protected
+  bootstrap/API operations.
+
+Do not add plaintext secret values to these manifests.
+
+## Image
+
+The committed manifest uses `gitea.coulomb.social/coulomb/core-hub:staging-placeholder`.
+Operators should patch or render the immutable image tag for the deployment and
+migration job before apply.
+
+## Apply Skeleton
+
+```bash
+kubectl apply -f k8s/railiance-staging/00-namespace.yaml
+kubectl -n core-hub-staging get secret core-hub-staging-env
+kubectl apply -f k8s/railiance-staging/20-runtime.yaml
+kubectl -n core-hub-staging set image job/core-hub-staging-migrate \
+  migrate=gitea.coulomb.social/coulomb/core-hub:<tag>
+kubectl -n core-hub-staging wait --for=condition=complete \
+  job/core-hub-staging-migrate --timeout=180s
+kubectl -n core-hub-staging set image deploy/core-hub-api \
+  api=gitea.coulomb.social/coulomb/core-hub:<tag>
+kubectl -n core-hub-staging rollout status deploy/core-hub-api --timeout=180s
+```
+
+For repeated migrations, delete the previous job first because Kubernetes Jobs
+are effectively immutable:
+
+```bash
+kubectl -n core-hub-staging delete job core-hub-staging-migrate --ignore-not-found
+```
--- a/scripts/check_staging_profile.py
+++ b/scripts/check_staging_profile.py
@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+REQUIRED_FILES = [
+    "Dockerfile",
+    "k8s/railiance-staging/00-namespace.yaml",
+    "k8s/railiance-staging/20-runtime.yaml",
+    "k8s/railiance-staging/README.md",
+    "docs/deployment/staging-profile.md",
+]
+RUNTIME_REQUIRED = [
+    "kind: ConfigMap",
+    "core-hub-staging-runtime",
+    "kind: Job",
+    "core-hub-staging-migrate",
+    "command: [\"alembic\", \"upgrade\", \"head\"]",
+    "kind: Deployment",
+    "core-hub-api",
+    "path: /readyz",
+    "path: /healthz",
+    "secretRef:",
+    "core-hub-staging-env",
+]
+DOC_REQUIRED = [
+    "CORE-WP-0005-T02",
+    "make deployed-smoke",
+    "CORE_HUB_DATABASE_URL",
+    "CORE_HUB_API_TOKEN",
+    "CORE_HUB_AUTO_CREATE_TABLES=1",
+    "railiance-apps",
+    "inter-hub-dry-run",
+]
+FORBIDDEN_RUNTIME = [
+    "stringData:",
+    "CORE_HUB_DATABASE_URL:",
+    "CORE_HUB_API_TOKEN:",
+    "postgresql+asyncpg://core_hub:",
+    "Bearer ",
+]
+FORBIDDEN_DOC = [
+    "postgresql+asyncpg://core_hub:core_hub@",
+    "CORE_HUB_API_TOKEN=",
+]
+
+
+def assert_contains(path: Path, needles: list[str]) -> list[str]:
+    text = path.read_text(encoding="utf-8")
+    return [needle for needle in needles if needle not in text]
+
+
+def assert_absent(path: Path, needles: list[str]) -> list[str]:
+    text = path.read_text(encoding="utf-8")
+    return [needle for needle in needles if needle in text]
+
+
+def main() -> int:
+    failures: list[str] = []
+    for relative in REQUIRED_FILES:
+        if not (ROOT / relative).exists():
+            failures.append(f"missing required file: {relative}")
+
+    runtime = ROOT / "k8s/railiance-staging/20-runtime.yaml"
+    docs = ROOT / "docs/deployment/staging-profile.md"
+    dockerfile = ROOT / "Dockerfile"
+
+    if runtime.exists():
+        for missing in assert_contains(runtime, RUNTIME_REQUIRED):
+            failures.append(f"runtime manifest missing: {missing}")
+        for present in assert_absent(runtime, FORBIDDEN_RUNTIME):
+            failures.append(f"runtime manifest must not contain: {present}")
+
+    if docs.exists():
+        for missing in assert_contains(docs, DOC_REQUIRED):
+            failures.append(f"staging profile docs missing: {missing}")
+        for present in assert_absent(docs, FORBIDDEN_DOC):
+            failures.append(f"staging profile docs must not contain: {present}")
+
+    if dockerfile.exists():
+        docker_text = dockerfile.read_text(encoding="utf-8")
+        for needle in ["uv sync --no-dev --frozen", "EXPOSE 8010", "CORE_HUB_AUTO_CREATE_TABLES"]:
+            if needle not in docker_text:
+                failures.append(f"Dockerfile missing: {needle}")
+
+    if failures:
+        for failure in failures:
+            print(f"FAIL: {failure}")
+        return 1
+    print("staging profile check passed")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
--- a/tests/test_staging_profile.py
+++ b/tests/test_staging_profile.py
@@ -0,0 +1,13 @@
+import importlib.util
+from pathlib import Path
+
+
+def test_staging_profile_check_passes() -> None:
+    checker = Path(__file__).resolve().parents[1] / "scripts" / "check_staging_profile.py"
+    spec = importlib.util.spec_from_file_location("check_staging_profile", checker)
+    assert spec is not None
+    assert spec.loader is not None
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+
+    assert module.main() == 0
--- a/workplans/CORE-WP-0005-data-migration-and-cutover.md
+++ b/workplans/CORE-WP-0005-data-migration-and-cutover.md
@@ -38,6 +38,11 @@ state_hub_task_id: "e0ea0928-a3ea-4ece-bbb7-ee08f8a3279b"

 Import production-safe or full approved Inter-Hub data into Core Hub staging. The importer is ready; this now waits on an approved Inter-Hub export bundle and target staging database. Record counts and discrepancies from the generated migration report.

+2026-06-27 continuation: `CORE-WP-0008-T04` added the Core Hub staging
+deployment profile and migration job/runbook. This task still waits for an
+approved Inter-Hub export bundle and provisioned staging database before an
+actual import can run.
+
 ## Dual-Run Smokes

 ```task
--- a/workplans/CORE-WP-0008-api-first-ops-evidence-continuation.md
+++ b/workplans/CORE-WP-0008-api-first-ops-evidence-continuation.md
@@ -159,7 +159,7 @@ deployed execution gate rather than an implementation blocker.

 ```task
 id: CORE-WP-0008-T04
-status: todo
+status: done
 priority: high
 state_hub_task_id: "fc53da17-485d-438c-986f-38d688d75b51"
 ```
@@ -178,6 +178,19 @@ Minimum scope:
 Done when Core Hub can be deployed to staging and used by T02/T03 smokes without
 one-off shell reconstruction.

+Completed 2026-06-27: added the service-repo staging deployment profile:
+Dockerfile, `.dockerignore`, `k8s/railiance-staging/` namespace/runtime
+manifests, `docs/deployment/staging-profile.md`, `make
+staging-profile-check`, `make container-build`, and test coverage for the
+profile checker. The profile defines staging runtime config, database and
+secret references, Alembic migration job, health/readiness probes,
+Railiance-style build/apply commands, rollback notes, and the `CORE-WP-0005-T02`
+staging import handoff. Verified with `make staging-profile-check`,
+`make lint`, `make test`, and `make container-build
+IMAGE_REPOSITORY=core-hub IMAGE_TAG=staging-profile-check`. It
+intentionally does not contain plaintext secret values or claim that a real
+staging cluster has already been deployed.
+
 ## Task: Add CLI wrappers after API proof

 ```task