core-hub/docs/specs/testing-release-and-migration.md

Testing, Release, and Migration Spec

Test Layers

• unit tests for models, validators, and services
• API tests for route behavior and auth semantics
• contract tests from fixtures
• migration tests for Alembic revisions and imports
• consumer smokes for ops-hub and activity-core
• Playwright visual checks for protected operator UI desktop and mobile layouts

Operator UI Visual Checks

The protected /console route is checked with Playwright using make visual-check. The harness starts a disposable SQLite-backed Core Hub instance, seeds non-secret operator data, verifies unauthenticated access returns 401, renders authenticated desktop and mobile screenshots, checks for horizontal overflow and overlapping major regions, and asserts full API key material is absent from DOM state and visual artifacts. Generated screenshots are written under .local/visual-checks/ and must not be committed.

Install the Chromium runtime once per workstation with:

make playwright-install

Deployed API Smoke

The deployed /api/v2 smoke harness is run with make deployed-smoke. It expects CORE_HUB_BASE_URL plus an approved operator token supplied through CORE_HUB_OPERATOR_TOKEN_FILE or the environment variable named by CORE_HUB_OPERATOR_TOKEN_ENV (default: CORE_HUB_OPERATOR_TOKEN). The harness never prints token values.

Example staging invocation:

CORE_HUB_BASE_URL=https://core-hub-staging.example.invalid \
CORE_HUB_OPERATOR_TOKEN_FILE=/secure/operator/path/core-hub-operator-token \
CORE_HUB_SMOKE_OUTPUT=.local/smoke/core-hub-staging.json \
make deployed-smoke

The JSON report contains health/readiness status, catalog and protected-route checks, created smoke ids, counts, event ids, and API key prefixes only. If the harness creates a runtime key for the smoke API consumer, set CORE_HUB_RUNTIME_TOKEN_OUTPUT to an operator-controlled 0600 path and move the value into the approved custody system immediately. Do not commit .local/ reports or runtime token files.

Staging Deployment Profile

The staging deployment profile lives in docs/deployment/staging-profile.md and k8s/railiance-staging/. Validate it with make staging-profile-check. The profile defines the non-secret Kubernetes runtime shape, secret references, Alembic migration job, image build command, health/readiness checks, rollback notes, and the handoff to CORE-WP-0005-T02 staging import.

Release Gates

A Core Hub release cannot replace Inter-Hub until:

• public /api/v2 compatibility smokes pass;
• protected route auth behavior is proven without exposing secrets;
• ops-hub bootstrap smoke passes;
• activity-core sink smoke passes or is explicitly deferred to State Hub fallback;
• data import row counts and relationship checks pass;
• rollback path is documented;
• production operator approves cutover.

Migration Gates

• Extract Inter-Hub schema and route inventory.
• Build fixture set from representative production-safe records.
• Import data into Core Hub staging.
• Compare row counts and key relationships.
• Run dual-read or shadow smokes.
• Switch traffic only after smoke evidence is recorded.

Migration Bundle v1

Core Hub staging imports use core-hub.migration.v1 JSON bundles. The bundle is the handoff contract from the Haskell Inter-Hub export path into the Python/FastAPI replacement:

• schemaVersion: core-hub.migration.v1
• source: normally inter-hub-haskell
• exportedAt: export timestamp for operator traceability
• records.hubs
• records.hubCapabilityManifests
• records.apiConsumers
• records.apiKeys
• records.widgets
• records.interactionEvents

API key records may contain keyPrefix, keyHash, scopes, status, and lifecycle metadata. They must not contain full key material, raw tokens, or secrets. Existing runtime keys are not recoverable from Inter-Hub unless an approved custody path provides them; migration preserves only non-secret evidence.

The reference schema is contracts/schemas/migration-bundle.schema.json, with a minimal fixture at contracts/fixtures/migration/interhub-minimal.bundle.json.

Import Procedure

Validate a bundle without database access:

PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py validate \
  contracts/fixtures/migration/interhub-minimal.bundle.json

Dry-run against the configured target database:

CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
  /secure/operator/path/interhub-export.bundle.json --dry-run

Import after the dry-run report is reviewed:

CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
  /secure/operator/path/interhub-export.bundle.json

Each successful import writes a migration_runs row containing the source, schema version, bundle SHA-256, row counts, warnings, and relationship checks.

Rollback Procedure

For staging, reset the target database and replay the same bundle after fixing the export or mapper. For production cutover, take a database snapshot before import, keep Inter-Hub read-only during verification, and roll traffic back to Inter-Hub if row counts, relationship checks, or consumer smokes fail. Do not delete or rotate legacy key custody records during the rollback window.

Haskell Retirement Gate

Do not retire production Inter-Hub or the haskelseed/Haskell build lane until Core Hub serves the required compatibility surface and production traffic has moved.