core-hub/docs/specs/testing-release-and-migration.md

# Testing, Release, and Migration Spec

## Test Layers

- unit tests for models, validators, and services
- API tests for route behavior and auth semantics
- contract tests from fixtures
- migration tests for Alembic revisions and imports
- consumer smokes for ops-hub and activity-core
- Playwright visual checks for protected operator UI desktop and mobile layouts

## Operator UI Visual Checks

The protected `/console` route is checked with Playwright using `make visual-check`.
The harness starts a disposable SQLite-backed Core Hub instance, seeds non-secret
operator data, verifies unauthenticated access returns `401`, renders authenticated
desktop and mobile screenshots, checks for horizontal overflow and overlapping
major regions, and asserts full API key material is absent from DOM state and
visual artifacts. Generated screenshots are written under `.local/visual-checks/`
and must not be committed.

Install the Chromium runtime once per workstation with:

```bash
make playwright-install
```

## Deployed API Smoke

The deployed `/api/v2` smoke harness is run with `make deployed-smoke`. It
expects `CORE_HUB_BASE_URL` plus an approved operator token supplied through
`CORE_HUB_OPERATOR_TOKEN_FILE` or the environment variable named by
`CORE_HUB_OPERATOR_TOKEN_ENV` (default: `CORE_HUB_OPERATOR_TOKEN`). The harness
never prints token values.

Example staging invocation:

```bash
CORE_HUB_BASE_URL=https://core-hub-staging.example.invalid \
CORE_HUB_OPERATOR_TOKEN_FILE=/secure/operator/path/core-hub-operator-token \
CORE_HUB_SMOKE_OUTPUT=.local/smoke/core-hub-staging.json \
make deployed-smoke
```

The JSON report contains health/readiness status, catalog and protected-route
checks, created smoke ids, counts, event ids, and API key prefixes only. If the
harness creates a runtime key for the smoke API consumer, set
`CORE_HUB_RUNTIME_TOKEN_OUTPUT` to an operator-controlled 0600 path and move the
value into the approved custody system immediately. Do not commit `.local/`
reports or runtime token files.

## Staging Deployment Profile

The staging deployment profile lives in `docs/deployment/staging-profile.md`
and `k8s/railiance-staging/`. Validate it with `make
staging-profile-check`. The profile defines the non-secret Kubernetes
runtime shape, secret references, Alembic migration job, image build
command, health/readiness checks, rollback notes, and the handoff to
`CORE-WP-0005-T02` staging import.

## Release Gates

A Core Hub release cannot replace Inter-Hub until:

- public `/api/v2` compatibility smokes pass;
- protected route auth behavior is proven without exposing secrets;
- ops-hub bootstrap smoke passes;
- activity-core sink smoke passes or is explicitly deferred to State Hub fallback;
- data import row counts and relationship checks pass;
- rollback path is documented;
- production operator approves cutover.

## Migration Gates

- Extract Inter-Hub schema and route inventory.
- Build fixture set from representative production-safe records.
- Import data into Core Hub staging.
- Compare row counts and key relationships.
- Run dual-read or shadow smokes.
- Switch traffic only after smoke evidence is recorded.

## Migration Bundle v1

Core Hub staging imports use `core-hub.migration.v1` JSON bundles. The bundle is
the handoff contract from the Haskell Inter-Hub export path into the Python/FastAPI
replacement:

- `schemaVersion`: `core-hub.migration.v1`
- `source`: normally `inter-hub-haskell`
- `exportedAt`: export timestamp for operator traceability
- `records.hubs`
- `records.hubCapabilityManifests`
- `records.apiConsumers`
- `records.apiKeys`
- `records.widgets`
- `records.interactionEvents`

API key records may contain `keyPrefix`, `keyHash`, scopes, status, and lifecycle
metadata. They must not contain full key material, raw tokens, or secrets. Existing
runtime keys are not recoverable from Inter-Hub unless an approved custody path
provides them; migration preserves only non-secret evidence.

The reference schema is
`contracts/schemas/migration-bundle.schema.json`, with a minimal fixture at
`contracts/fixtures/migration/interhub-minimal.bundle.json`.

## Import Procedure

Validate a bundle without database access:

```bash
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py validate \
  contracts/fixtures/migration/interhub-minimal.bundle.json
```

Dry-run against the configured target database:

```bash
CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
  /secure/operator/path/interhub-export.bundle.json --dry-run
```

Import after the dry-run report is reviewed:

```bash
CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
  /secure/operator/path/interhub-export.bundle.json
```

Each successful import writes a `migration_runs` row containing the source,
schema version, bundle SHA-256, row counts, warnings, and relationship checks.

## Rollback Procedure

For staging, reset the target database and replay the same bundle after fixing the
export or mapper. For production cutover, take a database snapshot before import,
keep Inter-Hub read-only during verification, and roll traffic back to Inter-Hub if
row counts, relationship checks, or consumer smokes fail. Do not delete or rotate
legacy key custody records during the rollback window.

## Haskell Retirement Gate

Do not retire production Inter-Hub or the haskelseed/Haskell build lane until Core Hub serves the required compatibility surface and production traffic has moved.