generated from coulomb/repo-seed
137 lines
5.1 KiB
Markdown
137 lines
5.1 KiB
Markdown
# Testing, Release, and Migration Spec
|
|
|
|
## Test Layers
|
|
|
|
- unit tests for models, validators, and services
|
|
- API tests for route behavior and auth semantics
|
|
- contract tests from fixtures
|
|
- migration tests for Alembic revisions and imports
|
|
- consumer smokes for ops-hub and activity-core
|
|
- Playwright visual checks for protected operator UI desktop and mobile layouts
|
|
|
|
## Operator UI Visual Checks
|
|
|
|
The protected `/console` route is checked with Playwright using `make visual-check`.
|
|
The harness starts a disposable SQLite-backed Core Hub instance, seeds non-secret
|
|
operator data, verifies unauthenticated access returns `401`, renders authenticated
|
|
desktop and mobile screenshots, checks for horizontal overflow and overlapping
|
|
major regions, and asserts full API key material is absent from DOM state and
|
|
visual artifacts. Generated screenshots are written under `.local/visual-checks/`
|
|
and must not be committed.
|
|
|
|
Install the Chromium runtime once per workstation with:
|
|
|
|
```bash
|
|
make playwright-install
|
|
```
|
|
|
|
## Deployed API Smoke
|
|
|
|
The deployed `/api/v2` smoke harness is run with `make deployed-smoke`. It
|
|
expects `CORE_HUB_BASE_URL` plus an approved operator token supplied through
|
|
`CORE_HUB_OPERATOR_TOKEN_FILE` or the environment variable named by
|
|
`CORE_HUB_OPERATOR_TOKEN_ENV` (default: `CORE_HUB_OPERATOR_TOKEN`). The harness
|
|
never prints token values.
|
|
|
|
Example staging invocation:
|
|
|
|
```bash
|
|
CORE_HUB_BASE_URL=https://core-hub-staging.example.invalid \
|
|
CORE_HUB_OPERATOR_TOKEN_FILE=/secure/operator/path/core-hub-operator-token \
|
|
CORE_HUB_SMOKE_OUTPUT=.local/smoke/core-hub-staging.json \
|
|
make deployed-smoke
|
|
```
|
|
|
|
The JSON report contains health/readiness status, catalog and protected-route
|
|
checks, created smoke ids, counts, event ids, and API key prefixes only. If the
|
|
harness creates a runtime key for the smoke API consumer, set
|
|
`CORE_HUB_RUNTIME_TOKEN_OUTPUT` to an operator-controlled 0600 path and move the
|
|
value into the approved custody system immediately. Do not commit `.local/`
|
|
reports or runtime token files.
|
|
|
|
## Release Gates
|
|
|
|
A Core Hub release cannot replace Inter-Hub until:
|
|
|
|
- public `/api/v2` compatibility smokes pass;
|
|
- protected route auth behavior is proven without exposing secrets;
|
|
- ops-hub bootstrap smoke passes;
|
|
- activity-core sink smoke passes or is explicitly deferred to State Hub fallback;
|
|
- data import row counts and relationship checks pass;
|
|
- rollback path is documented;
|
|
- production operator approves cutover.
|
|
|
|
## Migration Gates
|
|
|
|
- Extract Inter-Hub schema and route inventory.
|
|
- Build fixture set from representative production-safe records.
|
|
- Import data into Core Hub staging.
|
|
- Compare row counts and key relationships.
|
|
- Run dual-read or shadow smokes.
|
|
- Switch traffic only after smoke evidence is recorded.
|
|
|
|
## Migration Bundle v1
|
|
|
|
Core Hub staging imports use `core-hub.migration.v1` JSON bundles. The bundle is
|
|
the handoff contract from the Haskell Inter-Hub export path into the Python/FastAPI
|
|
replacement:
|
|
|
|
- `schemaVersion`: `core-hub.migration.v1`
|
|
- `source`: normally `inter-hub-haskell`
|
|
- `exportedAt`: export timestamp for operator traceability
|
|
- `records.hubs`
|
|
- `records.hubCapabilityManifests`
|
|
- `records.apiConsumers`
|
|
- `records.apiKeys`
|
|
- `records.widgets`
|
|
- `records.interactionEvents`
|
|
|
|
API key records may contain `keyPrefix`, `keyHash`, scopes, status, and lifecycle
|
|
metadata. They must not contain full key material, raw tokens, or secrets. Existing
|
|
runtime keys are not recoverable from Inter-Hub unless an approved custody path
|
|
provides them; migration preserves only non-secret evidence.
|
|
|
|
The reference schema is
|
|
`contracts/schemas/migration-bundle.schema.json`, with a minimal fixture at
|
|
`contracts/fixtures/migration/interhub-minimal.bundle.json`.
|
|
|
|
## Import Procedure
|
|
|
|
Validate a bundle without database access:
|
|
|
|
```bash
|
|
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py validate \
|
|
contracts/fixtures/migration/interhub-minimal.bundle.json
|
|
```
|
|
|
|
Dry-run against the configured target database:
|
|
|
|
```bash
|
|
CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
|
|
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
|
|
/secure/operator/path/interhub-export.bundle.json --dry-run
|
|
```
|
|
|
|
Import after the dry-run report is reviewed:
|
|
|
|
```bash
|
|
CORE_HUB_DATABASE_URL=postgresql+asyncpg://... \
|
|
PYTHONPATH=src uv run --extra dev python scripts/core_hub_migrate.py import \
|
|
/secure/operator/path/interhub-export.bundle.json
|
|
```
|
|
|
|
Each successful import writes a `migration_runs` row containing the source,
|
|
schema version, bundle SHA-256, row counts, warnings, and relationship checks.
|
|
|
|
## Rollback Procedure
|
|
|
|
For staging, reset the target database and replay the same bundle after fixing the
|
|
export or mapper. For production cutover, take a database snapshot before import,
|
|
keep Inter-Hub read-only during verification, and roll traffic back to Inter-Hub if
|
|
row counts, relationship checks, or consumer smokes fail. Do not delete or rotate
|
|
legacy key custody records during the rollback window.
|
|
|
|
## Haskell Retirement Gate
|
|
|
|
Do not retire production Inter-Hub or the haskelseed/Haskell build lane until Core Hub serves the required compatibility surface and production traffic has moved.
|