core-hub/docs/specs/ops-evidence-contract.md

Ops Evidence Contract

Purpose

Define the Core Hub-owned operations evidence contract that replaces the old standalone ops-hub model list from CUST-WP-0025-T14.

This contract reconciles the Custodian service inventory lane, Core Hub v2 registry resources, activity-core probe output, migration/cutover evidence, and non-secret credential-custody rules. It is intentionally API-first: CLI and UI surfaces must consume these resources instead of inventing their own evidence shape.

Scope

In scope:

• runtime service and endpoint readiness evidence;
• Core Hub registry/bootstrap evidence for ops-hub replacement work;
• activity-core Core Hub sink proof;
• migration import, deployment, outcome, and cutover readiness evidence;
• non-secret credential and access metadata.

Out of scope for this contract:

• secret value retrieval or vending;
• SSH certificate issuance;
• operator approval itself;
• the full future ops-hub product boundary if post-cutover work proves a separate service is needed.

Source Inputs

Source	Current artifact	Core Hub use
Custodian service inventory	CUST-WP-0047 service/location/evidence records	Seed widgets and interaction events; preserve source ids in metadata.
Core Hub v2 resources	hubs, manifests, API consumers, API keys, widgets, events, migration runs	Canonical API/read model for replacement evidence.
activity-core probe output	core-hub-interaction-event sink reports	Append interaction_events with source run and verification metadata.
Migration tooling	scripts/core_hub_migrate.py, MigrationRun	Store bundle hash, source, dry-run flag, counts, diagnostics, status.
Operator CLI	deployed-smoke, ops-bootstrap-status, migration, readiness-summary	Non-secret evidence report generator over the same API resources.

Canonical API Resources

Implemented Now

Resource	API path	Purpose	Required non-secret fields
Hub	/api/v2/hubs	Domain hub or replacement runtime record.	id, slug, name, domain, hubKind, hubFamily, vsmFunction, vsmSystem, status.
Hub capability manifest	/api/v2/hub-capability-manifests	Declared widgets/events/scopes/endpoints for a hub.	id, hubId, hubSlug, manifestVersion, status, declared widget/event/policy lists.
API consumer	/api/v2/api-consumers	Runtime or operator consumer metadata.	id, slug, name, hubCapabilityManifestId, rate/quota values, keyPrefix, status.
API key issuance result	/api/v2/api-consumers/{consumer_id}/api-keys	One-time generated key plus stored hash/prefix.	returned full key only at creation; persist only id, consumer id, keyPrefix, hash, scopes, status.
Widget	/api/v2/widgets	Operator-facing evidence target/read model item.	id, hubId, name, widgetType, capabilityRef, viewContext, policyScope, status.
Interaction event	/api/v2/interaction-events	Append-only operational evidence.	id, widgetId, eventType, viewContext, metadata, createdAt if exposed.
Hub registry	/api/v2/hub-registry	Read model for hubs and manifests.	hubs[], hubCapabilityManifests[].
Migration run	stored as migration_runs	Import/validate evidence.	id, source, schemaVersion, bundleSha256, dryRun, status, counts, diagnostics, createdAt.

Required Read-Model Gaps

These are the gaps that must close before broad UI expansion or cutover claims:

Gap	Required path or behavior	Why it matters
Migration run read API	GET /api/v2/migration-runs and optional GET /api/v2/migration-runs/{id}	UI and readiness reports need to inspect import evidence without reading the database directly.
Deployment records	Replace deferred empty /api/v2/deployment-records with persisted non-secret deployment facts.	Deployed smoke and cutover readiness need image/tag/runtime state evidence.
Outcome signals	Replace deferred empty /api/v2/outcome-signals with persisted pass/fail/degraded signals.	Operator screens need to distinguish evidence from result interpretation.
Service inventory mapping	Persist or derive a mapping from Custodian service ids to Core Hub widget ids/events.	Lets CUST-WP-0047 inventory become Core Hub evidence without losing provenance.
Interaction-event filters	Add query filters for hubId, widgetId, eventType, viewContext, and time window.	Needed for compact evidence streams and activity-core sink verification.
Registry containment summary	Expose counts and containment booleans now computed by smoke reports.	Avoids each CLI/UI consumer re-deriving the same readiness checks.

Evidence Vocabulary

Event types should be explicit and versioned through the catalog once promoted. Initial operational event types:

Event type	Producer	Meaning	Required metadata
ops-endpoint-verified	deployed smoke, ops probe, operator CLI	A service endpoint or registry path responded as expected.	runId, source, endpoint label, status code or health state, verified boolean.
core-hub-deployed-smoke	scripts/core_hub_deployed_smoke.py	Deployed Core Hub API smoke result.	runId, hub id/slug, manifest id/status, API consumer id/status/keyPrefix, widget id/count, event id/type, registry containment booleans.
core-hub-activity-sink-verified	activity-core	activity-core posted and verified a Core Hub interaction event.	runId, widget id, event id, eventType, status, verified boolean, source workplan/task if available.
core-hub-migration-validated	migration CLI	A bundle validation completed without import.	source, schemaVersion, bundleSha256, dryRun true, counts, diagnostics summary.
core-hub-migration-imported	migration CLI	A non-dry-run import completed.	source, schemaVersion, bundleSha256, dryRun false, counts, diagnostics summary, migrationRunId.
core-hub-cutover-gate	operator CLI	A readiness summary evaluated a gate.	gate name, ok boolean, reason, evidence ids/counts, report path or State Hub progress id.

All event metadata and body fields must be non-secret. Allowed access metadata:

• key prefixes;
• key hash algorithm labels or stored hash ids;
• OpenBao path/version labels only when they do not reveal values;
• route catalog ids;
• token accessor ids only when explicitly approved by the owning subsystem;
• Kubernetes Secret name and populated-key count, not secret data.

Forbidden fields:

• API keys, bearer tokens, SSH private keys, database passwords, OIDC refresh tokens, cookie values, raw Authorization headers, or copied Secret manifests;
• one-time API key material after the creation response has left the operator process;
• plaintext excerpts from secret files or OpenBao responses.

Service Inventory Mapping

CUST-WP-0047 service inventory rows map into Core Hub as follows:

Inventory concept	Core Hub target
service id / slug	Hub.slug when the service is itself a hub; otherwise Widget.body.serviceId.
service display name	Hub.name or Widget.name.
service domain/location	Hub.domain, Widget.viewContext, and event metadata.
evidence type	InteractionEvent.eventType.
evidence source path	InteractionEvent.metadata.sourcePath or sourceWorkplan.
readiness or blocker state	Widget.status, event metadata, and later outcome signals.
credential route	metadata route id only, never a credential value.

Mapping rules:

• Keep original inventory ids in metadata so State Hub/Custodian references stay traceable.
• Prefer one widget per operator-visible evidence target, not one widget per log line.
• Append interaction events for observations; only change widget status when the latest stable operator-facing state changes.
• Activity-core may append events to existing widgets only after the widget id mapping is explicitly configured.

Readiness Summary Inputs

readiness-summary is the cutover-facing aggregation. Required gates:

Gate	Source report	Required evidence
deployed_api_smoke	make deployed-smoke or operator CLI deployed-smoke	ok true, runId, check count, event id.
staging_import	migration import report	ok true, dryRun false, bundleSha256, counts, migrationRunId.
activity_core_sink	activity-core sink report	at least one posted and verified event.

Optional but useful gate:

• legacy_inter_hub_reference: explicit rollback/compatibility evidence while Inter-Hub remains available.

Cutover is not ready until every required gate is true and the operator records an approval decision outside this contract.

API / CLI / UI Order

1. API resources and read routes are canonical.
2. CLI wrappers generate and summarize evidence over the same API.
3. UI screens are read-heavy views over API and CLI-compatible summaries.

The /console rebuild must not read migration tables directly once the migration-runs API exists.

Implementation Backlog

Priority	Item	Acceptance
P0	Add migration run read routes.	/api/v2/migration-runs lists stored runs with source, bundle hash, dryRun, status, counts, diagnostics summary.
P0	Add interaction-event filters.	API supports filtering by event type, widget id, view context, and time window.
P0	Define service inventory import fixture.	A fixture maps at least one Custodian inventory service into a widget plus event with provenance metadata.
P1	Persist deployment records.	/api/v2/deployment-records stops returning only an empty compatibility collection.
P1	Persist outcome signals.	/api/v2/outcome-signals can store non-secret pass/fail/degraded signals linked to events or migration runs.
P1	Registry readiness summary route.	API exposes counts/containment booleans used by deployed smoke and UI readiness overview.

Acceptance For CUST-WP-0025-T14

This spec closes the T14 definition gate. Implementation remains in follow-up Core Hub tasks and the deployed evidence gates:

• CUST-WP-0025-T16 for deployed Core Hub and activity-core smokes;
• CUST-WP-0025-T17 for cutover/legacy coupling;
• CUST-WP-0025-T18 for the UI rebuild after API/CLI evidence is stable;
• CORE-WP-0005 for staging import and cutover readiness;
• future Core Hub work for the P0/P1 read-model gaps above.