inter-hub/docs/research/ops-hub-evidence-intake-current-state.md

# Ops Hub Evidence Intake - Current State

Date: 2026-06-15

Workplan: `IHUB-WP-0022`

## Summary

Inter-Hub has the generic v2 API surface needed for activity-core evidence
intake, but the activity-core path is not live yet. The safe implementation
slice is therefore contract-first:

- document the ops-hub widget mapping shape;
- document the Inter-Hub event payload shape;
- keep `OPS_HUB_KEY` outside Git;
- accept State Hub fallback as the temporary safety path;
- wait on live ops-hub manifest/widgets, key provisioning, and production
  smoke before enabling per-entity Inter-Hub submission.

## Inter-Hub API Surface

The current repo supports the necessary primitives through `/api/v2`.

`Web.Controller.Api.V2.Widgets`:

- `GET /api/v2/widgets` is authenticated and paginated.
- `POST /api/v2/widgets` requires `hubId`, `name`, and `widgetType`.
- Optional fields are `capabilityRef`, `viewContext`, `policyScope`,
  `status`, and `adapterSpecId`.
- `policyScope` defaults to `internal` when omitted.
- Valid widget statuses are `active`, `deprecated`, and `draft`.
- Widget type and policy scope are validated through the type registries.
- Widget creation creates an initial `WidgetVersion` snapshot.

`Web.Controller.Api.V2.InteractionEvents`:

- `GET /api/v2/interaction-events` is authenticated and paginated.
- Supported list filters are `widgetId` and `eventType`.
- `POST /api/v2/interaction-events` requires `widgetId` and `eventType`.
- `viewContext` is optional and is persisted as `viewContextRef`.
- `metadata` is accepted as a JSON object when the request content type is
  `application/json`.
- The event type must exist in `event_type_registry`.
- If the API consumer is bound to an active manifest, the event type must also
  be declared by that manifest.
- `occurredAt` is server-set. Activity-core should send its observed timestamp
  inside `metadata.attributes.observed_at`.
- Actor attribution is `actorType = "api"` for this endpoint.

`docs/new-hub-quickstart.md` and `scripts/ops-hub-bootstrap-smoke.py` already
show the bootstrap shape for a single ops-hub endpoint event. The activity-core
intake needs that pattern expanded from one smoke widget/event to a durable
five-event contract.

## Activity-Core Contract

The neighboring `activity-core` repo already defines the intended event
vocabulary under `event-types/`:

- `ops-service-observed`
- `ops-endpoint-verified`
- `ops-access-path-checked`
- `ops-backup-verified`
- `ops-inventory-drift`

The current activity-core sink implementation is intentionally conservative:

- `state-hub-progress` is implemented and idempotent.
- It posts `ops_inventory_probe` progress with compact non-secret detail.
- The idempotency key is `activity_core_run_id + context_key + event_type`.
- The compact probe strips raw response bodies, headers, credentials, URL query
  strings, and token-like material.
- Inter-Hub sink names are recognized, but the sink currently returns
  `missing_inter_hub_config` or `inter_hub_sink_deferred`; it does not submit
  events yet.
- Inter-Hub mode requires `INTER_HUB_URL`, `OPS_HUB_KEY`, and either
  `OPS_HUB_WIDGET_MAPPING`, `widget_mapping`, or `capability_mapping`.

Activity-core deployment placeholders exist in
`activity-core/k8s/railiance/20-runtime.yaml`:

- `INTER_HUB_URL` is present but empty.
- `OPS_HUB_WIDGET_MAPPING` is present but empty.
- `OPS_HUB_KEY` is created only as an empty Secret placeholder by
  `bootstrap-secrets.sh`.

## Fallback Evidence State

State Hub was queried directly for live fallback evidence:

```text
GET http://127.0.0.1:8000/progress/?event_type=ops_inventory_probe&limit=20
```

Result on 2026-06-15: an empty list.

That means the fallback sink is implemented and tested in activity-core, but no
live `ops_inventory_probe` progress event is available for Inter-Hub to accept
as closure evidence yet.

## Production Gates

Known gates before per-entity Inter-Hub submission can be treated as live:

1. The production Inter-Hub deployment must include commit `5101eb5` or an
   equivalent fix for PostgreSQL `COUNT(*)` decoding in widget creation and
   API rate-limit reads.
2. The active `ops-hub` manifest must declare the five activity-core event
   types, the selected widget types, the annotation category, and the policy
   scope.
3. Seed widgets named by the mapping contract must exist in the target
   environment.
4. `OPS_HUB_KEY` must be provisioned outside Git, preferably in OpenBao at
   `platform/operators/ops-hub/runtime`, field `OPS_HUB_KEY`.
5. Activity-core must receive `INTER_HUB_URL`, `OPS_HUB_KEY`, and
   `OPS_HUB_WIDGET_MAPPING` through its runtime config/Secret path.
6. A controlled smoke must submit one event for each declared event type and
   verify that an undeclared event type is rejected.

## Recommended Manifest Vocabulary

Use one policy scope for the first slice:

- `ops-evidence`

Use one annotation category:

- `ops-risk`

Use these widget types unless the operator prefers to keep a smaller aggregate
surface:

- `ops-service-card`
- `ops-endpoint-card`
- `ops-access-path-card`
- `ops-backup-card`
- `ops-drift-card`

Use the activity-core event types exactly as published:

- `ops-service-observed`
- `ops-endpoint-verified`
- `ops-access-path-checked`
- `ops-backup-verified`
- `ops-inventory-drift`

## Open Questions

- Does ops-hub already have a production manifest that should be patched rather
  than replaced?
- Should the first production mapping use only aggregate widgets, or seed
  per-entity widgets for the known Railiance inventory?
- Which OpenBao or cluster Secret path should activity-core consume for
  `OPS_HUB_KEY`?
- Should activity-core close `ACTIVITY-WP-0007/T06` after a live State Hub
  fallback event with explicit Inter-Hub deferral, or only after real
  Inter-Hub submission?