Set up Core Hub framework planning

docs/research/2026-06-27-core-hub-lineage-and-platform-reset.md

@@ -0,0 +1,133 @@
+# Core Hub Lineage and Platform Reset Research
+
+Date: 2026-06-27
+Author: codex
+Status: seed research artifact
+
+## Question
+
+How should Core Hub rebuild the intent of Inter-Hub while retiring Haskell/IHP dependencies and preserving the working infrastructure lessons from State Hub?
+
+## Summary
+
+Core Hub should not be a direct port of the Haskell Inter-Hub codebase. It should be a contract-first rebuild of the production interaction framework idea. The important asset is the framework intent: shared hub manifests, typed events, widgets, registries, API consumers, evidence, and operator surfaces. The costly part was the implementation substrate: Haskell/IHP/Nix/GHC/devenv and the special haskelseed build path.
+
+The recommended posture is to extract the contract, preserve compatibility where current consumers depend on `/api/v2`, then rebuild on the natural Coulomb platform: Python, FastAPI, Pydantic, SQLAlchemy, Alembic, Postgres, pytest, OpenAPI, Docker/Kubernetes, and whynot-design-aligned UI adapters.
+
+## Generation 1: State Hub
+
+State Hub proved several durable ideas:
+
+- Workplans should live in repository files first.
+- The hub can act as a read/cache/index layer over files.
+- Progress events, tasks, messages, and decisions give agents shared operational memory.
+- Simple HTTP/REST plus Postgres is enough to make the coordination loop useful.
+- File-first synchronization makes agent work inspectable and recoverable.
+
+Limits observed:
+
+- State Hub is coordination infrastructure, not a full interaction framework.
+- It does not naturally model rich domain hub manifests, widgets, registry federation, or UI composition.
+- The read-model role should be preserved until a replacement proves compatibility.
+
+## Generation 2: Inter-Hub
+
+Inter-Hub introduced the right higher-level idea: separate domain hubs should publish into a shared framework with manifests, widgets, interaction events, annotations, registries, API consumers, and an integrated operator UI.
+
+Useful surfaces to preserve:
+
+- `/api/v2/hubs`
+- `/api/v2/hub-capability-manifests`
+- `/api/v2/api-consumers`
+- `/api/v2/widgets`
+- `/api/v2/interaction-events`
+- `/api/v2/annotations`
+- `/api/v2/requirement-candidates`
+- `/api/v2/decision-records`
+- `/api/v2/deployment-records`
+- `/api/v2/outcome-signals`
+- `/api/v2/widget-types`
+- `/api/v2/event-types`
+- `/api/v2/annotation-categories`
+- `/api/v2/policy-scopes`
+- `/api/v2/token`
+- `/api/v2/openapi.json`
+- `/api/v2/openapi.yaml`
+- `/api/v2/docs`
+- SDK endpoints if existing consumers still use them
+
+Operational blockers observed:
+
+- Haskell/IHP demanded too much local and CI infrastructure.
+- Nix/GHC/devenv builds were slow and fragile for this environment.
+- The haskelseed path became a production gate instead of an implementation detail.
+- Basic API issues, such as Postgres `COUNT(*)` bigint decoding, became hard to prove live because the build/deploy loop was expensive.
+- UI ambition was tied to a monolithic framework rather than a neutral component contract.
+
+## Haskell Dependency Boundary
+
+Actual Haskell implementations:
+
+1. `inter-hub`: production framework and API service. High-impact. Retire only after Core Hub compatibility, migration, and cutover evidence.
+2. `ihp-railiance-probe`: small IHP/GHC/Nix probe. Low-impact. Can be renamed or archived early.
+
+Haskell support infrastructure:
+
+- haskelseed runner labels and Gitea workflow paths
+- `haskell-build` VM and build-agent capability
+- IHP/GHC/Nix/devenv setup
+- production image build path for Inter-Hub
+
+Protocol consumers that are not Haskell:
+
+- `ops-hub`: Python tooling that calls Inter-Hub `/api/v2` bootstrap endpoints.
+- `activity-core`: Python/FastAPI/Temporal stack with optional Inter-Hub sink and State Hub fallback.
+- `the-custodian`: planning and workplan gates.
+- Other repos mostly reference Inter-Hub protocols, docs, or concepts rather than depending on Haskell directly.
+
+## whynot-design Lesson
+
+The whynot-design direction suggests the right architecture pattern:
+
+- Maintain a canonical design or interaction contract.
+- Derive implementation adapters from that contract.
+- Keep generated or derived layers distinct from hand-authored behavior.
+- Use parity and drift checks rather than assuming codegen solves design.
+- Avoid binding the core model to React, IHP, or any single UI framework.
+
+For Core Hub, the equivalent is:
+
+- Contract/IR: schema, OpenAPI, JSON Schema, event catalogs, capability manifests, fixtures, and compatibility tests.
+- Runtime adapters: FastAPI service, Python clients, UI component adapters, import/export tools.
+- Drift checks: contract tests against legacy Inter-Hub fixtures and known consumers.
+
+## Recommended Core Hub Architecture
+
+Core Hub should be layered:
+
+1. Contract and IR layer: OpenAPI, JSON Schema, SQL/Alembic schema, catalogs, fixtures, and compatibility examples.
+2. Service layer: FastAPI, Pydantic v2 DTOs, SQLAlchemy async models, Alembic migrations, asyncpg/Postgres, and httpx clients.
+3. Compatibility layer: `/api/v2` routes, response-shape tests, and auth/error semantics.
+4. UI layer: operator console, whynot-design tokens/components, and Lit/custom-element adapters where useful.
+5. Migration layer: Inter-Hub schema import/export, row-count checks, fixture replay, and dual-run smokes.
+
+## Key Risks
+
+- Accidentally breaking ops-hub bootstrap endpoints.
+- Losing Inter-Hub data history during migration.
+- Treating API key hashes/prefixes as recoverable secrets. Runtime keys may need approved regeneration.
+- Retiring Haskell repos before production traffic has moved.
+- Recreating framework coupling by making the UI or service implementation the contract.
+
+## Decision Recommendation
+
+Create Core Hub as the new third-generation repo and proceed in stages:
+
+1. Freeze contract and compatibility scope.
+2. Build FastAPI/Postgres baseline.
+3. Implement `/api/v2` compatibility for current consumers.
+4. Migrate data and run side-by-side smokes.
+5. Cut over production DNS/service path.
+6. Rename/archive Haskell repos and retire the build infrastructure.
+
+The IHP probe can be retired first. Production Inter-Hub should be renamed to `inter-hub-haskell` only after Core Hub passes compatibility and migration gates.

docs/specs/README.md

@@ -0,0 +1,25 @@
+# Core Hub Specs
+
+This directory is the specification map for Core Hub. The specs are intentionally ahead of implementation so the framework contract stays stable while the runtime is rebuilt.
+
+## Spec Set
+
+- [Architecture](core-hub-architecture.md) - component model and boundaries
+- [Contract and IR](contract-ir.md) - canonical framework contract, schemas, fixtures, and adapter rules
+- [API v2 Compatibility](api-v2-compatibility.md) - Inter-Hub compatibility surface to preserve during transition
+- [Data Model](data-model.md) - initial entity model and migration posture
+- [Event and Registry Model](event-and-registry-model.md) - event catalogs, manifests, widgets, and capability registries
+- [Auth, Access, and Custody](auth-access-and-custody.md) - API auth semantics and secret routing rules
+- [Workplan Coordination](workplan-coordination.md) - file-first workplans, tasks, progress, messages, decisions
+- [UI and Operator Console](ui-operator-console.md) - whynot-design aligned UI direction
+- [Testing, Release, and Migration](testing-release-and-migration.md) - verification, deployment, and cutover gates
+
+## Contract Rule
+
+The implementation may change. The contract changes only through a workplan and a compatibility note.
+
+Any route, schema, event type, or manifest field used by existing consumers must have one of:
+
+- preserved behavior;
+- documented replacement and migration path;
+- explicit cancellation decision with affected consumers listed.

docs/specs/api-v2-compatibility.md

@@ -0,0 +1,59 @@
+# API v2 Compatibility Spec
+
+## Purpose
+
+Core Hub must preserve the Inter-Hub API surfaces that existing consumers rely on until each consumer has moved to a new explicit Core Hub contract.
+
+## Initial Compatibility Surface
+
+Public/read surfaces:
+
+- `GET /api/v2/hubs`
+- `GET /api/v2/hub-capability-manifests`
+- `GET /api/v2/widget-types`
+- `GET /api/v2/event-types`
+- `GET /api/v2/annotation-categories`
+- `GET /api/v2/policy-scopes`
+- `GET /api/v2/openapi.json`
+- `GET /api/v2/openapi.yaml`
+- `GET /api/v2/docs`
+
+Protected/write or authenticated surfaces:
+
+- `POST /api/v2/token`
+- `/api/v2/api-consumers`
+- `/api/v2/widgets`
+- `/api/v2/interaction-events`
+- `/api/v2/annotations`
+- `/api/v2/requirement-candidates`
+- `/api/v2/decision-records`
+- `/api/v2/deployment-records`
+- `/api/v2/outcome-signals`
+- `/api/v2/hub-registry`
+
+Potentially preserved SDK endpoints:
+
+- `/api/v2/sdk`
+- `/api/v2/sdk/ihf-client.ts`
+- `/api/v2/sdk/ihf-client.py`
+
+## Known Consumers
+
+- `ops-hub` bootstrap and Inter-Hub gate probes
+- `activity-core` optional Inter-Hub sink
+- Custodian workplans and operator smokes
+- Downstream docs or probes that check the public hub registry
+
+## Acceptance Standard
+
+A route is compatible when:
+
+- method and path match;
+- auth behavior matches public/protected expectations;
+- success status and response shape satisfy captured fixtures;
+- error shape is documented;
+- at least one consumer smoke passes against Core Hub.
+
+## Transition Rule
+
+Do not remove an Inter-Hub-compatible route until every known consumer either passes against the new Core Hub-native route or has a recorded cancellation decision.

docs/specs/auth-access-and-custody.md

@@ -0,0 +1,27 @@
+# Auth, Access, and Custody Spec
+
+## Purpose
+
+Define Core Hub authentication behavior and credential boundaries without turning Core Hub into a secret-vending system.
+
+## Rules
+
+- Core Hub may authenticate API clients and record non-secret access metadata.
+- Core Hub must not store raw provider tokens, database passwords, OpenBao tokens, SSH private keys, or login secrets.
+- API key full values are shown only at creation time if Core Hub owns key issuance.
+- Stored API key material must be hashed; logs and progress notes must use prefixes or opaque ids only.
+- Secret retrieval and custody follow ops-warden/OpenBao/key-cape routing.
+
+## Inter-Hub Compatibility
+
+Core Hub should preserve protected-route behavior expected by existing consumers:
+
+- public registry reads should remain public where they are public today;
+- protected write routes should return authorization errors before executing business logic;
+- bootstrap flows should support operator-created or approved API consumers;
+- rate limiting and request logs should use database types compatible with Postgres.
+
+## Open Questions
+
+- Whether token exchange should remain `/api/v2/token` compatible or become a Core Hub-native auth grant endpoint with compatibility alias.
+- Whether API consumers are global, per-hub, or scoped by registry capability.

docs/specs/contract-ir.md

@@ -0,0 +1,39 @@
+# Contract and IR Spec
+
+## Purpose
+
+The Contract/IR layer is the canonical source of truth for Core Hub behavior. Runtime code, UI adapters, generated clients, and migration tools consume this layer.
+
+## Initial Contract Artifacts
+
+- OpenAPI document for HTTP APIs
+- JSON Schemas for manifests, widgets, events, annotations, decisions, deployments, outcomes, and registry facts
+- Alembic migrations and SQL schema snapshots
+- fixture datasets for compatibility tests
+- event type catalog
+- widget type catalog
+- capability manifest schema
+- API auth and error envelope conventions
+
+## Adapter Rule
+
+Adapters may scaffold code or validate drift, but they must not become the contract. A field exists because it is in the Contract/IR, not because a runtime class happens to expose it.
+
+## Compatibility Rule
+
+When migrating from Inter-Hub, capture every preserved route with:
+
+- route and method
+- auth requirement
+- request schema
+- response schema
+- error semantics
+- known consumers
+- fixture examples
+- migration status
+
+## Open Questions
+
+- Whether Core Hub should keep SDK download endpoints or move generated clients to package artifacts.
+- Which admin UI routes from Inter-Hub need API-compatible successors vs. product redesign.
+- Whether State Hub file sync remains external or becomes a first-class Core Hub ingestion mode.

docs/specs/core-hub-architecture.md

@@ -0,0 +1,46 @@
+# Core Hub Architecture Spec
+
+## Goal
+
+Build the production interaction framework as a small set of explicit layers rather than a monolith.
+
+## Layers
+
+1. Contract/IR layer
+   - OpenAPI
+   - JSON Schema
+   - Alembic/SQL schema
+   - event, widget, and capability catalogs
+   - fixtures and compatibility examples
+
+2. Runtime service layer
+   - FastAPI app
+   - Pydantic v2 request/response DTOs
+   - SQLAlchemy async persistence
+   - Alembic migrations
+   - asyncpg/Postgres
+   - httpx clients for integration probes
+
+3. Compatibility layer
+   - Inter-Hub `/api/v2` routes required by current consumers
+   - auth/error compatibility
+   - response-shape fixtures
+   - smoke scripts for ops-hub and activity-core
+
+4. Operator UI layer
+   - dashboard/console surfaces
+   - whynot-design aligned tokens/components
+   - adapter-friendly component model
+
+5. Migration and operations layer
+   - import/export tools
+   - row-count and referential checks
+   - dual-run smokes
+   - release/cutover runbook
+
+## Boundary Decisions
+
+- Core Hub owns the framework contract and primary implementation.
+- Domain hubs own their domain data and domain-specific runtime logic.
+- Credential systems keep owning secrets; Core Hub records non-secret evidence and routing metadata only.
+- State Hub remains active until Core Hub has a proven compatibility and migration path.

docs/specs/data-model.md

@@ -0,0 +1,55 @@
+# Data Model Spec
+
+## Purpose
+
+Define the initial Core Hub entity model and the migration stance from State Hub and Inter-Hub.
+
+## Initial Entity Families
+
+Coordination:
+
+- repositories
+- workstreams
+- workplans
+- tasks
+- progress events
+- messages
+- decisions
+
+Framework:
+
+- hubs
+- hub capability manifests
+- API consumers
+- API keys or key references
+- widgets
+- widget types
+- interaction events
+- annotations and annotation threads
+- requirements
+- decision records
+- deployment records
+- outcome signals
+- policy scopes
+- registry facts
+
+Operations:
+
+- request logs
+- auth audit metadata
+- migration batches
+- compatibility fixtures
+- import/export runs
+
+## Migration Posture
+
+- Import Inter-Hub schema facts through explicit migrations or import tools, not ad hoc SQL editing.
+- Preserve identifiers where consumers or historical references depend on them.
+- Treat API key full values as non-recoverable unless an approved custody path says otherwise.
+- Store only hashes, prefixes, labels, and non-secret lifecycle metadata.
+- Record row counts, relationship checks, and fixture replays for every migration batch.
+
+## Open Questions
+
+- Whether Core Hub owns the canonical workplan task tables or continues reading from State Hub until cutover.
+- Whether Inter-Hub admin-only entities should be migrated as historical records or redesigned as Core Hub-native resources.

docs/specs/event-and-registry-model.md

@@ -0,0 +1,43 @@
+# Event and Registry Model Spec
+
+## Purpose
+
+Define how domain hubs publish structured facts and interaction evidence into Core Hub.
+
+## Event Principles
+
+- Event types are cataloged.
+- Payload schemas are versioned.
+- Producers identify hub, repo, actor, source system, and correlation identifiers where available.
+- Events are append-oriented; derived read models can be rebuilt.
+- Event payloads must not contain secrets.
+
+## Registry Principles
+
+- Capability and manifest records are discoverable.
+- Domain hubs publish what they provide, what they consume, and how operators or agents can interact with them.
+- Registry entries link to source docs and tests when possible.
+- Drift between repo files and Core Hub records is detectable.
+
+## Initial Catalog Families
+
+- hub capability manifests
+- widget types and widget instances
+- interaction event types
+- annotation categories
+- policy scopes
+- API consumers
+- outcome correlations
+- capability registry entries
+
+## Acceptance Standard
+
+A registry family is ready when it has:
+
+- schema;
+- fixture;
+- read endpoint;
+- write/upsert path if needed;
+- validation errors;
+- consumer guidance;
+- at least one compatibility or integration test.

docs/specs/testing-release-and-migration.md

@@ -0,0 +1,35 @@
+# Testing, Release, and Migration Spec
+
+## Test Layers
+
+- unit tests for models, validators, and services
+- API tests for route behavior and auth semantics
+- contract tests from fixtures
+- migration tests for Alembic revisions and imports
+- consumer smokes for ops-hub and activity-core
+- optional Playwright visual checks once UI exists
+
+## Release Gates
+
+A Core Hub release cannot replace Inter-Hub until:
+
+- public `/api/v2` compatibility smokes pass;
+- protected route auth behavior is proven without exposing secrets;
+- ops-hub bootstrap smoke passes;
+- activity-core sink smoke passes or is explicitly deferred to State Hub fallback;
+- data import row counts and relationship checks pass;
+- rollback path is documented;
+- production operator approves cutover.
+
+## Migration Gates
+
+- Extract Inter-Hub schema and route inventory.
+- Build fixture set from representative production-safe records.
+- Import data into Core Hub staging.
+- Compare row counts and key relationships.
+- Run dual-read or shadow smokes.
+- Switch traffic only after smoke evidence is recorded.
+
+## Haskell Retirement Gate
+
+Do not retire production Inter-Hub or the haskelseed/Haskell build lane until Core Hub serves the required compatibility surface and production traffic has moved.

docs/specs/ui-operator-console.md

@@ -0,0 +1,31 @@
+# UI and Operator Console Spec
+
+## Purpose
+
+Define the Core Hub operator interface without coupling the framework to one UI runtime.
+
+## UI Direction
+
+- Use whynot-design tokens and component semantics where possible.
+- Prefer custom-element/Lit adapters for framework-neutral surfaces.
+- Keep the canonical UI contract separate from generated or hand-authored implementation code.
+- Use Playwright visual checks once UI components exist.
+
+## Initial Console Areas
+
+- hub registry and health
+- active workplans and blocked tasks
+- progress/event stream
+- messages and agent coordination
+- decisions and deployment records
+- API consumers and non-secret access metadata
+- compatibility/migration dashboard
+
+## Design Constraint
+
+The operator console is an operational tool. It should be dense, readable, and stable, not a marketing surface.
+
+## Open Questions
+
+- Whether initial UI is server-rendered, Lit/custom-element, or a thin static frontend consuming FastAPI.
+- Which Inter-Hub admin screens should be preserved as compatibility views vs. redesigned.

docs/specs/workplan-coordination.md

@@ -0,0 +1,33 @@
+# Workplan Coordination Spec
+
+## Purpose
+
+Preserve the productive State Hub pattern while Core Hub evolves beyond it.
+
+## File-First Rule
+
+Workplans originate in repo files. Core Hub may index, cache, validate, and display them, but the repo file remains the source of truth until a recorded migration decision changes that rule.
+
+## Core Resources
+
+- workplan files
+- workstreams
+- tasks
+- progress events
+- messages
+- decisions
+- human-needed gates
+- blocking reasons
+- repo briefs
+
+## Agent Requirements
+
+- Agents can orient offline from repo files.
+- Agents can sync to the hub read model after edits.
+- Tasks expose status, priority, and blocking notes.
+- Progress events are non-secret and audit-friendly.
+- Human-needed gates are visible without leaking credentials.
+
+## Compatibility Goal
+
+Core Hub should be able to ingest existing ADR-001 workplans and State Hub records before it attempts to replace State Hub operations.