tegwick
8ab6e6c9c5
Implements CUST-WP-0007. Resolves inconsistencies I-1, I-2, I-5, I-6
identified in the GEMS audit (GenericEntityModellingSystem.md).
Pass 1 (e1f2a3b4c5d6): domain_id FK on extension_points and
technical_debt (replaces raw string column); repo_id FK on contributions.
Fixes domain-filtering bugs in EP/TD dashboard pages.
Pass 2 (f2a3b4c5d6e7): repo_id nullable FK on workstreams, aligning
the GEMS primary attachment with ADR-001 (repo > topic). Dashboard
pages updated to prefer repo->domain over topic->domain.
Pass 3 (a3b4c5d6e7f8): SBOMSnapshot container entity (GEMS Complex
between Repository and SBOMEntry). Ingest is now additive — each call
creates a new snapshot; history is retained. List/report endpoints
filter to latest snapshot per repo via _latest_snapshot_ids_subquery().
New endpoints: GET /sbom/snapshots/, GET /sbom/snapshots/{id}/.
Dashboard gains a Snapshot History section.
Also adds GEMS analysis artefacts: wiki/GEMS-StateHub-TypeRegistry.md,
wiki/GEMS-StateHub-SWOT.md, workplans/CUST-WP-0006 (analysis),
workplans/CUST-WP-0007 (migration, now completed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
271 lines
6.5 KiB
Markdown
271 lines
6.5 KiB
Markdown
---
|
|
id: CUST-WP-0007
|
|
type: workplan
|
|
title: GEMS Migration — Three-Pass State-Hub Alignment
|
|
domain: custodian
|
|
status: completed
|
|
owner: custodian
|
|
topic_slug: the-custodian
|
|
repo_slug: the-custodian
|
|
created: 2026-03-02
|
|
updated: 2026-03-02
|
|
state_hub_workstream_id: "22e18151-fc83-438c-b732-10e056e64a20"
|
|
---
|
|
|
|
# CUST-WP-0007 — GEMS Migration: Three-Pass State-Hub Alignment
|
|
|
|
Implements the migration decided in CUST-WP-0006. Fixes all structural
|
|
inconsistencies identified in the GEMS audit (I-1 through I-6) in three
|
|
independently releasable passes.
|
|
|
|
Decisions resolved: DEC-GEMS-001 (Option C), DEC-GEMS-002, DEC-GEMS-003,
|
|
DEC-GEMS-004. DEC-GEMS-005 and DEC-GEMS-006 deferred.
|
|
|
|
---
|
|
|
|
## Pass 1 — Fix Domain FK Inconsistencies
|
|
|
|
**Scope:** Resolve I-1 and I-6. No API breaking changes. Fixes observable
|
|
dashboard domain-filtering bugs on EP/TD pages.
|
|
|
|
**Alembic migration ID:** `e1f2a3b4c5d6` (down_revision: `d3e4f5a6b7c8`)
|
|
|
|
### Task T01: Alembic migration — Pass 1
|
|
|
|
```task
|
|
id: T01
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Operations:
|
|
1. Add `domain_id` UUID FK (nullable) to `extension_points`
|
|
2. Add `domain_id` UUID FK (nullable) to `technical_debt`
|
|
3. Add `repo_id` UUID FK (nullable) to `contributions`
|
|
4. Backfill `extension_points.domain_id` from `domains.slug` match
|
|
5. Backfill `technical_debt.domain_id` from `domains.slug` match
|
|
6. Make `domain_id` NOT NULL on both tables (all rows must be backfilled first)
|
|
7. Drop `domain` String column from `extension_points`
|
|
8. Drop `domain` String column from `technical_debt`
|
|
|
|
### Task T02: Update ExtensionPoint and TechnicalDebt models
|
|
|
|
```task
|
|
id: T02
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Replace `domain: Mapped[str]` with `domain_id: Mapped[uuid.UUID]` FK +
|
|
`domain: Mapped["Domain"]` relationship. Add domain_slug property.
|
|
|
|
### Task T03: Update Contribution model
|
|
|
|
```task
|
|
id: T03
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
Add `repo_id: Mapped[uuid.UUID | None]` nullable FK to `managed_repos`.
|
|
|
|
### Task T04: Update EP and TD routers
|
|
|
|
```task
|
|
id: T04
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
- Filter by `domain_id` FK instead of domain string
|
|
- Accept `domain` slug in create/filter params, resolve to `domain_id`
|
|
|
|
### Task T05: Update MCP EP/TD tools
|
|
|
|
```task
|
|
id: T05
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
`register_extension_point` and `register_technical_debt` still accept
|
|
`domain` as a slug string. Router resolves to `domain_id` FK.
|
|
|
|
### Task T06: Update EP/TD dashboard pages
|
|
|
|
```task
|
|
id: T06
|
|
status: done
|
|
priority: medium
|
|
assignee: custodian
|
|
```
|
|
|
|
`extensions.md` and `techdept.md` load domain list from `/domains/` API and
|
|
use `domain_id` FK for filtering. Remove reliance on string comparison.
|
|
|
|
---
|
|
|
|
## Pass 2 — Align Workstream with ADR-001
|
|
|
|
**Scope:** Resolve I-2. Breaking change to workstream schema. `topic_id`
|
|
becomes a nullable secondary annotation; `repo_id` becomes the primary FK.
|
|
Workplan frontmatter format gains `repo_slug` field.
|
|
|
|
**Alembic migration ID:** `f2a3b4c5d6e7` (down_revision: `e1f2a3b4c5d6`)
|
|
|
|
### Task T07: Alembic migration — Pass 2
|
|
|
|
```task
|
|
id: T07
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Operations:
|
|
1. Add `repo_id` UUID FK (nullable) to `workstreams`
|
|
2. Backfill `repo_id` using heuristic: workstream → topic → domain → first
|
|
repo for that domain (adequate for current data; all custodian workstreams
|
|
map to the-custodian repo)
|
|
3. For topics without a repo: leave nullable (MCP tooling handles this)
|
|
|
|
### Task T08: Update Workstream model
|
|
|
|
```task
|
|
id: T08
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Add `repo_id: Mapped[uuid.UUID | None]` nullable FK to `managed_repos`.
|
|
Keep `topic_id` as nullable secondary. Add `repo` relationship.
|
|
|
|
### Task T09: Update workstream router and MCP tools
|
|
|
|
```task
|
|
id: T09
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
- `create_workstream` MCP tool: add optional `repo_id` / `repo_slug` param
|
|
- Workstream read schema: expose `repo_id` and `repo_slug`
|
|
- Dependency resolution in `state/summary` uses `repo.domain` when available
|
|
|
|
### Task T10: Update workplan frontmatter format
|
|
|
|
```task
|
|
id: T10
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
Add `repo_slug` field to ADR-001 workplan frontmatter spec. Update existing
|
|
workplan files (CUST-WP-0001 through CUST-WP-0006) to include `repo_slug`.
|
|
|
|
### Task T11: Update Dependencies dashboard domain resolution
|
|
|
|
```task
|
|
id: T11
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
`dependencies.md` currently resolves domain via `topicMap[w.topic_id]?.domain_slug`.
|
|
Change to prefer `wsMap[w.id]?.repo?.domain_slug` when available.
|
|
|
|
---
|
|
|
|
## Pass 3 — SBOMSnapshot Container
|
|
|
|
**Scope:** Resolve I-5. Adds `sbom_snapshots` as a container entity between
|
|
Repository and SBOMEntry. Enables snapshot history and diff queries.
|
|
|
|
**Alembic migration ID:** `a3b4c5d6e7f8` (down_revision: `f2a3b4c5d6e7`)
|
|
|
|
### Task T12: Alembic migration — Pass 3
|
|
|
|
```task
|
|
id: T12
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Operations:
|
|
1. Create `sbom_snapshots` table (id, repo_id FK, snapshot_at, source, created_at)
|
|
2. Add `snapshot_id` UUID FK (nullable) to `sbom_entries`
|
|
3. Backfill: for each (repo_id, snapshot_at) group in sbom_entries, create one
|
|
sbom_snapshots row; set snapshot_id on all matching entries
|
|
4. Make `snapshot_id` NOT NULL on `sbom_entries`
|
|
5. Consider: drop `repo_id` from `sbom_entries` (reachable via snapshot)
|
|
|
|
### Task T13: Add SBOMSnapshot model
|
|
|
|
```task
|
|
id: T13
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
New model `api/models/sbom_snapshot.py` with FK to managed_repos.
|
|
|
|
### Task T14: Update SBOMEntry model
|
|
|
|
```task
|
|
id: T14
|
|
status: done
|
|
priority: critical
|
|
assignee: custodian
|
|
```
|
|
|
|
Add `snapshot_id` FK to `sbom_snapshots`. Update `repo` relationship to go
|
|
via snapshot.
|
|
|
|
### Task T15: Update SBOM router and ingest API
|
|
|
|
```task
|
|
id: T15
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
- Ingest creates/finds a snapshot record, then creates entries under it
|
|
- New endpoints: `GET /sbom/snapshots/`, `GET /sbom/snapshots/{id}/`
|
|
- Existing `GET /sbom/` still returns flat entries for backward compatibility
|
|
|
|
### Task T16: Update MCP ingest tool and SBOM resources
|
|
|
|
```task
|
|
id: T16
|
|
status: done
|
|
priority: high
|
|
assignee: custodian
|
|
```
|
|
|
|
`ingest_sbom_tool` returns `snapshot_id` in result. New MCP resource:
|
|
`state://sbom/snapshots/{repo_slug}`.
|
|
|
|
### Task T17: Update SBOM dashboard
|
|
|
|
```task
|
|
id: T17
|
|
status: done
|
|
priority: medium
|
|
assignee: custodian
|
|
```
|
|
|
|
`sbom.md` "By Repo" section adds a snapshot history row showing ingest dates
|
|
with package count delta from previous snapshot.
|