feat(gems): three-pass schema migration aligning state-hub with GEMS
Implements CUST-WP-0007. Resolves inconsistencies I-1, I-2, I-5, I-6
identified in the GEMS audit (GenericEntityModellingSystem.md).
Pass 1 (e1f2a3b4c5d6): domain_id FK on extension_points and
technical_debt (replaces raw string column); repo_id FK on contributions.
Fixes domain-filtering bugs in EP/TD dashboard pages.
Pass 2 (f2a3b4c5d6e7): repo_id nullable FK on workstreams, aligning
the GEMS primary attachment with ADR-001 (repo > topic). Dashboard
pages updated to prefer repo->domain over topic->domain.
Pass 3 (a3b4c5d6e7f8): SBOMSnapshot container entity (GEMS Complex
between Repository and SBOMEntry). Ingest is now additive — each call
creates a new snapshot; history is retained. List/report endpoints
filter to latest snapshot per repo via _latest_snapshot_ids_subquery().
New endpoints: GET /sbom/snapshots/, GET /sbom/snapshots/{id}/.
Dashboard gains a Snapshot History section.
Also adds GEMS analysis artefacts: wiki/GEMS-StateHub-TypeRegistry.md,
wiki/GEMS-StateHub-SWOT.md, workplans/CUST-WP-0006 (analysis),
workplans/CUST-WP-0007 (migration, now completed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
205
wiki/GEMS-StateHub-SWOT.md
Normal file
205
wiki/GEMS-StateHub-SWOT.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# SWOT Analysis — Migrating State-Hub to GEMS
|
||||
|
||||
Evaluation of migrating the Custodian State Hub data store from its current
|
||||
ad-hoc relational schema to the Generic Entity Modelling System (GEMS) as
|
||||
defined in `wiki/GenericEntityModellingSystem.md` and instantiated in
|
||||
`wiki/GEMS-StateHub-TypeRegistry.md`.
|
||||
|
||||
**Created:** 2026-03-02
|
||||
**Author:** Custodian (analytical session)
|
||||
|
||||
---
|
||||
|
||||
## Migration Options Under Consideration
|
||||
|
||||
Before the SWOT, three architectural options are in scope:
|
||||
|
||||
**Option A — Full Generic Entity Model**
|
||||
Single `entities` table + `attachments` junction + JSONB payload. True GEMS
|
||||
implementation. All current typed tables dissolved into the entity model.
|
||||
|
||||
**Option B — Typed-Table Approach with GEMS Constraints**
|
||||
Keep typed tables (domains, topics, workstreams, etc.) but add:
|
||||
- A universal `entity_id` abstraction layer
|
||||
- An `attachments` junction table for secondary attachments
|
||||
- Application-level GEMS constraint validation
|
||||
- Fix all structural inconsistencies (I-1 through I-8 in CUST-WP-0006)
|
||||
|
||||
**Option C — Incremental Normalization (Pattern C from GEMS §9)**
|
||||
Fix the most critical inconsistencies immediately (I-1, I-2, I-5), leave
|
||||
lesser items wrapped/deferred. No generic entity table introduced.
|
||||
|
||||
---
|
||||
|
||||
## SWOT Analysis
|
||||
|
||||
### Strengths
|
||||
|
||||
**S1 — Uniform modeling surface eliminates special-casing (all options)**
|
||||
Currently each entity type has bespoke FKs, bespoke routers, and bespoke MCP
|
||||
tools. GEMS gives a predictable pattern: every entity has a primary context and
|
||||
optional secondaries. New entity types follow the same pattern with zero
|
||||
schema design work.
|
||||
|
||||
**S2 — Fixes real, observable bugs (Options B and C)**
|
||||
The domain string inconsistency (I-1) causes SBOM and EP/TD dashboard views to
|
||||
silently display wrong or missing domain associations. The Workstream/Topic
|
||||
container mismatch (I-2) causes domain attribution to fail in the Dependencies
|
||||
view. These are current user-visible defects — migration resolves them.
|
||||
|
||||
**S3 — ADR-001 alignment (Options B and C)**
|
||||
ADR-001 mandates that workstreams originate in repos. The current schema forces
|
||||
workstreams under Topics. Migrating Workstream.primary → Repository would bring
|
||||
the schema into conformance with the governing ADR.
|
||||
|
||||
**S4 — Enables first-class graph queries (Option A, partially B)**
|
||||
With Relations as first-class entities, queries like "what decisions influenced
|
||||
which tasks?" or "what dependencies cross domain boundaries?" become uniform
|
||||
and indexable. Currently these require ad-hoc multi-table joins.
|
||||
|
||||
**S5 — Incremental migration is supported by the model (all options)**
|
||||
GEMS §9 explicitly defines integration patterns for existing systems. Pattern C
|
||||
(progressive normalization) allows working systems to remain stable while the
|
||||
most valuable types are migrated first.
|
||||
|
||||
**S6 — Future-proofs multi-domain cross-system queries**
|
||||
As more repositories are registered and domains become interdependent, the
|
||||
current schema's inconsistencies compound. GEMS alignment now prevents
|
||||
exponential complexity accumulation.
|
||||
|
||||
---
|
||||
|
||||
### Weaknesses
|
||||
|
||||
**W1 — Option A requires full schema rewrite (high risk)**
|
||||
Dissolving typed tables into a generic entity model means every router, every
|
||||
MCP tool, every dashboard data loader, and every Alembic migration must be
|
||||
rewritten. This is weeks of work with high regression risk.
|
||||
|
||||
**W2 — Loss of SQL-level type safety (Option A)**
|
||||
Typed tables give the database schema as documentation and enforce type-correct
|
||||
relations at the DB constraint level (FK types, enum columns). A generic entity
|
||||
table with JSONB payloads moves type enforcement to the application layer, which
|
||||
is easier to break silently.
|
||||
|
||||
**W3 — GEMS does not define a concrete SQL schema**
|
||||
The GEMS document is conceptual. Translating the attachment list model into
|
||||
PostgreSQL requires design decisions (indexed JSONB vs. junction table, UUID
|
||||
ordering, etc.) that are not trivial and have performance implications.
|
||||
|
||||
**W4 — ProgressEvent's multi-attach pattern doesn't map cleanly to GEMS**
|
||||
ProgressEvent's current schema (nullable topic_id, workstream_id, task_id,
|
||||
decision_id) is intentionally flexible for an append-only log. GEMS's "exactly
|
||||
one primary attachment" rule may force awkward choices (e.g. always using
|
||||
Workstream as primary even for domain-level events).
|
||||
|
||||
**W5 — Ecosystem root is of uncertain value**
|
||||
Adding an explicit Ecosystem singleton adds ceremony for little practical query
|
||||
benefit in the current six-domain setup. It may become valuable when the system
|
||||
grows to multi-tenant or multi-ecosystem scope, but is premature now.
|
||||
|
||||
---
|
||||
|
||||
### Opportunities
|
||||
|
||||
**O1 — Snapshot diffing for SBOM (SBOMSnapshot entity)**
|
||||
Adding a SBOMSnapshot container (resolves I-5) enables: "what packages were
|
||||
added/removed between ingests?" This is a direct user value feature, not just
|
||||
architectural cleanup.
|
||||
|
||||
**O2 — Unified contribution and decision provenance graph**
|
||||
With Relation entities, you can model "Decision D motivates Workstream W" or
|
||||
"Contribution C implements Decision D" as queryable, auditable edges. This is the
|
||||
foundation for a richer Custodian agent that can reason about the provenance of
|
||||
work items.
|
||||
|
||||
**O3 — Generic dashboard patterns**
|
||||
Once GEMS is in place, dashboard pages can share a single entity-browsing
|
||||
component rather than one bespoke page per entity type. This reduces UI technical
|
||||
debt significantly.
|
||||
|
||||
**O4 — Enabling cross-repo task relations (DependsOn at Repository scope)**
|
||||
With Relations as first-class, it becomes natural to register "Task A in repo X
|
||||
blocks Task B in repo Y" — a cross-repo dependency that the current
|
||||
WorkstreamDependency table cannot model.
|
||||
|
||||
**O5 — Type registry as a self-documenting schema**
|
||||
A GEMS Type Registry is human-readable, machine-validatable, and version-controlled.
|
||||
It replaces the current implicit understanding of "what can be attached to what"
|
||||
with an explicit contract.
|
||||
|
||||
---
|
||||
|
||||
### Threats
|
||||
|
||||
**T1 — Risk of over-engineering a working system**
|
||||
The state-hub currently works well enough for its intended read-model role. A
|
||||
full schema rewrite to achieve theoretical elegance could introduce regressions,
|
||||
stall other domain work, and deliver minimal user-visible value in the short term.
|
||||
|
||||
**T2 — ADR-001 workplan file format would need updates**
|
||||
If Workstream moves from Topic to Repository as its primary container, every
|
||||
existing workplan frontmatter field (`topic_slug`) would need to become or add
|
||||
`repo_slug`. All workplan files across all registered repos require updating.
|
||||
|
||||
**T3 — Hybrid state during incremental migration is confusing**
|
||||
Pattern C leaves the system in a mixed state for an extended period: some
|
||||
entities are GEMS-conformant, others are legacy. Tooling must handle both
|
||||
shapes simultaneously, increasing maintenance burden.
|
||||
|
||||
**T4 — Dashboard rewrites could introduce new bugs**
|
||||
The dashboard is the primary UI for the hub. Rewriting data loaders and query
|
||||
patterns risks introducing visual regressions that would go unnoticed without a
|
||||
test suite (there is currently none for the dashboard).
|
||||
|
||||
**T5 — No migration dry-run tooling exists**
|
||||
The current `make sync-workplans` doesn't exist yet (CUST-WP future deliverable).
|
||||
Running migrations against production data without a rollback path is risky.
|
||||
|
||||
---
|
||||
|
||||
## Verdict and Recommended Path
|
||||
|
||||
**Recommended: Option C — Incremental Normalization**
|
||||
|
||||
Proceed in three targeted passes, each independently releasable:
|
||||
|
||||
**Pass 1 — Fix structural inconsistencies (I-1, I-6): low risk, high consistency gain**
|
||||
- Migrate `ExtensionPoint.domain` (String) → `domain_id` FK + back-fill
|
||||
- Migrate `TechnicalDebt.domain` (String) → `domain_id` FK + back-fill
|
||||
- Add `repo_id` FK to `Contribution` (nullable initially)
|
||||
- This pass has zero API breaking changes; only DB schema and router filter logic change.
|
||||
|
||||
**Pass 2 — Align Workstream with ADR-001 (I-2): medium risk, architectural gain**
|
||||
- Add `repo_id` FK to `Workstream` (nullable initially, then enforce)
|
||||
- Update MCP `create_workstream` to require `repo_id`
|
||||
- Update workplan frontmatter format to include `repo_slug`
|
||||
- Migrate `dependencies.md` to use `repo` instead of `topic` for domain resolution
|
||||
- Decision DEC-GEMS-002 must be resolved before this pass begins
|
||||
|
||||
**Pass 3 — Add SBOMSnapshot container (I-5): medium risk, feature gain**
|
||||
- Add `sbom_snapshots` table + FK from `sbom_entries`
|
||||
- Update ingest API to create/find snapshot per repo+timestamp
|
||||
- Enable snapshot history and diff queries in SBOM dashboard
|
||||
- Decision DEC-GEMS-004 must be resolved before this pass begins
|
||||
|
||||
**Deferred:** Full generic entity model (Option A), Ecosystem root (I-7),
|
||||
DependsOn as first-class Relation (I-8), ManagedRepo.topic_id cleanup (I-4).
|
||||
These are tracked as extension points; revisit after Passes 1-3 are stable.
|
||||
|
||||
---
|
||||
|
||||
## Decision Dependency Map
|
||||
|
||||
```
|
||||
DEC-GEMS-001 (architecture) ──────────────────────────────────► Pass 3+
|
||||
DEC-GEMS-002 (workstream/topic vs repo) ──────────────────────► Pass 2
|
||||
DEC-GEMS-003 (domain string → FK) ────────────────────────────► Pass 1
|
||||
DEC-GEMS-004 (SBOMSnapshot container) ────────────────────────► Pass 3
|
||||
DEC-GEMS-005 (Ecosystem root) ─────────────────────────────────► Deferred
|
||||
DEC-GEMS-006 (DependsOn as Relation entity) ───────────────────► Deferred
|
||||
```
|
||||
|
||||
Pass 1 can begin as soon as DEC-GEMS-003 is resolved (expected: trivially yes).
|
||||
Pass 2 requires DEC-GEMS-002 resolution (breaking change; needs explicit approval).
|
||||
Pass 3 requires DEC-GEMS-004 resolution.
|
||||
134
wiki/GEMS-StateHub-TypeRegistry.md
Normal file
134
wiki/GEMS-StateHub-TypeRegistry.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# GEMS State-Hub Type Registry
|
||||
|
||||
Domain-specific instantiation of the Generic Entity Modelling System
|
||||
(`wiki/GenericEntityModellingSystem.md`) for the Custodian State Hub.
|
||||
|
||||
**Status:** Draft — subject to revision pending decision DEC-GEMS-001 through DEC-GEMS-006.
|
||||
**Created:** 2026-03-02
|
||||
|
||||
---
|
||||
|
||||
## Hierarchy Overview
|
||||
|
||||
```
|
||||
Ecosystem (implicit root — singleton)
|
||||
└── Domain (Complex)
|
||||
├── Topic (Complex) organizes focus areas
|
||||
└── Repository (Complex)
|
||||
├── Workstream (Complex) organizes tasks
|
||||
├── SBOMSnapshot (Complex) organizes SBOM entries
|
||||
├── Task (Atom) ← secondary: Workstream
|
||||
├── Decision (Atom) ← secondary: Topic | Workstream
|
||||
├── TechnicalDebt (Atom)
|
||||
├── ExtensionPoint (Atom)
|
||||
├── Contribution (Atom)
|
||||
└── ProgressEvent (Atom) ← secondary: Workstream | Task | Decision
|
||||
|
||||
SBOMSnapshot
|
||||
└── SBOMEntry (Atom)
|
||||
|
||||
DependsOn (Relation, primary=Domain)
|
||||
from: Workstream → to: Workstream
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Type Registry Table
|
||||
|
||||
| Type | Kind | Primary Attachment Type | Allowed Secondary Attachments | Payload / Key Fields |
|
||||
|---|---|---|---|---|
|
||||
| **Ecosystem** | Complex | — (root) | — | name, description |
|
||||
| **Domain** | Complex | Ecosystem | — | slug, name, status |
|
||||
| **Topic** | Complex | Domain | — | slug, title, status |
|
||||
| **Repository** | Complex | Domain | Topic (optional scope annotation) | slug, name, local_path, remote_url |
|
||||
| **Workstream** | Complex | Repository | Topic (organizer) | slug, title, status, owner, due_date |
|
||||
| **SBOMSnapshot** | Complex | Repository | — | snapshot_at, source |
|
||||
| **Task** | Atom | Workstream | — | title, status, priority, assignee, due_date |
|
||||
| **Decision** | Atom | Repository | Topic \| Workstream (context) | title, type, status, rationale, deadline |
|
||||
| **TechnicalDebt** | Atom | Repository | Topic \| Workstream (context) | td_id, debt_type, severity, status |
|
||||
| **ExtensionPoint** | Atom | Repository | Topic \| Workstream (context) | ep_id, ep_type, priority, status |
|
||||
| **Contribution** | Atom | Repository | — | type, target_org, target_repo, status |
|
||||
| **ProgressEvent** | Atom | Workstream | Task \| Decision (context) | summary, event_type, author |
|
||||
| **SBOMEntry** | Atom | SBOMSnapshot | — | package_name, version, ecosystem, license_spdx |
|
||||
| **DependsOn** | Relation | Domain | — | attachments[1]=from_ws, attachments[2]=to_ws, description |
|
||||
|
||||
---
|
||||
|
||||
## Validation Invariants
|
||||
|
||||
Following GEMS §5.2:
|
||||
|
||||
1. **Primary chain must be acyclic.** No entity may be its own ancestor via primary
|
||||
attachments.
|
||||
|
||||
2. **Primary attachment kind/type must match the registry.** A Task's primary must be
|
||||
a Workstream; a Workstream's primary must be a Repository, etc.
|
||||
|
||||
3. **Context-consistency for secondary attachments.** If Task has a secondary attachment
|
||||
to a Workstream, that Workstream's primary must be the same Repository as the Task's
|
||||
context (inherited via Workstream.primary).
|
||||
|
||||
4. **Relation endpoint types must match the relation's type definition.** DependsOn.from
|
||||
and DependsOn.to must both be Workstream entities within the same Domain.
|
||||
|
||||
5. **Relation primary must be a Complex.** DependsOn.primary = Domain (the relation-space
|
||||
that "owns" the inter-workstream dependency graph).
|
||||
|
||||
---
|
||||
|
||||
## Mapping: Current Tables → GEMS
|
||||
|
||||
| Current Table | Target GEMS Type | Status | Change Required |
|
||||
|---|---|---|---|
|
||||
| `domains` | Domain (Complex) | Correct | None |
|
||||
| `topics` | Topic (Complex) | Correct | None |
|
||||
| `managed_repos` | Repository (Complex) | Mostly correct | Remove nullable topic_id; add optional secondary |
|
||||
| `workstreams` | Workstream (Complex) | **Broken** | Change primary from topic_id to repo_id |
|
||||
| `tasks` | Task (Atom) | Correct | None |
|
||||
| `decisions` | Decision (Atom) | Ambiguous | Change primary to repo_id; topic/workstream become secondaries |
|
||||
| `technical_debt` | TechnicalDebt (Atom) | **Broken** | domain string → repo_id FK |
|
||||
| `extension_points` | ExtensionPoint (Atom) | **Broken** | domain string → repo_id FK |
|
||||
| `contributions` | Contribution (Atom) | Incomplete | Add repo_id FK |
|
||||
| `progress_events` | ProgressEvent (Atom) | Ambiguous | Clarify primary vs. secondary |
|
||||
| `sbom_entries` | SBOMEntry (Atom) | **Broken** | Add SBOMSnapshot container |
|
||||
| `workstream_dependencies` | DependsOn (Relation) | Acceptable | Consider Relation entity model |
|
||||
| *(missing)* | SBOMSnapshot (Complex) | **Missing** | New table required |
|
||||
| *(missing)* | Ecosystem (Complex) | Missing | Optional singleton |
|
||||
|
||||
---
|
||||
|
||||
## Query Patterns Enabled
|
||||
|
||||
After full GEMS alignment:
|
||||
|
||||
```
|
||||
# All workstreams in a domain
|
||||
Workstream WHERE primary.primary.slug = "railiance"
|
||||
|
||||
# All open tasks for a given repo
|
||||
Task WHERE primary.primary.slug = "activity-core" AND status != "done"
|
||||
|
||||
# Dependency graph for a domain
|
||||
DependsOn WHERE primary.slug = "custodian"
|
||||
|
||||
# SBOM history for a repo
|
||||
SBOMSnapshot WHERE primary.slug = "the-custodian" ORDER BY snapshot_at DESC
|
||||
|
||||
# All tech debt in a domain (currently broken — domain is a string)
|
||||
TechnicalDebt WHERE primary.primary.slug = "custodian"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
- `Topic` and `Repository` are both children of `Domain` but are distinct organizers.
|
||||
Topic = "focus area / project agenda"; Repository = "git repo / code artefact boundary".
|
||||
A Topic may align with one or more repositories, but neither owns the other.
|
||||
|
||||
- `Workstream` moving from Topic → Repository is the most disruptive change. It resolves
|
||||
the ADR-001 contradiction (workplans must live in repos, but workstreams live under topics).
|
||||
|
||||
- `ProgressEvent` retains its multi-attach flexibility (topic, workstream, task, decision)
|
||||
but those become secondary attachments. The primary should be the Workstream (or Repository
|
||||
if no workstream context).
|
||||
364
wiki/GenericEntityModellingSystem.md
Normal file
364
wiki/GenericEntityModellingSystem.md
Normal file
@@ -0,0 +1,364 @@
|
||||
## Generic entity modeling system
|
||||
|
||||
A domain-agnostic data modeling system for organizing “entities under management” in a rigorous, flexible, and extensible way.
|
||||
|
||||
### Goals
|
||||
|
||||
* **Rigorous**: clear invariants, predictable querying, safe evolution.
|
||||
* **Flexible**: new entity types and new relations without migrations that rewrite everything.
|
||||
* **Extensible**: supports multiple domains, sub-domains, and incremental adoption over existing data.
|
||||
|
||||
---
|
||||
|
||||
# 1. Core concepts
|
||||
|
||||
## 1.1 Entity
|
||||
|
||||
An **Entity** is the atomic unit of identity and lifecycle.
|
||||
|
||||
**Entity fields (conceptual)**
|
||||
|
||||
* `id` (immutable unique identifier)
|
||||
* `kind` ∈ {`Atom`, `Complex`, `Relation`}
|
||||
* `type` (domain-specific type name, e.g. `Task`, `Repository`, `Customer`)
|
||||
* `payload` (type-specific attributes; ideally versioned)
|
||||
* `attachments` (ordered list of entity references)
|
||||
* `meta` (timestamps, version, permissions, provenance)
|
||||
|
||||
### Entity kinds
|
||||
|
||||
* **Atom**: primary facts / content objects.
|
||||
* **Complex**: organizational containers and structure owners (hierarchy, collections, indexes, contexts).
|
||||
* **Relation**: first-class edge object that encodes a relationship between entities; owned by a Complex.
|
||||
|
||||
---
|
||||
|
||||
# 2. Attachments
|
||||
|
||||
## 2.1 Attachment list
|
||||
|
||||
Every entity has an ordered list:
|
||||
|
||||
* `attachments: [EntityRef]`
|
||||
* `attachments[0]` is the **Primary Attachment**.
|
||||
|
||||
### Derived notion: “Part-of”
|
||||
|
||||
If an entity’s primary attachment is an **Atom**, then the entity is a **Part** of that Atom.
|
||||
|
||||
This is a *classification* derived from data, not a separate stored relation.
|
||||
|
||||
## 2.2 Attachment roles (recommended)
|
||||
|
||||
To avoid ambiguity and allow validation, each attachment can optionally have a role label.
|
||||
|
||||
Conceptually:
|
||||
|
||||
* `attachment = { targetId, position, role }`
|
||||
|
||||
Common roles:
|
||||
|
||||
* `primary` (implicit by position 0)
|
||||
* `index` (entity appears in this complex for navigation/search)
|
||||
* `provenance` (source reference)
|
||||
* `tag` (classification)
|
||||
* `context` (additional scope)
|
||||
|
||||
Ordering remains canonical; roles improve clarity and constraints.
|
||||
|
||||
---
|
||||
|
||||
# 3. Hierarchy and layering
|
||||
|
||||
## 3.1 Primary chain
|
||||
|
||||
The **Primary Chain** of an entity is obtained by repeatedly following `attachments[0]`.
|
||||
|
||||
**Invariant (recommended):** the primary chain must be **acyclic**.
|
||||
|
||||
This yields a robust layering model:
|
||||
|
||||
* Every entity “lives in” a context (a Complex), or is “part of” an Atom.
|
||||
* You can always answer: “Where does this belong?” by walking the primary chain.
|
||||
|
||||
## 3.2 Roots and scopes
|
||||
|
||||
A system should define at least one **root Complex** (e.g. `Ecosystem`, `Workspace`, `Tenant`).
|
||||
|
||||
All managed entities must be reachable from a root by following primary attachments.
|
||||
|
||||
---
|
||||
|
||||
# 4. Relations as first-class entities
|
||||
|
||||
## 4.1 Relation entity
|
||||
|
||||
A **Relation** is an entity whose purpose is to define a connection among other entities.
|
||||
|
||||
**Key rule**
|
||||
|
||||
* A Relation’s **primary attachment MUST be a Complex**.
|
||||
That Complex is the **relation-space** (the context that “owns” the relationship).
|
||||
|
||||
This avoids “atoms knowing” relation details: atoms remain content, complexes and relations hold structure.
|
||||
|
||||
## 4.2 Relation endpoints convention
|
||||
|
||||
To make relations queryable and consistent, standardize attachment slots:
|
||||
|
||||
* `attachments[0] = contextComplex` (primary; relation-space)
|
||||
* `attachments[1] = fromEndpoint`
|
||||
* `attachments[2] = toEndpoint`
|
||||
* `attachments[3..] = optional extra endpoints` (evidence, via, stakeholder, etc.)
|
||||
|
||||
Relation semantics live in:
|
||||
|
||||
* `type` (e.g. `DependsOn`, `Implements`, `References`)
|
||||
* and/or payload fields like `{ relType: "...", strength: ..., rationale: ... }`
|
||||
|
||||
---
|
||||
|
||||
# 5. Type system and constraints
|
||||
|
||||
## 5.1 Entity Type Registry
|
||||
|
||||
Maintain a registry of types describing:
|
||||
|
||||
* `kind`: Atom/Complex/Relation
|
||||
* allowed primary attachment kinds/types
|
||||
* allowed secondary attachment kinds/types
|
||||
* payload schema (optional but recommended)
|
||||
* indexing / query defaults
|
||||
|
||||
Example (conceptual):
|
||||
|
||||
* `Task`: kind=Atom, primary must be `Repository` (Complex)
|
||||
* `Repository`: kind=Complex, primary must be `Domain` (Complex)
|
||||
|
||||
## 5.2 Validation invariants (recommended minimum)
|
||||
|
||||
1. **Exactly one primary attachment** (position 0).
|
||||
2. **Primary chain must be acyclic**.
|
||||
3. **Primary attachment kind/type constraints** must match the registry.
|
||||
4. **Context-consistency constraints** for organizer complexes:
|
||||
|
||||
* if `Task` has a secondary attachment to `Workstream`,
|
||||
then `Task.primary == Workstream.primary` (same repository).
|
||||
5. **Relation constraints**:
|
||||
|
||||
* primary must be Complex
|
||||
* endpoint types must match relation type definition
|
||||
* relation context must match endpoint context rules (usually same repo/domain)
|
||||
|
||||
These constraints give rigor without hard-coding a single domain model.
|
||||
|
||||
---
|
||||
|
||||
# 6. Query model (domain-agnostic)
|
||||
|
||||
These queries exist in any domain:
|
||||
|
||||
## 6.1 Locate context
|
||||
|
||||
* `context(entity)` = walk primary chain to root, or to the nearest scope boundary (e.g. nearest Domain/Workspace).
|
||||
|
||||
## 6.2 Membership
|
||||
|
||||
* Members of a Complex: all entities with `primary == complexId`.
|
||||
|
||||
## 6.3 Parts of an Atom
|
||||
|
||||
* Parts of an Atom: all entities with `primary == atomId`.
|
||||
|
||||
## 6.4 Relations in a relation-space
|
||||
|
||||
* Relations owned by a Complex: all Relation entities with `primary == complexId`.
|
||||
|
||||
## 6.5 Neighborhood (graph view)
|
||||
|
||||
* For entity X: find all relations in the same relation-space where X appears as endpoint.
|
||||
|
||||
---
|
||||
|
||||
# 7. Example domain: Ecosystem → Domain → Repository → Workstreams/SBOMs
|
||||
|
||||
This section makes the system concrete using your types.
|
||||
|
||||
## 7.1 Complexes
|
||||
|
||||
* `Ecosystem` (Complex, root)
|
||||
* `Domain` (Complex, primary = Ecosystem)
|
||||
* `Repository` (Complex, primary = Domain)
|
||||
* `Workstream` (Complex, primary = Repository) — organizes work items
|
||||
* `SBOM` (Complex, primary = Repository) — organizes dependencies
|
||||
|
||||
## 7.2 Atoms
|
||||
|
||||
* `Decision` (Atom, primary = Repository)
|
||||
* `Task` (Atom, primary = Repository)
|
||||
* `TechDebt` (Atom, primary = Repository)
|
||||
* `Extend` (Atom, primary = Repository)
|
||||
* `Dependency` (Atom, primary = Repository)
|
||||
|
||||
## 7.3 Organizing via secondary attachments
|
||||
|
||||
* A Task in a Workstream:
|
||||
|
||||
* `Task.attachments = [Repo42, Workstream7]`
|
||||
* A Dependency in an SBOM:
|
||||
|
||||
* `Dependency.attachments = [Repo42, Sbom3]`
|
||||
|
||||
Atoms remain ignorant of *how* the workstream orders tasks; the workstream can store structure.
|
||||
|
||||
## 7.4 Relation examples
|
||||
|
||||
### Task → Task dependency (repo-scoped)
|
||||
|
||||
Relation type: `DependsOn` (Relation)
|
||||
|
||||
* `DependsOn.attachments = [Repo42, TaskA, TaskB]`
|
||||
* payload: `{ critical: true, reason: "API contract needed first" }`
|
||||
|
||||
### Decision influences tasks (repo-scoped)
|
||||
|
||||
Relation type: `Motivates` (Relation)
|
||||
|
||||
* `Motivates.attachments = [Repo42, Decision9, TaskA]`
|
||||
|
||||
### Dependency graph inside an SBOM (sbom-scoped)
|
||||
|
||||
Relation type: `Requires` (Relation)
|
||||
|
||||
* `Requires.attachments = [Sbom3, DependencyX, DependencyY]`
|
||||
* payload: `{ scope: "runtime" }`
|
||||
|
||||
This cleanly separates:
|
||||
|
||||
* planning relations (Repo relation-space)
|
||||
* supply-chain relations (SBOM relation-space)
|
||||
|
||||
---
|
||||
|
||||
# 8. Applying the modeling system to a new domain
|
||||
|
||||
You can apply this to any domain by following a small method.
|
||||
|
||||
## 8.1 Step-by-step method
|
||||
|
||||
### Step 1 — Choose a root Complex
|
||||
|
||||
Pick the top-level scope:
|
||||
|
||||
* `Workspace`, `Tenant`, `Organization`, `Ecosystem`, etc.
|
||||
|
||||
### Step 2 — Identify “containers” vs “content”
|
||||
|
||||
* Containers become **Complexes** (projects, folders, accounts, repositories, case files).
|
||||
* Content objects become **Atoms** (documents, customers, invoices, tickets, assets).
|
||||
|
||||
Rule of thumb:
|
||||
|
||||
* If it *organizes* others or defines a scope, it’s a Complex.
|
||||
* If it’s a “thing” with intrinsic content/lifecycle, it’s an Atom.
|
||||
|
||||
### Step 3 — Define the primary hierarchy (layering)
|
||||
|
||||
Decide what “belongs to what” as the default place where entities live.
|
||||
Example pattern:
|
||||
|
||||
* `Atom.primary = nearest containing Complex`
|
||||
|
||||
### Step 4 — Define organizer complexes (optional)
|
||||
|
||||
Introduce complexes like `Workstream`, `Board`, `Collection`, `SBOM`, `Timeline` that provide structure.
|
||||
Use **secondary attachments** from atoms to these complexes.
|
||||
|
||||
### Step 5 — Define relation-spaces
|
||||
|
||||
Choose where relations live:
|
||||
|
||||
* typically in the “owning” complex (project/repo/case)
|
||||
* sometimes in a specialized complex (SBOM, timeline, graph)
|
||||
|
||||
### Step 6 — Create a Type Registry + constraints
|
||||
|
||||
For each type, specify:
|
||||
|
||||
* kind
|
||||
* required primary attachment type(s)
|
||||
* optional secondary attachment types
|
||||
* allowed relation endpoints (if relation type)
|
||||
|
||||
### Step 7 — Migrate incrementally
|
||||
|
||||
Start with primary attachments and identity first.
|
||||
Add organizer complexes and relations later without breaking identity.
|
||||
|
||||
---
|
||||
|
||||
# 9. Applying it to an existing domain with pre-existing entities
|
||||
|
||||
The key is to **wrap** existing entities as Entities in this system without rewriting them all at once.
|
||||
|
||||
## 9.1 Integration patterns
|
||||
|
||||
### Pattern A — “Entity wrapper” over existing tables/documents
|
||||
|
||||
* Keep existing storage unchanged.
|
||||
* Create an `Entity` record that references external storage:
|
||||
|
||||
* payload contains `{ externalType, externalId, sourceSystem }`
|
||||
* Attachments, relations, and organization are managed in the new layer.
|
||||
|
||||
This is the safest “overlay” approach.
|
||||
|
||||
### Pattern B — “Dual write” for new objects
|
||||
|
||||
* New entities are created in the new model as the source of truth.
|
||||
* Optionally mirrored into legacy storage for compatibility.
|
||||
|
||||
### Pattern C — “Progressive normalization”
|
||||
|
||||
* Start overlay-style.
|
||||
* Gradually move the most valuable types (e.g., Tasks, Decisions) into native entities.
|
||||
* Leave rarely touched legacy objects wrapped indefinitely.
|
||||
|
||||
## 9.2 Migration steps for existing data
|
||||
|
||||
1. **Assign stable IDs**
|
||||
|
||||
* If legacy IDs exist, reuse them with a namespace prefix.
|
||||
2. **Create root complexes**
|
||||
|
||||
* e.g. one `Ecosystem` or per-tenant `Workspace`.
|
||||
3. **Attach existing entities to a primary context**
|
||||
|
||||
* even if initially coarse (everything attaches to one domain/project).
|
||||
4. **Introduce finer complexes**
|
||||
|
||||
* split into domains, repos/projects later by moving primary attachments.
|
||||
5. **Add relations incrementally**
|
||||
|
||||
* create relation entities for the relationships you query most.
|
||||
6. **Backfill organizer complexes**
|
||||
|
||||
* workstreams, boards, SBOMs, etc., via secondary attachments.
|
||||
|
||||
Because relations and organization are additive, you can evolve structure without breaking identity.
|
||||
|
||||
---
|
||||
|
||||
# 10. What this system buys you
|
||||
|
||||
* A **uniform modeling surface** across domains.
|
||||
* A **clean separation** of content (atoms) from structure (complexes + relations).
|
||||
* **Multiple overlapping organizations** via secondary attachments without duplication.
|
||||
* **First-class relationships** with auditability and contextual ownership.
|
||||
* **Incremental adoption** over legacy systems.
|
||||
|
||||
## Extension Points
|
||||
|
||||
This could be turned into a compact “spec” format (like a small RFC) plus a concrete “Type Registry” table for your example (including recommended relation types and constraints).
|
||||
|
||||
xxx
|
||||
Reference in New Issue
Block a user