the-custodian/wiki/GenericEntityModellingSystem.md

## Generic entity modeling system

A domain-agnostic data modeling system for organizing “entities under management” in a rigorous, flexible, and extensible way.

### Goals

* **Rigorous**: clear invariants, predictable querying, safe evolution.
* **Flexible**: new entity types and new relations without migrations that rewrite everything.
* **Extensible**: supports multiple domains, sub-domains, and incremental adoption over existing data.

---

# 1. Core concepts

## 1.1 Entity

An **Entity** is the atomic unit of identity and lifecycle.

**Entity fields (conceptual)**

* `id` (immutable unique identifier)
* `kind` ∈ {`Atom`, `Complex`, `Relation`}
* `type` (domain-specific type name, e.g. `Task`, `Repository`, `Customer`)
* `payload` (type-specific attributes; ideally versioned)
* `attachments` (ordered list of entity references)
* `meta` (timestamps, version, permissions, provenance)

### Entity kinds

* **Atom**: primary facts / content objects.
* **Complex**: organizational containers and structure owners (hierarchy, collections, indexes, contexts).
* **Relation**: first-class edge object that encodes a relationship between entities; owned by a Complex.

---

# 2. Attachments

## 2.1 Attachment list

Every entity has an ordered list:

* `attachments: [EntityRef]`
* `attachments[0]` is the **Primary Attachment**.

### Derived notion: “Part-of”

If an entity’s primary attachment is an **Atom**, then the entity is a **Part** of that Atom.

This is a *classification* derived from data, not a separate stored relation.

## 2.2 Attachment roles (recommended)

To avoid ambiguity and allow validation, each attachment can optionally have a role label.

Conceptually:

* `attachment = { targetId, position, role }`

Common roles:

* `primary` (implicit by position 0)
* `index` (entity appears in this complex for navigation/search)
* `provenance` (source reference)
* `tag` (classification)
* `context` (additional scope)

Ordering remains canonical; roles improve clarity and constraints.

---

# 3. Hierarchy and layering

## 3.1 Primary chain

The **Primary Chain** of an entity is obtained by repeatedly following `attachments[0]`.

**Invariant (recommended):** the primary chain must be **acyclic**.

This yields a robust layering model:

* Every entity “lives in” a context (a Complex), or is “part of” an Atom.
* You can always answer: “Where does this belong?” by walking the primary chain.

## 3.2 Roots and scopes

A system should define at least one **root Complex** (e.g. `Ecosystem`, `Workspace`, `Tenant`).

All managed entities must be reachable from a root by following primary attachments.

---

# 4. Relations as first-class entities

## 4.1 Relation entity

A **Relation** is an entity whose purpose is to define a connection among other entities.

**Key rule**

* A Relation’s **primary attachment MUST be a Complex**.
  That Complex is the **relation-space** (the context that “owns” the relationship).

This avoids “atoms knowing” relation details: atoms remain content, complexes and relations hold structure.

## 4.2 Relation endpoints convention

To make relations queryable and consistent, standardize attachment slots:

* `attachments[0] = contextComplex` (primary; relation-space)
* `attachments[1] = fromEndpoint`
* `attachments[2] = toEndpoint`
* `attachments[3..] = optional extra endpoints` (evidence, via, stakeholder, etc.)

Relation semantics live in:

* `type` (e.g. `DependsOn`, `Implements`, `References`)
* and/or payload fields like `{ relType: "...", strength: ..., rationale: ... }`

---

# 5. Type system and constraints

## 5.1 Entity Type Registry

Maintain a registry of types describing:

* `kind`: Atom/Complex/Relation
* allowed primary attachment kinds/types
* allowed secondary attachment kinds/types
* payload schema (optional but recommended)
* indexing / query defaults

Example (conceptual):

* `Task`: kind=Atom, primary must be `Repository` (Complex)
* `Repository`: kind=Complex, primary must be `Domain` (Complex)

## 5.2 Validation invariants (recommended minimum)

1. **Exactly one primary attachment** (position 0).
2. **Primary chain must be acyclic**.
3. **Primary attachment kind/type constraints** must match the registry.
4. **Context-consistency constraints** for organizer complexes:

   * if `Task` has a secondary attachment to `Workstream`,
     then `Task.primary == Workstream.primary` (same repository).
5. **Relation constraints**:

   * primary must be Complex
   * endpoint types must match relation type definition
   * relation context must match endpoint context rules (usually same repo/domain)

These constraints give rigor without hard-coding a single domain model.

---

# 6. Query model (domain-agnostic)

These queries exist in any domain:

## 6.1 Locate context

* `context(entity)` = walk primary chain to root, or to the nearest scope boundary (e.g. nearest Domain/Workspace).

## 6.2 Membership

* Members of a Complex: all entities with `primary == complexId`.

## 6.3 Parts of an Atom

* Parts of an Atom: all entities with `primary == atomId`.

## 6.4 Relations in a relation-space

* Relations owned by a Complex: all Relation entities with `primary == complexId`.

## 6.5 Neighborhood (graph view)

* For entity X: find all relations in the same relation-space where X appears as endpoint.

---

# 7. Example domain: Ecosystem → Domain → Repository → Workstreams/SBOMs

This section makes the system concrete using your types.

## 7.1 Complexes

* `Ecosystem` (Complex, root)
* `Domain` (Complex, primary = Ecosystem)
* `Repository` (Complex, primary = Domain)
* `Workstream` (Complex, primary = Repository) — organizes work items
* `SBOM` (Complex, primary = Repository) — organizes dependencies

## 7.2 Atoms

* `Decision` (Atom, primary = Repository)
* `Task` (Atom, primary = Repository)
* `TechDebt` (Atom, primary = Repository)
* `Extend` (Atom, primary = Repository)
* `Dependency` (Atom, primary = Repository)

## 7.3 Organizing via secondary attachments

* A Task in a Workstream:

  * `Task.attachments = [Repo42, Workstream7]`
* A Dependency in an SBOM:

  * `Dependency.attachments = [Repo42, Sbom3]`

Atoms remain ignorant of *how* the workstream orders tasks; the workstream can store structure.

## 7.4 Relation examples

### Task → Task dependency (repo-scoped)

Relation type: `DependsOn` (Relation)

* `DependsOn.attachments = [Repo42, TaskA, TaskB]`
* payload: `{ critical: true, reason: "API contract needed first" }`

### Decision influences tasks (repo-scoped)

Relation type: `Motivates` (Relation)

* `Motivates.attachments = [Repo42, Decision9, TaskA]`

### Dependency graph inside an SBOM (sbom-scoped)

Relation type: `Requires` (Relation)

* `Requires.attachments = [Sbom3, DependencyX, DependencyY]`
* payload: `{ scope: "runtime" }`

This cleanly separates:

* planning relations (Repo relation-space)
* supply-chain relations (SBOM relation-space)

---

# 8. Applying the modeling system to a new domain

You can apply this to any domain by following a small method.

## 8.1 Step-by-step method

### Step 1 — Choose a root Complex

Pick the top-level scope:

* `Workspace`, `Tenant`, `Organization`, `Ecosystem`, etc.

### Step 2 — Identify “containers” vs “content”

* Containers become **Complexes** (projects, folders, accounts, repositories, case files).
* Content objects become **Atoms** (documents, customers, invoices, tickets, assets).

Rule of thumb:

* If it *organizes* others or defines a scope, it’s a Complex.
* If it’s a “thing” with intrinsic content/lifecycle, it’s an Atom.

### Step 3 — Define the primary hierarchy (layering)

Decide what “belongs to what” as the default place where entities live.
Example pattern:

* `Atom.primary = nearest containing Complex`

### Step 4 — Define organizer complexes (optional)

Introduce complexes like `Workstream`, `Board`, `Collection`, `SBOM`, `Timeline` that provide structure.
Use **secondary attachments** from atoms to these complexes.

### Step 5 — Define relation-spaces

Choose where relations live:

* typically in the “owning” complex (project/repo/case)
* sometimes in a specialized complex (SBOM, timeline, graph)

### Step 6 — Create a Type Registry + constraints

For each type, specify:

* kind
* required primary attachment type(s)
* optional secondary attachment types
* allowed relation endpoints (if relation type)

### Step 7 — Migrate incrementally

Start with primary attachments and identity first.
Add organizer complexes and relations later without breaking identity.

---

# 9. Applying it to an existing domain with pre-existing entities

The key is to **wrap** existing entities as Entities in this system without rewriting them all at once.

## 9.1 Integration patterns

### Pattern A — “Entity wrapper” over existing tables/documents

* Keep existing storage unchanged.
* Create an `Entity` record that references external storage:

  * payload contains `{ externalType, externalId, sourceSystem }`
* Attachments, relations, and organization are managed in the new layer.

This is the safest “overlay” approach.

### Pattern B — “Dual write” for new objects

* New entities are created in the new model as the source of truth.
* Optionally mirrored into legacy storage for compatibility.

### Pattern C — “Progressive normalization”

* Start overlay-style.
* Gradually move the most valuable types (e.g., Tasks, Decisions) into native entities.
* Leave rarely touched legacy objects wrapped indefinitely.

## 9.2 Migration steps for existing data

1. **Assign stable IDs**

   * If legacy IDs exist, reuse them with a namespace prefix.
2. **Create root complexes**

   * e.g. one `Ecosystem` or per-tenant `Workspace`.
3. **Attach existing entities to a primary context**

   * even if initially coarse (everything attaches to one domain/project).
4. **Introduce finer complexes**

   * split into domains, repos/projects later by moving primary attachments.
5. **Add relations incrementally**

   * create relation entities for the relationships you query most.
6. **Backfill organizer complexes**

   * workstreams, boards, SBOMs, etc., via secondary attachments.

Because relations and organization are additive, you can evolve structure without breaking identity.

---

# 10. What this system buys you

* A **uniform modeling surface** across domains.
* A **clean separation** of content (atoms) from structure (complexes + relations).
* **Multiple overlapping organizations** via secondary attachments without duplication.
* **First-class relationships** with auditability and contextual ownership.
* **Incremental adoption** over legacy systems.

## Extension Points

This could be turned into a compact “spec” format (like a small RFC) plus a concrete “Type Registry” table for your example (including recommended relation types and constraints).

xxx