research: Roam Research deep dive (block-graph DataScript DB, transclusion, datalog, Roam Depot extension API); UC-50/51/52

The modern bookend to the Xanadu/ZigZag dives: where those are unbuilt
ideals, Roam shipped fine-grained addressing (:block/uid), live
transclusion (block embeds), bidirectional links, and a queryable
structured space (DataScript datoms + Datalog). Studied as a candidate
DB-backed/API-attached shard (XWiki family) and as a concrete
engine-hosts-adapter surface (Roam Depot onload/onunload over
window.roamAlphaAPI). Added UC-50 (attach block-graph DB shard, block<->page
mapping), UC-51 (adopt native span IDs as portable span addresses), UC-52
(delegate derived views to a shard's native query engine); enriched
UC-32/34/35/38. Boundary: Roam is one candidate shard mapped into the
Markdown-first page model, not a substrate and not the federation layer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 11:36:50 +02:00
parent 7c22c468af
commit dfff9ab42e
5 changed files with 447 additions and 9 deletions

View File

@@ -6,7 +6,8 @@ Promoted from `research/260608-c2-wiki-origins/`,
`research/260608-yawex-prior-art/`, `research/260608-federation-concepts/`,
`research/260608-wikiengines-overview/`, `research/260613-xwiki-deep-dive/`,
`research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`,
`research/260614-xanadu-deep-dive/`, and `research/260614-zigzag-deep-dive/`.
`research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`, and
`research/260614-roam-deep-dive/`.
See InfoTechPrimers on coulomb.social for use-case catalog conventions.
## Conventions
@@ -188,7 +189,11 @@ projection. Provenance and staleness must be explicit. Xanadu deep dive sharpens
transclusion should be **content-identity based and bidirectional** (content is "knowably
in more than one place" and aware of its appearances), not a one-way path fetch —
enables UC-45 reverse lookup (`research/260614-xanadu-deep-dive/findings.md` §4). See
UC-44 for the whole-page composition-manifest form.
UC-44 for the whole-page composition-manifest form. **Roam ships this**: block embeds
transclude by `:block/uid` live (not copied), proving transclusion is a cheap data-layer
capability over an addressable store (`research/260614-roam-deep-dive/findings.md` §3,
§7) — argues for transclusion in core over the union, surfaced by UI. Depends on span
addressing (UC-51).
**Priority:** Later
### UC-33 — Git-branch an information space
@@ -217,7 +222,10 @@ XObjects against an XClass schema (`research/260613-xwiki-deep-dive/findings.md`
§2.3); UC-39 covers the extreme where the page is *only* structure. TWiki shows the
**git-friendly** variant: TWiki Forms store fields as `%META:FIELD%` *inside the
topic text file*, so structure is diffable rather than locked in a DB
(`research/260613-twiki-deep-dive/findings.md` §2.3).
(`research/260613-twiki-deep-dive/findings.md` §2.3). Roam is the modern exemplar:
`key:: value` attributes over a datom graph, queryable via Datalog
(`research/260614-roam-deep-dive/findings.md` §3, §4) — structure to preserve via
sidecar metadata, and a candidate for query delegation (UC-52).
**Priority:** Later
### UC-35 — Attach a shard with coarse write granularity
@@ -229,7 +237,10 @@ respect that granularity instead of assuming per-page writes.
**Source:** wikiengines, intent
**Notes:** Capability-aware adapters: the **capability profile must encode write
granularity** (per-page / per-file / per-space / append-only). Overlay and
patch flows must adapt (findings §5 #1). Affects conflict scope and locking.
patch flows must adapt (findings §5 #1). Affects conflict scope and locking. Roam marks
the **fine extreme** — block-level writes (`block.create/update/move/delete`), the
opposite of TiddlyWiki's whole-file granularity
(`research/260614-roam-deep-dive/findings.md` §6).
**Priority:** Later
### UC-36 — Supply a git-addressable history to an internal-history engine
@@ -278,7 +289,11 @@ surfacing — enough to host a high-fidelity adapter
adapter (needs deploy access, higher fidelity) is open (findings §8 Q3).
Generalizes beyond XWiki: TWiki's plugin handler API (`beforeSaveHandler`,
`afterRenameHandler`, REST handlers) is an equivalent host surface
(`research/260613-twiki-deep-dive/findings.md` §3).
(`research/260613-twiki-deep-dive/findings.md` §3). **Roam Depot** is the modern
template: an extension default-exports `onload`/`onunload` and drives
`window.roamAlphaAPI` read (`q`/`pull`) + write (`block.*`/`page.create`)
(`research/260614-roam-deep-dive/findings.md` §5) — note Roam's API runs *inside* the
client, so write-through requires in-engine hosting (findings §11 Q2).
**Priority:** Later
### UC-39 — Attach a wiki-as-application-platform shard (pages as typed records)
@@ -443,6 +458,51 @@ genealogy edge recorded at fork/remix/import time (UC-26, UC-29) in the coordina
journal. Makes provenance a traversal, not a buried footer (complements UC-24).
**Priority:** Later
### UC-50 — Attach a block-graph database wiki as a shard, mapping blocks to the page model
**Actor:** Maintainer
**Goal:** Attach a block-first, DB-backed graph tool (Roam-style) as a shard via its
query/CRUD API, mapping its blocks onto shard-wiki's Markdown page model without
flattening the structure.
**Source:** roam, intent
**Notes:** Roam's graph is a DataScript datom DB where a *page* is the node carrying
`:node/title` and nested *blocks* are addressable spans
(`research/260614-roam-deep-dive/findings.md` §2, §6). Clean mapping rule: titled node
= page, nested blocks = spans, attributes = sidecar metadata (honors UC-34 no-lossy-
flatten). DB-backed/API-attached path (like XWiki UC-38), **not** the file-store path
(UC-40). Write-through requires hosting the adapter inside the engine (Roam Depot),
else degrade to read/projection/overlay-target (findings §11 Q2).
**Priority:** Later
### UC-51 — Adopt a shard's native span IDs as portable span addresses
**Actor:** Orchestrator / adapter
**Goal:** Where a backend mints stable sub-page identifiers (Roam `:block/uid`), adopt
them as the span address for transclusion, overlay, and reverse-lookup, rather than
inventing one.
**Source:** roam, xanadu, intent
**Notes:** Roam ships the fine-grained stable address that Xanadu's tumblers
(`research/260614-xanadu-deep-dive/findings.md` §3) and the catalog left open — a short
opaque per-block UID, public and referenceable (`research/260614-roam-deep-dive/findings.md`
§2, §7). Enables UC-44/45 at sub-page granularity; falls back to content-fingerprint
(UC-46) or path+range where no native ID exists. Open: use native ID directly or wrap in
a shard-scoped address to avoid cross-shard collision and survive projection (findings
§11 Q1).
**Priority:** Later
### UC-52 — Delegate derived views to a shard's native query engine
**Actor:** Orchestrator
**Goal:** When a shard exposes a native query engine, compute derived views (backlinks,
recent changes, typed queries) by delegating to it instead of scanning the projection.
**Source:** roam, intent
**Notes:** In Roam, derived views *are* Datalog queries over the datom graph
(`research/260614-roam-deep-dive/findings.md` §4). Makes "native query" an adapter
capability the orchestrator can key off (vs. computing views itself over the
projection). Operationalizes the ZigZag "dimensions + rasters" insight
(`research/260614-zigzag-deep-dive/findings.md` §5) and relates to UC-05/UC-34.
**Priority:** Later
---
## B. Knowledge work and collaboration
@@ -678,6 +738,9 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD.
| UC-47 | | | ‡ | | ✓ |
| UC-48 | | | ‡ | | |
| UC-49 | | | ‡ | | ✓ |
| UC-50 | | | | § | ✓ |
| UC-51 | | | | § | ✓ |
| UC-52 | | | | § | ✓ |
| UC-08 | ✓ | | |
| UC-09 | ✓ | | |
| UC-10 | ✓ | | |
@@ -823,6 +886,36 @@ Architecture logged for `SHARD-WP-0002`: a dimensional projection layer; genealo
edges in the coordination journal; clone↔transclusion as a possible shared primitive
(findings §9). Pairs with the Xanadu dive (clone ↔ transclusion convergence).
### roam mapping
(§ UC-50UC-52 are placed in the **wikiengines** matrix column as the nearest existing
source — Roam is a shipped tool — but their true lineage is the **Roam deep dive**,
`research/260614-roam-deep-dive/findings.md`.)
| Roam mechanism (findings §) | Catalog UC |
|-----------------------------|------------|
| Block-graph DataScript DB; page = `:node/title` node, blocks = spans (§2, §6) | UC-50 |
| `:block/uid` — shipped stable sub-page address (§2, §7) | UC-51 |
| Derived views = Datalog queries over the graph (§4) | UC-52 |
| Block embeds = shipped live transclusion by uid (§3) | UC-32 (enriched; + UC-44/45) |
| `key:: value` attributes + Datalog = structured/typed pages (§3, §4) | UC-34 (enriched; + UC-39) |
| Roam Depot `onload/onunload` + `roamAlphaAPI` = engine-hosts-adapter template (§5) | UC-38 (enriched) |
| Block-level write granularity = the fine extreme (§6) | UC-35 (enriched) |
| Hosted history is internal-only, not portable Git (§6, §8.2) | links UC-36 |
Note: Roam is the **modern bookend** to the Nelson dives — where Xanadu and ZigZag are
unbuilt ideals, Roam *shipped* fine-grained addressing, transclusion, bidirectional
links, and a queryable structured space. Its payoff is **empirical evidence** on
shard-wiki's open questions (span addressing is tractable via native block IDs;
transclusion is a data-layer capability; derived views are queries —
`research/260614-roam-deep-dive/findings.md` §7). **Boundary recorded:** Roam is *one
candidate shard* — DB-backed / API-attached (the XWiki family, not the file-store
family), block-first, with no portable Git history — **mapped into** shard-wiki's
Markdown-first page model, not adopted as a substrate and never the federation layer
(findings §8.2). Architecture logged for `SHARD-WP-0002` (T14): native-span-ID and
native-query capabilities, block↔page mapping, and Roam as a second
engine-hosts-adapter exemplar alongside XWiki.
---
## Open questions
@@ -840,4 +933,7 @@ edges in the coordination journal; clone↔transclusion as a possible shared pri
7. Is the **dimensional/zzstructure model** (UC-47/48) a public navigation API and UI
paradigm, or only an internal organizing concept for the derived views?
8. How is the **many-to-many link graph** reconciled with the rank-based dimensional
model — clones, a parallel graph index, or both? (ZigZag dive §10.)
model — clones, a parallel graph index, or both? (ZigZag dive §10.)
9. Is **in-engine adapter hosting** (e.g. a Roam Depot extension for write-through) an
accepted deployment mode generally, or do we restrict API-only backends to
read/projection/overlay-target? (Roam dive §11 Q2; ties UC-38, UC-50.)