# Findings — Roam Research: the block graph as a queryable database, and its extension API Date: 2026-06-14 Source kind: **modern shipped product** — a block-graph note tool; a *candidate shard backend* (DB-backed / API-attached, like XWiki), and the place where several Nelson ideas actually shipped Lens: shard-wiki — fine-grained addressing, transclusion, bidirectional links, a structured/queryable page space, and the engine-hosts-adapter extension path > Why Roam, and why now. The previous two dives were Ted Nelson's *unbuilt* ideals — > Xanadu (`research/260614-xanadu-deep-dive/`: reference-not-copy, transclusion, > stable fine-grained addresses) and ZigZag (`research/260614-zigzag-deep-dive/`: an > information space as many co-equal dimensions over a queryable structure). **Roam > Research is where a large slice of both actually shipped to a mainstream audience.** > Every block has a stable address; block references and embeds are working > transclusion; the whole graph is a queryable database; links are bidirectional. So > Roam is the natural modern bookend: it lets us check shard-wiki's open questions > (portable span addressing, transclusion, structured pages) against a system that > solved them *in production* — and study its **extension architecture** as a concrete > adapter-host surface. This dive treats Roam as a *candidate shard* (capability profile in §6) and as *evidence* on the open questions the Nelson dives left (§7), not as something to copy wholesale — Roam is block-first and DB-backed, shard-wiki is Markdown-page-first and Git-backed (divergences in §8.2). --- ## 1. What Roam is, in architecture terms Roam Research is a networked-thought outliner whose **entire graph is a client-side DataScript database** (DataScript = a ClojureScript reimplementation of Datomic's data model and Datalog query engine). The store is not files; it is a set of immutable **datoms**. - A **datom** is one fact: `[entity, attribute, value, transaction]` — the EAV(T) model. The graph is the accumulation of datoms; edits are transactions. - Synchronization and Roam's deep undo fall out of the transaction log — a parallel worth noting to Git's immutable commit history (but it is *not* Git; see §8.2). This makes Roam a **structured, queryable** wiki-shaped store — the same family as XWiki (objects + DB) rather than TWiki/Foswiki (flat files), which matters for how an adapter attaches (§6). --- ## 2. The block model — everything is a block, every block is addressable Every paragraph *and* every page is a **block** (an entity). Key attributes: | Attribute | Meaning | |-----------|---------| | `:block/uid` | **nine-character public ID** — the block's stable, referenceable address (e.g. `((GGv3cyL6Y))`) | | `:block/string` | the block's text content | | `:block/order` | position among siblings | | `:block/children` | immediate child entity-ids | | `:block/parents` | full ancestor chain (incl. the page) | | `:block/refs` | edges to blocks/pages this block references | | `:block/page` | the owning page entity | | `:create/time`, `:edit/time`, `:create/email`, `:edit/email` | provenance metadata | | `:node/title` | **page-only** — the page's title (this attribute is what distinguishes a page from an ordinary block) | Structurally the DB is a **forest**: each page is a tree of nested blocks; a "page" is just the top-level block that carries `:node/title`. The single most shard-wiki-relevant fact: **`:block/uid` gives every block — i.e. every sub-page span — a stable, first-class, public address.** This is the *shipped* form of the Xanadu tumbler / the shard-wiki "portable span address" open question (§7). --- ## 3. References, transclusion, bidirectional links - **Page references** `[[Page]]` and `#tags`, and **block references** `((uid))`, all create `:block/refs` edges. The link graph is therefore queryable, not parsed-on-read. - **Block embeds** (`{{embed: ((uid))}}`) are **working transclusion**: a block's content rendered live in another location by reference, not copied — when the source changes, the embed reflects it. This is Xanadu transclusion / shard-wiki UC-32/UC-44, shipped at block granularity. - **Bidirectional links** come in two flavours, exactly shard-wiki's BackLinks problem: - **Linked references** — explicit `:block/refs` edges (queryable backlinks). - **Unlinked references** — bare text mentions with no edge, surfaced by text search. - **Attributes** `key:: value` are a convention on top of blocks/refs that turns a page into a lightweight record — Roam's answer to structured/typed pages (UC-34/UC-39), queryable via Datalog. --- ## 4. Querying — Datalog over the graph Roam exposes **Datalog** (`:find` / `:where`) plus **pull** expressions: ``` [:find ?uid :where [?b :block/string ?s] [?b :block/uid ?uid] ...] ``` This means Roam's "derived views" (a page's linked references, a `{{query}}` block, a table) are **just queries over the datom graph** — not bespoke features. For shard-wiki this is the strongest evidence yet that *derived views = queries over a structured space* (the ZigZag "dimensions + rasters" insight, made executable), and that an adapter could **delegate view computation to a shard's native query engine** instead of scanning (UC-52). --- ## 5. Extension architecture — Roam Depot and `window.roamAlphaAPI` Roam's official extension system (**Roam Depot**) is a concrete model of the *engine-hosts-adapter* path (UC-38): - **Distribution:** an extension is a GitHub repo (`README.md`, `extension.js`, optional `extension.css` / `CHANGELOG.md` / `build.sh`), cataloged in the `roam-depot` repo via a metadata JSON (`name`, `author`, `tags`, `source_repo`, `source_commit`, optional `stripe_account` for paid extensions). - **Lifecycle:** `extension.js` default-exports a map with **`onload`** and **`onunload`** functions; everything created on load must be torn down on unload. An `extensionAPI` object provides a **settings panel** and `settings.get/set`. - **Data API — `window.roamAlphaAPI`** (the surface an adapter would target): - **Read:** `q` (Datalog query → entity ids), `pull` / `pull-many` (fetch entity attributes), full graph traversal. - **Write:** `data.block.create` / `update` / `move` / `delete`, `data.page.create`. - **UI:** command palette, blocks, settings. Crucially, this API runs **inside the Roam client** (global `window.roamAlphaAPI`). There is no first-class external REST write API; programmatic external access uses graph **export** (EDN / JSON / Markdown) or unofficial private APIs. So a *live, write-through* Roam adapter must be **hosted inside Roam as a Depot extension** — which is exactly the UC-38 engine-side direction (cf. XWiki components, TWiki plugin handlers). --- ## 6. Roam as a shard — capability profile | Capability | Roam | Notes for the adapter contract | |------------|------|--------------------------------| | Read | **yes** | `q`/`pull` live (in-app) or EDN/JSON/Markdown export (snapshot) | | Write | **yes, in-app** | `block.create/update/move/delete`, `page.create` — but only via the in-client API → needs a Depot-hosted adapter (UC-38) or unofficial private API | | Write granularity | **block-level (fine)** | the opposite extreme from TiddlyWiki's whole-file writes — sharpens UC-35 | | Identity / addressing | **block `:block/uid` + page `:node/title`** | shipped sub-page stable addressing (UC-51) | | Transclusion | **yes** | block embeds by uid (UC-32/44/45) | | Backlinks | **yes** | linked (`:block/refs`) + unlinked (text) (UC-05/18) | | Structured data | **yes** | attributes + Datalog (UC-34/39) | | Native query | **yes (Datalog)** | adapter could delegate views (UC-52) | | Diff / merge | **no native** | transaction log exists but not exposed as page diffs | | Version history | **hosted, internal-only** | not portable Git history → needs supplement/import (UC-36, like Confluence/MediaWiki) | | Lock | **no** | — | | Syntax | **Roam-flavored Markdown** | not CommonMark; block-outline structure → translation/mapping needed (cf. UC-42) | Verdict: Roam is a legitimate **DB-backed, API-attached shard** — the XWiki family, not the file-store family. Its standout offerings are **block-level addressing** and **shipped transclusion**; its costs are **block↔page impedance** and **no portable history**. --- ## 7. Evidence on shard-wiki's open questions (the payoff) The Nelson dives left open questions; Roam answers several *empirically*: 1. **"What is a portable fine-grained span address?"** (Xanadu §11 Q1, the tumbler problem). Roam's answer: **a short opaque per-block UID, minted by the store, public and referenceable.** shard-wiki lesson — fine-grained addressing is *tractable* when the backend mints stable block/span IDs; the adapter should **adopt native block IDs as span addresses where they exist** (UC-51), and fall back to content-fingerprint or path+range where they do not (cf. Xanadu content-identity, UC-46). 2. **"Does transclusion belong in core or adapter/UI?"** (catalog Q6). Roam shows transclusion is cheap and natural *when the store is addressable and queryable*; it lives at the **data layer** (refs/embeds), surfaced by UI. Argues for transclusion as a **core capability over an addressable union**, not a UI-only trick. 3. **"Derived views — core or adapter?"** Roam shows them as **queries over a structured space**. Where a shard exposes a native query engine, **delegate** (UC-52); where it does not, the orchestrator computes them over the projection. 4. **Block↔page mapping** is the new question Roam *raises*: in a block graph, what is a "page"? Roam's own answer (`:node/title` node = page; nested blocks = spans) is a clean mapping rule for the adapter (UC-50). --- ## 8. Mapping to shard-wiki INTENT (compare, do not equate) ### 8.1 Reinforcements - **Fine-grained addressing is achievable** (block UIDs) — de-risks UC-44/45/51 and the Xanadu tumbler worry. - **Transclusion + bidirectional links + structured data** are not exotic; a shipped tool does all three over one addressable, queryable store — validating shard-wiki's ambitions for UC-32/34/39/05. - **Engine-hosts-adapter (UC-38)** has a clean modern template: Depot's `onload/onunload` + `roamAlphaAPI` read/write surface. ### 8.2 Deliberate divergences (design bugs if conflated) 1. **Block-DB substrate vs. Markdown-file page model.** Roam's "everything is a block in a DataScript DB" must **not** tempt shard-wiki away from its Markdown-first, backend-neutral page model (INTENT Stability Note). Roam is *one shard shape*, mapped *into* the page model, not the model itself. 2. **Client-side proprietary DB vs. Git coordination journal.** Roam's transaction log is not portable history; shard-wiki keeps Git as the coordination layer. A Roam shard contributes snapshots/projections, and its history needs **supplement/import** (UC-36), not adoption as canonical. 3. **Single graph vs. federation.** A Roam graph is **one sovereign shard**, never the federation layer. Do not model the union as "a big Roam graph." 4. **Roam-flavored Markdown + outline structure** ≠ CommonMark pages; the adapter owns a **lossy-aware translation** (block outline ↔ page + headings/lists), cf. UC-42. ### 8.3 What Roam teaches that shard-wiki should keep - **Mint/adopt stable sub-page IDs.** The cheapest path to transclusion, overlay, and reverse-lookup at span granularity is a backend that already addresses spans — lean on it (UC-51), degrade gracefully otherwise. - **Treat a structured shard's query engine as a capability**, and delegate derived views to it when present (UC-52). - **A clean block↔page rule** (`:node/title` node = page) keeps a block backend usable without flattening (UC-50, complementing UC-34's no-lossy-flatten rule). --- ## 9. Use-case seeds → catalog (promoted 2026-06-14) Last existing UC is **UC-49**. New UCs **UC-50–UC-52** added; existing UCs enriched. | Seed | Catalog action | |------|----------------| | **Attach a block-graph DB wiki (Roam-style) as a shard via its query/CRUD API, mapping blocks to the page model** (`:node/title` node = page; nested blocks = spans) | **UC-50 (new)** | | **Adopt a shard's native block/span IDs as portable span addresses** for transclusion/overlay (the shipped answer to the span-address question) | **UC-51 (new)** | | **Delegate derived views to a shard's native query engine** (Datalog) where advertised | **UC-52 (new)** | | Shipped transclusion via block embeds + shipped fine-grained addressing | **enriches UC-32** (and UC-44/45) | | Attributes + Datalog = shipped structured/typed pages | **enriches UC-34** (and UC-39) | | Roam Depot `onload/onunload` + `roamAlphaAPI` = modern engine-hosts-adapter template | **enriches UC-38** | | Block-level write granularity = the fine extreme opposite TiddlyWiki | **enriches UC-35** | | Hosted history is internal-only, not portable Git | links **UC-36** | --- ## 10. Architecture notes for SHARD-WP-0002 (no UC) - The adapter contract should model **"native span/block IDs"** as a capability: a shard advertises whether it mints stable sub-page addresses; transclusion/overlay/reverse- lookup capabilities key off it (ties UC-44/45/46/51). - The contract should model a **"native query"** capability so the orchestrator can delegate derived-view computation (UC-52) vs. scanning the projection. - A **block↔page mapping** belongs in the adapter, with the no-lossy-flatten rule of UC-34: page = titled node, blocks = addressable spans, attributes = sidecar metadata. - Roam joins XWiki as a **DB-backed / API-attached** exemplar (vs. TWiki/Foswiki file stores) and as a **second engine-hosts-adapter** template (vs. XWiki components / TWiki handlers) for the T14 adapter-binding task. --- ## 11. Open questions (for spec / workplans) 1. When a backend mints native span IDs (Roam UIDs), are they used **directly** as the shard-wiki span address, or **wrapped** in a shard-scoped address so they survive projection/overlay and don't collide across shards? 2. For a **write-through** Roam shard the adapter must run *inside* Roam (Depot extension). Is in-engine hosting an accepted adapter deployment mode generally, or do we restrict Roam to **read/projection/overlay-target** (graceful degradation) when we cannot deploy inside it? 3. How lossy is the **block-outline ↔ Markdown-page** mapping, and is round-trip fidelity required or best-effort (cf. UC-42 Foswiki TML↔HTML)? 4. Do we consume Roam's **export (EDN/JSON)** as the projection source, or the live `roamAlphaAPI`? Snapshot vs. live freshness trade-off (cf. UC-31). --- ## 12. Sources | Source | Used for | |--------|----------| | zsolt.blog — "Deep Dive Into Roam's Data Structure" (https://www.zsolt.blog/2021/01/Roam-Data-Structure-Query.html) | Datom/EAV model, block attributes (`:block/uid` etc.), refs/embeds, linked vs unlinked references, Datalog/pull | | Roam Research Datalog Cheatsheet (https://gist.github.com/2b3pro/231e4f230ed41e3f52e8a89ebf49848b) | Attribute names, `:q`/`:pull` query shapes | | GitHub — Roam-Research/roam-depot (https://github.com/Roam-Research/roam-depot) | Extension repo/manifest structure, `extension.js` `onload`/`onunload`, `extensionAPI` settings, `roamAlphaAPI` read/write/ui surface | | "Introduction to the Roam Alpha API" (https://www.putyourleftfoot.in/introduction-to-the-roam-alpha-api) | `window.roamAlphaAPI` `q`/`pull` semantics, global-scope dependency | | David Vargas — "How To Create Your Own Roam Extensions" (https://davidvargas.me/blog/how_to_create_your_own_roam_extensions) | Extension authoring lifecycle, write methods | | Wikipedia / general — Roam Research, DataScript, Datomic | Datalog/Datomic lineage, client-side DB framing | Cross-references: `research/260614-xanadu-deep-dive/findings.md` (span addressing, transclusion), `research/260614-zigzag-deep-dive/findings.md` (queryable dimensions), `research/260613-xwiki-deep-dive/findings.md` (DB-backed engine + extension host), `spec/UseCaseCatalog.md` (UC-05, UC-18, UC-32, UC-34/35/36/38/39, UC-44/45/46), `workplans/SHARD-WP-0002-federation-architecture.md` (T14 adapter binding). --- ## 13. Traceability - New UCs: **UC-50, UC-51, UC-52** → `spec/UseCaseCatalog.md`. - Enriched UCs: **UC-32, UC-34, UC-35, UC-38** (and links UC-36, UC-39, UC-44/45/46). - Architecture (no UC): native-span-ID capability; native-query capability; block↔page mapping; Roam as DB-backed/API-attached + engine-hosts-adapter exemplar → `SHARD-WP-0002` (T14). - Boundary recorded: Roam is **one candidate shard**, mapped into the Markdown-first page model; not a substrate, not the federation layer (INTENT Stability Note).