coulomb/shard-wiki
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

research: Notion deep dive (closed block-DB SaaS, external REST API only, database-as-pages); UC-57/58/59

Browse Source 
The closed/hosted/schema-rich extreme: everything is a block (UUID id,
type, properties, ordered child content, single parent), pages are blocks
and database rows are pages with a schema, Postgres-backed hosted SaaS.
Databases add typed properties + relations + rollups + formulas across
many views = the apex of wiki-page-as-structured-record. Extension model
has no in-app plugin runtime; the only extensibility is the external REST
API (+ webhooks 2026) inside a tight envelope (~3 rps, eventual
consistency, recursive child fetch, scoped/revocable per-page grants).
Adds the third attachment mode (external-API-only) alongside file-store
(Obsidian/TWiki) and in-engine host (Roam/XWiki); Notion enforces no
silent remote mutation via scoped grants. Added UC-57 (attach closed
external-API-only shard w/ operational envelope + scoped grant), UC-58
(typed database w/ schema+relations+views, no flattening), UC-59
(lossy-aware translation w/ fidelity report); enriched
UC-31/34/36/39/50/51/52/54/56. Boundary: one external-API candidate shard,
best as projection/mirror/overlay/backup, not a substrate and not the
federation layer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-06-14 13:44:09 +02:00
parent f0be5799aa
commit 64796f7b3a
5 changed files with 488 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
SCOPE.md
View File

@@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary.
|-------|-------| |-------|-------|
| Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
| Intent | `INTENT.md` established; authorization-in-core amendments drafted | | Intent | `INTENT.md` established; authorization-in-core amendments drafted |
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian deep dives (`research/260614-*/`) | | Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion deep dives (`research/260614-*/`) |
| Demand | NetKingdom integration asks captured, not yet negotiated | | Demand | NetKingdom integration asks captured, not yet negotiated |
| Spec | Architecture blueprint drafted; UseCaseCatalog 56 UCs from research; PRD/TSD scaffolds | | Spec | Architecture blueprint drafted; UseCaseCatalog 59 UCs from research; PRD/TSD scaffolds |
| Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (10 tasks) | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (10 tasks) |
## In Scope (today) ## In Scope (today)

56
research/260614-notion-deep-dive/README.md Normal file
View File

@@ -0,0 +1,56 @@
# 260614 — Notion deep dive (closed block-database SaaS, attached only through its API)
Date: 2026-06-14
## What this is
A focused study of **Notion** — architecture and extension model — read through
shard-wiki's lens. Notion is the **closed, hosted, schema-rich** end of the spectrum and
the dive that stress-tests shard-wiki's hardest constraints (*graceful degradation*,
*no silent remote mutation*, *Markdown-first that must degrade*).
It contrasts sharply with its neighbours:
- vs **Roam** — both are block databases with per-block UUIDs, but Roam is a *client*
DataScript DB reached only via an *in-app* JS API, while Notion is a *server* Postgres
store reached via a real *external* REST API (no in-engine hosting needed, but
rate-limited, eventually consistent, scoped by per-page grants).
- vs **Obsidian** — the mirror image of "file over app": Notion is **app over file**, a
closed hosted store with no portable files.
- vs **XWiki** — both are structured wiki-as-app-platforms; Notion is the **apex of
database-as-pages** (schema + relations + rollups + views) but closed and REST-only.
Distinctive material:
- **Data model** — everything is a **block** (UUID id, type, properties, ordered child
`content`, single `parent`); pages are blocks, **database rows are pages with a
schema**; Postgres-backed hosted SaaS
- **Databases** — typed properties, **relations** (bidirectional), **rollups**,
**formulas**, and many **views** (table/board/calendar/gallery) of one row-set
- **Extension model** — **no in-app plugin runtime**; extension = **external REST API**
integrations (+ webhooks since 2026), with ~3 rps rate limit, eventual consistency,
recursive child fetch, payload caps, and **scoped/revocable per-page access grants**
## Contents
| Path | Role |
|------|------|
| `findings.md` | Block model, databases, page-model impedance, the API-only extension model, capability profile, scoped consent, INTENT mapping, UC seeds, architecture notes, sources |
## Status
Initial deep dive complete. Three new use cases promoted to `spec/UseCaseCatalog.md`
(UC-57 attach a closed external-API-only hosted shard with rate-limit/eventual-
consistency/scoped-grant constraints, UC-58 attach a typed database with schema +
relations + views without flattening, UC-59 lossy-aware translation with a fidelity
report); UC-31/34/36/39/50/51/52/54/56 enriched. Logged for `SHARD-WP-0002` (T14):
an **operational-envelope** capability section, **access-grant semantics**, a
**translation-fidelity** capability, and a three-way **attachment-mode taxonomy**
(file-store / in-engine-hosted / external-API-only).
**Key takeaways recorded:** Notion *enforces* "no silent remote mutation" via scoped,
revocable per-integration grants (ties the authz-in-core decision); it is the heaviest
**lossy-translation** case (proprietary blocks/rich-text, lossy Markdown export); and it
adds the **external-API-only** attachment mode (full write-through without in-engine
hosting, but bounded by rate limits and eventual consistency). **Boundary:** one
external-API candidate shard — best as projection/mirror/overlay/backup — not a
substrate and not the federation layer.
</content>

302
research/260614-notion-deep-dive/findings.md Normal file
View File

@@ -0,0 +1,302 @@
# Findings — Notion: a closed block-database SaaS, attached only through its API
Date: 2026-06-14
Source kind: **modern shipped product** — a hosted block-database SaaS; a *candidate
shard* of a distinct family (closed, external-REST-API-only, schema-rich, heaviest
translation cost); has **no in-app plugin runtime**
Lens: shard-wiki — the external-API attachment mode, database-as-pages, lossy
translation/fidelity, scoped consent, and graceful degradation against a sovereign
closed backend
> Where Notion sits in the set. Roam and Notion are both **block databases with
> per-block UUIDs**, but they are opposite on access: Roam is a *client-side*
> DataScript DB reachable only through an *in-app* JS API (write-through needs an
> adapter hosted inside Roam); **Notion is a *server-side* Postgres store reachable
> through a real *external* REST API** — attachable from outside with no in-engine
> hosting, but rate-limited, eventually consistent, and **scoped by explicit
> per-integration page grants**. And against Obsidian's "file over app," Notion is the
> mirror image: **app over file** — a closed hosted store with no portable files at
> all. So Notion is the dive that stress-tests shard-wiki's hardest constraints:
> *graceful degradation*, *no silent remote mutation*, *union without erasure*, and
> *Markdown-first that must degrade* against a proprietary, schema-rich, non-Markdown
> backend.
Pairs with — and contrasts against — the Roam dive (block-DB, in-app vs external API),
the Obsidian dive (closed-hosted vs file-over-app), and the XWiki dive (both are
structured wiki-as-app-platforms; Notion is the apex of database-as-pages but closed
and REST-only).
---
## 1. Core data model — everything is a block
Notion's data model is uniform: **"Everything you see in Notion is a block. Text,
images, lists, a row in a database, even pages themselves — these are all blocks."**
Each block record holds:
| Field | Meaning |
|-------|---------|
| **id** | a randomly generated **UUID v4**, visible in the page URL — the block's stable address |
| **type** | determines rendering + which properties apply (text, to-do, heading, **page**, **child_database**, …) |
| **properties** | type-specific attributes (e.g. `title` text; for database rows, the typed property values) |
| **content** | an **ordered array of child block IDs** — nesting |
| **parent** | a single upward pointer — used for **permission inheritance** |
Blocks form a **render tree** (content + parent pointers); indentation is *structural*,
not cosmetic — it changes block relationships. **Pages are blocks**; **databases are
blocks whose children are pages**; a database **row is a page** with typed properties.
Stored in **Postgres** on Notion's servers (hosted SaaS — not local, not files).
The shard-wiki-relevant facts: (a) **per-block UUIDs** give native sub-page addressing
(UC-51); (b) the **page = block, database-row = page-with-schema** identity is a real
page-model impedance (§3); (c) the store is **closed and server-side** — reachable only
through the API (§4).
---
## 2. Databases — schema, typed properties, relations, views
A Notion **database** is a collection of pages sharing a **schema** of typed
**properties**: `title`, `rich_text`, `select` / `multi_select`, `number`, `date`,
`checkbox`, `person`, `files`, **`relation`** (a typed link to rows in another
database — shown on *both* sides, i.e. bidirectional), **`rollup`** (an aggregate over a
relation), and **`formula`** (computed). The same row-set is shown through multiple
**views** — **table, board (kanban), calendar, gallery, list, timeline** — each a
filtered/sorted/grouped projection of the same data.
For shard-wiki this is the **apex of "wiki page as structured record"** (stronger than
XWiki XObjects, UC-39): a page can be *bodiless typed data*, pages are joined by
**typed inter-database relations** (a graph of typed links), and "the page" is routinely
**a row in a schema** rather than prose. Notion databases also ship the ZigZag insight
commercially: **one row-set, many views/dimensions** (UC-47/48) and **filtered/linked
databases = query-defined pages** (UC-54).
---
## 3. Page-model impedance
Mapping Notion to a Markdown-first page model is the **heaviest translation case** in
the research set:
- **Block/rich-text ≠ Markdown.** Notion's rich text is an annotated-span model; many
block types (synced blocks, columns, callouts, embeds, database views) have no clean
Markdown equivalent. Notion's own **export to Markdown/CSV is lossy** (databases →
CSV, relations/rollups/formulas flatten or drop).
- **Database-row-as-page + schema** must map onto pages + sidecar metadata without
discarding the schema or the relations (extends UC-34/UC-39).
- Therefore translation must be **lossy-aware with a fidelity report** — surface what
did *not* round-trip rather than silently flattening (UC-59). This is *different from*
UC-42 (Foswiki TML↔HTML *lossless* round-trip); Notion is fundamentally lossy.
---
## 4. Extension model — there is no plugin runtime; only the API
This is the defining architectural fact for shard-wiki: **Notion has no third-party
in-app plugin/extension system.** There is no marketplace of code that runs inside
Notion (the only "in-app modding" is the *unofficial* Notion Enhancer desktop patcher,
out of scope). **Extension = external integration via the public REST API** (plus
embeds and, recently, webhooks).
The **public REST API** (`api.notion.com/v1`):
- **Resources:** `pages`, `blocks` (retrieve / append / update / delete children),
`databases` (retrieve, **query** with filters+sorts, create), `users`, `search`,
`comments`.
- **Authorization:** an **internal integration token** (workspace-owned) or **OAuth
2.0** (public integrations). Critically, **an integration only sees pages a user has
explicitly connected to it** — the user "approves the app and connects specific
pages" via *Add connections*. Access is a **scoped, revocable, per-page grant** (§6).
- **Operational constraints the adapter must encode:**
- **Rate limit** ~**3 requests/second** average (429 `rate_limited`); **no paid
increase**.
- **Eventual consistency / no live read model** — the API is not real-time; reads can
lag.
- **Recursive retrieval** — a block returns only **first-level children**; full-page
reconstruction requires walking `has_children` recursively.
- **Payload caps** — 1000 blocks / 500 KB per request; child arrays ≤ 100.
- **Webhooks** (added 2026) deliver page/database change events — a push transport for
UC-31, replacing pure polling.
- **Version history:** Notion keeps **internal page history** (retention bounded by
plan); it is **not portable** and not exposed as git — a UC-36 *supplementation* case
(like Confluence/MediaWiki), not an import case.
- **Publish:** pages can be **published to the web** as read-only public pages — an
outbound publish surface (UC-56).
Consequence: Notion is attachable **only** as an **external-API shard** — but, unlike
Roam, **no in-engine adapter hosting is needed** (the REST API is external). The cost is
the operational envelope (rate-limit, eventual consistency, scoped grant) that the
capability profile must model (UC-57).
---
## 5. Notion as a shard — capability profile
| Capability | Notion | Notes for the adapter contract |
|------------|--------|--------------------------------|
| Read | **yes (external REST)** | rate-limited (~3 rps), eventually consistent, recursive child fetch |
| Write | **yes (external REST)** | append/update/delete blocks, create pages — no in-engine host needed (vs Roam) |
| Write granularity | **block-level (fine)** | like Roam; per-block ops |
| Identity / addressing | **block UUID v4** | native sub-page addressing (UC-51), store-minted (like Roam, not in-file) |
| Structured data | **yes (apex)** | databases: schema + typed properties + relations + rollups + formulas (UC-34/39/58) |
| Native query | **yes** | database query API (filters/sorts) → delegate views (UC-52) |
| Views / dimensions | **yes** | table/board/calendar/gallery = many views of one row-set (UC-47/48/54) |
| Subscribe | **webhooks (2026)** | push events; else poll (UC-31) |
| Version history | **internal, not portable** | supplement via coordination journal (UC-36) |
| Diff / merge | **no native** | — |
| Lock | **no** | — |
| Publish | **publish-to-web** | outbound read-only (UC-56) |
| Access model | **scoped per-page grant (OAuth/token)** | explicit consent; revocable (UC-57, authz) |
| Syntax / content | **proprietary block + rich text** | **lossy** to Markdown; needs fidelity-aware translation (UC-59) |
Verdict: Notion is a legitimate but **demanding** shard — **external-API-attached,
schema-rich, fine-grained, closed**. It behaves best as a **projected / mirrored /
overlay / backup** participant; full write-through is possible but bounded by rate limits
and eventual consistency. The strongest reasons to attach it: structured databases
(UC-58) and block-UUID addressing (UC-51); the strongest cautions: lossy translation
(UC-59) and no portable history (UC-36).
---
## 6. Scoped consent and "no silent remote mutation"
Notion *enforces* one of shard-wiki's INTENT constraints at the platform level: an
integration can touch **only** the pages a user has explicitly **connected** to it, and
the grant is **revocable**. This is a clean, real-world model of **no silent remote
mutation** and of a shard granting the orchestrator **scoped, consented access** — and
it ties directly to the settled **authz-in-core / authn-delegated** decision
([[shard-wiki-auth-in-core-decision]]): authentication to Notion is delegated (OAuth /
integration token), while shard-wiki's *own* authorization decides what to do with the
granted scope. The adapter contract should treat **"scoped, revocable grant"** as a
first-class attachment property (UC-57), not an afterthought.
---
## 7. Mapping to shard-wiki INTENT (compare, do not equate)
### 7.1 Reinforcements
- **Graceful degradation** has its sharpest test here: a closed SaaS with no files, rate
limits, eventual consistency, and lossy export must still be usable as
read/projection/overlay/backup. If the adapter contract handles Notion, it handles
most things.
- **No silent remote mutation** is *modeled by the platform* (scoped grants) — Notion
validates the principle (§6).
- **Database-as-pages** validates that "wiki page" must stretch to **typed records with
relations**, not just prose (UC-34/39/58).
- **Block UUIDs** reconfirm (with Roam) that native sub-page addressing is real and
adoptable (UC-51).
### 7.2 Deliberate divergences (design bugs if conflated)
1. **Closed hosted store; no sovereignty over bytes.** shard-wiki cannot make Notion
git-native or local. It can mirror/project/overlay/back-up and **supply** a
git-addressable history (UC-36) — never claim to own Notion's store.
2. **Lossy, proprietary content.** Do **not** pretend Notion round-trips to Markdown.
Translate lossily *with a fidelity report* and preserve non-mappable elements as
provenance/sidecar (UC-59) — union without erasure includes erasure *of fidelity*
being made visible.
3. **External-API-only, rate-limited, eventually consistent.** Projection must be
**cache/poll/webhook**, not a live read model; sync is bounded — encode this in the
capability profile (UC-57). Do not design flows that assume cheap, instant, unlimited
reads.
4. **One workspace (or its granted page set) = one shard**, never the federation layer.
### 7.3 What Notion teaches that shard-wiki should keep
- Model **operational envelope** (rate limit, consistency class, payload caps,
pagination) as explicit capability-profile fields — Notion makes them unignorable.
- Model **scoped, revocable consent** as a first-class attachment property (UC-57, §6).
- Treat **translation fidelity as data**: a per-shard, per-page report of what projects
cleanly vs. degrades (UC-59) — applies beyond Notion.
- Recognize **external-REST attach** as a distinct, *preferred-where-available* mode:
full write-through without in-engine hosting (contrast Roam) — but pay the operational
envelope.
---
## 8. Use-case seeds → catalog (promoted 2026-06-14)
Last existing UC is **UC-56**. New UCs **UC-57–UC-59** added; existing UCs enriched.
| Seed | Catalog action |
|------|----------------|
| **Attach a closed hosted shard via its external REST API only** — no file store, no in-app runtime — honoring rate limits, eventual consistency, payload caps, and a scoped/revocable access grant | **UC-57 (new)** |
| **Attach a typed database (schema + relations + rollups + multiple views) as a shard** without flattening the schema or the inter-record relations | **UC-58 (new)** |
| **Translate a proprietary block/rich-text model to/from Markdown with an explicit fidelity report**, preserving non-mappable elements rather than silently dropping them | **UC-59 (new)** |
| Block UUIDs = store-minted native span addresses (external-API variant) | **enriches UC-51** |
| External-API block-DB attach (no in-engine host) — contrast Roam's in-app-only | **enriches UC-50** |
| Database query API + filtered/linked DBs | **enriches UC-52, UC-54** |
| Database-as-pages apex; typed records + relations | **enriches UC-34, UC-39** |
| Webhooks (2026) as a push transport | **enriches UC-31** |
| Internal-only page history, not portable | **enriches UC-36** |
| Publish-to-web outbound | **enriches UC-56** |
| Scoped, revocable per-integration grant; no silent mutation | links **UC-57** + [[shard-wiki-auth-in-core-decision]] |
---
## 9. Architecture notes for SHARD-WP-0002 (no UC)
- Add an **operational-envelope** section to the adapter capability profile: rate limit,
consistency class (live / eventually-consistent / snapshot), payload/pagination caps,
recursive-fetch requirement, push-vs-poll transport. Notion is the forcing example.
- Add **access-grant semantics**: scope (which pages), revocability, auth mode
(delegated token/OAuth) — ties the authz-in-core decision and "no silent mutation".
- Add a **translation-fidelity capability**: adapters declare and report what content
round-trips vs. degrades (UC-59); generalizes UC-42 (lossless) to the lossy case.
- **Attachment-mode taxonomy** now spans: file-store direct (Obsidian/TWiki, UC-40),
in-engine hosted adapter (Roam/XWiki, UC-38/50), and **external-API-only**
(Notion, UC-57). T14 binding should enumerate all three.
- **Database/schema/relations** as a unit (UC-58) presses the page-model spec: collection
+ schema + typed relations, not just a page.
---
## 10. Open questions (for spec / workplans)
1. Is **external-API-only with a tight rate limit** (Notion) viable for write-through at
wiki scale, or do we cap Notion at read/projection/overlay/backup by default?
2. How are **inter-database relations** (UC-58) represented in the union — as typed links
in the link graph, as a separate relation index (cf. ZigZag many-to-many), or both?
3. What is the **fidelity report** format (UC-59), and where does it surface — provenance
panel, projection metadata, reconciliation review?
4. For **scoped grants** (§6), how does shard-wiki represent partial visibility (only
*some* of a workspace's pages granted) without misrepresenting the shard as complete?
5. Do we consume Notion **webhooks** (push) or poll, given eventual consistency and the
rate limit (UC-31)?
---
## 11. Sources
| Source | Used for |
|--------|----------|
| Notion — "The data model behind Notion's flexibility" (https://www.notion.com/blog/data-model-behind-notion) | Everything-is-a-block; block record (id/type/properties/content/parent); render tree; pages/databases as blocks; Postgres |
| Notion Docs — Request limits (https://developers.notion.com/reference/request-limits) | ~3 rps rate limit, 429, payload caps, recursive first-level children |
| Notion Docs — Authorization (https://developers.notion.com/docs/authorization) | Internal token vs OAuth; integration connected to specific pages; scoped grant |
| Hookdeck / ClickUp — Notion webhooks guides (https://hookdeck.com/webhooks/platforms/guide-to-notion-webhooks-features-and-best-practices) | Webhook support (2026), page/database change events |
| Truto / Rollout — Notion API architecture & essentials (https://truto.one/blog/how-to-integrate-with-the-notion-api-architecture-guide-for-b2b-saas/) | REST endpoints (pages/blocks/databases/search), integration patterns, no in-app plugin model |
| General API knowledge — database property types, views, relations/rollups/formulas, export-to-Markdown lossiness | §2, §3 |
Cross-references: `research/260614-roam-deep-dive/findings.md` (block-DB/UUID, in-app vs
external API), `research/260614-obsidian-deep-dive/findings.md` (closed-hosted vs
file-over-app), `research/260613-xwiki-deep-dive/findings.md` (structured wiki-app-
platform), `spec/UseCaseCatalog.md` (UC-31, UC-34, UC-36, UC-39, UC-50/51/52, UC-54,
UC-56), `workplans/SHARD-WP-0002-federation-architecture.md` (T14), and the authz
decision [[shard-wiki-auth-in-core-decision]].
---
## 12. Traceability
- New UCs: **UC-57, UC-58, UC-59** → `spec/UseCaseCatalog.md`.
- Enriched UCs: **UC-31, UC-34, UC-36, UC-39, UC-50, UC-51, UC-52, UC-54, UC-56**.
- Architecture (no UC): operational-envelope + access-grant + translation-fidelity
capability fields; three-way attachment-mode taxonomy; database/schema/relations in the
page model → `SHARD-WP-0002` (T14).
- Decision link: scoped/revocable grant + no-silent-mutation → [[shard-wiki-auth-in-core-decision]].
- Boundary recorded: Notion is **one external-API candidate shard** — closed, hosted,
schema-rich, lossy-to-Markdown — best as projection/mirror/overlay/backup; not a
substrate, not the federation layer (INTENT graceful-degradation + no-silent-mutation).
</content>

3
research/README.md
View File

@@ -20,4 +20,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
| 2026-06-14 | `260614-xanadu-deep-dive/` | Project Xanadu — EDL/reference-not-copy, transclusion, addressing; UC-44/45/46 | | 2026-06-14 | `260614-xanadu-deep-dive/` | Project Xanadu — EDL/reference-not-copy, transclusion, addressing; UC-44/45/46 |
| 2026-06-14 | `260614-zigzag-deep-dive/` | ZigZag/zzstructure — information space as orthogonal dimensions; UC-47/48/49 | | 2026-06-14 | `260614-zigzag-deep-dive/` | ZigZag/zzstructure — information space as orthogonal dimensions; UC-47/48/49 |
| 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 | | 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 |
| 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 | | 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 |
| 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 |

137
spec/UseCaseCatalog.md
View File

@@ -7,7 +7,8 @@ Promoted from `research/260608-c2-wiki-origins/`,
`research/260608-wikiengines-overview/`, `research/260613-xwiki-deep-dive/`, `research/260608-wikiengines-overview/`, `research/260613-xwiki-deep-dive/`,
`research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`, `research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`,
`research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`, `research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`,
`research/260614-roam-deep-dive/`, and `research/260614-obsidian-deep-dive/`. `research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`, and
`research/260614-notion-deep-dive/`.
See InfoTechPrimers on coulomb.social for use-case catalog conventions. See InfoTechPrimers on coulomb.social for use-case catalog conventions.
## Conventions ## Conventions
@@ -177,7 +178,10 @@ triggering projection refresh or reconciliation review.
Transport is adapter concern; freshness surfaced in UC-24. XWiki deep dive adds a Transport is adapter concern; freshness surfaced in UC-24. XWiki deep dive adds a
concrete push transport: an engine event bus (`ObservationManager` concrete push transport: an engine event bus (`ObservationManager`
`DocumentUpdatedEvent`) consumed by a listener, vs polling REST history `DocumentUpdatedEvent`) consumed by a listener, vs polling REST history
(`research/260613-xwiki-deep-dive/findings.md` §2.5, §8 Q4). (`research/260613-xwiki-deep-dive/findings.md` §2.5, §8 Q4). Notion adds **webhooks**
(2026) delivering page/database change events — a SaaS push transport, weighed against
eventual consistency and the rate limit (`research/260614-notion-deep-dive/findings.md`
§4).
**Priority:** Later **Priority:** Later
### UC-32 — Transclude remote span with live freshness ### UC-32 — Transclude remote span with live freshness
@@ -230,7 +234,9 @@ topic text file*, so structure is diffable rather than locked in a DB
sidecar metadata, and a candidate for query delegation (UC-52). Obsidian shows the sidecar metadata, and a candidate for query delegation (UC-52). Obsidian shows the
**git-diffable in-file** variant: YAML frontmatter/properties live in the Markdown text **git-diffable in-file** variant: YAML frontmatter/properties live in the Markdown text
and are queried by the Dataview plugin (`research/260614-obsidian-deep-dive/findings.md` and are queried by the Dataview plugin (`research/260614-obsidian-deep-dive/findings.md`
§3) — structure as portable text, not DB state. §3) — structure as portable text, not DB state. **Notion** is the apex of the DB-locked
end: typed database properties with relations/rollups/formulas, lossy to Markdown —
see UC-58 (`research/260614-notion-deep-dive/findings.md` §2).
**Priority:** Later **Priority:** Later
### UC-35 — Attach a shard with coarse write granularity ### UC-35 — Attach a shard with coarse write granularity
@@ -265,7 +271,10 @@ internal RCS table (`xwikircs`) is a concrete instance
(TWiki RCS `.txt,v`) it can instead be **imported** — see UC-41. Demand evidence: (TWiki RCS `.txt,v`) it can instead be **imported** — see UC-41. Demand evidence:
**Obsidian Git is a top-7 plugin (2.7M)** — users manually bolt portable git history **Obsidian Git is a top-7 plugin (2.7M)** — users manually bolt portable git history
onto a vault that has none natively (`research/260614-obsidian-deep-dive/findings.md` onto a vault that has none natively (`research/260614-obsidian-deep-dive/findings.md`
§5, §7), exactly the gap the coordination journal fills as core. §5, §7), exactly the gap the coordination journal fills as core. **Notion** is the
closed-SaaS instance: internal page history bounded by plan, **not portable** and not
exported as git (`research/260614-notion-deep-dive/findings.md` §4) — a supplementation
case like Confluence/MediaWiki, not an import case.
**Priority:** Later **Priority:** Later
### UC-37 — Attach a static engine export as a read-only backup shard ### UC-37 — Attach a static engine export as a read-only backup shard
@@ -317,7 +326,9 @@ or can a page be purely typed data (`research/260613-xwiki-deep-dive/findings.md
§8 Q2)? Affects union views, diff, and search. Deferred to `SHARD-WP-0002`. §8 Q2)? Affects union views, diff, and search. Deferred to `SHARD-WP-0002`.
Foswiki shows a middle point: MetaDataPlugin stores **multiple** structured records Foswiki shows a middle point: MetaDataPlugin stores **multiple** structured records
per topic, still as `%META%` in text (`research/260613-foswiki-deep-dive/findings.md` per topic, still as `%META%` in text (`research/260613-foswiki-deep-dive/findings.md`
§4) — the page model must allow N typed records, not one form. §4) — the page model must allow N typed records, not one form. **Notion** pushes
further: a *database* is a collection with a schema whose rows are pages joined by typed
relations — see UC-58 (`research/260614-notion-deep-dive/findings.md` §2).
**Priority:** Later **Priority:** Later
### UC-40 — Attach a file-backed engine's on-disk store directly ### UC-40 — Attach a file-backed engine's on-disk store directly
@@ -482,7 +493,11 @@ flattening the structure.
= page, nested blocks = spans, attributes = sidecar metadata (honors UC-34 no-lossy- = page, nested blocks = spans, attributes = sidecar metadata (honors UC-34 no-lossy-
flatten). DB-backed/API-attached path (like XWiki UC-38), **not** the file-store path flatten). DB-backed/API-attached path (like XWiki UC-38), **not** the file-store path
(UC-40). Write-through requires hosting the adapter inside the engine (Roam Depot), (UC-40). Write-through requires hosting the adapter inside the engine (Roam Depot),
else degrade to read/projection/overlay-target (findings §11 Q2). else degrade to read/projection/overlay-target (findings §11 Q2). **Notion** is the
external-API variant of the same block-DB family: a server Postgres store with a real
**external** REST API, so write-through needs **no in-engine hosting** — but is bounded
by rate limits and eventual consistency (UC-57,
`research/260614-notion-deep-dive/findings.md` §4).
**Priority:** Later **Priority:** Later
### UC-51 — Adopt a shard's native span IDs as portable span addresses ### UC-51 — Adopt a shard's native span IDs as portable span addresses
@@ -501,7 +516,8 @@ a shard-scoped address to avoid cross-shard collision and survive projection (fi
§11 Q1). Obsidian shows the **text-embedded** variant: `^block-id` and heading anchors §11 Q1). Obsidian shows the **text-embedded** variant: `^block-id` and heading anchors
live in the Markdown text — git-diffable and portable (survives a file copy) but opt-in live in the Markdown text — git-diffable and portable (survives a file copy) but opt-in
(`research/260614-obsidian-deep-dive/findings.md` §2), vs Roam's mandatory DB-minted (`research/260614-obsidian-deep-dive/findings.md` §2), vs Roam's mandatory DB-minted
UID. UID. Notion is a second store-minted case: per-block **UUID v4**, exposed via the REST
API (`research/260614-notion-deep-dive/findings.md` §1).
**Priority:** Later **Priority:** Later
### UC-52 — Delegate derived views to a shard's native query engine ### UC-52 — Delegate derived views to a shard's native query engine
@@ -517,7 +533,9 @@ projection). Operationalizes the ZigZag "dimensions + rasters" insight
(`research/260614-zigzag-deep-dive/findings.md` §5) and relates to UC-05/UC-34. Obsidian (`research/260614-zigzag-deep-dive/findings.md` §5) and relates to UC-05/UC-34. Obsidian
nuance: query can be an *ecosystem plugin* (Dataview), **not core** — so "native query" nuance: query can be an *ecosystem plugin* (Dataview), **not core** — so "native query"
is adapter/plugin-provided and must not be assumed is adapter/plugin-provided and must not be assumed
(`research/260614-obsidian-deep-dive/findings.md` §7). (`research/260614-obsidian-deep-dive/findings.md` §7). **Notion** ships a native
database **query API** (filters/sorts) — a strong delegation target
(`research/260614-notion-deep-dive/findings.md` §2, §4).
**Priority:** Later **Priority:** Later
### UC-53 — Attach a local-first Markdown vault with a live concurrent native editor ### UC-53 — Attach a local-first Markdown vault with a live concurrent native editor
@@ -547,7 +565,9 @@ Roam `{{query}}` — query-defined dynamic pages
(`research/260614-obsidian-deep-dive/findings.md` §7). Distinct from UC-44 (reference/ (`research/260614-obsidian-deep-dive/findings.md` §7). Distinct from UC-44 (reference/
span manifest) and UC-52 (delegating *computation* of an existing view): here the *page span manifest) and UC-52 (delegating *computation* of an existing view): here the *page
itself* is a query. Open: core page type vs. adapter vs. reference-UI/plugin (findings itself* is a query. Open: core page type vs. adapter vs. reference-UI/plugin (findings
§11 Q2). May delegate execution to a shard's native query engine (UC-52). §11 Q2). May delegate execution to a shard's native query engine (UC-52). **Notion**
linked/filtered databases are the commercial instance — one row-set surfaced through many
filtered views (`research/260614-notion-deep-dive/findings.md` §2).
**Priority:** Later **Priority:** Later
### UC-55 — Carry non-Markdown content types as typed or opaque assets ### UC-55 — Carry non-Markdown content types as typed or opaque assets
@@ -575,7 +595,61 @@ read-only target — a static site or hosted page set — preserving provenance.
publish (`research/260614-obsidian-deep-dive/findings.md` §7, §10). Formalizes the publish (`research/260614-obsidian-deep-dive/findings.md` §7, §10). Formalizes the
`publish` capability from INTENT's adapter-capability list; **complements UC-37** `publish` capability from INTENT's adapter-capability list; **complements UC-37**
(inbound static-export *attach*) as its outbound mirror. Open: core vs. publish-adapter (inbound static-export *attach*) as its outbound mirror. Open: core vs. publish-adapter
family; interaction with overlays and projection freshness (findings §11 Q4). family; interaction with overlays and projection freshness (findings §11 Q4). **Notion**
adds the closed-SaaS instance: **publish-to-web** renders pages as read-only public
pages (`research/260614-notion-deep-dive/findings.md` §4).
**Priority:** Later
### UC-57 — Attach a closed hosted shard via its external API only
**Actor:** Maintainer
**Goal:** Attach a closed, hosted SaaS (Notion-style) that has no file store and no
in-app plugin runtime, reachable only through its external REST API — honoring its rate
limits, eventual consistency, payload/pagination caps, and a scoped, revocable access
grant.
**Source:** notion, intent
**Notes:** Notion is external-REST-attachable with **no in-engine hosting** (contrast
Roam UC-50) but inside a tight operational envelope: ~3 req/s, eventual consistency,
recursive first-level-children fetch, and integrations that see **only explicitly
connected pages** (`research/260614-notion-deep-dive/findings.md` §4, §6). The capability
profile must encode this **operational envelope** and **scoped/revocable grant**;
projection is cache/poll/webhook, not a live read model. Best as projection/mirror/
overlay/backup; write-through possible but bounded. Links the authz-in-core decision and
"no silent remote mutation". Third attachment mode alongside file-store (UC-40) and
in-engine host (UC-38/50).
**Priority:** Later
### UC-58 — Attach a typed database with schema, relations, and views
**Actor:** Maintainer
**Goal:** Attach a structured database (Notion-style) as a shard — preserving its
schema (typed properties), its inter-record **relations** and **rollups/formulas**, and
its multiple **views** — without flattening the schema or the relations into prose.
**Source:** notion, intent
**Notes:** The apex of UC-34/UC-39: not annotations on prose, nor one form per page, but
a **collection with a schema** whose rows are pages joined by **typed, bidirectional
relations**, shown through many views (table/board/calendar/gallery)
(`research/260614-notion-deep-dive/findings.md` §2). Presses the page model: collection +
schema + relations, not just a page (findings §9). Relations map to the union link graph
and/or a relation index (cf. ZigZag many-to-many, UC-47/48); views relate to UC-54.
Deferred to `SHARD-WP-0002`.
**Priority:** Later
### UC-59 — Translate a proprietary content model with an explicit fidelity report
**Actor:** Orchestrator / adapter
**Goal:** Project and edit a shard whose content model does not round-trip to Markdown
(Notion blocks/rich-text, databases, relations) by translating **lossily but
transparently** — surfacing what degraded or did not map, rather than silently dropping
it.
**Source:** notion, intent
**Notes:** Notion is the heaviest translation case — proprietary block + rich-text model,
and its own Markdown/CSV export is lossy (`research/260614-notion-deep-dive/findings.md`
§3). **Distinct from UC-42** (Foswiki TML↔HTML *lossless* round-trip): here translation
is fundamentally lossy, so **fidelity becomes data** — a per-shard/per-page report of
what projects cleanly vs. degrades, with non-mappable elements preserved as
provenance/sidecar. Embodies union-without-erasure by making *loss of fidelity* visible.
Open: report format and where it surfaces (findings §10 Q3).
**Priority:** Later **Priority:** Later
--- ---
@@ -822,6 +896,9 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD.
| UC-54 | | | | ¶ | ✓ | | UC-54 | | | | ¶ | ✓ |
| UC-55 | | | | ¶ | ✓ | | UC-55 | | | | ¶ | ✓ |
| UC-56 | | | | ¶ | ✓ | | UC-56 | | | | ¶ | ✓ |
| UC-57 | | | | ◊ | ✓ |
| UC-58 | | | | ◊ | ✓ |
| UC-59 | | | | ◊ | ✓ |
| UC-08 | ✓ | | | | UC-08 | ✓ | | |
| UC-09 | ✓ | | | | UC-09 | ✓ | | |
| UC-10 | ✓ | | | | UC-10 | ✓ | | |
@@ -1041,6 +1118,39 @@ target (`research/260614-obsidian-deep-dive/findings.md` §8.2). Architecture lo
non-Markdown content types in the page model, outbound publish, external-writer non-Markdown content types in the page model, outbound publish, external-writer
tolerance. tolerance.
### notion mapping
(◊ UC-57–UC-59 are placed in the **wikiengines** matrix column as the nearest existing
source — Notion is a shipped tool — but their true lineage is the **Notion deep dive**,
`research/260614-notion-deep-dive/findings.md`.)
| Notion mechanism (findings §) | Catalog UC |
|-------------------------------|------------|
| Closed hosted SaaS, external-REST-only, rate-limited/eventually-consistent, scoped grant (§4, §6) | UC-57 (new) |
| Database = schema + typed properties + relations + rollups + views (§2) | UC-58 (new) |
| Proprietary block/rich-text, lossy to Markdown (§3) | UC-59 (new) |
| Per-block UUID v4 via REST = store-minted span address (§1) | UC-51 (enriched) |
| Server Postgres + external REST API = block-DB without in-engine hosting (§4) | UC-50 (enriched) |
| Database query API; filtered/linked DBs (§2, §4) | UC-52, UC-54 (enriched) |
| Database-as-pages apex; typed records + relations (§2) | UC-34, UC-39 (enriched) |
| Webhooks (2026) as push transport (§4) | UC-31 (enriched) |
| Internal-only page history, not portable (§4) | UC-36 (enriched) |
| Publish-to-web outbound (§4) | UC-56 (enriched) |
| Scoped, revocable per-integration grant; no silent mutation (§6) | UC-57 + [[shard-wiki-auth-in-core-decision]] |
Note: Notion is the **closed/hosted/schema-rich** extreme and the dive that stress-tests
*graceful degradation*, *no silent remote mutation*, and *Markdown-first that must
degrade*. It adds the **third attachment mode** — external-API-only (full write-through
without in-engine hosting, but bounded by rate limits and eventual consistency) —
alongside file-store (UC-40) and in-engine host (UC-38/50). It also *enforces* no silent
mutation via scoped, revocable per-integration page grants
(`research/260614-notion-deep-dive/findings.md` §6). **Boundary recorded:** one
external-API candidate shard — best as projection/mirror/overlay/backup — not a
substrate and not the federation layer. Architecture logged for `SHARD-WP-0002` (T14):
an operational-envelope capability section (rate limit, consistency class, payload caps,
push-vs-poll), access-grant semantics, a translation-fidelity capability, and the
three-way attachment-mode taxonomy.
--- ---
## Open questions ## Open questions
@@ -1065,4 +1175,9 @@ tolerance.
10. How does shard-wiki represent **non-Markdown content** (UC-55) — typed asset with a 10. How does shard-wiki represent **non-Markdown content** (UC-55) — typed asset with a
Markdown stub, opaque blob with provenance, or a pluggable content-type registry? Markdown stub, opaque blob with provenance, or a pluggable content-type registry?
11. Is a **query-defined page** (UC-54) a core page type, an adapter feature, or a 11. Is a **query-defined page** (UC-54) a core page type, an adapter feature, or a
reference-UI/plugin concern? (Shared with catalog Q7.) reference-UI/plugin concern? (Shared with catalog Q7.)
12. Is **external-API-only write-through** under a tight rate limit (Notion) viable at
wiki scale, or do we default such shards to read/projection/overlay/backup? (Notion
dive §10 Q1; ties UC-57.)
13. How are **inter-database relations** (UC-58) represented in the union — typed links
in the link graph, a separate relation index (cf. ZigZag many-to-many), or both?