diff --git a/SCOPE.md b/SCOPE.md index 08ea96a..7c3e1ce 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary. |-------|-------| | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | -| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian deep dives (`research/260614-*/`) | +| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion deep dives (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | Architecture blueprint drafted; UseCaseCatalog 56 UCs from research; PRD/TSD scaffolds | +| Spec | Architecture blueprint drafted; UseCaseCatalog 59 UCs from research; PRD/TSD scaffolds | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (10 tasks) | ## In Scope (today) diff --git a/research/260614-notion-deep-dive/README.md b/research/260614-notion-deep-dive/README.md new file mode 100644 index 0000000..cbe7937 --- /dev/null +++ b/research/260614-notion-deep-dive/README.md @@ -0,0 +1,56 @@ +# 260614 — Notion deep dive (closed block-database SaaS, attached only through its API) + +Date: 2026-06-14 + +## What this is + +A focused study of **Notion** — architecture and extension model — read through +shard-wiki's lens. Notion is the **closed, hosted, schema-rich** end of the spectrum and +the dive that stress-tests shard-wiki's hardest constraints (*graceful degradation*, +*no silent remote mutation*, *Markdown-first that must degrade*). + +It contrasts sharply with its neighbours: +- vs **Roam** — both are block databases with per-block UUIDs, but Roam is a *client* + DataScript DB reached only via an *in-app* JS API, while Notion is a *server* Postgres + store reached via a real *external* REST API (no in-engine hosting needed, but + rate-limited, eventually consistent, scoped by per-page grants). +- vs **Obsidian** — the mirror image of "file over app": Notion is **app over file**, a + closed hosted store with no portable files. +- vs **XWiki** — both are structured wiki-as-app-platforms; Notion is the **apex of + database-as-pages** (schema + relations + rollups + views) but closed and REST-only. + +Distinctive material: +- **Data model** — everything is a **block** (UUID id, type, properties, ordered child + `content`, single `parent`); pages are blocks, **database rows are pages with a + schema**; Postgres-backed hosted SaaS +- **Databases** — typed properties, **relations** (bidirectional), **rollups**, + **formulas**, and many **views** (table/board/calendar/gallery) of one row-set +- **Extension model** — **no in-app plugin runtime**; extension = **external REST API** + integrations (+ webhooks since 2026), with ~3 rps rate limit, eventual consistency, + recursive child fetch, payload caps, and **scoped/revocable per-page access grants** + +## Contents + +| Path | Role | +|------|------| +| `findings.md` | Block model, databases, page-model impedance, the API-only extension model, capability profile, scoped consent, INTENT mapping, UC seeds, architecture notes, sources | + +## Status + +Initial deep dive complete. Three new use cases promoted to `spec/UseCaseCatalog.md` +(UC-57 attach a closed external-API-only hosted shard with rate-limit/eventual- +consistency/scoped-grant constraints, UC-58 attach a typed database with schema + +relations + views without flattening, UC-59 lossy-aware translation with a fidelity +report); UC-31/34/36/39/50/51/52/54/56 enriched. Logged for `SHARD-WP-0002` (T14): +an **operational-envelope** capability section, **access-grant semantics**, a +**translation-fidelity** capability, and a three-way **attachment-mode taxonomy** +(file-store / in-engine-hosted / external-API-only). + +**Key takeaways recorded:** Notion *enforces* "no silent remote mutation" via scoped, +revocable per-integration grants (ties the authz-in-core decision); it is the heaviest +**lossy-translation** case (proprietary blocks/rich-text, lossy Markdown export); and it +adds the **external-API-only** attachment mode (full write-through without in-engine +hosting, but bounded by rate limits and eventual consistency). **Boundary:** one +external-API candidate shard — best as projection/mirror/overlay/backup — not a +substrate and not the federation layer. + diff --git a/research/260614-notion-deep-dive/findings.md b/research/260614-notion-deep-dive/findings.md new file mode 100644 index 0000000..686b91d --- /dev/null +++ b/research/260614-notion-deep-dive/findings.md @@ -0,0 +1,302 @@ +# Findings — Notion: a closed block-database SaaS, attached only through its API + +Date: 2026-06-14 +Source kind: **modern shipped product** — a hosted block-database SaaS; a *candidate +shard* of a distinct family (closed, external-REST-API-only, schema-rich, heaviest +translation cost); has **no in-app plugin runtime** +Lens: shard-wiki — the external-API attachment mode, database-as-pages, lossy +translation/fidelity, scoped consent, and graceful degradation against a sovereign +closed backend + +> Where Notion sits in the set. Roam and Notion are both **block databases with +> per-block UUIDs**, but they are opposite on access: Roam is a *client-side* +> DataScript DB reachable only through an *in-app* JS API (write-through needs an +> adapter hosted inside Roam); **Notion is a *server-side* Postgres store reachable +> through a real *external* REST API** — attachable from outside with no in-engine +> hosting, but rate-limited, eventually consistent, and **scoped by explicit +> per-integration page grants**. And against Obsidian's "file over app," Notion is the +> mirror image: **app over file** — a closed hosted store with no portable files at +> all. So Notion is the dive that stress-tests shard-wiki's hardest constraints: +> *graceful degradation*, *no silent remote mutation*, *union without erasure*, and +> *Markdown-first that must degrade* against a proprietary, schema-rich, non-Markdown +> backend. + +Pairs with — and contrasts against — the Roam dive (block-DB, in-app vs external API), +the Obsidian dive (closed-hosted vs file-over-app), and the XWiki dive (both are +structured wiki-as-app-platforms; Notion is the apex of database-as-pages but closed +and REST-only). + +--- + +## 1. Core data model — everything is a block + +Notion's data model is uniform: **"Everything you see in Notion is a block. Text, +images, lists, a row in a database, even pages themselves — these are all blocks."** + +Each block record holds: + +| Field | Meaning | +|-------|---------| +| **id** | a randomly generated **UUID v4**, visible in the page URL — the block's stable address | +| **type** | determines rendering + which properties apply (text, to-do, heading, **page**, **child_database**, …) | +| **properties** | type-specific attributes (e.g. `title` text; for database rows, the typed property values) | +| **content** | an **ordered array of child block IDs** — nesting | +| **parent** | a single upward pointer — used for **permission inheritance** | + +Blocks form a **render tree** (content + parent pointers); indentation is *structural*, +not cosmetic — it changes block relationships. **Pages are blocks**; **databases are +blocks whose children are pages**; a database **row is a page** with typed properties. +Stored in **Postgres** on Notion's servers (hosted SaaS — not local, not files). + +The shard-wiki-relevant facts: (a) **per-block UUIDs** give native sub-page addressing +(UC-51); (b) the **page = block, database-row = page-with-schema** identity is a real +page-model impedance (§3); (c) the store is **closed and server-side** — reachable only +through the API (§4). + +--- + +## 2. Databases — schema, typed properties, relations, views + +A Notion **database** is a collection of pages sharing a **schema** of typed +**properties**: `title`, `rich_text`, `select` / `multi_select`, `number`, `date`, +`checkbox`, `person`, `files`, **`relation`** (a typed link to rows in another +database — shown on *both* sides, i.e. bidirectional), **`rollup`** (an aggregate over a +relation), and **`formula`** (computed). The same row-set is shown through multiple +**views** — **table, board (kanban), calendar, gallery, list, timeline** — each a +filtered/sorted/grouped projection of the same data. + +For shard-wiki this is the **apex of "wiki page as structured record"** (stronger than +XWiki XObjects, UC-39): a page can be *bodiless typed data*, pages are joined by +**typed inter-database relations** (a graph of typed links), and "the page" is routinely +**a row in a schema** rather than prose. Notion databases also ship the ZigZag insight +commercially: **one row-set, many views/dimensions** (UC-47/48) and **filtered/linked +databases = query-defined pages** (UC-54). + +--- + +## 3. Page-model impedance + +Mapping Notion to a Markdown-first page model is the **heaviest translation case** in +the research set: + +- **Block/rich-text ≠ Markdown.** Notion's rich text is an annotated-span model; many + block types (synced blocks, columns, callouts, embeds, database views) have no clean + Markdown equivalent. Notion's own **export to Markdown/CSV is lossy** (databases → + CSV, relations/rollups/formulas flatten or drop). +- **Database-row-as-page + schema** must map onto pages + sidecar metadata without + discarding the schema or the relations (extends UC-34/UC-39). +- Therefore translation must be **lossy-aware with a fidelity report** — surface what + did *not* round-trip rather than silently flattening (UC-59). This is *different from* + UC-42 (Foswiki TML↔HTML *lossless* round-trip); Notion is fundamentally lossy. + +--- + +## 4. Extension model — there is no plugin runtime; only the API + +This is the defining architectural fact for shard-wiki: **Notion has no third-party +in-app plugin/extension system.** There is no marketplace of code that runs inside +Notion (the only "in-app modding" is the *unofficial* Notion Enhancer desktop patcher, +out of scope). **Extension = external integration via the public REST API** (plus +embeds and, recently, webhooks). + +The **public REST API** (`api.notion.com/v1`): + +- **Resources:** `pages`, `blocks` (retrieve / append / update / delete children), + `databases` (retrieve, **query** with filters+sorts, create), `users`, `search`, + `comments`. +- **Authorization:** an **internal integration token** (workspace-owned) or **OAuth + 2.0** (public integrations). Critically, **an integration only sees pages a user has + explicitly connected to it** — the user "approves the app and connects specific + pages" via *Add connections*. Access is a **scoped, revocable, per-page grant** (§6). +- **Operational constraints the adapter must encode:** + - **Rate limit** ~**3 requests/second** average (429 `rate_limited`); **no paid + increase**. + - **Eventual consistency / no live read model** — the API is not real-time; reads can + lag. + - **Recursive retrieval** — a block returns only **first-level children**; full-page + reconstruction requires walking `has_children` recursively. + - **Payload caps** — 1000 blocks / 500 KB per request; child arrays ≤ 100. + - **Webhooks** (added 2026) deliver page/database change events — a push transport for + UC-31, replacing pure polling. +- **Version history:** Notion keeps **internal page history** (retention bounded by + plan); it is **not portable** and not exposed as git — a UC-36 *supplementation* case + (like Confluence/MediaWiki), not an import case. +- **Publish:** pages can be **published to the web** as read-only public pages — an + outbound publish surface (UC-56). + +Consequence: Notion is attachable **only** as an **external-API shard** — but, unlike +Roam, **no in-engine adapter hosting is needed** (the REST API is external). The cost is +the operational envelope (rate-limit, eventual consistency, scoped grant) that the +capability profile must model (UC-57). + +--- + +## 5. Notion as a shard — capability profile + +| Capability | Notion | Notes for the adapter contract | +|------------|--------|--------------------------------| +| Read | **yes (external REST)** | rate-limited (~3 rps), eventually consistent, recursive child fetch | +| Write | **yes (external REST)** | append/update/delete blocks, create pages — no in-engine host needed (vs Roam) | +| Write granularity | **block-level (fine)** | like Roam; per-block ops | +| Identity / addressing | **block UUID v4** | native sub-page addressing (UC-51), store-minted (like Roam, not in-file) | +| Structured data | **yes (apex)** | databases: schema + typed properties + relations + rollups + formulas (UC-34/39/58) | +| Native query | **yes** | database query API (filters/sorts) → delegate views (UC-52) | +| Views / dimensions | **yes** | table/board/calendar/gallery = many views of one row-set (UC-47/48/54) | +| Subscribe | **webhooks (2026)** | push events; else poll (UC-31) | +| Version history | **internal, not portable** | supplement via coordination journal (UC-36) | +| Diff / merge | **no native** | — | +| Lock | **no** | — | +| Publish | **publish-to-web** | outbound read-only (UC-56) | +| Access model | **scoped per-page grant (OAuth/token)** | explicit consent; revocable (UC-57, authz) | +| Syntax / content | **proprietary block + rich text** | **lossy** to Markdown; needs fidelity-aware translation (UC-59) | + +Verdict: Notion is a legitimate but **demanding** shard — **external-API-attached, +schema-rich, fine-grained, closed**. It behaves best as a **projected / mirrored / +overlay / backup** participant; full write-through is possible but bounded by rate limits +and eventual consistency. The strongest reasons to attach it: structured databases +(UC-58) and block-UUID addressing (UC-51); the strongest cautions: lossy translation +(UC-59) and no portable history (UC-36). + +--- + +## 6. Scoped consent and "no silent remote mutation" + +Notion *enforces* one of shard-wiki's INTENT constraints at the platform level: an +integration can touch **only** the pages a user has explicitly **connected** to it, and +the grant is **revocable**. This is a clean, real-world model of **no silent remote +mutation** and of a shard granting the orchestrator **scoped, consented access** — and +it ties directly to the settled **authz-in-core / authn-delegated** decision +([[shard-wiki-auth-in-core-decision]]): authentication to Notion is delegated (OAuth / +integration token), while shard-wiki's *own* authorization decides what to do with the +granted scope. The adapter contract should treat **"scoped, revocable grant"** as a +first-class attachment property (UC-57), not an afterthought. + +--- + +## 7. Mapping to shard-wiki INTENT (compare, do not equate) + +### 7.1 Reinforcements + +- **Graceful degradation** has its sharpest test here: a closed SaaS with no files, rate + limits, eventual consistency, and lossy export must still be usable as + read/projection/overlay/backup. If the adapter contract handles Notion, it handles + most things. +- **No silent remote mutation** is *modeled by the platform* (scoped grants) — Notion + validates the principle (§6). +- **Database-as-pages** validates that "wiki page" must stretch to **typed records with + relations**, not just prose (UC-34/39/58). +- **Block UUIDs** reconfirm (with Roam) that native sub-page addressing is real and + adoptable (UC-51). + +### 7.2 Deliberate divergences (design bugs if conflated) + +1. **Closed hosted store; no sovereignty over bytes.** shard-wiki cannot make Notion + git-native or local. It can mirror/project/overlay/back-up and **supply** a + git-addressable history (UC-36) — never claim to own Notion's store. +2. **Lossy, proprietary content.** Do **not** pretend Notion round-trips to Markdown. + Translate lossily *with a fidelity report* and preserve non-mappable elements as + provenance/sidecar (UC-59) — union without erasure includes erasure *of fidelity* + being made visible. +3. **External-API-only, rate-limited, eventually consistent.** Projection must be + **cache/poll/webhook**, not a live read model; sync is bounded — encode this in the + capability profile (UC-57). Do not design flows that assume cheap, instant, unlimited + reads. +4. **One workspace (or its granted page set) = one shard**, never the federation layer. + +### 7.3 What Notion teaches that shard-wiki should keep + +- Model **operational envelope** (rate limit, consistency class, payload caps, + pagination) as explicit capability-profile fields — Notion makes them unignorable. +- Model **scoped, revocable consent** as a first-class attachment property (UC-57, §6). +- Treat **translation fidelity as data**: a per-shard, per-page report of what projects + cleanly vs. degrades (UC-59) — applies beyond Notion. +- Recognize **external-REST attach** as a distinct, *preferred-where-available* mode: + full write-through without in-engine hosting (contrast Roam) — but pay the operational + envelope. + +--- + +## 8. Use-case seeds → catalog (promoted 2026-06-14) + +Last existing UC is **UC-56**. New UCs **UC-57–UC-59** added; existing UCs enriched. + +| Seed | Catalog action | +|------|----------------| +| **Attach a closed hosted shard via its external REST API only** — no file store, no in-app runtime — honoring rate limits, eventual consistency, payload caps, and a scoped/revocable access grant | **UC-57 (new)** | +| **Attach a typed database (schema + relations + rollups + multiple views) as a shard** without flattening the schema or the inter-record relations | **UC-58 (new)** | +| **Translate a proprietary block/rich-text model to/from Markdown with an explicit fidelity report**, preserving non-mappable elements rather than silently dropping them | **UC-59 (new)** | +| Block UUIDs = store-minted native span addresses (external-API variant) | **enriches UC-51** | +| External-API block-DB attach (no in-engine host) — contrast Roam's in-app-only | **enriches UC-50** | +| Database query API + filtered/linked DBs | **enriches UC-52, UC-54** | +| Database-as-pages apex; typed records + relations | **enriches UC-34, UC-39** | +| Webhooks (2026) as a push transport | **enriches UC-31** | +| Internal-only page history, not portable | **enriches UC-36** | +| Publish-to-web outbound | **enriches UC-56** | +| Scoped, revocable per-integration grant; no silent mutation | links **UC-57** + [[shard-wiki-auth-in-core-decision]] | + +--- + +## 9. Architecture notes for SHARD-WP-0002 (no UC) + +- Add an **operational-envelope** section to the adapter capability profile: rate limit, + consistency class (live / eventually-consistent / snapshot), payload/pagination caps, + recursive-fetch requirement, push-vs-poll transport. Notion is the forcing example. +- Add **access-grant semantics**: scope (which pages), revocability, auth mode + (delegated token/OAuth) — ties the authz-in-core decision and "no silent mutation". +- Add a **translation-fidelity capability**: adapters declare and report what content + round-trips vs. degrades (UC-59); generalizes UC-42 (lossless) to the lossy case. +- **Attachment-mode taxonomy** now spans: file-store direct (Obsidian/TWiki, UC-40), + in-engine hosted adapter (Roam/XWiki, UC-38/50), and **external-API-only** + (Notion, UC-57). T14 binding should enumerate all three. +- **Database/schema/relations** as a unit (UC-58) presses the page-model spec: collection + + schema + typed relations, not just a page. + +--- + +## 10. Open questions (for spec / workplans) + +1. Is **external-API-only with a tight rate limit** (Notion) viable for write-through at + wiki scale, or do we cap Notion at read/projection/overlay/backup by default? +2. How are **inter-database relations** (UC-58) represented in the union — as typed links + in the link graph, as a separate relation index (cf. ZigZag many-to-many), or both? +3. What is the **fidelity report** format (UC-59), and where does it surface — provenance + panel, projection metadata, reconciliation review? +4. For **scoped grants** (§6), how does shard-wiki represent partial visibility (only + *some* of a workspace's pages granted) without misrepresenting the shard as complete? +5. Do we consume Notion **webhooks** (push) or poll, given eventual consistency and the + rate limit (UC-31)? + +--- + +## 11. Sources + +| Source | Used for | +|--------|----------| +| Notion — "The data model behind Notion's flexibility" (https://www.notion.com/blog/data-model-behind-notion) | Everything-is-a-block; block record (id/type/properties/content/parent); render tree; pages/databases as blocks; Postgres | +| Notion Docs — Request limits (https://developers.notion.com/reference/request-limits) | ~3 rps rate limit, 429, payload caps, recursive first-level children | +| Notion Docs — Authorization (https://developers.notion.com/docs/authorization) | Internal token vs OAuth; integration connected to specific pages; scoped grant | +| Hookdeck / ClickUp — Notion webhooks guides (https://hookdeck.com/webhooks/platforms/guide-to-notion-webhooks-features-and-best-practices) | Webhook support (2026), page/database change events | +| Truto / Rollout — Notion API architecture & essentials (https://truto.one/blog/how-to-integrate-with-the-notion-api-architecture-guide-for-b2b-saas/) | REST endpoints (pages/blocks/databases/search), integration patterns, no in-app plugin model | +| General API knowledge — database property types, views, relations/rollups/formulas, export-to-Markdown lossiness | §2, §3 | + +Cross-references: `research/260614-roam-deep-dive/findings.md` (block-DB/UUID, in-app vs +external API), `research/260614-obsidian-deep-dive/findings.md` (closed-hosted vs +file-over-app), `research/260613-xwiki-deep-dive/findings.md` (structured wiki-app- +platform), `spec/UseCaseCatalog.md` (UC-31, UC-34, UC-36, UC-39, UC-50/51/52, UC-54, +UC-56), `workplans/SHARD-WP-0002-federation-architecture.md` (T14), and the authz +decision [[shard-wiki-auth-in-core-decision]]. + +--- + +## 12. Traceability + +- New UCs: **UC-57, UC-58, UC-59** → `spec/UseCaseCatalog.md`. +- Enriched UCs: **UC-31, UC-34, UC-36, UC-39, UC-50, UC-51, UC-52, UC-54, UC-56**. +- Architecture (no UC): operational-envelope + access-grant + translation-fidelity + capability fields; three-way attachment-mode taxonomy; database/schema/relations in the + page model → `SHARD-WP-0002` (T14). +- Decision link: scoped/revocable grant + no-silent-mutation → [[shard-wiki-auth-in-core-decision]]. +- Boundary recorded: Notion is **one external-API candidate shard** — closed, hosted, + schema-rich, lossy-to-Markdown — best as projection/mirror/overlay/backup; not a + substrate, not the federation layer (INTENT graceful-degradation + no-silent-mutation). + diff --git a/research/README.md b/research/README.md index d61f27b..ad620e9 100644 --- a/research/README.md +++ b/research/README.md @@ -20,4 +20,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and | 2026-06-14 | `260614-xanadu-deep-dive/` | Project Xanadu — EDL/reference-not-copy, transclusion, addressing; UC-44/45/46 | | 2026-06-14 | `260614-zigzag-deep-dive/` | ZigZag/zzstructure — information space as orthogonal dimensions; UC-47/48/49 | | 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 | -| 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 | \ No newline at end of file +| 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 | +| 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 | \ No newline at end of file diff --git a/spec/UseCaseCatalog.md b/spec/UseCaseCatalog.md index 1a64ae2..3afc09c 100644 --- a/spec/UseCaseCatalog.md +++ b/spec/UseCaseCatalog.md @@ -7,7 +7,8 @@ Promoted from `research/260608-c2-wiki-origins/`, `research/260608-wikiengines-overview/`, `research/260613-xwiki-deep-dive/`, `research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`, `research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`, -`research/260614-roam-deep-dive/`, and `research/260614-obsidian-deep-dive/`. +`research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`, and +`research/260614-notion-deep-dive/`. See InfoTechPrimers on coulomb.social for use-case catalog conventions. ## Conventions @@ -177,7 +178,10 @@ triggering projection refresh or reconciliation review. Transport is adapter concern; freshness surfaced in UC-24. XWiki deep dive adds a concrete push transport: an engine event bus (`ObservationManager` `DocumentUpdatedEvent`) consumed by a listener, vs polling REST history -(`research/260613-xwiki-deep-dive/findings.md` §2.5, §8 Q4). +(`research/260613-xwiki-deep-dive/findings.md` §2.5, §8 Q4). Notion adds **webhooks** +(2026) delivering page/database change events — a SaaS push transport, weighed against +eventual consistency and the rate limit (`research/260614-notion-deep-dive/findings.md` +§4). **Priority:** Later ### UC-32 — Transclude remote span with live freshness @@ -230,7 +234,9 @@ topic text file*, so structure is diffable rather than locked in a DB sidecar metadata, and a candidate for query delegation (UC-52). Obsidian shows the **git-diffable in-file** variant: YAML frontmatter/properties live in the Markdown text and are queried by the Dataview plugin (`research/260614-obsidian-deep-dive/findings.md` -§3) — structure as portable text, not DB state. +§3) — structure as portable text, not DB state. **Notion** is the apex of the DB-locked +end: typed database properties with relations/rollups/formulas, lossy to Markdown — +see UC-58 (`research/260614-notion-deep-dive/findings.md` §2). **Priority:** Later ### UC-35 — Attach a shard with coarse write granularity @@ -265,7 +271,10 @@ internal RCS table (`xwikircs`) is a concrete instance (TWiki RCS `.txt,v`) it can instead be **imported** — see UC-41. Demand evidence: **Obsidian Git is a top-7 plugin (2.7M)** — users manually bolt portable git history onto a vault that has none natively (`research/260614-obsidian-deep-dive/findings.md` -§5, §7), exactly the gap the coordination journal fills as core. +§5, §7), exactly the gap the coordination journal fills as core. **Notion** is the +closed-SaaS instance: internal page history bounded by plan, **not portable** and not +exported as git (`research/260614-notion-deep-dive/findings.md` §4) — a supplementation +case like Confluence/MediaWiki, not an import case. **Priority:** Later ### UC-37 — Attach a static engine export as a read-only backup shard @@ -317,7 +326,9 @@ or can a page be purely typed data (`research/260613-xwiki-deep-dive/findings.md §8 Q2)? Affects union views, diff, and search. Deferred to `SHARD-WP-0002`. Foswiki shows a middle point: MetaDataPlugin stores **multiple** structured records per topic, still as `%META%` in text (`research/260613-foswiki-deep-dive/findings.md` -§4) — the page model must allow N typed records, not one form. +§4) — the page model must allow N typed records, not one form. **Notion** pushes +further: a *database* is a collection with a schema whose rows are pages joined by typed +relations — see UC-58 (`research/260614-notion-deep-dive/findings.md` §2). **Priority:** Later ### UC-40 — Attach a file-backed engine's on-disk store directly @@ -482,7 +493,11 @@ flattening the structure. = page, nested blocks = spans, attributes = sidecar metadata (honors UC-34 no-lossy- flatten). DB-backed/API-attached path (like XWiki UC-38), **not** the file-store path (UC-40). Write-through requires hosting the adapter inside the engine (Roam Depot), -else degrade to read/projection/overlay-target (findings §11 Q2). +else degrade to read/projection/overlay-target (findings §11 Q2). **Notion** is the +external-API variant of the same block-DB family: a server Postgres store with a real +**external** REST API, so write-through needs **no in-engine hosting** — but is bounded +by rate limits and eventual consistency (UC-57, +`research/260614-notion-deep-dive/findings.md` §4). **Priority:** Later ### UC-51 — Adopt a shard's native span IDs as portable span addresses @@ -501,7 +516,8 @@ a shard-scoped address to avoid cross-shard collision and survive projection (fi §11 Q1). Obsidian shows the **text-embedded** variant: `^block-id` and heading anchors live in the Markdown text — git-diffable and portable (survives a file copy) but opt-in (`research/260614-obsidian-deep-dive/findings.md` §2), vs Roam's mandatory DB-minted -UID. +UID. Notion is a second store-minted case: per-block **UUID v4**, exposed via the REST +API (`research/260614-notion-deep-dive/findings.md` §1). **Priority:** Later ### UC-52 — Delegate derived views to a shard's native query engine @@ -517,7 +533,9 @@ projection). Operationalizes the ZigZag "dimensions + rasters" insight (`research/260614-zigzag-deep-dive/findings.md` §5) and relates to UC-05/UC-34. Obsidian nuance: query can be an *ecosystem plugin* (Dataview), **not core** — so "native query" is adapter/plugin-provided and must not be assumed -(`research/260614-obsidian-deep-dive/findings.md` §7). +(`research/260614-obsidian-deep-dive/findings.md` §7). **Notion** ships a native +database **query API** (filters/sorts) — a strong delegation target +(`research/260614-notion-deep-dive/findings.md` §2, §4). **Priority:** Later ### UC-53 — Attach a local-first Markdown vault with a live concurrent native editor @@ -547,7 +565,9 @@ Roam `{{query}}` — query-defined dynamic pages (`research/260614-obsidian-deep-dive/findings.md` §7). Distinct from UC-44 (reference/ span manifest) and UC-52 (delegating *computation* of an existing view): here the *page itself* is a query. Open: core page type vs. adapter vs. reference-UI/plugin (findings -§11 Q2). May delegate execution to a shard's native query engine (UC-52). +§11 Q2). May delegate execution to a shard's native query engine (UC-52). **Notion** +linked/filtered databases are the commercial instance — one row-set surfaced through many +filtered views (`research/260614-notion-deep-dive/findings.md` §2). **Priority:** Later ### UC-55 — Carry non-Markdown content types as typed or opaque assets @@ -575,7 +595,61 @@ read-only target — a static site or hosted page set — preserving provenance. publish (`research/260614-obsidian-deep-dive/findings.md` §7, §10). Formalizes the `publish` capability from INTENT's adapter-capability list; **complements UC-37** (inbound static-export *attach*) as its outbound mirror. Open: core vs. publish-adapter -family; interaction with overlays and projection freshness (findings §11 Q4). +family; interaction with overlays and projection freshness (findings §11 Q4). **Notion** +adds the closed-SaaS instance: **publish-to-web** renders pages as read-only public +pages (`research/260614-notion-deep-dive/findings.md` §4). +**Priority:** Later + +### UC-57 — Attach a closed hosted shard via its external API only + +**Actor:** Maintainer +**Goal:** Attach a closed, hosted SaaS (Notion-style) that has no file store and no +in-app plugin runtime, reachable only through its external REST API — honoring its rate +limits, eventual consistency, payload/pagination caps, and a scoped, revocable access +grant. +**Source:** notion, intent +**Notes:** Notion is external-REST-attachable with **no in-engine hosting** (contrast +Roam UC-50) but inside a tight operational envelope: ~3 req/s, eventual consistency, +recursive first-level-children fetch, and integrations that see **only explicitly +connected pages** (`research/260614-notion-deep-dive/findings.md` §4, §6). The capability +profile must encode this **operational envelope** and **scoped/revocable grant**; +projection is cache/poll/webhook, not a live read model. Best as projection/mirror/ +overlay/backup; write-through possible but bounded. Links the authz-in-core decision and +"no silent remote mutation". Third attachment mode alongside file-store (UC-40) and +in-engine host (UC-38/50). +**Priority:** Later + +### UC-58 — Attach a typed database with schema, relations, and views + +**Actor:** Maintainer +**Goal:** Attach a structured database (Notion-style) as a shard — preserving its +schema (typed properties), its inter-record **relations** and **rollups/formulas**, and +its multiple **views** — without flattening the schema or the relations into prose. +**Source:** notion, intent +**Notes:** The apex of UC-34/UC-39: not annotations on prose, nor one form per page, but +a **collection with a schema** whose rows are pages joined by **typed, bidirectional +relations**, shown through many views (table/board/calendar/gallery) +(`research/260614-notion-deep-dive/findings.md` §2). Presses the page model: collection + +schema + relations, not just a page (findings §9). Relations map to the union link graph +and/or a relation index (cf. ZigZag many-to-many, UC-47/48); views relate to UC-54. +Deferred to `SHARD-WP-0002`. +**Priority:** Later + +### UC-59 — Translate a proprietary content model with an explicit fidelity report + +**Actor:** Orchestrator / adapter +**Goal:** Project and edit a shard whose content model does not round-trip to Markdown +(Notion blocks/rich-text, databases, relations) by translating **lossily but +transparently** — surfacing what degraded or did not map, rather than silently dropping +it. +**Source:** notion, intent +**Notes:** Notion is the heaviest translation case — proprietary block + rich-text model, +and its own Markdown/CSV export is lossy (`research/260614-notion-deep-dive/findings.md` +§3). **Distinct from UC-42** (Foswiki TML↔HTML *lossless* round-trip): here translation +is fundamentally lossy, so **fidelity becomes data** — a per-shard/per-page report of +what projects cleanly vs. degrades, with non-mappable elements preserved as +provenance/sidecar. Embodies union-without-erasure by making *loss of fidelity* visible. +Open: report format and where it surfaces (findings §10 Q3). **Priority:** Later --- @@ -822,6 +896,9 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD. | UC-54 | | | | ¶ | ✓ | | UC-55 | | | | ¶ | ✓ | | UC-56 | | | | ¶ | ✓ | +| UC-57 | | | | ◊ | ✓ | +| UC-58 | | | | ◊ | ✓ | +| UC-59 | | | | ◊ | ✓ | | UC-08 | ✓ | | | | UC-09 | ✓ | | | | UC-10 | ✓ | | | @@ -1041,6 +1118,39 @@ target (`research/260614-obsidian-deep-dive/findings.md` §8.2). Architecture lo non-Markdown content types in the page model, outbound publish, external-writer tolerance. +### notion mapping + +(◊ UC-57–UC-59 are placed in the **wikiengines** matrix column as the nearest existing +source — Notion is a shipped tool — but their true lineage is the **Notion deep dive**, +`research/260614-notion-deep-dive/findings.md`.) + +| Notion mechanism (findings §) | Catalog UC | +|-------------------------------|------------| +| Closed hosted SaaS, external-REST-only, rate-limited/eventually-consistent, scoped grant (§4, §6) | UC-57 (new) | +| Database = schema + typed properties + relations + rollups + views (§2) | UC-58 (new) | +| Proprietary block/rich-text, lossy to Markdown (§3) | UC-59 (new) | +| Per-block UUID v4 via REST = store-minted span address (§1) | UC-51 (enriched) | +| Server Postgres + external REST API = block-DB without in-engine hosting (§4) | UC-50 (enriched) | +| Database query API; filtered/linked DBs (§2, §4) | UC-52, UC-54 (enriched) | +| Database-as-pages apex; typed records + relations (§2) | UC-34, UC-39 (enriched) | +| Webhooks (2026) as push transport (§4) | UC-31 (enriched) | +| Internal-only page history, not portable (§4) | UC-36 (enriched) | +| Publish-to-web outbound (§4) | UC-56 (enriched) | +| Scoped, revocable per-integration grant; no silent mutation (§6) | UC-57 + [[shard-wiki-auth-in-core-decision]] | + +Note: Notion is the **closed/hosted/schema-rich** extreme and the dive that stress-tests +*graceful degradation*, *no silent remote mutation*, and *Markdown-first that must +degrade*. It adds the **third attachment mode** — external-API-only (full write-through +without in-engine hosting, but bounded by rate limits and eventual consistency) — +alongside file-store (UC-40) and in-engine host (UC-38/50). It also *enforces* no silent +mutation via scoped, revocable per-integration page grants +(`research/260614-notion-deep-dive/findings.md` §6). **Boundary recorded:** one +external-API candidate shard — best as projection/mirror/overlay/backup — not a +substrate and not the federation layer. Architecture logged for `SHARD-WP-0002` (T14): +an operational-envelope capability section (rate limit, consistency class, payload caps, +push-vs-poll), access-grant semantics, a translation-fidelity capability, and the +three-way attachment-mode taxonomy. + --- ## Open questions @@ -1065,4 +1175,9 @@ tolerance. 10. How does shard-wiki represent **non-Markdown content** (UC-55) — typed asset with a Markdown stub, opaque blob with provenance, or a pluggable content-type registry? 11. Is a **query-defined page** (UC-54) a core page type, an adapter feature, or a - reference-UI/plugin concern? (Shared with catalog Q7.) \ No newline at end of file + reference-UI/plugin concern? (Shared with catalog Q7.) +12. Is **external-API-only write-through** under a tight rate limit (Notion) viable at + wiki scale, or do we default such shards to read/projection/overlay/backup? (Notion + dive §10 Q1; ties UC-57.) +13. How are **inter-database relations** (UC-58) represented in the union — typed links + in the link graph, a separate relation index (cf. ZigZag many-to-many), or both? \ No newline at end of file