From e00c160a4399aaa5e0768b790f2f8f98fe479258 Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 14 Jun 2026 14:21:31 +0200 Subject: [PATCH] research: Joplin deep dive (SQLite-local/Markdown-on-sync, interchange-format attach, E2EE content opacity); UC-60/61 Architecturally distinct from the Obsidian/Roam/Notion trio: content is Markdown, the local store is SQLite, and the sync representation is a folder of per-item Markdown+metadata files on a backend of choice (filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin Server/Cloud). Two new findings: (1) attach the sync/interchange mirror, not the app or the SQLite store, offline and app-independent, landing on INTENT's WebDAV/Nextcloud/S3 backends (UC-60, distinct from native on-disk store UC-40 and app-files UC-53); (2) E2EE => content opacity, a proposed twelfth capability spectrum for encrypted-at-rest shards (UC-61). Also: Joplin is the file-sync-daemon boundary case (attach the mirror as pages, never re-sync); store-minted page-level :/id links (addressing spectrum middle); dual surface (plugin host + local Data API on localhost:41184). Enriched UC-31/36/38/40/51/55. Catalog now 61 UCs. Architecture for SHARD-WP-0002 T11/T14: interchange-format attach, content-opacity field, local-REST sub-mode, format-aware file-store profiles. Co-Authored-By: Claude Opus 4.8 --- SCOPE.md | 4 +- research/260614-joplin-deep-dive/README.md | 49 ++++ research/260614-joplin-deep-dive/findings.md | 269 +++++++++++++++++++ research/README.md | 3 +- spec/UseCaseCatalog.md | 106 +++++++- 5 files changed, 419 insertions(+), 12 deletions(-) create mode 100644 research/260614-joplin-deep-dive/README.md create mode 100644 research/260614-joplin-deep-dive/findings.md diff --git a/SCOPE.md b/SCOPE.md index dba591c..2dff729 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary. |-------|-------| | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | -| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion deep dives (`research/260614-*/`) | +| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | Architecture blueprint drafted; UseCaseCatalog 59 UCs from research; PRD/TSD scaffolds | +| Spec | Architecture blueprint drafted; UseCaseCatalog 61 UCs from research; PRD/TSD scaffolds | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract) | ## In Scope (today) diff --git a/research/260614-joplin-deep-dive/README.md b/research/260614-joplin-deep-dive/README.md new file mode 100644 index 0000000..d4fb59d --- /dev/null +++ b/research/260614-joplin-deep-dive/README.md @@ -0,0 +1,49 @@ +# 260614 — Joplin deep dive (SQLite-local, Markdown-on-sync, interchange-format attach) + +Date: 2026-06-14 + +## What this is + +A focused study of **Joplin** — the open-source Markdown note app — read through +shard-wiki's lens. Joplin looks like "open-source Obsidian" but is a genuinely different +point in the space: **content is Markdown, the local store is SQLite, and the *sync +representation* is a folder of per-item Markdown+metadata files on a backend you choose** +(filesystem, WebDAV, **Nextcloud**, Dropbox, OneDrive, **S3**, Joplin Server/Cloud). It +also ships **end-to-end encryption** and is itself a **sync layer over heterogeneous +storage**. + +Distinctive material: +- **Architecture** — SQLite local store (documented schema); Markdown content; + notebooks → notes → resources + tags + to-dos; 32-char item IDs; `:/` note links + (rename-stable, page-level) +- **Sync** — serialized to plain-text items on a chosen target; tombstones; **conflict + notes** (keep-both) → enables the **"attach the sync mirror, not the app"** pattern on + WebDAV/Nextcloud/S3 +- **E2EE** — items encrypted before leaving the device → **content opacity** (ciphertext + at rest) +- **Extension surfaces** — a **plugin API** (`joplin.data`/`workspace`/`contentScripts`/ + views) *and* a **local Data API** (REST on `localhost:41184`, token) used by the Web + Clipper; plus JEX/Markdown/RAW export + +## Contents + +| Path | Role | +|------|------| +| `findings.md` | Architecture, sync/interchange attach, E2EE/content-opacity, plugin + Data API surfaces, capability profile, INTENT mapping, UC seeds, architecture notes, sources | + +## Status + +Initial deep dive complete. Two new use cases promoted to `spec/UseCaseCatalog.md` +(UC-60 attach a tool's documented sync/interchange representation on third-party storage +without the app; UC-61 attach an encrypted-at-rest shard with content opacity); +UC-31/36/38/40/51/55 enriched. Logged for `SHARD-WP-0002` (T11, T14): the +interchange/sync-representation attach surface, **content opacity** as a proposed twelfth +capability spectrum, a local-REST attach sub-mode, and format-aware file-store profiles. + +**Key takeaways recorded:** the best attach surface is often a tool's **interchange/sync +representation** (offline, app-independent — Joplin items on Nextcloud/WebDAV/S3, the +INTENT-named backends), not its live store or API; **encryption-at-rest** demands a +content-opacity capability so encrypted shards degrade to backup/structure-shell; and +Joplin is the **file-sync-daemon boundary case** — shard-wiki attaches its mirror as +pages and never re-syncs (not-a-sync-daemon). + diff --git a/research/260614-joplin-deep-dive/findings.md b/research/260614-joplin-deep-dive/findings.md new file mode 100644 index 0000000..a38d2cc --- /dev/null +++ b/research/260614-joplin-deep-dive/findings.md @@ -0,0 +1,269 @@ +# Findings — Joplin: SQLite locally, Markdown-on-sync, and the interchange-format attach + +Date: 2026-06-14 +Source kind: **modern shipped product** — an open-source Markdown note app; a *candidate +shard* whose distinctive trait is a **documented file-based sync/interchange format on +third-party storage** plus optional **end-to-end encryption** +Lens: shard-wiki — attachment surfaces, the "attach the sync mirror not the app" +pattern, content opacity (encryption), note-level stable IDs, and the file-sync-daemon +boundary + +> Why Joplin earns a dive after Obsidian/Roam/Notion. It looks like "Obsidian but +> open-source," yet its architecture is a genuinely different point in the space: +> **content is Markdown but the local store is SQLite, and the *sync representation* is +> a folder of per-item Markdown+metadata files on a backend you choose** (filesystem, +> WebDAV, **Nextcloud**, Dropbox, OneDrive, **S3**, Joplin Server/Cloud). That sync +> mirror — not the app, not the SQLite DB — is the interesting attach surface, and it +> lands squarely on the **WebDAV/Nextcloud backends INTENT already names**. Joplin also +> ships **E2EE**, forcing the question of a shard whose content is *opaque ciphertext*, +> and it is itself a **sync layer over heterogeneous storage** — the very "file-sync +> daemon" shard-wiki says it is *not*, making Joplin a sharp boundary case. + +Contrast set: Obsidian (file-over-app: native store *is* files), Roam (client DB, +in-app API only), Notion (hosted DB, external API only), TWiki/Foswiki (file+RCS native +store). Joplin is none of these exactly — **DB-local, files-on-sync** — which is the +whole point. + +--- + +## 1. Core architecture — SQLite local, Markdown content + +- **Local store:** a **SQLite database** (`~/.config/joplin-desktop/database.sqlite`), + schema documented. So, unlike Obsidian, the *native* local store is **not** plain + files — you cannot point shard-wiki at a folder of `.md` and read the live store + (you'd read SQLite, fragile while the app runs). +- **Content:** notes are **Markdown** (Joplin-flavored: `:/imageId` resource links, + internal `:/` note links, checkboxes/to-dos). Markdown-first — INTENT-aligned at + the *content* layer even though the *storage* layer is a DB. +- **Model:** `folders` (notebooks, hierarchical) → `notes` → `resources` (attachments); + cross-cutting `tags`; notes can be **to-dos**. Every item has a **32-char hex ID**; + notes link by ID (`:/`), so links survive rename/move (store-minted, page-level). +- **Clients:** desktop (Electron), mobile, and a **terminal/CLI** app — multiple clients + edit one logical store, reconciled by sync (§2). +- **Revisions:** Joplin keeps **internal note history** locally — not git, not portable + → a *supplement* case (UC-36), like Notion/Confluence. + +--- + +## 2. Sync — the distinctive layer (and the attach opportunity) + +Joplin's headline feature is **sync to a backend of your choice**. On sync, the SQLite +store is serialized to a **folder of plain-text items** — one file per note / notebook / +tag / resource-metadata, each carrying the **Markdown body plus a metadata block** +(`id`, `parent_id`, `type_`, `created_time`, `updated_time`, …) — written to the chosen +target: **filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin Server, Joplin +Cloud**. Deletions propagate via tombstones; concurrent edits across clients produce +**conflict notes** (keep-both, not silent overwrite). + +Two consequences for shard-wiki: + +1. **Attach the sync mirror, not the app (UC-60).** A Nextcloud/WebDAV/S3/filesystem + directory full of Joplin sync items is a **file-store shard** — readable *without + Joplin running and without touching SQLite* — provided the adapter understands the + **documented item format** (body + metadata footer, `:/id` links, resource items). + This is a new attach pattern: the backend's **interchange/sync representation** as + the attach surface, distinct from a native on-disk store (UC-40) or app files (UC-53). + It also realizes INTENT's WebDAV/Nextcloud participants concretely. +2. **Joplin is a file-sync layer over heterogeneous storage** — exactly what shard-wiki + says it is *not* (INTENT: "not a file-sync daemon"). Joplin syncs **one** logical + store across backends; shard-wiki **federates many** logical stores with wiki-page + semantics. Attaching a Joplin sync mirror means **reading its file representation as + pages**, never re-driving Joplin's sync — and never becoming a second sync engine + over the same target (don't double-sync). + +--- + +## 3. End-to-end encryption — content opacity + +Joplin offers **E2EE**: items are encrypted **before** leaving the device, so the sync +target holds **ciphertext** regardless of provider. (Security wart, noted: the master +key is stored *in clear* in the local SQLite — irrelevant to attaching the mirror, but +a reason never to treat the local DB as trusted.) + +For shard-wiki this introduces a capability dimension the prior dives did not: +**content opacity**. An encrypted Joplin sync target can be attached only as a +**backup/mirror/structure-shell** participant — item IDs, counts, and change events may +be visible, but **bodies are undecryptable without keys**. The adapter must **never +present ciphertext as readable content** and must degrade to "present-but-opaque" with +provenance. This extends the synthesis capability spectra with a proposed twelfth field, +**content opacity** (`plaintext → encrypted-at-rest/opaque`), feeding `SHARD-WP-0002` +T11 (§9). + +--- + +## 4. Extension surfaces — a plugin host *and* a local REST API + +Joplin exposes **two** programmatic surfaces (plus export): + +**A. Plugin API (in-app host).** TypeScript plugins, distributed via **npm + a plugin +repository**, loaded by the app: +- `joplin.data` — CRUD over the data model, shaped like REST (GET/POST/PUT/DELETE) on + `notes`/`folders`/`tags`/`resources`. +- `joplin.workspace` — selection + events (note selected, content changed, sync events). +- `joplin.contentScripts` — **markdown-it** render plugins and **CodeMirror** editor + extensions (the syntax/rendering extension point). +- `joplin.views` — panels, dialogs, toolbar/menu items, custom editors; + `joplin.commands`, `joplin.settings`. + +**B. Data API (local REST).** A REST service the desktop app serves on +**`localhost:41184`** (the "clipper server"), **token-authenticated**, used by the Web +Clipper and any local integration: endpoints `/notes`, `/folders`, `/tags`, +`/resources` (incl. `/resources/:id/file`), `/revisions`, `/search`; verbs +GET/POST/PUT/DELETE; pagination (`page`, `limit`≤100, `order_by`, `order_dir`). Notably +**plugins can use this API even when the clipper server is off**. + +**C. Export** — **JEX** (tar of raw items), **Markdown directory**, **RAW** (Joplin +export directory), **ENEX** import (Evernote). Snapshot import/attach surface (UC-28/37). + +So Joplin spans attachment modes: **in-app plugin host** (like Roam/Obsidian-plugin), +**local-REST** (like Notion's external API but *localhost + app-must-run* — a sub-mode), +the **sync-mirror file-store** (§2, the novel one), and **export snapshots**. + +--- + +## 5. Joplin as a shard — capability profile + +| Capability | Joplin | Notes for the adapter contract | +|------------|--------|--------------------------------| +| Read | **yes** | sync-mirror files (no app) · local Data API (app running) · plugin · export | +| Write | **yes** | Data API / plugin (`joplin.data`); writing the sync mirror directly is risky (Joplin owns that format) | +| Write granularity | **per-note (page)** | notebooks/notes/resources; not block-level | +| Identity / addressing | **32-char item ID; `:/` links** | store-minted, **page-level**, survives rename (between Obsidian path and Roam block-UUID) (UC-51) | +| Structure | notebooks (hierarchy) + tags + to-dos | modest; metadata in the sync item footer (in-text on the mirror) | +| History | **internal revisions, not portable** | supplement via coordination journal (UC-36) | +| Native query | search API (FTS) | delegate text search; no datalog/DB-query (weaker than Roam/Notion) | +| Subscribe | sync is **poll-based**; conflict notes | poll; conflicts surface as keep-both items (UC-31, UC-07) | +| Content opacity | **optional E2EE → ciphertext** | **new dimension**: opaque-at-rest; mirror usable only as backup/structure without keys (UC-61) | +| Transclusion | none (note-level links only) | — | +| Diff / merge | none native | git-level if mirror is on a git-backed target | +| Publish | via export / static-site pipelines | outbound (UC-56) | +| Content types | Markdown + **resources** (attachments) | non-Markdown assets with IDs (UC-55) | +| Attach modes | sync-mirror file-store · local-REST · plugin · export | multi-mode, per-binding (UC-60, UC-57, UC-38) | + +Verdict: a solid **Markdown-first candidate shard** whose best attach surface is the +**sync mirror on a file/WebDAV/S3 target** (offline, app-independent, INTENT-named), +with the local Data API or a plugin for live write-through. Standout: the +interchange-format attach (UC-60) and the encryption/opacity case (UC-61). Limits: DB +local store, note-level (not block) addressing, internal-only history, weak query. + +--- + +## 6. Mapping to shard-wiki INTENT (compare, do not equate) + +### 6.1 Reinforcements + +- **Interchange-format attach** validates that a shard's *documented sync/export + representation* — not just its live store or API — is a legitimate, often *best*, + attach surface (offline, app-independent). Generalizes beyond Joplin (UC-60). +- **WebDAV/Nextcloud/S3 become concrete** participants via Joplin's sync targets — the + backends INTENT names, now with a real format to parse. +- **Conflict-as-data** (Joplin conflict notes = keep-both) is union-without-erasure in + the wild (reinforces UC-07, divergence; consensus policy T9). +- **Store-minted, rename-stable note IDs** (`:/id`) confirm the addressing spectrum's + middle (page-level stable ID) between path (Obsidian) and block-UUID (Roam/Notion) + (UC-51). + +### 6.2 Deliberate divergences (design bugs if conflated) + +1. **Joplin is a sync daemon; shard-wiki is not.** Attach the **mirror as pages**; never + re-implement Joplin's sync or run a competing sync over the same target. (INTENT + not-a-file-sync-daemon.) +2. **DB-local; don't read SQLite live.** The native store is a DB; treat it as opaque. + Use the sync mirror, the Data API, or a plugin — not the live `database.sqlite`. +3. **Joplin owns the sync format; treat the mirror read-mostly.** Writing items into the + sync folder behind Joplin's back risks corruption/conflict; prefer overlay/projection + or write through the Data API/plugin (overlay before mutation, no silent mutation). +4. **Encrypted shards are opaque.** Never present ciphertext as content; degrade to + backup/structure with provenance (UC-61, graceful degradation). + +### 6.3 What Joplin teaches that shard-wiki should keep + +- Add **"interchange/sync representation"** as a recognized attach surface in the + contract — sometimes preferable to the live store/API (UC-60). +- Add **content opacity** (encryption-at-rest) as a capability-profile field so encrypted + shards degrade correctly (UC-61). +- Expect **multi-client concurrent editing reconciled by the backend's own sync** + (desktop/mobile/CLI) — a different concurrency than single-app file edits (extends + UC-53) or multi-user server writes; conflicts may pre-exist as keep-both items. + +--- + +## 7. Use-case seeds → catalog (promoted 2026-06-14) + +Last existing UC is **UC-59**. New UCs **UC-60, UC-61** added; existing UCs enriched. + +| Seed | Catalog action | +|------|----------------| +| **Attach a tool's documented sync/interchange representation on a third-party storage target** (Joplin items on Nextcloud/WebDAV/S3) as a file-store shard, without the app | **UC-60 (new)** | +| **Attach an encrypted-at-rest shard** (E2EE sync target): content opaque without keys; participate as backup/mirror/structure-shell, never presenting ciphertext as readable | **UC-61 (new)** | +| Store-minted, rename-stable **note-level** IDs (`:/id`) — the middle of the addressing spectrum | **enriches UC-51** | +| Native store can be a **DB while the sync representation is files** → attach surface ≠ native store | **enriches UC-40** | +| Internal revisions, not portable → supplement | **enriches UC-36** | +| Dual surfaces: in-app plugin host + **local Data API** (localhost, app-running) | **enriches UC-38** (links UC-57) | +| Resources/attachments = non-Markdown assets with IDs | **enriches UC-55** | +| Poll-based sync; conflict notes (keep-both) | **enriches UC-31** (links UC-07) | + +--- + +## 8. Architecture notes for SHARD-WP-0002 (no UC) + +- **Attachment-mode taxonomy gains a refinement:** within file-store, distinguish + *native on-disk store* (Obsidian/TWiki, UC-40) from *interchange/sync representation* + (Joplin mirror, UC-60). Both are "files," but format ownership and write-safety differ. + (T14.) +- **Add a "content opacity" capability field** (proposed **twelfth spectrum**: + `plaintext → encrypted-at-rest/opaque`) so encrypted shards degrade to + backup/structure-shell. Feeds T11 the next time the eleven spectra + (`research/260614-shard-spectrum-synthesis/findings.md` §2) are revised. (T11/T14.) +- **Local-REST as an attach sub-mode** (localhost, app-must-run, token) sits between + in-engine-host (UC-38) and external-API (UC-57) — note in T14. +- **Format-aware file-store adapters:** the contract should let a file-store adapter + declare a *format profile* (plain Markdown vs Joplin-item vs Foswiki-PlainFile) so the + same "folder of files" attach can parse tool-specific item formats. (T11/T14.) + +--- + +## 9. Open questions (for spec / workplans) + +1. For an **encrypted shard** (UC-61), what *is* visible without keys — item IDs/counts/ + timestamps (structure shell) or nothing? Does shard-wiki ever hold keys, or only ever + treat such shards as opaque backups? +2. Is **writing to a tool's sync mirror** (UC-60) ever safe, or are interchange-format + shards **read/projection/overlay-only** by policy (Joplin owns the format)? +3. Does shard-wiki parse Joplin's **item format** via a dedicated format profile, or + only attach the **export** (JEX/Markdown dir) as a cleaner snapshot (UC-37)? +4. How do we avoid **double-sync** when a shard's storage target is itself driven by a + sync daemon (Joplin) we don't control? + +--- + +## 10. Sources + +| Source | Used for | +|--------|----------| +| Joplin Data API reference (https://joplinapp.org/help/api/references/rest_api/) | Local REST on `localhost:41184`, token auth, endpoints, 32-char IDs, item fields, pagination | +| Joplin Plugin API docs (https://joplinapp.org/api/references/plugin_api/classes/joplin.html) | `joplin.data` / `joplin.workspace` / `joplin.contentScripts` / views / settings | +| Joplin — Extending Joplin (https://joplinapp.org/help/api/) | Plugin distribution (npm + repository), Data API vs plugin access | +| Joplin — WebDAV / Nextcloud sync (https://joplinapp.org/help/apps/sync/nextcloud/) | Sync targets; items as plain-text files on the target | +| Joplin FAQ + community (https://joplinapp.org/help/faq/) | SQLite local store; Markdown content; E2EE before leaving device | +| Obsidian vs Joplin comparison (https://petronellatech.com/blog/obsidian-vs-joplin/) | Architecture contrast (SQLite+sync vs file-over-app), E2EE framing | + +Cross-references: `research/260614-obsidian-deep-dive/findings.md` (file-over-app +contrast), `research/260614-notion-deep-dive/findings.md` (external-API, scoped grant), +`research/260614-shard-spectrum-synthesis/findings.md` (the spectra this extends), +`spec/UseCaseCatalog.md` (UC-31, UC-36, UC-38, UC-40, UC-51, UC-55, UC-57), +`workplans/SHARD-WP-0002-federation-architecture.md` (T11, T14). + +--- + +## 11. Traceability + +- New UCs: **UC-60, UC-61** → `spec/UseCaseCatalog.md`. +- Enriched UCs: **UC-31, UC-36, UC-38, UC-40, UC-51, UC-55** (links UC-07, UC-57). +- Architecture (no UC): interchange/sync-representation attach surface; **content + opacity** as a proposed twelfth capability spectrum; local-REST sub-mode; format-aware + file-store profiles → `SHARD-WP-0002` (T11, T14). +- Boundary recorded: Joplin is a **sync daemon over one store**; shard-wiki attaches its + **mirror as pages** and does not re-sync — and treats DB-local store and encrypted + content as opaque (INTENT not-a-sync-daemon, graceful degradation, no silent mutation). + diff --git a/research/README.md b/research/README.md index fd0f71b..fbfab31 100644 --- a/research/README.md +++ b/research/README.md @@ -22,4 +22,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and | 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 | | 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 | | 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 | -| 2026-06-14 | `260614-shard-spectrum-synthesis/` | Synthesis — shard family matrix + eleven capability spectra across nine systems; feeds SHARD-WP-0002 T11–T16 | \ No newline at end of file +| 2026-06-14 | `260614-shard-spectrum-synthesis/` | Synthesis — shard family matrix + eleven capability spectra across nine systems; feeds SHARD-WP-0002 T11–T16 | +| 2026-06-14 | `260614-joplin-deep-dive/` | Joplin — SQLite-local/Markdown-on-sync, interchange-format attach, E2EE content opacity; UC-60/61 | \ No newline at end of file diff --git a/spec/UseCaseCatalog.md b/spec/UseCaseCatalog.md index 3afc09c..6e47425 100644 --- a/spec/UseCaseCatalog.md +++ b/spec/UseCaseCatalog.md @@ -7,8 +7,8 @@ Promoted from `research/260608-c2-wiki-origins/`, `research/260608-wikiengines-overview/`, `research/260613-xwiki-deep-dive/`, `research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`, `research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`, -`research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`, and -`research/260614-notion-deep-dive/`. +`research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`, +`research/260614-notion-deep-dive/`, and `research/260614-joplin-deep-dive/`. See InfoTechPrimers on coulomb.social for use-case catalog conventions. ## Conventions @@ -181,7 +181,9 @@ concrete push transport: an engine event bus (`ObservationManager` (`research/260613-xwiki-deep-dive/findings.md` §2.5, §8 Q4). Notion adds **webhooks** (2026) delivering page/database change events — a SaaS push transport, weighed against eventual consistency and the rate limit (`research/260614-notion-deep-dive/findings.md` -§4). +§4). **Joplin** is poll-based and turns concurrent edits into **conflict notes** +(keep-both, not silent overwrite) — conflict-as-data, links UC-07 +(`research/260614-joplin-deep-dive/findings.md` §2). **Priority:** Later ### UC-32 — Transclude remote span with live freshness @@ -274,7 +276,9 @@ onto a vault that has none natively (`research/260614-obsidian-deep-dive/finding §5, §7), exactly the gap the coordination journal fills as core. **Notion** is the closed-SaaS instance: internal page history bounded by plan, **not portable** and not exported as git (`research/260614-notion-deep-dive/findings.md` §4) — a supplementation -case like Confluence/MediaWiki, not an import case. +case like Confluence/MediaWiki, not an import case. **Joplin** keeps internal note +revisions locally, likewise not portable (`research/260614-joplin-deep-dive/findings.md` +§1) — supplement. **Priority:** Later ### UC-37 — Attach a static engine export as a read-only backup shard @@ -310,7 +314,11 @@ Generalizes beyond XWiki: TWiki's plugin handler API (`beforeSaveHandler`, template: an extension default-exports `onload`/`onunload` and drives `window.roamAlphaAPI` read (`q`/`pull`) + write (`block.*`/`page.create`) (`research/260614-roam-deep-dive/findings.md` §5) — note Roam's API runs *inside* the -client, so write-through requires in-engine hosting (findings §11 Q2). +client, so write-through requires in-engine hosting (findings §11 Q2). **Joplin** offers +*two* surfaces: an in-app plugin host (`joplin.data`/`workspace`/`contentScripts`) **and** +a **local Data API** (REST on `localhost:41184`, token, app-running) — a local-REST attach +sub-mode between in-engine host and external API (UC-57, +`research/260614-joplin-deep-dive/findings.md` §4). **Priority:** Later ### UC-39 — Attach a wiki-as-application-platform shard (pages as typed records) @@ -350,7 +358,10 @@ direct-attach target than RCS diffs (`research/260613-foswiki-deep-dive/findings §2.2). An **Obsidian vault** is the cleanest case of all — a plain Markdown folder, Markdown-first and git-friendly — and uniquely **dual-attachable**: file-store direct *or* via an in-app plugin adapter for write-through + live file events (UC-53, -`research/260614-obsidian-deep-dive/findings.md` §4, §10). +`research/260614-obsidian-deep-dive/findings.md` §4, §10). Counter-example: **Joplin**'s +native store is a **DB** (SQLite), so its file-attach surface is the *sync/interchange +mirror* on a WebDAV/S3 target, not the native store — see UC-60 +(`research/260614-joplin-deep-dive/findings.md` §2). **Priority:** Later ### UC-41 — Import an engine's native file history into the coordination journal @@ -517,7 +528,10 @@ a shard-scoped address to avoid cross-shard collision and survive projection (fi live in the Markdown text — git-diffable and portable (survives a file copy) but opt-in (`research/260614-obsidian-deep-dive/findings.md` §2), vs Roam's mandatory DB-minted UID. Notion is a second store-minted case: per-block **UUID v4**, exposed via the REST -API (`research/260614-notion-deep-dive/findings.md` §1). +API (`research/260614-notion-deep-dive/findings.md` §1). **Joplin** marks the spectrum's +middle: a store-minted **page-level** 32-char ID with `:/` links that survive +rename/move (`research/260614-joplin-deep-dive/findings.md` §1) — coarser than a block +UUID, more stable than a path. **Priority:** Later ### UC-52 — Delegate derived views to a shard's native query engine @@ -582,7 +596,9 @@ content in "Markdown" vaults (`research/260614-obsidian-deep-dive/findings.md` Extends UC-34's no-lossy-flatten rule from structured-text to binary/spatial content. Pushes the **wiki page model**: page vs. typed asset vs. opaque blob — a page-model decision, not just adapter config (findings §10). JSON Canvas is an open format worth -first-class support. +first-class support. **Joplin** resources (attachments, each with an ID, linked `:/`) +are the same demand in a sync-mirror shard (`research/260614-joplin-deep-dive/findings.md` +§5). **Priority:** Later ### UC-56 — Publish a curated projection to an external read-only target @@ -652,6 +668,42 @@ provenance/sidecar. Embodies union-without-erasure by making *loss of fidelity* Open: report format and where it surfaces (findings §10 Q3). **Priority:** Later +### UC-60 — Attach a tool's documented sync / interchange representation + +**Actor:** Maintainer +**Goal:** Attach a backend's documented **sync or interchange representation** — a folder +of per-item files a tool writes to a third-party storage target — as a file-store shard, +without running the tool and without touching its native store. +**Source:** joplin, intent +**Notes:** Joplin's native store is **SQLite**, but on sync it serializes to per-item +**Markdown+metadata files** on filesystem / WebDAV / **Nextcloud** / Dropbox / OneDrive / +**S3** (`research/260614-joplin-deep-dive/findings.md` §2). That mirror is attachable +offline and app-independently *if* the adapter understands the documented item format +(body + metadata footer, `:/id` links, resources). **Distinct from UC-40** (native +on-disk store) and **UC-53** (app files *are* the store): here the attach surface is the +**interchange/sync representation**, and the tool **owns the format** → treat +read/projection/overlay-mostly, never re-sync (INTENT not-a-file-sync-daemon). Realizes +INTENT's WebDAV/Nextcloud/S3 participants. Needs a **format profile** in the adapter +(findings §8). +**Priority:** Later + +### UC-61 — Attach an encrypted-at-rest shard with content opacity + +**Actor:** Maintainer +**Goal:** Attach a shard whose stored content is encrypted (E2EE sync target) and +participate correctly when bodies are undecryptable — as backup / mirror / structure- +shell — without ever presenting ciphertext as readable content. +**Source:** joplin, intent +**Notes:** Joplin encrypts items **before** they leave the device, so the sync target +holds ciphertext (`research/260614-joplin-deep-dive/findings.md` §3). Introduces a new +capability dimension — **content opacity** (`plaintext → encrypted-at-rest/opaque`) — +proposed as a twelfth spectrum for the adapter contract (findings §8; extends +`research/260614-shard-spectrum-synthesis/findings.md` §2). Graceful degradation extreme: +IDs/counts/timestamps may be visible while bodies are opaque; never silently surface +undecryptable content. Open: does shard-wiki ever hold keys, or only treat such shards as +opaque backups (findings §9 Q1)? Ties [[shard-wiki-auth-in-core-decision]]. +**Priority:** Later + --- ## B. Knowledge work and collaboration @@ -899,6 +951,8 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD. | UC-57 | | | | ◊ | ✓ | | UC-58 | | | | ◊ | ✓ | | UC-59 | | | | ◊ | ✓ | +| UC-60 | | | | ‖ | ✓ | +| UC-61 | | | | ‖ | ✓ | | UC-08 | ✓ | | | | UC-09 | ✓ | | | | UC-10 | ✓ | | | @@ -1151,6 +1205,36 @@ an operational-envelope capability section (rate limit, consistency class, paylo push-vs-poll), access-grant semantics, a translation-fidelity capability, and the three-way attachment-mode taxonomy. +### joplin mapping + +(‖ UC-60–UC-61 are placed in the **wikiengines** matrix column as the nearest existing +source — Joplin is a shipped tool — but their true lineage is the **Joplin deep dive**, +`research/260614-joplin-deep-dive/findings.md`.) + +| Joplin mechanism (findings §) | Catalog UC | +|-------------------------------|------------| +| SQLite-local, Markdown-on-sync; per-item files on WebDAV/Nextcloud/S3 (§2) | UC-60 (new) | +| E2EE → ciphertext at rest; content opacity (§3) | UC-61 (new) | +| Store-minted page-level 32-char ID, rename-stable `:/id` links (§1) | UC-51 (enriched) | +| Native store is a DB; attach surface is the sync mirror, not the store (§2) | UC-40 (enriched) | +| Internal revisions, not portable (§1) | UC-36 (enriched) | +| Plugin host + local Data API (localhost:41184, token) (§4) | UC-38 (enriched; links UC-57) | +| Resources (attachments) with IDs (§5) | UC-55 (enriched) | +| Poll sync; conflict notes (keep-both) (§2) | UC-31 (enriched; links UC-07) | + +Note: Joplin is the **SQLite-local / Markdown-on-sync** case — its best attach surface is +the **interchange/sync representation** (per-item files on a WebDAV/Nextcloud/S3 target), +offline and app-independent, landing on INTENT's named backends. It adds two findings for +the adapter contract: the **interchange/sync-representation attach surface** (distinct +from native on-disk store UC-40 and app-files UC-53), and **content opacity** (a proposed +twelfth capability spectrum for encrypted-at-rest shards, extending +`research/260614-shard-spectrum-synthesis/findings.md` §2). **Boundary recorded:** Joplin +is a **file-sync daemon over one store**; shard-wiki attaches its **mirror as pages** and +never re-syncs, and treats the DB-local store and encrypted content as opaque +(`research/260614-joplin-deep-dive/findings.md` §6). Architecture logged for +`SHARD-WP-0002` (T11/T14): interchange-format attach, content-opacity field, local-REST +sub-mode, format-aware file-store profiles. + --- ## Open questions @@ -1180,4 +1264,8 @@ three-way attachment-mode taxonomy. wiki scale, or do we default such shards to read/projection/overlay/backup? (Notion dive §10 Q1; ties UC-57.) 13. How are **inter-database relations** (UC-58) represented in the union — typed links - in the link graph, a separate relation index (cf. ZigZag many-to-many), or both? \ No newline at end of file + in the link graph, a separate relation index (cf. ZigZag many-to-many), or both? +14. For an **encrypted shard** (UC-61), what is visible without keys (IDs/structure vs + nothing), and does shard-wiki ever hold keys or only treat it as an opaque backup? +15. Is writing to a tool's **sync mirror** (UC-60) ever safe, or are interchange-format + shards read/projection/overlay-only by policy? (Joplin dive §9.) \ No newline at end of file