research: Joplin deep dive (SQLite-local/Markdown-on-sync, interchange-format attach, E2EE content opacity); UC-60/61

Architecturally distinct from the Obsidian/Roam/Notion trio: content is
Markdown, the local store is SQLite, and the sync representation is a
folder of per-item Markdown+metadata files on a backend of choice
(filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin
Server/Cloud). Two new findings: (1) attach the sync/interchange mirror,
not the app or the SQLite store, offline and app-independent, landing on
INTENT's WebDAV/Nextcloud/S3 backends (UC-60, distinct from native
on-disk store UC-40 and app-files UC-53); (2) E2EE => content opacity, a
proposed twelfth capability spectrum for encrypted-at-rest shards (UC-61).
Also: Joplin is the file-sync-daemon boundary case (attach the mirror as
pages, never re-sync); store-minted page-level :/id links (addressing
spectrum middle); dual surface (plugin host + local Data API on
localhost:41184). Enriched UC-31/36/38/40/51/55. Catalog now 61 UCs.
Architecture for SHARD-WP-0002 T11/T14: interchange-format attach,
content-opacity field, local-REST sub-mode, format-aware file-store
profiles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

research/260614-joplin-deep-dive/README.md

@@ -0,0 +1,49 @@
+# 260614 — Joplin deep dive (SQLite-local, Markdown-on-sync, interchange-format attach)
+
+Date: 2026-06-14
+
+## What this is
+
+A focused study of **Joplin** — the open-source Markdown note app — read through
+shard-wiki's lens. Joplin looks like "open-source Obsidian" but is a genuinely different
+point in the space: **content is Markdown, the local store is SQLite, and the *sync
+representation* is a folder of per-item Markdown+metadata files on a backend you choose**
+(filesystem, WebDAV, **Nextcloud**, Dropbox, OneDrive, **S3**, Joplin Server/Cloud). It
+also ships **end-to-end encryption** and is itself a **sync layer over heterogeneous
+storage**.
+
+Distinctive material:
+- **Architecture** — SQLite local store (documented schema); Markdown content;
+  notebooks → notes → resources + tags + to-dos; 32-char item IDs; `:/<id>` note links
+  (rename-stable, page-level)
+- **Sync** — serialized to plain-text items on a chosen target; tombstones; **conflict
+  notes** (keep-both) → enables the **"attach the sync mirror, not the app"** pattern on
+  WebDAV/Nextcloud/S3
+- **E2EE** — items encrypted before leaving the device → **content opacity** (ciphertext
+  at rest)
+- **Extension surfaces** — a **plugin API** (`joplin.data`/`workspace`/`contentScripts`/
+  views) *and* a **local Data API** (REST on `localhost:41184`, token) used by the Web
+  Clipper; plus JEX/Markdown/RAW export
+
+## Contents
+
+| Path | Role |
+|------|------|
+| `findings.md` | Architecture, sync/interchange attach, E2EE/content-opacity, plugin + Data API surfaces, capability profile, INTENT mapping, UC seeds, architecture notes, sources |
+
+## Status
+
+Initial deep dive complete. Two new use cases promoted to `spec/UseCaseCatalog.md`
+(UC-60 attach a tool's documented sync/interchange representation on third-party storage
+without the app; UC-61 attach an encrypted-at-rest shard with content opacity);
+UC-31/36/38/40/51/55 enriched. Logged for `SHARD-WP-0002` (T11, T14): the
+interchange/sync-representation attach surface, **content opacity** as a proposed twelfth
+capability spectrum, a local-REST attach sub-mode, and format-aware file-store profiles.
+
+**Key takeaways recorded:** the best attach surface is often a tool's **interchange/sync
+representation** (offline, app-independent — Joplin items on Nextcloud/WebDAV/S3, the
+INTENT-named backends), not its live store or API; **encryption-at-rest** demands a
+content-opacity capability so encrypted shards degrade to backup/structure-shell; and
+Joplin is the **file-sync-daemon boundary case** — shard-wiki attaches its mirror as
+pages and never re-syncs (not-a-sync-daemon).
+</content>

research/260614-joplin-deep-dive/findings.md

@@ -0,0 +1,269 @@
+# Findings — Joplin: SQLite locally, Markdown-on-sync, and the interchange-format attach
+
+Date: 2026-06-14
+Source kind: **modern shipped product** — an open-source Markdown note app; a *candidate
+shard* whose distinctive trait is a **documented file-based sync/interchange format on
+third-party storage** plus optional **end-to-end encryption**
+Lens: shard-wiki — attachment surfaces, the "attach the sync mirror not the app"
+pattern, content opacity (encryption), note-level stable IDs, and the file-sync-daemon
+boundary
+
+> Why Joplin earns a dive after Obsidian/Roam/Notion. It looks like "Obsidian but
+> open-source," yet its architecture is a genuinely different point in the space:
+> **content is Markdown but the local store is SQLite, and the *sync representation* is
+> a folder of per-item Markdown+metadata files on a backend you choose** (filesystem,
+> WebDAV, **Nextcloud**, Dropbox, OneDrive, **S3**, Joplin Server/Cloud). That sync
+> mirror — not the app, not the SQLite DB — is the interesting attach surface, and it
+> lands squarely on the **WebDAV/Nextcloud backends INTENT already names**. Joplin also
+> ships **E2EE**, forcing the question of a shard whose content is *opaque ciphertext*,
+> and it is itself a **sync layer over heterogeneous storage** — the very "file-sync
+> daemon" shard-wiki says it is *not*, making Joplin a sharp boundary case.
+
+Contrast set: Obsidian (file-over-app: native store *is* files), Roam (client DB,
+in-app API only), Notion (hosted DB, external API only), TWiki/Foswiki (file+RCS native
+store). Joplin is none of these exactly — **DB-local, files-on-sync** — which is the
+whole point.
+
+---
+
+## 1. Core architecture — SQLite local, Markdown content
+
+- **Local store:** a **SQLite database** (`~/.config/joplin-desktop/database.sqlite`),
+  schema documented. So, unlike Obsidian, the *native* local store is **not** plain
+  files — you cannot point shard-wiki at a folder of `.md` and read the live store
+  (you'd read SQLite, fragile while the app runs).
+- **Content:** notes are **Markdown** (Joplin-flavored: `:/imageId` resource links,
+  internal `:/<id>` note links, checkboxes/to-dos). Markdown-first — INTENT-aligned at
+  the *content* layer even though the *storage* layer is a DB.
+- **Model:** `folders` (notebooks, hierarchical) → `notes` → `resources` (attachments);
+  cross-cutting `tags`; notes can be **to-dos**. Every item has a **32-char hex ID**;
+  notes link by ID (`:/<id>`), so links survive rename/move (store-minted, page-level).
+- **Clients:** desktop (Electron), mobile, and a **terminal/CLI** app — multiple clients
+  edit one logical store, reconciled by sync (§2).
+- **Revisions:** Joplin keeps **internal note history** locally — not git, not portable
+  → a *supplement* case (UC-36), like Notion/Confluence.
+
+---
+
+## 2. Sync — the distinctive layer (and the attach opportunity)
+
+Joplin's headline feature is **sync to a backend of your choice**. On sync, the SQLite
+store is serialized to a **folder of plain-text items** — one file per note / notebook /
+tag / resource-metadata, each carrying the **Markdown body plus a metadata block**
+(`id`, `parent_id`, `type_`, `created_time`, `updated_time`, …) — written to the chosen
+target: **filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin Server, Joplin
+Cloud**. Deletions propagate via tombstones; concurrent edits across clients produce
+**conflict notes** (keep-both, not silent overwrite).
+
+Two consequences for shard-wiki:
+
+1. **Attach the sync mirror, not the app (UC-60).** A Nextcloud/WebDAV/S3/filesystem
+   directory full of Joplin sync items is a **file-store shard** — readable *without
+   Joplin running and without touching SQLite* — provided the adapter understands the
+   **documented item format** (body + metadata footer, `:/id` links, resource items).
+   This is a new attach pattern: the backend's **interchange/sync representation** as
+   the attach surface, distinct from a native on-disk store (UC-40) or app files (UC-53).
+   It also realizes INTENT's WebDAV/Nextcloud participants concretely.
+2. **Joplin is a file-sync layer over heterogeneous storage** — exactly what shard-wiki
+   says it is *not* (INTENT: "not a file-sync daemon"). Joplin syncs **one** logical
+   store across backends; shard-wiki **federates many** logical stores with wiki-page
+   semantics. Attaching a Joplin sync mirror means **reading its file representation as
+   pages**, never re-driving Joplin's sync — and never becoming a second sync engine
+   over the same target (don't double-sync).
+
+---
+
+## 3. End-to-end encryption — content opacity
+
+Joplin offers **E2EE**: items are encrypted **before** leaving the device, so the sync
+target holds **ciphertext** regardless of provider. (Security wart, noted: the master
+key is stored *in clear* in the local SQLite — irrelevant to attaching the mirror, but
+a reason never to treat the local DB as trusted.)
+
+For shard-wiki this introduces a capability dimension the prior dives did not:
+**content opacity**. An encrypted Joplin sync target can be attached only as a
+**backup/mirror/structure-shell** participant — item IDs, counts, and change events may
+be visible, but **bodies are undecryptable without keys**. The adapter must **never
+present ciphertext as readable content** and must degrade to "present-but-opaque" with
+provenance. This extends the synthesis capability spectra with a proposed twelfth field,
+**content opacity** (`plaintext → encrypted-at-rest/opaque`), feeding `SHARD-WP-0002`
+T11 (§9).
+
+---
+
+## 4. Extension surfaces — a plugin host *and* a local REST API
+
+Joplin exposes **two** programmatic surfaces (plus export):
+
+**A. Plugin API (in-app host).** TypeScript plugins, distributed via **npm + a plugin
+repository**, loaded by the app:
+- `joplin.data` — CRUD over the data model, shaped like REST (GET/POST/PUT/DELETE) on
+  `notes`/`folders`/`tags`/`resources`.
+- `joplin.workspace` — selection + events (note selected, content changed, sync events).
+- `joplin.contentScripts` — **markdown-it** render plugins and **CodeMirror** editor
+  extensions (the syntax/rendering extension point).
+- `joplin.views` — panels, dialogs, toolbar/menu items, custom editors;
+  `joplin.commands`, `joplin.settings`.
+
+**B. Data API (local REST).** A REST service the desktop app serves on
+**`localhost:41184`** (the "clipper server"), **token-authenticated**, used by the Web
+Clipper and any local integration: endpoints `/notes`, `/folders`, `/tags`,
+`/resources` (incl. `/resources/:id/file`), `/revisions`, `/search`; verbs
+GET/POST/PUT/DELETE; pagination (`page`, `limit`≤100, `order_by`, `order_dir`). Notably
+**plugins can use this API even when the clipper server is off**.
+
+**C. Export** — **JEX** (tar of raw items), **Markdown directory**, **RAW** (Joplin
+export directory), **ENEX** import (Evernote). Snapshot import/attach surface (UC-28/37).
+
+So Joplin spans attachment modes: **in-app plugin host** (like Roam/Obsidian-plugin),
+**local-REST** (like Notion's external API but *localhost + app-must-run* — a sub-mode),
+the **sync-mirror file-store** (§2, the novel one), and **export snapshots**.
+
+---
+
+## 5. Joplin as a shard — capability profile
+
+| Capability | Joplin | Notes for the adapter contract |
+|------------|--------|--------------------------------|
+| Read | **yes** | sync-mirror files (no app) · local Data API (app running) · plugin · export |
+| Write | **yes** | Data API / plugin (`joplin.data`); writing the sync mirror directly is risky (Joplin owns that format) |
+| Write granularity | **per-note (page)** | notebooks/notes/resources; not block-level |
+| Identity / addressing | **32-char item ID; `:/<id>` links** | store-minted, **page-level**, survives rename (between Obsidian path and Roam block-UUID) (UC-51) |
+| Structure | notebooks (hierarchy) + tags + to-dos | modest; metadata in the sync item footer (in-text on the mirror) |
+| History | **internal revisions, not portable** | supplement via coordination journal (UC-36) |
+| Native query | search API (FTS) | delegate text search; no datalog/DB-query (weaker than Roam/Notion) |
+| Subscribe | sync is **poll-based**; conflict notes | poll; conflicts surface as keep-both items (UC-31, UC-07) |
+| Content opacity | **optional E2EE → ciphertext** | **new dimension**: opaque-at-rest; mirror usable only as backup/structure without keys (UC-61) |
+| Transclusion | none (note-level links only) | — |
+| Diff / merge | none native | git-level if mirror is on a git-backed target |
+| Publish | via export / static-site pipelines | outbound (UC-56) |
+| Content types | Markdown + **resources** (attachments) | non-Markdown assets with IDs (UC-55) |
+| Attach modes | sync-mirror file-store · local-REST · plugin · export | multi-mode, per-binding (UC-60, UC-57, UC-38) |
+
+Verdict: a solid **Markdown-first candidate shard** whose best attach surface is the
+**sync mirror on a file/WebDAV/S3 target** (offline, app-independent, INTENT-named),
+with the local Data API or a plugin for live write-through. Standout: the
+interchange-format attach (UC-60) and the encryption/opacity case (UC-61). Limits: DB
+local store, note-level (not block) addressing, internal-only history, weak query.
+
+---
+
+## 6. Mapping to shard-wiki INTENT (compare, do not equate)
+
+### 6.1 Reinforcements
+
+- **Interchange-format attach** validates that a shard's *documented sync/export
+  representation* — not just its live store or API — is a legitimate, often *best*,
+  attach surface (offline, app-independent). Generalizes beyond Joplin (UC-60).
+- **WebDAV/Nextcloud/S3 become concrete** participants via Joplin's sync targets — the
+  backends INTENT names, now with a real format to parse.
+- **Conflict-as-data** (Joplin conflict notes = keep-both) is union-without-erasure in
+  the wild (reinforces UC-07, divergence; consensus policy T9).
+- **Store-minted, rename-stable note IDs** (`:/id`) confirm the addressing spectrum's
+  middle (page-level stable ID) between path (Obsidian) and block-UUID (Roam/Notion)
+  (UC-51).
+
+### 6.2 Deliberate divergences (design bugs if conflated)
+
+1. **Joplin is a sync daemon; shard-wiki is not.** Attach the **mirror as pages**; never
+   re-implement Joplin's sync or run a competing sync over the same target. (INTENT
+   not-a-file-sync-daemon.)
+2. **DB-local; don't read SQLite live.** The native store is a DB; treat it as opaque.
+   Use the sync mirror, the Data API, or a plugin — not the live `database.sqlite`.
+3. **Joplin owns the sync format; treat the mirror read-mostly.** Writing items into the
+   sync folder behind Joplin's back risks corruption/conflict; prefer overlay/projection
+   or write through the Data API/plugin (overlay before mutation, no silent mutation).
+4. **Encrypted shards are opaque.** Never present ciphertext as content; degrade to
+   backup/structure with provenance (UC-61, graceful degradation).
+
+### 6.3 What Joplin teaches that shard-wiki should keep
+
+- Add **"interchange/sync representation"** as a recognized attach surface in the
+  contract — sometimes preferable to the live store/API (UC-60).
+- Add **content opacity** (encryption-at-rest) as a capability-profile field so encrypted
+  shards degrade correctly (UC-61).
+- Expect **multi-client concurrent editing reconciled by the backend's own sync**
+  (desktop/mobile/CLI) — a different concurrency than single-app file edits (extends
+  UC-53) or multi-user server writes; conflicts may pre-exist as keep-both items.
+
+---
+
+## 7. Use-case seeds → catalog (promoted 2026-06-14)
+
+Last existing UC is **UC-59**. New UCs **UC-60, UC-61** added; existing UCs enriched.
+
+| Seed | Catalog action |
+|------|----------------|
+| **Attach a tool's documented sync/interchange representation on a third-party storage target** (Joplin items on Nextcloud/WebDAV/S3) as a file-store shard, without the app | **UC-60 (new)** |
+| **Attach an encrypted-at-rest shard** (E2EE sync target): content opaque without keys; participate as backup/mirror/structure-shell, never presenting ciphertext as readable | **UC-61 (new)** |
+| Store-minted, rename-stable **note-level** IDs (`:/id`) — the middle of the addressing spectrum | **enriches UC-51** |
+| Native store can be a **DB while the sync representation is files** → attach surface ≠ native store | **enriches UC-40** |
+| Internal revisions, not portable → supplement | **enriches UC-36** |
+| Dual surfaces: in-app plugin host + **local Data API** (localhost, app-running) | **enriches UC-38** (links UC-57) |
+| Resources/attachments = non-Markdown assets with IDs | **enriches UC-55** |
+| Poll-based sync; conflict notes (keep-both) | **enriches UC-31** (links UC-07) |
+
+---
+
+## 8. Architecture notes for SHARD-WP-0002 (no UC)
+
+- **Attachment-mode taxonomy gains a refinement:** within file-store, distinguish
+  *native on-disk store* (Obsidian/TWiki, UC-40) from *interchange/sync representation*
+  (Joplin mirror, UC-60). Both are "files," but format ownership and write-safety differ.
+  (T14.)
+- **Add a "content opacity" capability field** (proposed **twelfth spectrum**:
+  `plaintext → encrypted-at-rest/opaque`) so encrypted shards degrade to
+  backup/structure-shell. Feeds T11 the next time the eleven spectra
+  (`research/260614-shard-spectrum-synthesis/findings.md` §2) are revised. (T11/T14.)
+- **Local-REST as an attach sub-mode** (localhost, app-must-run, token) sits between
+  in-engine-host (UC-38) and external-API (UC-57) — note in T14.
+- **Format-aware file-store adapters:** the contract should let a file-store adapter
+  declare a *format profile* (plain Markdown vs Joplin-item vs Foswiki-PlainFile) so the
+  same "folder of files" attach can parse tool-specific item formats. (T11/T14.)
+
+---
+
+## 9. Open questions (for spec / workplans)
+
+1. For an **encrypted shard** (UC-61), what *is* visible without keys — item IDs/counts/
+   timestamps (structure shell) or nothing? Does shard-wiki ever hold keys, or only ever
+   treat such shards as opaque backups?
+2. Is **writing to a tool's sync mirror** (UC-60) ever safe, or are interchange-format
+   shards **read/projection/overlay-only** by policy (Joplin owns the format)?
+3. Does shard-wiki parse Joplin's **item format** via a dedicated format profile, or
+   only attach the **export** (JEX/Markdown dir) as a cleaner snapshot (UC-37)?
+4. How do we avoid **double-sync** when a shard's storage target is itself driven by a
+   sync daemon (Joplin) we don't control?
+
+---
+
+## 10. Sources
+
+| Source | Used for |
+|--------|----------|
+| Joplin Data API reference (https://joplinapp.org/help/api/references/rest_api/) | Local REST on `localhost:41184`, token auth, endpoints, 32-char IDs, item fields, pagination |
+| Joplin Plugin API docs (https://joplinapp.org/api/references/plugin_api/classes/joplin.html) | `joplin.data` / `joplin.workspace` / `joplin.contentScripts` / views / settings |
+| Joplin — Extending Joplin (https://joplinapp.org/help/api/) | Plugin distribution (npm + repository), Data API vs plugin access |
+| Joplin — WebDAV / Nextcloud sync (https://joplinapp.org/help/apps/sync/nextcloud/) | Sync targets; items as plain-text files on the target |
+| Joplin FAQ + community (https://joplinapp.org/help/faq/) | SQLite local store; Markdown content; E2EE before leaving device |
+| Obsidian vs Joplin comparison (https://petronellatech.com/blog/obsidian-vs-joplin/) | Architecture contrast (SQLite+sync vs file-over-app), E2EE framing |
+
+Cross-references: `research/260614-obsidian-deep-dive/findings.md` (file-over-app
+contrast), `research/260614-notion-deep-dive/findings.md` (external-API, scoped grant),
+`research/260614-shard-spectrum-synthesis/findings.md` (the spectra this extends),
+`spec/UseCaseCatalog.md` (UC-31, UC-36, UC-38, UC-40, UC-51, UC-55, UC-57),
+`workplans/SHARD-WP-0002-federation-architecture.md` (T11, T14).
+
+---
+
+## 11. Traceability
+
+- New UCs: **UC-60, UC-61** → `spec/UseCaseCatalog.md`.
+- Enriched UCs: **UC-31, UC-36, UC-38, UC-40, UC-51, UC-55** (links UC-07, UC-57).
+- Architecture (no UC): interchange/sync-representation attach surface; **content
+  opacity** as a proposed twelfth capability spectrum; local-REST sub-mode; format-aware
+  file-store profiles → `SHARD-WP-0002` (T11, T14).
+- Boundary recorded: Joplin is a **sync daemon over one store**; shard-wiki attaches its
+  **mirror as pages** and does not re-sync — and treats DB-local store and encrypted
+  content as opaque (INTENT not-a-sync-daemon, graceful degradation, no silent mutation).
+</content>

research/README.md

@@ -22,4 +22,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
 | 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 |
 | 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 |
 | 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 |
-| 2026-06-14 | `260614-shard-spectrum-synthesis/` | Synthesis — shard family matrix + eleven capability spectra across nine systems; feeds SHARD-WP-0002 T11–T16 |
+| 2026-06-14 | `260614-shard-spectrum-synthesis/` | Synthesis — shard family matrix + eleven capability spectra across nine systems; feeds SHARD-WP-0002 T11–T16 |
+| 2026-06-14 | `260614-joplin-deep-dive/` | Joplin — SQLite-local/Markdown-on-sync, interchange-format attach, E2EE content opacity; UC-60/61 |