shard-wiki/research/260614-joplin-deep-dive/findings.md

Findings — Joplin: SQLite locally, Markdown-on-sync, and the interchange-format attach

Date: 2026-06-14 Source kind: modern shipped product — an open-source Markdown note app; a candidate shard whose distinctive trait is a documented file-based sync/interchange format on third-party storage plus optional end-to-end encryption Lens: shard-wiki — attachment surfaces, the "attach the sync mirror not the app" pattern, content opacity (encryption), note-level stable IDs, and the file-sync-daemon boundary

Why Joplin earns a dive after Obsidian/Roam/Notion. It looks like "Obsidian but open-source," yet its architecture is a genuinely different point in the space: content is Markdown but the local store is SQLite, and the sync representation is a folder of per-item Markdown+metadata files on a backend you choose (filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin Server/Cloud). That sync mirror — not the app, not the SQLite DB — is the interesting attach surface, and it lands squarely on the WebDAV/Nextcloud backends INTENT already names. Joplin also ships E2EE, forcing the question of a shard whose content is opaque ciphertext, and it is itself a sync layer over heterogeneous storage — the very "file-sync daemon" shard-wiki says it is not, making Joplin a sharp boundary case.

Contrast set: Obsidian (file-over-app: native store is files), Roam (client DB, in-app API only), Notion (hosted DB, external API only), TWiki/Foswiki (file+RCS native store). Joplin is none of these exactly — DB-local, files-on-sync — which is the whole point.

---

1. Core architecture — SQLite local, Markdown content

• Local store: a SQLite database (~/.config/joplin-desktop/database.sqlite), schema documented. So, unlike Obsidian, the native local store is not plain files — you cannot point shard-wiki at a folder of .md and read the live store (you'd read SQLite, fragile while the app runs).
• Content: notes are Markdown (Joplin-flavored: :/imageId resource links, internal :/<id> note links, checkboxes/to-dos). Markdown-first — INTENT-aligned at the content layer even though the storage layer is a DB.
• Model: folders (notebooks, hierarchical) → notes → resources (attachments); cross-cutting tags; notes can be to-dos. Every item has a 32-char hex ID; notes link by ID (:/<id>), so links survive rename/move (store-minted, page-level).
• Clients: desktop (Electron), mobile, and a terminal/CLI app — multiple clients edit one logical store, reconciled by sync (§2).
• Revisions: Joplin keeps internal note history locally — not git, not portable → a supplement case (UC-36), like Notion/Confluence.

---

2. Sync — the distinctive layer (and the attach opportunity)

Joplin's headline feature is sync to a backend of your choice. On sync, the SQLite store is serialized to a folder of plain-text items — one file per note / notebook / tag / resource-metadata, each carrying the Markdown body plus a metadata block (id, parent_id, type_, created_time, updated_time, …) — written to the chosen target: filesystem, WebDAV, Nextcloud, Dropbox, OneDrive, S3, Joplin Server, Joplin Cloud. Deletions propagate via tombstones; concurrent edits across clients produce conflict notes (keep-both, not silent overwrite).

Two consequences for shard-wiki:

1. Attach the sync mirror, not the app (UC-60). A Nextcloud/WebDAV/S3/filesystem directory full of Joplin sync items is a file-store shard — readable without Joplin running and without touching SQLite — provided the adapter understands the documented item format (body + metadata footer, :/id links, resource items). This is a new attach pattern: the backend's interchange/sync representation as the attach surface, distinct from a native on-disk store (UC-40) or app files (UC-53). It also realizes INTENT's WebDAV/Nextcloud participants concretely.
2. Joplin is a file-sync layer over heterogeneous storage — exactly what shard-wiki says it is not (INTENT: "not a file-sync daemon"). Joplin syncs one logical store across backends; shard-wiki federates many logical stores with wiki-page semantics. Attaching a Joplin sync mirror means reading its file representation as pages, never re-driving Joplin's sync — and never becoming a second sync engine over the same target (don't double-sync).

---

3. End-to-end encryption — content opacity

Joplin offers E2EE: items are encrypted before leaving the device, so the sync target holds ciphertext regardless of provider. (Security wart, noted: the master key is stored in clear in the local SQLite — irrelevant to attaching the mirror, but a reason never to treat the local DB as trusted.)

For shard-wiki this introduces a capability dimension the prior dives did not: content opacity. An encrypted Joplin sync target can be attached only as a backup/mirror/structure-shell participant — item IDs, counts, and change events may be visible, but bodies are undecryptable without keys. The adapter must never present ciphertext as readable content and must degrade to "present-but-opaque" with provenance. This extends the synthesis capability spectra with a proposed twelfth field, content opacity (plaintext → encrypted-at-rest/opaque), feeding SHARD-WP-0002 T11 (§9).

---

4. Extension surfaces — a plugin host and a local REST API

Joplin exposes two programmatic surfaces (plus export):

A. Plugin API (in-app host). TypeScript plugins, distributed via npm + a plugin repository, loaded by the app:

• joplin.data — CRUD over the data model, shaped like REST (GET/POST/PUT/DELETE) on notes/folders/tags/resources.
• joplin.workspace — selection + events (note selected, content changed, sync events).
• joplin.contentScripts — markdown-it render plugins and CodeMirror editor extensions (the syntax/rendering extension point).
• joplin.views — panels, dialogs, toolbar/menu items, custom editors; joplin.commands, joplin.settings.

B. Data API (local REST). A REST service the desktop app serves on localhost:41184 (the "clipper server"), token-authenticated, used by the Web Clipper and any local integration: endpoints /notes, /folders, /tags, /resources (incl. /resources/:id/file), /revisions, /search; verbs GET/POST/PUT/DELETE; pagination (page, limit≤100, order_by, order_dir). Notably plugins can use this API even when the clipper server is off.

C. Export — JEX (tar of raw items), Markdown directory, RAW (Joplin export directory), ENEX import (Evernote). Snapshot import/attach surface (UC-28/37).

So Joplin spans attachment modes: in-app plugin host (like Roam/Obsidian-plugin), local-REST (like Notion's external API but localhost + app-must-run — a sub-mode), the sync-mirror file-store (§2, the novel one), and export snapshots.

---

5. Joplin as a shard — capability profile

Capability	Joplin	Notes for the adapter contract
Read	yes	sync-mirror files (no app) · local Data API (app running) · plugin · export
Write	yes	Data API / plugin (joplin.data); writing the sync mirror directly is risky (Joplin owns that format)
Write granularity	per-note (page)	notebooks/notes/resources; not block-level
Identity / addressing	32-char item ID; :/<id> links	store-minted, page-level, survives rename (between Obsidian path and Roam block-UUID) (UC-51)
Structure	notebooks (hierarchy) + tags + to-dos	modest; metadata in the sync item footer (in-text on the mirror)
History	internal revisions, not portable	supplement via coordination journal (UC-36)
Native query	search API (FTS)	delegate text search; no datalog/DB-query (weaker than Roam/Notion)
Subscribe	sync is poll-based; conflict notes	poll; conflicts surface as keep-both items (UC-31, UC-07)
Content opacity	optional E2EE → ciphertext	new dimension: opaque-at-rest; mirror usable only as backup/structure without keys (UC-61)
Transclusion	none (note-level links only)	—
Diff / merge	none native	git-level if mirror is on a git-backed target
Publish	via export / static-site pipelines	outbound (UC-56)
Content types	Markdown + resources (attachments)	non-Markdown assets with IDs (UC-55)
Attach modes	sync-mirror file-store · local-REST · plugin · export	multi-mode, per-binding (UC-60, UC-57, UC-38)

Verdict: a solid Markdown-first candidate shard whose best attach surface is the sync mirror on a file/WebDAV/S3 target (offline, app-independent, INTENT-named), with the local Data API or a plugin for live write-through. Standout: the interchange-format attach (UC-60) and the encryption/opacity case (UC-61). Limits: DB local store, note-level (not block) addressing, internal-only history, weak query.

---

6. Mapping to shard-wiki INTENT (compare, do not equate)

6.1 Reinforcements

• Interchange-format attach validates that a shard's documented sync/export representation — not just its live store or API — is a legitimate, often best, attach surface (offline, app-independent). Generalizes beyond Joplin (UC-60).
• WebDAV/Nextcloud/S3 become concrete participants via Joplin's sync targets — the backends INTENT names, now with a real format to parse.
• Conflict-as-data (Joplin conflict notes = keep-both) is union-without-erasure in the wild (reinforces UC-07, divergence; consensus policy T9).
• Store-minted, rename-stable note IDs (:/id) confirm the addressing spectrum's middle (page-level stable ID) between path (Obsidian) and block-UUID (Roam/Notion) (UC-51).

6.2 Deliberate divergences (design bugs if conflated)

1. Joplin is a sync daemon; shard-wiki is not. Attach the mirror as pages; never re-implement Joplin's sync or run a competing sync over the same target. (INTENT not-a-file-sync-daemon.)
2. DB-local; don't read SQLite live. The native store is a DB; treat it as opaque. Use the sync mirror, the Data API, or a plugin — not the live database.sqlite.
3. Joplin owns the sync format; treat the mirror read-mostly. Writing items into the sync folder behind Joplin's back risks corruption/conflict; prefer overlay/projection or write through the Data API/plugin (overlay before mutation, no silent mutation).
4. Encrypted shards are opaque. Never present ciphertext as content; degrade to backup/structure with provenance (UC-61, graceful degradation).

6.3 What Joplin teaches that shard-wiki should keep

• Add "interchange/sync representation" as a recognized attach surface in the contract — sometimes preferable to the live store/API (UC-60).
• Add content opacity (encryption-at-rest) as a capability-profile field so encrypted shards degrade correctly (UC-61).
• Expect multi-client concurrent editing reconciled by the backend's own sync (desktop/mobile/CLI) — a different concurrency than single-app file edits (extends UC-53) or multi-user server writes; conflicts may pre-exist as keep-both items.

---

7. Use-case seeds → catalog (promoted 2026-06-14)

Last existing UC is UC-59. New UCs UC-60, UC-61 added; existing UCs enriched.

Seed	Catalog action
Attach a tool's documented sync/interchange representation on a third-party storage target (Joplin items on Nextcloud/WebDAV/S3) as a file-store shard, without the app	UC-60 (new)
Attach an encrypted-at-rest shard (E2EE sync target): content opaque without keys; participate as backup/mirror/structure-shell, never presenting ciphertext as readable	UC-61 (new)
Store-minted, rename-stable note-level IDs (:/id) — the middle of the addressing spectrum	enriches UC-51
Native store can be a DB while the sync representation is files → attach surface ≠ native store	enriches UC-40
Internal revisions, not portable → supplement	enriches UC-36
Dual surfaces: in-app plugin host + local Data API (localhost, app-running)	enriches UC-38 (links UC-57)
Resources/attachments = non-Markdown assets with IDs	enriches UC-55
Poll-based sync; conflict notes (keep-both)	enriches UC-31 (links UC-07)

---

8. Architecture notes for SHARD-WP-0002 (no UC)

• Attachment-mode taxonomy gains a refinement: within file-store, distinguish native on-disk store (Obsidian/TWiki, UC-40) from interchange/sync representation (Joplin mirror, UC-60). Both are "files," but format ownership and write-safety differ. (T14.)
• Add a "content opacity" capability field (proposed twelfth spectrum: plaintext → encrypted-at-rest/opaque) so encrypted shards degrade to backup/structure-shell. Feeds T11 the next time the eleven spectra (research/260614-shard-spectrum-synthesis/findings.md §2) are revised. (T11/T14.)
• Local-REST as an attach sub-mode (localhost, app-must-run, token) sits between in-engine-host (UC-38) and external-API (UC-57) — note in T14.
• Format-aware file-store adapters: the contract should let a file-store adapter declare a format profile (plain Markdown vs Joplin-item vs Foswiki-PlainFile) so the same "folder of files" attach can parse tool-specific item formats. (T11/T14.)

---

9. Open questions (for spec / workplans)

1. For an encrypted shard (UC-61), what is visible without keys — item IDs/counts/ timestamps (structure shell) or nothing? Does shard-wiki ever hold keys, or only ever treat such shards as opaque backups?
2. Is writing to a tool's sync mirror (UC-60) ever safe, or are interchange-format shards read/projection/overlay-only by policy (Joplin owns the format)?
3. Does shard-wiki parse Joplin's item format via a dedicated format profile, or only attach the export (JEX/Markdown dir) as a cleaner snapshot (UC-37)?
4. How do we avoid double-sync when a shard's storage target is itself driven by a sync daemon (Joplin) we don't control?

---

10. Sources

Source	Used for
Joplin Data API reference (https://joplinapp.org/help/api/references/rest_api/)	Local REST on localhost:41184, token auth, endpoints, 32-char IDs, item fields, pagination
Joplin Plugin API docs (https://joplinapp.org/api/references/plugin_api/classes/joplin.html)	joplin.data / joplin.workspace / joplin.contentScripts / views / settings
Joplin — Extending Joplin (https://joplinapp.org/help/api/)	Plugin distribution (npm + repository), Data API vs plugin access
Joplin — WebDAV / Nextcloud sync (https://joplinapp.org/help/apps/sync/nextcloud/)	Sync targets; items as plain-text files on the target
Joplin FAQ + community (https://joplinapp.org/help/faq/)	SQLite local store; Markdown content; E2EE before leaving device
Obsidian vs Joplin comparison (https://petronellatech.com/blog/obsidian-vs-joplin/)	Architecture contrast (SQLite+sync vs file-over-app), E2EE framing

Cross-references: research/260614-obsidian-deep-dive/findings.md (file-over-app contrast), research/260614-notion-deep-dive/findings.md (external-API, scoped grant), research/260614-shard-spectrum-synthesis/findings.md (the spectra this extends), spec/UseCaseCatalog.md (UC-31, UC-36, UC-38, UC-40, UC-51, UC-55, UC-57), workplans/SHARD-WP-0002-federation-architecture.md (T11, T14).

---

11. Traceability

• New UCs: UC-60, UC-61 → spec/UseCaseCatalog.md.
• Enriched UCs: UC-31, UC-36, UC-38, UC-40, UC-51, UC-55 (links UC-07, UC-57).
• Architecture (no UC): interchange/sync-representation attach surface; content opacity as a proposed twelfth capability spectrum; local-REST sub-mode; format-aware file-store profiles → SHARD-WP-0002 (T11, T14).
• Boundary recorded: Joplin is a sync daemon over one store; shard-wiki attaches its mirror as pages and does not re-sync — and treats DB-local store and encrypted content as opaque (INTENT not-a-sync-daemon, graceful degradation, no silent mutation).