research: TiddlyWiki deep dive (single-file wiki, write-granularity extreme); UC-78

SHARD-WP-0003 T3. Whole-file write-granularity anchor: an entire wiki (content
+ app engine) in one self-contained HTML file -> save rewrites the whole file,
no per-page atomicity. Node .tid file-per-tiddler substrate is git-diffable/
fine-grained, so the same engine spans the granularity spectrum by substrate
(cf. Logseq file/DB UC-62, backend-swap UC-43). Tiddler = flexible-field
record (UC-34); filter expressions = native-query tier (UC-52). UC-78
(single-file container-format attach). Enriched UC-35/40/34/52/43. Marks T3
done. Feeds SHARD-WP-0002 T11/T14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

SCOPE.md

@@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary.
 |-------|-------|
 | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
 | Intent | `INTENT.md` established; authorization-in-core amendments drafted |
-| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis deep dives & shard-spectrum synthesis (`research/260614-*/`) |
+| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
 | Demand | NetKingdom integration asks captured, not yet negotiated |
-| Spec | Architecture blueprint drafted; UseCaseCatalog 77 UCs from research; PRD/TSD scaffolds |
+| Spec | Architecture blueprint drafted; UseCaseCatalog 78 UCs from research; PRD/TSD scaffolds |
 | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract); `SHARD-WP-0003` active (9 engine dives); `SHARD-WP-0004` active (8 computational-knowledge dives) |
 
 ## In Scope (today)

research/260614-tiddlywiki-deep-dive/README.md

@@ -0,0 +1,16 @@
+# 260614 — TiddlyWiki deep dive
+
+Deep dive on **TiddlyWiki** (TW5): an entire wiki — tiddlers **plus** the app engine — in
+**one self-contained HTML file**, the **whole-file write-granularity** anchor of the
+synthesis matrix, with a Node.js **file-per-tiddler** (`.tid`) substrate as the git-diffable
+alternative, a tiddler/field record model, and **filter expressions** as the native query
+language.
+
+- `findings.md` — the single-file model, tiddler data model, dual substrate, filters,
+  capability profile, INTENT mapping, UC seed (UC-78), architecture notes for SHARD-WP-0002,
+  open questions, sources, traceability.
+
+Catalog yield: UC-78 (attach a single-file self-contained wiki as one shard — parse tiddlers
+out, project; write = rewrite the whole file, the coarsest write-granularity anchor).
+Enriched UC-35/40/34/52/43. Feeds SHARD-WP-0002 T11 (write-granularity extreme) and T14
+(single-file vs file-per-tiddler binding).

research/260614-tiddlywiki-deep-dive/findings.md

@@ -0,0 +1,164 @@
+# TiddlyWiki — deep dive (findings)
+
+**Date:** 2026-06-14 · **Source:** SHARD-WP-0003 T3 · **Subject:** TiddlyWiki / TiddlyWiki5
+(TW5), Jeremy Ruston's self-contained personal wiki.
+
+## Why this dive
+
+The synthesis matrix names **whole-file write granularity** as one extreme of the
+write-granularity spectrum, anchored by TiddlyWiki. This dive confirms the anchor and the
+*portability extreme* it implies — a wiki that is **a single HTML file you can email** — and
+finds the twist: TiddlyWiki also has a **Node.js file-per-tiddler** substrate, so it spans
+the granularity spectrum the way Logseq spans file/DB (UC-62). The question: how does
+shard-wiki attach a backend whose *entire content is one file*?
+
+## 1. The single-file model
+
+Classic TiddlyWiki ships as **one `.html` file** that contains **both**:
+
+1. the **TiddlyWiki core** (the JavaScript engine, parser, renderer, UI), and
+2. **every tiddler** (all content), serialized into the file.
+
+Open it in a browser and the file *is* the running application. There is **no server and no
+build step** — the app reconstitutes itself from the file it was loaded from. This is the
+portability extreme: a complete, self-hosting wiki in a single, emailable, USB-stick-able
+artifact that runs offline anywhere.
+
+**Saving** is the catch: a browser page cannot normally overwrite the file it came from, so
+TiddlyWiki uses **"savers"** — TiddlyFox/browser extension, the File System Access API, a
+Node.js server, TiddlySpot/put-savers, or "download a new copy." Crucially, **a save
+rewrites the *entire* HTML file** (core + all tiddlers re-serialized). Hence **whole-file
+write granularity**: there is no concept of writing one page in isolation in the
+single-file mode — every save touches the whole artifact.
+
+## 2. The tiddler data model
+
+The atomic unit is the **tiddler** — a named record with **fields**:
+
+- Core fields: **`title`** (the identity), **`text`** (the body), **`tags`**, **`created`**,
+  **`modified`**, **`type`** (content type, e.g. `text/vnd.tiddlywiki`, `text/markdown`),
+  plus **arbitrary custom fields** (any key→value). A tiddler is effectively a **flexible
+  flat record** — closer to a typed-field record (UC-34) than to prose-with-frontmatter.
+- **Everything is a tiddler**: not just pages, but tags, macros, templates, themes, plugins,
+  and the wiki's own configuration are all tiddlers. A **plugin is a bundle of tiddlers**.
+- Content markup is **WikiText** (TW5's own), though `type` can mark a tiddler as Markdown,
+  JSON, image, etc. **Transclusion** is native: `{{SomeTiddler}}` embeds another tiddler;
+  `{{SomeTiddler!!field}}` embeds a field.
+
+## 3. The dual substrate — single-file vs file-per-tiddler
+
+TiddlyWiki on **Node.js** stores each tiddler as a **separate `.tid` file** on disk: a small
+text file with a header of `field: value` lines, a blank line, then the body. The Node
+server assembles these into the same wiki at serve time. This substrate is:
+
+- **git-diffable and fine-grained** — one file per tiddler, line-level diffs, per-tiddler
+  history — the *opposite* end of the granularity spectrum from the single HTML file.
+- the natural attach surface for a *versioned, multi-author* TiddlyWiki.
+
+So TiddlyWiki **spans the write-granularity spectrum by substrate** (single-file = whole-file
+write; Node = file-per-tiddler write), exactly as Logseq spans file/DB (UC-62) and as the
+backend-swap question (UC-43) anticipates.
+
+## 4. Native query — filter expressions
+
+TiddlyWiki's query language is **filter expressions** over tiddler fields, e.g.
+`[tag[todo]!tag[done]sort[modified]]` — a compact DSL that selects/orders tiddlers by field
+and tag. Lists, tables, and dynamic views are built from filters. This is a **native-query
+capability** (UC-52 tier) — less expressive than SPARQL/datalog but real, and computed over
+the tiddler store.
+
+## 5. Capability profile
+
+| Dimension (synthesis spectrum) | TiddlyWiki (single-file) | TiddlyWiki (Node `.tid`) |
+|--------------------------------|--------------------------|--------------------------|
+| Attachment mode | **file-store: one HTML file** | **file-store: dir of `.tid` files** |
+| Addressing granularity | tiddler (`title`) within the file | tiddler = one file |
+| Content identity | **`title`** (placement-bound) | title ↔ filename |
+| Structure | flat record store w/ arbitrary **fields** + tags | same |
+| History | none in-file (whole-file save) | **per-file git history** |
+| Merge model | whole-file replace (no merge) | git 3-way per tiddler |
+| Native query | **filter expressions** | filter expressions |
+| Translation | WikiText (or per-tiddler `type`: markdown/json/…) | same |
+| **Write granularity** | **whole file** (the anchor) | **file per tiddler** |
+| Operational envelope | trivial — a browser; no server | a Node server |
+| Access grant | file access = full access | server/file perms |
+| Content opacity | transparent (parse the HTML store) | transparent text |
+| Provenance | created/modified fields | git + fields |
+
+## 6. INTENT mapping
+
+### Reinforcements
+
+- **Graceful degradation**: a single-file TiddlyWiki is a *trivial* read-only / projection /
+  backup shard — parse the tiddlers out of the HTML, project pages; no server needed. The
+  limited-backend-still-usable principle at its simplest.
+- **Markdown-first but backend-neutral**: tiddlers carry a `type`, so Markdown tiddlers
+  coexist with WikiText — the page model's content-type field maps directly.
+- **Typed fields** (UC-34): arbitrary tiddler fields are a flexible record model the page
+  model already accommodates.
+- **Backend-swap under stable identity** (UC-43): single-file ↔ Node `.tid` is the same
+  logical wiki on two substrates — the migration UC-43 anticipates, within one engine.
+
+### Divergences (boundaries / notes)
+
+- **Whole-file write granularity** is a real constraint: in single-file mode shard-wiki
+  cannot write one page atomically — an overlay applied "to one page" still **rewrites the
+  whole file** (T11). This is the coarsest write tier; model it explicitly so overlays/locks
+  account for it (a write to any page conflicts with any concurrent write).
+- **Identity = title**, file-local; cross-shard identity (T16) layered above.
+- **The app is in the file**: when parsing a single-file TiddlyWiki, shard-wiki must extract
+  the **tiddler store** and ignore the embedded engine — i.e. treat the HTML as a *container
+  format*, not as page content (don't mistake the app for content).
+
+### What to keep
+
+1. **Single-file self-contained wiki as a first-class file-store shard** — container-format
+   parse, whole-file write granularity (UC-78); the portability/granularity extreme.
+2. **Whole-file write granularity as a named tier** (T11) with overlay/lock implications.
+3. **Dual-substrate binding** (single-file vs `.tid` dir) as another instance of
+   substrate-choice under one identity (UC-43/UC-62).
+
+## 7. UC seed
+
+| # | Seed | Disposition |
+|---|------|-------------|
+| UC-78 | Attach a **single-file self-contained wiki** (TiddlyWiki HTML) as one shard — parse tiddlers out of the container, project pages; **write = rewrite the whole file** (whole-file write granularity, the coarsest tier) | **new** |
+| — | whole-file write granularity anchor + overlay/lock implications | enrich **UC-35** |
+| — | single HTML file as a file-store shard (container format) | enrich **UC-40** |
+| — | tiddler arbitrary fields = flexible record | enrich **UC-34** |
+| — | filter expressions as a native-query tier | enrich **UC-52** |
+| — | single-file ↔ Node `.tid` substrate swap | enrich **UC-43** |
+
+## 8. Architecture notes for SHARD-WP-0002
+
+- **T11 (capability / write granularity):** confirm **whole-file** as the coarsest named
+  write tier (anchored by single-file TiddlyWiki), with the implication that an overlay to
+  *any* page conflicts with concurrent writes (no per-page atomicity). File-per-tiddler is
+  the fine tier on the same engine.
+- **T14 (attach binding):** a single-file wiki binds as a **container-format file-store**
+  (parse tiddler store, ignore embedded engine); a Node TiddlyWiki binds as a **dir of
+  `.tid` files** (git-diffable). One engine, two bindings — parameterize like UC-43.
+- **Native query:** filter expressions are a low-mid native-query tier between "none" and
+  datalog/SPARQL — delegate where present (UC-52).
+
+## 9. Open questions
+
+1. In single-file mode, how does shard-wiki represent **per-page overlays** when writes are
+   whole-file — buffer overlays and re-serialize, or require the Node `.tid` substrate for
+   write-through and treat single-file as read/projection/backup only?
+2. Is a single-file TiddlyWiki's **embedded plugins/config** ever relevant to the union, or
+   strictly ignored as app-internals (parse only content tiddlers)?
+3. Does shard-wiki expose tiddler **filter expressions** as a delegated query, or only its
+   own union query over projected tiddlers?
+
+## 10. Sources
+
+- TiddlyWiki.com — *Tiddlers*, *TiddlerFields*, *Filters*, *Saving*, *Node.js* docs
+- *TiddlyWiki5* GitHub (Jermolene/TiddlyWiki5) — `.tid` file format, store structure
+- prior: `research/260614-logseq-deep-dive/` (file/DB dual substrate, UC-62)
+
+## 11. Traceability
+
+New UC **UC-78** carries the marker **⊡** in the wikiengines column of
+`spec/UseCaseCatalog.md`. Enriched: UC-35, UC-40, UC-34, UC-52, UC-43. Architecture
+cross-refs: SHARD-WP-0002 T11 (whole-file tier), T14 (dual binding).

research/README.md

@@ -30,4 +30,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
 | 2026-06-14 | `260614-wikijs-deep-dive/` | Wiki.js — storage-module engine (DB↔Git Markdown), GraphQL API, pluggable modules ≈ adapter-contract prior art; UC-68/69 |
 | 2026-06-14 | `260614-federated-wiki-deep-dive/` | Federated Wiki — fork-with-provenance, per-page semantic-action journal (story=replay), neighborhood/roster + chorus; prior art for our coordination journal / overlay / union pillars; UC-70/71/72 |
 | 2026-06-14 | `260614-wikibase-deep-dive/` | Wikibase/Wikidata — typed entity-statement knowledge graph (claim+qualifiers+refs+rank), SPARQL/RDF + federated SERVICE, opaque stable IDs, statement-level provenance; structure & query far-end; UC-73/74/75 |
-| 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 |
+| 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 |
+| 2026-06-14 | `260614-tiddlywiki-deep-dive/` | TiddlyWiki — entire wiki (content + app) in one self-contained HTML file = whole-file write-granularity extreme; Node `.tid` file-per-tiddler substrate (git-diffable); tiddler/field records, filter-expression query; UC-78 |

spec/UseCaseCatalog.md

@@ -14,7 +14,8 @@ Promoted from `research/260608-c2-wiki-origins/`,
 `research/260614-trilium-deep-dive/`, `research/260614-wikijs-deep-dive/`, and
 `research/260614-federated-wiki-deep-dive/`, and
 `research/260614-wikibase-deep-dive/`, and
-`research/260614-forge-wikis-deep-dive/`.
+`research/260614-forge-wikis-deep-dive/`, and
+`research/260614-tiddlywiki-deep-dive/`.
 See InfoTechPrimers on coulomb.social for use-case catalog conventions.
 
 ## Conventions
@@ -272,7 +273,12 @@ granularity** (per-page / per-file / per-space / append-only). Overlay and
 patch flows must adapt (findings §5 #1). Affects conflict scope and locking. Roam marks
 the **fine extreme** — block-level writes (`block.create/update/move/delete`), the
 opposite of TiddlyWiki's whole-file granularity
-(`research/260614-roam-deep-dive/findings.md` §6).  
+(`research/260614-roam-deep-dive/findings.md` §6). The **TiddlyWiki** dive confirms the
+coarse anchor and its consequence: in single-file mode a save **rewrites the whole HTML
+file**, so an overlay to *any* page **conflicts with any concurrent write** (no per-page
+atomicity) — whereas its Node `.tid` substrate is **file-per-tiddler** (fine), so the *same
+engine* sits at both ends by substrate (UC-78,
+`research/260614-tiddlywiki-deep-dive/findings.md` §1, §3; cf. UC-43/UC-62).  
 **Priority:** Later
 
 ### UC-36 — Supply a git-addressable history to an internal-history engine
@@ -1051,6 +1057,24 @@ external-API host sub-mode (UC-38) beside the file-store attach (UC-76). Feeds
 SHARD-WP-0002 T11 (capability flag), T14 (binding).  
 **Priority:** Later
 
+### UC-78 — Attach a single-file self-contained wiki (whole-file write granularity)
+
+**Actor:** Orchestrator / adapter  
+**Goal:** Attach a **single-file self-contained wiki** (a TiddlyWiki HTML file that bundles
+both content and the app engine) as one shard — **parse the tiddler store out of the
+container**, project pages, and treat **writing as rewriting the whole file**.  
+**Source:** wikiengines, intent  
+**Notes:** Classic TiddlyWiki is *one HTML file* = engine + every tiddler
+(`research/260614-tiddlywiki-deep-dive/findings.md` §1) — the **portability and
+write-granularity extreme**. Write granularity is **whole-file** (the coarsest tier): an
+overlay to any one page still rewrites the entire artifact, so per-page atomicity does not
+exist (T11) — model overlays/locks accordingly, or require the Node.js **`.tid`
+file-per-tiddler** substrate for fine-grained write-through and keep single-file as
+read/projection/backup. Treat the HTML as a **container format** (extract content tiddlers,
+ignore the embedded engine). Feeds SHARD-WP-0002 T11 (whole-file tier), T14 (single-file vs
+`.tid` binding; cf. UC-43/UC-62).  
+**Priority:** Later
+
 ---
 
 ## B. Knowledge work and collaboration
@@ -1325,6 +1349,7 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD.
 | UC-75 | | | | ⬡ | ✓ |
 | UC-76 | | | | ⎇ | ✓ |
 | UC-77 | | | | ⎇ | ✓ |
+| UC-78 | | | | ⊡ | ✓ |
 | UC-08 | ✓ | | |
 | UC-09 | ✓ | | |
 | UC-10 | ✓ | | |
@@ -1827,6 +1852,35 @@ overlay→review→apply flow is shard-wiki's to provide. Architecture logged fo
 `SHARD-WP-0002` (T14/T11): `.wiki.git` clone as the canonical file-store attach, wiki API as
 an optional per-forge capability, git log adopted directly as the journal.
 
+### tiddlywiki mapping
+
+(⊡ UC-78 is placed in the **wikiengines** matrix column; lineage = the **TiddlyWiki deep
+dive**, `research/260614-tiddlywiki-deep-dive/findings.md`.)
+
+| TiddlyWiki mechanism (findings §) | Catalog UC |
+|-----------------------------------|------------|
+| One HTML file = engine + all tiddlers; save rewrites whole file (§1) | UC-78 (new) |
+| Whole-file write granularity = coarsest tier; no per-page atomicity (§1, §6) | UC-35 (enriched) |
+| Single HTML file as a container-format file-store shard (§1) | UC-40 (enriched) |
+| Tiddler = flat record with arbitrary fields + tags (§2) | UC-34 (enriched) |
+| Filter expressions `[tag[x]sort[...]]` = native-query tier (§4) | UC-52 (enriched) |
+| Single-file ↔ Node `.tid` file-per-tiddler substrate swap (§3) | UC-43 (enriched); links UC-62 |
+| Transclusion `{{tiddler}}` / `{{tiddler!!field}}` (§2) | links UC-32 |
+| Identity = `title` (file-local) (§2, §5) | links UC-25 |
+
+Note: TiddlyWiki is the **write-granularity and portability extreme** — a complete wiki
+(content **plus** the app engine) in **one self-contained HTML file**, where every save
+**rewrites the whole file** (whole-file write granularity, the coarsest tier: no per-page
+atomicity — model overlays/locks accordingly). Its Node.js substrate stores **one `.tid`
+file per tiddler**, git-diffable and fine-grained — so TiddlyWiki **spans the granularity
+spectrum by substrate** exactly as Logseq spans file/DB (UC-62) and UC-43 anticipates. The
+tiddler is a **flexible record with arbitrary fields** (UC-34), and **filter expressions**
+are a real native-query tier (UC-52). **Boundary recorded:** treat the single HTML file as a
+**container format** — extract content tiddlers, ignore the embedded engine; prefer the
+`.tid` substrate for fine-grained write-through, single-file as read/projection/backup.
+Architecture logged for `SHARD-WP-0002` (T11/T14): whole-file as the coarsest write tier,
+single-file-container vs `.tid`-dir dual binding.
+
 ---
 
 ## Open questions
@@ -1894,5 +1948,8 @@ an optional per-forge capability, git log adopted directly as the journal.
     git+Markdown adapter with the **forge wiki repo** shard (parameterized by repo/path), or
     stay distinct? Since forge wikis lack wiki-MRs, does shard-wiki supply the
     overlay→review→apply layer? (Forge-wikis dive §8.)
+26. For a **single-file wiki** (UC-78), does shard-wiki support per-page overlays by
+    buffering and re-serializing the whole file, or require the Node `.tid` substrate for
+    write-through and treat single-file as read/projection/backup only? (TiddlyWiki dive §9.)
 23. How does shard-wiki **honor/surface a shard's path-based access rules** (UC-06) in a
     projection without re-implementing its ACL engine? (Wiki.js dive §9.)

workplans/SHARD-WP-0003-engine-dives-batch.md

@@ -84,7 +84,7 @@ feed `SHARD-WP-0002` T12/T16.
 
 ```task
 id: SHARD-WP-0003-T3
-status: todo
+status: done
 priority: medium
 state_hub_task_id: "a0ab1aba-9aa2-4214-b130-67ba2805e64b"
 ```