From fee47514a2360ddd19d59ecc0d6291a93ebceb6d Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 14 Jun 2026 19:46:37 +0200 Subject: [PATCH] research: TiddlyWiki deep dive (single-file wiki, write-granularity extreme); UC-78 SHARD-WP-0003 T3. Whole-file write-granularity anchor: an entire wiki (content + app engine) in one self-contained HTML file -> save rewrites the whole file, no per-page atomicity. Node .tid file-per-tiddler substrate is git-diffable/ fine-grained, so the same engine spans the granularity spectrum by substrate (cf. Logseq file/DB UC-62, backend-swap UC-43). Tiddler = flexible-field record (UC-34); filter expressions = native-query tier (UC-52). UC-78 (single-file container-format attach). Enriched UC-35/40/34/52/43. Marks T3 done. Feeds SHARD-WP-0002 T11/T14. Co-Authored-By: Claude Opus 4.8 --- SCOPE.md | 4 +- .../260614-tiddlywiki-deep-dive/README.md | 16 ++ .../260614-tiddlywiki-deep-dive/findings.md | 164 ++++++++++++++++++ research/README.md | 3 +- spec/UseCaseCatalog.md | 61 ++++++- workplans/SHARD-WP-0003-engine-dives-batch.md | 2 +- 6 files changed, 244 insertions(+), 6 deletions(-) create mode 100644 research/260614-tiddlywiki-deep-dive/README.md create mode 100644 research/260614-tiddlywiki-deep-dive/findings.md diff --git a/SCOPE.md b/SCOPE.md index be3b7a0..773ac7e 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary. |-------|-------| | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | -| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis deep dives & shard-spectrum synthesis (`research/260614-*/`) | +| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | Architecture blueprint drafted; UseCaseCatalog 77 UCs from research; PRD/TSD scaffolds | +| Spec | Architecture blueprint drafted; UseCaseCatalog 78 UCs from research; PRD/TSD scaffolds | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract); `SHARD-WP-0003` active (9 engine dives); `SHARD-WP-0004` active (8 computational-knowledge dives) | ## In Scope (today) diff --git a/research/260614-tiddlywiki-deep-dive/README.md b/research/260614-tiddlywiki-deep-dive/README.md new file mode 100644 index 0000000..48d929d --- /dev/null +++ b/research/260614-tiddlywiki-deep-dive/README.md @@ -0,0 +1,16 @@ +# 260614 — TiddlyWiki deep dive + +Deep dive on **TiddlyWiki** (TW5): an entire wiki — tiddlers **plus** the app engine — in +**one self-contained HTML file**, the **whole-file write-granularity** anchor of the +synthesis matrix, with a Node.js **file-per-tiddler** (`.tid`) substrate as the git-diffable +alternative, a tiddler/field record model, and **filter expressions** as the native query +language. + +- `findings.md` — the single-file model, tiddler data model, dual substrate, filters, + capability profile, INTENT mapping, UC seed (UC-78), architecture notes for SHARD-WP-0002, + open questions, sources, traceability. + +Catalog yield: UC-78 (attach a single-file self-contained wiki as one shard — parse tiddlers +out, project; write = rewrite the whole file, the coarsest write-granularity anchor). +Enriched UC-35/40/34/52/43. Feeds SHARD-WP-0002 T11 (write-granularity extreme) and T14 +(single-file vs file-per-tiddler binding). diff --git a/research/260614-tiddlywiki-deep-dive/findings.md b/research/260614-tiddlywiki-deep-dive/findings.md new file mode 100644 index 0000000..70d17c9 --- /dev/null +++ b/research/260614-tiddlywiki-deep-dive/findings.md @@ -0,0 +1,164 @@ +# TiddlyWiki — deep dive (findings) + +**Date:** 2026-06-14 · **Source:** SHARD-WP-0003 T3 · **Subject:** TiddlyWiki / TiddlyWiki5 +(TW5), Jeremy Ruston's self-contained personal wiki. + +## Why this dive + +The synthesis matrix names **whole-file write granularity** as one extreme of the +write-granularity spectrum, anchored by TiddlyWiki. This dive confirms the anchor and the +*portability extreme* it implies — a wiki that is **a single HTML file you can email** — and +finds the twist: TiddlyWiki also has a **Node.js file-per-tiddler** substrate, so it spans +the granularity spectrum the way Logseq spans file/DB (UC-62). The question: how does +shard-wiki attach a backend whose *entire content is one file*? + +## 1. The single-file model + +Classic TiddlyWiki ships as **one `.html` file** that contains **both**: + +1. the **TiddlyWiki core** (the JavaScript engine, parser, renderer, UI), and +2. **every tiddler** (all content), serialized into the file. + +Open it in a browser and the file *is* the running application. There is **no server and no +build step** — the app reconstitutes itself from the file it was loaded from. This is the +portability extreme: a complete, self-hosting wiki in a single, emailable, USB-stick-able +artifact that runs offline anywhere. + +**Saving** is the catch: a browser page cannot normally overwrite the file it came from, so +TiddlyWiki uses **"savers"** — TiddlyFox/browser extension, the File System Access API, a +Node.js server, TiddlySpot/put-savers, or "download a new copy." Crucially, **a save +rewrites the *entire* HTML file** (core + all tiddlers re-serialized). Hence **whole-file +write granularity**: there is no concept of writing one page in isolation in the +single-file mode — every save touches the whole artifact. + +## 2. The tiddler data model + +The atomic unit is the **tiddler** — a named record with **fields**: + +- Core fields: **`title`** (the identity), **`text`** (the body), **`tags`**, **`created`**, + **`modified`**, **`type`** (content type, e.g. `text/vnd.tiddlywiki`, `text/markdown`), + plus **arbitrary custom fields** (any key→value). A tiddler is effectively a **flexible + flat record** — closer to a typed-field record (UC-34) than to prose-with-frontmatter. +- **Everything is a tiddler**: not just pages, but tags, macros, templates, themes, plugins, + and the wiki's own configuration are all tiddlers. A **plugin is a bundle of tiddlers**. +- Content markup is **WikiText** (TW5's own), though `type` can mark a tiddler as Markdown, + JSON, image, etc. **Transclusion** is native: `{{SomeTiddler}}` embeds another tiddler; + `{{SomeTiddler!!field}}` embeds a field. + +## 3. The dual substrate — single-file vs file-per-tiddler + +TiddlyWiki on **Node.js** stores each tiddler as a **separate `.tid` file** on disk: a small +text file with a header of `field: value` lines, a blank line, then the body. The Node +server assembles these into the same wiki at serve time. This substrate is: + +- **git-diffable and fine-grained** — one file per tiddler, line-level diffs, per-tiddler + history — the *opposite* end of the granularity spectrum from the single HTML file. +- the natural attach surface for a *versioned, multi-author* TiddlyWiki. + +So TiddlyWiki **spans the write-granularity spectrum by substrate** (single-file = whole-file +write; Node = file-per-tiddler write), exactly as Logseq spans file/DB (UC-62) and as the +backend-swap question (UC-43) anticipates. + +## 4. Native query — filter expressions + +TiddlyWiki's query language is **filter expressions** over tiddler fields, e.g. +`[tag[todo]!tag[done]sort[modified]]` — a compact DSL that selects/orders tiddlers by field +and tag. Lists, tables, and dynamic views are built from filters. This is a **native-query +capability** (UC-52 tier) — less expressive than SPARQL/datalog but real, and computed over +the tiddler store. + +## 5. Capability profile + +| Dimension (synthesis spectrum) | TiddlyWiki (single-file) | TiddlyWiki (Node `.tid`) | +|--------------------------------|--------------------------|--------------------------| +| Attachment mode | **file-store: one HTML file** | **file-store: dir of `.tid` files** | +| Addressing granularity | tiddler (`title`) within the file | tiddler = one file | +| Content identity | **`title`** (placement-bound) | title ↔ filename | +| Structure | flat record store w/ arbitrary **fields** + tags | same | +| History | none in-file (whole-file save) | **per-file git history** | +| Merge model | whole-file replace (no merge) | git 3-way per tiddler | +| Native query | **filter expressions** | filter expressions | +| Translation | WikiText (or per-tiddler `type`: markdown/json/…) | same | +| **Write granularity** | **whole file** (the anchor) | **file per tiddler** | +| Operational envelope | trivial — a browser; no server | a Node server | +| Access grant | file access = full access | server/file perms | +| Content opacity | transparent (parse the HTML store) | transparent text | +| Provenance | created/modified fields | git + fields | + +## 6. INTENT mapping + +### Reinforcements + +- **Graceful degradation**: a single-file TiddlyWiki is a *trivial* read-only / projection / + backup shard — parse the tiddlers out of the HTML, project pages; no server needed. The + limited-backend-still-usable principle at its simplest. +- **Markdown-first but backend-neutral**: tiddlers carry a `type`, so Markdown tiddlers + coexist with WikiText — the page model's content-type field maps directly. +- **Typed fields** (UC-34): arbitrary tiddler fields are a flexible record model the page + model already accommodates. +- **Backend-swap under stable identity** (UC-43): single-file ↔ Node `.tid` is the same + logical wiki on two substrates — the migration UC-43 anticipates, within one engine. + +### Divergences (boundaries / notes) + +- **Whole-file write granularity** is a real constraint: in single-file mode shard-wiki + cannot write one page atomically — an overlay applied "to one page" still **rewrites the + whole file** (T11). This is the coarsest write tier; model it explicitly so overlays/locks + account for it (a write to any page conflicts with any concurrent write). +- **Identity = title**, file-local; cross-shard identity (T16) layered above. +- **The app is in the file**: when parsing a single-file TiddlyWiki, shard-wiki must extract + the **tiddler store** and ignore the embedded engine — i.e. treat the HTML as a *container + format*, not as page content (don't mistake the app for content). + +### What to keep + +1. **Single-file self-contained wiki as a first-class file-store shard** — container-format + parse, whole-file write granularity (UC-78); the portability/granularity extreme. +2. **Whole-file write granularity as a named tier** (T11) with overlay/lock implications. +3. **Dual-substrate binding** (single-file vs `.tid` dir) as another instance of + substrate-choice under one identity (UC-43/UC-62). + +## 7. UC seed + +| # | Seed | Disposition | +|---|------|-------------| +| UC-78 | Attach a **single-file self-contained wiki** (TiddlyWiki HTML) as one shard — parse tiddlers out of the container, project pages; **write = rewrite the whole file** (whole-file write granularity, the coarsest tier) | **new** | +| — | whole-file write granularity anchor + overlay/lock implications | enrich **UC-35** | +| — | single HTML file as a file-store shard (container format) | enrich **UC-40** | +| — | tiddler arbitrary fields = flexible record | enrich **UC-34** | +| — | filter expressions as a native-query tier | enrich **UC-52** | +| — | single-file ↔ Node `.tid` substrate swap | enrich **UC-43** | + +## 8. Architecture notes for SHARD-WP-0002 + +- **T11 (capability / write granularity):** confirm **whole-file** as the coarsest named + write tier (anchored by single-file TiddlyWiki), with the implication that an overlay to + *any* page conflicts with concurrent writes (no per-page atomicity). File-per-tiddler is + the fine tier on the same engine. +- **T14 (attach binding):** a single-file wiki binds as a **container-format file-store** + (parse tiddler store, ignore embedded engine); a Node TiddlyWiki binds as a **dir of + `.tid` files** (git-diffable). One engine, two bindings — parameterize like UC-43. +- **Native query:** filter expressions are a low-mid native-query tier between "none" and + datalog/SPARQL — delegate where present (UC-52). + +## 9. Open questions + +1. In single-file mode, how does shard-wiki represent **per-page overlays** when writes are + whole-file — buffer overlays and re-serialize, or require the Node `.tid` substrate for + write-through and treat single-file as read/projection/backup only? +2. Is a single-file TiddlyWiki's **embedded plugins/config** ever relevant to the union, or + strictly ignored as app-internals (parse only content tiddlers)? +3. Does shard-wiki expose tiddler **filter expressions** as a delegated query, or only its + own union query over projected tiddlers? + +## 10. Sources + +- TiddlyWiki.com — *Tiddlers*, *TiddlerFields*, *Filters*, *Saving*, *Node.js* docs +- *TiddlyWiki5* GitHub (Jermolene/TiddlyWiki5) — `.tid` file format, store structure +- prior: `research/260614-logseq-deep-dive/` (file/DB dual substrate, UC-62) + +## 11. Traceability + +New UC **UC-78** carries the marker **⊡** in the wikiengines column of +`spec/UseCaseCatalog.md`. Enriched: UC-35, UC-40, UC-34, UC-52, UC-43. Architecture +cross-refs: SHARD-WP-0002 T11 (whole-file tier), T14 (dual binding). diff --git a/research/README.md b/research/README.md index c88fe4c..b613b85 100644 --- a/research/README.md +++ b/research/README.md @@ -30,4 +30,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and | 2026-06-14 | `260614-wikijs-deep-dive/` | Wiki.js — storage-module engine (DB↔Git Markdown), GraphQL API, pluggable modules ≈ adapter-contract prior art; UC-68/69 | | 2026-06-14 | `260614-federated-wiki-deep-dive/` | Federated Wiki — fork-with-provenance, per-page semantic-action journal (story=replay), neighborhood/roster + chorus; prior art for our coordination journal / overlay / union pillars; UC-70/71/72 | | 2026-06-14 | `260614-wikibase-deep-dive/` | Wikibase/Wikidata — typed entity-statement knowledge graph (claim+qualifiers+refs+rank), SPARQL/RDF + federated SERVICE, opaque stable IDs, statement-level provenance; structure & query far-end; UC-73/74/75 | -| 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 | \ No newline at end of file +| 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 | +| 2026-06-14 | `260614-tiddlywiki-deep-dive/` | TiddlyWiki — entire wiki (content + app) in one self-contained HTML file = whole-file write-granularity extreme; Node `.tid` file-per-tiddler substrate (git-diffable); tiddler/field records, filter-expression query; UC-78 | \ No newline at end of file diff --git a/spec/UseCaseCatalog.md b/spec/UseCaseCatalog.md index f96b483..69faac6 100644 --- a/spec/UseCaseCatalog.md +++ b/spec/UseCaseCatalog.md @@ -14,7 +14,8 @@ Promoted from `research/260608-c2-wiki-origins/`, `research/260614-trilium-deep-dive/`, `research/260614-wikijs-deep-dive/`, and `research/260614-federated-wiki-deep-dive/`, and `research/260614-wikibase-deep-dive/`, and -`research/260614-forge-wikis-deep-dive/`. +`research/260614-forge-wikis-deep-dive/`, and +`research/260614-tiddlywiki-deep-dive/`. See InfoTechPrimers on coulomb.social for use-case catalog conventions. ## Conventions @@ -272,7 +273,12 @@ granularity** (per-page / per-file / per-space / append-only). Overlay and patch flows must adapt (findings §5 #1). Affects conflict scope and locking. Roam marks the **fine extreme** — block-level writes (`block.create/update/move/delete`), the opposite of TiddlyWiki's whole-file granularity -(`research/260614-roam-deep-dive/findings.md` §6). +(`research/260614-roam-deep-dive/findings.md` §6). The **TiddlyWiki** dive confirms the +coarse anchor and its consequence: in single-file mode a save **rewrites the whole HTML +file**, so an overlay to *any* page **conflicts with any concurrent write** (no per-page +atomicity) — whereas its Node `.tid` substrate is **file-per-tiddler** (fine), so the *same +engine* sits at both ends by substrate (UC-78, +`research/260614-tiddlywiki-deep-dive/findings.md` §1, §3; cf. UC-43/UC-62). **Priority:** Later ### UC-36 — Supply a git-addressable history to an internal-history engine @@ -1051,6 +1057,24 @@ external-API host sub-mode (UC-38) beside the file-store attach (UC-76). Feeds SHARD-WP-0002 T11 (capability flag), T14 (binding). **Priority:** Later +### UC-78 — Attach a single-file self-contained wiki (whole-file write granularity) + +**Actor:** Orchestrator / adapter +**Goal:** Attach a **single-file self-contained wiki** (a TiddlyWiki HTML file that bundles +both content and the app engine) as one shard — **parse the tiddler store out of the +container**, project pages, and treat **writing as rewriting the whole file**. +**Source:** wikiengines, intent +**Notes:** Classic TiddlyWiki is *one HTML file* = engine + every tiddler +(`research/260614-tiddlywiki-deep-dive/findings.md` §1) — the **portability and +write-granularity extreme**. Write granularity is **whole-file** (the coarsest tier): an +overlay to any one page still rewrites the entire artifact, so per-page atomicity does not +exist (T11) — model overlays/locks accordingly, or require the Node.js **`.tid` +file-per-tiddler** substrate for fine-grained write-through and keep single-file as +read/projection/backup. Treat the HTML as a **container format** (extract content tiddlers, +ignore the embedded engine). Feeds SHARD-WP-0002 T11 (whole-file tier), T14 (single-file vs +`.tid` binding; cf. UC-43/UC-62). +**Priority:** Later + --- ## B. Knowledge work and collaboration @@ -1325,6 +1349,7 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD. | UC-75 | | | | ⬡ | ✓ | | UC-76 | | | | ⎇ | ✓ | | UC-77 | | | | ⎇ | ✓ | +| UC-78 | | | | ⊡ | ✓ | | UC-08 | ✓ | | | | UC-09 | ✓ | | | | UC-10 | ✓ | | | @@ -1827,6 +1852,35 @@ overlay→review→apply flow is shard-wiki's to provide. Architecture logged fo `SHARD-WP-0002` (T14/T11): `.wiki.git` clone as the canonical file-store attach, wiki API as an optional per-forge capability, git log adopted directly as the journal. +### tiddlywiki mapping + +(⊡ UC-78 is placed in the **wikiengines** matrix column; lineage = the **TiddlyWiki deep +dive**, `research/260614-tiddlywiki-deep-dive/findings.md`.) + +| TiddlyWiki mechanism (findings §) | Catalog UC | +|-----------------------------------|------------| +| One HTML file = engine + all tiddlers; save rewrites whole file (§1) | UC-78 (new) | +| Whole-file write granularity = coarsest tier; no per-page atomicity (§1, §6) | UC-35 (enriched) | +| Single HTML file as a container-format file-store shard (§1) | UC-40 (enriched) | +| Tiddler = flat record with arbitrary fields + tags (§2) | UC-34 (enriched) | +| Filter expressions `[tag[x]sort[...]]` = native-query tier (§4) | UC-52 (enriched) | +| Single-file ↔ Node `.tid` file-per-tiddler substrate swap (§3) | UC-43 (enriched); links UC-62 | +| Transclusion `{{tiddler}}` / `{{tiddler!!field}}` (§2) | links UC-32 | +| Identity = `title` (file-local) (§2, §5) | links UC-25 | + +Note: TiddlyWiki is the **write-granularity and portability extreme** — a complete wiki +(content **plus** the app engine) in **one self-contained HTML file**, where every save +**rewrites the whole file** (whole-file write granularity, the coarsest tier: no per-page +atomicity — model overlays/locks accordingly). Its Node.js substrate stores **one `.tid` +file per tiddler**, git-diffable and fine-grained — so TiddlyWiki **spans the granularity +spectrum by substrate** exactly as Logseq spans file/DB (UC-62) and UC-43 anticipates. The +tiddler is a **flexible record with arbitrary fields** (UC-34), and **filter expressions** +are a real native-query tier (UC-52). **Boundary recorded:** treat the single HTML file as a +**container format** — extract content tiddlers, ignore the embedded engine; prefer the +`.tid` substrate for fine-grained write-through, single-file as read/projection/backup. +Architecture logged for `SHARD-WP-0002` (T11/T14): whole-file as the coarsest write tier, +single-file-container vs `.tid`-dir dual binding. + --- ## Open questions @@ -1894,5 +1948,8 @@ an optional per-forge capability, git log adopted directly as the journal. git+Markdown adapter with the **forge wiki repo** shard (parameterized by repo/path), or stay distinct? Since forge wikis lack wiki-MRs, does shard-wiki supply the overlay→review→apply layer? (Forge-wikis dive §8.) +26. For a **single-file wiki** (UC-78), does shard-wiki support per-page overlays by + buffering and re-serializing the whole file, or require the Node `.tid` substrate for + write-through and treat single-file as read/projection/backup only? (TiddlyWiki dive §9.) 23. How does shard-wiki **honor/surface a shard's path-based access rules** (UC-06) in a projection without re-implementing its ACL engine? (Wiki.js dive §9.) \ No newline at end of file diff --git a/workplans/SHARD-WP-0003-engine-dives-batch.md b/workplans/SHARD-WP-0003-engine-dives-batch.md index ccd566b..266bf09 100644 --- a/workplans/SHARD-WP-0003-engine-dives-batch.md +++ b/workplans/SHARD-WP-0003-engine-dives-batch.md @@ -84,7 +84,7 @@ feed `SHARD-WP-0002` T12/T16. ```task id: SHARD-WP-0003-T3 -status: todo +status: done priority: medium state_hub_task_id: "a0ab1aba-9aa2-4214-b130-67ba2805e64b" ```