From 4543f5dc4341c78fcac12e21832ace2f1f452e9a Mon Sep 17 00:00:00 2001 From: tegwick Date: Sun, 14 Jun 2026 17:50:27 +0200 Subject: [PATCH] research: Wiki.js deep dive (storage-module engine, DB<->Git Markdown, GraphQL); UC-68/69 The closest existing engine to shard-wiki's own design: DB-canonical (Postgres/MySQL/SQLite) but with a pluggable storage-module abstraction that bidirectionally syncs clean Markdown (+ YAML frontmatter) to Git (also FS/S3/Azure), each provider acting as backup or source of truth. Two big findings: (1) the storage-module interface is concrete adapter-contract prior art alongside Foswiki::Store, and the closer one (medium = Markdown in Git); (2) the engine-maintained bidirectional Git mirror is the ideal file-store attach (clean MD + git history) and, being bidirectional, makes git commit a write path (overlay/patch as a commit, no API). Also GraphQL API (introspection = capability discovery; selective fields = efficient projection) and authn-delegated auth modules + path-based rule ACL. Added UC-68 (engine-maintained bidirectional Git mirror, write-by-commit), UC-69 (typed/introspectable API for schema discovery + selective projection); enriched UC-06/36/38/40/42/57. Catalog now 69 UCs. Architecture for SHARD-WP-0002 T11/T14: storage-module abstraction as 2nd adapter-contract prior art, engine-maintained Git mirror as attach+write surface, GraphQL introspection for capability discovery. Co-Authored-By: Claude Opus 4.8 --- SCOPE.md | 4 +- research/260614-wikijs-deep-dive/README.md | 49 ++++ research/260614-wikijs-deep-dive/findings.md | 249 +++++++++++++++++++ research/README.md | 3 +- spec/UseCaseCatalog.md | 109 +++++++- 5 files changed, 402 insertions(+), 12 deletions(-) create mode 100644 research/260614-wikijs-deep-dive/README.md create mode 100644 research/260614-wikijs-deep-dive/findings.md diff --git a/SCOPE.md b/SCOPE.md index 6a22ee7..7638716 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary. |-------|-------| | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | -| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium deep dives & shard-spectrum synthesis (`research/260614-*/`) | +| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | Architecture blueprint drafted; UseCaseCatalog 67 UCs from research; PRD/TSD scaffolds | +| Spec | Architecture blueprint drafted; UseCaseCatalog 69 UCs from research; PRD/TSD scaffolds | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract) | ## In Scope (today) diff --git a/research/260614-wikijs-deep-dive/README.md b/research/260614-wikijs-deep-dive/README.md new file mode 100644 index 0000000..f86c0fd --- /dev/null +++ b/research/260614-wikijs-deep-dive/README.md @@ -0,0 +1,49 @@ +# 260614 — Wiki.js deep dive (the storage-module engine that already speaks Git + Markdown) + +Date: 2026-06-14 + +## What this is + +A focused study of **Wiki.js** — the modern Node.js server wiki — read through +shard-wiki's lens. Wiki.js is the **closest existing engine to shard-wiki's own design**: +it is DB-canonical (Postgres/MySQL/SQLite) **but abstracts persistence behind pluggable +"storage modules"** — Git, local filesystem, S3, Azure Blob — each acting as **backup or +source of truth**, with the **Git module bidirectionally syncing clean `.md` files (+ YAML +frontmatter)** to a repo. That storage-module interface is, in effect, a shipped version +of shard-wiki's adapter contract — the second concrete prior art after **Foswiki::Store**, +and the closer one (medium = Markdown in Git). + +Distinctive material: +- **Architecture** — Node.js/Vue; **GraphQL API** over all resources; DB-canonical; + Markdown (primary) + HTML + AsciiDoc; path-based hierarchy; modular + (storage/auth/search/render/editor) +- **Storage modules** — provider connector (Git/FS/S3/Azure) with properties + create/ + update/delete/sync methods, **backup-or-source-of-truth**, `injectMetadata()` YAML + frontmatter; **Git module commits every change → version-controlled clean Markdown**, + bidirectional +- **GraphQL** — typed, **introspectable** (capability/schema discovery), **selective-field** + projection +- **Auth modules** (delegated authn) + **path-based rule ACL** + +## Contents + +| Path | Role | +|------|------| +| `findings.md` | Architecture, storage modules (≈ adapter contract), Git mirror as attach+write surface, GraphQL, access control, capability profile, INTENT mapping, UC seeds, architecture notes, sources | + +## Status + +Initial deep dive complete. Two new use cases promoted to `spec/UseCaseCatalog.md` (UC-68 +attach an engine-maintained bidirectional Git mirror of clean Markdown, incl. write-by- +commit; UC-69 attach via a typed, introspectable API — GraphQL — for schema discovery + +selective projection); UC-06/36/38/40/42/57 enriched. Logged for `SHARD-WP-0002` +(T11/T14): the **storage-module abstraction as a second adapter-contract prior art** (with +Foswiki::Store), the **engine-maintained Git mirror** as preferred attach+write surface, +and **GraphQL introspection** for capability discovery + selective projection. + +**Key takeaway recorded:** Wiki.js is the most shard-wiki-shaped engine yet — its +storage-module interface is near-isomorphic to the adapter contract, and its +Markdown-in-Git mirror is the ideal file-store attach (clean MD, git history, +write-by-commit). **Boundary:** one server-engine candidate shard, DB-canonical with a +maintained Git mirror; don't double-sync; honor its access rules; not the federation layer. + diff --git a/research/260614-wikijs-deep-dive/findings.md b/research/260614-wikijs-deep-dive/findings.md new file mode 100644 index 0000000..b7f4cd0 --- /dev/null +++ b/research/260614-wikijs-deep-dive/findings.md @@ -0,0 +1,249 @@ +# Findings — Wiki.js: the storage-module engine that already speaks Git + Markdown + +Date: 2026-06-14 +Source kind: **modern shipped product** — a server wiki engine; a *candidate shard* and +the closest existing engine to shard-wiki's own design (DB-canonical, but with a +**pluggable storage-module abstraction** that bidirectionally syncs clean Markdown to +**Git**/FS/S3/Azure) +Lens: shard-wiki — storage-module-as-adapter-contract prior art, the engine-maintained +Git mirror as an attach + write surface, GraphQL as a typed/introspectable API, and +path-based access rules + +> Why Wiki.js is special in this set. Every prior engine/tool stored content in *one* +> substrate and exposed *one or two* surfaces. Wiki.js is DB-canonical (Postgres/MySQL/ +> SQLite) **but abstracts persistence behind pluggable "storage modules"** — Git, local +> filesystem, S3, Azure Blob — each of which can act as **backup *or* source of truth**, +> and the **Git module bidirectionally syncs clean `.md` files (with YAML frontmatter)** +> to a repo. In other words, Wiki.js already does, internally, much of what shard-wiki's +> adapter contract is meant to do: a **versioned, capability-bearing, multi-provider +> persistence interface over Markdown**. It is the second concrete prior art for the +> contract after **Foswiki::Store** — and arguably the closer one, because its content +> is **Markdown in Git**, exactly shard-wiki's native shape. + +Contrast set: the engine dives (XWiki DB/components, TWiki/Foswiki file+RCS) and the +modern tools. Wiki.js = **modern server engine, DB-canonical, Git-storage-of-Markdown, +GraphQL API, pluggable modules**. + +--- + +## 1. Core architecture + +- **Stack:** Node.js backend, Vue front end; **everything is accessible via a GraphQL + API** (`/graphql`, with a Playground). The DB is **Postgres** (primary; MySQL/SQLite + supported), read-replica-capable. +- **Content:** **Markdown (primary)**, plus **HTML** and **AsciiDoc**, via pluggable + **editors** (Markdown, WYSIWYG/visual, raw HTML). Pages live on **paths** (a path-based + hierarchy) with tags. +- **Modular architecture:** typed pluggable **modules** — **storage**, **auth**, + **search**, **rendering**, **editor**. The contract-relevant ones are *storage* (§2) + and *auth* (§4). +- **3.0:** a large rewrite is in development; storage remains a headline subsystem. + +--- + +## 2. Storage modules — the headline (≈ shard-wiki's adapter contract) + +A **storage module** "connects Wiki.js with a local or remote storage provider, to act as +**backup or source of truth** for content. It consists of **properties** (user-set) and +**methods called on events** — create, update, delete content, and sync." Providers +include **Git, local filesystem, S3, Azure Blob**. + +The **Git module** is the most-used: **every page change commits to a Git repository**, +turning the wiki into **version-controlled `.md` files**. Content is written as **clean +Markdown**, and `injectMetadata()` prepends **YAML frontmatter** (for Markdown) or HTML +comments (for HTML) carrying page metadata. The sync is **bidirectional** with the remote +repo, and access to the synced files honors Wiki.js **access rules** (§4). + +Why this matters to shard-wiki — three distinct payoffs: + +1. **Prior art for the adapter contract.** Wiki.js's storage-module interface is a + shipped instance of what `SHARD-WP-0002` T11 specifies: a *versioned, capability- + bearing, multi-provider* persistence abstraction, with a backup-vs-source-of-truth + choice and lifecycle methods (create/update/delete/sync). It joins **Foswiki::Store** + as concrete prior art — and is closer, because its medium is **Markdown in Git**. +2. **The ideal file-store attach.** A Wiki.js instance with the Git module **is a git + repo of clean Markdown + YAML frontmatter**, maintained by the engine. shard-wiki can + attach that repo **directly** (UC-40) — better than reading the DB — and get **git + history for free** (UC-36, *adopt* not supplement, via the mirror). +3. **A write path via Git, no API needed.** Because the Git sync is **bidirectional**, + shard-wiki can **write-through by committing Markdown** to the repo; Wiki.js **ingests + git→DB**. This is overlay/patch application as a *git commit*, not an API call (UC-68). + Caution: Wiki.js owns the DB↔git sync — **don't double-sync**, and coordinate which + side is **source of truth** (configurable). + +--- + +## 3. GraphQL — a typed, introspectable external API + +Unlike the REST/local-REST tools (Joplin, Trilium, Notion), Wiki.js exposes a **GraphQL +API** over all resources. Two shard-wiki-relevant properties: + +- **Introspection = capability/schema discovery.** A GraphQL schema is self-describing; + an adapter can introspect the content/page schema rather than hard-coding it — feeding + T11's *capability discovery*. +- **Selective-field projection.** GraphQL fetches **exactly the fields a projection + needs** (page body vs. metadata vs. tags), reducing over-fetch — relevant to projection + efficiency and the operational envelope. (UC-69; an external-API sub-mode beside + Notion's REST, UC-57.) + +--- + +## 4. Auth modules + path-based access rules + +- **Auth modules** (local, LDAP, OAuth, SAML, …) — **authentication is delegated to + pluggable providers**, exactly shard-wiki's **authn-delegated** stance + ([[shard-wiki-auth-in-core-decision]]). +- **Access rules** are **path-based / rule-based ACL**: allow/deny access to all or + specific sections of the wiki per group; the Git-synced files honor the same rules. For + shard-wiki this is a richer ACL than TWiki's per-topic flags (UC-06) — **rule-based by + path pattern** — and informs how a projection should **honor/surface a shard's + restricted regions** (union without erasure: show that a region is restricted, don't + silently drop or expose it). + +--- + +## 5. Wiki.js as a shard — capability profile + +| Capability | Wiki.js | Notes for the adapter contract | +|------------|---------|--------------------------------| +| Read | yes (Git mirror / GraphQL / DB) | **Git mirror = clean Markdown** is the best surface | +| Write | yes (GraphQL **or git commit**) | bidirectional Git ingest = write-by-commit (UC-68) | +| Write granularity | per-page | — | +| Identity / addressing | path (+ page id) | path-based; YAML frontmatter carries metadata/id | +| Structure | YAML frontmatter (in-file via Git module) | git-diffable in-text structure (good) | +| History | **git-native via Git module** (else DB versions) | adopt the mirror's git history (UC-36) | +| Native query | GraphQL; pluggable search (DB/Elastic/Algolia) | delegate search; GraphQL selective fetch (UC-69) | +| Translation | Markdown primary; HTML/AsciiDoc too | mostly native; multi-format = light translation (UC-42) | +| Attach modes | **file-store (engine-maintained Git mirror)** / external-API (GraphQL) | clean MD in git is the ideal file-store (UC-40, UC-68) | +| Access control | **path-based rule ACL**; delegated auth modules | authn-delegated; honor restricted regions (UC-06) | +| Storage abstraction | **pluggable storage modules (Git/FS/S3/Azure)** | **adapter-contract prior art** (T11) | +| Content types | pages + media/assets | assets via storage modules | + +Verdict: **the most shard-wiki-shaped engine yet.** Best attached via its **Git storage +mirror** (clean Markdown + frontmatter + git history, with write-by-commit), or via +**GraphQL** for live/selective access. Its standout gift is **architectural**: a shipped +storage-module abstraction that is near-isomorphic to shard-wiki's adapter contract. + +--- + +## 6. Mapping to shard-wiki INTENT (compare, do not equate) + +### 6.1 Reinforcements + +- **Storage modules = the adapter contract, shipped.** A versioned, capability-bearing, + multi-provider persistence interface with backup-vs-source-of-truth — exactly T11's + shape, in production (with Foswiki::Store as the other prior art). +- **Markdown-in-Git is the native shape.** Wiki.js validates that a serious engine can + keep content as **clean `.md` + frontmatter in git** — shard-wiki's preferred medium — + while still offering a rich app over a DB. +- **authn-delegated** (auth modules) and **git coordination** (Git storage) are both core + shard-wiki stances, here shipped. + +### 6.2 Deliberate divergences (design bugs if conflated) + +1. **DB is Wiki.js's canonical store; the Git repo is a maintained mirror.** Treat the + mirror as the attach surface, but respect that **Wiki.js runs the DB↔git sync** — + don't double-sync; coordinate source-of-truth (UC-68). +2. **Bidirectional write needs care.** Committing Markdown is a real write path, but + races with the engine's own sync; apply overlays as commits the engine can ingest + cleanly, never out-of-band DB edits (overlay before mutation, no silent mutation). +3. **One Wiki.js instance = one shard**, not the federation layer; its access rules are + the shard's, to be honored/surfaced, not re-implemented. +4. **HTML/AsciiDoc pages** need light translation; Markdown pages are native. + +### 6.3 What Wiki.js teaches that shard-wiki should keep + +- **Model the contract on a real storage-module interface** (Wiki.js + Foswiki::Store): + properties + lifecycle methods (create/update/delete/sync) + a **backup-vs-source-of- + truth** flag + a provider list. (T11.) +- **Prefer the engine-maintained Git mirror** as the attach surface for DB-canonical + engines that offer one — clean Markdown, git history, and write-by-commit (UC-68). +- **Use GraphQL introspection for capability discovery and selective projection** where a + shard offers it (UC-69). + +--- + +## 7. Use-case seeds → catalog (promoted 2026-06-14) + +Last existing UC is **UC-67**. New UCs **UC-68, UC-69** added; existing UCs enriched. + +| Seed | Catalog action | +|------|----------------| +| **Attach an engine-maintained bidirectional Git mirror of clean Markdown** (Wiki.js Git storage module): attach the repo as a file-store shard (clean MD + YAML frontmatter + git history) and optionally **write-through by committing Markdown the engine ingests**; coordinate source-of-truth, don't double-sync | **UC-68 (new)** | +| **Attach via a typed, introspectable API (GraphQL)**: discover the shard's schema by introspection (capability discovery) and fetch only the fields a projection needs (selective projection) | **UC-69 (new)** | +| Pluggable **storage modules** (Git/FS/S3/Azure) = adapter-contract prior art; modular auth/search/render/editor | **enriches UC-38** | +| Git storage = **clean Markdown in git**, the ideal engine-maintained file-store attach | **enriches UC-40** | +| Git history natively via the storage module (**adopt** via mirror) | **enriches UC-36** | +| GraphQL external-API variant (typed/introspectable/selective) | **enriches UC-57** | +| Multi-format (Markdown primary; HTML/AsciiDoc) | **enriches UC-42** | +| Path-based rule ACL; delegated auth modules | **enriches UC-06** (links authz decision) | +| Engine-maintained clean git shard | links UC-02 | + +--- + +## 8. Architecture notes for SHARD-WP-0002 (no UC) + +- **Second adapter-contract prior art (T11).** Wiki.js's **storage-module interface** + (properties + create/update/delete/sync methods + backup-vs-source-of-truth + multi- + provider Git/FS/S3/Azure) is, with **Foswiki::Store**, the model to base the contract + on — and the closer one (medium = Markdown in Git). Cite both in T11. +- **Engine-maintained Git mirror (T14).** Add to the attachment taxonomy: for a + DB-canonical engine that offers a bidirectional Git-of-Markdown mirror, the **mirror is + the preferred attach + write surface** (clean MD, git history, write-by-commit). A + cleaner sibling of Joplin's proprietary interchange mirror (UC-60) and the file-store + native-store kind (UC-40). +- **GraphQL → capability discovery + selective projection (T11/T14).** Typed/ + introspectable APIs give schema discovery (capability negotiation) and reduce over-fetch + — note alongside Notion's REST external-API mode. +- **authn-delegated + path-based ACL** reinforce the settled authz decision and inform how + projections honor/surface restricted regions (union without erasure). + +--- + +## 9. Open questions (for spec / workplans) + +1. For an engine with a bidirectional Git mirror (UC-68), is the **mirror or the DB the + source of truth** from shard-wiki's view, and how do we avoid races with the engine's + own sync (write-by-commit cadence, locking)? +2. When both a **Git mirror and GraphQL** are available, which does a binding prefer — + mirror for content/history, GraphQL for live/selective metadata? Can one binding use + both? +3. How does shard-wiki **honor/surface path-based access rules** (UC-06) in a projection + without re-implementing the shard's ACL engine? +4. Should the adapter contract's **storage-module shape** be standardized directly on + Wiki.js + Foswiki::Store, or generalized further? + +--- + +## 10. Sources + +| Source | Used for | +|--------|----------| +| Wiki.js — GraphQL API docs (https://docs.requarks.io/dev/api) | Everything via GraphQL; `/graphql` Playground; access/modify all resources | +| Wiki.js — Git storage (https://docs.requarks.io/storage/git) | Per-change commit to git; clean `.md`; bidirectional sync with remote repo | +| Wiki.js — Storage overview + dev/storage.md (https://docs.requarks.io/storage ; https://github.com/requarks/wiki-docs/blob/master/dev/storage.md) | Storage module = provider connector; backup-or-source-of-truth; properties + create/update/delete/sync methods; `injectMetadata()` YAML frontmatter; providers Git/FS/S3/Azure | +| Wiki.js 3.0 storage preview (https://beta.js.wiki/blog/2021-wiki-js-3-feature-preview-storage) | Storage subsystem direction (3.0) | +| Wikipedia — Wiki.js (https://en.wikipedia.org/wiki/Wiki.js) | Node.js/Vue, DB options, formats, modular architecture | +| BrightCoding / Grokipedia overviews | Auth providers, access rules, editors, read replicas | + +Cross-references: `research/260613-foswiki-deep-dive/findings.md` (Foswiki::Store, the +other storage-interface prior art), `research/260614-joplin-deep-dive/findings.md` +(interchange/sync-mirror contrast), `research/260614-notion-deep-dive/findings.md` +(external-API/REST contrast), `research/260614-shard-spectrum-synthesis/findings.md` +(attachment-mode spectrum), `spec/UseCaseCatalog.md` (UC-02, UC-06, UC-36, UC-38, UC-40, +UC-42, UC-57), `workplans/SHARD-WP-0002-federation-architecture.md` (T11, T14). + +--- + +## 11. Traceability + +- New UCs: **UC-68, UC-69** → `spec/UseCaseCatalog.md`. +- Enriched UCs: **UC-06, UC-36, UC-38, UC-40, UC-42, UC-57** (links UC-02, authz decision). +- Architecture (no UC): **storage-module abstraction = 2nd adapter-contract prior art** + (with Foswiki::Store); engine-maintained Git mirror as attach+write surface; GraphQL + introspection → capability discovery + selective projection; path-based ACL + authn- + delegated → `SHARD-WP-0002` (T11, T14). +- Boundary recorded: Wiki.js is **one server-engine candidate shard**, DB-canonical with a + **bidirectional Git-of-Markdown mirror** that is the preferred attach surface; not a + substrate and not the federation layer (don't double-sync; honor its access rules). + diff --git a/research/README.md b/research/README.md index eda87d3..cf67dad 100644 --- a/research/README.md +++ b/research/README.md @@ -26,4 +26,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and | 2026-06-14 | `260614-joplin-deep-dive/` | Joplin — SQLite-local/Markdown-on-sync, interchange-format attach, E2EE content opacity; UC-60/61 | | 2026-06-14 | `260614-logseq-deep-dive/` | Logseq — block-graph on plain Markdown files, in-file block IDs, derived Datalog index; UC-62/63 | | 2026-06-14 | `260614-localfirst-workspaces-deep-dive/` | Anytype · AFFiNE · AppFlowy — CRDT local-first workspaces (any-sync/Yjs/Yrs), native merge, P2P/E2EE; UC-64/65 | -| 2026-06-14 | `260614-trilium-deep-dive/` | Trilium/TriliumNext — note cloning (DAG hierarchy), attribute inheritance/templates, HTML-native, scripting+ETAPI; UC-66/67 | \ No newline at end of file +| 2026-06-14 | `260614-trilium-deep-dive/` | Trilium/TriliumNext — note cloning (DAG hierarchy), attribute inheritance/templates, HTML-native, scripting+ETAPI; UC-66/67 | +| 2026-06-14 | `260614-wikijs-deep-dive/` | Wiki.js — storage-module engine (DB↔Git Markdown), GraphQL API, pluggable modules ≈ adapter-contract prior art; UC-68/69 | \ No newline at end of file diff --git a/spec/UseCaseCatalog.md b/spec/UseCaseCatalog.md index df3ce22..af941cf 100644 --- a/spec/UseCaseCatalog.md +++ b/spec/UseCaseCatalog.md @@ -10,8 +10,8 @@ Promoted from `research/260608-c2-wiki-origins/`, `research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`, `research/260614-notion-deep-dive/`, `research/260614-joplin-deep-dive/`, `research/260614-logseq-deep-dive/`, -`research/260614-localfirst-workspaces-deep-dive/`, and -`research/260614-trilium-deep-dive/`. +`research/260614-localfirst-workspaces-deep-dive/`, +`research/260614-trilium-deep-dive/`, and `research/260614-wikijs-deep-dive/`. See InfoTechPrimers on coulomb.social for use-case catalog conventions. ## Conventions @@ -94,7 +94,10 @@ external provider. shard-wiki delegates authentication, owns authorization (`spec/ArchitectureBlueprint.md`). yawex's model traces to **TWiki's per-topic `ALLOW/DENY TOPICVIEW/CHANGE/RENAME`** — the origin reference for shard-wiki's -optional per-page ACL at L4 (`research/260613-twiki-deep-dive/findings.md` §4). +optional per-page ACL at L4 (`research/260613-twiki-deep-dive/findings.md` §4). **Wiki.js** +is the modern form: **path-based rule ACL** (allow/deny by path pattern per group) + +delegated auth modules (`research/260614-wikijs-deep-dive/findings.md` §4) — a projection +should **honor/surface restricted regions**, not silently drop or expose them. **Priority:** Later ### UC-07 — Detect and reconcile cross-shard divergence @@ -289,7 +292,10 @@ case like Confluence/MediaWiki, not an import case. **Joplin** keeps internal no revisions locally, likewise not portable (`research/260614-joplin-deep-dive/findings.md` §1) — supplement. **CRDT** shards (Anytype/AFFiNE/AppFlowy) keep a CRDT update log, not git — also supplement, or snapshot the replica -(`research/260614-localfirst-workspaces-deep-dive/findings.md` §4). +(`research/260614-localfirst-workspaces-deep-dive/findings.md` §4). **Wiki.js** is the +*adopt* case via mirror: its Git storage module commits every change to a repo, so the +engine's history **is** git even though its canonical store is a DB (UC-68, +`research/260614-wikijs-deep-dive/findings.md` §2). **Priority:** Later ### UC-37 — Attach a static engine export as a read-only backup shard @@ -332,7 +338,11 @@ sub-mode between in-engine host and external API (UC-57, `research/260614-joplin-deep-dive/findings.md` §4). **Trilium** is the same dual pattern: **scripting** (frontend/backend code notes against a Script API) as the in-app host, plus **ETAPI** (token REST) as a designed external surface for a self-hosted server -(`research/260614-trilium-deep-dive/findings.md` §5). +(`research/260614-trilium-deep-dive/findings.md` §5). **Wiki.js** generalizes the host +surface into a **pluggable module system** (storage/auth/search/render/editor); its +**storage-module** interface is — with `Foswiki::Store` — concrete **adapter-contract +prior art** (versioned, multi-provider Git/FS/S3/Azure, backup-or-source-of-truth) +(`research/260614-wikijs-deep-dive/findings.md` §2, §8). **Priority:** Later ### UC-39 — Attach a wiki-as-application-platform shard (pages as typed records) @@ -375,7 +385,10 @@ Markdown-first and git-friendly — and uniquely **dual-attachable**: file-store `research/260614-obsidian-deep-dive/findings.md` §4, §10). Counter-example: **Joplin**'s native store is a **DB** (SQLite), so its file-attach surface is the *sync/interchange mirror* on a WebDAV/S3 target, not the native store — see UC-60 -(`research/260614-joplin-deep-dive/findings.md` §2). +(`research/260614-joplin-deep-dive/findings.md` §2). **Wiki.js** is the clean middle: also +DB-canonical, but its Git storage module maintains a repo of **plain Markdown** that is the +ideal engine-maintained file-store attach — see UC-68 +(`research/260614-wikijs-deep-dive/findings.md` §2). **Priority:** Later ### UC-41 — Import an engine's native file history into the coordination journal @@ -408,7 +421,10 @@ syntax to be safe (findings §9 Q2)? Without this, non-Markdown shards degrade t read-only (UC-03) — a graceful-degradation floor, not the goal. **Trilium** is the HTML-native case: text notes are HTML (CKEditor5), so participation needs HTML↔Markdown translation — more tractable than Notion's blocks but still lossy for some constructs -(`research/260614-trilium-deep-dive/findings.md` §4, links UC-59). +(`research/260614-trilium-deep-dive/findings.md` §4, links UC-59). **Wiki.js** is +multi-format (**Markdown primary**, plus HTML and AsciiDoc) with pluggable editors — mostly +native, light translation for the non-MD pages (`research/260614-wikijs-deep-dive/findings.md` +§1). **Priority:** Later ### UC-43 — Tolerate a shard's storage-backend swap without losing identity @@ -673,7 +689,9 @@ overlay/backup; write-through possible but bounded. Links the authz-in-core deci in-engine host (UC-38/50). Self-hostable variants: **AFFiNE Cloud** / **AppFlowy Cloud** (self-host sync servers) and **Anytype**'s open API + MCP are endpoint forms within a user's trust boundary (`research/260614-localfirst-workspaces-deep-dive/findings.md` §4, -§9) — see also UC-65 (P2P, no single endpoint). +§9) — see also UC-65 (P2P, no single endpoint). **Wiki.js** uses **GraphQL** (typed, +introspectable, selective-field) rather than REST — schema discovery + reduced over-fetch, +see UC-69 (`research/260614-wikijs-deep-dive/findings.md` §3). **Priority:** Later ### UC-58 — Attach a typed database with schema, relations, and views @@ -856,6 +874,40 @@ effective values (snapshot) vs compute live from the shard's tree/templates (fin Q2). Templates also reinforce UC-15 (blueprints). **Priority:** Later +### UC-68 — Attach an engine-maintained bidirectional Git mirror of clean Markdown + +**Actor:** Maintainer +**Goal:** Attach a DB-canonical engine that bidirectionally syncs its content to a Git +repo of **clean Markdown + YAML frontmatter** (Wiki.js Git storage module): use the repo +as a file-store shard (with git history) and optionally **write-through by committing +Markdown the engine ingests** — coordinating which side is source of truth, without +double-syncing. +**Source:** wikijs, intent +**Notes:** Wiki.js commits every page change to git as clean `.md` +(`injectMetadata()` prepends YAML frontmatter), bidirectionally with the remote repo +(`research/260614-wikijs-deep-dive/findings.md` §2). Cleaner than Joplin's proprietary +interchange mirror (UC-60) — it is **plain Markdown** — and richer than a read-only native +store (UC-40): the bidirectional ingest makes **git commit a write path** (overlay/patch +as a commit, no API). Caution: the **engine owns the DB↔git sync** — don't double-sync; +coordinate source-of-truth (configurable). Gives **git history natively** (UC-36, adopt +via mirror). Open: mirror-vs-DB source of truth; write-by-commit races (findings §9 Q1). +**Priority:** Later + +### UC-69 — Attach via a typed, introspectable API (schema discovery + selective projection) + +**Actor:** Orchestrator / adapter +**Goal:** Attach a shard through a typed, introspectable API (Wiki.js GraphQL) — using +**schema introspection** for capability/shape discovery and **selective-field queries** to +fetch only what a projection needs. +**Source:** wikijs, intent +**Notes:** Wiki.js exposes a **GraphQL** API over all resources +(`research/260614-wikijs-deep-dive/findings.md` §3). Two advantages over a REST +external-API (UC-57): **introspection** = self-describing schema → feeds capability +discovery (T11); **selective fields** = fetch body vs metadata vs tags as needed → +reduces over-fetch (projection efficiency, operational envelope). An external-API +sub-mode beside Notion's REST. +**Priority:** Later + --- ## B. Knowledge work and collaboration @@ -1116,6 +1168,8 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD. | UC-65 | | | | ⌘ | ✓ | | UC-66 | | | | ⊕ | ✓ | | UC-67 | | | | ⊕ | ✓ | +| UC-68 | | | | ⚓ | ✓ | +| UC-69 | | | | ⚓ | ✓ | | UC-08 | ✓ | | | | UC-09 | ✓ | | | | UC-10 | ✓ | | | @@ -1491,6 +1545,39 @@ substrate and not the federation layer. Architecture logged for `SHARD-WP-0002` (T11/T12/T14/T15/T16): DAG namespace + identity/placement split, computed/inherited metadata, per-item content opacity, HTML source model, scripting + ETAPI host surfaces. +### wikijs mapping + +(⚓ UC-68–UC-69 are placed in the **wikiengines** matrix column as the nearest existing +source — Wiki.js is a shipped engine — but their true lineage is the **Wiki.js deep dive**, +`research/260614-wikijs-deep-dive/findings.md`.) + +| Wiki.js mechanism (findings §) | Catalog UC | +|--------------------------------|------------| +| Bidirectional Git storage module → clean Markdown + YAML frontmatter in a repo (§2) | UC-68 (new) | +| GraphQL API: introspectable schema + selective-field projection (§3) | UC-69 (new) | +| Pluggable **storage modules** (Git/FS/S3/Azure) = adapter-contract prior art; modular auth/search/render/editor (§2, §8) | UC-38 (enriched) | +| Git storage = clean Markdown in git — ideal engine-maintained file-store attach (§2) | UC-40 (enriched) | +| Git history natively via the storage module (adopt via mirror) (§2) | UC-36 (enriched) | +| GraphQL external-API variant (typed/introspectable/selective) (§3) | UC-57 (enriched) | +| Multi-format: Markdown primary; HTML/AsciiDoc (§1) | UC-42 (enriched) | +| Path-based rule ACL; delegated auth modules (§4) | UC-06 (enriched; links authz decision) | +| Engine-maintained clean git shard (§2, §5) | links UC-02 | + +Note: Wiki.js is the **most shard-wiki-shaped engine** studied — DB-canonical but with a +**pluggable storage-module abstraction** that bidirectionally syncs **clean Markdown to +Git** (also FS/S3/Azure), each provider acting as **backup or source of truth**. That +interface is, with **Foswiki::Store**, concrete **prior art for the adapter contract** +(and the closer one — its medium is Markdown in Git). Its Git mirror is the **ideal +engine-maintained file-store attach** (clean MD + frontmatter + git history) and, being +**bidirectional**, makes **git commit a write path** (UC-68). It also uses **GraphQL** +(introspection → capability discovery; selective fields → efficient projection, UC-69) and +ships **authn-delegated auth modules + path-based rule ACL**. **Boundary recorded:** one +server-engine candidate shard; the Git mirror is the preferred attach surface but the +engine owns the DB↔git sync (don't double-sync); honor its access rules; not the federation +layer. Architecture logged for `SHARD-WP-0002` (T11/T14): storage-module abstraction as a +second adapter-contract prior art, engine-maintained Git mirror as attach+write surface, +GraphQL introspection for capability discovery + selective projection. + --- ## Open questions @@ -1540,4 +1627,8 @@ metadata, per-item content opacity, HTML source model, scripting + ETAPI host su page transcluded into multiple locations? Does shard-wiki adopt Trilium's identity ≠ placement (note/branch) split as its own page model? (Trilium dive §11.) 21. Are **inherited/templated attributes** (UC-67) materialized (snapshot) or computed - live from the shard's tree/templates, and how is per-attribute provenance recorded? \ No newline at end of file + live from the shard's tree/templates, and how is per-attribute provenance recorded? +22. For an **engine-maintained Git mirror** (UC-68), is the mirror or the engine DB the + source of truth, and how do we write-by-commit without racing the engine's own sync? +23. How does shard-wiki **honor/surface a shard's path-based access rules** (UC-06) in a + projection without re-implementing its ACL engine? (Wiki.js dive §9.) \ No newline at end of file