shard-wiki/research/260614-shard-spectrum-synthesis/findings.md

Synthesis — the shard spectrum: one capability model across the full dive set

Date: 2026-06-14 · revised 2026-06-14 (v2) — extended from nine systems to the full set (added Joplin, Logseq, the CRDT local-first cohort — Anytype/AFFiNE/AppFlowy — and Trilium); spectra grown from eleven to thirteen. Source kind: synthesis — consolidates every deep dive into a single comparative model feeding the shard adapter contract (SHARD-WP-0002 T11–T16) Lens: shard-wiki — what a backend must expose to participate, expressed as spectra of capability rather than a yes/no checklist

Purpose. Fourteen tools/systems have now been studied (plus the wiki-engines landscape): two conceptual ancestors (Xanadu, ZigZag); four engines (XWiki, TWiki, Foswiki + landscape); and the modern note/PKB tools (Roam, Obsidian, Notion, Joplin, Logseq, Anytype, AFFiNE, AppFlowy, Trilium), against the federation and origin research. This document reads them across each other. The payoff is a small set of capability spectra — now thirteen — each anchored at both ends by a real system, with federation operations degrading by position. That spectrum is the adapter contract's design surface. v2 folds the post-engine use cases (UC-44– UC-67) into the SHARD-WP-0002 tasks.

Inputs: research/260608-{federation-concepts,wikiengines-overview,c2-wiki-origins,yawex-prior-art}, research/260613-{xwiki,twiki,foswiki}-deep-dive, research/260614-{xanadu,zigzag,roam,obsidian,notion,joplin,logseq,localfirst-workspaces,trilium}-deep-dive. Output target: spec/TechnicalSpecificationDocument.md (adapter contract) via SHARD-WP-0002.

---

1. The shard family matrix

Candidate-shard backends across the dimensions that matter to the contract. (Xanadu and ZigZag are not shards — they are the conceptual ideals each column aspires to; listed last as reference anchors.)

Backend	Substrate	Attach mode(s)	Addressing	Structure	History	Merge	Query	→Markdown	Opacity
Git folder / repo	files	file-store	path (+ commit)	flat MD + frontmatter	git-native	git/text	no	native	none
Obsidian	files	file-store / plugin	path + in-file ^id	frontmatter (in-file)	none (Git plugin)	git/text	plugin (Dataview)	native (OFM)	none
Logseq	files (MD/Org) → SQLite (DB-graph)	file-store / plugin	in-file block id:: (sweet spot)	key:: props (in-file)	none (git)	git/text	Datalog (derived)	native-ish	none
TWiki / Foswiki	files + RCS / pluggable	file-store / API	path	%META% in-file (N records)	open file (RCS/PlainFile)	git/text	no	lossless (TML)	none
Trilium	SQLite (one file)	ETAPI / scripting / DB	noteId + branchId (id≠place)	labels+relations, inherited+templated; DAG	internal revisions	conflict-res	attr search	lossy (HTML)	per-note
Joplin	SQLite-local	sync-mirror / local-REST / plugin	page-level :/id (store)	notebooks + tags	internal revisions	conflict-notes	search	lossy	E2EE (whole)
XWiki	DB (Hibernate)	in-engine host / REST	path	XObjects/XClass	internal (xwikircs)	git/text	yes (XWQL)	engine syntax	none
Roam	client DataScript	in-app host only	store UUID	key:: attrs	internal (txn log)	(in-app)	yes (Datalog)	(Roam MD)	none
Notion	hosted Postgres	external API only	store UUID	DB schema + relations + rollups	internal, not portable	internal	yes (DB query)	lossy	none
Anytype	CRDT (any-sync)	replica / P2P node	object id (store)	typed object graph (ontology)	CRDT log	native-CRDT	graph	lossy	E2EE (whole)
AFFiNE	CRDT (Yjs)	replica / self-host	block id (store)	blocks; one-data-many-views	CRDT log	native-CRDT	DB filters	lossy	optional
AppFlowy	CRDT (Yrs)	replica / self-host	block id (store)	Notion-style DB + views	CRDT log	native-CRDT	DB query	lossy	self-host
TiddlyWiki	single file	file-store	path	typed tiddlers	none	git/text	no	varies	none
MediaWiki / Confluence	DB	API	path	wikitext / macros	internal-only	internal	limited	lossy	none
— Xanadu (ideal)	permascroll	—	tumbler (span)	spans + links	permanent	content-merge	—	—	—
— ZigZag (ideal)	cells/dims	—	cell id	N dimensions	—	—	dimension walk	—	—

Reading top to bottom is roughly shard-wiki's difficulty gradient: Git/Obsidian/ Logseq are friction-free file-store cases (Logseq adding block addressing on files); TWiki/Foswiki/Trilium add translation and (Trilium) a DAG + computed metadata; XWiki/Roam/Notion add DB structure and store addressing; the CRDT cohort (Anytype/AFFiNE/AppFlowy) adds a new merge model and (Anytype) P2P+E2EE; Notion is the hardest hosted case. Xanadu/ZigZag mark the ideals.

---

2. The capability spectra (the contract's real shape) — thirteen

Each capability is not boolean — it is a position on a spectrum anchored at each end by a real system. The contract should model positions; federation ops degrade by position.

1. Addressing granularity — none → whole-page(path) → page-level store id(Joplin :/id, Trilium noteId) → in-file span(Obsidian ^id) → in-file block(Logseq id::, the sweet spot: block-level AND git-diffable) → store-minted span(Roam/Notion/CRDT UUID) → portable tumbler(Xanadu ideal). (UC-51, UC-44/45.)
2. Content identity — none → path/title → fingerprint(hash) → span-set/equivalence (Xanadu). Drives equivalence + reverse-transclusion. (UC-46, UC-27.)
3. Identity vs placement (new emphasis) — path = identity(most) → identity separated from placement(Trilium note/branch; a page in many locations = a DAG). The clean model for a page under multiple paths/shards. (UC-66, UC-22.)
4. Structure — flat Markdown → in-file frontmatter/key::(Obsidian/Logseq) → in-file %META%(TWiki) → typed objects(XWiki) → DB schema+relations+rollups(Notion/AppFlowy) → typed object graph/ontology(Anytype) → computed: inherited+templated(Trilium). In-text federates; DB-locked needs sidecar+fidelity; computed needs effective-vs- own provenance. (UC-34, UC-39, UC-58, UC-67.)
5. History — none → internal-only/not-portable(Notion/Joplin/Trilium) → CRDT update log(Anytype/AFFiNE/AppFlowy) → open file format(TWiki RCS) → git-native(Git/Obsidian/ Logseq). Internal/CRDT-log ⇒ supplement; open-file ⇒ import; git ⇒ adopt. (UC-36, UC-41.)
6. Merge model (new, 13th) — none → git/text merge → conflict-notes/keep-both (Joplin) → native-CRDT conflict-free(Anytype/AFFiNE/AppFlowy). A CRDT shard merges itself — never impose git/text merge; speak the CRDT (replica) or stay projection/ overlay. (UC-64.)
7. Native query — none → text search → build-your-own derived index(Logseq DataScript over files; shard-wiki can do likewise) → datalog/graph(Roam/Anytype) → DB query (Notion/AppFlowy/XWiki). Delegate where present; build an index over the projection where not. (UC-52, UC-63, UC-05, UC-54.)
8. Translation to Markdown — native → lossless round-trip(Foswiki TML↔HTML) → lossy- with-fidelity-report(HTML/CKEditor Trilium; Notion blocks; CRDT/object models). Lossless ⇒ writable; lossy ⇒ read/projection floor + visible fidelity loss. (UC-42, UC-59, UC-03.)
9. Attachment mode (expanded) — a per-binding, capability-gated choice; a backend may offer several:
   • file-store — native on-disk store(Obsidian/Logseq/TWiki) or interchange/ sync mirror(Joplin items on WebDAV/S3) (UC-40, UC-53, UC-60, UC-62)
   • in-engine host — adapter inside the app via its API (Roam/Obsidian/Logseq/ Trilium scripting, XWiki components) (UC-38, UC-50)
   • local-REST — localhost API, app-running (Joplin Data API; Trilium ETAPI) (UC-38)
   • external-API — remote REST from outside (Notion) (UC-57)
   • CRDT replica — hold a local CRDT replica (Anytype/AFFiNE/AppFlowy) (UC-64)
   • P2P / no-central-endpoint — replica or peer/node, not a URL (Anytype) (UC-65)
10. Operational envelope — local/unbounded → realtime CRDT/WebSocket → rate-limited+ eventually-consistent+paginated(Notion ~3 rps). Sets live vs cache/poll/webhook. (UC-57, UC-31.)
11. Access grant — open(L0) → token → OAuth scoped+revocable(Notion) → P2P key/invite (Anytype). The backend may enforce no-silent-mutation. (UC-57, UC-65, shard-wiki-auth-in-core-decision.)
12. Content opacity (new, 12th) — plaintext → encrypted-at-rest whole-shard(Joplin/ Anytype E2EE) → per-item(Trilium protected notes — encrypted and plaintext coexist). Opaque content ⇒ backup/structure-shell only; never present ciphertext as readable. (UC-61.)
13. Write granularity — whole-file(TiddlyWiki) → per-page/note(Git/Obsidian/Joplin/ Trilium) → per-block(Roam/Notion/Logseq/CRDT). Sets overlay/patch/lock/conflict scope. (UC-35.)

*(Content types — Markdown-only → typed records → non-Markdown assets (Excalidraw/Canvas/ whiteboards/objects) — remains a cross-cutting page-model demand, tracked under structure

• T12 rather than as a standalone capability. UC-55.)*

Design consequence: T11's capability vocabulary = these thirteen spectra, not a flat read/write/diff/... list. The flat verbs remain the operations; the spectra are the profile saying how well each verb is supported and how it degrades.

---

3. Cross-cutting findings (the through-lines)

• Files-canonical, index-derived is the winning architecture. Obsidian's MetadataCache, Logseq's DataScript-over-files, Git's working tree, and shard-wiki's projection model agree: the graph/backlinks/structured index are derived and rebuildable, never a second source of truth. Roam/Notion/CRDT invert this (store canonical) and pay in portability. shard-wiki keeps files+journal canonical.
• Fine-grained addressing is real, adoptable, and can be git-diffable. Roam/Notion/ CRDT mint store UUIDs; Obsidian uses in-file ^id; Logseq resolves the tension — block-level id:: that is also git-diffable text. Adopt native IDs where present, fingerprint where not; portable span address stays the open ideal (Xanadu).
• Identity ≠ placement. (new) Trilium's note vs branch (a note cloned into many locations = a DAG) is the clean model for a page under multiple paths or across shards — separate what a page is from where it sits. The namespace-level form of the clone/ reference primitive.
• Transclusion ⇄ clone ⇄ embed ⇄ cloned-note is one primitive. Xanadu transclusion, ZigZag clone, Roam/Logseq embed, Obsidian ![[ ]], Notion synced block, Trilium note cloning — one "reference-not-copy" primitive over an addressable union (UC-32/44/45/ 51/66).
• CRDT changes the merge math. (new) The CRDT cohort merges concurrent edits itself; shard-wiki must not impose git/text merge — speak the CRDT (replica) or stay a projection/overlay. A new merge-model capability (spectrum 6).
• Structure & history federate iff in text; metadata can be computed. %META%/ frontmatter/key:: diff and travel; XObjects/Notion-DB/CRDT lock in. (new) Trilium adds computed metadata (inherited+templated) — represent effective-vs-own, don't flatten.
• Content opacity is per-item, not only whole-shard. (new) Joplin/Anytype encrypt whole-shard; Trilium encrypts per-note — the opacity capability must be granular.
• The attach surface is not always the native store. (new) Joplin's best surface is its sync mirror (not its SQLite); Logseq offers file-graph or DB-graph, and is migrating substrate (file→SQLite). Bind to capabilities, not to "it's files."
• The page model must stretch many ways at once: prose Markdown, typed/computed records (N-per-page, relations, inheritance), non-Markdown assets, reference/query- defined pages, and multi-placement (DAG) identity. The heaviest demand on T12.

---

4. How the post-engine use cases fold into the workplan

UC-44–UC-67 (from the 260614 dives) map onto the adapter-contract tasks:

UCs	Theme	Lands in
UC-35, UC-50, UC-53, UC-57, UC-60, UC-62, UC-64, UC-65	attachment modes (file-store native/mirror, in-engine, local-REST, external-API, CRDT-replica, P2P) + operational envelope + merge model	T11 + T14
UC-34, UC-39, UC-55, UC-58, UC-67	structured/typed/computed payload, non-MD assets, DB schema+relations	T12
UC-36	internal/CRDT-log history = supplement	T13
UC-42, UC-59	translation: lossless vs lossy-with-fidelity-report (HTML/blocks/CRDT)	T15
UC-31	webhooks / realtime / push-vs-poll	T6 / T11 envelope
UC-57 §6, UC-61, UC-65	scoped grant; content opacity (whole/ per-item); P2P key	T11 (access-grant + opacity)
UC-44, UC-45, UC-46, UC-51, UC-63, UC-66	span addressing, content identity, identity≠placement, transclusion-as-reference, derived index	T16 (+ T12)
UC-47, UC-48, UC-52, UC-54	dimensional navigation, query delegation/build, query-defined pages	T16 (+ T5/T10)

No new task needed — T16 already owns the addressing/identity/navigation thread; v2 adds identity≠placement (UC-66) and build-your-own derived index (UC-63) to it, and computed metadata (UC-67) + non-MD/multi-placement to T12.

---

5. Recommendations (decisions to record under SHARD-WP-0002)

1. Model capabilities as the thirteen spectra (§2), not flat verbs. (T11.)
2. Two new spectra: content opacity (per-item) and merge model. A CRDT shard's merge is native — never git-merge it; an encrypted shard degrades to structure-shell. (T11.)
3. Adopt native span IDs (prefer in-file/git-diffable, à la Logseq id::); fingerprint otherwise; keep portable-span-address open. (T16/T11, UC-51/46.)
4. Separate page identity from placement (Trilium note/branch) in the page/namespace model — a page is one entity with N placements (paths/shards). (T12/T16, UC-66.)
5. One reference-not-copy primitive for transclusion/clone/embed/cloned-note over the addressable union. (T16, UC-32/44/45/66.)
6. Prefer in-text structure; tolerate DB structure via sidecar+fidelity; represent computed (inherited/templated) metadata as effective-vs-own. (T12/T15, UC-34/58/67.)
7. Attachment mode is a per-binding, capability-gated choice across the full set: file-store (native or interchange-mirror), in-engine-host, local-REST, external-API, CRDT-replica, P2P/no-central-endpoint. Bind to capabilities — substrate can migrate (Logseq). (T14, UC-40/38/50/57/60/62/64/65 + UC-43.)
8. Query: delegate to a native engine where present; build a derived index over the projection where not (Logseq pattern). (T16/T5, UC-52/63/54.)
9. Keep files + coordination journal canonical; all indexes/projections derived and rebuildable. (Architecture invariant, INTENT.)

These honor INTENT: mechanism over policy (spectra not hard-coded behaviors), union without erasure (fidelity + effective-vs-own + identity/placement all preserve information), graceful degradation (every capability has a read/projection floor; opaque/ CRDT shards degrade to backup/projection), no silent mutation (access-grant + opacity + overlay + respect native merge), shard sovereignty (no backend forced to change substrate; substrate may migrate), Markdown-first/backend-neutral (in-text preferred, DB/ CRDT/HTML tolerated).

---

6. Open questions escalated by the synthesis

1. Portable span address across heterogeneous backends — wrap native IDs (Roam/Notion/ CRDT UUID, Logseq id::, Trilium noteId) in a shard-scoped address? (T16.)
2. CRDT shards — embed a CRDT client (Yjs/Yrs) for a live replica, or consume snapshots? Overlays as CRDT ops or out-of-band patches? (T14, UC-64.)
3. Identity vs placement — does shard-wiki adopt the note/branch split as its own page model, and is a cloned page one page with N placements or a transclusion? (T12/T16, UC-66.)
4. Computed metadata — materialize effective attributes (snapshot) or compute live from the shard's tree/templates, with per-attribute provenance? (T12, UC-67.)
5. Page model breadth — can one model carry prose + typed/computed records + non-MD assets + query-defined pages + multi-placement identity coherently? (T12.)
6. External-API write-through at scale (Notion ~3 rps) vs read/projection default. (T14, UC-57.)
7. Content opacity — what is visible for an opaque shard/item (IDs/structure vs nothing); does shard-wiki ever hold keys? (T11, UC-61.)

---

7. Sources

A synthesis; primary sources are the fourteen dives' findings.md files plus research/260608-{federation-concepts,wikiengines-overview,c2-wiki-origins,yawex-prior-art}. No new external research was performed in v2.

Cross-references: spec/UseCaseCatalog.md (UC-26–UC-67), workplans/SHARD-WP-0002-federation-architecture.md (T1–T16), INTENT.md (constraints), shard-wiki-auth-in-core-decision.

---

8. Traceability

• Consolidates: all fourteen deep dives + federation/origin research into one capability model (v2).
• Feeds: SHARD-WP-0002 T11 (thirteen-spectra vocabulary, incl. content-opacity + merge-model), T12 (page-model breadth: computed metadata, identity≠placement, non-MD), T13 (history incl. CRDT-log = supplement), T14 (full attachment-mode taxonomy incl. CRDT-replica + P2P + interchange-mirror + local-REST), T15 (lossy incl. HTML), T16 (addressing, content identity, identity≠placement, derived index, dimensional/query).
• UC coverage extended in the workplan from UC-34–UC-59 to UC-34–UC-67.
• No UCs added (synthesis only); no boundary changes (INTENT Stability Note untouched).