generated from coulomb/repo-seed
synthesis v2: extend shard spectrum to full dive set (13 spectra); re-fold into SHARD-WP-0002
Extend the cross-dive synthesis from nine systems to the full fourteen (added Joplin, Logseq, the CRDT cohort Anytype/AFFiNE/AppFlowy, Trilium). Spectra grown 11 -> 13: new merge-model (none/git/conflict-notes/ native-CRDT) and content-opacity (plaintext/whole-shard-E2EE/per-item), plus identity-vs-placement emphasis (Trilium note/branch DAG), Logseq's in-file id:: addressing sweet spot, build-your-own derived query index, and a six-mode attachment taxonomy (file-store native/interchange-mirror, in-engine-host, local-REST, external-API, CRDT-replica, P2P). New through-lines: CRDT changes the merge math; identity != placement; metadata can be computed; attach surface != native store (+ substrate migration). UC fold-in extended UC-44-59 -> UC-44-67. Folded into SHARD-WP-0002: T11 thirteen spectra; T12 computed metadata + multi-placement/DAG identity; T13 CRDT-log supplement; T14 full six-mode attachment taxonomy; T15 HTML source model; T16 identity!=placement + derived index. Context inputs += four dives; acceptance UC-26-67. No new tasks, no new UCs (synthesis only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,47 +1,55 @@
|
||||
# 260614 — Shard spectrum synthesis (one capability model across nine systems)
|
||||
# 260614 — Shard spectrum synthesis (one capability model across the full dive set)
|
||||
|
||||
Date: 2026-06-14
|
||||
Date: 2026-06-14 · **revised 2026-06-14 (v2)**
|
||||
|
||||
## What this is
|
||||
|
||||
A **synthesis** (no new external research) that reads the nine studied systems *across*
|
||||
each other and distills them into a single comparative model feeding the **shard adapter
|
||||
A **synthesis** (no new external research) that reads every studied system *across* the
|
||||
others and distills them into a single comparative model feeding the **shard adapter
|
||||
contract** (`SHARD-WP-0002`).
|
||||
|
||||
The nine: two conceptual ancestors (**Xanadu**, **ZigZag**), four engines (**XWiki,
|
||||
TWiki, Foswiki** + the wiki-engines landscape), and three modern tools (**Roam,
|
||||
Obsidian, Notion**), against the federation/origin research.
|
||||
**v2** extends from nine systems to the full set and grows the spectra from eleven to
|
||||
**thirteen**. The systems: two conceptual ancestors (**Xanadu, ZigZag**), four engines
|
||||
(**XWiki, TWiki, Foswiki** + the wiki-engines landscape), and the modern note/PKB tools
|
||||
(**Roam, Obsidian, Notion, Joplin, Logseq, Anytype, AFFiNE, AppFlowy, Trilium**), against
|
||||
the federation/origin research.
|
||||
|
||||
Centerpieces:
|
||||
- **The shard family matrix** — every candidate backend × {substrate, attach mode,
|
||||
addressing, structure, history, query, transclusion, →Markdown, write granularity},
|
||||
with Xanadu/ZigZag as the ideal anchors.
|
||||
- **The eleven capability spectra** — the claim that the adapter contract should model
|
||||
*positions on spectra* (addressing, content identity, structure, history, query,
|
||||
translation, attachment mode, operational envelope, access grant, write granularity,
|
||||
content types), each anchored at both ends by a real system, with federation ops
|
||||
degrading by position.
|
||||
- **UC-44–UC-59 → workplan task mapping**, including one genuinely new thread (T16:
|
||||
addressing, content identity, dimensional/query navigation).
|
||||
- **The shard family matrix** — every candidate backend × {substrate, attach mode(s),
|
||||
addressing, structure, history, **merge**, query, →Markdown, **opacity**}, with Xanadu/
|
||||
ZigZag as the ideal anchors.
|
||||
- **The thirteen capability spectra** — the claim that the adapter contract should model
|
||||
*positions on spectra*, each anchored at both ends by a real system, with federation
|
||||
ops degrading by position. v2 adds **content opacity** (12th) and **merge model**
|
||||
(13th), plus emphasis on **identity vs placement**, and expands the attachment-mode
|
||||
spectrum (file-store native/interchange-mirror, in-engine-host, local-REST,
|
||||
external-API, CRDT-replica, P2P/no-central-endpoint).
|
||||
- **UC-44–UC-67 → workplan task mapping** (T11–T16).
|
||||
|
||||
## Contents
|
||||
|
||||
| Path | Role |
|
||||
|------|------|
|
||||
| `findings.md` | Family matrix, the eleven spectra, cross-cutting through-lines, UC→task fold-in, recommendations/decisions, escalated open questions |
|
||||
| `findings.md` | Family matrix, the thirteen spectra, cross-cutting through-lines, UC→task fold-in, recommendations/decisions, escalated open questions |
|
||||
|
||||
## Status
|
||||
|
||||
Synthesis complete. No new use cases (consolidation only). Feeds `SHARD-WP-0002`: T11
|
||||
reframed around the eleven spectra; T12 page-model breadth (prose + typed records +
|
||||
non-Markdown assets + query-defined); T13 history confirmed; T14 three-mode attachment
|
||||
taxonomy; T15 lossy-with-fidelity-report; **new T16** (addressing, content identity,
|
||||
dimensional/query navigation). UC coverage in the workplan extended UC-34–UC-43 →
|
||||
UC-34–UC-59.
|
||||
Synthesis v2 complete. No new use cases (consolidation only). Feeds `SHARD-WP-0002`: T11
|
||||
reframed around the **thirteen spectra** (incl. content-opacity + merge-model); T12
|
||||
page-model breadth (prose + typed/**computed** records + non-Markdown assets +
|
||||
query-defined + **multi-placement/DAG identity**); T13 history (incl. CRDT-log =
|
||||
supplement); T14 **full attachment-mode taxonomy** (CRDT-replica + P2P + interchange-mirror
|
||||
+ local-REST added); T15 lossy-with-fidelity-report (incl. HTML); T16 (addressing, content
|
||||
identity, **identity≠placement**, derived index, dimensional/query). UC coverage extended
|
||||
UC-34–UC-59 → **UC-34–UC-67**.
|
||||
|
||||
**Through-lines recorded:** files-canonical/index-derived is the winning architecture;
|
||||
fine-grained addressing is real and adoptable (adopt native, fingerprint otherwise);
|
||||
transclusion ⇄ clone ⇄ embed is one primitive; structure/history federate iff in text;
|
||||
attach mode is a per-binding capability-gated choice; Notion proves the platform can
|
||||
*enforce* no-silent-mutation via scoped grants.
|
||||
**New through-lines (v2):** CRDT changes the merge math (native merge — never impose git
|
||||
merge); identity ≠ placement (Trilium note/branch) is the model for multi-location/
|
||||
multi-shard pages; metadata can be computed (inherited/templated), not just stored;
|
||||
content opacity is per-item, not only whole-shard; the attach surface is not always the
|
||||
native store (Joplin sync mirror; Logseq file-vs-DB and its substrate migration); the
|
||||
block-graph-on-files sweet spot (Logseq `id::`) resolves the addressing tension.
|
||||
**Carried from v1:** files-canonical/index-derived; fine-grained addressing is adoptable;
|
||||
transclusion=clone=embed is one primitive; structure/history federate iff in-text; attach
|
||||
mode is per-binding; Notion proves the platform can enforce no-silent-mutation.
|
||||
</content>
|
||||
|
||||
@@ -1,26 +1,28 @@
|
||||
# Synthesis — the shard spectrum: one capability model across nine systems
|
||||
# Synthesis — the shard spectrum: one capability model across the full dive set
|
||||
|
||||
Date: 2026-06-14
|
||||
Source kind: **synthesis** — consolidates the engine and tool deep dives plus the two
|
||||
Nelson conceptual systems into a single comparative model that feeds the **shard adapter
|
||||
contract** (`SHARD-WP-0002` T11–T16)
|
||||
Date: 2026-06-14 · **revised 2026-06-14 (v2)** — extended from nine systems to the full
|
||||
set (added Joplin, Logseq, the CRDT local-first cohort — Anytype/AFFiNE/AppFlowy — and
|
||||
Trilium); spectra grown from eleven to **thirteen**.
|
||||
Source kind: **synthesis** — consolidates every deep dive into a single comparative model
|
||||
feeding the **shard adapter contract** (`SHARD-WP-0002` T11–T16)
|
||||
Lens: shard-wiki — what a backend must expose to participate, expressed as *spectra* of
|
||||
capability rather than a yes/no checklist
|
||||
|
||||
> Purpose. Nine systems have been studied: two conceptual ancestors (**Xanadu**,
|
||||
> **ZigZag**), four engines (**XWiki, TWiki, Foswiki**, + the wiki-engines landscape),
|
||||
> and three modern tools (**Roam, Obsidian, Notion**), against the federation and
|
||||
> origin research (federation-concepts, c2, yawex). They were studied one at a time;
|
||||
> this document reads them *across* each other. The payoff is a small set of
|
||||
> **capability spectra** — addressing, structure, history, query, translation,
|
||||
> attachment mode, operational envelope, access grant — each anchored at both ends by a
|
||||
> real system. That spectrum *is* the adapter contract's design surface. It also folds
|
||||
> the post-engine use cases (**UC-44–UC-59**) into the existing `SHARD-WP-0002` tasks.
|
||||
> Purpose. Fourteen tools/systems have now been studied (plus the wiki-engines
|
||||
> landscape): two conceptual ancestors (**Xanadu, ZigZag**); four engines (**XWiki,
|
||||
> TWiki, Foswiki** + landscape); and the modern note/PKB tools (**Roam, Obsidian,
|
||||
> Notion, Joplin, Logseq, Anytype, AFFiNE, AppFlowy, Trilium**), against the federation
|
||||
> and origin research. This document reads them *across* each other. The payoff is a
|
||||
> small set of **capability spectra** — now **thirteen** — each anchored at both ends by
|
||||
> a real system, with federation operations degrading by position. That spectrum *is*
|
||||
> the adapter contract's design surface. v2 folds the post-engine use cases (**UC-44–
|
||||
> UC-67**) into the `SHARD-WP-0002` tasks.
|
||||
|
||||
Inputs: `research/260608-federation-concepts`, `research/260608-wikiengines-overview`,
|
||||
Inputs: `research/260608-{federation-concepts,wikiengines-overview,c2-wiki-origins,yawex-prior-art}`,
|
||||
`research/260613-{xwiki,twiki,foswiki}-deep-dive`,
|
||||
`research/260614-{xanadu,zigzag,roam,obsidian,notion}-deep-dive`. Output target:
|
||||
`spec/TechnicalSpecificationDocument.md` (adapter contract) via `SHARD-WP-0002`.
|
||||
`research/260614-{xanadu,zigzag,roam,obsidian,notion,joplin,logseq,localfirst-workspaces,trilium}-deep-dive`.
|
||||
Output target: `spec/TechnicalSpecificationDocument.md` (adapter contract) via
|
||||
`SHARD-WP-0002`.
|
||||
|
||||
---
|
||||
|
||||
@@ -30,180 +32,221 @@ Candidate-shard backends across the dimensions that matter to the contract. (Xan
|
||||
ZigZag are *not* shards — they are the conceptual ideals each column aspires to; listed
|
||||
last as reference anchors.)
|
||||
|
||||
| Backend | Substrate | Attach mode | Addressing | Structure | History | Native query | Transclusion | →Markdown | Write granularity |
|
||||
|---------|-----------|-------------|-----------|-----------|---------|--------------|-------------|-----------|-------------------|
|
||||
| **Git folder / repo** | files | file-store | path (+ commit) | flat MD + frontmatter | **git-native** | no | no | native | per-file |
|
||||
| **Obsidian vault** | files | file-store **or** in-app plugin | path + **in-file `^id`** | frontmatter (in-file) | none (Git plugin) | plugin (Dataview) | in-file `![[ ]]` | native (OFM) | per-file |
|
||||
| **TWiki** | files + RCS | file-store / API | path | `%META%` in-file | **open file (RCS)** | no | include macro | lossless (TML) | per-topic |
|
||||
| **Foswiki** | pluggable store | file-store / API | path | `%META%` in-file, N records | open file / PlainFile | no | include macro | lossless (TML) | per-topic |
|
||||
| **XWiki** | DB (Hibernate) | in-engine host / REST | path | **XObjects/XClass** | internal (`xwikircs`) | yes (XWQL) | macro | engine syntax | per-page / per-object |
|
||||
| **Roam** | client DataScript | **in-app host only** | **store UUID** | `key::` attrs | internal (txn log) | **yes (Datalog)** | **block embed** | (Roam MD) | **per-block** |
|
||||
| **Notion** | hosted Postgres | **external API only** | **store UUID** | **DB schema + relations + rollups** | internal, not portable | **yes (DB query)** | synced block | **lossy** | **per-block** |
|
||||
| **TiddlyWiki** | single file | file-store | path | typed tiddlers | none | no | transclusion | varies | **whole-file** |
|
||||
| **MediaWiki / Confluence** | DB | API | path | wikitext / macros | internal-only | limited | templates | lossy | per-page |
|
||||
| — *Xanadu (ideal)* | permascroll | — | **tumbler (span)** | spans + links | permanent | — | **content-identity** | — | span |
|
||||
| — *ZigZag (ideal)* | cells/dims | — | cell id | **N dimensions** | — | dimension walk | clone | — | cell |
|
||||
| Backend | Substrate | Attach mode(s) | Addressing | Structure | History | Merge | Query | →Markdown | Opacity |
|
||||
|---------|-----------|----------------|-----------|-----------|---------|-------|-------|-----------|---------|
|
||||
| **Git folder / repo** | files | file-store | path (+ commit) | flat MD + frontmatter | **git-native** | git/text | no | native | none |
|
||||
| **Obsidian** | files | file-store **/** plugin | path + in-file `^id` | frontmatter (in-file) | none (Git plugin) | git/text | plugin (Dataview) | native (OFM) | none |
|
||||
| **Logseq** | files (MD/Org) → SQLite (DB-graph) | file-store **/** plugin | **in-file block `id::`** (sweet spot) | `key::` props (in-file) | none (git) | git/text | **Datalog (derived)** | native-ish | none |
|
||||
| **TWiki / Foswiki** | files + RCS / pluggable | file-store **/** API | path | `%META%` in-file (N records) | **open file (RCS/PlainFile)** | git/text | no | lossless (TML) | none |
|
||||
| **Trilium** | SQLite (one file) | **ETAPI** / scripting / DB | `noteId` + **`branchId`** (id≠place) | labels+relations, **inherited+templated**; **DAG** | internal revisions | conflict-res | attr search | **lossy (HTML)** | **per-note** |
|
||||
| **Joplin** | SQLite-local | **sync-mirror** / local-REST / plugin | page-level `:/id` (store) | notebooks + tags | internal revisions | conflict-notes | search | lossy | E2EE (whole) |
|
||||
| **XWiki** | DB (Hibernate) | in-engine host / REST | path | **XObjects/XClass** | internal (`xwikircs`) | git/text | yes (XWQL) | engine syntax | none |
|
||||
| **Roam** | client DataScript | **in-app host only** | **store UUID** | `key::` attrs | internal (txn log) | (in-app) | yes (Datalog) | (Roam MD) | none |
|
||||
| **Notion** | hosted Postgres | **external API only** | store UUID | **DB schema + relations + rollups** | internal, not portable | internal | yes (DB query) | **lossy** | none |
|
||||
| **Anytype** | **CRDT (any-sync)** | **replica / P2P node** | object id (store) | **typed object graph (ontology)** | CRDT log | **native-CRDT** | graph | lossy | **E2EE (whole)** |
|
||||
| **AFFiNE** | **CRDT (Yjs)** | replica / self-host | block id (store) | blocks; **one-data-many-views** | CRDT log | **native-CRDT** | DB filters | lossy | optional |
|
||||
| **AppFlowy** | **CRDT (Yrs)** | replica / self-host | block id (store) | **Notion-style DB + views** | CRDT log | **native-CRDT** | DB query | lossy | self-host |
|
||||
| **TiddlyWiki** | single file | file-store | path | typed tiddlers | none | git/text | no | varies | none |
|
||||
| **MediaWiki / Confluence** | DB | API | path | wikitext / macros | internal-only | internal | limited | lossy | none |
|
||||
| — *Xanadu (ideal)* | permascroll | — | **tumbler (span)** | spans + links | permanent | content-merge | — | — | — |
|
||||
| — *ZigZag (ideal)* | cells/dims | — | cell id | **N dimensions** | — | — | dimension walk | — | — |
|
||||
|
||||
Reading the matrix top to bottom is reading shard-wiki's difficulty gradient: **Git
|
||||
folder → Obsidian** are the friction-free, INTENT-native cases; **TWiki/Foswiki** add
|
||||
syntax translation but stay file/diffable; **XWiki/Roam/Notion** add DB structure, store
|
||||
addressing, and (for Roam/Notion) non-file substrates; **Notion** is the hardest
|
||||
(closed, hosted, lossy, rate-limited). Xanadu/ZigZag mark where the *ideals* live.
|
||||
Reading top to bottom is roughly shard-wiki's difficulty gradient: **Git/Obsidian/
|
||||
Logseq** are friction-free file-store cases (Logseq adding block addressing on files);
|
||||
**TWiki/Foswiki/Trilium** add translation and (Trilium) a DAG + computed metadata;
|
||||
**XWiki/Roam/Notion** add DB structure and store addressing; the **CRDT cohort
|
||||
(Anytype/AFFiNE/AppFlowy)** adds a new merge model and (Anytype) P2P+E2EE; **Notion** is
|
||||
the hardest hosted case. Xanadu/ZigZag mark the ideals.
|
||||
|
||||
---
|
||||
|
||||
## 2. The capability spectra (the contract's real shape)
|
||||
## 2. The capability spectra (the contract's real shape) — thirteen
|
||||
|
||||
Each capability is **not boolean** — it is a position on a spectrum with a real anchor at
|
||||
each end. This is the core synthesis claim: the adapter contract should model *positions*,
|
||||
and federation operations should degrade by position.
|
||||
Each capability is **not boolean** — it is a position on a spectrum anchored at each end
|
||||
by a real system. The contract should model *positions*; federation ops degrade by
|
||||
position.
|
||||
|
||||
1. **Addressing granularity** — `none → whole-page(path) → in-file span(Obsidian ^id) →
|
||||
store-minted span(Roam/Notion UUID) → portable tumbler(Xanadu ideal)`. (UC-51, UC-44/45.)
|
||||
2. **Content identity** — `none → path/title match → content fingerprint(hash) →
|
||||
span-set/equivalence(Xanadu)`. Drives equivalence + reverse-transclusion. (UC-46, UC-27.)
|
||||
3. **Structure** — `flat Markdown → in-file frontmatter → in-file %META% → DB
|
||||
objects(XWiki) → DB schema+relations+rollups(Notion)`. The git-diffable forms federate
|
||||
easily; DB-locked forms need sidecar + fidelity reporting. (UC-34, UC-39, UC-58.)
|
||||
4. **History** — `none → internal-only/not-portable(Notion/Confluence) → open file
|
||||
format(TWiki RCS) → git-native(Git/Obsidian-with-Git)`. Internal ⇒ *supplement*;
|
||||
open-file ⇒ *import*; git-native ⇒ *adopt*. (UC-36, UC-41.)
|
||||
5. **Native query** — `none → text search → datalog(Roam) → DB query(Notion/XWiki)`.
|
||||
Where present, **delegate** derived-view computation; else compute over the projection.
|
||||
(UC-52, UC-05, UC-54.)
|
||||
6. **Translation to Markdown** — `native → lossless round-trip(Foswiki TML↔HTML) →
|
||||
lossy-with-fidelity-report(Notion)`. Lossless ⇒ writable; lossy ⇒ read/projection floor
|
||||
+ visible fidelity loss. (UC-42, UC-59, UC-03.)
|
||||
7. **Attachment mode** — `file-store direct(Obsidian/TWiki) → in-engine hosted
|
||||
adapter(Roam/XWiki) → external-API only(Notion)`. A backend may offer **more than one**
|
||||
(Obsidian offers both; TWiki file-or-API). (UC-40, UC-38, UC-50, UC-57.)
|
||||
8. **Operational envelope** — `local/unbounded → rate-limited+eventually-consistent+
|
||||
paginated(Notion ~3 rps)`. Determines whether projection can be live or must be
|
||||
cache/poll/webhook. (UC-57, UC-31.)
|
||||
9. **Access grant** — `open(L0) → token → OAuth scoped + revocable(Notion)`. The backend
|
||||
may *enforce* no-silent-mutation (Notion scoped grants). Ties the authz-in-core ladder.
|
||||
(UC-57, [[shard-wiki-auth-in-core-decision]].)
|
||||
10. **Write granularity** — `whole-file(TiddlyWiki) → per-page(Git/Obsidian/TWiki) →
|
||||
per-block(Roam/Notion)`. Sets overlay/patch/lock/conflict scope. (UC-35.)
|
||||
11. **Content types** — `Markdown-only → typed records(XWiki/Notion DB) → non-Markdown
|
||||
assets(Excalidraw/Canvas/attachments)`. The page model must stretch to all three.
|
||||
(UC-55, UC-58.)
|
||||
1. **Addressing granularity** — `none → whole-page(path) → page-level store id(Joplin
|
||||
:/id, Trilium noteId) → in-file span(Obsidian ^id) → in-file block(Logseq id::, the
|
||||
sweet spot: block-level AND git-diffable) → store-minted span(Roam/Notion/CRDT UUID) →
|
||||
portable tumbler(Xanadu ideal)`. (UC-51, UC-44/45.)
|
||||
2. **Content identity** — `none → path/title → fingerprint(hash) → span-set/equivalence
|
||||
(Xanadu)`. Drives equivalence + reverse-transclusion. (UC-46, UC-27.)
|
||||
3. **Identity vs placement** *(new emphasis)* — `path = identity(most) → identity
|
||||
separated from placement(Trilium note/branch; a page in many locations = a DAG)`. The
|
||||
clean model for a page under multiple paths/shards. (UC-66, UC-22.)
|
||||
4. **Structure** — `flat Markdown → in-file frontmatter/key::(Obsidian/Logseq) → in-file
|
||||
%META%(TWiki) → typed objects(XWiki) → DB schema+relations+rollups(Notion/AppFlowy) →
|
||||
typed object graph/ontology(Anytype) → computed: inherited+templated(Trilium)`.
|
||||
In-text federates; DB-locked needs sidecar+fidelity; **computed** needs effective-vs-
|
||||
own provenance. (UC-34, UC-39, UC-58, UC-67.)
|
||||
5. **History** — `none → internal-only/not-portable(Notion/Joplin/Trilium) → CRDT update
|
||||
log(Anytype/AFFiNE/AppFlowy) → open file format(TWiki RCS) → git-native(Git/Obsidian/
|
||||
Logseq)`. Internal/CRDT-log ⇒ *supplement*; open-file ⇒ *import*; git ⇒ *adopt*.
|
||||
(UC-36, UC-41.)
|
||||
6. **Merge model** *(new, 13th)* — `none → git/text merge → conflict-notes/keep-both
|
||||
(Joplin) → native-CRDT conflict-free(Anytype/AFFiNE/AppFlowy)`. A CRDT shard merges
|
||||
itself — **never impose git/text merge**; speak the CRDT (replica) or stay projection/
|
||||
overlay. (UC-64.)
|
||||
7. **Native query** — `none → text search → build-your-own derived index(Logseq DataScript
|
||||
over files; shard-wiki can do likewise) → datalog/graph(Roam/Anytype) → DB query
|
||||
(Notion/AppFlowy/XWiki)`. Delegate where present; **build an index over the projection**
|
||||
where not. (UC-52, UC-63, UC-05, UC-54.)
|
||||
8. **Translation to Markdown** — `native → lossless round-trip(Foswiki TML↔HTML) → lossy-
|
||||
with-fidelity-report(HTML/CKEditor Trilium; Notion blocks; CRDT/object models)`.
|
||||
Lossless ⇒ writable; lossy ⇒ read/projection floor + visible fidelity loss. (UC-42,
|
||||
UC-59, UC-03.)
|
||||
9. **Attachment mode** *(expanded)* — a **per-binding, capability-gated** choice; a
|
||||
backend may offer several:
|
||||
- **file-store** — *native on-disk store*(Obsidian/Logseq/TWiki) **or** *interchange/
|
||||
sync mirror*(Joplin items on WebDAV/S3) (UC-40, UC-53, UC-60, UC-62)
|
||||
- **in-engine host** — adapter inside the app via its API (Roam/Obsidian/Logseq/
|
||||
Trilium scripting, XWiki components) (UC-38, UC-50)
|
||||
- **local-REST** — localhost API, app-running (Joplin Data API; Trilium ETAPI) (UC-38)
|
||||
- **external-API** — remote REST from outside (Notion) (UC-57)
|
||||
- **CRDT replica** — hold a local CRDT replica (Anytype/AFFiNE/AppFlowy) (UC-64)
|
||||
- **P2P / no-central-endpoint** — replica or peer/node, not a URL (Anytype) (UC-65)
|
||||
10. **Operational envelope** — `local/unbounded → realtime CRDT/WebSocket → rate-limited+
|
||||
eventually-consistent+paginated(Notion ~3 rps)`. Sets live vs cache/poll/webhook.
|
||||
(UC-57, UC-31.)
|
||||
11. **Access grant** — `open(L0) → token → OAuth scoped+revocable(Notion) → P2P key/invite
|
||||
(Anytype)`. The backend may *enforce* no-silent-mutation. (UC-57, UC-65,
|
||||
[[shard-wiki-auth-in-core-decision]].)
|
||||
12. **Content opacity** *(new, 12th)* — `plaintext → encrypted-at-rest whole-shard(Joplin/
|
||||
Anytype E2EE) → per-item(Trilium protected notes — encrypted and plaintext coexist)`.
|
||||
Opaque content ⇒ backup/structure-shell only; never present ciphertext as readable.
|
||||
(UC-61.)
|
||||
13. **Write granularity** — `whole-file(TiddlyWiki) → per-page/note(Git/Obsidian/Joplin/
|
||||
Trilium) → per-block(Roam/Notion/Logseq/CRDT)`. Sets overlay/patch/lock/conflict
|
||||
scope. (UC-35.)
|
||||
|
||||
Design consequence: **T11's capability vocabulary should be these eleven spectra**, not
|
||||
the original flat `read/write/diff/...` list. The flat verbs remain as the *operations*;
|
||||
the spectra are the *profile* that says how well each verb is supported and how it
|
||||
degrades.
|
||||
*(Content types — Markdown-only → typed records → non-Markdown assets (Excalidraw/Canvas/
|
||||
whiteboards/objects) — remains a cross-cutting page-model demand, tracked under structure
|
||||
+ T12 rather than as a standalone capability. UC-55.)*
|
||||
|
||||
Design consequence: **T11's capability vocabulary = these thirteen spectra**, not a flat
|
||||
`read/write/diff/...` list. The flat verbs remain the *operations*; the spectra are the
|
||||
*profile* saying how well each verb is supported and how it degrades.
|
||||
|
||||
---
|
||||
|
||||
## 3. Cross-cutting findings (the through-lines)
|
||||
|
||||
- **Files-canonical, index-derived is the winning architecture.** Obsidian's
|
||||
MetadataCache, Git's working tree, and shard-wiki's own projection model all agree:
|
||||
the link graph / backlinks / structured index are **derived and rebuildable**, never a
|
||||
second source of truth. Roam/Notion invert this (DB canonical) and pay for it in
|
||||
portability. shard-wiki keeps files+journal canonical; everything else is projection.
|
||||
(Reinforces UC-05, UC-17–20; INTENT.)
|
||||
- **Fine-grained addressing is real and adoptable** — Roam (`:block/uid`) and Notion
|
||||
(UUID) ship it store-minted; Obsidian ships it in-file (`^id`). The Xanadu tumbler is
|
||||
the unreached ideal (portable, transfinite). shard-wiki should **adopt native span IDs
|
||||
where present, fingerprint where not** (UC-51 + UC-46), and treat a portable span
|
||||
address as the open hard problem (Xanadu §11 Q1).
|
||||
- **Transclusion ⇄ clone ⇄ embed is one primitive.** Xanadu transclusion, ZigZag clones,
|
||||
Roam block embeds, Obsidian `![[ ]]`, Notion synced blocks are the same idea at
|
||||
different fidelities. A single internal "reference-not-copy" primitive over an
|
||||
addressable union serves UC-32/44/45/51. (Convergence first noted in the ZigZag dive.)
|
||||
- **Structure and history federate iff they live in text.** The sharpest recurring
|
||||
engine insight: `%META%`-in-file (TWiki/Foswiki) and frontmatter (Obsidian) diff and
|
||||
travel; XObjects and Notion DBs lock in. The contract should *prefer* in-text
|
||||
structure and *tolerate* DB structure via sidecar + fidelity report.
|
||||
- **Three attachment modes, and a backend may span them.** The clean taxonomy
|
||||
(file-store / in-engine-host / external-API) plus the observation that one backend can
|
||||
offer several (Obsidian: file *or* plugin; TWiki: file *or* API) means **attach mode is
|
||||
a per-binding choice, capability-gated**, not a fixed property of the backend.
|
||||
- **Notion proves the platform can enforce INTENT.** Scoped, revocable grants are
|
||||
no-silent-mutation made mechanical — evidence the principle is implementable, and a
|
||||
model for how shard-wiki represents partial/consented visibility.
|
||||
- **The page model must stretch four ways at once:** prose Markdown, typed records
|
||||
(with N-per-page and inter-record relations), non-Markdown assets, and
|
||||
reference/query-defined pages. This is the heaviest single demand on T12.
|
||||
MetadataCache, **Logseq's DataScript-over-files**, Git's working tree, and shard-wiki's
|
||||
projection model agree: the graph/backlinks/structured index are **derived and
|
||||
rebuildable**, never a second source of truth. Roam/Notion/CRDT invert this (store
|
||||
canonical) and pay in portability. shard-wiki keeps files+journal canonical.
|
||||
- **Fine-grained addressing is real, adoptable, and can be git-diffable.** Roam/Notion/
|
||||
CRDT mint store UUIDs; Obsidian uses in-file `^id`; **Logseq resolves the tension —
|
||||
block-level `id::` that is also git-diffable text**. Adopt native IDs where present,
|
||||
fingerprint where not; portable span address stays the open ideal (Xanadu).
|
||||
- **Identity ≠ placement.** *(new)* Trilium's **note vs branch** (a note cloned into many
|
||||
locations = a DAG) is the clean model for a page under multiple paths or across shards —
|
||||
separate *what a page is* from *where it sits*. The namespace-level form of the clone/
|
||||
reference primitive.
|
||||
- **Transclusion ⇄ clone ⇄ embed ⇄ cloned-note is one primitive.** Xanadu transclusion,
|
||||
ZigZag clone, Roam/Logseq embed, Obsidian `![[ ]]`, Notion synced block, **Trilium note
|
||||
cloning** — one "reference-not-copy" primitive over an addressable union (UC-32/44/45/
|
||||
51/66).
|
||||
- **CRDT changes the merge math.** *(new)* The CRDT cohort merges concurrent edits itself;
|
||||
shard-wiki must **not impose git/text merge** — speak the CRDT (replica) or stay a
|
||||
projection/overlay. A new **merge-model** capability (spectrum 6).
|
||||
- **Structure & history federate iff in text; metadata can be computed.** `%META%`/
|
||||
frontmatter/`key::` diff and travel; XObjects/Notion-DB/CRDT lock in. *(new)* Trilium
|
||||
adds **computed metadata** (inherited+templated) — represent effective-vs-own, don't
|
||||
flatten.
|
||||
- **Content opacity is per-item, not only whole-shard.** *(new)* Joplin/Anytype encrypt
|
||||
whole-shard; **Trilium encrypts per-note** — the opacity capability must be granular.
|
||||
- **The attach surface is not always the native store.** *(new)* Joplin's best surface is
|
||||
its **sync mirror** (not its SQLite); Logseq offers file-graph **or** DB-graph, and is
|
||||
**migrating substrate** (file→SQLite). Bind to capabilities, not to "it's files."
|
||||
- **The page model must stretch many ways at once:** prose Markdown, typed/computed
|
||||
records (N-per-page, relations, inheritance), non-Markdown assets, reference/query-
|
||||
defined pages, and **multi-placement (DAG) identity**. The heaviest demand on T12.
|
||||
|
||||
---
|
||||
|
||||
## 4. How the post-engine use cases fold into the workplan
|
||||
|
||||
UC-44–UC-59 (from the five 260614 dives) map onto the existing adapter-contract tasks,
|
||||
with one genuinely new thread needing a home (T16):
|
||||
UC-44–UC-67 (from the 260614 dives) map onto the adapter-contract tasks:
|
||||
|
||||
| UCs | Theme | Lands in |
|
||||
|-----|-------|----------|
|
||||
| UC-35, UC-50, UC-53, UC-57 | attachment mode + operational envelope + write granularity | **T11** (capabilities) + **T14** (binding) |
|
||||
| UC-34, UC-39, UC-55, UC-58 | structured/typed payload, non-Markdown assets, DB schema+relations | **T12** (page model) |
|
||||
| UC-36 | internal-only history (Notion) = supplement | **T13** (history) |
|
||||
| UC-42, UC-59 | translation: lossless vs lossy-with-fidelity-report | **T15** (translation) |
|
||||
| UC-31 | webhooks / push-vs-poll under eventual consistency | **T6** / **T11** envelope |
|
||||
| UC-57 §6 | scoped, revocable grant; no silent mutation | **T11** access-grant + authz decision |
|
||||
| **UC-44, UC-45, UC-46, UC-51** | span addressing, content identity, transclusion-as-reference | **T8** + **new T16** |
|
||||
| **UC-47, UC-48, UC-52, UC-54** | dimensional navigation, query delegation, query-defined pages | **T5/T10** + **new T16** |
|
||||
| UC-35, UC-50, UC-53, UC-57, **UC-60, UC-62, UC-64, UC-65** | attachment modes (file-store native/mirror, in-engine, local-REST, external-API, CRDT-replica, P2P) + operational envelope + merge model | **T11** + **T14** |
|
||||
| UC-34, UC-39, UC-55, UC-58, **UC-67** | structured/typed/**computed** payload, non-MD assets, DB schema+relations | **T12** |
|
||||
| UC-36 | internal/CRDT-log history = supplement | **T13** |
|
||||
| UC-42, UC-59 | translation: lossless vs lossy-with-fidelity-report (HTML/blocks/CRDT) | **T15** |
|
||||
| UC-31 | webhooks / realtime / push-vs-poll | **T6** / **T11** envelope |
|
||||
| UC-57 §6, **UC-61, UC-65** | scoped grant; **content opacity** (whole/ per-item); P2P key | **T11** (access-grant + opacity) |
|
||||
| UC-44, UC-45, UC-46, UC-51, **UC-63, UC-66** | span addressing, content identity, **identity≠placement**, transclusion-as-reference, derived index | **T16** (+ T12) |
|
||||
| UC-47, UC-48, UC-52, UC-54 | dimensional navigation, query delegation/build, query-defined pages | **T16** (+ T5/T10) |
|
||||
|
||||
**New thread (T16): Addressing, content identity, and dimensional/query navigation.** No
|
||||
existing task owns (a) the portable span-address scheme + content-identity model that
|
||||
transclusion/overlay/reverse-lookup need, nor (b) the dimensional/query view model
|
||||
(ZigZag dimensions, Roam/Notion query, Dataview query-defined pages). These are the
|
||||
Nelson-ideals-meet-modern-tools thread and warrant their own design task.
|
||||
No new task needed — **T16** already owns the addressing/identity/navigation thread; v2
|
||||
adds **identity≠placement (UC-66)** and **build-your-own derived index (UC-63)** to it,
|
||||
and **computed metadata (UC-67)** + **non-MD/multi-placement** to **T12**.
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommendations (decisions to record under SHARD-WP-0002)
|
||||
|
||||
1. **Model capabilities as the eleven spectra (§2), not flat verbs.** Keep verbs as
|
||||
operations; add the spectra as the profile. (T11.)
|
||||
2. **Adopt native span IDs; fingerprint otherwise; keep portable-span-address open.**
|
||||
(T16, UC-51/46.)
|
||||
3. **One reference-not-copy primitive** for transclusion/clone/embed over the addressable
|
||||
union. (T16/T8, UC-32/44/45.)
|
||||
4. **Prefer in-text structure; tolerate DB structure via sidecar + fidelity report.**
|
||||
(T12/T15, UC-34/58/59.)
|
||||
5. **Attach mode is a per-binding, capability-gated choice** across the three modes; a
|
||||
backend may offer several. (T14, UC-40/38/50/57.)
|
||||
6. **Add operational-envelope and access-grant to the profile** as first-class fields.
|
||||
(T11, UC-57.)
|
||||
7. **Delegate derived views to native query engines where present; else compute over the
|
||||
projection.** (T16/T5, UC-52/54.)
|
||||
8. **Keep files + coordination journal canonical; all indexes/projections derived and
|
||||
1. **Model capabilities as the thirteen spectra (§2)**, not flat verbs. (T11.)
|
||||
2. **Two new spectra: content opacity (per-item) and merge model.** A CRDT shard's merge
|
||||
is native — never git-merge it; an encrypted shard degrades to structure-shell. (T11.)
|
||||
3. **Adopt native span IDs (prefer in-file/git-diffable, à la Logseq `id::`); fingerprint
|
||||
otherwise; keep portable-span-address open.** (T16/T11, UC-51/46.)
|
||||
4. **Separate page identity from placement** (Trilium note/branch) in the page/namespace
|
||||
model — a page is one entity with N placements (paths/shards). (T12/T16, UC-66.)
|
||||
5. **One reference-not-copy primitive** for transclusion/clone/embed/cloned-note over the
|
||||
addressable union. (T16, UC-32/44/45/66.)
|
||||
6. **Prefer in-text structure; tolerate DB structure via sidecar+fidelity; represent
|
||||
computed (inherited/templated) metadata as effective-vs-own.** (T12/T15, UC-34/58/67.)
|
||||
7. **Attachment mode is a per-binding, capability-gated choice** across the full set:
|
||||
file-store (native *or* interchange-mirror), in-engine-host, local-REST, external-API,
|
||||
CRDT-replica, P2P/no-central-endpoint. Bind to capabilities — substrate can migrate
|
||||
(Logseq). (T14, UC-40/38/50/57/60/62/64/65 + UC-43.)
|
||||
8. **Query: delegate to a native engine where present; build a derived index over the
|
||||
projection where not** (Logseq pattern). (T16/T5, UC-52/63/54.)
|
||||
9. **Keep files + coordination journal canonical; all indexes/projections derived and
|
||||
rebuildable.** (Architecture invariant, INTENT.)
|
||||
|
||||
These honor INTENT: mechanism over policy (spectra not hard-coded behaviors), union
|
||||
without erasure (fidelity reporting makes loss visible), graceful degradation (every
|
||||
capability has a read/projection floor), no silent mutation (access-grant + overlay),
|
||||
shard sovereignty (no backend forced to change substrate), Markdown-first/backend-neutral
|
||||
(in-text preferred, DB tolerated).
|
||||
without erasure (fidelity + effective-vs-own + identity/placement all preserve
|
||||
information), graceful degradation (every capability has a read/projection floor; opaque/
|
||||
CRDT shards degrade to backup/projection), no silent mutation (access-grant + opacity +
|
||||
overlay + respect native merge), shard sovereignty (no backend forced to change
|
||||
substrate; substrate may migrate), Markdown-first/backend-neutral (in-text preferred, DB/
|
||||
CRDT/HTML tolerated).
|
||||
|
||||
---
|
||||
|
||||
## 6. Open questions escalated by the synthesis
|
||||
|
||||
1. **Portable span address** — still unsolved across heterogeneous backends (Xanadu's
|
||||
unbuilt part). Wrap native IDs in a shard-scoped address? (T16.)
|
||||
2. **External-API write-through at scale** — is Notion-class write viable under ~3 rps,
|
||||
or read/projection/overlay/backup by default? (T14, UC-57.)
|
||||
3. **Inter-record relations in the union** (UC-58) — typed links in the link graph, a
|
||||
separate relation index (ZigZag many-to-many), or both? (T16/T5.)
|
||||
4. **Page model breadth** — can one model carry prose + typed records + non-Markdown
|
||||
assets + query-defined pages without becoming incoherent? (T12.)
|
||||
5. **Dimensional navigation surface** — public API/UI paradigm or internal organizing
|
||||
concept? (T16, UC-47/48.)
|
||||
1. **Portable span address** across heterogeneous backends — wrap native IDs (Roam/Notion/
|
||||
CRDT UUID, Logseq `id::`, Trilium noteId) in a shard-scoped address? (T16.)
|
||||
2. **CRDT shards** — embed a CRDT client (Yjs/Yrs) for a live replica, or consume
|
||||
snapshots? Overlays as CRDT ops or out-of-band patches? (T14, UC-64.)
|
||||
3. **Identity vs placement** — does shard-wiki adopt the note/branch split as its own page
|
||||
model, and is a cloned page one page with N placements or a transclusion? (T12/T16,
|
||||
UC-66.)
|
||||
4. **Computed metadata** — materialize effective attributes (snapshot) or compute live
|
||||
from the shard's tree/templates, with per-attribute provenance? (T12, UC-67.)
|
||||
5. **Page model breadth** — can one model carry prose + typed/computed records + non-MD
|
||||
assets + query-defined pages + multi-placement identity coherently? (T12.)
|
||||
6. **External-API write-through at scale** (Notion ~3 rps) vs read/projection default.
|
||||
(T14, UC-57.)
|
||||
7. **Content opacity** — what is visible for an opaque shard/item (IDs/structure vs
|
||||
nothing); does shard-wiki ever hold keys? (T11, UC-61.)
|
||||
|
||||
---
|
||||
|
||||
## 7. Sources
|
||||
|
||||
This is a synthesis; primary sources are the nine dives' `findings.md` files plus
|
||||
`research/260608-federation-concepts`, `research/260608-wikiengines-overview`,
|
||||
`research/260608-c2-wiki-origins`, `research/260608-yawex-prior-art`. No new external
|
||||
research was performed.
|
||||
A synthesis; primary sources are the fourteen dives' `findings.md` files plus
|
||||
`research/260608-{federation-concepts,wikiengines-overview,c2-wiki-origins,yawex-prior-art}`.
|
||||
No new external research was performed in v2.
|
||||
|
||||
Cross-references: `spec/UseCaseCatalog.md` (UC-26–UC-59),
|
||||
Cross-references: `spec/UseCaseCatalog.md` (UC-26–UC-67),
|
||||
`workplans/SHARD-WP-0002-federation-architecture.md` (T1–T16),
|
||||
`INTENT.md` (constraints), [[shard-wiki-auth-in-core-decision]].
|
||||
|
||||
@@ -211,10 +254,13 @@ Cross-references: `spec/UseCaseCatalog.md` (UC-26–UC-59),
|
||||
|
||||
## 8. Traceability
|
||||
|
||||
- Consolidates: nine deep dives + federation/origin research into one capability model.
|
||||
- Feeds: `SHARD-WP-0002` T11 (eleven-spectra vocabulary), T12 (page-model breadth), T13
|
||||
(history confirmed), T14 (three-mode attachment taxonomy), T15 (lossy fidelity report),
|
||||
and a **new T16** (addressing, content identity, dimensional/query navigation).
|
||||
- UC coverage extended in the workplan from UC-34–UC-43 to **UC-34–UC-59**.
|
||||
- Consolidates: all fourteen deep dives + federation/origin research into one capability
|
||||
model (v2).
|
||||
- Feeds: `SHARD-WP-0002` T11 (**thirteen-spectra** vocabulary, incl. content-opacity +
|
||||
merge-model), T12 (page-model breadth: computed metadata, identity≠placement, non-MD),
|
||||
T13 (history incl. CRDT-log = supplement), T14 (full attachment-mode taxonomy incl.
|
||||
CRDT-replica + P2P + interchange-mirror + local-REST), T15 (lossy incl. HTML), T16
|
||||
(addressing, content identity, identity≠placement, derived index, dimensional/query).
|
||||
- UC coverage extended in the workplan from UC-34–UC-59 to **UC-34–UC-67**.
|
||||
- No UCs added (synthesis only); no boundary changes (INTENT Stability Note untouched).
|
||||
</content>
|
||||
|
||||
Reference in New Issue
Block a user