synthesis: shard spectrum across nine systems; fold into SHARD-WP-0002 (T11 eleven spectra, new T16)

Cross-dive synthesis (research/260614-shard-spectrum-synthesis/) reading
the two Nelson conceptual systems, four engines, and three modern tools
across each other: a shard family matrix and eleven capability spectra
(addressing, content identity, structure, history, native query,
translation, attachment mode, operational envelope, access grant, write
granularity, content types) — positions anchored at both ends by a real
system, federation ops degrading by position. Through-lines:
files-canonical/index-derived wins; fine-grained addressing is adoptable;
transclusion=clone=embed is one primitive; structure/history federate iff
in-text; attach mode is a per-binding choice; Notion proves the platform
can enforce no-silent-mutation.

Folded into SHARD-WP-0002: T11 reframed around the eleven spectra; T12
page model stretched four ways (prose + typed records + non-Markdown
assets + query-defined); T13 adds Notion supplement; T14 rewritten as the
three-mode attachment taxonomy (file-store / in-engine-host /
external-API); T15 adds lossy-with-fidelity-report; new T16 (addressing,
content identity, dimensional/query navigation). UC coverage UC-34-43 ->
UC-34-59. Workplan now 16 tasks. No new UCs (synthesis only).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

research/260614-shard-spectrum-synthesis/README.md

@@ -0,0 +1,47 @@
+# 260614 — Shard spectrum synthesis (one capability model across nine systems)
+
+Date: 2026-06-14
+
+## What this is
+
+A **synthesis** (no new external research) that reads the nine studied systems *across*
+each other and distills them into a single comparative model feeding the **shard adapter
+contract** (`SHARD-WP-0002`).
+
+The nine: two conceptual ancestors (**Xanadu**, **ZigZag**), four engines (**XWiki,
+TWiki, Foswiki** + the wiki-engines landscape), and three modern tools (**Roam,
+Obsidian, Notion**), against the federation/origin research.
+
+Centerpieces:
+- **The shard family matrix** — every candidate backend × {substrate, attach mode,
+  addressing, structure, history, query, transclusion, →Markdown, write granularity},
+  with Xanadu/ZigZag as the ideal anchors.
+- **The eleven capability spectra** — the claim that the adapter contract should model
+  *positions on spectra* (addressing, content identity, structure, history, query,
+  translation, attachment mode, operational envelope, access grant, write granularity,
+  content types), each anchored at both ends by a real system, with federation ops
+  degrading by position.
+- **UC-44–UC-59 → workplan task mapping**, including one genuinely new thread (T16:
+  addressing, content identity, dimensional/query navigation).
+
+## Contents
+
+| Path | Role |
+|------|------|
+| `findings.md` | Family matrix, the eleven spectra, cross-cutting through-lines, UC→task fold-in, recommendations/decisions, escalated open questions |
+
+## Status
+
+Synthesis complete. No new use cases (consolidation only). Feeds `SHARD-WP-0002`: T11
+reframed around the eleven spectra; T12 page-model breadth (prose + typed records +
+non-Markdown assets + query-defined); T13 history confirmed; T14 three-mode attachment
+taxonomy; T15 lossy-with-fidelity-report; **new T16** (addressing, content identity,
+dimensional/query navigation). UC coverage in the workplan extended UC-34–UC-43 →
+UC-34–UC-59.
+
+**Through-lines recorded:** files-canonical/index-derived is the winning architecture;
+fine-grained addressing is real and adoptable (adopt native, fingerprint otherwise);
+transclusion ⇄ clone ⇄ embed is one primitive; structure/history federate iff in text;
+attach mode is a per-binding capability-gated choice; Notion proves the platform can
+*enforce* no-silent-mutation via scoped grants.
+</content>

research/260614-shard-spectrum-synthesis/findings.md

@@ -0,0 +1,220 @@
+# Synthesis — the shard spectrum: one capability model across nine systems
+
+Date: 2026-06-14
+Source kind: **synthesis** — consolidates the engine and tool deep dives plus the two
+Nelson conceptual systems into a single comparative model that feeds the **shard adapter
+contract** (`SHARD-WP-0002` T11–T16)
+Lens: shard-wiki — what a backend must expose to participate, expressed as *spectra* of
+capability rather than a yes/no checklist
+
+> Purpose. Nine systems have been studied: two conceptual ancestors (**Xanadu**,
+> **ZigZag**), four engines (**XWiki, TWiki, Foswiki**, + the wiki-engines landscape),
+> and three modern tools (**Roam, Obsidian, Notion**), against the federation and
+> origin research (federation-concepts, c2, yawex). They were studied one at a time;
+> this document reads them *across* each other. The payoff is a small set of
+> **capability spectra** — addressing, structure, history, query, translation,
+> attachment mode, operational envelope, access grant — each anchored at both ends by a
+> real system. That spectrum *is* the adapter contract's design surface. It also folds
+> the post-engine use cases (**UC-44–UC-59**) into the existing `SHARD-WP-0002` tasks.
+
+Inputs: `research/260608-federation-concepts`, `research/260608-wikiengines-overview`,
+`research/260613-{xwiki,twiki,foswiki}-deep-dive`,
+`research/260614-{xanadu,zigzag,roam,obsidian,notion}-deep-dive`. Output target:
+`spec/TechnicalSpecificationDocument.md` (adapter contract) via `SHARD-WP-0002`.
+
+---
+
+## 1. The shard family matrix
+
+Candidate-shard backends across the dimensions that matter to the contract. (Xanadu and
+ZigZag are *not* shards — they are the conceptual ideals each column aspires to; listed
+last as reference anchors.)
+
+| Backend | Substrate | Attach mode | Addressing | Structure | History | Native query | Transclusion | →Markdown | Write granularity |
+|---------|-----------|-------------|-----------|-----------|---------|--------------|-------------|-----------|-------------------|
+| **Git folder / repo** | files | file-store | path (+ commit) | flat MD + frontmatter | **git-native** | no | no | native | per-file |
+| **Obsidian vault** | files | file-store **or** in-app plugin | path + **in-file `^id`** | frontmatter (in-file) | none (Git plugin) | plugin (Dataview) | in-file `![[ ]]` | native (OFM) | per-file |
+| **TWiki** | files + RCS | file-store / API | path | `%META%` in-file | **open file (RCS)** | no | include macro | lossless (TML) | per-topic |
+| **Foswiki** | pluggable store | file-store / API | path | `%META%` in-file, N records | open file / PlainFile | no | include macro | lossless (TML) | per-topic |
+| **XWiki** | DB (Hibernate) | in-engine host / REST | path | **XObjects/XClass** | internal (`xwikircs`) | yes (XWQL) | macro | engine syntax | per-page / per-object |
+| **Roam** | client DataScript | **in-app host only** | **store UUID** | `key::` attrs | internal (txn log) | **yes (Datalog)** | **block embed** | (Roam MD) | **per-block** |
+| **Notion** | hosted Postgres | **external API only** | **store UUID** | **DB schema + relations + rollups** | internal, not portable | **yes (DB query)** | synced block | **lossy** | **per-block** |
+| **TiddlyWiki** | single file | file-store | path | typed tiddlers | none | no | transclusion | varies | **whole-file** |
+| **MediaWiki / Confluence** | DB | API | path | wikitext / macros | internal-only | limited | templates | lossy | per-page |
+| — *Xanadu (ideal)* | permascroll | — | **tumbler (span)** | spans + links | permanent | — | **content-identity** | — | span |
+| — *ZigZag (ideal)* | cells/dims | — | cell id | **N dimensions** | — | dimension walk | clone | — | cell |
+
+Reading the matrix top to bottom is reading shard-wiki's difficulty gradient: **Git
+folder → Obsidian** are the friction-free, INTENT-native cases; **TWiki/Foswiki** add
+syntax translation but stay file/diffable; **XWiki/Roam/Notion** add DB structure, store
+addressing, and (for Roam/Notion) non-file substrates; **Notion** is the hardest
+(closed, hosted, lossy, rate-limited). Xanadu/ZigZag mark where the *ideals* live.
+
+---
+
+## 2. The capability spectra (the contract's real shape)
+
+Each capability is **not boolean** — it is a position on a spectrum with a real anchor at
+each end. This is the core synthesis claim: the adapter contract should model *positions*,
+and federation operations should degrade by position.
+
+1. **Addressing granularity** — `none → whole-page(path) → in-file span(Obsidian ^id) →
+   store-minted span(Roam/Notion UUID) → portable tumbler(Xanadu ideal)`. (UC-51, UC-44/45.)
+2. **Content identity** — `none → path/title match → content fingerprint(hash) →
+   span-set/equivalence(Xanadu)`. Drives equivalence + reverse-transclusion. (UC-46, UC-27.)
+3. **Structure** — `flat Markdown → in-file frontmatter → in-file %META% → DB
+   objects(XWiki) → DB schema+relations+rollups(Notion)`. The git-diffable forms federate
+   easily; DB-locked forms need sidecar + fidelity reporting. (UC-34, UC-39, UC-58.)
+4. **History** — `none → internal-only/not-portable(Notion/Confluence) → open file
+   format(TWiki RCS) → git-native(Git/Obsidian-with-Git)`. Internal ⇒ *supplement*;
+   open-file ⇒ *import*; git-native ⇒ *adopt*. (UC-36, UC-41.)
+5. **Native query** — `none → text search → datalog(Roam) → DB query(Notion/XWiki)`.
+   Where present, **delegate** derived-view computation; else compute over the projection.
+   (UC-52, UC-05, UC-54.)
+6. **Translation to Markdown** — `native → lossless round-trip(Foswiki TML↔HTML) →
+   lossy-with-fidelity-report(Notion)`. Lossless ⇒ writable; lossy ⇒ read/projection floor
+   + visible fidelity loss. (UC-42, UC-59, UC-03.)
+7. **Attachment mode** — `file-store direct(Obsidian/TWiki) → in-engine hosted
+   adapter(Roam/XWiki) → external-API only(Notion)`. A backend may offer **more than one**
+   (Obsidian offers both; TWiki file-or-API). (UC-40, UC-38, UC-50, UC-57.)
+8. **Operational envelope** — `local/unbounded → rate-limited+eventually-consistent+
+   paginated(Notion ~3 rps)`. Determines whether projection can be live or must be
+   cache/poll/webhook. (UC-57, UC-31.)
+9. **Access grant** — `open(L0) → token → OAuth scoped + revocable(Notion)`. The backend
+   may *enforce* no-silent-mutation (Notion scoped grants). Ties the authz-in-core ladder.
+   (UC-57, [[shard-wiki-auth-in-core-decision]].)
+10. **Write granularity** — `whole-file(TiddlyWiki) → per-page(Git/Obsidian/TWiki) →
+    per-block(Roam/Notion)`. Sets overlay/patch/lock/conflict scope. (UC-35.)
+11. **Content types** — `Markdown-only → typed records(XWiki/Notion DB) → non-Markdown
+    assets(Excalidraw/Canvas/attachments)`. The page model must stretch to all three.
+    (UC-55, UC-58.)
+
+Design consequence: **T11's capability vocabulary should be these eleven spectra**, not
+the original flat `read/write/diff/...` list. The flat verbs remain as the *operations*;
+the spectra are the *profile* that says how well each verb is supported and how it
+degrades.
+
+---
+
+## 3. Cross-cutting findings (the through-lines)
+
+- **Files-canonical, index-derived is the winning architecture.** Obsidian's
+  MetadataCache, Git's working tree, and shard-wiki's own projection model all agree:
+  the link graph / backlinks / structured index are **derived and rebuildable**, never a
+  second source of truth. Roam/Notion invert this (DB canonical) and pay for it in
+  portability. shard-wiki keeps files+journal canonical; everything else is projection.
+  (Reinforces UC-05, UC-17–20; INTENT.)
+- **Fine-grained addressing is real and adoptable** — Roam (`:block/uid`) and Notion
+  (UUID) ship it store-minted; Obsidian ships it in-file (`^id`). The Xanadu tumbler is
+  the unreached ideal (portable, transfinite). shard-wiki should **adopt native span IDs
+  where present, fingerprint where not** (UC-51 + UC-46), and treat a portable span
+  address as the open hard problem (Xanadu §11 Q1).
+- **Transclusion ⇄ clone ⇄ embed is one primitive.** Xanadu transclusion, ZigZag clones,
+  Roam block embeds, Obsidian `![[ ]]`, Notion synced blocks are the same idea at
+  different fidelities. A single internal "reference-not-copy" primitive over an
+  addressable union serves UC-32/44/45/51. (Convergence first noted in the ZigZag dive.)
+- **Structure and history federate iff they live in text.** The sharpest recurring
+  engine insight: `%META%`-in-file (TWiki/Foswiki) and frontmatter (Obsidian) diff and
+  travel; XObjects and Notion DBs lock in. The contract should *prefer* in-text
+  structure and *tolerate* DB structure via sidecar + fidelity report.
+- **Three attachment modes, and a backend may span them.** The clean taxonomy
+  (file-store / in-engine-host / external-API) plus the observation that one backend can
+  offer several (Obsidian: file *or* plugin; TWiki: file *or* API) means **attach mode is
+  a per-binding choice, capability-gated**, not a fixed property of the backend.
+- **Notion proves the platform can enforce INTENT.** Scoped, revocable grants are
+  no-silent-mutation made mechanical — evidence the principle is implementable, and a
+  model for how shard-wiki represents partial/consented visibility.
+- **The page model must stretch four ways at once:** prose Markdown, typed records
+  (with N-per-page and inter-record relations), non-Markdown assets, and
+  reference/query-defined pages. This is the heaviest single demand on T12.
+
+---
+
+## 4. How the post-engine use cases fold into the workplan
+
+UC-44–UC-59 (from the five 260614 dives) map onto the existing adapter-contract tasks,
+with one genuinely new thread needing a home (T16):
+
+| UCs | Theme | Lands in |
+|-----|-------|----------|
+| UC-35, UC-50, UC-53, UC-57 | attachment mode + operational envelope + write granularity | **T11** (capabilities) + **T14** (binding) |
+| UC-34, UC-39, UC-55, UC-58 | structured/typed payload, non-Markdown assets, DB schema+relations | **T12** (page model) |
+| UC-36 | internal-only history (Notion) = supplement | **T13** (history) |
+| UC-42, UC-59 | translation: lossless vs lossy-with-fidelity-report | **T15** (translation) |
+| UC-31 | webhooks / push-vs-poll under eventual consistency | **T6** / **T11** envelope |
+| UC-57 §6 | scoped, revocable grant; no silent mutation | **T11** access-grant + authz decision |
+| **UC-44, UC-45, UC-46, UC-51** | span addressing, content identity, transclusion-as-reference | **T8** + **new T16** |
+| **UC-47, UC-48, UC-52, UC-54** | dimensional navigation, query delegation, query-defined pages | **T5/T10** + **new T16** |
+
+**New thread (T16): Addressing, content identity, and dimensional/query navigation.** No
+existing task owns (a) the portable span-address scheme + content-identity model that
+transclusion/overlay/reverse-lookup need, nor (b) the dimensional/query view model
+(ZigZag dimensions, Roam/Notion query, Dataview query-defined pages). These are the
+Nelson-ideals-meet-modern-tools thread and warrant their own design task.
+
+---
+
+## 5. Recommendations (decisions to record under SHARD-WP-0002)
+
+1. **Model capabilities as the eleven spectra (§2), not flat verbs.** Keep verbs as
+   operations; add the spectra as the profile. (T11.)
+2. **Adopt native span IDs; fingerprint otherwise; keep portable-span-address open.**
+   (T16, UC-51/46.)
+3. **One reference-not-copy primitive** for transclusion/clone/embed over the addressable
+   union. (T16/T8, UC-32/44/45.)
+4. **Prefer in-text structure; tolerate DB structure via sidecar + fidelity report.**
+   (T12/T15, UC-34/58/59.)
+5. **Attach mode is a per-binding, capability-gated choice** across the three modes; a
+   backend may offer several. (T14, UC-40/38/50/57.)
+6. **Add operational-envelope and access-grant to the profile** as first-class fields.
+   (T11, UC-57.)
+7. **Delegate derived views to native query engines where present; else compute over the
+   projection.** (T16/T5, UC-52/54.)
+8. **Keep files + coordination journal canonical; all indexes/projections derived and
+   rebuildable.** (Architecture invariant, INTENT.)
+
+These honor INTENT: mechanism over policy (spectra not hard-coded behaviors), union
+without erasure (fidelity reporting makes loss visible), graceful degradation (every
+capability has a read/projection floor), no silent mutation (access-grant + overlay),
+shard sovereignty (no backend forced to change substrate), Markdown-first/backend-neutral
+(in-text preferred, DB tolerated).
+
+---
+
+## 6. Open questions escalated by the synthesis
+
+1. **Portable span address** — still unsolved across heterogeneous backends (Xanadu's
+   unbuilt part). Wrap native IDs in a shard-scoped address? (T16.)
+2. **External-API write-through at scale** — is Notion-class write viable under ~3 rps,
+   or read/projection/overlay/backup by default? (T14, UC-57.)
+3. **Inter-record relations in the union** (UC-58) — typed links in the link graph, a
+   separate relation index (ZigZag many-to-many), or both? (T16/T5.)
+4. **Page model breadth** — can one model carry prose + typed records + non-Markdown
+   assets + query-defined pages without becoming incoherent? (T12.)
+5. **Dimensional navigation surface** — public API/UI paradigm or internal organizing
+   concept? (T16, UC-47/48.)
+
+---
+
+## 7. Sources
+
+This is a synthesis; primary sources are the nine dives' `findings.md` files plus
+`research/260608-federation-concepts`, `research/260608-wikiengines-overview`,
+`research/260608-c2-wiki-origins`, `research/260608-yawex-prior-art`. No new external
+research was performed.
+
+Cross-references: `spec/UseCaseCatalog.md` (UC-26–UC-59),
+`workplans/SHARD-WP-0002-federation-architecture.md` (T1–T16),
+`INTENT.md` (constraints), [[shard-wiki-auth-in-core-decision]].
+
+---
+
+## 8. Traceability
+
+- Consolidates: nine deep dives + federation/origin research into one capability model.
+- Feeds: `SHARD-WP-0002` T11 (eleven-spectra vocabulary), T12 (page-model breadth), T13
+  (history confirmed), T14 (three-mode attachment taxonomy), T15 (lossy fidelity report),
+  and a **new T16** (addressing, content identity, dimensional/query navigation).
+- UC coverage extended in the workplan from UC-34–UC-43 to **UC-34–UC-59**.
+- No UCs added (synthesis only); no boundary changes (INTENT Stability Note untouched).
+</content>

research/README.md

@@ -21,4 +21,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
 | 2026-06-14 | `260614-zigzag-deep-dive/` | ZigZag/zzstructure — information space as orthogonal dimensions; UC-47/48/49 |
 | 2026-06-14 | `260614-roam-deep-dive/` | Roam Research — block-graph DataScript DB, transclusion, datalog, Roam Depot extension API; UC-50/51/52 |
 | 2026-06-14 | `260614-obsidian-deep-dive/` | Obsidian — file-over-app vaults, plugin API, ecosystem-popularity → UC signal; UC-53/54/55/56 |
-| 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 |
+| 2026-06-14 | `260614-notion-deep-dive/` | Notion — closed block-DB SaaS, external REST API only, database-as-pages; UC-57/58/59 |
+| 2026-06-14 | `260614-shard-spectrum-synthesis/` | Synthesis — shard family matrix + eleven capability spectra across nine systems; feeds SHARD-WP-0002 T11–T16 |