shard-wiki/research/260614-xanadu-deep-dive/findings.md

# Findings — Project Xanadu: the docuverse, the EDL, and reference-not-copy

Date: 2026-06-14
Source kind: **conceptual / architectural prior art** (not a deployable engine, not a
candidate shard backend)
Lens: shard-wiki orchestration — projection, overlay, transclusion, provenance,
addressing, coordination journal

> Reading guide. Every previous deep dive (`xwiki`, `twiki`, `foswiki`) studied a
> *shippable engine* as a candidate **shard**. Xanadu is different: it is the
> **deepest conceptual ancestor** of shard-wiki's own model and is *not* a backend
> we would ever attach. It is studied here for its **mechanisms** —
> reference-not-copy documents, separated link tables, content-identity transclusion,
> stable fine-grained addressing — several of which shard-wiki already reinvents
> under different names (projection, overlay, union BackLinks, coordination journal).
> The job of this file is to (a) name what shard-wiki inherits, (b) flag what would
> be a **design bug** to inherit, and (c) extract use cases that are genuinely new
> versus the federation track already in the catalog.

Pairs with — and deliberately extends, does not repeat —
`research/260608-federation-concepts/findings.md` §3.3, which treats Xanadu only as a
six-row pattern table and (correctly) labels it "speculative design / pattern
language, not deployable federation." This dive goes underneath those patterns into
the actual data architecture (EDL, istream, enfilades, tumblers) because *that* is
where the resonance with shard-wiki's storage-neutral page model lives.

---

## 1. Origin and status

Project Xanadu (Ted Nelson, 1960– ; the term *hypertext* and *transclusion* are his)
is the original electronic-literature system: a global **docuverse** of permanent,
addressable documents joined by visible, two-way links. It famously never shipped at
scale; partial artifacts exist — **Udanax Green / Udanax Gold** (source released 1999
to establish prior art against patents), **XanaduSpace**, and **OpenXanadu** (2014, a
small working browser demo built on the EDL format). Treat all of these as
*reference designs*, not running software to integrate.

Why it matters to shard-wiki: Nelson's critique of the Web — **"one-way,
ever-breaking links and no management of version or contents"** — is precisely the
gap shard-wiki addresses for wikis. shard-wiki is, in effect, a *pragmatic, federated,
Git-backed, heterogeneous-backend* descendant of the Xanadu idea, scoped to
Markdown wiki pages and explicitly **refusing** Xanadu's one-global-store premise.

---

## 2. The headline insight — a document is a manifest of spans, not content

The single most shard-wiki-relevant Xanadu artifact is the **EDL (Edit Decision
List)**, also called a **xanadoc**:

- A xanadoc is a *connected document that contains no original content*. It is a file
  listing (a) **spans** — "portions of content to bring in" addressed in remote
  sources — and (b) **xanalinks** — tables saying what to connect to what.
- The client reads the EDL, **fetches each span from its source**, and **assembles**
  the document locally per the xanalinks. Content is referenced, never embedded.
- Structure is strictly separated: **spans first** (content references), **xanalinks
  after** (relationships). Content and link structure live apart.

This is *the same move* shard-wiki makes:

| Xanadu (xanadoc / EDL) | shard-wiki concept |
|------------------------|--------------------|
| Document = ordered list of span references, no embedded content | **Projection** — lazy, cache-like view assembled from shard storage, not a copy |
| Spans pulled from multiple remote sources at view time | **Union of pages** across heterogeneous shards |
| Xanalinks stored separately from content | **Overlay** — non-destructive edits/annotations as separate objects, "overlay before mutation" |
| "All media should be permanized and addressable" | **Coordination journal** + Git content-addressable storage |

shard-wiki's INTENT lines — *"prefer lazy projection over eager copying"* and *"union
without erasure"* — are the xanadoc principle restated for wikis.

---

## 3. Addressing and the storage substrate

Xanadu's machinery underneath the EDL:

- **Tumblers** (Miller / Gregory): a transfinite-number addressing scheme where one
  address simultaneously encodes **machine, author, document version, byte span, and
  links**. A single tumbler is a stable, fine-grained, hierarchical pointer into the
  whole docuverse.
- **istream (invariant stream):** a growing pool of *shareable content pieces*.
  Documents do not store text; they reference istream content by virtual address. A
  family of **enfilades** (tree structures using *dsps* = relative displacements and
  *wids* = ranges) maps **virtual addresses ↔ istream addresses** bidirectionally.
- **Spanfilade:** indexes which istream spans each document uses. Other filades:
  **granfilade** (storage across disks/network), **POOMfilade** (permutation-of-order
  matrix mapping document position → istream location).
- Implicit substrate: content pieces are **invariant** (append-only, never rewritten);
  versions are new arrangements of shared, permanent pieces. Nothing is deleted, so
  references never break.

shard-wiki mapping:

- **istream + invariance ≈ Git's content-addressable, immutable blob store**, and the
  **coordination journal** as the append-only record. shard-wiki gets "references
  never break" *for Git-native shards for free*; for non-Git shards it must
  approximate via the journal/projection cache.
- **Tumbler ≈ the still-open question of a portable, stable, fine-grained span
  address** that survives projection, overlay, and versioning across heterogeneous
  backends. shard-wiki has no such addressing scheme yet; this is the hardest part
  Xanadu solved on paper and the part that never shipped. (See §10, §11.)

---

## 4. Links and transclusion — separate, two-way, content-identity

- **Links are first-class objects stored apart from content** (the xanalinks tables),
  and are **visible and followable from all endpoints** (rule 7) — i.e. inherently
  **bidirectional**. Backlinks are not derived after the fact; they are the same
  object seen from the other end.
- **Transclusion** = Nelson's "the same content *knowably* in more than one place."
  The content piece **remembers its identity and can trace back to all its
  appearances**. This is the crucial delta from every weak modern form (server-side
  includes, HTML embeds, Roam block-refs): those *copy or cache*; Xanadu keeps one
  permanent instance that is **aware of where it is reused**.

shard-wiki mapping:

- **Separate link tables ≈ overlays and union BackLinks as first-class.** An overlay
  is shard-wiki's xanalink: a non-destructive object spanning content it does not own.
- **Content-identity, bidirectional transclusion** is *stronger* than UC-32 as
  currently written (which is path/span fetch with freshness). Xanadu says the
  *content itself* knows its appearances → enables **reverse transclusion** ("where is
  this paragraph used across all shards?") and **content-identity equivalence** ("these
  two pages in different shards are versions of the same content"). These are new
  capabilities, surfaced as UC-45 and UC-46 below.

---

## 5. Versioning and comparison — by span-set intersection

Because a document is a set of spans over an invariant pool, **comparing two documents
or two versions = intersecting their span-sets** (the spanfilade operation). Shared
spans are literally shared subtrees; differences fall out of the set arithmetic. The
same operation surfaces links between documents.

shard-wiki mapping: this is a **content-identity diff/merge** model. shard-wiki's
diff/merge capability (an adapter capability in the contract) is today implicitly
path/title- and text-based. Xanadu shows a path-independent alternative: detect that
page A in shard X and page B in shard Y are *the same or derived content* by span
overlap, **without relying on matching titles or paths**. This directly serves UC-27
(view parallel versions of equivalent pages) by giving an *equivalence-detection
mechanism* rather than assuming naming conventions align across sovereign shards.

---

## 6. Rights and economics — transcopyright, micropayment, implicit permission

Three of the 17 rules are an economic/rights layer:

- Rule 8: **publication grants implicit permission to link/transclude.**
- Rule 9: **granular royalty / micropayment** on any accessed portion.
- **Transcopyright:** pre-granted permission for virtual republication *by reference*,
  with the attribution chain to origin preserved automatically because the content is
  transcluded, not copied.

shard-wiki mapping — handle with the **mechanism-over-policy** rule:

- The *mechanism* shard-wiki should provide: a reference (overlay/transclusion) that
  **carries provenance and reuse terms with it**, so attribution and permission travel
  with the span. This strengthens UC-29 (remix with portable attribution).
- The *policy* (whether reuse requires permission, whether anything is metered) must
  **not be hard-coded**. Xanadu baked one global economic policy into the substrate;
  shard-wiki keeps editorial/economic policy configurable. Baking in a payments or
  permission model would be a design-bug per INTENT.
- This intersects the **settled authz-in-core / authn-delegated decision**
  ([[shard-wiki-auth-in-core-decision]]): "publication grants implicit linking
  permission" is exactly a *policy* that an information space might or might not adopt
  on its L0→L4 ladder. shard-wiki must be able to *represent* the permission on a
  transclusion; it must not *assume* Xanadu's answer.

---

## 7. Xanadu through shard-wiki's lens — it is architecture, not a shard

Unlike the engine dives, there is **no capability profile to fill in** — Xanadu is not
a backend you attach. The useful framing is the inverse: *which shard-wiki primitives
are Xanadu mechanisms in disguise?*

| shard-wiki primitive | Xanadu mechanism | Inheritance verdict |
|----------------------|------------------|---------------------|
| Projection (lazy view, no copy) | xanadoc / EDL span assembly | **Inherit** — same idea, scope to wiki pages |
| Overlay (non-destructive edit) | xanalinks as separate objects | **Inherit** — overlays are xanalinks |
| Union BackLinks | two-way links visible from all endpoints | **Inherit** — generalize to sub-page spans |
| Transclusion (UC-32) | content-identity, content-aware reuse | **Inherit, strengthen** — add reverse + equivalence |
| Coordination journal | invariant istream / permanent storage | **Inherit** (Git gives this for Git-native shards) |
| Stable span address | tumblers | **Aspire** — hard, partly unshippable; degrade gracefully |
| Editorial/economic policy | transcopyright + micropayment | **Reject as substrate** — keep configurable |
| One global docuverse | the docuverse premise | **Reject** — see §8.2 |

---

## 8. Mapping to shard-wiki INTENT (compare, do not equate)

### 8.1 Reinforcements

- **Reference-not-copy** is the core of both. Xanadu validates shard-wiki's
  "lazy projection over eager copying" and "union without erasure" as a coherent,
  decades-deep design lineage, not an ad-hoc preference.
- **Provenance is structural, not decorative.** In Xanadu, knowing where content came
  from and where it is reused is *built into the storage model*. shard-wiki's "never
  hide authorship, conflicts, freshness, backend limits" is the same commitment.
- **Links/edits as separate objects** validates overlays-before-mutation and
  first-class BackLinks.

### 8.2 Deliberate divergences (design bugs if conflated)

1. **No single docuverse / no universal address space.** Xanadu requires one global
   permanent store, universal addressing, and its own published client–server
   protocol (rules 12, 17). shard-wiki's whole reason to exist is the opposite:
   **federate heterogeneous, sovereign backends** (Git repos, Gitea, Obsidian,
   WebDAV, Coulomb), each keeping its own storage, history, identity, and limits.
   Adopting Xanadu's universality would violate **shard sovereignty** and **graceful
   degradation**. shard-wiki's addressing must tolerate backends that *cannot* offer
   stable span addresses, treating fine-grained addressing as an **adapter
   capability**, not a precondition.
2. **One canonical content instance vs. parallel divergent versions.** Xanadu
   transclusion centers a *single source of truth*. shard-wiki must support
   **equivalent-but-divergent** pages across shards with conflicts *visible* (UC-27),
   not collapsed into one instance. Content-identity is a *detection* tool here, not a
   mandate to unify.
3. **Baked-in economic/permission policy.** See §6 — reject as substrate, keep as
   configurable policy.
4. **Permanence as a hard requirement.** Xanadu forbids deletion globally. shard-wiki
   cannot impose that on sovereign shards; permanence holds for the **coordination
   journal** and Git-native shards, and is *approximated* (cache/projection/snapshot)
   for backends that delete. This is graceful degradation, not a weaker promise.

### 8.3 What Xanadu teaches that shard-wiki should not lose

- A page view can legitimately be a **composition manifest** (a list of references),
  not a file. Build the page model so a page *can* be authored as references into
  other shards (UC-44), even if the common case is a plain Markdown file.
- **Content identity** (not path/title) is the durable basis for transclusion,
  equivalence, and reverse lookup across sovereign shards (UC-45, UC-46). Lean on
  Git blob hashes where available; define a content-fingerprint fallback elsewhere.
- Getting **stable fine-grained addressing** right is the hard, valuable, historically
  *unshippable* part. Scope it as an adapter capability with explicit degradation
  rather than promising tumbler-grade universality.

---

## 9. Use-case seeds → catalog (promoted 2026-06-14)

Last existing UC is **UC-43**. New UCs **UC-44–UC-46** added; several existing UCs
enriched (no new scenario, stronger mechanism/notes).

| Seed | Catalog action |
|------|----------------|
| **Compose-by-reference page** — author a page whose body is an ordered list of spans pulled from multiple shards, stored as a manifest (xanadoc/EDL), not a copy | **UC-44 (new)** |
| **Reverse transclusion** — find every page/shard where a given span (paragraph/section) appears or is transcluded | **UC-45 (new)** |
| **Content-identity equivalence** — detect that two pages in different shards are the same or derived content via span/content overlap, without matching titles or paths | **UC-46 (new)** |
| Content-identity, content-aware (bidirectional) transclusion | **enriches UC-32** |
| Equivalence detection mechanism for parallel versions | **enriches UC-27** |
| Reference carries provenance + reuse terms (transcopyright as representable policy) | **enriches UC-29**; links [[shard-wiki-auth-in-core-decision]] |
| Provenance: content remembers its appearances | **enriches UC-24** |

---

## 10. Adapter-contract / architecture notes for SHARD-WP-0002

Logged as architecture (no UC):

- **Fine-grained span addressing is an adapter capability**, not a core assumption.
  The shard adapter contract should model whether a shard can mint a *stable address
  for a sub-page span* that survives edits/versions (tumbler-grade), down through
  whole-page-only, down to path-only. Transclusion/overlay capabilities depend on it.
- **Content identity** should be a contract-level concept: a shard advertises how it
  fingerprints content (Git blob hash, normalized-text hash, none). UC-45/UC-46 and
  cross-shard diff/merge consume it.
- **Composition manifests** (UC-44) imply the wiki page model must permit a page whose
  canonical form is a reference list. This is an INTENT-level page-model decision —
  flag for the page-model spec, not just the adapter contract.
- Reuse-terms metadata on a reference (UC-29 / §6) is **policy data the core carries
  but does not interpret** — consistent with mechanism-over-policy and the L0→L4
  authz ladder.

---

## 11. Open questions (for spec / workplans)

1. What is shard-wiki's **portable span address**? Git blob+range works for Git-native
   shards; what is the fallback for Obsidian/WebDAV/Gitea-wiki, and how does it
   survive a shard's storage swap (cf. UC-43 Foswiki RCS↔PlainFile)?
2. Is **compose-by-reference (UC-44)** core orchestrator, adapter-provided, or
   reference-UI — and is it MVP or deferred with UC-32?
3. Does **content-identity equivalence (UC-46)** belong in core (cross-shard union
   logic) or as an adapter-provided index? How expensive is span-set intersection at
   wiki scale without enfilades?
4. How far do we take **reverse transclusion (UC-45)** — exact span identity only, or
   fuzzy/derived-content tracking? The latter is research-grade.
5. Where does **reuse-terms/transcopyright metadata** sit on the L0→L4 ladder, and
   does any tier ever *enforce* it, or is it always advisory provenance?

---

## 12. Sources

| Source | Used for |
|--------|----------|
| Wikipedia — Project Xanadu (https://en.wikipedia.org/wiki/Project_Xanadu) | 17 rules, tumblers, history, Web critique |
| xanadu.com — The Edit Decision List / Xanadoc File (https://xanadu.com/xuEDL.html) | EDL/xanadoc structure: spans + xanalinks, client assembly |
| Wikipedia — Enfilade (Xanadu) (https://en.wikipedia.org/wiki/Enfilade_(Xanadu)) | istream, enfilades (dsp/wid), spanfilade, granfilade, POOMfilade, version comparison by span-set intersection |
| Wikipedia — Transclusion (https://en.wikipedia.org/wiki/Transclusion) | "same content knowably in more than one place," content-aware reuse, transcopyright, micropayment vs. weak modern forms |
| Maggie Appleton — Xanadu Patterns (https://maggieappleton.com/xanadu-patterns) | pattern naming: visible links, parallel documents, transpointing windows, modular blocks, stable addresses, annotation |

Cross-references: `research/260608-federation-concepts/findings.md` §3.3 (prior
surface treatment), `spec/UseCaseCatalog.md` (UC-24, UC-27, UC-29, UC-32),
`workplans/SHARD-WP-0002-federation-architecture.md` (adapter contract).

---

## 13. Traceability

- New UCs: **UC-44, UC-45, UC-46** → `spec/UseCaseCatalog.md` (section F, federation).
- Enriched UCs: **UC-24, UC-27, UC-29, UC-32**.
- Architecture (no UC): span-addressing capability, content-identity, composition
  manifest, reuse-terms metadata → `SHARD-WP-0002`.
- Decision link: transcopyright-as-policy → [[shard-wiki-auth-in-core-decision]].
</content>
</invoke>