research: federation concepts for unified information spaces

Document Federated Wiki mechanics (fork, journal, inverted browser model), Caulfield lifecycle concepts, and comparison with git-backed wikis, ActivityPub/XWiki, Xanadu patterns, and yawex REMOTE/VIRTUAL. Map findings to shard-wiki INTENT without equating fedwiki to orchestration.
2026-06-08 14:43:13 +02:00
parent 5d8608aeae
commit 3160648b61
4 changed files with 439 additions and 2 deletions
--- a/SCOPE.md
+++ b/SCOPE.md
@@ -19,7 +19,7 @@ Learnings update both SCOPE and INTENT where necessary.
 |-------|-------|
 | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
 | Intent | `INTENT.md` established; authorization-in-core amendments drafted |
-| Research | yawex prior art; c2 wiki origins (`research/260608-*/`) |
+| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`) |
 | Demand | NetKingdom integration asks captured, not yet negotiated |
 | Spec | Architecture blueprint drafted; UseCaseCatalog 25 UCs from research; PRD/TSD scaffolds |
 | Work | `SHARD-WP-0001` active — 6 design tasks, all todo |
--- a/research/260608-federation-concepts/README.md
+++ b/research/260608-federation-concepts/README.md
@@ -0,0 +1,27 @@
+# 260608 — Federation concepts for unified information spaces
+
+Date: 2026-06-08
+
+## What this is
+
+Research into **federation models** for decentralized but unified knowledge
+spaces — with emphasis on **Federated Wiki** (Ward Cunningham, ~2011+) and
+comparable approaches (git-backed wikis, ActivityPub, Xanadu patterns, yawex
+REMOTE/VIRTUAL).
+
+Complements `research/260608-c2-wiki-origins/` (pre-federation wiki culture)
+and `research/260608-yawex-prior-art/` (Perl engine resolution states).
+
+Focus: terms, architectural patterns, and **mapping to `shard-wiki` INTENT**
+— not implementation specification.
+
+## Contents
+
+| Path | Role |
+|------|------|
+| `findings.md` | Glossary, model comparison, shard-wiki mapping, open questions, sources |
+
+## Status
+
+Initial exploration complete. Findings may inform `spec/` and future
+workplans; not yet promoted to specification.
--- a/research/260608-federation-concepts/findings.md
+++ b/research/260608-federation-concepts/findings.md
@@ -0,0 +1,408 @@
+# Findings — Federation concepts for decentralized, unified information spaces
+
+Date: 2026-06-08 · Status: research draft
+
+Scope: federation models that let independently stored knowledge participate in
+a **joined view** without erasing provenance, sovereignty, or backend
+differences. Primary anchor: **Federated Wiki** and Ward Cunningham's
+post-c2 inversion. Secondary: git-backed wikis, ActivityPub wiki extensions,
+Xanadu hypertext patterns, and yawex multi-wiki resolution.
+
+**Complements:** `research/260608-c2-wiki-origins/` (federation explicitly
+deferred there) and `research/260608-yawex-prior-art/` (REMOTE/VIRTUAL as
+adapter foreshadowing).
+
+---
+
+## 1. Why federation (the problem statement)
+
+Centralized wikis concentrate **meaning** and **records** on one server:
+
+| Problem | Manifestation |
+|---------|---------------|
+| **Ownership** | Contributors build on someone else's host; site shutdown = loss risk |
+| **Consensus pressure** | Early edits must survive notability/deletion fights (Wikipedia model) |
+| **Heat death** | Bounded communities exhaust their charter; maintenance mode → colony collapse |
+| **Reuse friction** | Copy-paste HTML/DB content across sites is slow, lossy, attribution-prone |
+| **Backend lock-in** | One engine, one storage model, one permission model |
+
+Federation responses split along two axes:
+
+1. **Where records live** — per-user site, per-instance repo, personal pod, mirrored git clone.
+2. **How unity is achieved** — fork + spread, git push/pull, protocol activities, projection/cache, orchestrated union.
+
+`shard-wiki` (per `INTENT.md`) targets axis (2) via an **orchestration layer**
+over heterogeneous **shards**, with Git as coordination substrate — not a
+single federation mechanism.
+
+---
+
+## 2. Federated Wiki (Smallest Federated Wiki / SFW)
+
+### 2.1 Historical anchor
+
+| Fact | Detail |
+|------|--------|
+| Origin | Ward Cunningham; launched at IndieWebCamp 2011 |
+| Motivation | Regret that c2's 35,000+ pages lived on Ward's server, not contributors' |
+| c2 migration | WikiWikiWeb moved to Federated Wiki ~2015 after read-only period |
+| GitHub lineage | Fork button inspired by GitHub; "radical code sharing" as model |
+| Current code | [github.com/fedwiki/wiki](https://github.com/fedwiki/wiki) (CoffeeScript; active releases through 2025) |
+
+Ward's framing (Wired, 2012): the wiki's radical idea was an **edit button on
+every page**; federated wiki's radical idea is a **fork button on every page**.
+
+### 2.2 Inverted architecture
+
+Traditional wiki vs federated wiki (Caulfield, NWACC 2014, summarizing Ward):
+
+```
+Traditional:  many people → one server → server owns records → consensus on server
+Federated:    many people → many servers → each owns records → browser composes union
+```
+
+**Meaning is made in the browser.** The client pulls JSON page records from
+multiple origins (CORS) and presents them as one navigable space. Consensus
+emerges through **which versions spread** across the network, not through a
+single server arbiter.
+
+### 2.3 Page model (JSON + journal)
+
+From [viki.wiki JSON schema](https://viki.wiki/json-schema.html):
+
+```
+page = { title, story, journal }
+story = [ item ]           # paragraph-like plugin items
+journal = [ action ]       # edit history that reconstructs story
+action.type = create|add|move|edit|remove|fork
+action.site = origin host  # present when content came from elsewhere
+```
+
+Key properties:
+
+| Property | Implication |
+|----------|-------------|
+| **Data files, not DB render** | Fork copies JSON, not scraped HTML — reuse stays structured |
+| **JSON items, not HTML** | Target site re-renders via plugins; display differences don't break remix |
+| **Journal travels with page** | Attribution/history portable; no separate blame log |
+| **Plugin architecture** | Item `type` selects renderer; missing plugins surfaced to user |
+| **Fork action** | Whole-page copy from remote site into local site with provenance |
+| **Item IDs preserved** | Stable anchors through edits; aliasing guards duplicate IDs |
+
+### 2.4 Interaction patterns
+
+| Pattern | Behavior |
+|---------|----------|
+| **Fork** | Copy remote page to your site; edit locally; original owner may merge back |
+| **Drag-and-drop fork** | Cross-site page transfer in ~15 seconds (OER case) |
+| **Chorus of voices** | Multiple versions of "same" topic coexist; linked, not merged by default |
+| **Neighborhood / river** | Discovery via recent changes across federated sites (e.g. fedwikiriver.com) |
+| **Happening** | Time-bounded collaborative event on a topic-specific subdomain/site |
+
+### 2.5 Mike Caulfield — lifecycle and pedagogy
+
+Caulfield articulated fedwiki's social model beyond Ward's mechanics:
+
+**Kinneavy triangle / knowledge lifecycle** (composition theory):
+
+| Phase | Mode | Typical medium | Fedwiki fit |
+|-------|------|----------------|-------------|
+| **I** | Personal capture | Notes, journal | Each person's site |
+| **You** | Dialogic | Conversation, annotation | Fork chains, cross-site references |
+| **It** | Expository | Stable shared article | Spread of refined forks |
+
+Fedwiki supports **I → You → It** in one system; centralized wikis force **It**
+(consensus) from the first edit.
+
+**Bounded conversations / expected heat death** (2015):
+
+- Wiki sites are **bounded conversations**, not permanent monuments.
+- Sites are **expected to die**; valuable material is **forked into the next happening**.
+- **Reverse bit-rot** — Smalltalk-era term Ward revived: objects/pages improve each
+  time reused across systems (refactor on fork), defying entropy of abandoned forums.
+- Contrast: colony collapse on centralized forums (Blue Hampshire example) vs
+  humane site retirement with selective carry-forward.
+
+**OER reuse** (2015): WordPress copy-paste ~10–15 min/page vs fedwiki fork ~15 sec;
+bottleneck is software, not human willingness to remix.
+
+**Dissent as feature** (Ogden via Wired): Wikipedia forces one perspective;
+fedwiki enables multiple controversial pages, still linked for exploration.
+
+### 2.6 Strengths and limits
+
+| Strengths | Limits |
+|-----------|--------|
+| Personal sovereignty over pages | Requires running a site (mitigated by one-click AWS deploy) |
+| Low-friction remix with provenance | Small niche ecosystem |
+| Multi-perspective knowledge | Weak team-governance / permissions story vs enterprise wikis |
+| JSON portability | Plugin fragmentation; non-Markdown item model |
+| Strong conceptual clarity | Not a general adapter layer for arbitrary backends |
+
+---
+
+## 3. Other federation models
+
+### 3.1 Git-backed distributed wikis (ikiwiki pattern)
+
+[ikiwiki distributed wikis](https://ikiwiki.info/tips/distributed_wikis/) documents
+a **spectrum of decentralization**:
+
+| Level | Setup | Federation mechanism |
+|-------|-------|---------------------|
+| 0 | Single server | None |
+| 1 | HTML mirror | Read-only copy |
+| 2 | Split web + git host | Clone/pull |
+| 3 | Pinger mirrors | Central bare repo + ping-on-edit propagation |
+| 4 | Fully decentralized | Independent wikis push/pull via git to peers |
+
+**Branching a wiki** = clone origin read-only (`git://`), edit locally without
+push-back — explicit fork. Conflicts surface as markers in rendered pages.
+
+**Relevance to shard-wiki:** Strong alignment with **Git-addressable
+coordination** and **coordination journal** in `INTENT.md`. ikiwiki federates
+*homogeneous* git-backed wikis; shard-wiki must also attach non-git shards
+(Obsidian, WebDAV, Gitea wiki, Coulomb, etc.).
+
+**Gollum / GitHub wikis:** Same backing-store idea — pages as files in a repo;
+federation is git sync, not a wiki protocol. Multi-repo Gollum setups are
+operational, not semantic union.
+
+### 3.2 ActivityPub wiki federation (XWiki)
+
+[XWiki ActivityPub Application](https://extensions.xwiki.org/xwiki/bin/view/Extension/ActivityPub%20Application/)
+(v1.7.x) connects XWiki instances to the **fediverse**:
+
+| Capability | Notes |
+|------------|-------|
+| Follow user / wiki actor | Receive Create/Update notifications |
+| Share documents | Push page content to followers; view remote in modal |
+| Messaging / discussions | Note objects threaded as discussions |
+| Like, mention | Social engagement primitives |
+| Webfinger + ActivityPub endpoints | Standard discovery |
+
+**Caveats (documented by XWiki):** Federation "not well supported yet" for
+full activity routing; Mastodon mention of Page entities incomplete; security
+signing partial.
+
+**Contrast with fedwiki:** ActivityPub federates **activities and notifications**
+between **compatible app instances**; content may be copied on share, but the
+model is social-graph + event stream, not per-user fork sovereignty. Closer to
+**fediverse** than **fedwiki** UX.
+
+**Contrast with shard-wiki:** ActivityPub could be an **adapter transport** for
+some shards, but shard-wiki's core is wiki-page semantics (paths, links,
+overlays, provenance), not activity streams.
+
+### 3.3 Xanadu / transclusion lineage
+
+Ted Nelson's [Project Xanadu](https://en.wikipedia.org/wiki/Project_Xanadu)
+(1960s–) pursued **serious electronic literature** with patterns still relevant:
+
+| Pattern | Intent | Modern partial implementations |
+|---------|--------|-------------------------------|
+| **Visible links** | Show destination context, not opaque jumps | Hover previews, Open Graph unfurls |
+| **Parallel documents** | Side-by-side source + derivative | Multi-pane editors (Obsidian, Roam) |
+| **Transclusion** | Live inclusion of remote span with origin pointer | Embeds, block references (weak) |
+| **Transcopyright** | Permissioning for reuse | Mostly unsolved at web scale |
+| **Stable addresses** | Fine-grained, durable pointers | Block UIDs (Roam); URL rot elsewhere |
+| **Bi-directional links** | Backlinks as first-class | Obsidian, Roam, shard-wiki BackLinks UC |
+
+Xanadu is **speculative design / pattern language**, not deployable federation.
+Useful for shard-wiki **provenance, transclusion, and link resolution** thinking —
+especially union BackLinks and projection freshness.
+
+### 3.4 Personal / local-first wikis (TiddlyWiki, Obsidian)
+
+Not federation protocols, but **sovereignty** models:
+
+- **TiddlyWiki** — single-file portable wiki; complete user ownership.
+- **Obsidian** — local vault + optional sync; community plugins for publish/remix.
+
+Shard-wiki INTENT explicitly lists these as shard participants via adapters.
+Federation is **attachment to a root information space**, not replacing the
+local tool.
+
+### 3.5 yawex multi-wiki resolution (cross-reference)
+
+From `research/260608-yawex-prior-art/findings.md`:
+
+| yawex state | Federation reading |
+|-------------|-------------------|
+| `REMOTE` | Jump to another wiki's page — remote shard navigation |
+| `VIRTUAL` | Local processing, remote content — projection |
+| Multi-wiki config | Multiple named wikis = multiple shards |
+
+**Decision already recorded:** inspiration only; shard-wiki designs resolution
+fresh for heterogeneous adapters.
+
+### 3.6 Solid / personal data pods (brief)
+
+Solid aims at **user-controlled pods** + linked data apps. Distributed wiki
+discussions (forum threads, TiddlyWiki-on-Solid experiments) explore **personal
+wikis on pods** vs **team commons**.
+
+| Axis | Solid-style | Fedwiki | shard-wiki |
+|------|-------------|---------|------------|
+| Unit of sovereignty | Pod / user | Site / person | Shard |
+| Unity mechanism | Linked data queries | Browser composition + fork | Orchestrated union + Git journal |
+| Team wiki | Secondary; app-dependent | Weak | Supported via shards + authz |
+
+No mature Solid wiki federation standard emerged; treat as **adjacent personal-
+data sovereignty** research, not direct prior art.
+
+---
+
+## 4. Comparison matrix
+
+| Model | Unit of ownership | Unity mechanism | Consensus model | History / provenance | Heterogeneous backends |
+|-------|-------------------|-----------------|-----------------|----------------------|------------------------|
+| **Centralized wiki** (c2, MediaWiki) | Site operator | Single namespace | On-page consensus | Server revision log | No |
+| **Federated Wiki** | Per-site (per person) | Browser pulls JSON; fork | Spread of versions | Journal per page | No (all SFW-shaped) |
+| **ikiwiki distributed** | Per git clone | git push/pull, pingers | Merge conflicts in git | Git commits | No (all ikiwiki) |
+| **ActivityPub (XWiki)** | Per instance | Activity stream | Social follow/share | Activity + page store | No (XWiki only) |
+| **Xanadu** | Document address | Transclusion | Parallel versions | Version trails (concept) | N/A (unbuilt) |
+| **shard-wiki (INTENT)** | Per shard | Orchestrator union | Configurable policy | Git coordination journal + shard history | **Yes** (adapter contract) |
+
+---
+
+## 5. Glossary (federation track)
+
+| Term | Meaning | Primary source |
+|------|---------|----------------|
+| **Fork** | Copy a page (or site branch) to a new home; divergent editing | Fedwiki, git |
+| **Journal** | Ordered actions reconstructing page state; travels with page | Fedwiki JSON schema |
+| **Story** | Array of typed items (paragraph, image, …) forming page body | Fedwiki |
+| **Happening** | Time-bounded fedwiki collaboration on a topic site | Caulfield |
+| **Bounded conversation** | Site with expected end; not infinite commons | Caulfield |
+| **Reverse bit-rot** | Content improves through reuse across contexts | Caulfield / Ward |
+| **Chorus of voices** | Multiple perspectives on same topic, linked | Ward |
+| **Inverted model** | Records distributed; meaning composed client-side | Ward |
+| **Projection** | Local view of remote content without claiming ownership | shard-wiki / yawex VIRTUAL |
+| **Overlay** | Local edit against remote/canonical without immediate mutation | shard-wiki |
+| **Shard** | Independently meaningful page store with own capabilities | shard-wiki |
+| **Coordination journal** | Git-backed change record for an information space | shard-wiki |
+| **Transclusion** | Include live span from elsewhere with origin visible | Xanadu |
+| **Fediverse** | Network of ActivityPub-speaking apps | XWiki AP, Mastodon |
+| **Pinger / pingee** | Edit-triggered mirror refresh between ikiwiki sites | ikiwiki |
+| **Wiki actor** | Entire wiki as ActivityPub actor | XWiki AP |
+
+---
+
+## 6. Mapping to shard-wiki INTENT (compare, do not equate)
+
+### 6.1 Strong resonances
+
+| Fedwiki / federation idea | shard-wiki concept | Notes |
+|---------------------------|-------------------|-------|
+| Personal ownership of edits | Shard sovereignty | shard-wiki generalizes beyond per-person sites |
+| Fork with provenance | Overlay + patch flows | shard-wiki separates draft from destructive apply |
+| Journal / history with page | Coordination journal + shard revision | Git journal is space-level; shard retains own history |
+| Browser-composed union | Union of pages across shards | Orchestrator presents coherent graph |
+| JSON not HTML for remix | Markdown-first page model | Different format; same structural intent |
+| Lazy pull from remote | Projection | yawex VIRTUAL; explicit freshness |
+| Chorus of voices | Union without erasure | Show provenance, divergence, equivalents |
+| Bounded conversation | Configurable policy | Mechanism not policy — space may be permanent or ephemeral |
+| Reverse bit-rot on reuse | Reconciliation / sync | Policy-driven; not automatic fork |
+| Derived views (recent, links) | UC-05 union views | BackLinks, RecentChanges across shards |
+
+### 6.2 Deliberate divergences (design bugs if conflated)
+
+| Fedwiki assumption | shard-wiki correction |
+|--------------------|----------------------|
+| Every participant runs SFW-shaped JSON site | **Adapter contract** for Gitea, folders, Obsidian, WebDAV, engines |
+| Fork = default edit primitive | **Capability-aware**: read-only shards need overlay, not fork |
+| Client composes in browser only | Orchestrator may serve CLI, agents, CI — not browser-only |
+| Per-site = per-person | Shards may be team repos, org wikis, app databases |
+| No central coordination | **Git-addressable coordination layer** per information space |
+| Plugin item types | Markdown-first; engine-specific render is out of core scope |
+| Implicit spread-as-consensus | **Mechanism over policy** — canonical source is explicit config |
+
+### 6.3 What shard-wiki adds beyond fedwiki
+
+1. **Heterogeneous attachment** — not all shards speak the same page JSON.
+2. **Capability matrix** — read/write/diff/merge/lock/publish per shard.
+3. **Overlay-before-mutation** — respect remote sovereignty and limited backends.
+4. **Semantic wiki sync** — not generic file mirroring.
+5. **Enterprise authz ladder** — L0 open → delegated IAM (without owning identity).
+6. **Divergence detection** — equivalent pages across shards, reconcilable.
+
+### 6.4 What fedwiki teaches that shard-wiki should not lose
+
+1. **Frictionless reuse** — if remix takes 15 minutes, federation fails socially.
+2. **Provenance by default** — history must travel or be reconstructible.
+3. **Multi-perspective truth** — union ≠ single canonical article.
+4. **Expected lifecycle** — information spaces may be ephemeral; plan for carry-forward.
+5. **Personal sovereignty** — contributors must not be hostage to one operator's server.
+
+---
+
+## 7. Use-case seeds (not yet in UseCaseCatalog)
+
+Candidates for future promotion from this research:
+
+| ID | Use case | Source |
+|----|----------|--------|
+| UC-FED-01 | **Fork page from remote shard into writable shard** | Fedwiki fork; maps to overlay or import policy |
+| UC-FED-02 | **View multiple versions of equivalent page** | Fedwiki chorus; union without erasure |
+| UC-FED-03 | **Carry forward pages from closed/archived shard** | Bounded conversation / reverse bit-rot |
+| UC-FED-04 | **Remix with portable attribution** | Fedwiki journal; coordination journal |
+| UC-FED-05 | **Time-bounded collaboration space** | Happening; optional space lifecycle policy |
+| UC-FED-06 | **Subscribe to remote shard changes** | ikiwiki pinger, ActivityPub Create/Update |
+| UC-FED-07 | **Transclude remote span with live freshness** | Xanadu; projection + provenance |
+| UC-FED-08 | **Git-branch a wiki information space** | ikiwiki branch; coordination journal fork |
+
+---
+
+## 8. Open questions (for spec / workplans)
+
+1. **Fork vs overlay vs import** — When does shard-wiki copy content into a
+   writable shard vs keep an overlay vs link-only reference?
+2. **Equivalent page identity** — How are "same topic" pages matched across
+   shards (title, path, link graph, explicit alias)?
+3. **Journal format** — Does shard-wiki adopt fedwiki-like action journals,
+   Git commits only, or both per shard type?
+4. **Browser vs server composition** — Where does union merging run for agents
+   and non-UI consumers?
+5. **ActivityPub as adapter** — Optional transport for change notification, or
+   out of scope?
+6. **Ephemeral spaces** — Should information spaces have first-class lifecycle
+   (archived, read-only, merged-into-successor)?
+7. **Plugin/item extensibility** — Markdown extensions vs structured blocks for
+   remixable assessments (fedwiki OER lesson)?
+8. **Consensus without erasure** — Policy presets: fedwiki-spread, git-merge,
+   designated canonical shard, vote-to-merge?
+
+---
+
+## 9. Sources
+
+| Source | URL |
+|--------|-----|
+| Wikipedia — Federated Wiki | https://en.wikipedia.org/wiki/Federated_Wiki |
+| Wired — Wiki Inventor Sticks a Fork | https://www.wired.com/2012/07/wiki-inventor/ |
+| viki.wiki — JSON Schema | https://viki.wiki/json-schema.html |
+| Fedwiki GitHub | https://github.com/fedwiki/wiki |
+| Caulfield — Federated Education (NWACC 2014) | https://hapgood.us/2014/11/06/federated-education-new-directions-in-digital-collaboration/ |
+| Caulfield — Bounded Conversations | https://hapgood.us/2015/01/21/rethinking-wiki-lifecycle-sites-as-bounded-conversations/ |
+| Caulfield — OER Case for Federated Wiki | https://hapgood.us/2015/05/05/the-oer-case-for-federated-wiki/ |
+| ikiwiki — distributed wikis | https://ikiwiki.info/tips/distributed_wikis/ |
+| XWiki — ActivityPub Application | https://extensions.xwiki.org/xwiki/bin/view/Extension/ActivityPub%20Application/ |
+| Maggie Appleton — Xanadu Patterns | https://maggieappleton.com/xanadu-patterns |
+| Wikipedia — Project Xanadu | https://en.wikipedia.org/wiki/Project_Xanadu |
+| shard-wiki — yawex prior art | `research/260608-yawex-prior-art/findings.md` |
+| shard-wiki — c2 origins | `research/260608-c2-wiki-origins/findings.md` |
+| shard-wiki — INTENT | `INTENT.md` |
+
+---
+
+## 10. Traceability
+
+| This document section | Informs (future) |
+|-----------------------|------------------|
+| §2 Federated Wiki | Adapter design, page model, overlay/fork policy |
+| §3.1 ikiwiki | Git coordination journal, mirror/branch modes |
+| §3.2 ActivityPub | Optional notification adapter |
+| §3.3 Xanadu | Link resolution, transclusion, provenance UI |
+| §6 Mapping | Architecture blueprint guardrails |
+| §7 UC seeds | `spec/UseCaseCatalog.md` promotion pass |
--- a/research/README.md
+++ b/research/README.md
@@ -11,4 +11,6 @@ when multiple files or sources are involved. Findings here inform `spec/` and
 | Date | Path | Summary |
 |------|------|---------|
 | 2026-06-08 | `260608-yawex-prior-art/` | yawex 0.7.4 Perl wiki prior art; federation design seeds |
-| 2026-06-08 | `260608-c2-wiki-origins/` | Ward Cunningham & WikiWikiWeb origins; terms and use cases |
+| 2026-06-08 | `260608-c2-wiki-origins/` | Ward Cunningham & WikiWikiWeb origins; terms and use cases |
+| 2026-06-08 | `260608-federation-concepts/` | Federated Wiki, git/ActivityPub/Xanadu federation models |
+| 2026-06-08 | `260608-wikiengines-overview/` | Wiki engine landscape survey (Perplexity-assisted) |