shard-wiki/workplans/SHARD-WP-0002-federation-architecture.md

id, type, title, domain, repo, status, owner, topic_slug, created, updated, depends_on, state_hub_workstream_id

id	type	title	domain	repo	status	owner	topic_slug	created	updated	depends_on	state_hub_workstream_id
SHARD-WP-0002	workplan	federation architecture design	whynot	shard-wiki	active	tegwick	whynot	2026-06-08	2026-06-13	SHARD-WP-0001	2af4c46d-cbfd-40ea-a94b-d9e60b0f9945

SHARD-WP-0002 — Federation architecture design

Goal

Produce a federation architecture specification for shard-wiki: positioning against prior art (Federated Wiki, git-backed wikis, ActivityPub), documented decisions and tradeoffs, and ADR-ready design notes that resolve the open questions raised by research/260608-federation-concepts/ and UseCaseCatalog UC-26–UC-33.

Primary deliverable: spec/FederationArchitecture.md (created and filled by this workplan's tasks).

Secondary deliverable (T11–T15, added 2026-06-13): the shard adapter contract constraints distilled from the engine deep dives, feeding spec/TechnicalSpecificationDocument.md. Federation architecture decides what the union does; the adapter contract decides what a backend must expose to participate. The two are co-dependent (the capability matrix T10 consumes the contract vocabulary T11 defines), so they live in one workplan.

Context

• Federation research: research/260608-federation-concepts/findings.md
• Engine deep dives (adapter-contract inputs, T11–T15):
  • research/260608-wikiengines-overview/findings.md (landscape; capability heterogeneity, write granularity)
  • research/260613-xwiki-deep-dive/findings.md (DB/component app-platform)
  • research/260613-twiki-deep-dive/findings.md (file+RCS app-platform)
  • research/260613-foswiki-deep-dive/findings.md (Foswiki::Store versioned store interface = adapter-contract prior art)
• Use cases: spec/UseCaseCatalog.md § A (UC-26–UC-43; UC-34–43 are the engine-attachment / adapter-contract cases)
• Aspiration: INTENT.md (orchestrator, not engine; mechanism over policy; capability-aware adapters; Markdown-first, backend-neutral)
• Related workplan: SHARD-WP-0001 (page resolution, namespaces, overlays, provenance — federation architecture must align with, not duplicate, those outputs)

Non-goal: Implement federation. This workplan produces architecture and decision records only.

Decision topics (overview)

Topic	Key tradeoff	Primary UCs
Orchestrator positioning	Adapter layer vs fedwiki-style homogeneous network	UC-02, UC-26
Remix primitives	Fork vs overlay vs import vs link-only	UC-04, UC-26, UC-29
Equivalent page identity	Title/path/alias/graph matching; chorus vs canonical	UC-27, UC-07
History model	Per-shard journal vs Git coordination journal vs both	UC-29, UC-33
Union composition	Server orchestrator vs client composition	UC-05, UC-27
Change notification	Git ping, ActivityPub, poll — adapter transports	UC-31
Space lifecycle	Permanent vs ephemeral; archive and carry-forward	UC-28, UC-30
Transclusion depth	Whole-page projection vs inline span	UC-03, UC-32
Consensus presets	Spread, merge, designated canonical — policy not core	UC-07, UC-27
Capability matrix	Which federation ops require which adapter capabilities	UC-02–UC-07

Adapter-contract topics (T11–T15, from engine deep dives)

Topic	Key tradeoff	Primary UCs
Adapter contract & capabilities	Versioned interface + capability vocabulary; write granularity	UC-02, UC-35, UC-38
Structured page model	Typed frontmatter vs sidecar vs blob; bodiless / multi-record pages	UC-34, UC-39
History portability	Supplement (DB-internal) vs import (open file history)	UC-36, UC-41
Adapter binding	Runtime API vs on-disk store; engine-hosted vs external; backend-swap	UC-38, UC-40, UC-43
Syntax translation	Lossless non-Markdown round-trip vs read-only degradation	UC-42, UC-03

---

Architecture positioning and boundaries

id: SHARD-WP-0002-T1
status: todo
priority: high
state_hub_task_id: "ea8fdb22-6c7f-4ac1-9799-1346abf3c3b7"

Write the opening sections of spec/FederationArchitecture.md:

• shard-wiki as orchestration layer over heterogeneous shards (contrast Federated Wiki homogeneous JSON sites, ikiwiki homogeneous git wikis, ActivityPub activity streams).
• Explicit compare, do not equate mapping from federation research §6.
• Architectural boundaries: what core owns vs adapters vs UI vs policy config.
• Relationship to SHARD-WP-0001 outputs (resolution, namespaces, overlays).

Tradeoffs to document: Central Git coordination journal vs fully decentralized peer sync; browser-composed union vs server-side orchestrator for agents/CLI.

---

Remix primitives: fork, overlay, import, reference

id: SHARD-WP-0002-T2
status: todo
priority: high
state_hub_task_id: "fb7d4bce-5d2e-4602-9b63-85934d90e82d"

Define when each remix primitive applies and how they interact:

Primitive	Typical trigger	Writes to remote?
Reference	Link only	No
Projection	Read remote page	No (cache optional)
Overlay	Edit read-only shard	No until explicit apply
Import / fork	Copy into writable shard	Source unchanged

Resolve federation research open question #1. Cover capability-limited shards (read-only, no diff/merge). Map to UC-26, UC-04, UC-29.

Tradeoffs: Fedwiki fork-as-default vs overlay-before-mutation; copy cost and attribution portability vs link-only federation.

---

Equivalent page identity and multi-version presentation

id: SHARD-WP-0002-T3
status: todo
priority: high
state_hub_task_id: "8f2a333d-ddcc-4cc6-b6ed-1ba9b178eee3"

Specify how shard-wiki identifies "the same topic" across shards:

• Matching signals: normalized title, path, explicit alias table, link-graph equivalence, manual curator binding.
• Presentation: chorus-of-voices (UC-27) vs designated canonical (policy).
• Link to UC-07 divergence detection and reconciliation triggers.

Tradeoffs: Automatic matching false positives vs manual curation burden; showing all versions vs default canonical with alternates visible.

---

History, attribution, and coordination journal

id: SHARD-WP-0002-T4
status: todo
priority: high
state_hub_task_id: "5f39f48d-5142-4078-a84f-3245ec1add7e"

Define how edit history and attribution flow across federation operations:

• Per-shard revision model (Git commit, engine history, fedwiki-style journal where applicable).
• Information-space coordination journal role for cross-shard operations (fork, import, reconcile, space branch).
• Portable attribution requirements for UC-29 (frictionless remix).

Tradeoffs: Fedwiki journal-per-page vs Git-only; duplication of history vs reconstructibility from coordination journal; storage cost of embedded media on import.

---

Union composition layer

id: SHARD-WP-0002-T5
status: todo
priority: medium
state_hub_task_id: "3ff71e11-d0e9-4fda-b916-d6c34c51aa51"

Decide where the union view is assembled:

• Orchestrator API (server-side graph for agents, CI, non-browser clients).
• Optional client-side composition (fedwiki-style browser pull) as a consumer pattern, not the only path.
• Caching, freshness, and invalidation interaction with UC-03, UC-05, UC-31.

Tradeoffs: Single composition point (simpler provenance) vs distributed composition (fedwiki resilience); cache staleness vs live-pull latency.

---

Change notification and subscription transports

id: SHARD-WP-0002-T6
status: todo
priority: medium
state_hub_task_id: "9596e5e8-8d6b-4ed4-bcbc-ebb45e3168be"

Specify change-notification as an optional adapter capability:

• Transports: git hook / pinger (ikiwiki), ActivityPub Create/Update (XWiki), WebDAV ETag, polling fallback.
• What a subscription triggers: projection refresh, reconciliation queue, RecentChanges union entry (UC-31, UC-17).
• Out-of-scope vs in-scope for v1.

Tradeoffs: Push freshness vs implementation complexity; ActivityPub as notification-only vs content-bearing share; dependency on external fediverse infrastructure.

---

Information space lifecycle

id: SHARD-WP-0002-T7
status: todo
priority: medium
state_hub_task_id: "38134064-51ce-4f5a-80bf-b2cfbe381c59"

Model lifecycle states for root entities / information spaces:

• Active, read-only archived, retired (detached), merged-into-successor.
• Carry-forward workflow (UC-28, UC-30): selective import from archived shard.
• Space-level fork / branch (UC-33): coordination journal + shard config branch.

Tradeoffs: First-class ephemeral "happenings" vs permanent spaces only; automatic archive vs manual; whether retirement deletes projections or preserves read-only union entries.

---

Transclusion and projection depth

id: SHARD-WP-0002-T8
status: todo
priority: medium
state_hub_task_id: "5607732b-612a-4550-bb17-b8cd34979cf4"

Distinguish projection levels:

Level	UC	Behavior
Whole-page projection	UC-03	Lazy full page from remote shard
Block/span transclusion	UC-32	Inline embed with origin + freshness
Link reference	UC-08	Pointer only

Define provenance display requirements and staleness semantics for UC-32. Reference Xanadu transclusion patterns without adopting unbuilt Xanadu scope.

Tradeoffs: Live transclusion fragility (link rot, remote down) vs snapshot on import; core orchestrator support vs Markdown extension + adapter.

---

Consensus and reconciliation policy catalog

id: SHARD-WP-0002-T9
status: todo
priority: medium
state_hub_task_id: "adfca8b3-eb21-497d-9f51-65dc9269c810"

Document configurable policy presets (mechanism over policy):

• Fedwiki-spread (versions coexist; popularity implicit).
• Designated canonical shard (explicit write target).
• Git-merge reconciliation (ikiwiki-style).
• Overlay-only (no destructive merge on read-only sources).
• Vote-to-merge / editorial gate (optional, UI-layer).

Map presets to UC-07, UC-27. Core provides primitives; policy selects preset.

Tradeoffs: Default policy for L0 open mode vs team mode; exposing conflicts vs auto-merge when backends support it.

---

Federation operations capability matrix

id: SHARD-WP-0002-T10
status: todo
priority: medium
state_hub_task_id: "c7a93d06-8631-43b4-bc7f-1b0a1cd1436f"

Produce a capability matrix: which federation operations require which adapter capabilities (read, write, diff, merge, lock, version, publish, notify, transclude-source).

Cross-check against INTENT capability-aware adapters principle. Identify graceful degradation paths for limited backends (read-only participant, projection-only, patch target).

This matrix consumes the capability vocabulary defined in T11 — keep the two in sync. Feeds spec/TechnicalSpecificationDocument.md adapter contract section.

---

Shard adapter contract (folded from engine deep dives, 2026-06-13)

These tasks distill the cross-cutting adapter-contract constraints surfaced by the four engine deep dives (wiki-engines landscape, XWiki, TWiki, Foswiki). They answer what a backend must expose to participate as a shard, complementing the federation tasks above (which answer what the union does). Output target: spec/TechnicalSpecificationDocument.md (adapter contract section), cross-linked from spec/FederationArchitecture.md.

Recurring cross-engine insight to honor: structure and history in text files federate far more easily than in a database, and Foswiki::Store already proves the versioned-interface-with-swappable-backends pattern the contract should adopt.

Adapter contract: capability model & versioned interface

id: SHARD-WP-0002-T11
status: todo
priority: high
state_hub_task_id: "a7379621-6694-488b-94ca-846b8e27e745"

Define the capability-aware shard adapter contract as a versioned interface, taking Foswiki::Store + Foswiki::Meta as direct prior art (a real engine that already separates store backend from core behind a stable interface).

• Capability vocabulary: read, write, diff, merge, lock, version, publish, notify, transclude-source, translate-syntax, structured-payload (reconcile with T10).
• Write granularity as a first-class capability dimension: per-page / per-file / per-space / append-only (TiddlyWiki single-file vs DB whole-space vs TWiki per-topic).
• Versioning/compatibility rules for the contract itself; capability discovery.

Inputs: 260608-wikiengines-overview §3, §5; 260613-foswiki-deep-dive §2, §6. Provides the vocabulary T10's matrix consumes. UCs: UC-02, UC-35, UC-38.

Tradeoffs: Rich capability set (precise degradation) vs contract complexity; static profile vs runtime capability negotiation.

---

Page model: structured / typed payload representation

id: SHARD-WP-0002-T12
status: todo
priority: high
state_hub_task_id: "7334a4a4-ba75-4fac-a8b4-8350d342b299"

Decide how the backend-neutral, Markdown-first page model carries structured data without flattening:

• Source shapes: XWiki XObjects against an XClass; TWiki/Foswiki DataForms stored as %META:FIELD% in text; Foswiki MetaDataPlugin multiple records per topic; Semantic MediaWiki annotations.
• Representation choice: typed frontmatter vs sidecar file vs opaque provenance blob.
• Bodiless pages (UC-39): may a page be purely structured, no Markdown body?
• N typed records per page (not one form) must be representable.

Inputs: xwiki §2.3; twiki §2.3; foswiki §4. UCs: UC-34, UC-39.

Tradeoffs: Lossless fidelity vs a uniform queryable model; diff/search over structured fields vs prose.

---

History portability: supplement vs import

id: SHARD-WP-0002-T13
status: todo
priority: high
state_hub_task_id: "6837862a-8f57-410d-9200-a6a5dcf1a7b9"

Define how the coordination journal relates to an engine's native history (aligns with and refines T4):

• Supplement — engine history is DB-internal and non-portable (Confluence, MediaWiki, XWiki xwikircs): the journal begins now / mirrors forward (UC-36).
• Import — engine history is an open file format (TWiki/yawex RCS .txt,v, Foswiki PlainFile timestamped copies): backfill into Git preserving author/timestamp (UC-41).
• Authoritative journal vs mirror-reconciled-back; one-time vs continuous.

Inputs: xwiki §2.4; twiki §2.1; foswiki §2.2. UCs: UC-36, UC-41.

Tradeoffs: Import fidelity vs effort; duplicated history vs single source of truth.

---

Adapter binding: attach path, hosting, backend-swap

id: SHARD-WP-0002-T14
status: todo
priority: medium
state_hub_task_id: "f8835969-d118-4738-952a-5e67e5209f3d"

Decide how and where an adapter binds to a backend:

• Dual-path attach — runtime API vs reading the on-disk store directly (consistency vs offline/fidelity); capability-gate the choice (UC-40).
• Hosting — engine-side adapter via native extension API (XWiki components/REST/ UIX; TWiki/Foswiki plugin handlers + REST + registerTagHandler) vs external adapter (no engine change, lower fidelity) (UC-38).
• Backend-swap tolerance — shard identity/provenance survives a storage-format change (Foswiki RCS↔PlainFile; folder→Git) (UC-43).

Inputs: twiki §5, §6; xwiki §3; foswiki §2. UCs: UC-38, UC-40, UC-43.

Tradeoffs: Fidelity vs deploy access; consistency vs offline capability.

---

Syntax translation & content fidelity

id: SHARD-WP-0002-T15
status: todo
priority: medium
state_hub_task_id: "22b57b3a-b06b-4ff0-a34a-667a0386bf25"

Specify how non-Markdown shards participate in the Markdown-first model:

• Read native markup (TWiki/Foswiki TML, XWiki syntax) into the page model and accept Markdown overlays back via bidirectional, lossless translation (UC-42).
• Feasibility proof: Foswiki WysiwygPlugin (TML→HTML for editing, HTML→TML losslessly on save).
• Graceful degradation: when lossless translation is unavailable, the shard is a read-only/projection participant (UC-03), never silently corrupted.
• Open: may overlays be stored in Markdown, or must they round-trip in native syntax?

Inputs: foswiki §5; twiki §2.4; xwiki §2.5. UCs: UC-42, UC-03.

Tradeoffs: Translation coverage vs read-only floor; Markdown overlays (portable) vs native-syntax overlays (safe round-trip).

---

Acceptance criteria

• spec/FederationArchitecture.md exists with all ten federation topic sections (T1–T10) and an explicit decisions / deferred / open table per topic.
• The adapter contract section of spec/TechnicalSpecificationDocument.md exists, populated by T11–T15, with the capability vocabulary (T11) reconciled against the T10 federation-ops matrix.
• Each decision honors INTENT: mechanism over policy, union without erasure, overlay before mutation, no silent remote mutation, shard sovereignty, capability-aware adapters, Markdown-first / backend-neutral.
• UC-26–UC-43 are traceable to architecture / adapter-contract sections (UC-34–UC-43 to T11–T15 specifically).
• Conflicts or dependencies on SHARD-WP-0001 outputs are listed (e.g. namespace model affects equivalent-page identity; page model T12 affects resolution/overlays).
• SHARD-WP-0001 and SHARD-WP-0002 can proceed in parallel where topics are independent; merge conflicts in spec are resolved before implementation workplan starts.

Suggested task order

Federation architecture (T1–T10):

1. T1 positioning (frames everything)
2. T2 remix primitives + T3 equivalent identity (parallel)
3. T4 history model
4. T5 composition + T6 notification (parallel)
5. T7 lifecycle + T8 transclusion (parallel)
6. T9 policy catalog + T10 capability matrix (parallel, finalize doc)

Adapter contract (T11–T15) — can start in parallel with T1; T11 first as it frames the rest, and pair it with T10 (shared capability vocabulary): 7. T11 contract & capabilities (frames T12–T15; sync with T10) 8. T12 page model + T13 history portability (parallel; T13 aligns with T4) 9. T14 binding + T15 syntax translation (parallel, finalize adapter-contract spec)