Files
shard-wiki/workplans/SHARD-WP-0002-federation-architecture.md
tegwick d65f9e21f3 spec(SHARD-WP-0002): adapter contract (TSD §A, T11-T16+T18); workplan done
Adds the normative Shard Adapter Contract as TechnicalSpecificationDocument §A:
A.1 versioned capability contract (verbs + orthogonal-core spectra), A.2 verified
conformance suite (profiles not self-asserted), A.3 attachment-mode taxonomy +
image-is-not-a-store boundary, A.4 page model (incl. computational shapes,
identity/placement/equivalence, layered provenance), A.5 history portability,
A.6 syntax translation + fidelity report, A.7 addressing/navigation, A.8 gated
computational content. Updates TSD references/UC-count/next-work. Flips all 18
WP-0002 tasks + workplan done. Design layer complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:25:50 +02:00

868 lines
45 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
id: SHARD-WP-0002
type: workplan
title: "federation architecture design"
domain: whynot
repo: shard-wiki
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-08"
updated: "2026-06-15"
depends_on:
- SHARD-WP-0001
state_hub_workstream_id: "2af4c46d-cbfd-40ea-a94b-d9e60b0f9945"
---
# SHARD-WP-0002 — Federation architecture design
## Goal
Produce a **federation architecture specification** for shard-wiki: positioning
against prior art (Federated Wiki, git-backed wikis, ActivityPub), documented
**decisions and tradeoffs**, and ADR-ready design notes that resolve the open
questions raised by `research/260608-federation-concepts/` and UseCaseCatalog
UC-26UC-33.
Primary deliverable: `spec/FederationArchitecture.md` (created and filled by
this workplan's tasks).
Secondary deliverable (T11T16): the **shard adapter contract** constraints distilled
from the engine deep dives (T11T15, added 2026-06-13) and the modern-tool dives +
cross-dive synthesis (T16 and scope extensions, added 2026-06-14), feeding
`spec/TechnicalSpecificationDocument.md`. Federation architecture decides *what the
union does*; the adapter contract decides *what a backend must expose to participate*.
The two are co-dependent (the capability matrix T10 consumes the contract vocabulary
T11 defines), so they live in one workplan.
Tertiary deliverable (T17T18, added 2026-06-15): two **selectable-axis** deliverables
re-folded from the later syntheses — **T17 the federation-model taxonomy** (federation is
plural and composable, from synthesis **v3**) and **T18 the computational / executable
content capability** (from the SHARD-WP-0004 "computational page model" synthesis).
The cross-dive synthesis (`research/260614-shard-spectrum-synthesis/findings.md`, **v3
2026-06-14** — extended across ~23 systems incl. the SHARD-WP-0003 engine batch: Federated
Wiki, Wikibase, the git-forge wikis, TiddlyWiki, ikiwiki, Quip, MojoMojo, Oddmuse,
UseModWiki, on top of v2's Joplin/Logseq/CRDT-cohort/Trilium) reframes the contract around
**fifteen capability spectra** — the thirteen of v2 (addressing, content identity,
**identity-vs-placement**, structure, history, **merge model**, native query, translation,
attachment mode, operational envelope, access grant, **content opacity**, write
granularity) plus **#14 provenance granularity** (v3: per-shard → per-page → per-edit →
per-statement/value) and **#15 computational / liveness** (SHARD-WP-0004: static → captured-
output → live-over-files → view-time → irreducibly-live) — positions on a spectrum anchored
at both ends by a real system, with federation ops degrading by position, rather than a flat
verb checklist. v3 also adds a **coordination-layer axis** (the federation-model taxonomy,
T17) and the **computational page model** adds a **two-axis projection model** (replication↔
derivation-projection × live↔snapshot, T16/T18).
## Context
- Federation research: `research/260608-federation-concepts/findings.md`
- Engine deep dives (adapter-contract inputs, T11T15):
- `research/260608-wikiengines-overview/findings.md` (landscape; capability
heterogeneity, write granularity)
- `research/260613-xwiki-deep-dive/findings.md` (DB/component app-platform)
- `research/260613-twiki-deep-dive/findings.md` (file+RCS app-platform)
- `research/260613-foswiki-deep-dive/findings.md` (`Foswiki::Store` versioned
store interface = adapter-contract prior art)
- Conceptual + modern-tool dives (T16 + scope extensions, 2026-06-14):
- `research/260614-xanadu-deep-dive/findings.md` (reference-not-copy, transclusion,
tumbler span addressing — ideal)
- `research/260614-zigzag-deep-dive/findings.md` (information space as co-equal
dimensions; clone = transclusion — ideal)
- `research/260614-roam-deep-dive/findings.md` (block-DB, store UUIDs, Datalog,
in-app host)
- `research/260614-obsidian-deep-dive/findings.md` (file-over-app, in-file `^id`,
MetadataCache derived index, dual attach, ecosystem popularity)
- `research/260614-notion-deep-dive/findings.md` (closed SaaS, external-API-only,
DB schema+relations, operational envelope, scoped grant)
- `research/260614-joplin-deep-dive/findings.md` (SQLite-local, interchange/sync-mirror
attach, E2EE content opacity, local-REST)
- `research/260614-logseq-deep-dive/findings.md` (block-graph on plain files, in-file
`id::`, derived Datalog index, file→SQLite substrate migration)
- `research/260614-localfirst-workspaces-deep-dive/findings.md` (Anytype/AFFiNE/AppFlowy:
CRDT substrate, native merge, P2P/E2EE, CRDT-replica attach)
- `research/260614-trilium-deep-dive/findings.md` (note cloning → DAG / identity≠placement,
inherited+templated computed metadata, HTML-native, scripting+ETAPI, per-item opacity)
- Engine batch + federation-model inputs (T17 + spectra extensions, 2026-06-14, SHARD-WP-0003):
- `research/260614-federated-wiki-deep-dive/findings.md` (fork+journal federation; story =
semantic-action replay — the coordination-journal anchor)
- `research/260614-wikibase-deep-dive/findings.md` (typed entity-statement graph; SPARQL +
federated `SERVICE` = query-time graph-join federation; per-statement provenance/rank)
- `research/260614-forge-wikis-deep-dive/findings.md` (Gitea/GitLab/GitHub: git IS the
store — resolves the engine-mirror write-race; the home case)
- `research/260614-{tiddlywiki,ikiwiki,quip,mojomojo,oddmuse,usemodwiki}-deep-dive/findings.md`
(whole-file write-granularity; VCS-replication+ping federation; direct-DB binding; the
flat-file floor)
- Computational / interactive-knowledge inputs (T18 + page-model/projection extensions,
2026-06-14/15, SHARD-WP-0004):
- `research/260614-{literate-programming,jupyter,mathematica,processing,strudel,
squeak-pharo,glamorous-toolkit}-deep-dive/findings.md` (one-source-many-projections;
notebook computed-output provenance; program-as-page; live↔snapshot; moldable views;
image-is-not-a-store)
- `research/260614-computational-page-model-synthesis/findings.md` (**the computational
page model**: source/derivation/projection, two axes, four page shapes, execution-as-
gated-capability)
- Cross-dive synthesis: `research/260614-shard-spectrum-synthesis/findings.md` (**synthesis
v3**: the fifteen capability spectra, shard family matrix across ~23 systems, the
federation-model taxonomy §2.5, UC-34UC-82 → task fold-in)
- Use cases: `spec/UseCaseCatalog.md` (UC-26UC-84; UC-34UC-43 engine-attachment, UC-44UC-67
conceptual + modern-tool, UC-68UC-82 engine-batch, UC-83UC-84 computational)
- Aspiration: `INTENT.md` (orchestrator, not engine; mechanism over policy;
capability-aware adapters; Markdown-first, backend-neutral)
- Related workplan: `SHARD-WP-0001` (page resolution, namespaces, overlays,
provenance — federation architecture must align with, not duplicate, those
outputs)
**Non-goal:** Implement federation. This workplan produces architecture and
decision records only.
## Decision topics (overview)
| Topic | Key tradeoff | Primary UCs |
|-------|--------------|-------------|
| Orchestrator positioning | Adapter layer vs fedwiki-style homogeneous network | UC-02, UC-26 |
| Remix primitives | Fork vs overlay vs import vs link-only | UC-04, UC-26, UC-29 |
| Equivalent page identity | Title/path/alias/graph matching; chorus vs canonical | UC-27, UC-07 |
| History model | Per-shard journal vs Git coordination journal vs both | UC-29, UC-33 |
| Union composition | Server orchestrator vs client composition | UC-05, UC-27 |
| Change notification | Git ping, ActivityPub, poll — adapter transports | UC-31 |
| Space lifecycle | Permanent vs ephemeral; archive and carry-forward | UC-28, UC-30 |
| Transclusion depth | Whole-page projection vs inline span | UC-03, UC-32 |
| Consensus presets | Spread, merge, designated canonical — policy not core | UC-07, UC-27 |
| Capability matrix | Which federation ops require which adapter capabilities | UC-02UC-07 |
| **Federation-model taxonomy** (T17) | Federation is **plural & composable**: fork+journal / VCS-replication+ping / query-time graph-join / feed-aggregation / activity-streams / engine-mirror — selectable, not one model | UC-26, UC-31, UC-33, UC-71, UC-72, UC-74 |
### Adapter-contract topics (T11T16, from engine + modern-tool dives)
| Topic | Key tradeoff | Primary UCs |
|-------|--------------|-------------|
| Adapter contract & capabilities | **Thirteen spectra** (not flat verbs); write granularity; operational envelope; access grant; **merge model**; **content opacity** | UC-02, UC-35, UC-38, UC-57, UC-61, UC-64 |
| Structured page model | Typed frontmatter vs sidecar vs blob; bodiless / multi-record; DB schema+relations; **computed (inherited/templated) metadata**; non-Markdown assets; query-defined pages; **multi-placement (DAG) identity** | UC-34, UC-39, UC-55, UC-58, UC-54, UC-66, UC-67 |
| History portability | Supplement (DB-internal incl. Notion/Joplin/Trilium; **CRDT-log**) vs import (open file history) | UC-36, UC-41 |
| Adapter binding | **Full mode taxonomy**: file-store (native *or* interchange/sync-mirror) / in-engine-host / local-REST / external-API / **CRDT-replica** / **P2P-no-central-endpoint**; per-binding choice; backend-swap / substrate-migration | UC-38, UC-40, UC-43, UC-50, UC-57, UC-60, UC-62, UC-64, UC-65 |
| Syntax translation | Lossless round-trip vs **lossy-with-fidelity-report** (HTML/blocks/CRDT) vs read-only degradation | UC-42, UC-59, UC-03 |
| Addressing, identity & navigation (T16) | Span addressing; content identity; **identity≠placement**; transclusion-as-reference; **derived query index**; dimensional/query views; **two-axis projection** (replication↔derivation × live↔snapshot); **moldable view registry** | UC-44, UC-45, UC-46, UC-47, UC-48, UC-51, UC-52, UC-54, UC-63, UC-66, UC-83 |
| **Computational / executable content** (T18) | Source canonical, render = projection; **in scope as page-model+projection, out as execution platform**; execute = gated capability, degrade to snapshot | UC-54, UC-55, UC-83, UC-84 |
---
## Architecture positioning and boundaries
```task
id: SHARD-WP-0002-T1
status: done
priority: high
state_hub_task_id: "ea8fdb22-6c7f-4ac1-9799-1346abf3c3b7"
```
Write the opening sections of `spec/FederationArchitecture.md`:
- shard-wiki as **orchestration layer** over heterogeneous shards (contrast
Federated Wiki homogeneous JSON sites, ikiwiki homogeneous git wikis,
ActivityPub activity streams).
- Explicit **compare, do not equate** mapping from federation research §6.
- Architectural boundaries: what core owns vs adapters vs UI vs policy config.
- Relationship to `SHARD-WP-0001` outputs (resolution, namespaces, overlays).
**Tradeoffs to document:** Central Git coordination journal vs fully
decentralized peer sync; browser-composed union vs server-side orchestrator for
agents/CLI.
---
## Remix primitives: fork, overlay, import, reference
```task
id: SHARD-WP-0002-T2
status: done
priority: high
state_hub_task_id: "fb7d4bce-5d2e-4602-9b63-85934d90e82d"
```
Define when each remix primitive applies and how they interact:
| Primitive | Typical trigger | Writes to remote? |
|-----------|-----------------|-------------------|
| **Reference** | Link only | No |
| **Projection** | Read remote page | No (cache optional) |
| **Overlay** | Edit read-only shard | No until explicit apply |
| **Import / fork** | Copy into writable shard | Source unchanged |
Resolve federation research open question #1. Cover capability-limited shards
(read-only, no diff/merge). Map to UC-26, UC-04, UC-29.
**Tradeoffs:** Fedwiki fork-as-default vs overlay-before-mutation; copy cost
and attribution portability vs link-only federation.
---
## Equivalent page identity and multi-version presentation
```task
id: SHARD-WP-0002-T3
status: done
priority: high
state_hub_task_id: "8f2a333d-ddcc-4cc6-b6ed-1ba9b178eee3"
```
Specify how shard-wiki identifies "the same topic" across shards:
- Matching signals: normalized title, path, explicit alias table, link-graph
equivalence, manual curator binding.
- Presentation: chorus-of-voices (UC-27) vs designated canonical (policy).
- Link to UC-07 divergence detection and reconciliation triggers.
**Tradeoffs:** Automatic matching false positives vs manual curation burden;
showing all versions vs default canonical with alternates visible.
---
## History, attribution, and coordination journal
```task
id: SHARD-WP-0002-T4
status: done
priority: high
state_hub_task_id: "5f39f48d-5142-4078-a84f-3245ec1add7e"
```
Define how edit history and attribution flow across federation operations:
- Per-shard revision model (Git commit, engine history, fedwiki-style journal
where applicable).
- Information-space **coordination journal** role for cross-shard operations
(fork, import, reconcile, space branch).
- Portable attribution requirements for UC-29 (frictionless remix).
**Tradeoffs:** Fedwiki journal-per-page vs Git-only; duplication of history vs
reconstructibility from coordination journal; storage cost of embedded media on
import.
---
## Union composition layer
```task
id: SHARD-WP-0002-T5
status: done
priority: medium
state_hub_task_id: "3ff71e11-d0e9-4fda-b916-d6c34c51aa51"
```
Decide where the **union view** is assembled:
- Orchestrator API (server-side graph for agents, CI, non-browser clients).
- Optional client-side composition (fedwiki-style browser pull) as a consumer
pattern, not the only path.
- Caching, freshness, and invalidation interaction with UC-03, UC-05, UC-31.
**Tradeoffs:** Single composition point (simpler provenance) vs distributed
composition (fedwiki resilience); cache staleness vs live-pull latency.
---
## Change notification and subscription transports
```task
id: SHARD-WP-0002-T6
status: done
priority: medium
state_hub_task_id: "9596e5e8-8d6b-4ed4-bcbc-ebb45e3168be"
```
Specify change-notification as an **optional adapter capability**:
- Transports: git hook / pinger (ikiwiki), ActivityPub Create/Update (XWiki),
WebDAV ETag, polling fallback.
- What a subscription triggers: projection refresh, reconciliation queue,
RecentChanges union entry (UC-31, UC-17).
- Out-of-scope vs in-scope for v1.
**Tradeoffs:** Push freshness vs implementation complexity; ActivityPub as
notification-only vs content-bearing share; dependency on external fediverse
infrastructure.
---
## Information space lifecycle
```task
id: SHARD-WP-0002-T7
status: done
priority: medium
state_hub_task_id: "38134064-51ce-4f5a-80bf-b2cfbe381c59"
```
Model lifecycle states for root entities / information spaces:
- Active, read-only archived, retired (detached), merged-into-successor.
- Carry-forward workflow (UC-28, UC-30): selective import from archived shard.
- Space-level fork / branch (UC-33): coordination journal + shard config branch.
**Tradeoffs:** First-class ephemeral "happenings" vs permanent spaces only;
automatic archive vs manual; whether retirement deletes projections or preserves
read-only union entries.
---
## Transclusion and projection depth
```task
id: SHARD-WP-0002-T8
status: done
priority: medium
state_hub_task_id: "5607732b-612a-4550-bb17-b8cd34979cf4"
```
Distinguish projection levels:
| Level | UC | Behavior |
|-------|-----|----------|
| Whole-page projection | UC-03 | Lazy full page from remote shard |
| Block/span transclusion | UC-32 | Inline embed with origin + freshness |
| Link reference | UC-08 | Pointer only |
Define provenance display requirements and staleness semantics for UC-32.
Reference Xanadu transclusion patterns without adopting unbuilt Xanadu scope.
**Tradeoffs:** Live transclusion fragility (link rot, remote down) vs snapshot
on import; core orchestrator support vs Markdown extension + adapter.
---
## Consensus and reconciliation policy catalog
```task
id: SHARD-WP-0002-T9
status: done
priority: medium
state_hub_task_id: "adfca8b3-eb21-497d-9f51-65dc9269c810"
```
Document **configurable policy presets** (mechanism over policy):
- Fedwiki-spread (versions coexist; popularity implicit).
- Designated canonical shard (explicit write target).
- Git-merge reconciliation (ikiwiki-style).
- Overlay-only (no destructive merge on read-only sources).
- Vote-to-merge / editorial gate (optional, UI-layer).
Map presets to UC-07, UC-27. Core provides primitives; policy selects preset.
**Tradeoffs:** Default policy for L0 open mode vs team mode; exposing conflicts
vs auto-merge when backends support it.
---
## Federation operations capability matrix
```task
id: SHARD-WP-0002-T10
status: done
priority: medium
state_hub_task_id: "c7a93d06-8631-43b4-bc7f-1b0a1cd1436f"
```
Produce a capability matrix: which federation operations require which adapter
capabilities (read, write, diff, merge, lock, version, publish, notify,
transclude-source).
Cross-check against INTENT capability-aware adapters principle. Identify
graceful degradation paths for limited backends (read-only participant,
projection-only, patch target).
This matrix **consumes the capability vocabulary defined in T11** — keep the two
in sync. Feeds `spec/TechnicalSpecificationDocument.md` adapter contract section.
---
# Shard adapter contract (folded from engine deep dives, 2026-06-13)
These tasks distill the cross-cutting **adapter-contract constraints** surfaced by
the four engine deep dives (wiki-engines landscape, XWiki, TWiki, Foswiki). They
answer *what a backend must expose to participate as a shard*, complementing the
federation tasks above (which answer *what the union does*). Output target:
`spec/TechnicalSpecificationDocument.md` (adapter contract section), cross-linked
from `spec/FederationArchitecture.md`.
Recurring cross-engine insight to honor: **structure and history in text files
federate far more easily than in a database**, and **Foswiki::Store already proves
the versioned-interface-with-swappable-backends pattern** the contract should adopt.
## Adapter contract: capability model & versioned interface
```task
id: SHARD-WP-0002-T11
status: done
priority: high
state_hub_task_id: "a7379621-6694-488b-94ca-846b8e27e745"
```
Define the **capability-aware shard adapter contract** as a *versioned* interface,
taking `Foswiki::Store` + `Foswiki::Meta` as direct prior art (a real engine that
already separates store backend from core behind a stable interface).
- **Operation verbs:** `read, write, diff, merge, lock, version, publish, notify,
transclude-source, translate-syntax, structured-payload, derive-projection,
execute/evaluate` (reconcile with T10; `derive-projection`/`execute` are gated, T18).
- **Capability profile = the fifteen spectra** (synthesis v3 §2 + SHARD-WP-0004), each a
*position* not a boolean, so federation ops degrade by position:
1. addressing granularity (none → path → page-level store id → in-file span → in-file
block `id::` → store-UUID → portable tumbler)
2. content identity (none → path/title → fingerprint → span-set)
3. identity vs placement (path=identity → **identity separated from placement**:
Trilium note/branch; a page in many locations = a DAG)
4. structure (flat MD → frontmatter/`key::` → %META% → typed objects → DB
schema+relations → object-graph/ontology → **computed: inherited+templated**)
5. history (none → internal-only / **CRDT-log** → open-file → git-native)
6. **merge model** (none → git/text → conflict-notes/keep-both → **native-CRDT**)
7. native query (none → text → **build-your-own derived index** → datalog/graph → DB query)
8. translation (native → lossless → lossy-with-fidelity-report; incl. HTML)
9. attachment mode (file-store [native *or* interchange/sync-mirror] → in-engine-host →
local-REST → external-API → CRDT-replica → P2P/no-central-endpoint)
10. operational envelope (local/unbounded → realtime CRDT/WebSocket → rate-limited/
eventually-consistent/paginated)
11. access grant (open → token → OAuth scoped+revocable → P2P key/invite)
12. **content opacity** (plaintext → **structured re-evaluable value** [Wolfram expr] →
encrypted-at-rest whole-shard → **per-item** → proprietary-lossy-exportable)
13. write granularity (whole-file [TiddlyWiki] → per-page/note → section/anchor →
per-block → story-item)
14. **provenance granularity** (v3: per-shard → per-page → per-edit →
per-statement/value, e.g. Wikibase claim references + rank)
15. **computational / liveness** (SHARD-WP-0004: static source → captured-output snapshot
→ live-over-files → view-time render → irreducibly-live/temporal) — pairs with the
T16/T18 projection axes
- Versioning/compatibility rules for the contract itself; capability discovery
(static profile vs runtime negotiation).
**Inputs:** `260608-wikiengines-overview` §3, §5; `260613-foswiki-deep-dive` §2, §6;
`260614-shard-spectrum-synthesis` §2, §5 (**v3** — fifteen spectra incl. provenance
granularity + computational/liveness); `260614-wikibase-deep-dive` (per-statement
provenance/rank). **Provides** the vocabulary T10's matrix consumes; the computational/
execute capabilities are detailed in **T18**. **UCs:** UC-02, UC-35, UC-38, UC-57, UC-61,
UC-64, UC-75, UC-83, UC-84.
**Tradeoffs:** Rich capability set (precise degradation) vs contract complexity;
static profile vs runtime capability negotiation.
---
## Page model: structured / typed payload representation
```task
id: SHARD-WP-0002-T12
status: done
priority: high
state_hub_task_id: "7334a4a4-ba75-4fac-a8b4-8350d342b299"
```
Decide how the **backend-neutral, Markdown-first page model carries structured data
without flattening**:
The model must stretch **many ways at once** (synthesis v2 §3): prose Markdown, typed
*and computed* records, non-Markdown assets, reference/query-defined pages, and
**multi-placement (DAG) identity**.
- Source shapes: XWiki XObjects against an XClass; TWiki/Foswiki DataForms stored as
`%META:FIELD%` in text; Foswiki MetaDataPlugin **multiple records per topic**;
Semantic MediaWiki annotations; **Notion databases** (schema + typed properties +
inter-record **relations** + rollups/formulas) — the apex of database-as-pages.
- Representation choice: typed frontmatter vs sidecar file vs opaque provenance blob.
Prefer **in-text structure** (frontmatter/`%META%`, git-diffable); tolerate DB
structure via sidecar + fidelity report (T15).
- **Bodiless pages** (UC-39): may a page be purely structured, no Markdown body?
- **N typed records per page** (not one form) must be representable.
- **Non-Markdown content types** (UC-55): drawings (Excalidraw), spatial canvases
(JSON Canvas), images, attachments — typed asset vs opaque blob vs content-type
registry, with provenance, not flattened away.
- **Inter-record relations** (UC-58): represented as typed links in the union link
graph, a separate relation index (cf. ZigZag many-to-many), or both (T16).
- **Query-defined / reference pages** (UC-44, UC-54): a page whose canonical form is a
span manifest (Xanadu EDL) or a saved query (Dataview/Notion linked DB) — coordinate
with T16.
- **Computed metadata** (UC-67): Trilium attributes are **inherited (down the tree) +
templated**, so effective metadata = own + inherited + templated. Represent
**effective-vs-own with per-attribute provenance** (own / inherited-from / template);
decide materialize (snapshot) vs compute-live.
- **Multi-placement / DAG identity** (UC-66): Trilium **note (identity) vs branch
(placement)** — a page may occupy several locations at once. Separate **page identity
from placement** (one entity, N placements/paths/shards); no single canonical path —
coordinate with T16 (the namespace-level clone/reference primitive).
- **Typed-graph statements** (UC-73): Wikibase claim + qualifiers + references + rank — the
structure far-end (a true knowledge graph), beyond Notion DB / XWiki XObjects.
- **Computational page shapes** (UC-83, UC-84 — detailed in **T18**): the model must also
carry (a) **one-source-many-projections** (literate source → co-equal derived views, UC-83);
(b) **notebook** — **ordered/nestable cells** where code cells own **embedded computed
outputs** (derived output stored *inside* the source) with weak provenance, outputs being
MIME blobs *or* **structured re-evaluable values** (UC-84); (c) **program-as-page** (source
text, no cached output); (d) **live/temporal content** (source + marked recording). All
reduce to *(source, derivation rule, projection with provenance + liveness)*.
**Inputs:** `xwiki` §2.3; `twiki` §2.3; `foswiki` §4; `notion` §2§3; `obsidian` §3;
`trilium` §2§3; `localfirst-workspaces` §1§3; `wikibase` §1; `literate-programming`;
`jupyter`; `mathematica`; `shard-spectrum-synthesis` §3 (v3);
`computational-page-model-synthesis` §3. **UCs:** UC-34, UC-39, UC-55, UC-58, UC-54, UC-44,
UC-66, UC-67, UC-73, UC-83, UC-84.
**Tradeoffs:** Lossless fidelity vs a uniform queryable model; diff/search over
structured fields vs prose; one coherent model vs many content shapes; effective-vs-own
metadata snapshot vs live computation.
---
## History portability: supplement vs import
```task
id: SHARD-WP-0002-T13
status: done
priority: high
state_hub_task_id: "6837862a-8f57-410d-9200-a6a5dcf1a7b9"
```
Define how the **coordination journal** relates to an engine's native history (aligns
with and refines T4):
History is a spectrum (synthesis v2 §2): `none → internal-only / CRDT-log → open-file →
git-native`.
- **Adopt** — already git-native (Git folder/repo, Obsidian-with-Git, Logseq files): use
as-is.
- **Supplement** — engine history is non-portable: DB-internal (Confluence, MediaWiki,
XWiki `xwikircs`, **Notion**, **Joplin/Trilium** revisions) or a **CRDT update log**
(Anytype/AFFiNE/AppFlowy): the journal begins now / mirrors forward / snapshots the
replica (UC-36).
- **Import** — engine history is an open file format (TWiki/yawex RCS `.txt,v`,
Foswiki PlainFile timestamped copies): backfill into Git preserving author/timestamp
(UC-41).
- **DB-version-rows** (MojoMojo) — relational version rows as a third import source beside
git commits and RCS files; **partial/truncated** flat-file history (Oddmuse `keep/`) must
be reported honestly ("history begins at", UC-24), never implied complete.
- **Embedded-output documents** (notebooks, T18) — `.ipynb` JSON diffs are noisy; record
**text-pairing (Jupytext)** / **cell-aware merge (nbdime)** and the **outputs-as-derived**
(nbstripout) stance as the history-portability strategy for source-with-embedded-output.
- Authoritative journal vs mirror-reconciled-back; one-time vs continuous.
**Inputs:** `xwiki` §2.4; `twiki` §2.1; `foswiki` §2.2; `notion` §4; `joplin` §1;
`trilium` §1; `localfirst-workspaces` §4; `mojomojo` §1, §4; `oddmuse` §2; `jupyter` §3.
**UCs:** UC-36, UC-41, UC-24.
**Tradeoffs:** Import fidelity vs effort; duplicated history vs single source of truth.
---
## Adapter binding: attach path, hosting, backend-swap
```task
id: SHARD-WP-0002-T14
status: done
priority: medium
state_hub_task_id: "f8835969-d118-4738-952a-5e67e5209f3d"
```
Decide **how and where an adapter binds to a backend**:
**Full attachment-mode taxonomy** (synthesis v2 §2, §3), with a backend possibly
offering more than one — so attach mode is a **per-binding, capability-gated choice**,
not a fixed property:
- **File-store direct** — read the on-disk store as a folder shard, in two sub-kinds:
- *native store* — Obsidian vault, **Logseq** `pages/`+`journals/` (with a format
profile parsing `id::`/`key::`), TWiki `data/`, Git repo: offline, git-native, but
risks inconsistent reads under a running engine (UC-40, UC-53, UC-62).
- *interchange / sync mirror* — a tool's documented sync representation on a third-party
target (**Joplin** items on WebDAV/Nextcloud/S3): app-independent, read/projection/
overlay-mostly (the tool owns the format; never re-sync) (UC-60).
- **In-engine hosted adapter** — adapter runs *inside* the app via its extension API
(XWiki components; TWiki/Foswiki handlers; **Roam Depot** over `roamAlphaAPI`;
**Obsidian plugin** over `App.vault`; **Logseq** plugin; **Trilium** scripting code
notes): high fidelity, needs deploy access (UC-38, UC-50).
- **Local-REST** — a localhost API served by the running app (**Joplin** Data API;
**Trilium** ETAPI): app-must-run, token auth (UC-38, links UC-57).
- **External-API only** — attach from outside via a remote REST API, no in-engine
hosting (**Notion**): full write-through without deploy access, but bounded by the
**operational envelope** (rate limit, eventual consistency, payload caps) and a
**scoped, revocable access grant** that enforces no-silent-mutation (UC-57).
- **CRDT replica** — hold a local CRDT replica (**Anytype/AFFiNE/AppFlowy**: any-sync/
Yjs/Yrs): the backend merges natively — respect CRDT semantics, don't git-merge; or a
self-host sync endpoint (AFFiNE/AppFlowy Cloud) (UC-64).
- **P2P / no-central-endpoint** — bind a replica or a named peer/node, not a URL
(**Anytype** any-sync), content possibly E2EE (UC-65).
- **Dual / multi attach** — a backend offering several modes (Obsidian/Logseq: file *or*
plugin; TWiki: file *or* API; Joplin: mirror *or* Data API): pick per binding; declare
which is authoritative (UC-53).
- **Git-IS-the-store** (forge wikis Gitea/GitLab/GitHub, ikiwiki) — the wiki is a
`.wiki.git` / source repo of Markdown; git-clone is universal and git **is** the canonical
store and journal at once (the home case; resolves the engine-mirror write-race — bind to
git, not to a DB-mirror) (UC-76, UC-79).
- **Direct-DB read** (MojoMojo) — no file store and no API; the adapter maps a relational
**schema → page model + journal** (a versioned coupling that can drift, UC-43); default
read/projection/overlay (UC-81).
- **Backend-swap / substrate-migration tolerance** — shard identity/provenance survives a
storage change (Foswiki RCS↔PlainFile; folder→Git; **Logseq file→SQLite DB-graph**) —
bind to capabilities, not to "it's files" (UC-43).
- **Boundary — image-is-not-a-store** (Squeak/Pharo, T18): a monolithic live-memory blob
(image, kernel) is **never an attach target**; it participates only via **export→files**
(a degrading derivation-projection). Generalizes "attach the files, not the kernel/image"
(notebooks UC-84, GT/Lepiter).
**Inputs:** `twiki` §5§6; `xwiki` §3; `foswiki` §2; `roam` §5; `obsidian` §4;
`notion` §4, §6; `joplin` §2, §4; `logseq` §4§5; `localfirst-workspaces` §4; `trilium`
§5; `forge-wikis`; `ikiwiki` §1; `mojomojo` §2; `squeak-pharo` §2§3;
`shard-spectrum-synthesis` §3 (v3). **UCs:** UC-38, UC-40, UC-43, UC-50, UC-53,
UC-57, UC-60, UC-62, UC-64, UC-65, UC-76, UC-79, UC-81.
**Tradeoffs:** Fidelity vs deploy access; consistency vs offline capability; external
write-through vs rate-limit/eventual-consistency ceiling; CRDT-replica liveness vs
snapshot simplicity.
---
## Syntax translation & content fidelity
```task
id: SHARD-WP-0002-T15
status: done
priority: medium
state_hub_task_id: "22b57b3a-b06b-4ff0-a34a-667a0386bf25"
```
Specify how **non-Markdown shards** participate in the Markdown-first model:
Translation is a spectrum (synthesis §2): `native → lossless → lossy-with-fidelity-report`.
- Read native markup (TWiki/Foswiki TML, XWiki syntax) into the page model and accept
Markdown overlays back via **bidirectional, lossless translation** (UC-42).
- Feasibility proof: Foswiki **WysiwygPlugin** (TML→HTML for editing, HTML→TML
losslessly on save).
- **Lossy-with-fidelity-report** (UC-59) — for models that do *not* round-trip: **Notion**
blocks/rich-text/databases (export is lossy); **Trilium** **HTML** (CKEditor — more
tractable than blocks but still lossy); CRDT/object models. Translate lossily but **make
fidelity loss visible** — a per-shard/per-page report of what projects cleanly vs.
degrades, with non-mappable elements preserved as provenance/sidecar. Fidelity becomes
data (union without erasure). Distinct from the lossless case (UC-42).
- **HTML as a source model** (Trilium) joins TML/Notion-blocks/CRDT in the translation
capability — HTML↔Markdown, lossy-aware.
- **Typed-graph & non-Markdown computed content** (T18): Wikibase statements (UC-73),
notebook MIME-bundle outputs, and `.ipynb`/`.nb` JSON are **not Markdown** — keep the
native form canonical, treat any Markdown as a lossy projection; add a **structured
re-evaluable value** point to the opacity spectrum (Wolfram expression) between
transparent-text and opaque-blob.
- **Graceful degradation:** when neither lossless nor acceptable lossy translation is
available, the shard is a read-only/projection participant (UC-03), never silently
corrupted.
- Open: may overlays be stored in Markdown, or must they round-trip in native syntax?
**Inputs:** `foswiki` §5; `twiki` §2.4; `xwiki` §2.5; `notion` §3; `trilium` §4.
**UCs:** UC-42, UC-59, UC-03.
**Tradeoffs:** Translation coverage vs read-only floor; Markdown overlays (portable)
vs native-syntax overlays (safe round-trip); lossy-but-usable vs read-only-but-faithful.
---
## Addressing, content identity, and dimensional / query navigation
```task
id: SHARD-WP-0002-T16
status: done
priority: medium
state_hub_task_id: "b00ca669-59d6-454a-8d6e-f34694e35192"
```
New thread from the conceptual (Xanadu, ZigZag) + modern-tool (Roam, Notion, Obsidian/
Dataview) dives and the synthesis (`260614-shard-spectrum-synthesis` §4). No existing
task owns the addressing/identity/navigation layer that transclusion, overlay,
equivalence, and dimensional views depend on. Refines T8 (transclusion/projection depth)
and T5 (union composition); consumes T11 capabilities.
- **Span addressing** — a portable sub-page address. Adopt **native span IDs** where the
backend mints them (Roam `:block/uid`, Notion/CRDT UUID, Joplin/Trilium page-level id)
or carries them in-file (Obsidian `^id`, **Logseq `id::` — block-level AND git-diffable,
the sweet spot**); fall back to **content fingerprint** or path+range otherwise. Open:
wrap native IDs in a shard-scoped address so they survive projection and don't collide
across shards (Xanadu tumbler is the unreached ideal). (UC-51, UC-44.)
- **Content identity** — detect that two pages are the same / derived content by
fingerprint or span-set overlap, **path-independently** — the equivalence mechanism
UC-27 left open. Enables reverse transclusion (UC-45). (UC-46.)
- **Identity vs placement** — separate **page identity from placement** (Trilium **note vs
branch**: a page cloned into many locations = a DAG). One entity, N placements (paths/
shards); no single canonical path — the namespace-level form of the clone/reference
primitive. (UC-66, UC-22.)
- **Transclusion as one reference-not-copy primitive** — unify Xanadu transclusion,
ZigZag clone, Roam/Obsidian/Logseq embed, Notion synced block, **Trilium note cloning**,
and **literate named-chunk** assembly (`<<name>>` resolved by name at derivation time,
UC-83) over the addressable union (synthesis v3 §3). Compose-by-reference pages (UC-44) and
reverse lookup (UC-45) are views over it.
- **Two-axis projection model** (T18, the computational page model): split projection into
**replication-projection** (lazy cache of remote content — the current default) vs
**derivation-projection** (transform/compile/weave/evaluate a source), with facets
*materialization timing* (ahead-of-time vs view-time), *multiplicity* (one output vs N
co-equal projections), and *continuity* (one-shot vs continuous); crossed with the
**live↔snapshot** axis (static → captured → live-over-files → view-time → irreducibly-live).
Every computed/live view declares its liveness + provenance; the far end has no faithful
static form (source + marked recording).
- **Moldable view registry** (Glamorous Toolkit) — generalize "a projection" to an **open,
type-keyed set of co-equal, possibly-computed views, none canonical** (display-canonical is
policy). Unifies replication-/derivation-/dimensional-(ZigZag)/query-(Dataview) projection
and answers UC-55's pluggable-content-type/view-registry question. (UC-47, UC-48, UC-54.)
- **Dimensional / query navigation** — model the union so each relationship (namespace,
genealogy, version, shard, equivalence, links) is a navigable **dimension** (ZigZag),
and a page can be **query-defined** (Dataview/Notion linked DB, UC-54). **Delegate**
view computation to a shard's native query engine where present (Roam/Logseq Datalog,
Notion/AppFlowy DB query, XWiki XWQL); **else build a derived index over the
projection** (Logseq DataScript-over-files pattern, UC-63). (UC-47, UC-48, UC-52,
UC-54, UC-63.)
- **Boundary:** this is a **derived lens over canonical files+journal**, never a new
store (ZigZag dive §6); the many-to-many link graph stays a graph index, not a rank.
**Inputs:** `xanadu` §2§5; `zigzag` §2§6; `roam` §2§4, §7; `notion` §2; `obsidian`
§2§3; `logseq` §1§3; `trilium` §2§3; `wikibase` (federated `SERVICE` / graph query);
`literate-programming`; `glamorous-toolkit`; `shard-spectrum-synthesis` §3§5 (v3);
`computational-page-model-synthesis` §2. **UCs:** UC-44, UC-45, UC-46, UC-47, UC-48, UC-51,
UC-52, UC-54, UC-63, UC-66, UC-74, UC-83.
**Tradeoffs:** Native-ID adoption (cheap, backend-coupled) vs a portable address scheme
(harder, uniform); core navigation API vs internal organizing concept; query delegation
(fast, backend-dependent) vs orchestrator-computed (uniform, costly).
---
# Re-folded syntheses (2026-06-15)
These two tasks re-fold the later syntheses — **synthesis v3** (the federation-model
taxonomy) and the **SHARD-WP-0004 "computational page model"** — into this workplan as
**selectable-axis** deliverables. T17 is a federation-architecture concern (cross-links
T1T6); T18 is an adapter-contract concern (cross-links T11T16).
## Federation-model taxonomy: selectable / composable federation
```task
id: SHARD-WP-0002-T17
status: done
priority: high
state_hub_task_id: "529b32d2-b681-4711-82c5-5298410cfb37"
```
Specify federation as a **plural, composable coordination-layer axis** rather than a single
model (synthesis v3 §2.5). Document each model, its capability prerequisites, and how
shard-wiki selects/composes them per information space:
| Model | Anchor system | Coordination shape | Primary UCs |
|-------|---------------|--------------------|-------------|
| **Fork + journal** | Federated Wiki | copy-with-provenance + per-page semantic-action journal (story = replay) | UC-26, UC-71, UC-72 |
| **VCS-replication + ping** | ikiwiki | git clone/pull/push + XML-RPC change-ping | UC-31, UC-33, UC-79 |
| **Query-time graph-join** | Wikibase SPARQL `SERVICE` | join remote graphs at query time, no copy | UC-74 |
| **Feed aggregation** | RSS/Atom (ikiwiki `aggregate`) | inbound feed → pages | UC-03 |
| **Activity streams** | ActivityPub | Create/Update events as notification or content-bearing share | UC-31 |
| **Engine-mirror** | Wiki.js (DB↔Git) | engine syncs its own store to a git mirror | UC-68, UC-69 |
- The **coordination journal** (T4) is the **fork+journal** model's natural home; **git IS
the journal** in VCS-replication and engine-mirror (forge wikis make git the store *and*
journal at once, resolving the engine-mirror write-race — git-IS-store, T14).
- Models are **selectable per space and composable** (e.g. fork+journal locally, query-join
for a typed-graph shard, feed-aggregation for an external source) — mechanism over policy.
- Cross-link: T1 (positioning names the taxonomy), T2 (fork is one remix primitive among
many), T4 (journal shapes), T5 (composition), T6 (notification transports).
**Inputs:** `federated-wiki` (fork/journal); `ikiwiki` §2 (VCS-replication+ping, aggregate);
`wikibase` (federated `SERVICE`); `wikijs` (engine-mirror); `260608-federation-concepts`;
`shard-spectrum-synthesis` §2.5 (v3). **UCs:** UC-26, UC-31, UC-33, UC-68, UC-69, UC-71,
UC-72, UC-74, UC-79.
**Tradeoffs:** One uniform model (simpler) vs plural/composable (faithful to heterogeneity);
query-time join freshness vs cost; activity-streams reach vs fediverse dependency.
---
## Computational / executable content capability
```task
id: SHARD-WP-0002-T18
status: done
priority: medium
state_hub_task_id: "331c2a9b-57bb-4067-8d1f-9a3de10e2873"
```
Specify how shard-wiki treats **computational / executable / live page content**, re-folded
from the SHARD-WP-0004 *computational page model* synthesis. **Headline decision to record:
computational content is IN scope as a page-model + projection concern, OUT of scope as an
execution platform.**
- **The model:** source is canonical; everything rendered/computed is a **projection** placed
on two axes (replication↔derivation × live↔snapshot, defined in T16). Four computational
page shapes (detailed in T12): one-source-many-projections (UC-83), notebook with embedded
computed-output provenance (UC-84), program-as-page, live/temporal content.
- **Recognition + projection, not execution:** shard-wiki recognizes computational types,
attaches the **canonical source**, and presents derived forms as **provenance- and
liveness-marked projections** — needing no kernel/sandbox/runtime for the base case.
- **Execution as a gated capability** (T11 `derive-projection`/`execute` verbs): driving a
derivation (tangle/weave, re-execute a notebook, render a sketch, evaluate a pattern in the
viewer) is **off by default**, carries a **trust/sandbox** sub-concern, and **degrades to a
captured snapshot / static render / recording** when absent (graceful degradation).
- **One snapshot-provenance record** reused for notebook outputs, renders, and recordings
(run id, source rev, timestamp, environment "unguaranteed") — reuses UC-84 machinery and is
consistent with partial-history honesty (UC-82).
- **Hard boundaries** (design-bugs if violated): never host a kernel/runtime as the store
(**image-is-not-a-store**, T14); never present a derivation without output→source
provenance; never imply a static view captures a live artifact (union without erasure).
- **No INTENT amendment required** — this extends the page model (T12) and projection model
(T16) within existing constraints (mechanism over policy, capability-aware, degradable).
**Inputs:** the seven SHARD-WP-0004 dives (`literate-programming`, `jupyter`, `mathematica`,
`processing`, `strudel`, `squeak-pharo`, `glamorous-toolkit`);
`computational-page-model-synthesis` §1§6. **Feeds** T11 (verbs + spectra #12/#15), T12
(page shapes), T13 (paired-text history), T14 (image-is-not-a-store), T15 (non-Markdown
computed content), T16 (two-axis projection + moldable views). **UCs:** UC-54, UC-55, UC-83,
UC-84.
**Tradeoffs:** Recognize-only (safe, limited) vs sandboxed execution (rich, complex/risky);
live render fidelity vs snapshot simplicity; per-shard execute policy vs global off-by-default.
---
## Acceptance criteria
- `spec/FederationArchitecture.md` exists with all ten federation topic sections
(T1T10) and an explicit **decisions / deferred / open** table per topic.
- The **adapter contract** section of `spec/TechnicalSpecificationDocument.md` exists,
populated by T11T16 + T18, with the capability vocabulary (T11, the **fifteen spectra**)
reconciled against the T10 federation-ops matrix.
- The **federation-model taxonomy** (T17) is documented in `spec/FederationArchitecture.md`
as a selectable/composable coordination-layer axis cross-linked from T1T6.
- The **computational page model** (T18) is recorded with its headline decision
(in scope as page-model+projection, out as execution platform; execute = gated capability).
- Each decision honors INTENT: mechanism over policy, union without erasure,
overlay before mutation, no silent remote mutation, shard sovereignty,
capability-aware adapters, Markdown-first / backend-neutral.
- UC-26UC-84 are traceable to architecture / adapter-contract sections
(UC-34UC-43 to T11T15; UC-44UC-82 to T11T17; UC-83UC-84 to T18; per
`research/260614-shard-spectrum-synthesis/findings.md` §4 (v3) and
`research/260614-computational-page-model-synthesis/findings.md` §6).
- Conflicts or dependencies on `SHARD-WP-0001` outputs are listed (e.g.
namespace model affects equivalent-page identity; page model T12 affects
resolution/overlays).
- `SHARD-WP-0001` and `SHARD-WP-0002` can proceed in parallel where topics
are independent; merge conflicts in spec are resolved before implementation
workplan starts.
## Suggested task order
**Federation architecture (T1T10):**
1. T1 positioning (frames everything)
2. T2 remix primitives + T3 equivalent identity (parallel)
3. T4 history model
4. T5 composition + T6 notification (parallel)
5. T7 lifecycle + T8 transclusion (parallel)
6. T9 policy catalog + T10 capability matrix (parallel, finalize doc)
**Adapter contract (T11T16)** — can start in parallel with T1; T11 first as it
frames the rest, and pair it with T10 (shared capability vocabulary):
7. T11 contract & the fifteen capability spectra (frames T12T16; sync with T10)
8. T12 page model + T13 history portability (parallel; T13 aligns with T4)
9. T14 binding + T15 syntax translation (parallel)
10. T16 addressing, identity & navigation + the two-axis projection model (refines T8/T5)
**Re-folded syntheses (T17T18):**
11. T17 federation-model taxonomy (with T1; cross-links T2/T4/T5/T6)
12. T18 computational / executable content (after T11/T12/T16; finalize adapter-contract spec)