generated from coulomb/repo-seed
Cross-dive synthesis (research/260614-shard-spectrum-synthesis/) reading the two Nelson conceptual systems, four engines, and three modern tools across each other: a shard family matrix and eleven capability spectra (addressing, content identity, structure, history, native query, translation, attachment mode, operational envelope, access grant, write granularity, content types) — positions anchored at both ends by a real system, federation ops degrading by position. Through-lines: files-canonical/index-derived wins; fine-grained addressing is adoptable; transclusion=clone=embed is one primitive; structure/history federate iff in-text; attach mode is a per-binding choice; Notion proves the platform can enforce no-silent-mutation. Folded into SHARD-WP-0002: T11 reframed around the eleven spectra; T12 page model stretched four ways (prose + typed records + non-Markdown assets + query-defined); T13 adds Notion supplement; T14 rewritten as the three-mode attachment taxonomy (file-store / in-engine-host / external-API); T15 adds lossy-with-fidelity-report; new T16 (addressing, content identity, dimensional/query navigation). UC coverage UC-34-43 -> UC-34-59. Workplan now 16 tasks. No new UCs (synthesis only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
613 lines
26 KiB
Markdown
613 lines
26 KiB
Markdown
---
|
||
id: SHARD-WP-0002
|
||
type: workplan
|
||
title: "federation architecture design"
|
||
domain: whynot
|
||
repo: shard-wiki
|
||
status: active
|
||
owner: tegwick
|
||
topic_slug: whynot
|
||
created: "2026-06-08"
|
||
updated: "2026-06-13"
|
||
depends_on:
|
||
- SHARD-WP-0001
|
||
state_hub_workstream_id: "2af4c46d-cbfd-40ea-a94b-d9e60b0f9945"
|
||
---
|
||
|
||
# SHARD-WP-0002 — Federation architecture design
|
||
|
||
## Goal
|
||
|
||
Produce a **federation architecture specification** for shard-wiki: positioning
|
||
against prior art (Federated Wiki, git-backed wikis, ActivityPub), documented
|
||
**decisions and tradeoffs**, and ADR-ready design notes that resolve the open
|
||
questions raised by `research/260608-federation-concepts/` and UseCaseCatalog
|
||
UC-26–UC-33.
|
||
|
||
Primary deliverable: `spec/FederationArchitecture.md` (created and filled by
|
||
this workplan's tasks).
|
||
|
||
Secondary deliverable (T11–T16): the **shard adapter contract** constraints distilled
|
||
from the engine deep dives (T11–T15, added 2026-06-13) and the modern-tool dives +
|
||
cross-dive synthesis (T16 and scope extensions, added 2026-06-14), feeding
|
||
`spec/TechnicalSpecificationDocument.md`. Federation architecture decides *what the
|
||
union does*; the adapter contract decides *what a backend must expose to participate*.
|
||
The two are co-dependent (the capability matrix T10 consumes the contract vocabulary
|
||
T11 defines), so they live in one workplan.
|
||
|
||
The cross-dive synthesis (`research/260614-shard-spectrum-synthesis/findings.md`)
|
||
reframes the contract around **eleven capability spectra** (addressing, content
|
||
identity, structure, history, native query, translation, attachment mode, operational
|
||
envelope, access grant, write granularity, content types) — positions on a spectrum
|
||
anchored at both ends by a real system, with federation ops degrading by position —
|
||
rather than a flat verb checklist.
|
||
|
||
## Context
|
||
|
||
- Federation research: `research/260608-federation-concepts/findings.md`
|
||
- Engine deep dives (adapter-contract inputs, T11–T15):
|
||
- `research/260608-wikiengines-overview/findings.md` (landscape; capability
|
||
heterogeneity, write granularity)
|
||
- `research/260613-xwiki-deep-dive/findings.md` (DB/component app-platform)
|
||
- `research/260613-twiki-deep-dive/findings.md` (file+RCS app-platform)
|
||
- `research/260613-foswiki-deep-dive/findings.md` (`Foswiki::Store` versioned
|
||
store interface = adapter-contract prior art)
|
||
- Conceptual + modern-tool dives (T16 + scope extensions, 2026-06-14):
|
||
- `research/260614-xanadu-deep-dive/findings.md` (reference-not-copy, transclusion,
|
||
tumbler span addressing — ideal)
|
||
- `research/260614-zigzag-deep-dive/findings.md` (information space as co-equal
|
||
dimensions; clone = transclusion — ideal)
|
||
- `research/260614-roam-deep-dive/findings.md` (block-DB, store UUIDs, Datalog,
|
||
in-app host)
|
||
- `research/260614-obsidian-deep-dive/findings.md` (file-over-app, in-file `^id`,
|
||
MetadataCache derived index, dual attach, ecosystem popularity)
|
||
- `research/260614-notion-deep-dive/findings.md` (closed SaaS, external-API-only,
|
||
DB schema+relations, operational envelope, scoped grant)
|
||
- `research/260614-shard-spectrum-synthesis/findings.md` (**synthesis**: the eleven
|
||
capability spectra, shard family matrix, UC→task fold-in)
|
||
- Use cases: `spec/UseCaseCatalog.md` § A (UC-26–UC-59; UC-34–UC-43 are the
|
||
engine-attachment cases, UC-44–UC-59 the conceptual + modern-tool cases)
|
||
- Aspiration: `INTENT.md` (orchestrator, not engine; mechanism over policy;
|
||
capability-aware adapters; Markdown-first, backend-neutral)
|
||
- Related workplan: `SHARD-WP-0001` (page resolution, namespaces, overlays,
|
||
provenance — federation architecture must align with, not duplicate, those
|
||
outputs)
|
||
|
||
**Non-goal:** Implement federation. This workplan produces architecture and
|
||
decision records only.
|
||
|
||
## Decision topics (overview)
|
||
|
||
| Topic | Key tradeoff | Primary UCs |
|
||
|-------|--------------|-------------|
|
||
| Orchestrator positioning | Adapter layer vs fedwiki-style homogeneous network | UC-02, UC-26 |
|
||
| Remix primitives | Fork vs overlay vs import vs link-only | UC-04, UC-26, UC-29 |
|
||
| Equivalent page identity | Title/path/alias/graph matching; chorus vs canonical | UC-27, UC-07 |
|
||
| History model | Per-shard journal vs Git coordination journal vs both | UC-29, UC-33 |
|
||
| Union composition | Server orchestrator vs client composition | UC-05, UC-27 |
|
||
| Change notification | Git ping, ActivityPub, poll — adapter transports | UC-31 |
|
||
| Space lifecycle | Permanent vs ephemeral; archive and carry-forward | UC-28, UC-30 |
|
||
| Transclusion depth | Whole-page projection vs inline span | UC-03, UC-32 |
|
||
| Consensus presets | Spread, merge, designated canonical — policy not core | UC-07, UC-27 |
|
||
| Capability matrix | Which federation ops require which adapter capabilities | UC-02–UC-07 |
|
||
|
||
### Adapter-contract topics (T11–T15, from engine deep dives)
|
||
|
||
| Topic | Key tradeoff | Primary UCs |
|
||
|-------|--------------|-------------|
|
||
| Adapter contract & capabilities | Eleven spectra (not flat verbs); write granularity; operational envelope; access grant | UC-02, UC-35, UC-38, UC-57 |
|
||
| Structured page model | Typed frontmatter vs sidecar vs blob; bodiless / multi-record; DB schema+relations; non-Markdown assets; query-defined pages | UC-34, UC-39, UC-55, UC-58, UC-54 |
|
||
| History portability | Supplement (DB-internal, incl. Notion) vs import (open file history) | UC-36, UC-41 |
|
||
| Adapter binding | Three modes (file-store / in-engine-host / external-API); per-binding choice; backend-swap; scoped grant | UC-38, UC-40, UC-43, UC-50, UC-57 |
|
||
| Syntax translation | Lossless round-trip vs **lossy-with-fidelity-report** vs read-only degradation | UC-42, UC-59, UC-03 |
|
||
| Addressing, identity & navigation (T16) | Span addressing; content identity; transclusion-as-reference; dimensional/query views | UC-44, UC-45, UC-46, UC-47, UC-48, UC-51, UC-52, UC-54 |
|
||
|
||
---
|
||
|
||
## Architecture positioning and boundaries
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T1
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "ea8fdb22-6c7f-4ac1-9799-1346abf3c3b7"
|
||
```
|
||
|
||
Write the opening sections of `spec/FederationArchitecture.md`:
|
||
|
||
- shard-wiki as **orchestration layer** over heterogeneous shards (contrast
|
||
Federated Wiki homogeneous JSON sites, ikiwiki homogeneous git wikis,
|
||
ActivityPub activity streams).
|
||
- Explicit **compare, do not equate** mapping from federation research §6.
|
||
- Architectural boundaries: what core owns vs adapters vs UI vs policy config.
|
||
- Relationship to `SHARD-WP-0001` outputs (resolution, namespaces, overlays).
|
||
|
||
**Tradeoffs to document:** Central Git coordination journal vs fully
|
||
decentralized peer sync; browser-composed union vs server-side orchestrator for
|
||
agents/CLI.
|
||
|
||
---
|
||
|
||
## Remix primitives: fork, overlay, import, reference
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T2
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "fb7d4bce-5d2e-4602-9b63-85934d90e82d"
|
||
```
|
||
|
||
Define when each remix primitive applies and how they interact:
|
||
|
||
| Primitive | Typical trigger | Writes to remote? |
|
||
|-----------|-----------------|-------------------|
|
||
| **Reference** | Link only | No |
|
||
| **Projection** | Read remote page | No (cache optional) |
|
||
| **Overlay** | Edit read-only shard | No until explicit apply |
|
||
| **Import / fork** | Copy into writable shard | Source unchanged |
|
||
|
||
Resolve federation research open question #1. Cover capability-limited shards
|
||
(read-only, no diff/merge). Map to UC-26, UC-04, UC-29.
|
||
|
||
**Tradeoffs:** Fedwiki fork-as-default vs overlay-before-mutation; copy cost
|
||
and attribution portability vs link-only federation.
|
||
|
||
---
|
||
|
||
## Equivalent page identity and multi-version presentation
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T3
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "8f2a333d-ddcc-4cc6-b6ed-1ba9b178eee3"
|
||
```
|
||
|
||
Specify how shard-wiki identifies "the same topic" across shards:
|
||
|
||
- Matching signals: normalized title, path, explicit alias table, link-graph
|
||
equivalence, manual curator binding.
|
||
- Presentation: chorus-of-voices (UC-27) vs designated canonical (policy).
|
||
- Link to UC-07 divergence detection and reconciliation triggers.
|
||
|
||
**Tradeoffs:** Automatic matching false positives vs manual curation burden;
|
||
showing all versions vs default canonical with alternates visible.
|
||
|
||
---
|
||
|
||
## History, attribution, and coordination journal
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T4
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "5f39f48d-5142-4078-a84f-3245ec1add7e"
|
||
```
|
||
|
||
Define how edit history and attribution flow across federation operations:
|
||
|
||
- Per-shard revision model (Git commit, engine history, fedwiki-style journal
|
||
where applicable).
|
||
- Information-space **coordination journal** role for cross-shard operations
|
||
(fork, import, reconcile, space branch).
|
||
- Portable attribution requirements for UC-29 (frictionless remix).
|
||
|
||
**Tradeoffs:** Fedwiki journal-per-page vs Git-only; duplication of history vs
|
||
reconstructibility from coordination journal; storage cost of embedded media on
|
||
import.
|
||
|
||
---
|
||
|
||
## Union composition layer
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T5
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "3ff71e11-d0e9-4fda-b916-d6c34c51aa51"
|
||
```
|
||
|
||
Decide where the **union view** is assembled:
|
||
|
||
- Orchestrator API (server-side graph for agents, CI, non-browser clients).
|
||
- Optional client-side composition (fedwiki-style browser pull) as a consumer
|
||
pattern, not the only path.
|
||
- Caching, freshness, and invalidation interaction with UC-03, UC-05, UC-31.
|
||
|
||
**Tradeoffs:** Single composition point (simpler provenance) vs distributed
|
||
composition (fedwiki resilience); cache staleness vs live-pull latency.
|
||
|
||
---
|
||
|
||
## Change notification and subscription transports
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T6
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "9596e5e8-8d6b-4ed4-bcbc-ebb45e3168be"
|
||
```
|
||
|
||
Specify change-notification as an **optional adapter capability**:
|
||
|
||
- Transports: git hook / pinger (ikiwiki), ActivityPub Create/Update (XWiki),
|
||
WebDAV ETag, polling fallback.
|
||
- What a subscription triggers: projection refresh, reconciliation queue,
|
||
RecentChanges union entry (UC-31, UC-17).
|
||
- Out-of-scope vs in-scope for v1.
|
||
|
||
**Tradeoffs:** Push freshness vs implementation complexity; ActivityPub as
|
||
notification-only vs content-bearing share; dependency on external fediverse
|
||
infrastructure.
|
||
|
||
---
|
||
|
||
## Information space lifecycle
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T7
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "38134064-51ce-4f5a-80bf-b2cfbe381c59"
|
||
```
|
||
|
||
Model lifecycle states for root entities / information spaces:
|
||
|
||
- Active, read-only archived, retired (detached), merged-into-successor.
|
||
- Carry-forward workflow (UC-28, UC-30): selective import from archived shard.
|
||
- Space-level fork / branch (UC-33): coordination journal + shard config branch.
|
||
|
||
**Tradeoffs:** First-class ephemeral "happenings" vs permanent spaces only;
|
||
automatic archive vs manual; whether retirement deletes projections or preserves
|
||
read-only union entries.
|
||
|
||
---
|
||
|
||
## Transclusion and projection depth
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T8
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "5607732b-612a-4550-bb17-b8cd34979cf4"
|
||
```
|
||
|
||
Distinguish projection levels:
|
||
|
||
| Level | UC | Behavior |
|
||
|-------|-----|----------|
|
||
| Whole-page projection | UC-03 | Lazy full page from remote shard |
|
||
| Block/span transclusion | UC-32 | Inline embed with origin + freshness |
|
||
| Link reference | UC-08 | Pointer only |
|
||
|
||
Define provenance display requirements and staleness semantics for UC-32.
|
||
Reference Xanadu transclusion patterns without adopting unbuilt Xanadu scope.
|
||
|
||
**Tradeoffs:** Live transclusion fragility (link rot, remote down) vs snapshot
|
||
on import; core orchestrator support vs Markdown extension + adapter.
|
||
|
||
---
|
||
|
||
## Consensus and reconciliation policy catalog
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T9
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "adfca8b3-eb21-497d-9f51-65dc9269c810"
|
||
```
|
||
|
||
Document **configurable policy presets** (mechanism over policy):
|
||
|
||
- Fedwiki-spread (versions coexist; popularity implicit).
|
||
- Designated canonical shard (explicit write target).
|
||
- Git-merge reconciliation (ikiwiki-style).
|
||
- Overlay-only (no destructive merge on read-only sources).
|
||
- Vote-to-merge / editorial gate (optional, UI-layer).
|
||
|
||
Map presets to UC-07, UC-27. Core provides primitives; policy selects preset.
|
||
|
||
**Tradeoffs:** Default policy for L0 open mode vs team mode; exposing conflicts
|
||
vs auto-merge when backends support it.
|
||
|
||
---
|
||
|
||
## Federation operations capability matrix
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T10
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "c7a93d06-8631-43b4-bc7f-1b0a1cd1436f"
|
||
```
|
||
|
||
Produce a capability matrix: which federation operations require which adapter
|
||
capabilities (read, write, diff, merge, lock, version, publish, notify,
|
||
transclude-source).
|
||
|
||
Cross-check against INTENT capability-aware adapters principle. Identify
|
||
graceful degradation paths for limited backends (read-only participant,
|
||
projection-only, patch target).
|
||
|
||
This matrix **consumes the capability vocabulary defined in T11** — keep the two
|
||
in sync. Feeds `spec/TechnicalSpecificationDocument.md` adapter contract section.
|
||
|
||
---
|
||
|
||
# Shard adapter contract (folded from engine deep dives, 2026-06-13)
|
||
|
||
These tasks distill the cross-cutting **adapter-contract constraints** surfaced by
|
||
the four engine deep dives (wiki-engines landscape, XWiki, TWiki, Foswiki). They
|
||
answer *what a backend must expose to participate as a shard*, complementing the
|
||
federation tasks above (which answer *what the union does*). Output target:
|
||
`spec/TechnicalSpecificationDocument.md` (adapter contract section), cross-linked
|
||
from `spec/FederationArchitecture.md`.
|
||
|
||
Recurring cross-engine insight to honor: **structure and history in text files
|
||
federate far more easily than in a database**, and **Foswiki::Store already proves
|
||
the versioned-interface-with-swappable-backends pattern** the contract should adopt.
|
||
|
||
## Adapter contract: capability model & versioned interface
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T11
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "a7379621-6694-488b-94ca-846b8e27e745"
|
||
```
|
||
|
||
Define the **capability-aware shard adapter contract** as a *versioned* interface,
|
||
taking `Foswiki::Store` + `Foswiki::Meta` as direct prior art (a real engine that
|
||
already separates store backend from core behind a stable interface).
|
||
|
||
- **Operation verbs:** `read, write, diff, merge, lock, version, publish, notify,
|
||
transclude-source, translate-syntax, structured-payload` (reconcile with T10).
|
||
- **Capability profile = the eleven spectra** (synthesis §2), each a *position* not a
|
||
boolean, so federation ops degrade by position:
|
||
1. addressing granularity (none → path → in-file span → store-UUID → portable tumbler)
|
||
2. content identity (none → path/title → fingerprint → span-set)
|
||
3. structure (flat MD → frontmatter → %META% → DB objects → DB schema+relations)
|
||
4. history (none → internal-only → open-file → git-native)
|
||
5. native query (none → text → datalog → DB query)
|
||
6. translation (native → lossless → lossy-with-fidelity-report)
|
||
7. attachment mode (file-store → in-engine-host → external-API)
|
||
8. operational envelope (local/unbounded → rate-limited/eventually-consistent/paginated)
|
||
9. access grant (open → token → OAuth scoped+revocable)
|
||
10. write granularity (whole-file → per-page → per-block)
|
||
11. content types (Markdown-only → typed records → non-Markdown assets)
|
||
- Versioning/compatibility rules for the contract itself; capability discovery
|
||
(static profile vs runtime negotiation).
|
||
|
||
**Inputs:** `260608-wikiengines-overview` §3, §5; `260613-foswiki-deep-dive` §2, §6;
|
||
`260614-shard-spectrum-synthesis` §2, §5. **Provides** the vocabulary T10's matrix
|
||
consumes. **UCs:** UC-02, UC-35, UC-38, UC-57.
|
||
|
||
**Tradeoffs:** Rich capability set (precise degradation) vs contract complexity;
|
||
static profile vs runtime capability negotiation.
|
||
|
||
---
|
||
|
||
## Page model: structured / typed payload representation
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T12
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "7334a4a4-ba75-4fac-a8b4-8350d342b299"
|
||
```
|
||
|
||
Decide how the **backend-neutral, Markdown-first page model carries structured data
|
||
without flattening**:
|
||
|
||
The model must stretch **four ways at once** (synthesis §3): prose Markdown, typed
|
||
records, non-Markdown assets, and reference/query-defined pages.
|
||
|
||
- Source shapes: XWiki XObjects against an XClass; TWiki/Foswiki DataForms stored as
|
||
`%META:FIELD%` in text; Foswiki MetaDataPlugin **multiple records per topic**;
|
||
Semantic MediaWiki annotations; **Notion databases** (schema + typed properties +
|
||
inter-record **relations** + rollups/formulas) — the apex of database-as-pages.
|
||
- Representation choice: typed frontmatter vs sidecar file vs opaque provenance blob.
|
||
Prefer **in-text structure** (frontmatter/`%META%`, git-diffable); tolerate DB
|
||
structure via sidecar + fidelity report (T15).
|
||
- **Bodiless pages** (UC-39): may a page be purely structured, no Markdown body?
|
||
- **N typed records per page** (not one form) must be representable.
|
||
- **Non-Markdown content types** (UC-55): drawings (Excalidraw), spatial canvases
|
||
(JSON Canvas), images, attachments — typed asset vs opaque blob vs content-type
|
||
registry, with provenance, not flattened away.
|
||
- **Inter-record relations** (UC-58): represented as typed links in the union link
|
||
graph, a separate relation index (cf. ZigZag many-to-many), or both (T16).
|
||
- **Query-defined / reference pages** (UC-44, UC-54): a page whose canonical form is a
|
||
span manifest (Xanadu EDL) or a saved query (Dataview/Notion linked DB) — coordinate
|
||
with T16.
|
||
|
||
**Inputs:** `xwiki` §2.3; `twiki` §2.3; `foswiki` §4; `notion` §2–§3; `obsidian` §3;
|
||
`shard-spectrum-synthesis` §3. **UCs:** UC-34, UC-39, UC-55, UC-58, UC-54, UC-44.
|
||
|
||
**Tradeoffs:** Lossless fidelity vs a uniform queryable model; diff/search over
|
||
structured fields vs prose; one coherent model vs four content shapes.
|
||
|
||
---
|
||
|
||
## History portability: supplement vs import
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T13
|
||
status: todo
|
||
priority: high
|
||
state_hub_task_id: "6837862a-8f57-410d-9200-a6a5dcf1a7b9"
|
||
```
|
||
|
||
Define how the **coordination journal** relates to an engine's native history (aligns
|
||
with and refines T4):
|
||
|
||
History is a spectrum (synthesis §2): `none → internal-only → open-file → git-native`.
|
||
|
||
- **Adopt** — already git-native (Git folder/repo, Obsidian-with-Git): use as-is.
|
||
- **Supplement** — engine history is DB-internal and non-portable
|
||
(Confluence, MediaWiki, XWiki `xwikircs`, **Notion** internal page history): the
|
||
journal begins now / mirrors forward (UC-36).
|
||
- **Import** — engine history is an open file format (TWiki/yawex RCS `.txt,v`,
|
||
Foswiki PlainFile timestamped copies): backfill into Git preserving author/timestamp
|
||
(UC-41).
|
||
- Authoritative journal vs mirror-reconciled-back; one-time vs continuous.
|
||
|
||
**Inputs:** `xwiki` §2.4; `twiki` §2.1; `foswiki` §2.2; `notion` §4. **UCs:** UC-36, UC-41.
|
||
|
||
**Tradeoffs:** Import fidelity vs effort; duplicated history vs single source of truth.
|
||
|
||
---
|
||
|
||
## Adapter binding: attach path, hosting, backend-swap
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T14
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "f8835969-d118-4738-952a-5e67e5209f3d"
|
||
```
|
||
|
||
Decide **how and where an adapter binds to a backend**:
|
||
|
||
**Three attachment modes** (synthesis §2, §3), with a backend possibly offering more
|
||
than one — so attach mode is a **per-binding, capability-gated choice**, not a fixed
|
||
property:
|
||
|
||
- **File-store direct** — read the on-disk store as a folder shard (Obsidian vault,
|
||
TWiki `data/`, DokuWiki, Git repo): offline, git-native, but risks inconsistent reads
|
||
under a running engine (UC-40, UC-53).
|
||
- **In-engine hosted adapter** — adapter runs *inside* the engine via its extension API
|
||
(XWiki components/REST/UIX; TWiki/Foswiki plugin handlers; **Roam Depot**
|
||
`onload/onunload` over `roamAlphaAPI`; **Obsidian plugin** over `App.vault`): high
|
||
fidelity, needs deploy access (UC-38, UC-50).
|
||
- **External-API only** — attach from outside via the backend's REST API, no in-engine
|
||
hosting (**Notion**): full write-through without deploy access, but bounded by the
|
||
**operational envelope** (rate limit, eventual consistency, payload caps) and a
|
||
**scoped, revocable access grant** that enforces no-silent-mutation (UC-57).
|
||
- **Dual attach** — a backend offering several modes (Obsidian: file-store *or* plugin;
|
||
TWiki: file *or* API): pick per binding; declare which is authoritative (UC-53).
|
||
- **Backend-swap tolerance** — shard identity/provenance survives a storage-format
|
||
change (Foswiki RCS↔PlainFile; folder→Git) (UC-43).
|
||
|
||
**Inputs:** `twiki` §5, §6; `xwiki` §3; `foswiki` §2; `roam` §5; `obsidian` §4;
|
||
`notion` §4, §6; `shard-spectrum-synthesis` §3. **UCs:** UC-38, UC-40, UC-43, UC-50, UC-53, UC-57.
|
||
|
||
**Tradeoffs:** Fidelity vs deploy access; consistency vs offline capability; external
|
||
write-through vs rate-limit/eventual-consistency ceiling.
|
||
|
||
---
|
||
|
||
## Syntax translation & content fidelity
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T15
|
||
status: todo
|
||
priority: medium
|
||
state_hub_task_id: "22b57b3a-b06b-4ff0-a34a-667a0386bf25"
|
||
```
|
||
|
||
Specify how **non-Markdown shards** participate in the Markdown-first model:
|
||
|
||
Translation is a spectrum (synthesis §2): `native → lossless → lossy-with-fidelity-report`.
|
||
|
||
- Read native markup (TWiki/Foswiki TML, XWiki syntax) into the page model and accept
|
||
Markdown overlays back via **bidirectional, lossless translation** (UC-42).
|
||
- Feasibility proof: Foswiki **WysiwygPlugin** (TML→HTML for editing, HTML→TML
|
||
losslessly on save).
|
||
- **Lossy-with-fidelity-report** (UC-59) — for proprietary models that do *not*
|
||
round-trip (**Notion** blocks/rich-text/databases; its own Markdown export is lossy):
|
||
translate lossily but **make fidelity loss visible** — a per-shard/per-page report of
|
||
what projects cleanly vs. degrades, with non-mappable elements preserved as
|
||
provenance/sidecar. Fidelity becomes data (union without erasure). Distinct from the
|
||
lossless case (UC-42).
|
||
- **Graceful degradation:** when neither lossless nor acceptable lossy translation is
|
||
available, the shard is a read-only/projection participant (UC-03), never silently
|
||
corrupted.
|
||
- Open: may overlays be stored in Markdown, or must they round-trip in native syntax?
|
||
|
||
**Inputs:** `foswiki` §5; `twiki` §2.4; `xwiki` §2.5; `notion` §3. **UCs:** UC-42, UC-59, UC-03.
|
||
|
||
**Tradeoffs:** Translation coverage vs read-only floor; Markdown overlays (portable)
|
||
vs native-syntax overlays (safe round-trip); lossy-but-usable vs read-only-but-faithful.
|
||
|
||
---
|
||
|
||
## Addressing, content identity, and dimensional / query navigation
|
||
|
||
```task
|
||
id: SHARD-WP-0002-T16
|
||
status: todo
|
||
priority: medium
|
||
```
|
||
|
||
New thread from the conceptual (Xanadu, ZigZag) + modern-tool (Roam, Notion, Obsidian/
|
||
Dataview) dives and the synthesis (`260614-shard-spectrum-synthesis` §4). No existing
|
||
task owns the addressing/identity/navigation layer that transclusion, overlay,
|
||
equivalence, and dimensional views depend on. Refines T8 (transclusion/projection depth)
|
||
and T5 (union composition); consumes T11 capabilities.
|
||
|
||
- **Span addressing** — a portable sub-page address. Adopt **native span IDs** where the
|
||
backend mints them (Roam `:block/uid`, Notion UUID) or carries them in-file (Obsidian
|
||
`^id`); fall back to **content fingerprint** or path+range otherwise. Open: wrap native
|
||
IDs in a shard-scoped address so they survive projection and don't collide across
|
||
shards (Xanadu tumbler is the unreached ideal). (UC-51, UC-44.)
|
||
- **Content identity** — detect that two pages are the same / derived content by
|
||
fingerprint or span-set overlap, **path-independently** — the equivalence mechanism
|
||
UC-27 left open. Enables reverse transclusion (UC-45). (UC-46.)
|
||
- **Transclusion as one reference-not-copy primitive** — unify Xanadu transclusion,
|
||
ZigZag clone, Roam/Obsidian embed, Notion synced block over the addressable union
|
||
(synthesis §3). Compose-by-reference pages (UC-44) and reverse lookup (UC-45) are
|
||
views over it.
|
||
- **Dimensional / query navigation** — model the union so each relationship (namespace,
|
||
genealogy, version, shard, equivalence, links) is a navigable **dimension** (ZigZag),
|
||
and a page can be **query-defined** (Dataview/Notion linked DB, UC-54). **Delegate**
|
||
view computation to a shard's native query engine where present (Roam Datalog, Notion
|
||
DB query, XWiki XWQL); else compute over the projection. (UC-47, UC-48, UC-52, UC-54.)
|
||
- **Boundary:** this is a **derived lens over canonical files+journal**, never a new
|
||
store (ZigZag dive §6); the many-to-many link graph stays a graph index, not a rank.
|
||
|
||
**Inputs:** `xanadu` §2–§5; `zigzag` §2–§6; `roam` §2–§4, §7; `notion` §2; `obsidian`
|
||
§2–§3; `shard-spectrum-synthesis` §3–§5. **UCs:** UC-44, UC-45, UC-46, UC-47, UC-48,
|
||
UC-51, UC-52, UC-54.
|
||
|
||
**Tradeoffs:** Native-ID adoption (cheap, backend-coupled) vs a portable address scheme
|
||
(harder, uniform); core navigation API vs internal organizing concept; query delegation
|
||
(fast, backend-dependent) vs orchestrator-computed (uniform, costly).
|
||
|
||
---
|
||
|
||
## Acceptance criteria
|
||
|
||
- `spec/FederationArchitecture.md` exists with all ten federation topic sections
|
||
(T1–T10) and an explicit **decisions / deferred / open** table per topic.
|
||
- The **adapter contract** section of `spec/TechnicalSpecificationDocument.md` exists,
|
||
populated by T11–T16, with the capability vocabulary (T11, the **eleven spectra**)
|
||
reconciled against the T10 federation-ops matrix.
|
||
- Each decision honors INTENT: mechanism over policy, union without erasure,
|
||
overlay before mutation, no silent remote mutation, shard sovereignty,
|
||
capability-aware adapters, Markdown-first / backend-neutral.
|
||
- UC-26–UC-59 are traceable to architecture / adapter-contract sections
|
||
(UC-34–UC-43 to T11–T15; UC-44–UC-59 to T11–T16 per
|
||
`research/260614-shard-spectrum-synthesis/findings.md` §4).
|
||
- Conflicts or dependencies on `SHARD-WP-0001` outputs are listed (e.g.
|
||
namespace model affects equivalent-page identity; page model T12 affects
|
||
resolution/overlays).
|
||
- `SHARD-WP-0001` and `SHARD-WP-0002` can proceed in parallel where topics
|
||
are independent; merge conflicts in spec are resolved before implementation
|
||
workplan starts.
|
||
|
||
## Suggested task order
|
||
|
||
**Federation architecture (T1–T10):**
|
||
1. T1 positioning (frames everything)
|
||
2. T2 remix primitives + T3 equivalent identity (parallel)
|
||
3. T4 history model
|
||
4. T5 composition + T6 notification (parallel)
|
||
5. T7 lifecycle + T8 transclusion (parallel)
|
||
6. T9 policy catalog + T10 capability matrix (parallel, finalize doc)
|
||
|
||
**Adapter contract (T11–T16)** — can start in parallel with T1; T11 first as it
|
||
frames the rest, and pair it with T10 (shared capability vocabulary):
|
||
7. T11 contract & the eleven capability spectra (frames T12–T16; sync with T10)
|
||
8. T12 page model + T13 history portability (parallel; T13 aligns with T4)
|
||
9. T14 binding + T15 syntax translation (parallel)
|
||
10. T16 addressing, identity & navigation (refines T8/T5; finalize adapter-contract spec) |