From d65f9e21f3ba96246e63d20d41c163bbd067474c Mon Sep 17 00:00:00 2001 From: tegwick Date: Mon, 15 Jun 2026 02:25:50 +0200 Subject: [PATCH] =?UTF-8?q?spec(SHARD-WP-0002):=20adapter=20contract=20(TS?= =?UTF-8?q?D=20=C2=A7A,=20T11-T16+T18);=20workplan=20done?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the normative Shard Adapter Contract as TechnicalSpecificationDocument §A: A.1 versioned capability contract (verbs + orthogonal-core spectra), A.2 verified conformance suite (profiles not self-asserted), A.3 attachment-mode taxonomy + image-is-not-a-store boundary, A.4 page model (incl. computational shapes, identity/placement/equivalence, layered provenance), A.5 history portability, A.6 syntax translation + fidelity report, A.7 addressing/navigation, A.8 gated computational content. Updates TSD references/UC-count/next-work. Flips all 18 WP-0002 tasks + workplan done. Design layer complete. Co-Authored-By: Claude Opus 4.8 --- SCOPE.md | 2 +- spec/README.md | 3 +- spec/TechnicalSpecificationDocument.md | 140 +++++++++++++++++- .../SHARD-WP-0002-federation-architecture.md | 38 ++--- 4 files changed, 157 insertions(+), 26 deletions(-) diff --git a/SCOPE.md b/SCOPE.md index d526bd1..82035f0 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -22,7 +22,7 @@ Learnings update both SCOPE and INTENT where necessary. | Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | | Spec | CoreArchitectureBlueprint (whole-system architecture, hardened via SHARD-WP-0005) + ArchitectureBlueprint (auth/history) drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds | -| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` active (18 tasks: T1–T10 federation + T11–T16 adapter contract + T17 federation-model taxonomy + T18 computational content, re-folded from synthesis v3 + the computational page model); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) | +| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` **done** (18 tasks → `FederationArchitecture.md` [T1–T10, T17] + `TechnicalSpecificationDocument.md` §A adapter contract [T11–T16, T18]); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) | ## In Scope (today) diff --git a/spec/README.md b/spec/README.md index a4134e6..546808c 100644 --- a/spec/README.md +++ b/spec/README.md @@ -7,9 +7,10 @@ Background on document types: InfoTechPrimers on coulomb.social. | File | Status | Role | |------|--------|------| | `CoreArchitectureBlueprint.md` | draft for review | **Whole-system architecture** — layers, abstractions, load-bearing decisions (synthesised from all research) | +| `FederationArchitecture.md` | draft for review | federation design — *what the union does*: T1–T10 decision records + the federation-model taxonomy (SHARD-WP-0002) | | `FederationRequirements.md` | draft for review | yawex-derived union/federation design notes — resolution, namespace, derived views, provenance, overlay, links (ADR-01…06; SHARD-WP-0001) | | `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver | -| `TechnicalSpecificationDocument.md` | draft scaffold | How the system is built | +| `TechnicalSpecificationDocument.md` | draft + §A | How the system is built; **§A = the normative shard adapter contract** (T11–T16, T18; SHARD-WP-0002) | | `UseCaseCatalog.md` | draft | 84 use cases promoted from c2 + yawex + ~23-system research | | `ArchitectureBlueprint.md` | draft | Access, history, and identity sub-blueprint (the L0–L4 authorization ladder; referenced by CoreArchitectureBlueprint §9) | diff --git a/spec/TechnicalSpecificationDocument.md b/spec/TechnicalSpecificationDocument.md index 976c444..1a85a7c 100644 --- a/spec/TechnicalSpecificationDocument.md +++ b/spec/TechnicalSpecificationDocument.md @@ -33,7 +33,134 @@ per information space. ## 4. Architecture References -- `spec/ArchitectureBlueprint.md` — access, history, identity delegation +- `spec/CoreArchitectureBlueprint.md` — whole-system architecture (layers, the dual narrow + waist, the 15 capability spectra, projection, consistency); the normative source for §A. +- `spec/FederationArchitecture.md` — federation design (*what the union does*); §A is its + companion (*what a backend must expose*). +- `spec/FederationRequirements.md` — yawex-derived ADRs (resolution, page model, overlay). +- `spec/ArchitectureBlueprint.md` — access, history, identity delegation (L5). + +--- + +# A. Shard Adapter Contract (normative) + +Deliverable of **SHARD-WP-0002** (T11–T16, T18): the versioned interface a backend implements +to participate as a shard, and the *normative* rules core relies on. It is the **bottom narrow +waist** (blueprint §6); the page model is the top waist (§7). This section is normative where it +says **MUST/SHOULD**; design rationale lives in the blueprint (cited, not restated). + +## A.1 (T11) Capability model & versioned interface + +- The contract is a **versioned interface** (`Foswiki::Store`/`Foswiki::Meta` is the proof a + swappable-backend-behind-a-stable-interface works). A binding declares the contract version + it implements; core checks compatibility at registration. +- **Operation verbs:** `read, write, diff, merge, lock, version, publish, notify, + transclude-source, translate-syntax, structured-payload, derive-projection, execute`. The + last two are **gated, OFF by default** (A.6/T18). Verb support is part of the profile and MUST + reconcile with the T10 federation-ops matrix. +- **Capability profile = a position on each capability spectrum**, not a boolean checklist. The + full set is the **fifteen spectra** (blueprint §6.1); operationally they reduce to a **small + orthogonal core** (substrate, write-granularity, content-opacity, operational-envelope, + access-grant, computational/liveness) with the rest **implied** via published rules that + **forbid impossible profiles** (blueprint §6.5). Degradation reads only the **named + axis-interaction subset** (blueprint §6.5 table) — that table *is* the degradation contract. +- A profile MUST express **absence** cleanly (the Oddmuse floor); core never guesses a missing + capability. + +## A.2 (T11/§6.6) Conformance — profiles are verified, not asserted + +- The contract ships a **versioned conformance suite**. A binding is **admissible only if it + passes**: the suite exercises each declared verb and spectrum position and checks observed + behaviour matches the claim (a `write` round-trips, `notify` actually fires, an opaque shard + refuses plaintext query, implied-position rules hold). A lying/buggy profile is **rejected at + registration**, not run in production (blueprint §6.6). +- Conformance makes capability-as-data (I-3) and the §6.5 degradation contract **sound**. + Mismatch is reported as a capability-by-capability diff; an adapter may register + *degraded-but-honest* (drop the unsupported claim). + +## A.3 (T14) Adapter binding — attachment-mode taxonomy + +A backend MAY offer several modes; attach mode is a **per-binding, capability-gated choice** +with one declared authoritative (blueprint §6.3): + +- **file-store** (native vault/folder *or* an interchange/sync mirror) · **git-IS-store** (forge + wikis & ikiwiki — git is store *and* journal; the home case) · **in-engine-hosted** (XWiki + component, Obsidian/Logseq/Roam plugin, Trilium script) · **local-REST** (Joplin Data API, + Trilium ETAPI) · **external-API-only** (Notion) · **direct-DB** (MojoMojo schema→model) · + **CRDT-replica** (Anytype/AFFiNE/AppFlowy) · **P2P / no-central-endpoint**. +- **Backend-swap tolerance:** identity/provenance MUST survive a substrate change (bind to + capabilities, not "it's files"). **Boundary:** an **image / live-memory blob is never an + attach target** (I-12) — participation is export→files only. +- (UC-38, UC-40, UC-43, UC-50, UC-53, UC-57, UC-60, UC-62, UC-64, UC-65, UC-76, UC-79, UC-81.) + +## A.4 (T12) Page model — structured / computational payload + +The backend-neutral, Markdown-first page model MUST carry, without lossy flattening, every +shape in blueprint §7.1: prose; typed/computed records (incl. effective-vs-own computed +metadata); typed-graph statements (Wikibase); inline-embedded objects (Quip/Notion); +non-Markdown assets (typed asset / opaque blob / content-type registry); and the **four +computational shapes** (one-source-many-projections UC-83; notebook-with-embedded-output UC-84; +program-as-page; live/temporal). All reduce to `(content|source, structure, provenance +envelope, [derivation rule])`. **Identity is a stable handle; placement is separate; equivalence +is fingerprint-based** (blueprint §7.2, FederationRequirements ADR-01/02, I-9). The **provenance +envelope is layered** (page + span deltas; effective = page ⊕ delta, §7.3). (UC-34, UC-39, +UC-55, UC-58, UC-54, UC-44, UC-66, UC-67, UC-73, UC-83, UC-84.) + +## A.5 (T13) History portability — adopt / supplement / import + +Per the history axis: **adopt** git-native history as-is; **supplement** non-portable internal +history (DB / Notion / Joplin / Trilium revisions, CRDT-log) — the journal begins-now / mirrors +/ snapshots; **import** open file history (RCS, PlainFile, MojoMojo DB-version-rows) backfilled +preserving author/timestamp. **Partial/truncated history MUST be reported honestly** ("history +begins at", UC-24), never implied complete. Embedded-output documents (notebooks) use +paired-text (Jupytext) / cell-aware merge (nbdime). The space's own coordination history is the +event-sourced decision log (blueprint §8.1). (UC-36, UC-41, UC-24.) + +## A.6 (T15) Syntax translation & content fidelity + +Translation is a spectrum: **native → lossless → lossy-with-fidelity-report**. Read native +markup (TML, XWiki syntax, HTML) into the page model; accept Markdown overlays back via +**lossless bidirectional translation** where possible (Foswiki WysiwygPlugin is the proof). For +non-round-tripping models (Notion blocks, Trilium HTML, CRDT, typed-graph, notebook JSON/MIME): +translate lossily but **make the fidelity loss visible** — a per-shard/per-page report of what +projects cleanly vs degrades, with non-mappable elements preserved as provenance/sidecar (I-4). +Add a **structured-re-evaluable-value** point to the opacity spectrum (Wolfram expression). +Where no acceptable translation exists, the shard is a **read-only/projection** participant +(I-8), never silently corrupted. (UC-42, UC-59, UC-03, UC-73.) + +## A.7 (T16) Addressing, identity & navigation + +- **Span addressing:** adopt native span IDs where minted (Roam `:block/uid`, Logseq `id::`, + Notion/CRDT UUID), shard-scoped so they survive projection and don't collide; else a + position address (path+range) or content-fingerprint address. Portable tumbler is the ideal + (blueprint O-6). +- **Transclusion** = one reference-not-copy primitive over the addressable union (FederationArch + T8). **Query/navigation:** delegate to a shard's native query where capable (Datalog, DB + query, XWQL, SPARQL), **else build a derived index over the projection** (Logseq pattern); + dimensional/query-defined views are derived-tier. (UC-44–48, UC-51, UC-52, UC-54, UC-63, + UC-74.) + +## A.8 (T18) Computational / executable content capability + +**In scope as a page-model + projection concern; out of scope as an execution platform** +(blueprint §8.5). Core **recognises** computational types, attaches the **canonical source**, +and presents derived forms as **provenance- and liveness-marked projections**. `derive- +projection`/`execute` are **gated capabilities, OFF by default**, carrying a **trust/sandbox** +concern, **degrading to a captured snapshot / static render / recording**. One snapshot- +provenance record (run id, source rev, timestamp, environment "unguaranteed") serves notebooks, +renders, recordings. No INTENT amendment required. (UC-54, UC-55, UC-83, UC-84.) + +## A.9 Conformance & module mapping + +The contract maps to `src/shard_wiki/adapters/` (the bottom waist: `AdapterContract`, +`CapabilityProfile`, attachment-mode binding, the conformance suite) consuming `model/` +(page model + capability value types) and `provenance/` (blueprint §11). Each concrete adapter +ships its declared profile + a conformance pass. + +*Decided:* A.1–A.8 (versioned capability contract, verified conformance, binding taxonomy, +page model, history portability, translation, addressing, gated computational content). +*Deferred:* per-backend adapter specs (one per backend, later). *Open:* blueprint §12 O-items +(addressing O-6, axis interactions O-5, span-authz O-10). ## 5. Integration Boundaries @@ -47,10 +174,13 @@ Package scaffold only (`__version__`, smoke tests). Domain model not yet coded. ## 7. Use Cases -`spec/UseCaseCatalog.md` — 25 use cases (UC-01–UC-25) promoted from c2 wiki -origins and yawex prior-art research. +`spec/UseCaseCatalog.md` — 84 use cases (UC-01–UC-84) from c2/yawex origins, +the wiki-engine + modern-tool + computational research, and the syntheses. ## 8. Next Specification Work -Outputs from `SHARD-WP-0001` tasks (page resolution, namespaces, derived views, -provenance, overlays, link semantics) will be incorporated here as they complete. \ No newline at end of file +The design layer is complete: `SHARD-WP-0001` (→ `FederationRequirements.md`), +`SHARD-WP-0002` (→ `FederationArchitecture.md` + §A above), and the hardened +`CoreArchitectureBlueprint.md`. The next workplan is the **implementation** of the +domain model + adapter contract (starting from §A and blueprint §11's module layout), +likely with a first spike on the keystone (the event-sourced decision log + derived fold). \ No newline at end of file diff --git a/workplans/SHARD-WP-0002-federation-architecture.md b/workplans/SHARD-WP-0002-federation-architecture.md index eae18fc..83306b3 100644 --- a/workplans/SHARD-WP-0002-federation-architecture.md +++ b/workplans/SHARD-WP-0002-federation-architecture.md @@ -4,7 +4,7 @@ type: workplan title: "federation architecture design" domain: whynot repo: shard-wiki -status: active +status: done owner: tegwick topic_slug: whynot created: "2026-06-08" @@ -151,7 +151,7 @@ decision records only. ```task id: SHARD-WP-0002-T1 -status: todo +status: done priority: high state_hub_task_id: "ea8fdb22-6c7f-4ac1-9799-1346abf3c3b7" ``` @@ -175,7 +175,7 @@ agents/CLI. ```task id: SHARD-WP-0002-T2 -status: todo +status: done priority: high state_hub_task_id: "fb7d4bce-5d2e-4602-9b63-85934d90e82d" ``` @@ -201,7 +201,7 @@ and attribution portability vs link-only federation. ```task id: SHARD-WP-0002-T3 -status: todo +status: done priority: high state_hub_task_id: "8f2a333d-ddcc-4cc6-b6ed-1ba9b178eee3" ``` @@ -222,7 +222,7 @@ showing all versions vs default canonical with alternates visible. ```task id: SHARD-WP-0002-T4 -status: todo +status: done priority: high state_hub_task_id: "5f39f48d-5142-4078-a84f-3245ec1add7e" ``` @@ -245,7 +245,7 @@ import. ```task id: SHARD-WP-0002-T5 -status: todo +status: done priority: medium state_hub_task_id: "3ff71e11-d0e9-4fda-b916-d6c34c51aa51" ``` @@ -266,7 +266,7 @@ composition (fedwiki resilience); cache staleness vs live-pull latency. ```task id: SHARD-WP-0002-T6 -status: todo +status: done priority: medium state_hub_task_id: "9596e5e8-8d6b-4ed4-bcbc-ebb45e3168be" ``` @@ -289,7 +289,7 @@ infrastructure. ```task id: SHARD-WP-0002-T7 -status: todo +status: done priority: medium state_hub_task_id: "38134064-51ce-4f5a-80bf-b2cfbe381c59" ``` @@ -310,7 +310,7 @@ read-only union entries. ```task id: SHARD-WP-0002-T8 -status: todo +status: done priority: medium state_hub_task_id: "5607732b-612a-4550-bb17-b8cd34979cf4" ``` @@ -335,7 +335,7 @@ on import; core orchestrator support vs Markdown extension + adapter. ```task id: SHARD-WP-0002-T9 -status: todo +status: done priority: medium state_hub_task_id: "adfca8b3-eb21-497d-9f51-65dc9269c810" ``` @@ -359,7 +359,7 @@ vs auto-merge when backends support it. ```task id: SHARD-WP-0002-T10 -status: todo +status: done priority: medium state_hub_task_id: "c7a93d06-8631-43b4-bc7f-1b0a1cd1436f" ``` @@ -394,7 +394,7 @@ the versioned-interface-with-swappable-backends pattern** the contract should ad ```task id: SHARD-WP-0002-T11 -status: todo +status: done priority: high state_hub_task_id: "a7379621-6694-488b-94ca-846b8e27e745" ``` @@ -452,7 +452,7 @@ static profile vs runtime capability negotiation. ```task id: SHARD-WP-0002-T12 -status: todo +status: done priority: high state_hub_task_id: "7334a4a4-ba75-4fac-a8b4-8350d342b299" ``` @@ -515,7 +515,7 @@ metadata snapshot vs live computation. ```task id: SHARD-WP-0002-T13 -status: todo +status: done priority: high state_hub_task_id: "6837862a-8f57-410d-9200-a6a5dcf1a7b9" ``` @@ -555,7 +555,7 @@ git-native`. ```task id: SHARD-WP-0002-T14 -status: todo +status: done priority: medium state_hub_task_id: "f8835969-d118-4738-952a-5e67e5209f3d" ``` @@ -622,7 +622,7 @@ snapshot simplicity. ```task id: SHARD-WP-0002-T15 -status: todo +status: done priority: medium state_hub_task_id: "22b57b3a-b06b-4ff0-a34a-667a0386bf25" ``` @@ -665,7 +665,7 @@ vs native-syntax overlays (safe round-trip); lossy-but-usable vs read-only-but-f ```task id: SHARD-WP-0002-T16 -status: todo +status: done priority: medium state_hub_task_id: "b00ca669-59d6-454a-8d6e-f34694e35192" ``` @@ -739,7 +739,7 @@ T1–T6); T18 is an adapter-contract concern (cross-links T11–T16). ```task id: SHARD-WP-0002-T17 -status: todo +status: done priority: high state_hub_task_id: "529b32d2-b681-4711-82c5-5298410cfb37" ``` @@ -779,7 +779,7 @@ query-time join freshness vs cost; activity-streams reach vs fediverse dependenc ```task id: SHARD-WP-0002-T18 -status: todo +status: done priority: medium state_hub_task_id: "331c2a9b-57bb-4067-8d1f-9a3de10e2873" ```