diff --git a/SCOPE.md b/SCOPE.md index d511a85..be1d459 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -21,8 +21,8 @@ Learnings update both SCOPE and INTENT where necessary. | Intent | `INTENT.md` established; authorization-in-core amendments drafted | | Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | CoreArchitectureBlueprint (whole-system architecture) + ArchitectureBlueprint (auth/history) drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds | -| Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (18 tasks: T1–T10 federation + T11–T16 adapter contract + T17 federation-model taxonomy + T18 computational content, re-folded from synthesis v3 + the computational page model); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis) | +| Spec | CoreArchitectureBlueprint (whole-system architecture, hardened via SHARD-WP-0005) + ArchitectureBlueprint (auth/history) drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds | +| Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (18 tasks: T1–T10 federation + T11–T16 adapter contract + T17 federation-model taxonomy + T18 computational content, re-folded from synthesis v3 + the computational page model); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review) | ## In Scope (today) diff --git a/spec/CoreArchitectureBlueprint.md b/spec/CoreArchitectureBlueprint.md index 80f8df3..567c184 100644 --- a/spec/CoreArchitectureBlueprint.md +++ b/spec/CoreArchitectureBlueprint.md @@ -721,7 +721,27 @@ Hard import rules (enforced by import lint): --- -## 12. Canonical data flows (the architecture exercised) +## 12. Known scaling risks & open problems + +Tracked honestly rather than pretend-solved (review disposition F). Each has a **chosen +direction** and a **revisit trigger** — the thing that, if observed, forces a redesign. + +| # | Risk / open problem | Chosen direction | Revisit trigger | +|---|---------------------|------------------|-----------------| +| O-1 | **Equivalence blocking misses true matches** (LSH false negatives, §8.7) | accept a small miss rate; curator bindings are the escape hatch | measured recall below an agreed threshold on real corpora | +| O-2 | **Convergence bound for high-write CRDT shards under partition** (§8.6) | causal via journal + CRDT-native merge at the shard; no global bound promised | user-visible divergence that outlives a partition | +| O-3 | **Per-equivalence-set divergence tracking** (§8.6) | start with base-rev comparison; add vector clocks only if needed | 3-way concurrent divergence that base-rev mis-orders | +| O-4 | **Persisted derived-tier cost ceiling** (§8.7/§9.1) | per-tenant partition, incremental-maintained, rebuild is fallback | a tenant whose incremental cost still exceeds budget | +| O-5 | **Axis-interaction completeness** (§6.5) | the named interaction table is the contract; extend deliberately | a real adapter needing an interaction not in the table | +| O-6 | **Span-address portability across projection** (§7.2) | shard-scoped native-id wrapping now; tumbler later | cross-shard transclusion that native ids can't satisfy | +| O-7 | **Squash-compaction vs. perfect auditability** (§8.1) | compact the *path*, preserve reachable states; configurable | a compliance need for every intermediate keystroke | + +These are the spec-writing inputs for `SHARD-WP-0002`; none blocks the architecture, each +scopes an implementation spike. + +--- + +## 13. Canonical data flows (the architecture exercised) **A. Attach a shard.** Adapter binds (chosen attachment mode) → probes/declares a capability profile → core registers the shard under a root entity → if not git-native, the coordination @@ -746,30 +766,38 @@ gated off. History uses paired-text/nbdime per axis 5. --- -## 13. Key tradeoffs & decisions to confirm +## 14. Key tradeoffs & decisions -Resolved here: +Decided: -- **Capability spectra over a verb checklist** — accept richer contract complexity for precise, - uniform degradation. (Decided: spectra.) -- **Derived middle is a cache, not a store** — accept recompute cost for rebuildability, - provenance, and graceful degradation. (Decided: cache.) -- **Default federation = fork+journal over Git** — the home case; other models opt-in. (Decided.) -- **Execution off by default** — recognise+project always; execute only when gated on. (Decided.) +- **Capability spectra over a verb checklist** — richer contract for precise, uniform + degradation; tamed by an orthogonal core + implied positions + a named interaction table + (§6.5). (Decided.) +- **Three states; derived = f(canonical)** — sharded + coordination canonical, derived + disposable (§1). (Decided; supersedes the earlier "edges vs middle" framing.) +- **Incremental-first, rebuild-as-fallback** — the derived tier is continuously maintained from + change events; full rebuild is rare and need not be cheap (§8.7). (Decided — resolves the + earlier "union graph persistence" open item: **persisted, per-tenant, incrementally + maintained, rebuildable**, §9.1.) +- **Identity ≠ fingerprint** — page identity is a stable handle; fingerprints are for + equivalence (§7.2). (Decided.) +- **Consistency = read-your-writes (journal) + causal (derived) + eventual/freshness-labelled + (shards)**; conflict detection/representation is core, resolution is policy (§8.6). (Decided.) +- **Address scheme** — shard-scoped native-id wrapping now; portable tumbler later (§7.2, O-6). + (Decided.) +- **Default federation = fork+journal over Git**; other models opt-in (§8.3). (Decided.) +- **Execution off by default** — recognise+project always; execute only when gated (§8.5). (Decided.) +- **Derived tier is tenant-partitioned** (I-13, §9.1). (Decided.) -Open — to confirm before SHARD-WP-0002 spec-writing finalises: +Still open (carried to §12 / policy): -1. **Union graph persistence.** Pure-recompute (simplest, honours I-2 hardest) vs a persisted- - but-disposable cache (faster, must guarantee rebuild equivalence). *Recommendation: - persisted-but-disposable with a `rebuild` that must reproduce it byte-for-byte.* -2. **Address scheme.** Ship shard-scoped native-id wrapping now and treat a portable tumbler as - a later capability, or design the tumbler up front? *Recommendation: wrap native ids now.* -3. **L1 "attributed-but-open" mode** — ship it or jump L0→L2? (Carried from ArchitectureBlueprint.) -4. **Per-page ACL default** — off (per-shard/namespace) confirmed; revisit only if demand appears. +1. **L1 "attributed-but-open" mode** — ship it or jump L0→L2? (Carried from ArchitectureBlueprint.) +2. **Per-page ACL default** — off (per-shard/namespace) confirmed; revisit only if demand appears. +3. The implementation spikes in **§12** (O-1…O-7). --- -## 14. What this architecture is *not* +## 15. What this architecture is *not* - Not a wiki engine, UI, or rendering pipeline (those are consumers at L6). - Not a canonical-source-of-truth — shards keep sovereignty; the middle is derived. @@ -780,10 +808,16 @@ Open — to confirm before SHARD-WP-0002 spec-writing finalises: --- -## 15. Traceability +## 16. Traceability -- **INTENT** — every invariant in §2 cites an INTENT principle or boundary; no invariant - contradicts the Stability Note. +- **INTENT** — every invariant in §2 (I-1…I-13) cites an INTENT principle or boundary; no + invariant contradicts the Stability Note. +- **Review & hardening** — this revision folds in + `history/260615-core-architecture-blueprint-review.md` via **`SHARD-WP-0005`**: A-1→§1/§3/§4 + (three-state re-frame), B-1→§7.2 (identity vs equivalence), B-2→§8.6 (consistency/conflict), + B-3→§9.1+I-13 (tenant isolation), C-1/C-2→§8.7/§8.8 (incremental + indexed + invalidation), + C-3→§8.1 (history scaling), D-1→§6.5 (orthogonal core), D-2→§7.3 (layered provenance), + D-3→§8.4 (common-case projection), D-4→§11 (policy module + rail discipline); open items→§12. - **Research** — §6 (spectra) ← `260614-shard-spectrum-synthesis` v3; §8.3 (federation taxonomy) ← v3 §2.5; §8.4–§8.5 (two-axis projection, view registry, computational scope) ← `260614-computational-page-model-synthesis`; §7 page shapes ← the engine + modern-tool + @@ -801,7 +835,7 @@ Open — to confirm before SHARD-WP-0002 spec-writing finalises: --- -## 16. Stability note +## 17. Stability note This document defines shard-wiki's **internal** architecture; it may evolve as the spec workplans land. But the **thesis (§1)**, the **invariants (§2)**, and the **dual narrow waist diff --git a/workplans/SHARD-WP-0005-architecture-hardening.md b/workplans/SHARD-WP-0005-architecture-hardening.md index 0f777ac..62a0a11 100644 --- a/workplans/SHARD-WP-0005-architecture-hardening.md +++ b/workplans/SHARD-WP-0005-architecture-hardening.md @@ -4,7 +4,7 @@ type: workplan title: "core architecture hardening (blueprint review fixes)" domain: whynot repo: shard-wiki -status: active +status: done owner: tegwick topic_slug: whynot created: "2026-06-15" @@ -187,7 +187,7 @@ Fix findings **D-2, D-3, D-4** together (the structural elegance/efficiency clus ```task id: SHARD-WP-0005-T8 -status: todo +status: done priority: medium state_hub_task_id: "6ce3584f-ffa1-4568-8175-460adc8e2083" ``` @@ -207,7 +207,7 @@ Fix findings **B-3, C-3**: ```task id: SHARD-WP-0005-T9 -status: todo +status: done priority: medium state_hub_task_id: "c876a63c-4cfd-4794-be29-1d8226643c82" ```