shard-wiki/workplans/SHARD-WP-0005-architecture-hardening.md

id, type, title, domain, repo, status, owner, topic_slug, created, updated, depends_on, state_hub_workstream_id

id	type	title	domain	repo	status	owner	topic_slug	created	updated	depends_on	state_hub_workstream_id
SHARD-WP-0005	workplan	core architecture hardening (blueprint review fixes)	whynot	shard-wiki	done	tegwick	whynot	2026-06-15	2026-06-15	SHARD-WP-0002	7d277f39-f78f-4998-8cab-875f2bf69763

SHARD-WP-0005 — Core architecture hardening

Goal

Resolve the findings of the critical review (history/260615-core-architecture-blueprint-review.md) by hardening spec/CoreArchitectureBlueprint.md for correctness, scale, and elegance before implementation. Close every solvable finding; record every partially-open finding explicitly (consistency model, equivalence-at-scale strategy, axis-interaction subset) rather than pretending it is solved.

Primary deliverable: a revised spec/CoreArchitectureBlueprint.md (the review's A–F findings folded in) plus a new "Known scaling risks & open problems" section.

Context

• Review: history/260615-core-architecture-blueprint-review.md (findings A-1, B-1…B-3, C-1…C-3, D-1…D-4; disposition F).
• Architecture under revision: spec/CoreArchitectureBlueprint.md @ 9b5b393.
• Constraints: INTENT.md (no amendment expected — all fixes live inside existing boundaries); the synthesis inputs already folded into SHARD-WP-0002.

Non-goal: Implement anything. This workplan revises the architecture spec only.

Guiding aims

• Elegance: prefer fewer, orthogonal concepts; make the common case trivial and the exotic case possible (not the reverse).
• No pretend-solved: an honestly-open problem with a chosen direction beats a hand-wave.
• INTENT-preserving: every change must still honour the 12 invariants (or revise an invariant deliberately and say so).

---

Re-frame the state model: canonical (sharded + coordination) vs derived

id: SHARD-WP-0005-T1
status: done
priority: high
state_hub_task_id: "0bb17223-b458-41a4-b86c-c36b2689cf81"

Fix finding A-1 (and its I-2 contradiction). Replace the two-bucket thesis with three states: sharded-canonical (shard content), coordination-canonical (journal: overlays, curator equivalence bindings, alias tables, merge decisions — durable, born in the middle), derived-disposable (union graph, indexes, projections). Re-frame §1 as canonical (sharded + coordination) vs derived (disposable); make derived = f(canonical) literally true. Update I-2, the §3 dependency rule (only the disposable tier is rebuildable), §4 abstractions (name coordination-canonical state), and move "alias table / curator binding" out of L4-rebuildable into the coordination tier.

---

Split page identity from content equivalence

id: SHARD-WP-0005-T2
status: done
priority: high
state_hub_task_id: "d76a6b07-5157-426c-9343-a3e7efedf4a9"

Fix bug B-1. Separate two concepts §7.2/§8.4 conflate: page identity = a stable handle (shard-scoped uid, name-based, survives edits) used for references/placement; content equivalence = fingerprint / span-set overlap used to detect sameness, never as identity. State that a fingerprint identifies a version/content, not a page. Reconcile with identity≠placement (I-9): identity (stable) → placements (N) → equivalence (cross-identity sameness).

---

Consistency, concurrency & conflict model

id: SHARD-WP-0005-T3
status: done
priority: high
state_hub_task_id: "fb91f43f-3bf0-41f5-ad0a-bfd15a7fad17"

Fix bug B-2. Add a new section stating shard-wiki's consistency guarantee (choose and justify: e.g. causal consistency via the coordination journal; read-your-writes for local overlays; eventual convergence for projected union). Specify conflict detection + representation as core mechanism (divergence detection, keep-both/coexist representation), keeping only resolution as policy (I-7). Define overlay-apply semantics under source drift (rebase/refuse/three-way), and journal-commit vs shard-native-write ordering. Mark any residual as open (→ T9).

---

Scaling the union: incremental-first, equivalence indexing, rebuild-as-fallback

id: SHARD-WP-0005-T4
status: done
priority: high
state_hub_task_id: "eb885f03-2569-4ade-b3aa-174bc6062672"

Fix scaling findings C-1, C-2. Make incremental, change-driven maintenance the primary mechanism for the derived tier: the notify capability (or poll/ETag fallback) drives delta updates to union/index/projections; full rebuild is a rare fallback (and explicitly not required to be cheap for rate-limited shards — reconcile with axis-10). Replace O(N²) equivalence with a blocking/indexing strategy (normalised-title/path buckets, fingerprint shingling/LSH, candidate generation then verify) and incremental equivalence maintenance. Update §8.4 and I-2 (rebuildability is a correctness property of the disposable tier, not an operational expectation).

---

Cache freshness & invalidation protocol

id: SHARD-WP-0005-T5
status: done
priority: medium
state_hub_task_id: "b031715c-00d0-4a0a-99f1-4e74a236d04b"

Fix finding C-2 (invalidation). Design the replication-projection freshness/invalidation protocol: staleness semantics (TTL vs event-driven), push (notify/webhook/ActivityPub) vs poll (ETag/If-Modified) vs hybrid per capability profile, single-flight / coalescing to avoid thundering-herd refetch, and how freshness is surfaced in the provenance envelope. Tie to the operational-envelope axis (rate-limited shards favour event-driven + long TTL).

---

Capability spectra: orthogonal core, implied positions, interaction subset

id: SHARD-WP-0005-T6
status: done
priority: medium
state_hub_task_id: "f04ce101-0d95-4e1a-ab8b-80dfff9d2dda"

Fix elegance finding D-1. Identify a smaller orthogonal core of capability axes and mark the rest as derived/implied (e.g. attachment=git-IS-store ⟹ history=git-native ⟹ merge=git/text; opacity=encrypted ⟹ query/translation degrade). Explicitly enumerate the axis-interaction subset the degradation function actually depends on (so "no per-backend code" is a demonstrated claim, not an assertion), and forbid impossible profiles via the implied-position rules. Update §6.

---

Elegance pass: layered provenance, common-case projection, policy module & rails

id: SHARD-WP-0005-T7
status: done
priority: medium
state_hub_task_id: "b84b790f-d208-4a76-af9f-1402a6a87ac1"

Fix findings D-2, D-3, D-4 together (the structural elegance/efficiency cluster):

• Layered provenance (D-2): page-level envelope + span-level deltas (the same effective-vs-own pattern used for Trilium metadata), so per-span cost is near-zero when uniform. Update §4/§7.2 and the provenance rail.
• Common-case-trivial projection (D-3): default = plain lazy replication-projection; derivation/liveness/view-registry become an extension point invoked only for computational/typed content — not a taxonomy every projection instantiates. Re-shape §8.4–8.5.
• Policy module + rail discipline (D-4): add a policy/ module owning the §10 surface; pin provenance/ and capability types behind stable narrow interfaces to cap coupling. Update §11 and the dependency rules.

---

Security/multi-tenancy isolation & history scaling

id: SHARD-WP-0005-T8
status: done
priority: medium
state_hub_task_id: "6ce3584f-ffa1-4568-8175-460adc8e2083"

Fix findings B-3, C-3:

• Tenant isolation of derived state (B-3): the persisted derived tier is partitioned per tenant/root-entity; no cross-tenant union cache guarded only by read-time filtering. Reconcile I-2 + L5; state the isolation invariant. Update §9/§13.
• History scaling (C-3): a strategy for unbounded open-L0 history — git packing/gc, compaction/squash policy for low-value churn, per-shard history offload, and rate-limiting/anti-abuse hooks — without weakening recoverability (I-10). Update §8.1.

---

Known scaling risks & open problems; invariant + traceability refresh

id: SHARD-WP-0005-T9
status: done
priority: medium
state_hub_task_id: "c876a63c-4cfd-4794-be29-1d8226643c82"

Close out (finding F). Add a new "Known scaling risks & open problems" section listing the partially-open items with their chosen direction and the trigger that would force a revisit (consistency-model edge cases, equivalence-blocking false-negative rate, axis-interaction completeness, persisted-cache cost ceiling). Refresh the invariants table (any added/changed invariant), the §13 decisions (mark resolved vs still-open), and the §15 traceability (link this review + SHARD-WP-0005). Final check_repo_consistency pass.

---

Acceptance criteria

• Every review finding A-1, B-1–B-3, C-1–C-3, D-1–D-4 is either resolved in the blueprint or listed as a known open problem with a chosen direction (none silently dropped).
• The blueprint still honours all INTENT invariants, or revises one deliberately and visibly.
• spec/CoreArchitectureBlueprint.md reads as more elegant, not merely more detailed: fewer/orthogonal core concepts; common case trivial; exotic case possible.
• Each task committed; SCOPE/spec-README updated where status changes; state-hub synced.

Suggested task order

Correctness first (T1 → T2 → T3), then scale (T4 → T5), then elegance (T6 → T7), then hardening (T8), then close-out (T9).