shard-wiki/workplans/SHARD-WP-0005-architecture-hardening.md

---
id: SHARD-WP-0005
type: workplan
title: "core architecture hardening (blueprint review fixes)"
domain: whynot
repo: shard-wiki
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
updated: "2026-06-15"
depends_on:
  - SHARD-WP-0002
state_hub_workstream_id: "7d277f39-f78f-4998-8cab-875f2bf69763"
---

# SHARD-WP-0005 — Core architecture hardening

## Goal

Resolve the findings of the critical review
(`history/260615-core-architecture-blueprint-review.md`) by hardening
`spec/CoreArchitectureBlueprint.md` for **correctness, scale, and elegance** before
implementation. Close every *solvable* finding; record every *partially-open* finding
explicitly (consistency model, equivalence-at-scale strategy, axis-interaction subset) rather
than pretending it is solved.

Primary deliverable: a revised `spec/CoreArchitectureBlueprint.md` (the review's A–F findings
folded in) plus a new **"Known scaling risks & open problems"** section.

## Context

- Review: `history/260615-core-architecture-blueprint-review.md` (findings A-1, B-1…B-3,
  C-1…C-3, D-1…D-4; disposition F).
- Architecture under revision: `spec/CoreArchitectureBlueprint.md` @ 9b5b393.
- Constraints: `INTENT.md` (no amendment expected — all fixes live inside existing
  boundaries); the synthesis inputs already folded into `SHARD-WP-0002`.

**Non-goal:** Implement anything. This workplan revises the architecture spec only.

## Guiding aims

- **Elegance:** prefer fewer, orthogonal concepts; make the common case trivial and the
  exotic case possible (not the reverse).
- **No pretend-solved:** an honestly-open problem with a chosen direction beats a hand-wave.
- **INTENT-preserving:** every change must still honour the 12 invariants (or revise an
  invariant deliberately and say so).

---

## Re-frame the state model: canonical (sharded + coordination) vs derived

```task
id: SHARD-WP-0005-T1
status: done
priority: high
state_hub_task_id: "0bb17223-b458-41a4-b86c-c36b2689cf81"
```

Fix finding **A-1** (and its I-2 contradiction). Replace the two-bucket thesis with **three
states**: **sharded-canonical** (shard content), **coordination-canonical** (journal:
overlays, curator equivalence bindings, alias tables, merge decisions — durable, born in the
middle), **derived-disposable** (union graph, indexes, projections). Re-frame §1 as
**canonical (sharded + coordination) vs derived (disposable)**; make `derived = f(canonical)`
literally true. Update I-2, the §3 dependency rule (only the disposable tier is rebuildable),
§4 abstractions (name coordination-canonical state), and move "alias table / curator binding"
out of L4-rebuildable into the coordination tier.

---

## Split page identity from content equivalence

```task
id: SHARD-WP-0005-T2
status: done
priority: high
state_hub_task_id: "d76a6b07-5157-426c-9343-a3e7efedf4a9"
```

Fix bug **B-1**. Separate two concepts §7.2/§8.4 conflate: **page identity** = a *stable
handle* (shard-scoped uid, name-based, survives edits) used for references/placement; **content
equivalence** = fingerprint / span-set overlap used to *detect sameness*, never as identity.
State that a fingerprint identifies a *version/content*, not a *page*. Reconcile with
identity≠placement (I-9): identity (stable) → placements (N) → equivalence (cross-identity
sameness).

---

## Consistency, concurrency & conflict model

```task
id: SHARD-WP-0005-T3
status: done
priority: high
state_hub_task_id: "fb91f43f-3bf0-41f5-ad0a-bfd15a7fad17"
```

Fix bug **B-2**. Add a new section stating shard-wiki's **consistency guarantee** (choose and
justify: e.g. causal consistency via the coordination journal; read-your-writes for local
overlays; eventual convergence for projected union). Specify **conflict detection +
representation as core mechanism** (divergence detection, keep-both/coexist representation),
keeping only *resolution* as policy (I-7). Define **overlay-apply semantics under source
drift** (rebase/refuse/three-way), and journal-commit vs shard-native-write ordering. Mark any
residual as open (→ T9).

---

## Scaling the union: incremental-first, equivalence indexing, rebuild-as-fallback

```task
id: SHARD-WP-0005-T4
status: done
priority: high
state_hub_task_id: "eb885f03-2569-4ade-b3aa-174bc6062672"
```

Fix scaling findings **C-1, C-2**. Make **incremental, change-driven maintenance the primary
mechanism** for the derived tier: the `notify` capability (or poll/ETag fallback) drives
**delta updates** to union/index/projections; full rebuild is a rare fallback (and explicitly
*not required* to be cheap for rate-limited shards — reconcile with axis-10). Replace O(N²)
equivalence with a **blocking/indexing strategy** (normalised-title/path buckets, fingerprint
shingling/LSH, candidate generation then verify) and **incremental equivalence maintenance**.
Update §8.4 and I-2 (rebuildability is a *correctness property of the disposable tier*, not an
operational expectation).

---

## Cache freshness & invalidation protocol

```task
id: SHARD-WP-0005-T5
status: done
priority: medium
state_hub_task_id: "b031715c-00d0-4a0a-99f1-4e74a236d04b"
```

Fix finding **C-2 (invalidation)**. Design the replication-projection **freshness/invalidation
protocol**: staleness semantics (TTL vs event-driven), push (notify/webhook/ActivityPub) vs
poll (ETag/If-Modified) vs hybrid per capability profile, single-flight / coalescing to avoid
thundering-herd refetch, and how freshness is surfaced in the provenance envelope. Tie to the
operational-envelope axis (rate-limited shards favour event-driven + long TTL).

---

## Capability spectra: orthogonal core, implied positions, interaction subset

```task
id: SHARD-WP-0005-T6
status: done
priority: medium
state_hub_task_id: "f04ce101-0d95-4e1a-ab8b-80dfff9d2dda"
```

Fix elegance finding **D-1**. Identify a **smaller orthogonal core** of capability axes and
mark the rest as **derived/implied** (e.g. attachment=git-IS-store ⟹ history=git-native ⟹
merge=git/text; opacity=encrypted ⟹ query/translation degrade). Explicitly enumerate the
**axis-interaction subset** the degradation function actually depends on (so "no per-backend
code" is a demonstrated claim, not an assertion), and forbid impossible profiles via the
implied-position rules. Update §6.

---

## Elegance pass: layered provenance, common-case projection, policy module & rails

```task
id: SHARD-WP-0005-T7
status: done
priority: medium
state_hub_task_id: "b84b790f-d208-4a76-af9f-1402a6a87ac1"
```

Fix findings **D-2, D-3, D-4** together (the structural elegance/efficiency cluster):

- **Layered provenance** (D-2): page-level envelope + span-level *deltas* (the same
  effective-vs-own pattern used for Trilium metadata), so per-span cost is near-zero when
  uniform. Update §4/§7.2 and the provenance rail.
- **Common-case-trivial projection** (D-3): default = plain lazy replication-projection;
  derivation/liveness/view-registry become an **extension point** invoked only for
  computational/typed content — not a taxonomy every projection instantiates. Re-shape §8.4–8.5.
- **Policy module + rail discipline** (D-4): add a `policy/` module owning the §10 surface;
  pin `provenance/` and capability types behind **stable narrow interfaces** to cap coupling.
  Update §11 and the dependency rules.

---

## Security/multi-tenancy isolation & history scaling

```task
id: SHARD-WP-0005-T8
status: done
priority: medium
state_hub_task_id: "6ce3584f-ffa1-4568-8175-460adc8e2083"
```

Fix findings **B-3, C-3**:

- **Tenant isolation of derived state** (B-3): the persisted derived tier is **partitioned per
  tenant/root-entity**; no cross-tenant union cache guarded only by read-time filtering.
  Reconcile I-2 + L5; state the isolation invariant. Update §9/§13.
- **History scaling** (C-3): a strategy for unbounded open-L0 history — git packing/gc,
  **compaction/squash policy for low-value churn**, per-shard history offload, and
  rate-limiting/anti-abuse hooks — without weakening recoverability (I-10). Update §8.1.

---

## Known scaling risks & open problems; invariant + traceability refresh

```task
id: SHARD-WP-0005-T9
status: done
priority: medium
state_hub_task_id: "c876a63c-4cfd-4794-be29-1d8226643c82"
```

Close out (finding **F**). Add a new **"Known scaling risks & open problems"** section listing
the partially-open items with their chosen direction and the trigger that would force a
revisit (consistency-model edge cases, equivalence-blocking false-negative rate,
axis-interaction completeness, persisted-cache cost ceiling). Refresh the **invariants table**
(any added/changed invariant), the **§13 decisions** (mark resolved vs still-open), and the
**§15 traceability** (link this review + SHARD-WP-0005). Final `check_repo_consistency` pass.

---

## Acceptance criteria

- Every review finding A-1, B-1–B-3, C-1–C-3, D-1–D-4 is either **resolved in the blueprint**
  or **listed as a known open problem with a chosen direction** (none silently dropped).
- The blueprint still honours all INTENT invariants, or revises one *deliberately and visibly*.
- `spec/CoreArchitectureBlueprint.md` reads as **more elegant**, not merely more detailed:
  fewer/orthogonal core concepts; common case trivial; exotic case possible.
- Each task committed; SCOPE/spec-README updated where status changes; state-hub synced.

## Suggested task order

Correctness first (**T1 → T2 → T3**), then scale (**T4 → T5**), then elegance (**T6 → T7**),
then hardening (**T8**), then close-out (**T9**).