generated from coulomb/repo-seed
history+workplan: round-2 blueprint review; SHARD-WP-0006 hardening II
Records the round-2 critical review (history/260615-...-review-2) and establishes SHARD-WP-0006 to: reconcile overview with hardened body (§A), settle the journal/coordination-state model (event-sourced decision log; single-vs-multi-writer concurrency — B-1+B-3), require an adapter conformance suite (B-2), fix incremental-equivalence correctness + I-2 verification (B-4), and track §C as O-8..O-11. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
91
history/260615-core-architecture-blueprint-review-2.md
Normal file
91
history/260615-core-architecture-blueprint-review-2.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# Critical review (round 2) — CoreArchitectureBlueprint.md (hardened)
|
||||
|
||||
Date: 2026-06-15 · Reviewer: tegwick (with Claude) · Subject:
|
||||
`spec/CoreArchitectureBlueprint.md` after **SHARD-WP-0005** (commit f21b7b5) · Feeds:
|
||||
**SHARD-WP-0006**
|
||||
|
||||
A second hostile pass over the *hardened* blueprint. Round 1 found design bugs; this round
|
||||
finds (a) self-consistency regressions the surgical hardening introduced, and (b) deeper
|
||||
second-order gaps — some of which the hardening *sharpened*. Verdict: the architecture is now
|
||||
substantially sound; what remains in §B/§C are hard distributed-systems/operational questions,
|
||||
not design smells — except §A (a real regression) and three foundational gaps in §B.
|
||||
|
||||
---
|
||||
|
||||
## A. Self-consistency regressions introduced by surgical hardening
|
||||
|
||||
The 9 edits deepened §6–§9 but did not propagate to the **overview surfaces**, so the document
|
||||
now contradicts itself between its summary and its body (and readers trust the summary).
|
||||
|
||||
- **A-1 (real contradiction).** §4 still says "Addressing, **equivalence**, and transclusion
|
||||
key on identity" — the exact conflation T2 fixed in §7.2 (equivalence keys on *content
|
||||
fingerprint across distinct identities*). → **WP-0006 T1**
|
||||
- **A-2.** §4 "Projection — typed on two axes" and §4 "Provenance envelope … every artifact
|
||||
carries [full wrapper]" are stale vs T7's §8.4 (two-axis = extension point; trivial default)
|
||||
and §7.3 (layered effective-vs-own). → **T1**
|
||||
- **A-3.** §10 policy surface omits knobs the hardening added (freshness/staleness §8.8,
|
||||
squash-compaction §8.1, conflict-resolution preset §8.6, tenant-partition) — yet §11 defines
|
||||
`policy/` as "owns the §10 surface." The module contract points at a stale list. → **T1**
|
||||
- **A-4 (cosmetic).** §3 diagram + §11 header still say "L4 rebuildable cache" / "15 spectra,"
|
||||
advertising the pre-hardening model (§8.7 incremental-first; §6.5 orthogonal-core). → **T1**
|
||||
|
||||
Meta-point: surgical editing hardened the body but regressed whole-document coherence; v2
|
||||
needs an **overview-reconciliation pass**.
|
||||
|
||||
## B. Foundational gaps (serious; some sharpened by the hardening)
|
||||
|
||||
- **B-1 — The journal is now a concurrent-write DB, but it's single-writer Git.** §8.6's
|
||||
consistency model assumes "the journal is local Git, read-your-writes." L4 multi-tenant + the
|
||||
L6 Orchestrator API imply a server; HA/scale implies *multiple* instances. Concurrent commits
|
||||
of coordination-canonical state to one git journal = lock contention / merge races; Git is not
|
||||
a concurrent-write store. Either single-writer-per-space (an unstated HA ceiling) or a real
|
||||
concurrent coordination store with Git as an *export*. **T1 of WP-0005 worsened this** by
|
||||
loading more canonical state into the journal. The keystone unanswered question. → **T2**
|
||||
- **B-2 — Capability-as-data trusts self-reported profiles with no conformance check.** I-3 +
|
||||
§6.5's degradation contract assume the profile tells the truth. A buggy adapter (claims
|
||||
`merge=git/text`, corrupts; claims `notify`, never emits) silently poisons every degradation
|
||||
decision. No **adapter conformance suite** (declared profile == observed behavior) exists.
|
||||
Foundational for an architecture whose correctness rests on profile accuracy. → **T3**
|
||||
- **B-3 — "Coordination-canonical state in the journal" has no representation design.** T1
|
||||
relocated overlays/bindings/aliases/equivalence-sets/merges into "the journal" without saying
|
||||
*how* Git stores structured mutable state. "All equivalences touching X" over a git-of-files
|
||||
is O(scan) unless indexed — and an index is L4/derived. The new central concept is a black
|
||||
box; resolve *with* B-1. → **T2**
|
||||
- **B-4 — Incremental equivalence is under-specified/likely incorrect; I-2 only eventually
|
||||
true.** §8.7 re-verifies a changed page's *new* candidate set but not the pairs it *leaves*
|
||||
(a page exiting an LSH bucket can break an existing equivalence edge); the delta is not
|
||||
additive. Deeper: incremental maintenance drifts from `f(canonical)`, so I-2 holds only
|
||||
eventually, guaranteed solely by an expensive reconcile-against-rebuild. Needs a stated
|
||||
verification mechanism (background checker / digest-vs-sampled-rebuild). → **T4**
|
||||
|
||||
## C. Real but second-tier (track as open problems O-8…O-11)
|
||||
|
||||
- **C-1 — Mechanism-over-policy → operator burden; no preset bundles.** ~7 knob families with
|
||||
sub-modes and interactions; only authz (L0–L4) bundles into personas. Need named bundles
|
||||
("personal vault" / "team wiki" / "enterprise federation"). → **O-8 / T5**
|
||||
- **C-2 — Tenant partitioning (I-13) vs shard sharing + lazy projection.** A shard in two roots
|
||||
is cached twice → duplicate storage + double refresh on rate-limited backends. Shard
|
||||
exclusive-to-one-root or shareable? Unresolved. → **O-9 / T5**
|
||||
- **C-3 — Span-level authz + transclusion is an unmodeled leak path.** Authz is per
|
||||
page/shard/tenant; transclusion crosses shards at span granularity → a page can leak a span
|
||||
past its ACL (aggregation/inference). §7.3's ⊕ also stops being simple two-level inheritance
|
||||
across a transclusion boundary. → **O-10 / T5**
|
||||
- **C-4 — Union-under-unavailability undefined.** Freshness covers *stale*, nothing covers
|
||||
*down*. The dead-shard read path (partial union? error? last-known?) is unspecified though
|
||||
it's the commonest real failure. → **O-11 / T5**
|
||||
|
||||
## D. Recommended resolution (→ SHARD-WP-0006)
|
||||
|
||||
1. **§A reconciliation** (T1) — make the overview match the hardened body.
|
||||
2. **Journal & coordination-state model** (T2) — settle single-vs-multi-writer and separate the
|
||||
**coordination-state store** from the **content-history journal**. Likely resolution:
|
||||
**event-sourced coordination** — an append-only *decision log* is the coordination-canonical
|
||||
tier (git-addressable, I-6 preserved); the queryable current state (alias table, equivalence
|
||||
set) is a *derived fold* of the log (disposable). Append-logs tolerate concurrency far better
|
||||
than mutable-file Git; state a concurrency model. Resolves **B-1 + B-3** together.
|
||||
3. **Adapter conformance suite** (T3) — make a passing conformance run part of the contract
|
||||
(B-2): every adapter proves declared profile == observed behavior.
|
||||
4. **Incremental correctness + I-2 verification** (T4) — fix the leaving-bucket re-verification
|
||||
and propagation; add a background consistency-checker / derived-tier digest so I-2 is
|
||||
verifiable, not merely asserted (B-4).
|
||||
5. **Track §C** (T5) — O-8…O-11 with chosen direction + revisit trigger; close-out.
|
||||
@@ -14,4 +14,5 @@ design evolution.
|
||||
|
||||
| Date | Record | Subject |
|
||||
|------|--------|---------|
|
||||
| 2026-06-15 | `260615-core-architecture-blueprint-review.md` | Critical review of `spec/CoreArchitectureBlueprint.md` (commit 9b5b393); inputs to `SHARD-WP-0005` |
|
||||
| 2026-06-15 | `260615-core-architecture-blueprint-review.md` | Critical review of `spec/CoreArchitectureBlueprint.md` (commit 9b5b393); inputs to `SHARD-WP-0005` |
|
||||
| 2026-06-15 | `260615-core-architecture-blueprint-review-2.md` | Round-2 review of the hardened blueprint (post-`SHARD-WP-0005`, f21b7b5); inputs to `SHARD-WP-0006` |
|
||||
168
workplans/SHARD-WP-0006-architecture-hardening-2.md
Normal file
168
workplans/SHARD-WP-0006-architecture-hardening-2.md
Normal file
@@ -0,0 +1,168 @@
|
||||
---
|
||||
id: SHARD-WP-0006
|
||||
type: workplan
|
||||
title: "core architecture hardening II (round-2 review fixes)"
|
||||
domain: whynot
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
updated: "2026-06-15"
|
||||
depends_on:
|
||||
- SHARD-WP-0005
|
||||
---
|
||||
|
||||
# SHARD-WP-0006 — Core architecture hardening II
|
||||
|
||||
## Goal
|
||||
|
||||
Resolve the round-2 review (`history/260615-core-architecture-blueprint-review-2.md`):
|
||||
reconcile the document's overview with its hardened body (§A), settle the three foundational
|
||||
gaps (§B: journal concurrency + coordination-state representation; adapter conformance;
|
||||
incremental-maintenance correctness + I-2 verification), and track the second-tier items
|
||||
(§C) as open problems. Deliverable: a revised `spec/CoreArchitectureBlueprint.md` that is
|
||||
**self-consistent** and answers the keystone question — *single- vs multi-writer journal, and
|
||||
where mutable coordination state actually lives*.
|
||||
|
||||
## Context
|
||||
|
||||
- Round-2 review: `history/260615-core-architecture-blueprint-review-2.md` (A-1…A-4, B-1…B-4,
|
||||
C-1…C-4; resolution §D).
|
||||
- Round-1 review + first hardening: `history/260615-core-architecture-blueprint-review.md`,
|
||||
`SHARD-WP-0005`.
|
||||
- Constraint to honour: INTENT I-6 / "Git-addressable coordination layer" — any coordination
|
||||
store must remain git-addressable (history/patches/review/backup), not abandon Git.
|
||||
|
||||
**Non-goal:** Implement anything. Architecture spec revision only.
|
||||
|
||||
---
|
||||
|
||||
## §A overview-reconciliation pass
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0006-T1
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Make the overview match the hardened body (review A-1…A-4):
|
||||
|
||||
- **A-1 (bug):** §4 "Page identity vs placement" — remove "equivalence … key on identity";
|
||||
state equivalence keys on content fingerprint across identities (align with §7.2).
|
||||
- **A-2:** §4 "Projection" → trivial-default + extension-point framing (align §8.4); §4
|
||||
"Provenance envelope" → layered effective-vs-own (align §7.3).
|
||||
- **A-3:** §10 policy surface — add the missing knobs (freshness/staleness mode, history
|
||||
squash-compaction, conflict-resolution preset, tenant-partition); keep §11's `policy/`
|
||||
pointer accurate.
|
||||
- **A-4:** §3 diagram + §11 header — "incremental-maintained derived tier (rebuild = fallback)"
|
||||
and "orthogonal-core capability profile," not "rebuildable cache / 15 spectra."
|
||||
|
||||
---
|
||||
|
||||
## Journal & coordination-state model (single vs multi-writer)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0006-T2
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Settle the keystone (review B-1 + B-3 together). Decide and document:
|
||||
|
||||
- **Separate the content-history journal from the live coordination state.** Proposed
|
||||
resolution: **event-sourced coordination** — an append-only **decision log** (overlays,
|
||||
bindings, aliases, merges as events) is the **coordination-canonical** tier and is
|
||||
**git-addressable** (preserves I-6); the queryable *current* coordination state (effective
|
||||
alias table, equivalence set) is a **derived fold** of the log (disposable, indexable, L4).
|
||||
This resolves B-3 (representation = event log; queries = derived index) and refines the §1
|
||||
three-state framing (coordination-canonical = the decision *log*, not a mutable file).
|
||||
- **State the concurrency model (B-1).** How multiple orchestrator instances append safely:
|
||||
serialized append per space (with the HA story made explicit) and/or
|
||||
conflict-free ordering of independent appends; what "read-your-writes" means across
|
||||
instances; where Git is authoritative vs a materialised export.
|
||||
- Update §1 (refine coordination-canonical = decision log), §4 (Coordination journal /
|
||||
Overlay), §8.1/§8.2, §8.6 (consistency now grounded on the log + a stated concurrency model),
|
||||
§11 (a `coordination/` event-log store + derived-fold index).
|
||||
|
||||
---
|
||||
|
||||
## Adapter conformance suite (profiles must be verified, not asserted)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0006-T3
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Fix B-2. Make capability-as-data trustworthy: add to the adapter contract (§6) a
|
||||
**conformance-suite requirement** — a published battery that, given a binding, verifies its
|
||||
**declared capability profile matches observed behaviour** (each declared verb/position is
|
||||
exercised; mismatches fail registration). State that an adapter is only admissible if it
|
||||
passes; that the suite is versioned with the contract; and that profile claims are
|
||||
**verified, never self-asserted** (this is what makes I-3 / §6.5 sound). Update §6.4/§6.5 and
|
||||
add the conformance obligation to the acceptance posture.
|
||||
|
||||
---
|
||||
|
||||
## Incremental-maintenance correctness & I-2 verification
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0006-T4
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Fix B-4. In §8.7: correct the incremental equivalence algorithm — a changed page must
|
||||
**re-verify the candidate pairs it leaves** (bucket exits) and **propagate** to pages whose
|
||||
equivalence depended on it, not only its new candidates; the delta is not additive. Add an
|
||||
**I-2 verification mechanism**: a background consistency-checker comparing a **digest of the
|
||||
derived tier** against a sampled/periodic recompute, so `derived = f(canonical)` is *verified*,
|
||||
not merely asserted; on mismatch, recompute the affected partition. State that I-2 is an
|
||||
*eventually-verified* property with a named checker, and note the residual in §12.
|
||||
|
||||
---
|
||||
|
||||
## Track second-tier items; close-out
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0006-T5
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Add review §C as open problems in §12 (each with chosen direction + revisit trigger):
|
||||
|
||||
- **O-8** preset bundles ("personal vault" / "team wiki" / "enterprise federation") over the
|
||||
§10 policy surface — direction: ship a small set of named personas; trigger: operators
|
||||
mis-configuring interacting knobs.
|
||||
- **O-9** shard sharing across roots vs tenant partition (I-13) — direction: shard exclusive to
|
||||
one root by default, explicit shared-read binding otherwise; trigger: a shard legitimately
|
||||
needed live in two tenants.
|
||||
- **O-10** span-level authz under transclusion (aggregation/inference leak) + ⊕ composition
|
||||
across transclusion boundaries — direction: a transcluded span inherits the *stricter* of
|
||||
source and host authz; provenance ⊕ composes source-page-envelope under the host; trigger: a
|
||||
cross-authz transclusion in practice.
|
||||
- **O-11** union-under-shard-unavailability — direction: partial union + per-shard
|
||||
"unavailable" provenance + last-known-projection where policy allows; trigger: SLA needs on
|
||||
partial reads.
|
||||
|
||||
Refresh §14 decisions and §16 traceability (link this review + SHARD-WP-0006); refresh
|
||||
invariants if any added. Final `check_repo_consistency` pass.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Every round-2 finding A-1…A-4, B-1…B-4 is **resolved in the blueprint**; C-1…C-4 are tracked
|
||||
as O-8…O-11 with a chosen direction (none silently dropped).
|
||||
- The blueprint is **self-consistent**: overview (§1–§5, §10) agrees with body (§6–§9).
|
||||
- The **journal/coordination-state model** answers single-vs-multi-writer and preserves I-6
|
||||
(git-addressable coordination).
|
||||
- The **adapter conformance** obligation makes capability-as-data (I-3) verified, not asserted.
|
||||
- Each task committed; SCOPE/spec-README updated where status changes; state-hub synced.
|
||||
|
||||
## Suggested task order
|
||||
|
||||
T1 (reconcile, fast) → T2 (keystone journal/coordination model) → T3 (conformance) →
|
||||
T4 (incremental correctness) → T5 (track §C + close-out).
|
||||
Reference in New Issue
Block a user