Compare commits

..

105 Commits

Author SHA1 Message Date
776f5af5a7 Normalize agent instructions and workplan frontmatter (STATE-WP-0067)
- Align agent files with on-disk workplan prefixes (infer from workplan ids)
- Set workplan domain to registered domain_slug; add topic_slug where applicable
- Repair frontmatter delimiter formatting; migrate legacy task status literals
- Regenerate AGENTS.md, CLAUDE.md, and .claude/rules from State Hub templates
2026-06-22 23:16:28 +02:00
fd961c83b4 Add .repo-classification.yaml (CUST-WP-0050 T11 agent first-pass) 2026-06-22 17:47:43 +02:00
cca5bf83c3 Add credential routing instructions for all agent runtimes
Propagate shared credential-routing section (Codex, Claude, Grok, llm-connect)
from state-hub template via scripts/propagate_credential_routing.py.
2026-06-18 22:48:39 +02:00
def699c1eb feat(adapters): GitShardAdapter history adopt + cross-substrate integration (WP-0012 T3)
Adopt git-native history (TSD §A.5): a VERSION-gated history(key) surfaces the
commit list for a path (newest-first sha + subject) — declared by every git-IS-store
shard, read-only or not. Integration proves the union/overlay/edit machinery works
unchanged across folder + git substrates: resolve/chorus span both, edit through a
git shard fast-forwards as a commit, apply-under-drift refuses on an external commit
(sha drift) without clobbering, and a read-only git target keeps the overlay as a
draft. SCOPE updated; WP-0012 done. 196 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:41:19 +02:00
a4e0f52ec1 feat(adapters): GitShardAdapter write=commit + current_rev drift (WP-0012 T2)
Writable mode: write(key, body) stages and commits the file (skipping a no-op so
no empty commit is created), returning the page at the new commit sha. The
writable profile declares WRITE + VERSION with PER_PAGE granularity. current_rev
is the per-path commit sha, so a write — or an external commit to the same path —
moves it, driving apply-under-drift. Passes the conformance positive-write probe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:38:41 +02:00
4231daf94f feat(adapters): GitShardAdapter read path + git-IS-store profile (WP-0012 T1)
A second substrate validating the contract beyond plain folders: a git-IS-store
shard reading Markdown from a git repo. Keys are tracked *.md paths; read returns
a Page whose source_rev is the per-path last-commit sha (so an edit to one page
never drifts another); profile is git-IS-store / substrate=git / history=git-native
/ addressing=path, validated against the §6.5 implication rules. Passes the
conformance read path with honest absence of unclaimed verbs. Zero new deps
(git CLI via subprocess). No core changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:36:28 +02:00
37681d89b6 feat(incremental): wire maintained tier behind views; rebuild fallback (WP-0011 T4)
Route InformationSpace.all_pages through a maintained UnionIndex: equivalence is
served from the incrementally maintained index (curator bindings re-synced live
from the log fold + detected content edges), exposed in decision-log string form
so results are a behaviour-preserving superset. The index is built lazily and
rebuilt (bounded fallback) when the union mutates (attach/edit invalidate it);
reindex() forces a rebuild and verify_index() runs the I-2 self-healing checker.
all_pages() gains an optional equivalence_groups source (default = fold) so
direct callers are unaffected. SCOPE updated; WP-0011 done. 173 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:21:39 +02:00
a8e65235a8 feat(incremental): I-2 digest + consistency-checker (WP-0011 T3)
A Merkle-style digest summarizes the derived tier (per-identity fingerprint +
incident edges as order-independent leaves) so equal states have equal digests
and the digest is stable under equivalent event orders. A ConsistencyChecker
recomputes the authoritative fold from the current source, compares it over a
sampled region, and on mismatch scoped-recomputes just the affected identities —
self-healing missed-delta drift, corrupted internal state, and vanished pages.
Makes derived = f(canonical) verified, not asserted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:16:50 +02:00
d7d046cac0 test(incremental): delta maintenance == rebuild, retraction + split (WP-0011 T2)
Verify change-driven maintenance keeps the equivalence index equal to a
from-scratch rebuild under add / edit / remove: an edit into a new bucket
retracts the stale edge, an edit into equivalence adds one, and removing a
connector node propagates a retraction that splits a chorus. Equality checked
against a fresh build() oracle on every operation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:14:32 +02:00
0b3ab2086f feat(incremental): indexed equivalence — blocking + verify (WP-0011 T1)
Detect equivalence (distinct identities holding the same page) without pairwise
O(N²): MinHash/LSH bands over content shingles + normalized-title buckets
generate candidates (blocking), then exact-fingerprint or Jaccard>=threshold
confirm them (verify), with curator decision-log bindings always forming edges.
Groups are the connected components of the edge set. Includes the incremental
add/update/remove internals used by T2. Matches a brute-force oracle. New
incremental/ package (minhash primitives + EquivalenceIndex).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:13:06 +02:00
d85d019543 feat(views): wire derived views onto InformationSpace + integration (WP-0010 T5)
Expose backlinks(name), recent_changes(), all_pages(), site_map() on
InformationSpace. Integration test exercises all four over two shards (BackLinks
aggregate across shards, AllPages/SiteMap span the union, RecentChanges merges an
alias decision with shard edits). SCOPE updated; WP-0010 done. 152 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:05:12 +02:00
3a5acdcb28 feat(views): AllPages + SiteMap enumeration views (WP-0010 T4)
AllPages enumerates the union's distinct pages, collapsing chorus (same key
across shards) and equivalence-bound identities into one entry via union-find,
noting divergence when members' bodies differ (collapse acknowledged, not
silent). SiteMap builds the namespace tree from page placements, spanning shards.
Both derived/recomputable and presentation-free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:03:15 +02:00
34b0c539f3 feat(views): RecentChanges merged change feed (WP-0010 T3)
One newest-first feed merging the coordination journal (overlay/alias/fork/merge/
binding decisions, with actor + payload) and shard change signals (page
source_rev / mtime). Each entry carries provenance: the originating shard for an
edit, or 'coordination' (and the actor) for a decision. Non-temporal revision
tokens are skipped gracefully. Derived/recomputable; notify-streaming later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:59:11 +02:00
da540d4eea feat(views): BackLinks derived view over the union link graph (WP-0010 T2)
For any page name, the set of pages that link to it: extract wikilinks from every
union page (new UnionGraph.iter_pages enumeration) and index the resolved ones by
target name. Red-links create no backlinks; entries carry source provenance; a
chorus target aggregates the backlinks of all members under one name. Derived/
recomputable, stores nothing canonical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:56:48 +02:00
951b24300d feat(views): wikilink + red-link model (WP-0010 T1)
A CommonMark wikilink extension: extract [[Target]] / [[Target|label]] from a
page body (skipping fenced + inline code, preserving offsets), and resolve each
target through the union — resolved is a link, unresolved is a createable
red-link (never a dropped reference). CamelCase auto-linking is off by default,
opt-in per space, and never double-counts a target already inside [[...]]. Link
model + resolution are core; rendering stays L6. New views/ package.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:55:06 +02:00
c731c96634 feat(coordination): git backend wiring + verbatim log migration (WP-0009 T4)
InformationSpace.git_backed(space_id, repo_path) wires the git coordination log;
the default constructor stays in-memory for tests (new keyword-only store=). A
one-time importer (migrate_space / import_log / JSONL export+import) replays an
existing in-memory or JSON log into git verbatim — preserving seq, timestamp and
actor (union-without-erasure) and refusing out-of-order import. Same fold after
migration; no behavioural change to overlay/union. SCOPE updated; WP-0009 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:49:55 +02:00
f0fee65cc0 test(coordination): cross-process read-your-writes + fold parity (WP-0009 T3)
Verify the git backend's fold reads the durable log into CoordinationState with
unchanged semantics, and that read-your-writes holds across separate handles and
separate OS processes against the same space ref (one test spawns a real
subprocess that appends, then reads it back). Cross-process fold equals the
in-memory fold for the same event sequence (derived = f(log)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:47:08 +02:00
34432c2e15 feat(coordination): per-space append authority (lease) (WP-0009 T2)
A single append authority per space serializes appends into a total order: at
most one node holds a space's lease; only the holder writes, non-holders forward
their append intent to the holder. Leases are time-bounded and re-grantable, so
a dead holder's lease expires and a new node resumes from the log head (seq stays
contiguous). A stale ex-holder discovers it is no longer the holder and forwards
rather than writing, so a partitioned node cannot fork the log. Works over both
in-memory and git stores. Single-coordinator only (distributed leasing out of scope).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:45:52 +02:00
45a858ead0 feat(coordination): git-backed DecisionLog event store (WP-0009 T1)
Factor DecisionLog storage behind an EventStore abstraction: InMemoryEventStore
stays the default/test double, GitEventStore makes the coordination log
git-addressable. Each space is a ref (refs/spaces/<sha1>); append writes an
immutable one-blob commit and advances the ref under compare-and-swap, so the
commit chain is the per-space total order and a racing appender can never fork
the log. Deterministic stable-JSON event serialization. Zero runtime deps
(git CLI via subprocess). API and fold unchanged across backends.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:41:27 +02:00
b31e9bc337 Add capability registry scaffold and seed entries from reuse-surface
Bootstrap registry/indexes/capabilities.yaml and migrate helix_forge
capability entries owned by this repository for federation publishing.
2026-06-16 01:34:23 +02:00
e50dcc6b5c chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:57:09 +02:00
a165cced33 feat(engine): ext.struct typed-records built-in; close engine implementation (WP-0014 T6)
engine/extensions/struct.py: ext.struct (typed records) — in-text frontmatter
parse + ON_WRITE validation (allowed-fields, content-preserving), ON_READ tags
PageShape.TYPED_RECORD, ON_PROFILE raises structured-payload. Proves the framework:
feature absent when off (opaque prose, honest profile), present + profile-reflected
when on; works through InformationSpace edit. SCOPE updated. 6 tests, 107 total,
~97% coverage, pyflakes clean. Marks T6 + SHARD-WP-0014 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:56:22 +02:00
8393a9c55d chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:36:55 +02:00
ff96ee0c48 feat(engine): EngineShardAdapter — engine as a canonical-mode shard (WP-0014 T5)
engine/adapter.py: EngineShardAdapter implements adapters.ShardAdapter (read/write
run extension transform hooks; profile derived from active extensions, E-5;
current_rev for apply-under-drift) + build_engine_shard() helper (explicit ids or
activation provider). runtime.available() added. Engine shard passes assert_conformant
and attaches to an InformationSpace — resolve + edit (overlay->apply->write-through)
work, and the declared profile reflects the active extensions. 5 tests green, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:36:24 +02:00
8b353f1077 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:27:57 +02:00
b9bb1f7d10 feat(engine): capability profile derived from active extensions (WP-0014 T4, E-5)
engine/profile.py: engine_base_profile() (kernel-only c2-minimum profile),
ProfileContribution (an extension's ON_PROFILE contribution: axis raises + verbs),
derive_profile() folds active extensions' contributions onto the base in deterministic
order then validate() — so configuration->capability is one chain and composition can
never yield an impossible profile (encrypted+native-query rejected). 5 tests green,
96 total, pyflakes clean. Marks T4 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:27:11 +02:00
c40fa3c934 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:16:54 +02:00
54c2bf2ae5 feat(engine): per-shard extension activation (WP-0014 T3, ADR-0001)
engine/activation.py: ActivationContext (shard/tenant, no authz), pluggable
ActivationProvider protocol, StaticProvider standalone default (zero-dep, global
flags + per-shard scoping + per-ext config), ActivationResolver (candidate ids ->
active set / activation profile), and feature_control_provider() lazy factory
(returns None when feature_control_sdk absent -> degrade to static; OpenFeature-
shaped when present). Availability only. 6 tests green, coverage held, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:15:33 +02:00
6d8bd837a4 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:12:18 +02:00
b48a99d3c2 feat(engine): typed-extension runtime (WP-0014 T2)
engine/extension.py: Extension contract (id/provides/declares_types/depends_on/
conflicts_with + bound hooks), ExtensionRuntime (register-with-verification,
activate = dependency closure + conflict + type-collision checks rejecting
impossible profiles), and ActiveExtensions with deterministic (topological, id-tie)
hook dispatch — transform hooks chain, collect hooks gather. 9 tests green,
coverage floor held, pyflakes clean. Marks T2 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:11:18 +02:00
9b7f86ba69 test: harden suite with error-path contracts + coverage floor (98%)
Adds tests/test_error_paths.py covering real failure contracts (red-link
single() KeyError, unknown/unattached-shard apply_overlay, kernel.delete
missing, conformance survives a broken profile, Placement str). Adds a
[tool.coverage.report] fail_under=90 floor (engages on pytest --cov, not bare
pytest). 76 tests, 98% coverage, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:05:16 +02:00
74142096d0 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:58:54 +02:00
2100e956aa feat(engine): page-store kernel skeleton (WP-0014 T1)
engine/ package: EngineKernel (in-process page store with per-page version
history; create/edit-as-version, recoverable delete-tombstone, keys, current_rev)
+ wikilink extraction + in-shard link resolution / red-link detection (EC-1..EC-4).
Reuses model/provenance; git-IS-store backing slots in later. 6 tests green,
pyflakes clean, full suite green. Marks T1 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:57:31 +02:00
e62560eb5a chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:52:14 +02:00
b147d3e831 workplan: SHARD-WP-0014 wiki-engine implementation (kernel + typed-extension runtime + activation)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:51:12 +02:00
cdcf4b09aa adr: ADR-0001 engine activation via feature-control (OpenFeature, availability-only)
Records the accepted decision: shard-wiki's native engine uses feature-control via
OpenFeature for per-shard extension activation (availability only, never authz),
provider-pluggable with a LocalProvider standalone default (mirrors the identity
ladder), at the engine layer, consuming the mature feature-control.evaluate slice.
Adds spec/adr/ series + README; hub decision abf7830f recorded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:48:49 +02:00
b21efe307b chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:05:45 +02:00
e18397272a spec(SHARD-WP-0013 T6): wire-up + close-out
spec/README + SCOPE list WikiEngineCoreArchitecture.md; CoreArchitectureBlueprint
cross-links the engine as a canonical-mode shard (federation/union stay in the
orchestrator). reuse-surface engine capability promoted D2->D3 (4204255).
Marks T6 + SHARD-WP-0013 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:02:15 +02:00
0ee972f2e2 spec(SHARD-WP-0013 T5): WikiEngineCoreArchitecture.md — small core + typed extensions
Headless, API-first, agent-optimized native engine = canonical-mode shard backend.
Thesis: a page-store kernel with a typed-extension runtime; everything beyond the
c2-minimum is a typed extension activated per shard, and the shard's §A capability
profile is DERIVED from its active extensions (configuration->capability->conformance).
9 engine invariants (engine-is-one-shard, small kernel, per-shard activation,
profile-from-extensions, headless/agent-first, reuse-not-reinvent, typed+verified).
Kernel (4 concepts), typed-extension model (typed hooks + deterministic composition +
feature-control activation), T2 featureset/conflict-mediation realized, engine-as-shard,
agent-first API surface, module sketch, reuse (consumes feature-control/authorization;
G1 framework proposal), traceability, decisions/open, stability note. Marks T5 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:54:40 +02:00
bb1b54e0af chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 22:45:17 +02:00
b70f1c9acc intent(SHARD-WP-0013 T4): ratify additive native engine — headless, API-first, agent-optimized
Ratified by tegwick (decision 84ffdb48 resolved). INTENT.md amended: native
reference wiki-engine as a canonical-mode shard backend, reframed as HEADLESS &
API-FIRST — small typed-extension core optimized for integrating heterogeneous
data sources and efficient agent/automation access; no bundled UI. Edits:
Primary Utility framing + bullet, non-goal clarifier, new Design Principle,
Stability Note amendment. SCOPE.md: engine core in scope, UI/rendering out.
Orchestrator role unchanged; union-without-erasure + shard sovereignty preserved.
Marks T4 done; unblocks T5 (WikiEngineCoreArchitecture.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:43:46 +02:00
8de044bbde chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 22:29:53 +02:00
67d851be0b history(SHARD-WP-0013 T3): reuse-surface gap proposals + consumptions; derived-views registered
Registered capability.wiki.derived-views in reuse-surface (8 wiki.* total, index 20,
validate ok, pushed d985e68). Tracked contributions note (history/260615-...): proposes
two cross-cutting gaps to reuse-surface (G1 typed-extension-framework, G2
translation-fidelity) and records consumptions (feature-control.evaluate,
authorization.policy-evaluate). Sent to the reuse-surface agent via send_message. Marks T3 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:28:29 +02:00
46ec6f2a5f spec(SHARD-WP-0013 T2): engine-facing capability structure layer over the UC catalog
Adds 'Capability structure for the wiki engine (reuse-surface-aligned)' to
UseCaseCatalog.md (no UC renumbered): a small always-on engine core (EC-1..EC-5),
ten typed per-shard-activatable extensions (X-OVERLAY/AUTHZ/VIEW/STRUCT/ADDR/COMP/
PROV/COLLAB/FED/ATT) each mapped to a reuse-surface capability with activation
defaults, and a conflict-mediation map turning conflicting requirements into
mechanisms. Bridges demand -> the engine typed-extension model (T5). Marks T2 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:16:06 +02:00
ad4a2dbf5a chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 22:11:16 +02:00
23bc597343 chore(SHARD-WP-0013): T1 done — shard-wiki registered on reuse-surface (7 wiki.* capabilities)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:10:12 +02:00
ec7a1ec946 spec(SHARD-WP-0013): fold in confirmed reuse-surface coordinates + mechanism
reuse-surface = capability registry at /home/worsch/reuse-surface (helix_forge):
Markdown entries via templates/capability-entry.template.md, D/A/C/R maturity
vectors, reuse-surface CLI (validate/query/export/overlaps), hub at
reuse.coulomb.social. T1 now registers shard-wiki's capabilities as registry
entries; T3 contributes gaps back via REUSE-WP/agent message; T5 evaluates
reusing sibling feature-control for per-shard activation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 17:32:49 +02:00
d6d7cc555f chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 17:18:00 +02:00
1dd1def40f workplan: SHARD-WP-0013 wiki-engine prep (reuse-surface, UC systematization, WikiEngineCoreArchitecture)
Prep for an additive native wiki-engine framed as shard-wiki's reference
first-party shard backend (canonical-mode shard) — small core + typed-extension
framework, per-shard activation, integrated featureset with conflict mediation.
Tasks: confirm+register on the reuse surface; systematize the UC catalog around a
reuse-surface capability taxonomy; surface capability gaps back; narrow INTENT
amendment + ratified decision (gate); author WikiEngineCoreArchitecture.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 17:16:54 +02:00
6d341cd4e6 chore: restore SHARD-WP-0003 workstream id (re-link to existing finished workstream)
The 56af8185 workstream still exists in the DB (finished); clearing the id only
unlinked it (slug taken, cannot recreate). Restoring re-links the file.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 15:55:55 +02:00
1b3d4aaa39 chore: clear stale SHARD-WP-0003 workstream id (re-register on next sync)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:08:28 +02:00
6e49bd3a4b chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 14:07:24 +02:00
d2c73b02d9 workplans: register follow-up implementation roadmap (WP-0009..0012)
WP-0009 git-backed DecisionLog + per-space append authority (keystone backing);
WP-0010 derived views (wikilinks, BackLinks, RecentChanges, AllPages/SiteMap);
WP-0011 incremental union + equivalence index + I-2 verification;
WP-0012 second adapter (git-IS-store) validating the contract on a new substrate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 14:05:21 +02:00
042e198286 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 13:47:25 +02:00
3ea0cc1226 feat(space): wire write path into InformationSpace; integration (WP-0008 T6)
edit()/overlay()/apply_overlay() on InformationSpace. edit() unifies the write
path through one principled route — draft overlay then apply: write-through-capable
target fast-forwards (APPLIED), read-only target keeps the draft as local truth
(KEPT_DRAFT), external drift refuses (no clobber). Integration tests cover all
four. 64 tests green, pyflakes clean. Flips WP-0008 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:46:32 +02:00
4be2f190a0 feat(union): overlay-aware resolution (WP-0008 T5)
resolve() layers open overlays onto canonical pages (overlay_state=DRAFT always
surfaced; overlaid body projected when policy.show_drafts); draft-only edits make
a not-yet-existing page resolvable. Never hides an unapplied overlay (I-4). Policy
gains show_drafts. 4 tests green, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:29:06 +02:00
7d00ae758e feat(coordination): apply-under-drift (WP-0008 T4)
OverlayEngine.apply: read-only target → KEPT_DRAFT; base_rev==current →
fast-forward write-through (APPLIED, MERGE_DECIDED closes overlay); drift →
REFUSED_DRIFT (no clobber, I-5). 5 tests green, pyflakes clean. (blueprint §8.6)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 13:25:10 +02:00
715ab1ca00 feat(coordination): patch rendering (unified diff) (WP-0008 T3)
render_patch(overlay, base_body) → Patch (pure difflib unified diff, target-
labelled); is_empty when unchanged. 3 tests green, pyflakes clean. (ADR-05)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 12:32:47 +02:00
d797bc5ee4 feat(coordination): Overlay model + OverlayEngine.draft (WP-0008 T2)
Overlay value type (id, target, base_rev, body, state) recorded as an
OVERLAY_CREATED decision-log event (coordination-canonical); get()/open_overlays()
reconstruct from the fold. 4 tests green, pyflakes clean. (ADR-05, blueprint §8.2)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 11:56:53 +02:00
92d5774baf feat(adapters): writable FolderAdapter + positive write conformance (WP-0008 T1)
FolderAdapter(writable=True) declares WRITE+PER_PAGE, implements write() and
current_rev() (mtime token for drift detection). Conformance gains a
content-preserving positive write probe for WRITE-claiming adapters. 5 tests
green, full suite green, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 11:09:43 +02:00
e24f0034a0 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 10:29:21 +02:00
b2ea276c00 workplan: SHARD-WP-0008 write path (overlay engine, writable adapter, apply-under-drift)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 10:28:37 +02:00
9fedb31d8b chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 10:13:40 +02:00
517cf1d282 feat(space): InformationSpace orchestrator + integration; foundation slice complete (WP-0007 T7)
InformationSpace ties the slice together: conformance-gated attach → resolve →
read, with alias() recording coordination decisions in the log. Exposed from the
package root. End-to-end integration test (two folder shards → union read with
layered provenance + chorus + alias redirect + red-link + nonconformant-rejected).
pyflakes clean, 39 tests green. Flips WP-0007 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 10:12:38 +02:00
b44b2a74a4 feat(policy,union): policy leaf + UnionGraph resolution with chorus (WP-0007 T6)
policy/ leaf (CanonicalSource presets, default chorus). union/ UnionGraph:
identity-keyed resolve (alias-redirect via log fold → union lookup → chorus →
red-link); chorus records divergent peers in each page's provenance envelope
(union without erasure); designated-canonical orders the pick. Imports down only.
6 tests green. (blueprint §8.4, ADR-01)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 10:02:09 +02:00
24108b65aa feat(coordination): event-sourced DecisionLog keystone (WP-0007 T5)
Append-only, totally-ordered-per-space decision log (in-process append authority;
git+lease later) with event types overlay/binding/alias/merge/fork, and a derived
fold to CoordinationState (aliases LWW, transitively-merged equivalence groups,
open overlays). derived = f(log); read-your-writes. 6 tests green. (blueprint §8.1)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:45:11 +02:00
6d48a1d3e6 feat(adapters): conformance suite — profiles verified not asserted (WP-0007 T4)
run_conformance/assert_conformant verify declared profile == observed behaviour
(profile validates, READ round-trips to the right shard, unclaimed verbs raise
NotSupported); ConformanceReport gives a precise capability diff. FolderAdapter
passes; lying-read and dishonest-write stubs fail. 3 tests green. (TSD §A.2)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:42:59 +02:00
9a4e00a05a feat(adapters): ShardAdapter contract + read-only FolderAdapter (WP-0007 T3)
Versioned ShardAdapter ABC (shard_id/profile/keys/read mandatory; optional verbs
raise NotSupported by default = honest absence). FolderAdapter reads a dir of
Markdown into Pages (relpath=key, mtime=source_rev, provenance envelope),
declaring a validated read-only file-store profile. 4 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:37:02 +02:00
5a77ea879c feat(model): Identity/Placement/Span, Page, CapabilityProfile (WP-0007 T2)
Identity = stable shard-scoped handle (not a fingerprint); Placement separate;
Span carries a layered provenance delta. Page model (PageShape incl. the four
computational shapes). CapabilityProfile with orthogonal-core axes + validate()
applying §6.5 implication rules that reject impossible profiles. Imports only
provenance/. 8 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 09:00:49 +02:00
aca9bf30f9 feat(provenance): layered ProvenanceEnvelope + effective() leaf (WP-0007 T1)
Dependency-free leaf rail: ProvenanceEnvelope + SpanProvenanceDelta + effective()
(page envelope ⊕ span delta; zero-cost inheritance when uniform). Liveness/
Staleness/OverlayState enums. 6 tests green. (blueprint §7.3, §11)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 08:55:35 +02:00
3b4f10a349 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 08:45:49 +02:00
f2dd2e124a workplan: SHARD-WP-0007 foundation implementation (first vertical slice)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 07:36:36 +02:00
1f3aba7aad chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 02:26:48 +02:00
d65f9e21f3 spec(SHARD-WP-0002): adapter contract (TSD §A, T11-T16+T18); workplan done
Adds the normative Shard Adapter Contract as TechnicalSpecificationDocument §A:
A.1 versioned capability contract (verbs + orthogonal-core spectra), A.2 verified
conformance suite (profiles not self-asserted), A.3 attachment-mode taxonomy +
image-is-not-a-store boundary, A.4 page model (incl. computational shapes,
identity/placement/equivalence, layered provenance), A.5 history portability,
A.6 syntax translation + fidelity report, A.7 addressing/navigation, A.8 gated
computational content. Updates TSD references/UC-count/next-work. Flips all 18
WP-0002 tasks + workplan done. Design layer complete.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:25:50 +02:00
802a80231a spec(SHARD-WP-0002): FederationArchitecture.md — federation design (T1-T10, T17)
Decision records for the 10 federation topics + the federation-model taxonomy,
each with a Decided/Deferred/Open footer: star orchestrator positioning, remix
primitives (overlay-default), identity/placement/equivalence + chorus,
event-sourced coordination journal + per-space append authority, server-primary
incremental union, notify-driven freshness, space lifecycle, reference-not-copy
transclusion, detection-core/resolution-policy preset catalog, the federation-
ops capability matrix, and the six selectable/composable federation models.
References (not duplicates) CoreArchitectureBlueprint + FederationRequirements.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:24:11 +02:00
40575045ca chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 02:20:55 +02:00
a2f15806f5 spec(SHARD-WP-0001): yawex-derived federation requirements (ADR-01..06); workplan done
Delivers spec/FederationRequirements.md — six ADR-ready design notes mapping
the yawex prior art onto the hardened CoreArchitectureBlueprint:
ADR-01 cross-shard resolution (yawex PageLookUp states as a checklist, keyed
on identity), ADR-02 namespace/path + shard roles (path=placement not identity),
ADR-03 derived views core-vs-adapter (BackLinks/RecentChanges/AllPages/SiteMap
core, Search hybrid), ADR-04 concrete provenance envelope fields, ADR-05 overlay/
patch/append lifecycle, ADR-06 wikilink+red-link extension (model core, render UI).
Resolves findings open-Qs #1-#3; access-model thread ratified. Flips WP-0001 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:19:43 +02:00
9b5fceb451 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 02:04:39 +02:00
d39e7f7765 spec(SHARD-WP-0006 T5): track §C as O-8..O-12; refresh decisions/traceability; close-out
Adds O-8 (preset bundles), O-9 (shard sharing vs tenant partition), O-10
(span authz + transclusion ⊕), O-11 (union under unavailability), O-12
(append-log throughput) to §12. Refreshes §14 decisions (event-sourced
coordination, conformance, I-2 verification) and §16 traceability (round-2
review + WP-0006). Flips SHARD-WP-0006 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:03:58 +02:00
1ad70a9c8a spec(SHARD-WP-0006 T4): incremental-equivalence correctness + I-2 verification (§8.7)
Fixes B-4. Incremental delta is not additive: a change processes bucket
exits (retract unsupported edges) + entries (add) + propagation across
equivalence neighbours, not just new candidates. Adds an I-2 verification
mechanism: per-partition Merkle-style digest + background consistency-checker
vs sampled fold → scoped self-healing recompute on drift. I-2 now
eventually-verified, not asserted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:02:33 +02:00
3a753a6f3b spec(SHARD-WP-0006 T3): adapter conformance suite (§6.6)
Fixes B-2. Capability profiles are verified, not self-asserted: the contract
ships a versioned conformance suite that exercises each declared verb/position
against observed behaviour; passing is an admissibility precondition (lying
profiles rejected at registration); mismatch reported as a capability diff.
Makes I-3 / §6.5 sound rather than aspirational.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:01:37 +02:00
08a2148079 spec(SHARD-WP-0006 T2): event-sourced coordination + per-space append authority
Resolves B-1+B-3. Coordination-canonical state = an append-only decision log
in the git journal (events: overlay/binding/alias/merge); queryable current
state = a derived fold (tier-3, indexed). Concurrency: one append authority
per space (lease/leader) → totally-ordered per-space log, read-your-writes
across instances, HA via re-grantable lease, partition yields to log integrity.
Updates §1, §4, §8.1, §8.6, §11. I-6 strengthened (coordination state is now
git-addressable history/patch/review), not bypassed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:00:43 +02:00
cbd29e0a32 spec(SHARD-WP-0006 T1): overview-reconciliation pass (§A)
Fixes A-1..A-4: §4 identity/placement/equivalence (removes the equivalence-
keys-on-identity contradiction), §4 projection (trivial default + extension
point) and provenance (layered), §10 policy surface (adds freshness,
conflict-resolution, compaction, tenant-partition knobs + preset-bundle note),
§3 diagram + §11 header (incremental-first, orthogonal-core). Overview now
matches the hardened body.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:58:48 +02:00
079652b61f chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 01:57:04 +02:00
84ab25eeb9 history+workplan: round-2 blueprint review; SHARD-WP-0006 hardening II
Records the round-2 critical review (history/260615-...-review-2) and
establishes SHARD-WP-0006 to: reconcile overview with hardened body (§A),
settle the journal/coordination-state model (event-sourced decision log;
single-vs-multi-writer concurrency — B-1+B-3), require an adapter conformance
suite (B-2), fix incremental-equivalence correctness + I-2 verification (B-4),
and track §C as O-8..O-11.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:56:42 +02:00
229d24d1cf chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 01:44:05 +02:00
f21b7b5259 spec(SHARD-WP-0005 T9): known scaling risks & open problems; close-out
Adds §12 'Known scaling risks & open problems' (O-1..O-7 with chosen
direction + revisit trigger); renumbers §13-17. Refreshes §14 decisions
(several earlier 'open' items now decided), §16 traceability (links the
review + per-finding section map), and I-1..I-13. Flips SHARD-WP-0005 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:43:14 +02:00
c895d33091 spec(SHARD-WP-0005 T8): tenant isolation of derived tier + history scaling
Fixes B-3/C-3. §9.1 structural per-tenant partitioning of the derived tier
(no shared cross-tenant cache; read-time filtering as defence-in-depth;
reconciles I-2+L5 per partition); new invariant I-13. §8.1 history stays
recoverable AND bounded (gc/repack, squash-compaction of churn preserving
recoverable endpoints, per-shard offload, anti-abuse hooks).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:41:09 +02:00
59c36ac9d1 spec(SHARD-WP-0005 T7): elegance pass — layered provenance, common-case projection, policy module
Fixes D-2/D-3/D-4. §7.3 effective-vs-own provenance (page envelope + span
deltas, near-zero per-span cost). §8.4 projection trivial-by-default
(plain lazy replication), derivation/liveness/view-registry as extension
points only for the computational/typed tail. §11 adds policy/ + provenance/
as dependency-free leaf rails (mechanism never in a rail), tightened import rules.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:39:43 +02:00
012d151fe8 spec(SHARD-WP-0005 T6): orthogonal core spectra + implied positions + interaction subset (§6.5)
Fixes D-1. ~6 independent core axes (substrate, write-granularity, opacity,
envelope, access, liveness) with the rest implied via published rules that
forbid impossible profiles; a small named axis-interaction table is the
degradation contract (proof obligation behind 'core logic written once').

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:38:02 +02:00
af840576e9 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 01:37:11 +02:00
0383d78440 spec(SHARD-WP-0005 T5): cache freshness & invalidation protocol (§8.8)
Fixes C-2 (invalidation). Per-binding modes (event-driven push / validator
poll / TTL), hybrid default, operational-envelope coupling (rate-limited
shards must not poll-per-read), single-flight coalescing + batched bulk
invalidation, freshness always represented in the provenance envelope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:36:51 +02:00
dc451b0f4e spec(SHARD-WP-0005 T4): scale the union — incremental-first + indexed equivalence (§8.7)
Fixes C-1/C-2. Incremental change-driven maintenance (notify->delta) is
primary; full rebuild is a rare, envelope-respecting, concurrent fallback
(not required cheap). Equivalence via blocking/LSH candidate-gen + verify +
incremental maintenance, replacing O(N^2). Index is derived, per-tenant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:36:09 +02:00
04be66161e spec(SHARD-WP-0005 T3): consistency, concurrency & conflict model (§8.6)
Fixes bug B-2. States the guarantee (read-your-writes for journal-owned
coordination-canonical state; causal across the derived tier; eventual+
freshness-labelled for sharded inputs). Conflict detection+representation =
core mechanism, resolution = policy. Overlay-apply-under-drift semantics
(fast-forward / three-way / refuse+re-present) and journal ordering.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:35:00 +02:00
c8dbe9b573 spec(SHARD-WP-0005 T2): split page identity / placement / content equivalence
Fixes bug B-1: page identity is a stable assigned handle (survives edits),
not a content fingerprint; fingerprints identify versions/content for the
equivalence mechanism. Chain: identity -> placements -> equivalence. (§7.2)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:07:37 +02:00
b5d2cbc330 spec(SHARD-WP-0005 T1): re-frame state model — three states, derived=f(canonical)
Replaces the two-bucket thesis with sharded-canonical / coordination-canonical
(journal: overlays, bindings, aliases, merges) / derived-disposable. Fixes the
I-2 contradiction (curator bindings can't be rebuilt). Updates §1, I-2, §3
dependency rule, §4 abstractions, §8.4. (review A-1)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:06:54 +02:00
ee2449d987 chore(SHARD-WP-0005): link workstream + task ids 2026-06-15 01:05:17 +02:00
f1dc7aff61 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 01:03:30 +02:00
dd812abb81 history+workplan: CoreArchitectureBlueprint review; SHARD-WP-0005 hardening
Records the critical review (history/260615-...) and establishes SHARD-WP-0005
to fold its findings (A-1, B-1..B-3, C-1..C-3, D-1..D-4) into the blueprint:
correctness (state re-frame, identity/equivalence split, consistency model),
scale (incremental-first union, equivalence indexing, cache invalidation),
elegance (orthogonal spectra, layered provenance, common-case projection,
policy module), security/history scaling, and a known-open-problems section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 01:01:19 +02:00
9b5b393519 spec: CoreArchitectureBlueprint — whole-system architecture from research
The optimal architecture synthesised from INTENT + the full research arc:
- Thesis: canonical at the edges, derived in the middle (orchestrator not engine)
- Dual narrow waist: adapter contract (15 capability spectra) + page model
- 6 layers + provenance/capability rails; L4 union/projection is a rebuildable cache
- Federation-model taxonomy (plural/composable); two-axis projection model;
  moldable view registry; identity != placement; computational content in scope
  as page-model+projection, out as execution platform
- Concrete src/ module layout with downward-only dependency rule
- Canonical data flows; policy surface; tradeoffs; full traceability to INTENT/UCs
References ArchitectureBlueprint.md as the L5 authorization sub-blueprint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:41:14 +02:00
6a88eb7710 chore(SHARD-WP-0002): link T17/T18 state_hub_task_ids; sync task drift
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:27:07 +02:00
d898307b7e chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 00:23:15 +02:00
90a854f841 spec(SHARD-WP-0002): re-fold synthesis v3 + computational page model (T17, T18)
Folds the later syntheses into the federation/adapter-contract workplan:
- T11 capability spectra 13 -> 15 (+provenance granularity, +computational/
  liveness); +derive-projection/execute verbs
- T12 four computational page shapes (UC-83/84) + typed-graph statements
- T13 paired-text/nbdime + DB-version-rows + partial-history honesty
- T14 git-IS-store + direct-DB + image-is-not-a-store boundary
- T15 structured-re-evaluable-value opacity point; non-Markdown computed
- T16 two-axis projection model + moldable view registry + named-chunk transclusion
- NEW T17 federation-model taxonomy (fork+journal / VCS-replication+ping /
  query-join / feed / activity-streams / engine-mirror; selectable+composable)
- NEW T18 computational/executable content (in scope as page-model+projection,
  out as execution platform; execute = gated capability)
- Context/decision-tables/acceptance/order updated; UC range -> UC-26-UC-84

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:22:59 +02:00
3d137c96b6 research: SHARD-WP-0004 synthesis (the computational page model); workplan done
Consolidates the 8 computational/interactive-knowledge dives into one
source/derivation/projection model on two axes (replication<->derivation,
live<->snapshot); four computational page shapes; recommendation that
executable content is in scope as a page-model+projection concern, out of
scope as an execution platform (capability-gated, degrade to snapshot;
no INTENT amendment). Flips SHARD-WP-0004 status: done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:12:03 +02:00
f2f9f31df8 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 00:04:50 +02:00
134 changed files with 12330 additions and 228 deletions

View File

@@ -1,21 +0,0 @@
---
active: true
iteration: 1
session_id: b24b53ca-a25b-4a18-8331-925c4100d39f
max_iterations: 20
completion_promise: "HEUREKA"
workplan_id: SHARD-WP-0004
workplan_file: workplans/SHARD-WP-0004-computational-knowledge-systems.md
started_at: "2026-06-14T20:35:07Z"
---
Read the workplan at `workplans/SHARD-WP-0004-computational-knowledge-systems.md`.
If every task has `status: done` AND frontmatter `status: done`:
run `rm -f .claude/ralph-loop.local.md` first (deactivates the loop so the stop hook exits cleanly),
then output <promise>HEUREKA</promise>.
Otherwise implement the next `todo` task as described in the workplan.
Set task `in_progress` when starting, `done` when complete.
When all tasks are done set frontmatter `status: done`.

20
.claude/rules/agents.md Normal file
View File

@@ -0,0 +1,20 @@
## Kaizen Agents
Specialized agent personas available on demand via the state-hub MCP.
**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category
**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them
Common agents:
| Agent | Category | When to use |
|-------|----------|-------------|
| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature |
| `code-refactoring` | quality | Code quality analysis and safe refactoring |
| `test-maintenance` | testing | Diagnose and fix failing tests |
| `requirements-engineering` | process | Prevent interface/mock mismatches upfront |
| `keepaTodofile` | process | Maintain TODO.md during work |
| `project-management` | process | Track status, determine next steps |
| `datamodel-optimization` | quality | Optimize dataclasses and data structures |
All 17 agents: call `list_kaizen_agents()` for the full list.

View File

@@ -0,0 +1,8 @@
## Architecture
<!-- TODO: Describe the key design decisions and component structure.
Key modules, data flows, external integrations, state machines, etc. -->
## Quick Reference
`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference

View File

@@ -0,0 +1,50 @@
# Credential and access routing
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
for inference. Run this check **before** requesting secrets, API keys, SSH access,
login tokens, or database passwords — in any repo, not only `ops-warden`.
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
other credential need belongs to another subsystem. **Do not** message
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
### Lookup (do this first)
```bash
warden route find "<describe your need>" --json
warden route show <catalog-id> --json
```
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
| Agent runtime | How to orient |
| --- | --- |
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
### Quick routing table
| I need… | Owner | ops-warden executes? |
| --- | --- | --- |
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes**`warden sign` |
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
| Authorization decision | flex-auth | No — route only |
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
### Anti-patterns (do not do these)
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
- Pasting secrets into Git, State Hub, workplans, logs, or chat
### Other capabilities (reuse-surface)
Non-credential capabilities are usually discovered through **reuse-surface** federation
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
every repo's agent instructions because it is high-frequency, high-risk, and easy to
get wrong.
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`

View File

@@ -0,0 +1,38 @@
## First Session Protocol
Triggered when `get_domain_summary("consumer")` shows **no workstreams**.
The project is registered but work has not yet been structured.
**Step 1 — Read, don't write**
- `~/the-custodian/canon/projects/consumer/project_charter_v0.1.md` — purpose, scope
- `~/the-custodian/canon/projects/consumer/roadmap_v0.1.md` — planned phases
- Scan repo root: README, directory structure, existing code or docs
**Step 2 — Survey in-progress work**
Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete.
**Step 3 — Propose workstreams to Bernd**
Propose 13 workstreams — each a coherent strand, weeks to months, anchored to a
roadmap phase. **Wait for approval before creating.**
**Step 4 — Create workplan file first, then DB record (ADR-001)**
```
workplans/SHARD-WP-NNNN-<slug>.md ← write this first
```
Then register in the hub:
```
create_workstream(topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", title="...", owner="...", description="...")
create_task(workstream_id="<id>", title="...", priority="high|medium|low")
```
**Step 5 — Record the setup**
```
add_progress_event(
summary="First session: structured consumer into N workstreams, M tasks",
event_type="milestone",
topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0",
detail={"workstreams": [...], "tasks_created": M}
)
```
<!-- Delete or archive this file once past first session -->

View File

@@ -0,0 +1,8 @@
## Repo boundary
This repo owns **shard-wiki** only. It does not own:
<!-- TODO: List what belongs in adjacent repos, e.g.:
- SSH key management → railiance-infra/
- State hub code → state-hub/
-->

View File

@@ -0,0 +1,5 @@
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
**Domain:** consumer
**Repo slug:** shard-wiki
**Topic ID:** 4c2e5315-2cb9-447c-9d16-a39bdb0aabd0

View File

@@ -0,0 +1,85 @@
## Session Protocol
Dev Hub (State Hub API): http://127.0.0.1:8000
MCP server name in `~/.claude.json`: `dev-hub`
**Step 1 — Orient**
Read the offline-safe brief first — it works without a live hub connection:
```bash
cat .custodian-brief.md
```
Then call the MCP tool for richer cross-domain context when MCP tools are exposed:
```
get_domain_summary("consumer")
```
If MCP tools are unavailable in the current agent session, use the REST API:
```bash
curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool
```
If the hub is offline: `cd ~/state-hub && make api`
**Step 2 — Check inbox**
With MCP tools:
```
get_messages(to_agent="shard-wiki", unread_only=True)
```
Mark read with `mark_message_read(message_id)`. Reply or act on coordination
requests before proceeding.
Without MCP tools:
```bash
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
| python3 -m json.tool
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
-H "Content-Type: application/json" -d '{}'
```
**Step 3 — Scan workplans**
```bash
ls workplans/
```
For each file with `status: ready`, `active`, or `blocked`, note pending
`wait`/`todo`/`progress` tasks.
**Step 4 — Present brief**
1. **Active workstreams** for `consumer` — title, task counts, blocking decisions
2. **Pending tasks** from `workplans/` + any `[repo:shard-wiki]` hub tasks
3. **Goal guidance** — if `goal_guidance` in summary:
- `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"*
- `alignment_warnings`: flag if active work is not aligned with current goal
4. **Suggested next action** — highest-priority open item
5. **SBOM status** — flag if `last_sbom_at` is unset for this repo
If no workstreams: follow First Session Protocol (`first-session.md`).
**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()`
> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`)
> are First Session Protocol only. Work structure belongs in repo files (ADR-001).
**Session close:**
With MCP tools:
```
add_progress_event(summary="...", topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", workstream_id="<uuid>")
```
Without MCP tools:
```bash
curl -s -X POST http://127.0.0.1:8000/progress/ \
-H "Content-Type: application/json" \
-d '{"topic_id":"4c2e5315-2cb9-447c-9d16-a39bdb0aabd0","workstream_id":"<uuid>","event_type":"note","summary":"what changed","author":"codex"}'
```
If workplan files were modified, ensure the local copy is up to date first:
```bash
git -C <repo_path> pull --ff-only
cd ~/state-hub && make fix-consistency REPO=shard-wiki
```
For repos where implementation runs on a remote machine (e.g. CoulombCore),
use the combined target which pulls before fixing:
```bash
cd ~/state-hub && make fix-consistency-remote REPO=shard-wiki
```
**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback
will sync the file to match DB. **C-16** (repo behind remote) blocks all writes
until you pull — intentional to prevent clobbering remote progress.

View File

@@ -0,0 +1,19 @@
## Stack
<!-- TODO: Fill in language, frameworks, and key dependencies -->
- **Language:**
- **Key deps:**
## Dev Commands
```bash
# TODO: Fill in the standard commands for this repo
# Install dependencies
# Run tests
# Lint / type check
# Build / package (if applicable)
```

View File

@@ -0,0 +1,40 @@
## Workplan Convention (ADR-001)
File location: `workplans/SHARD-WP-NNNN-<slug>.md`
ID prefix: `SHARD-WP-`
Work items originate as files in this repo **before** being registered in the hub.
Canonical workplan/workstream frontmatter statuses are:
`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`.
Use `proposed` for a newly drafted plan, `ready` after review against current
repo state, and `finished` when implementation is complete. `stalled` and
`needs_review` are derived health labels, not stored statuses.
Closed workplans may be moved to `workplans/archived/` with a completion-date
prefix: `YYMMDD-SHARD-WP-NNNN-<slug>.md`. The frontmatter id remains
unchanged; the prefix is only for quick visual reference.
Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**:
`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids
`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed
directly. Promote anything requiring analysis, design, approval, dependencies, or
multiple planned phases into a normal workplan.
Ecosystem todos from other agents arrive as `[repo:shard-wiki]` hub tasks —
visible at session start. Pick one up by creating the workplan file, then registering
the workstream.
Task blocks use this shape:
```task
id: SHARD-WP-NNNN-T01
status: wait | todo | progress | done | cancel
priority: high | medium | low
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
```
Status progression is `todo``progress``done`; use `wait` for waiting or
blocked work and `cancel` for stopped work.
<!-- Ralph Loop rules and HEUREKA sequence: ~/.claude/CLAUDE.md — do not duplicate here -->

View File

@@ -2,41 +2,46 @@
# Custodian Brief — shard-wiki
**Domain:** whynot
**Last synced:** 2026-06-14 21:55 UTC
**Last synced:** 2026-06-15 22:57 UTC
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
## Active Workstreams
### computational / interactive-knowledge systems research
Progress: 6/8 done | workstream_id: `5fc3911a-1c68-4826-bbd2-b892dec8f981`
### second adapter — git-IS-store shard (contract validation on a new substrate)
Progress: 0/3 done | workstream_id: `9e24eeb0-c0f0-41e6-a1ca-88d71e4139ea`
**Open tasks:**
- · Processing.org and Processing.js `0f54cb0e`
- · Strudel.cc — live-coding REPL `2b099639`
- · GitShardAdapter — read over a git working tree/repo `8a1c7c80`
- · Write = commit; current_rev = sha (drift) `b47dfb86`
- · History adopt + integration with union/overlay `4c895f42`
### federation architecture design
Progress: 0/16 done | workstream_id: `2af4c46d-cbfd-40ea-a94b-d9e60b0f9945`
### incremental union maintenance + equivalence index + I-2 verification
Progress: 0/4 done | workstream_id: `78d48bcf-6482-4266-bc81-084b7ec1cd80`
**Open tasks:**
- · Architecture positioning and boundaries `ea8fdb22`
- · Remix primitives: fork, overlay, import, reference `fb7d4bce`
- · Equivalent page identity and multi-version presentation `8f2a333d`
- · History, attribution, and coordination journal `5f39f48d`
- · Union composition layer `3ff71e11`
- · Change notification and subscription transports `9596e5e8`
- · Information space lifecycle `38134064`
- … and 9 more open tasks
- · Equivalence index: blocking + verify `842f480b`
- · Incremental maintenance (delta, not additive) `2da4e0b8`
- · I-2 verification: digest + consistency-checker `b602ce31`
- · Wire incremental tier behind resolution + views `2f3d083c`
### shard-wiki requirements from yawex prior art
Progress: 0/6 done | workstream_id: `0ed023a2-760b-4990-b931-8ee1f41ea08f`
### derived views — wikilinks, BackLinks, RecentChanges, AllPages/SiteMap
Progress: 0/5 done | workstream_id: `2fe15330-ddf6-4b0f-8e55-ada341375d35`
**Open tasks:**
- · Design federation page-resolution model (yawex state space as inspiration) `ebc036e4`
- · Define namespace/path model and page+shard roles `431b4d28`
- · Specify union-level derived views (BackLinks, RecentChanges, AllPages, SiteMap, Search) `564545ec`
- · Provenance & freshness model for pages/revisions/projections `738326f5`
- · Overlay / lightweight-patch model (from yawex append/comment) `a268de6a`
- · Markdown link semantics: wikilink + red-link extension `a7499f3e`
- · Wikilink + red-link model `792660c3`
- · BackLinks (core) `431a54c3`
- · RecentChanges (core) `270c1c31`
- · AllPages / SiteMap (core) `898ba43e`
- · Wiring + integration `7157544b`
### git-backed DecisionLog + per-space append authority
Progress: 0/4 done | workstream_id: `4fb5b29b-955c-4f37-85cf-58b4643ab1ca`
**Open tasks:**
- · Git event-store backend (append = commit/object) `a8fcbb3e`
- · Per-space append authority (lease) `62abd162`
- · Fold over the git log + read-your-writes across processes `8cc3691e`
- · Migration + wiring `281e1db4`
---
## MCP Orientation (when available)

17
.repo-classification.yaml Normal file
View File

@@ -0,0 +1,17 @@
repo_classification:
standard: Repo Classification Standard
version: '1.0'
classified_at: '2026-06-22'
classified_by: agent
category: project
domain: consumer
secondary_domains: []
capability_tags:
- knowledge
- documentation
business_stake:
- product
- experience
business_mechanics:
- coordination
- operation

243
AGENTS.md
View File

@@ -1,62 +1,219 @@
# AGENTS.md
# shard-wiki — Agent Instructions
Guidance for agents working in `shard-wiki`.
## Repo Identity
## Read First
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
1. `INTENT.md` — aspiration and boundaries (stable; architectural changes are rare).
2. `SCOPE.md` — what we are achieving now and current maturity.
3. `.custodian-brief.md` — State Hub snapshot (generated; do not edit manually).
**Domain:** consumer
**Repo slug:** shard-wiki
**Topic ID:** `4c2e5315-2cb9-447c-9d16-a39bdb0aabd0`
**Workplan prefix:** `SHARD-WP-`
## Documentation Layout
---
This repo follows the CoulombSocial / HelixForge / MarkiTect documentation
layout (recommendation, not strict law). Efficient retrieval by purpose:
## State Hub Integration
| Path | Purpose |
|------|---------|
| `INTENT.md` | Aspiration and boundaries |
| `SCOPE.md` | Top-level view of current achievement; closes gap to INTENT |
| `research/` | Exploration results (`yymmdd-` prefix on files or subdirs) |
| `demand/` | Inbound requests not yet reviewed into spec or workplans |
| `spec/` | Implementation guardrails (PRD, TSD, use cases, architecture) |
| `workplans/` | State Hubregistered implementation tasks |
| `docs/` | Stakeholder documentation (users, developers, humans, agents) |
| `wiki/` | Perspective-free interconnected knowledge (wiki UI when connected) |
| `issues/` | Mirror of relevant open tickets when ticket systems are in use |
| `history/` | Archived material (`yymmdd-` prefix); out of scope for daily work |
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
there is no MCP server for Codex agents.
**Mode of operation:** close SCOPE → INTENT while learning; refine both as needed.
| Context | URL |
|---------|-----|
| Local workstation | `http://127.0.0.1:8000` |
| Remote via tunnel | `http://127.0.0.1:18000` |
## Domain Vocabulary
Honor terms from `INTENT.md`: shard, root entity, adapter contract, projection,
overlay, coordination journal, shard modes. Do not invent parallel vocabulary.
## Build And Test
### Orient at session start
```bash
pip install -e ".[dev]"
pytest
ruff check
ruff format
# Offline brief — works without hub connection
cat .custodian-brief.md
# Active workstreams for this domain
curl -s "http://127.0.0.1:8000/workstreams/?topic_id=4c2e5315-2cb9-447c-9d16-a39bdb0aabd0&status=active" \
| python3 -m json.tool
# Check inbox
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
| python3 -m json.tool
```
## State Hub
Mark a message read:
```bash
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
-H "Content-Type: application/json" -d '{}'
```
Workplans register with State Hub. After workplan changes:
### Log progress (required at session close)
```bash
cd ~/state-hub && make fix-consistency REPO=shard-wiki
curl -s -X POST http://127.0.0.1:8000/progress/ \
-H "Content-Type: application/json" \
-d '{
"summary": "what was done",
"event_type": "note",
"author": "codex",
"workstream_id": "<uuid>",
"task_id": "<uuid>"
}'
```
Finished or canceled workplans move to `history/` with a `yymmdd-` archive prefix.
Omit `workstream_id` / `task_id` when not applicable.
## Where To Put New Material
### Update task status
- Exploratory analysis → `research/yymmdd-<topic>/`
- Raw feature ask or external requirement → `demand/`
- Reviewed design ready to guide code → `spec/`
- Implementation tasks → `workplans/`
- User/dev/agent how-to → `docs/`
- Collaborative unstructured notes → `wiki/`
```bash
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
-H "Content-Type: application/json" \
-d '{"status": "progress"}'
# values: wait | todo | progress | done | cancel
```
### Flag a task for human review
```bash
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
-H "Content-Type: application/json" \
-d '{"needs_human": true, "intervention_note": "reason"}'
```
---
## Session Protocol
**Start:**
1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe)
2. Check inbox: `GET /messages/?to_agent=shard-wiki&unread_only=true`; mark read
3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks
4. Check human-needed tasks: `GET /tasks/?needs_human=true`
**During work:**
- Update task statuses in workplan files as tasks progress
- Record significant decisions via `POST /decisions/`
**Close:**
1. Update workplan file task statuses to reflect progress
2. Log: `POST /progress/` with a summary of what changed
3. Note for the custodian operator: after workplan file changes, run from
`~/state-hub`:
```bash
make fix-consistency REPO=shard-wiki
```
This syncs task status from files into the hub DB.
---
## Credential and access routing
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
for inference. Run this check **before** requesting secrets, API keys, SSH access,
login tokens, or database passwords — in any repo, not only `ops-warden`.
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
other credential need belongs to another subsystem. **Do not** message
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
### Lookup (do this first)
```bash
warden route find "<describe your need>" --json
warden route show <catalog-id> --json
```
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
| Agent runtime | How to orient |
| --- | --- |
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
### Quick routing table
| I need… | Owner | ops-warden executes? |
| --- | --- | --- |
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
| Authorization decision | flex-auth | No — route only |
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
### Anti-patterns (do not do these)
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
- Pasting secrets into Git, State Hub, workplans, logs, or chat
### Other capabilities (reuse-surface)
Non-credential capabilities are usually discovered through **reuse-surface** federation
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
every repo's agent instructions because it is high-frequency, high-risk, and easy to
get wrong.
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
<!-- REPO-AGENTS-EXTENSIONS -->
<!-- Append repo-specific agent instructions below this marker.
The state-hub template sync preserves content after this line. -->
---
## Workplan Convention (ADR-001)
Work items originate as files in this repo — not in the hub. The hub is a
read/cache/index layer that rebuilds from files.
**File location:** `workplans/SHARD-WP-NNNN-<slug>.md`
**Archived location:** finished workplans may move to
`workplans/archived/YYMMDD-SHARD-WP-NNNN-<slug>.md`. The `YYMMDD` prefix is
the completion/archive date; the frontmatter `id` does not change.
**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use
`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use
this only for low-risk work completed directly; create a normal workplan for
anything needing analysis, design, approval, dependencies, or multiple phases.
**Frontmatter:**
```yaml
---
id: SHARD-WP-NNNN
type: workplan
title: "..."
domain: consumer
repo: shard-wiki
status: proposed | ready | active | blocked | backlog | finished | archived
owner: codex
topic_slug: ...
created: "YYYY-MM-DD"
updated: "YYYY-MM-DD"
state_hub_workstream_id: "<uuid>" # written by fix-consistency — do not edit
---
```
Use `proposed` for a new draft, `ready` after review against current repo
state, and `finished` after implementation. `stalled` and `needs_review` are
derived health labels, not frontmatter statuses.
**Task block format** (one per `##` section):
```
## Task Title
` ` `task
id: SHARD-WP-NNNN-T01
status: wait | todo | progress | done | cancel
priority: high | medium | low
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
` ` `
Task description text.
```
Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work.
To create a new workplan:
1. Write the file following the format above
2. Notify the custodian operator to run `make fix-consistency REPO=shard-wiki`
(or send a message to the hub agent via `POST /messages/`)

View File

@@ -1,53 +1,12 @@
# CLAUDE.md
# shard-wiki — Claude Code Instructions
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Repository status
This is an **early-stage Python repository**. The package scaffold (`src/shard_wiki/`, `tests/`, `pyproject.toml`) exists with only smoke tests — the domain model is not yet implemented. Read `INTENT.md` (aspiration), `SCOPE.md` (current achievement), and `AGENTS.md` (layout and conventions) before designing anything. Close the gap from SCOPE to INTENT via `research/`, `spec/`, and `workplans/`.
## What this project is
`shard-wiki` is a **Git-based Markdown wiki orchestrator and federation layer**, not a wiki engine. It lets multiple heterogeneous wiki-shaped page stores (**shards**) attach to a shared root entity and be presented as a **union of pages**, while preserving each shard's separate storage, provenance, capabilities, and history.
The core job is orchestration across backends — Git repos, repo subdirectories (`wiki/`), Gitea wikis, local folders, Obsidian vaults, WebDAV/Nextcloud directories, Coulomb spaces — never replacing or homogenizing them.
## Core domain model (the concepts code must honor)
These abstractions come from `INTENT.md` and define the architecture. New code should map onto them rather than inventing parallel vocabulary:
- **Shard** — an independently meaningful page store attached to a root entity. Shards have *sovereignty*: their own backend, capabilities, limits, history, and identity model. Not all shards are Git-native.
- **Root entity / information space** — the joined space that shards attach to. Each information space should have a **Git-addressable coordination layer** (history, patches, review, backup, reconciliation) even when individual shards are not Git-native.
- **Shard adapter contract** — the versioned interface a backend implements to participate. Adapters are **capability-aware**: the core must model explicitly which operations a shard supports (read, write, diff, merge, lock, version, publish, accept patches) rather than assuming uniformity.
- **Wiki page model** — a stable, versioned, Markdown-first but backend-neutral representation of pages, paths, links, metadata, revisions.
- **Projection** — a lazy, cache-like local view of remote/external shard content. Prefer lazy projection over eager copying.
- **Overlay** — a non-destructive local edit against a remote, read-only, or capability-limited shard, representable as drafts/patches/commits/merge requests *before* destructive application ("overlay before mutation").
- **Coordination journal** — the Git-backed record of change flows for an information space.
- **Shard modes** — read-only, write-through, mirrored, projected, cached, canonical.
## Design constraints to enforce in code
These are hard boundaries from `INTENT.md`; treat violations as design bugs:
- **Mechanism over policy.** Provide primitives for federation, sync, overlays, patching, conflict detection, projection, reconciliation. Do *not* hard-code one editorial/sync/conflict/canonical-source policy — keep those configurable.
- **Union without erasure.** Always preserve provenance: which shard a page came from, its freshness, whether it is cached, whether it has overlays, whether it diverges from an equivalent page elsewhere. Never hide authorship, conflicts, freshness, or backend limitations.
- **No silent remote mutation.** Do not mutate remote systems without explicit adapter support and user intent.
- **Graceful degradation.** Limited backends must still be usable as read-only/cache/projection/backup/patch targets.
- **Not a file-sync daemon.** Synchronization is wiki-page-semantic, not generic file mirroring.
`INTENT.md` has a "Stability Note": changes that redefine what a shard is, Git's role, how root entities are modeled, or whether this is an orchestrator vs. an engine are **architectural changes** and should be rare and deliberate.
## Build, test, run
Python with a `src/` layout, built via hatchling, tested with pytest. Tests run against the source tree directly (`pythonpath = ["src"]` in `pyproject.toml`), so no install/editable step is required to run them.
```bash
pip install -e ".[dev]" # one-time: install dev tooling (pytest, pytest-cov, ruff)
pytest # run the full test suite
pytest tests/test_package.py::test_version_is_exposed # run a single test
pytest --cov # run with coverage
ruff check # lint
ruff format # format
```
Note: the system `pytest` is 7.4.x; `minversion` in `pyproject.toml` is pinned to `7.0` to match. Bump it if a newer pytest is installed into the dev environment.
@SCOPE.md
@.claude/rules/repo-identity.md
@.claude/rules/session-protocol.md
@.claude/rules/first-session.md
@.claude/rules/workplan-convention.md
@.claude/rules/stack-and-commands.md
@.claude/rules/architecture.md
@.claude/rules/repo-boundary.md
@.claude/rules/credential-routing.md
@.claude/rules/agents.md

View File

@@ -16,6 +16,8 @@ The goal is to allow independently stored and differently implemented wikis, pag
The repository provides a **shard orchestration layer** for interconnected Markdown and markup-based wiki content.
Equivalently, shard-wiki can be used as a **headless, API-first wiki engine** — optimized for **integrating heterogeneous data sources** and for **efficient access by agents and automation** — that ships its own native engine as one (canonical-mode) shard among many. There is no bundled UI: presentation and rendering are consumer concerns.
It allows wiki-like systems to:
* Attach heterogeneous page stores as shards of a shared information space
@@ -30,6 +32,7 @@ It allows wiki-like systems to:
* Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
* Allow existing wiki engines to become federation-capable through a shared API
* Allow non-federation-aware systems to participate through adapters and projections
* Serve as a **headless, API-first wiki engine** (a small typed-extension core) that integrates heterogeneous data sources and is consumed efficiently by agents and automation
It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a **composable federated wiki space**.
@@ -85,7 +88,7 @@ A mature `shard-wiki` should allow each participating shard to see the others as
This repository is **not** intended to:
* Replace all wiki engines with a single canonical wiki implementation
* Replace all wiki engines with a single canonical wiki implementation *(shard-wiki MAY still provide its own native, headless, API-first engine as one optional shard backend — see Design Principles — but never as a mandated or universal replacement)*
* Force every shard to use the same backend, database, directory layout, or storage format
* Require every participating system to become federation-aware
* Require every participating shard to be Git-native
@@ -148,6 +151,9 @@ Policy decisions such as conflict preference, canonical source selection, public
* **Composable integration**
Wiki engines should be able to use the `shard-wiki` API to become federation-enabled without reimplementing federation internally.
* **Native reference engine (additive, headless & API-first)**
shard-wiki MAY provide its own native wiki-engine as a **canonical-mode shard backend** — a **small core** with a **typed-extension framework**, activated **per shard** (only what you need). It is **headless and API-first** (no bundled UI; presentation/rendering are consumer concerns) and tuned for **integrating heterogeneous data sources** and **efficient agent/automation access**. It is *one shard type among many*, implemented against shard-wiki's own adapter contract; it does **not** replace other engines, mandate a single implementation, or change shard-wiki's role as an orchestrator. Shard sovereignty and union-without-erasure are preserved.
* **Open by default, progressively governed**
A standalone `shard-wiki` must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an *additive capability*, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).
@@ -201,3 +207,5 @@ Such changes should be rare, because they affect all downstream systems relying
In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether `shard-wiki` is an orchestrator rather than a wiki engine should be treated as architectural changes.
**Amendment — 2026-06-15 (SHARD-WP-0013 T4, decision `84ffdb48`):** admits an **additive** native reference wiki-engine — **headless, API-first**, a small typed-extension core — as a **canonical-mode shard backend** optimized for data-source integration and agent access. Deliberate, narrow scope change; shard-wiki remains an orchestrator and neither mandates nor replaces other engines. (Mirrors the earlier auth-in-core amendment precedent.)

View File

@@ -17,12 +17,12 @@ Learnings update both SCOPE and INTENT where necessary.
| Layer | State |
|-------|-------|
| Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. Incremental-first derived tier (SHARD-WP-0011): `incremental` (indexed equivalence via MinHash/LSH blocking + verify, change-driven delta maintenance with retraction/propagation, Merkle-style digest + self-healing I-2 consistency-checker, `UnionIndex` routed behind `InformationSpace.all_pages` with rebuild as explicit fallback). Second adapter (SHARD-WP-0012): `GitShardAdapter` — git-IS-store substrate (read=tracked *.md, write=commit, current_rev=per-path sha for drift, adopted git-native history), passes conformance, works across folder+git shards in union/overlay/edit with no core change (capability-as-data proven on a second substrate). 196 tests green, ~97% coverage |
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
| Demand | NetKingdom integration asks captured, not yet negotiated |
| Spec | Architecture blueprint drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds |
| Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1T10 federation + T11T16 adapter contract); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` active (8 computational-knowledge dives: T1 literate programming, T2 Mathematica, T3 Jupyter, T4 Processing, T5 Strudel, T6 Squeak + T8 Pharo (merged), T7 Glamorous Toolkit done — all 8 dives complete) |
| Spec | CoreArchitectureBlueprint (whole-system, hardened via SHARD-WP-0005/0006) + FederationArchitecture + FederationRequirements + TSD §A adapter contract + ArchitectureBlueprint (auth/history) + WikiEngineCoreArchitecture (headless API-first engine, SHARD-WP-0013) drafted; UseCaseCatalog 84 UCs (+ engine capability-structure layer); PRD scaffold |
| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` **done** (18 tasks`FederationArchitecture.md` [T1T10, T17] + `TechnicalSpecificationDocument.md` §A adapter contract [T11T16, T18]); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) |
## In Scope (today)
@@ -32,11 +32,15 @@ Learnings update both SCOPE and INTENT where necessary.
- Authorization model design (delegated authentication, core authorization).
- Shard adapter contract and wiki page model (to be specified, then implemented).
- Git-backed coordination journal for information spaces.
- A **native, headless, API-first wiki-engine core** (small typed-extension core, as a
canonical-mode shard backend) — design via SHARD-WP-0013; optimized for data-source
integration and agent access.
- State Hub workplan registration and consistency sync.
## Out Of Scope (today)
- A standalone wiki engine UI or rendering pipeline.
- A wiki-engine **UI or rendering pipeline** (the engine is headless/API-first; presentation
is a consumer concern). A bundled standalone UI is not provided.
- Authentication, credential storage, or user directory implementation.
- Hard-coded editorial, sync, or conflict-resolution policy.
- Generic file mirroring independent of wiki-page semantics.

View File

@@ -0,0 +1,91 @@
# Critical review (round 2) — CoreArchitectureBlueprint.md (hardened)
Date: 2026-06-15 · Reviewer: tegwick (with Claude) · Subject:
`spec/CoreArchitectureBlueprint.md` after **SHARD-WP-0005** (commit f21b7b5) · Feeds:
**SHARD-WP-0006**
A second hostile pass over the *hardened* blueprint. Round 1 found design bugs; this round
finds (a) self-consistency regressions the surgical hardening introduced, and (b) deeper
second-order gaps — some of which the hardening *sharpened*. Verdict: the architecture is now
substantially sound; what remains in §B/§C are hard distributed-systems/operational questions,
not design smells — except §A (a real regression) and three foundational gaps in §B.
---
## A. Self-consistency regressions introduced by surgical hardening
The 9 edits deepened §6§9 but did not propagate to the **overview surfaces**, so the document
now contradicts itself between its summary and its body (and readers trust the summary).
- **A-1 (real contradiction).** §4 still says "Addressing, **equivalence**, and transclusion
key on identity" — the exact conflation T2 fixed in §7.2 (equivalence keys on *content
fingerprint across distinct identities*). → **WP-0006 T1**
- **A-2.** §4 "Projection — typed on two axes" and §4 "Provenance envelope … every artifact
carries [full wrapper]" are stale vs T7's §8.4 (two-axis = extension point; trivial default)
and §7.3 (layered effective-vs-own). → **T1**
- **A-3.** §10 policy surface omits knobs the hardening added (freshness/staleness §8.8,
squash-compaction §8.1, conflict-resolution preset §8.6, tenant-partition) — yet §11 defines
`policy/` as "owns the §10 surface." The module contract points at a stale list. → **T1**
- **A-4 (cosmetic).** §3 diagram + §11 header still say "L4 rebuildable cache" / "15 spectra,"
advertising the pre-hardening model (§8.7 incremental-first; §6.5 orthogonal-core). → **T1**
Meta-point: surgical editing hardened the body but regressed whole-document coherence; v2
needs an **overview-reconciliation pass**.
## B. Foundational gaps (serious; some sharpened by the hardening)
- **B-1 — The journal is now a concurrent-write DB, but it's single-writer Git.** §8.6's
consistency model assumes "the journal is local Git, read-your-writes." L4 multi-tenant + the
L6 Orchestrator API imply a server; HA/scale implies *multiple* instances. Concurrent commits
of coordination-canonical state to one git journal = lock contention / merge races; Git is not
a concurrent-write store. Either single-writer-per-space (an unstated HA ceiling) or a real
concurrent coordination store with Git as an *export*. **T1 of WP-0005 worsened this** by
loading more canonical state into the journal. The keystone unanswered question. → **T2**
- **B-2 — Capability-as-data trusts self-reported profiles with no conformance check.** I-3 +
§6.5's degradation contract assume the profile tells the truth. A buggy adapter (claims
`merge=git/text`, corrupts; claims `notify`, never emits) silently poisons every degradation
decision. No **adapter conformance suite** (declared profile == observed behavior) exists.
Foundational for an architecture whose correctness rests on profile accuracy. → **T3**
- **B-3 — "Coordination-canonical state in the journal" has no representation design.** T1
relocated overlays/bindings/aliases/equivalence-sets/merges into "the journal" without saying
*how* Git stores structured mutable state. "All equivalences touching X" over a git-of-files
is O(scan) unless indexed — and an index is L4/derived. The new central concept is a black
box; resolve *with* B-1. → **T2**
- **B-4 — Incremental equivalence is under-specified/likely incorrect; I-2 only eventually
true.** §8.7 re-verifies a changed page's *new* candidate set but not the pairs it *leaves*
(a page exiting an LSH bucket can break an existing equivalence edge); the delta is not
additive. Deeper: incremental maintenance drifts from `f(canonical)`, so I-2 holds only
eventually, guaranteed solely by an expensive reconcile-against-rebuild. Needs a stated
verification mechanism (background checker / digest-vs-sampled-rebuild). → **T4**
## C. Real but second-tier (track as open problems O-8…O-11)
- **C-1 — Mechanism-over-policy → operator burden; no preset bundles.** ~7 knob families with
sub-modes and interactions; only authz (L0L4) bundles into personas. Need named bundles
("personal vault" / "team wiki" / "enterprise federation"). → **O-8 / T5**
- **C-2 — Tenant partitioning (I-13) vs shard sharing + lazy projection.** A shard in two roots
is cached twice → duplicate storage + double refresh on rate-limited backends. Shard
exclusive-to-one-root or shareable? Unresolved. → **O-9 / T5**
- **C-3 — Span-level authz + transclusion is an unmodeled leak path.** Authz is per
page/shard/tenant; transclusion crosses shards at span granularity → a page can leak a span
past its ACL (aggregation/inference). §7.3's ⊕ also stops being simple two-level inheritance
across a transclusion boundary. → **O-10 / T5**
- **C-4 — Union-under-unavailability undefined.** Freshness covers *stale*, nothing covers
*down*. The dead-shard read path (partial union? error? last-known?) is unspecified though
it's the commonest real failure. → **O-11 / T5**
## D. Recommended resolution (→ SHARD-WP-0006)
1. **§A reconciliation** (T1) — make the overview match the hardened body.
2. **Journal & coordination-state model** (T2) — settle single-vs-multi-writer and separate the
**coordination-state store** from the **content-history journal**. Likely resolution:
**event-sourced coordination** — an append-only *decision log* is the coordination-canonical
tier (git-addressable, I-6 preserved); the queryable current state (alias table, equivalence
set) is a *derived fold* of the log (disposable). Append-logs tolerate concurrency far better
than mutable-file Git; state a concurrency model. Resolves **B-1 + B-3** together.
3. **Adapter conformance suite** (T3) — make a passing conformance run part of the contract
(B-2): every adapter proves declared profile == observed behavior.
4. **Incremental correctness + I-2 verification** (T4) — fix the leaving-bucket re-verification
and propagation; add a background consistency-checker / derived-tier digest so I-2 is
verifiable, not merely asserted (B-4).
5. **Track §C** (T5) — O-8…O-11 with chosen direction + revisit trigger; close-out.

View File

@@ -0,0 +1,122 @@
# Critical review — CoreArchitectureBlueprint.md
Date: 2026-06-15 · Reviewer: tegwick (with Claude) · Subject:
`spec/CoreArchitectureBlueprint.md` @ commit **9b5b393** · Feeds: **SHARD-WP-0005**
A deliberately hostile review of the first whole-system architecture, to find where it
**breaks (correctness)**, **fails to scale**, and **could be more elegant/efficient** before
any implementation. Findings are prioritised; each is the input to a SHARD-WP-0005 task.
## Verdict in one line
The **layering and the dual narrow waist are sound and stay**. The **thesis is ~90% right**;
the missing 10% (curatorial / coordination-canonical state) breaks its clean story. There are
**two genuine bugs**, **two large unaddressed scaling risks**, and several **elegance/efficiency
debts** — all fixable without touching INTENT.
---
## A. The framing crack (fix resolves three issues)
**A-1 — Two buckets hide a third.** The thesis "canonical at the edges, derived in the middle"
omits **born-in-the-middle-but-canonical** state: overlays that are the local truth against a
read-only shard (Flow C), manual **curator equivalence bindings**, alias tables, merge
decisions. These encode human judgment or local-only content and **cannot be rebuilt** from
shards+journal.
**Contradiction:** I-2 declares L4 rebuildable, yet §8.4 puts "alias table, curator binding"
in L4. You cannot rebuild a curator's manual binding.
**Fix:** three states — **sharded-canonical**, **coordination-canonical** (journal: overlays,
bindings, aliases, merges — durable, born in the middle), **derived-disposable** (union graph,
indexes, projections). Re-frame §1 as **canonical (sharded + coordination) vs derived
(disposable)**; `derived = f(canonical)` then becomes actually true. → **T1**
---
## B. Where it breaks (correctness)
**B-1 — Identity conflated with content-fingerprint (BUG).** §7.2 derives page identity from
content fingerprint. That makes **editing a page change its identity**, breaking every
reference. Fingerprints identify *versions/equivalence*, not *identity*. Page identity must be
a **stable handle (uid)** surviving edits; fingerprints belong to the **equivalence** mechanism
(§8.4). One word, two concepts, wrong implementation for the stable one. → **T2**
**B-2 — No concurrency/consistency model.** Concurrent overlays on one page, overlay applied
after source drift, journal-commit vs shard-native-write ordering — all undefined. Conflict
handling is deferred to "policy presets," but **conflict *detection + representation* is core
mechanism**; only *resolution* is policy. The union's consistency guarantee is unstated
(eventually-consistent? read-your-writes? causal-via-journal?). → **T3**
**B-3 — Persisted union cache + multi-tenant = leak surface.** §13 recommends a persisted L4
cache; §9 protects content by *read-time* filtering on the provenance envelope. A persisted
cross-tenant union cache guarded only by read-time filtering is an L4 attack surface. Tension
between I-2 (persisted rebuildable cache), scale, and L5 isolation is unacknowledged. → **T8**
---
## C. Where it fails to scale
**C-1 — Equivalence detection is O(N²), no indexing/incremental story.** Fingerprint /
span-set-overlap across all pages of all shards is combinatorial (10 shards × 100k pages ≈
10¹² comparisons). No blocking/LSH/indexing, no incremental maintenance. Biggest scaling
hazard in the document. → **T4**
**C-2 — "Rebuildable cache" collides with the operational-envelope axis.** A byte-exact
rebuild requires reading *every page of every shard*, including rate-limited/paginated
external APIs (Notion) and irreducibly-live sources — hours-to-days. I-2 contradicts axis-10.
**Incremental, change-driven maintenance must be primary** (notify→delta), rebuild a rare
fallback. Cache invalidation — the actual hard problem — is named once and never designed. →
**T4, T5**
**C-3 — Unbounded history at open L0 = DoS/perf.** "Every write a commit" + "open for all" ⇒
the git journal grows without bound under bots/vandalism and git degrades on huge histories.
"History is the floor" has an unacknowledged cost: packing, compaction, per-shard offload. →
**T8**
---
## D. Elegance / efficiency debts
**D-1 — The 15 spectra assert a clean degradation function never demonstrated.** Either most
axes are irrelevant to most ops (then the 15-D profile is ceremony), or behavior depends on
several axes *jointly* (then "no per-backend code" becomes a sprawling axis-interaction matrix
— the flat-checklist problem in higher dimensions). And the axes **aren't orthogonal**
(git-native history ⟺ git-IS-store ⟺ git/text merge; encrypted opacity ⟹ query/translation
collapse). Model a **smaller orthogonal core** + **derived/implied** positions, and state the
**axis-interaction subset** the degradation logic truly uses. → **T6**
**D-2 — Provenance envelope isn't inherited; it'll dwarf the content.** Per-span envelopes at
block granularity = 10k near-identical envelopes for a 10k-block graph. The doc already
invented the right pattern for Trilium ("effective-vs-own with per-attribute provenance") and
failed to apply it to its own envelope. Make provenance **layered (page envelope + span
deltas)**. → **T7**
**D-3 — Projection machinery over-fit to the exotic tail.** Two-axis model + three facets +
view registry exist mostly for UC-83/84 (2 of 84 UCs); the 95% case (markdown in git) pays the
weight. Make the **common case trivial** (default = plain lazy replication) and
derivation/liveness an **extension point**, not a taxonomy every projection instantiates. →
**T7**
**D-4 — Cross-cutting rails are the highest-coupling components, presented as clean.**
`provenance/` and capability types are imported by every layer (god-modules); an envelope
change ripples everywhere. And **policy has no module** (§10 enumerates it; §11 omits it)
despite being consulted by L3/L4/L5. Give policy a home; pin the rails behind stable narrow
interfaces. → **T7**
---
## E. What explicitly stays
- The 6-layer model + the dual narrow waist (adapter contract / page model).
- Capability-as-data (I-3), union-without-erasure (I-4), overlay-before-mutation (I-5),
Git-addressable coordination (I-6), mechanism-over-policy (I-7), graceful degradation (I-8).
- The federation-model taxonomy and the auth ladder (ArchitectureBlueprint.md).
## F. Disposition
Some findings are **solvable now** (A-1, B-1, D-2, D-3, D-4, C-3); some are **partially open**
and should be tracked honestly rather than pretend-solved (B-2 consistency model: pick a
guarantee; C-1 equivalence-at-scale: pick a blocking strategy; D-1 axis interactions: enumerate
the real subset). SHARD-WP-0005 closes the solvable ones and records the open ones in a new
"Known scaling risks & open problems" section of the blueprint. → **T9**

View File

@@ -0,0 +1,45 @@
# reuse-surface contributions — shard-wiki (SHARD-WP-0013 T3)
Date: 2026-06-15 · From: shard-wiki (whynot) · To: reuse-surface (helix_forge) · Tracked list
of (A) capabilities shard-wiki registered, (B) **gaps** shard-wiki proposes the reuse surface
add, and (C) capabilities shard-wiki will **consume** rather than rebuild. Communicated to the
reuse-surface agent via state-hub `send_message`.
## A. Registered (T1 + T3) — 8 entries
`capability.wiki.{shard-orchestration, adapter-contract, page-model, coordination-journal,
overlay, federation-models, engine-typed-extensions, derived-views}` — committed in
reuse-surface, `validate` ok (20 entries total).
## B. Proposed gaps (cross-cutting; not shard-wiki-internal) → reuse-surface should own/define
- **G1 — `capability.platform.typed-extension-framework`** *(suggested)*
A reusable pattern: a **small core + a stringent typed-extension framework** where extensions
declare typed contracts, compose, and are **activated per context**. shard-wiki's wiki engine
(`capability.wiki.engine-typed-extensions`) is one instance, but the *pattern* is
cross-domain (any HelixForge capability platform). Evidence: shard-wiki UseCaseCatalog
"Capability structure" layer (core + 10 typed extensions + conflict-mediation map).
Suggested owner: helix_forge / reuse-surface. Relation: would `generalize`
`capability.wiki.engine-typed-extensions`.
- **G2 — `capability.content.translation-fidelity`** *(suggested)*
Lossless/lossy **content translation with an explicit fidelity report** (what round-trips
cleanly vs degrades, non-mappable elements preserved as provenance). Reusable well beyond
wiki (any format-bridging consumer). Evidence: shard-wiki TSD §A.6, UC-42/UC-59.
## C. Consumptions (reuse, do not rebuild)
- **`capability.feature-control.evaluate`** (helix_forge/feature-control) → shard-wiki's
**per-shard extension/feature activation** (the "activate only what you need" mechanism).
Already recorded as a relation on `capability.wiki.engine-typed-extensions`.
- **`capability.authorization.policy-evaluate`** (flex-auth) → shard-wiki's **X-AUTHZ** policy
decisions. shard-wiki owns the authz *model* (authz-in-core) but can reuse this evaluation
engine rather than building one.
- **`capability.statehub.{progress-log, workstream-coordinate}`** → already in use for
coordination across this work.
## Status
Gaps G1/G2 are **suggestions** to the reuse-surface owner (not unilateral registrations, since
they are cross-cutting, not shard-wiki-internal). Consumptions are recorded for the engine
architecture (T5) so it reuses rather than reinvents.

View File

@@ -1,8 +1,19 @@
# history/
Archived material that is no longer needed for daily work but should be kept.
Archived material and the project's **meta-history**: finished/canceled workplans kept for
the record, plus durable **reviews, critical assessments, and decision records** — the
reasoning behind the specs, captured at a point in time.
Use a `yymmdd-` prefix when archiving files or directories. Content here is
**out of scope** for regular tasks — consult only for research or diagnostics.
Use a `yymmdd-` prefix. Archived material is **out of scope** for regular tasks (consult only
for research or diagnostics); assessment/review records are point-in-time and may seed active
workplans, but are not edited after the fact — supersede with a new dated record and link back.
Finished or canceled workplans from `workplans/` are archived here.
Distinct from the **coordination journal** (a runtime Git-backed record of *content* change
flows inside an information space, an INTENT domain concept); `history/` is the *project's own*
design evolution.
| Date | Record | Subject |
|------|--------|---------|
| 2026-06-15 | `260615-core-architecture-blueprint-review.md` | Critical review of `spec/CoreArchitectureBlueprint.md` (commit 9b5b393); inputs to `SHARD-WP-0005` |
| 2026-06-15 | `260615-core-architecture-blueprint-review-2.md` | Round-2 review of the hardened blueprint (post-`SHARD-WP-0005`, f21b7b5); inputs to `SHARD-WP-0006` |
| 2026-06-15 | `260615-reuse-surface-contributions.md` | shard-wiki's reuse-surface registrations, proposed gaps (G1/G2), and consumptions (`SHARD-WP-0013` T3) |

View File

@@ -36,6 +36,11 @@ pythonpath = ["src"]
branch = true
source = ["shard_wiki"]
[tool.coverage.report]
show_missing = true
# Quality floor for `pytest --cov` / `coverage report` (not forced on a bare `pytest` run).
fail_under = 90
[tool.ruff]
src = ["src", "tests"]
target-version = "py311"

12
registry/README.md Normal file
View File

@@ -0,0 +1,12 @@
# Capability Registry
Markdown-first capability index for federation and reuse planning.
## Authoring
1. Copy a capability entry template (see reuse-surface `templates/capability-entry.template.md`).
2. Add the row to `indexes/capabilities.yaml`.
3. Run `reuse-surface validate` from a checkout with the CLI installed.
4. Merge to `main` and verify publish with `reuse-surface establish --publish-check`.
Federation contract: reuse-surface `docs/RegistryFederation.md`.

View File

@@ -0,0 +1,103 @@
---
id: capability.wiki.adapter-contract
name: Capability-Aware Shard Adapter Contract
summary: A versioned backend interface where each binding declares a verified capability profile (positions on capability spectra), so federation ops degrade by capability.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, adapter, capability, contract, conformance, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Fifteen capability spectra with an orthogonal core + implication rules, plus
a normative contract spec (TSD Section A); derived from a ~23-system synthesis.
availability:
current: A2
target: A5
confidence: medium
rationale: >
AdapterContract + a read/write FolderAdapter + a conformance suite that
verifies declared profile == observed behaviour exist as a source module.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- versioned interface with declared, conformance-verified capability profiles
- one concrete adapter (file-store) passes the conformance suite
broken_expectations:
- only one substrate implemented (git-IS-store, REST, CRDT adapters planned)
out_of_scope_expectations:
- hosting backends
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- single adapter implemented so far
discovery:
intent: >
Mediate heterogeneity at one narrow waist: a backend participates by implementing a
versioned interface and declaring a verified position on each capability spectrum.
includes:
- capability profile as data (orthogonal-core spectra + implied positions)
- operation verbs (read/write/diff/merge/notify/.../derive-projection/execute)
- a conformance suite (profiles verified, not self-asserted)
excludes:
- assuming uniform backend capabilities
use_cases:
- "shard-wiki UseCaseCatalog UC-34..UC-43, UC-50, UC-57, UC-60..UC-69 (shard attachment & adapter binding)"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/adapters/"
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.page-model
supports:
- capability.wiki.shard-orchestration
evidence:
documentation:
- "shard-wiki/spec/TechnicalSpecificationDocument.md (Section A)"
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 6)"
tests:
- "shard-wiki/tests/test_folder_adapter.py"
- "shard-wiki/tests/test_conformance.py"
consumer_guidance:
recommended_for:
- exposing any page store as a capability-described, conformance-checked shard
not_recommended_for:
- backends that cannot honestly describe their capabilities
known_limitations:
- reference implementation covers the file-store substrate only so far
---
# Capability-Aware Shard Adapter Contract
The bottom narrow waist of shard-wiki: a versioned interface plus a **verified** capability
profile per binding. Core logic is written once against capabilities (not per-backend), and
the conformance suite rejects profiles whose declared abilities don't match observed behaviour.
## Assessment notes
### Discovery
Fifteen spectra reduced to an orthogonal core with implication rules (CoreArchitectureBlueprint
Section 6.5); normative in TSD Section A.
### Availability
`adapters/` ships the contract, a folder adapter, and `assert_conformant`.

View File

@@ -0,0 +1,103 @@
---
id: capability.wiki.coordination-journal
name: Event-Sourced Coordination Journal
summary: An append-only, totally-ordered-per-space decision log (overlays, bindings, aliases, merges, forks) whose current state is a derived fold; git-addressable history.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, event-sourcing, coordination, git, journal, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Keystone resolved across two architecture reviews: coordination-canonical state
as an append-only decision log with a per-space append authority; current state
is a derived fold (derived = f(log)).
availability:
current: A2
target: A4
confidence: medium
rationale: >
In-memory DecisionLog + fold work as a source module; the git-backed store with a
per-space lease (the production backing) is planned.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- append-only, totally-ordered-per-space log with read-your-writes
- derived fold to aliases + transitively-merged equivalence groups
broken_expectations:
- git-backed storage and per-space lease/append-authority not yet implemented
out_of_scope_expectations:
- general-purpose event bus
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- in-memory backing only; cross-process durability pending
discovery:
intent: >
Make coordination-canonical decisions durable and git-addressable as events, with the
queryable current state always recomputable by replay.
includes:
- append-only decision log, totally ordered per information space
- derived fold to current coordination state (aliases, equivalence groups, overlays)
- per-space append authority (concurrency model)
excludes:
- storing derived/disposable union state
use_cases:
- "shard-wiki UseCaseCatalog UC-29, UC-33 (history, attribution, coordination journal)"
availability:
current_level: A2
target_level: A4
current_artifacts:
- "shard-wiki/src/shard_wiki/coordination/decision_log.py"
target_artifacts:
- git-backed log store with per-space lease
consumption_modes:
- source module
relations:
supports:
- capability.wiki.shard-orchestration
- capability.wiki.overlay
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.1)"
tests:
- "shard-wiki/tests/test_decision_log.py"
consumer_guidance:
recommended_for:
- durable, replayable, git-addressable coordination state for a federated space
not_recommended_for:
- high-frequency general event streaming
known_limitations:
- production git backing + lease are still on the roadmap (SHARD-WP-0009)
---
# Event-Sourced Coordination Journal
The keystone: coordination-canonical state (overlays, equivalence bindings, aliases, merges,
forks) is an append-only **decision log**, totally ordered per information space; the queryable
current state is a derived **fold** of the log (`derived = f(log)`). The log is git-addressable,
giving history/patch/review/backup for coordination decisions for free.
## Assessment notes
### Discovery
Resolved across the round-1/round-2 architecture reviews (CoreArchitectureBlueprint Section 8.1).
### Availability
`decision_log.py` ships an in-memory, totally-ordered log + fold; git+lease backing is planned.

View File

@@ -0,0 +1,87 @@
---
id: capability.wiki.derived-views
name: Wiki Derived Views
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages, SiteMap, and (delegate-or-derive) Search — carrying provenance.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, derived-views, backlinks, recentchanges, search, shard-wiki]
maturity:
discovery:
current: D3
target: D5
confidence: medium
rationale: >
Core-vs-adapter classification and behaviours are decided (FederationRequirements ADR-03);
implementation is planned (SHARD-WP-0010), not built.
availability:
current: A0
target: A4
confidence: low
rationale: >
Designed; no implementation yet. Informational/planning reuse only today.
external_evidence:
completeness:
level: C0
name: Absent
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations: []
broken_expectations:
- no derived view is implemented yet
out_of_scope_expectations:
- presentation / rendering of views
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- planning-stage
discovery:
intent: >
Provide recomputable, provenance-carrying views over the union (link graph, change feed,
enumeration, search) without introducing canonical state.
includes:
- BackLinks (link graph), RecentChanges (journal + shard signals), AllPages, SiteMap
- Search as delegate-to-native-or-derive-index
excludes:
- view presentation / UI
use_cases:
- "shard-wiki UseCaseCatalog UC-17..UC-21, UC-63"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/workplans/SHARD-WP-0010-derived-views.md"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.shard-orchestration
- capability.wiki.page-model
related_to:
- capability.wiki.engine-typed-extensions
evidence:
documentation:
- "shard-wiki/spec/FederationRequirements.md (ADR-03)"
consumer_guidance:
recommended_for:
- planning derived navigation/discovery over a federated wiki union
not_recommended_for:
- implementation reuse today (planning-stage)
known_limitations:
- not implemented; Search ranking policy undecided
---
# Wiki Derived Views
Recomputable views over the union (BackLinks, RecentChanges, AllPages, SiteMap, Search). All
are derived/disposable (no canonical state) and carry provenance; Search is delegate-to-native
where a shard's query capability allows, else a derived index. Planned in SHARD-WP-0010.

View File

@@ -0,0 +1,115 @@
---
id: capability.wiki.engine-typed-extensions
name: Wiki Engine with Typed Extensions
summary: A small-core wiki engine realizing a stringent typed-extension framework that addresses all wiki use cases and lets each shard activate only the features it needs.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, engine, typed-extensions, feature-activation, shard-wiki]
maturity:
discovery:
current: D3
target: D5
confidence: medium
rationale: >
Architecture authored (shard-wiki/spec/WikiEngineCoreArchitecture.md): small page-store
kernel + typed-extension framework, per-shard activation, engine-as-canonical-mode-shard,
and a conflict-mediation realization are explored. Detailed extension SDK/ABI and the API
protocol remain (so D3 Explored, not yet D4/D5).
availability:
current: A0
target: A4
confidence: low
rationale: >
Planned. No engine kernel or extensions exist yet; informational/planning reuse only.
external_evidence:
completeness:
level: C0
name: Absent
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations: []
broken_expectations:
- engine core and typed-extension mechanism not yet designed in detail
out_of_scope_expectations:
- replacing other wiki engines or mandating one implementation
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- planning-stage capability
discovery:
intent: >
Provide shard-wiki's reference first-party shard backend: a small core + a stringent
typed-extension framework covering all collected use cases, mediating conflicting
requirements into an integrated whole, with per-shard activation (only what you need).
includes:
- a minimal engine kernel (page lifecycle, storage via the adapter contract, the typing mechanism)
- typed extensions that declare contracts and compose
- per-shard feature activation
excludes:
- replacing or mandating other wiki engines (it is one shard type among many)
- a single canonical implementation for all wikis
use_cases:
- "shard-wiki UseCaseCatalog UC-08..UC-25 and the full catalog (the engine must cover all)"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
- "shard-wiki/spec/WikiEngineCoreArchitecture.md"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.adapter-contract
- capability.wiki.page-model
related_to:
- capability.feature-control.evaluate
- capability.authorization.policy-evaluate
evidence:
documentation:
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
consumer_guidance:
recommended_for:
- planning a composable, feature-activatable native wiki engine
not_recommended_for:
- implementation reuse today (planning-stage)
known_limitations:
- architecture authored; extension SDK/ABI + API protocol still to design; not yet built
promotion_history:
- date: "2026-06-15"
dimension: discovery
from: D2
to: D3
rationale: WikiEngineCoreArchitecture.md authored (kernel + typed-extension framework explored); INTENT amendment ratified.
author: shard-wiki
---
# Wiki Engine with Typed Extensions
shard-wiki's planned reference first-party shard backend — a *canonical-mode shard* it
implements natively: a small core plus a stringent typed-extension framework addressing all
collected use cases, mediating conflicting requirements into a consistent whole, with per-shard
activation (activate only what you need). It is one shard type among many — not a replacement
for other engines. Per-shard activation is a candidate consumer of
`capability.feature-control.evaluate`.
## Assessment notes
### Discovery
Architecture authored: `shard-wiki/spec/WikiEngineCoreArchitecture.md` (small kernel +
typed-extension framework; engine = canonical-mode shard). INTENT amendment ratified
(2026-06-15, decision 84ffdb48). Extension SDK/ABI + API protocol are the next deliverables.
### Availability
Planning-stage; informational reuse only.

View File

@@ -0,0 +1,97 @@
---
id: capability.wiki.federation-models
name: Selectable Federation-Model Taxonomy
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication+ping, query-time graph-join, feed, activity-streams, engine-mirror) selected per space.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, federation, taxonomy, composable, shard-wiki]
maturity:
discovery:
current: D4
target: D6
confidence: high
rationale: >
A six-model taxonomy distilled from a ~23-system synthesis, each model anchored in a
real system, with capability prerequisites and per-space/per-shard composition rules.
availability:
current: A0
target: A4
confidence: low
rationale: >
Designed and specified (FederationArchitecture T17) but not implemented; informational
reuse only today.
external_evidence:
completeness:
level: C1
name: Sparse
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- the model taxonomy and selection/composition rules are documented
broken_expectations:
- no federation transport is implemented yet
out_of_scope_expectations:
- mandating a single federation mechanism
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- design-stage; no runtime evidence
discovery:
intent: >
Treat federation as selectable and composable rather than one mechanism, so each space
picks fork+journal, VCS-replication, query-join, feed, activity-streams, or engine-mirror.
includes:
- the six federation models + their capability floors
- per-space selection and per-shard composition
excludes:
- imposing one homogeneous federation network
use_cases:
- "shard-wiki UseCaseCatalog UC-26, UC-31, UC-33, UC-71, UC-72, UC-74, UC-79"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/spec/FederationArchitecture.md (T17)"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.shard-orchestration
- capability.wiki.coordination-journal
evidence:
documentation:
- "shard-wiki/spec/FederationArchitecture.md"
- "shard-wiki/research/260614-shard-spectrum-synthesis/findings.md"
consumer_guidance:
recommended_for:
- planning a federation strategy that mixes models per source
not_recommended_for:
- implementation reuse today (design-stage)
known_limitations:
- no transport implemented; informational planning reuse only
---
# Selectable Federation-Model Taxonomy
Federation is plural and composable: fork+journal (Federated Wiki), VCS-replication+ping
(ikiwiki), query-time graph-join (Wikibase SERVICE), feed aggregation, activity streams
(ActivityPub), and engine-mirror (Wiki.js). A space selects a model and composes per shard;
the default is fork+journal over git. Design-stage capability — strong for planning reuse.
## Assessment notes
### Discovery
FederationArchitecture T17, distilled from the shard-spectrum synthesis (v3).
### Availability
Specified, not implemented — informational reuse only.

View File

@@ -0,0 +1,102 @@
---
id: capability.wiki.overlay
name: Overlay-Before-Mutation Write Path
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only, rate-limited, or lossy backends be edited safely without silent remote mutation.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, overlay, patch, write-path, conflict, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Overlay lifecycle and apply-under-drift semantics are specified (ADR-05, blueprint
Section 8.6) and implemented as a single principled write path.
availability:
current: A2
target: A4
confidence: medium
rationale: >
OverlayEngine (draft/patch/apply), writable adapter, and InformationSpace.edit
exist as a source module; three-way merge is not (refuse-on-drift only).
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- draft -> patch -> apply with fast-forward / refuse-on-drift / keep-draft outcomes
- no silent remote mutation; overlay_state surfaced in provenance
broken_expectations:
- three-way / auto merge not implemented (refuse-on-conflict only)
out_of_scope_expectations:
- federation propagation of applied overlays
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- early implementation; conflict handling is detect-and-refuse only
discovery:
intent: >
Make any sub-write-through backend editable safely: an edit is an overlay first, applied
only on explicit intent and only when the source has not drifted.
includes:
- overlay drafts recorded as coordination-canonical events
- patch rendering (unified diff)
- apply-under-drift (fast-forward / refuse / keep-draft)
excludes:
- destructive write without drift check
use_cases:
- "shard-wiki UseCaseCatalog UC-04, UC-26, UC-29 (remix primitives, overlay)"
availability:
current_level: A2
target_level: A4
current_artifacts:
- "shard-wiki/src/shard_wiki/coordination/overlay.py"
- "shard-wiki/src/shard_wiki/coordination/patch.py"
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.coordination-journal
- capability.wiki.adapter-contract
evidence:
documentation:
- "shard-wiki/spec/FederationRequirements.md (ADR-05)"
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.2, 8.6)"
tests:
- "shard-wiki/tests/test_apply.py"
- "shard-wiki/tests/test_write_path_integration.py"
consumer_guidance:
recommended_for:
- safe editing over read-only / rate-limited / lossy backends
not_recommended_for:
- workflows needing automatic conflict resolution today
known_limitations:
- merge is detect-and-refuse; three-way merge is future work
---
# Overlay-Before-Mutation Write Path
One principled write path: every edit drafts an overlay (a coordination-canonical event),
renders as a patch, and applies under drift checks — fast-forwarding a writable target,
keeping a local draft on a read-only target, and refusing (never clobbering) on external drift.
## Assessment notes
### Discovery
Specified in FederationRequirements ADR-05 and CoreArchitectureBlueprint Section 8.2/8.6.
### Availability
`overlay.py` + `patch.py` + `InformationSpace.edit` ship the path; built in SHARD-WP-0008.

View File

@@ -0,0 +1,104 @@
---
id: capability.wiki.page-model
name: Backend-Neutral Wiki Page Model
summary: A Markdown-first but stretchable page model with stable identity separate from placement and layered provenance, spanning prose to typed-graph and computational shapes.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, page-model, identity, provenance, markdown, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Page shapes (prose, typed records, typed-graph, inline-embedded, non-Markdown,
and four computational shapes) plus identity != placement and layered provenance
are specified and grounded in the dive research.
availability:
current: A2
target: A5
confidence: medium
rationale: >
Identity/Placement/Span/Page and layered ProvenanceEnvelope exist as a source
module; richer shapes (typed-graph, notebook) are modeled but not all built.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- stable identity distinct from placement and from content fingerprint
- layered (effective-vs-own) provenance with near-zero per-span cost
broken_expectations:
- non-prose shapes (typed-graph, notebook, inline-embedded) not fully realized
out_of_scope_expectations:
- rendering / presentation
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- prose shape is the only exercised path so far
discovery:
intent: >
One backend-neutral lingua franca every consumer sees; every shape reduces to
(content|source, structure, provenance envelope, optional derivation rule).
includes:
- page identity (stable handle) vs placement (N paths/shards) vs equivalence (fingerprint)
- layered provenance envelope (page + span deltas)
- page-shape taxonomy incl. computational shapes
excludes:
- deriving identity from content (a fingerprint identifies a version, not a page)
use_cases:
- "shard-wiki UseCaseCatalog UC-34, UC-39, UC-44..UC-49, UC-55, UC-73, UC-83, UC-84"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/model/"
- "shard-wiki/src/shard_wiki/provenance/"
consumption_modes:
- source module
relations:
supports:
- capability.wiki.adapter-contract
- capability.wiki.shard-orchestration
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 7)"
- "shard-wiki/spec/FederationRequirements.md (ADR-02, ADR-04)"
tests:
- "shard-wiki/tests/test_model.py"
- "shard-wiki/tests/test_provenance.py"
consumer_guidance:
recommended_for:
- a portable, provenance-carrying representation of wiki pages across backends
not_recommended_for:
- cases needing a single canonical path per page (use identity, not path)
known_limitations:
- non-prose shapes specified ahead of implementation
---
# Backend-Neutral Wiki Page Model
The top narrow waist: a Markdown-first model that stretches to typed records, typed-graph
statements, inline-embedded objects, non-Markdown assets, and computational shapes. Identity
is a stable handle; placement and equivalence are separate mechanisms; provenance is layered
(effective = page envelope + span delta).
## Assessment notes
### Discovery
Specified in CoreArchitectureBlueprint Section 7 and FederationRequirements ADR-02/04.
### Availability
`model/` + `provenance/` ship the prose path and the layered envelope today.

View File

@@ -0,0 +1,114 @@
---
id: capability.wiki.shard-orchestration
name: Wiki Shard Orchestration
summary: Present a union of pages across heterogeneous wiki-shaped shards while preserving each shard's provenance, capabilities, and history.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, federation, orchestration, union, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Grounded in 84 documented use cases and a twice-reviewed whole-system
architecture (CoreArchitectureBlueprint) derived from ~23 prior-art systems.
availability:
current: A2
target: A5
confidence: medium
rationale: >
InformationSpace orchestrator (attach -> resolve -> read, chorus on
ambiguity) works as a Python source module; network API and incremental
union are planned.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- attach folder shards and read a union page with layered provenance
- chorus presentation of equivalent-but-divergent pages (union without erasure)
broken_expectations:
- incremental union maintenance and equivalence index not yet built
- write-through federation transports not yet built
out_of_scope_expectations:
- hosting or replacing the underlying wiki engines
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- early implementation; 64 tests but no production exposure
discovery:
intent: >
Let independently stored, differently implemented wikis behave as one
coherent, versionable, inspectable information space without homogenizing them.
includes:
- union resolution across shards (identity-keyed)
- chorus / designated-canonical presentation of equivalent pages
- lazy replication projection of remote content with freshness
excludes:
- implementing a backend wiki engine (see capability.wiki.engine-typed-extensions)
- silent remote mutation
assumptions:
- canonical truth lives in shards + a git coordination journal; the union is derived
use_cases:
- "shard-wiki UseCaseCatalog UC-01..UC-07, UC-26..UC-33 (information space, federation, coordination)"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/union/"
- "shard-wiki/src/shard_wiki/space.py"
target_artifacts:
- orchestrator network API
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.adapter-contract
- capability.wiki.page-model
- capability.wiki.coordination-journal
supports:
- capability.wiki.federation-models
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md"
- "shard-wiki/spec/FederationArchitecture.md"
tests:
- "shard-wiki/tests/test_union.py"
- "shard-wiki/tests/test_integration.py"
consumer_guidance:
recommended_for:
- composing multiple Markdown/wiki stores into one provenance-preserving view
not_recommended_for:
- replacing a single wiki engine
known_limitations:
- resolution is recompute-on-read until the incremental tier lands
---
# Wiki Shard Orchestration
shard-wiki's core capability: orchestrate wiki-shaped content across heterogeneous *shards*
as a union of pages, preserving provenance, capabilities, and history per shard. Canonical
truth stays at the edges (shards + the git coordination journal); the union is a derived,
recomputable view (orchestrator, not engine).
## Assessment notes
### Discovery
Grounded by `UseCaseCatalog.md` (84 UCs) and the hardened `CoreArchitectureBlueprint.md`.
### Availability
`InformationSpace` provides attach/resolve/read today (source module); a network API is the
target availability step.

View File

@@ -0,0 +1,145 @@
version: 1
updated: '2026-06-16'
domain: helix_forge
capabilities:
- id: capability.wiki.shard-orchestration
name: Wiki Shard Orchestration
summary: Present a union of pages across heterogeneous wiki-shaped shards while
preserving each shard's provenance, capabilities, and history.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.shard-orchestration.md
tags:
- wiki
- federation
- orchestration
- union
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.adapter-contract
name: Capability-Aware Shard Adapter Contract
summary: A versioned backend interface where each binding declares a verified capability
profile, so federation ops degrade by capability.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.adapter-contract.md
tags:
- wiki
- adapter
- capability
- contract
- conformance
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.page-model
name: Backend-Neutral Wiki Page Model
summary: A Markdown-first but stretchable page model with stable identity separate
from placement and layered provenance.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.page-model.md
tags:
- wiki
- page-model
- identity
- provenance
- markdown
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.coordination-journal
name: Event-Sourced Coordination Journal
summary: An append-only, totally-ordered-per-space decision log whose current state
is a derived fold; git-addressable history.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.coordination-journal.md
tags:
- wiki
- event-sourcing
- coordination
- git
- journal
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.overlay
name: Overlay-Before-Mutation Write Path
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only
or limited backends be edited safely without silent remote mutation.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.overlay.md
tags:
- wiki
- overlay
- patch
- write-path
- conflict
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.federation-models
name: Selectable Federation-Model Taxonomy
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication,
query-join, feed, activity-streams, engine-mirror) selected per space.
vector: D4 / A0 / C1 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.federation-models.md
tags:
- wiki
- federation
- taxonomy
- composable
- shard-wiki
consumption_modes:
- informational
- id: capability.wiki.engine-typed-extensions
name: Wiki Engine with Typed Extensions
summary: A small-core wiki engine realizing a typed-extension framework that addresses
all wiki use cases and lets each shard activate only the features it needs.
vector: D3 / A0 / C0 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.engine-typed-extensions.md
tags:
- wiki
- engine
- typed-extensions
- feature-activation
- shard-wiki
consumption_modes:
- informational
- id: capability.wiki.derived-views
name: Wiki Derived Views
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages,
SiteMap, and (delegate-or-derive) Search — carrying provenance.
vector: D3 / A0 / C0 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.derived-views.md
tags:
- wiki
- derived-views
- backlinks
- recentchanges
- search
- shard-wiki
consumption_modes:
- informational

View File

@@ -0,0 +1,38 @@
# 260614 — The computational page model (SHARD-WP-0004 synthesis)
Date: 2026-06-15 · Source: **SHARD-WP-0004** post-batch synthesis (T1T8)
## What this is
The acceptance-criteria synthesis for SHARD-WP-0004 — *"the computational page model"*
reading the eight computational/interactive-knowledge dives across each other and distilling
them into **one model**: **source is canonical; everything rendered/computed is a
projection**, placed on **two axes** (projection-kind: replication vs derivation; liveness:
live↔snapshot), with a recommendation on an executable-content capability.
## The answer to the carried question
*Can a shard-wiki page be a live computational artifact?* **Yes — as a page-model +
projection concern, not as an execution platform.** Every system externalizes to a canonical
source and treats the live/computed form as derived; shard-wiki **recognizes** computational
content, **attaches the source**, and **presents derivations as provenance- and
liveness-marked projections**, with **execution as a gated capability (off by default,
degrade to snapshot)**. No INTENT amendment required.
## Key contributions
- **One model:** `(source, derivation rule, projection with provenance + liveness)` covers
all four computational page shapes (one-source-many-projections UC-83; notebook UC-84;
program-as-page; live/temporal content).
- **Two axes for T16:** replication vs **derivation-projection** (timing / multiplicity /
continuity facets) × the **live↔snapshot** axis (bounded at the irreducibly-live far end by
Strudel).
- **One snapshot-provenance record** reused for notebook outputs, renders, recordings.
- **Hard boundaries:** never host a kernel/runtime as store; **image-is-not-a-store**; never
present a derivation without output→source provenance.
## Contents
| Path | Role |
|------|------|
| `findings.md` | The source/derivation/projection model, the two axes, the four page shapes, provenance/reproducibility, the recommendation, the SHARD-WP-0002 fold-in, escalated open questions |

View File

@@ -0,0 +1,178 @@
# The computational page model — synthesis (SHARD-WP-0004)
**Date:** 2026-06-15 · **Source:** SHARD-WP-0004 (post-batch synthesis, T1T8) · **Kind:**
synthesis (no new external research) reading the eight computational/interactive-knowledge
dives *across* each other.
## What this is
The acceptance-criteria synthesis for SHARD-WP-0004. The batch asked one carried question:
*can a shard-wiki page be a live computational artifact — a source woven/evaluated into
rendered forms — and if so, how do projection, transclusion, provenance, and the adapter
contract treat the source, the environment, and the computed output?* This memo answers it
by distilling the eight dives into **one model**: the **source / derivation / projection**
view of computational content, anchored on two axes, plus a recommendation on whether
shard-wiki needs an executable-content capability.
Dives consolidated: **T1** literate programming (WEB/weave/tangle), **T2** Mathematica,
**T3** Jupyter, **T4** Processing/p5.js, **T5** Strudel, **T6** Squeak, **T7** Glamorous
Toolkit, **T8** Pharo. Catalog yield: **UC-83** (literate one-source-many-projections),
**UC-84** (notebook with computed-output provenance); the other six are **enrichment/boundary**
dives (UC-54/55/47/48 + the projection model).
## 1. The one finding: source is canonical, everything rendered is a projection
Every system in the batch, however "live," **externalizes to a canonical artifact** and
treats the rendered/computed form as **derived**:
| System | Canonical (attach this) | Derived (a projection) |
|--------|-------------------------|------------------------|
| Literate (WEB) | the WEB/`.nw`/`.org` **source** | woven docs **and** tangled code |
| Mathematica | the `.nb` (a Wolfram expression) | cached `Output` cells; CDF; `Dynamic` widgets |
| Jupyter | `.ipynb` source (ideally paired text) | embedded outputs; nbconvert/nbviewer renders |
| Processing | the **sketch source** | the view-time canvas render |
| Strudel | the **pattern source** | the live audio performance / a recording |
| GT / Lepiter | git-versionable **page/Tonel files** | moldable `gtView`s; live snippet results |
| Squeak/Pharo | exported **files** (Tonel/git) | the live image / running objects |
**This is the same principle shard-wiki already holds** (files-canonical, index/render
derived — ikiwiki UC-79, Logseq UC-62, nbstripout). The computational batch did not break it;
it **stress-tested it to the live extreme and it held**. *The image (Squeak/Pharo) is the
only would-be exception, and it is a boundary, not a counterexample: an image is not a store.*
## 2. Two axes the contract must add
The batch refines the **projection model** (SHARD-WP-0002 T16) with two orthogonal axes:
### 2.1 Projection kind: replication vs derivation
- **Replication-projection** (the existing default): a lazy **cache/copy** of remote content
(Obsidian/Notion mirrors, UC-53/57).
- **Derivation-projection** (new, from T1): a **transform/compile/weave/evaluate** of a
source into rendered forms — regenerable, may delegate to the source's tool, **degrades to
a captured snapshot** when the tool is absent. Covers weave/tangle, nbconvert, CDF, sketch
render, audio recording, `gtView`.
Sub-facets of derivation (from T4): **materialization timing** = *ahead-of-time* (CDF,
nbconvert, static HTML) vs *view-time* (Processing/Strudel); **multiplicity** = one output
(UC-79) vs **N co-equal projections** (UC-83 weave+tangle; UC-47/48/54 moldable views).
### 2.2 Liveness: the live↔snapshot axis (from T6, bounded by T5)
Every derived view sits on a spectrum, and the more live the source, the more its static form
is a clearly-marked degrading snapshot:
```
static source ── captured output ── live-over-files ── view-time one-shot ── continuous/
(literate) (notebook UC-84) (GT/Lepiter) (Processing) interactive
── irreducibly
live/temporal
(Strudel: source
+ recording only)
```
**Honesty rule (union-without-erasure):** a computed/live view must always declare *what it
is* — "captured at run N, environment unguaranteed" (UC-84), "one performance, time T, source
rev R" (UC-83/Strudel), "live render needs the runtime." Never present a snapshot as live or a
static page as capturing a live artifact.
## 3. The computational page-model shapes (T12)
The batch adds these page shapes (beyond prose / typed records / query-defined / inline-
embedded objects / typed-graph already catalogued):
1. **One-source-many-projections** (UC-83) — a source whose presented forms are co-equal
derivations (docs + code), each with output→source provenance; **named-chunk transclusion**
assembles fragments by name at derivation time (UC-32/44).
2. **Notebook** (UC-84) — **ordered/nestable cells** (Mathematica adds the outline tree)
where code cells own **embedded computed outputs** (the derived output is stored *inside*
the source) with **weak execution provenance**; outputs may be MIME blobs or **structured
re-evaluable values** (Mathematica) — a new point on the content-opacity spectrum.
3. **Program-as-page** (Processing) — canonical content = **source text**, presentation = an
**executable render** with **no cached output**; non-Markdown executable content.
4. **Live/temporal/generative content** (Strudel) — source canonical, render irreducibly
live; static = source + a marked recording.
All four reduce to **(source, derivation rule, projection with provenance + liveness)** — one
model, four positions.
## 4. Provenance & reproducibility (the honest weakness)
Computed output provenance is **real but fragile** everywhere: Jupyter/Mathematica
`execution_count`/`In`-`Out` can be **out-of-order**; **environment/versions/data are not
captured**; Strudel/Processing may be **non-deterministic**. Implication for the contract:
treat a computed output as a **snapshot with declared, incomplete provenance** (run id, source
rev, timestamp; environment "unguaranteed"), reusing **one snapshot-provenance machinery**
across notebooks, recordings, and renders (UC-84 is the template). This is consistent with
shard-wiki's existing "surface freshness/completeness, never imply more than you have"
(Oddmuse partial-history UC-82).
## 5. The recommendation
**Does shard-wiki need an executable/computational content capability? — Yes, but only as
recognition + projection + capability-gating, never as an execution engine.**
1. **Adopt the source/derivation/projection model** (no execution required). shard-wiki
**recognizes** computational content types (literate source, notebook, sketch, pattern),
**attaches the canonical source**, and **presents derived forms as projections with
provenance + liveness markers**. This alone delivers UC-83/UC-84 and the enrichment of
UC-54/55 — and needs **no kernel, no sandbox, no runtime**.
2. **Make execution a capability, off by default.** "Drive a derivation" (run tangle/weave,
re-execute a notebook, render a sketch, evaluate a pattern in the viewer) is a **gated
capability** with a **trust/sandbox** sub-concern (T11). Absent it, **degrade to the
captured snapshot / static render / recording** — the graceful-degradation rule, which the
batch shows always has an honest fallback (source is tiny and diffable everywhere).
3. **One projection model, two axes** (T16): projection-kind (replication vs derivation; with
timing + multiplicity facets) × liveness (live↔snapshot). The **moldable view registry**
(T7) is the unifying structure — an open, type-keyed set of co-equal projections, none
canonical-by-fact (display-canonical is policy).
4. **One snapshot-provenance record** reused for notebook outputs, renders, and recordings
(run id, source rev, timestamp, environment "unguaranteed").
5. **Hard boundaries** (design-bugs if violated): never host a kernel/runtime as the store;
**image-is-not-a-store** (attach exported files); never present a derivation without
output→source provenance; never imply a static view captures a live artifact.
**Net:** computational content is **in scope as a page-model + projection concern**, **out of
scope as an execution platform** — exactly the mechanism-over-policy, capability-aware,
degradable posture INTENT already mandates. No INTENT amendment is required; this extends the
page model and projection model within existing constraints.
## 6. Fold into SHARD-WP-0002
- **T12 (page model):** add the four computational shapes (§3); allow nestable cells and
structured re-evaluable outputs; "derived output may live inside the source" (notebook).
- **T16 (projection):** the **two-axis model** (§2) + the **moldable view registry** (§3/T7);
materialization-timing, multiplicity, continuity, and the live↔snapshot far end as explicit
projection metadata.
- **T11 (capabilities):** "derive/execute/render/evaluate" as gated capabilities with trust/
sandbox; default off → snapshot.
- **T15 (fidelity):** non-Markdown executable/computed content; lossy renders; the
structured-re-evaluable-value point on the content-opacity spectrum.
- **T13 (history):** paired-text (Jupytext) / cell-aware (nbdime) strategies for embedded-
output documents; outputs-as-derived (nbstripout ethos).
- **T14 (binding):** **image-is-not-a-store** boundary (export→files only).
## 7. Open questions (escalated)
1. Is **liveness** (and "irreducibly live / no faithful static form") an explicit first-class
metadata flag on every projection, so the union renders the honest fallback automatically?
(T5/T6 far-end question.)
2. Does shard-wiki **ever drive a derivation** (sandboxed), or strictly attach + present
snapshots? (Recurs UC-56/UC-83/UC-84/T4/T5 — a single capability/trust policy decision.)
3. Is a computed output's **structured re-evaluable value** (Mathematica/Wolfram) modeled as a
typed value or stored opaquely with provenance? (UC-55 open-Q #10; UC-84 Q3.)
4. Should **UC-83** and **UC-84** eventually merge as two positions of one "source +
derivations" shape, or stay distinct? (Kept distinct: UC-84's defining trait is *output
embedded in source with weak provenance*; UC-83's is *N co-equal external derivations*.)
## 8. Sources
The eight SHARD-WP-0004 dives: `research/260614-{literate-programming,mathematica,jupyter,
processing,strudel,squeak-pharo,glamorous-toolkit}-deep-dive/`. Prior projection/structure
anchors: `research/260614-{ikiwiki,logseq,zigzag}-deep-dive/`,
`research/260614-shard-spectrum-synthesis/`.
## 9. Traceability
No new UC (consolidation). Consolidates **UC-83, UC-84** and enrichments to **UC-32, UC-44,
UC-47, UC-48, UC-54, UC-55, UC-79, UC-37, UC-35**. Feeds **SHARD-WP-0002 T11/T12/T13/T14/T15/
T16** (see §6). Recommendation: computational content is **in scope as page-model + projection
mechanism, out of scope as an execution platform**; no INTENT amendment required.

View File

@@ -43,4 +43,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
| 2026-06-14 | `260614-mathematica-deep-dive/` | Mathematica Notebooks — the original computational notebook (`.nb` = a Wolfram expression); nestable cell groups, structured re-evaluable outputs, `Manipulate` live widgets, CDF; confirms UC-84 notebook shape is a genus; SHARD-WP-0004 T2; enrichment-only (reinforces UC-84; UC-54/55) |
| 2026-06-14 | `260614-squeak-pharo-deep-dive/` | Squeak & Pharo (image-based Smalltalk) — the live-object image (purest "live" end); image-is-not-a-store boundary (export→files only); Pharo Tonel/Iceberg externalizes code to git text; names the live↔snapshot projection axis; SHARD-WP-0004 **T6 + T8** (merged); boundary/enrichment-only, no new UC |
| 2026-06-14 | `260614-processing-deep-dive/` | Processing / p5.js — program-as-page rendered live at view time (no cached output); adds materialization-timing + continuity facets to projection; execute-in-viewer = capability+trust; SHARD-WP-0004 T4; enrichment-only (UC-54/55) |
| 2026-06-14 | `260614-strudel-deep-dive/` | Strudel.cc (TidalCycles JS) live-coding REPL — code as live time-based audio performance; the far live end (no faithful static form; static = source + marked recording); honesty test for graceful degradation; SHARD-WP-0004 T5; enrichment-only (UC-54/55) |
| 2026-06-14 | `260614-strudel-deep-dive/` | Strudel.cc (TidalCycles JS) live-coding REPL — code as live time-based audio performance; the far live end (no faithful static form; static = source + marked recording); honesty test for graceful degradation; SHARD-WP-0004 T5; enrichment-only (UC-54/55) |
| 2026-06-15 | `260614-computational-page-model-synthesis/` | **SHARD-WP-0004 synthesis***the computational page model*: source canonical / everything rendered is a projection; two axes (replication↔derivation, live↔snapshot); four computational page shapes; recommendation — executable content in scope as page-model+projection, out of scope as an execution platform (no INTENT amendment); feeds SHARD-WP-0002 T11T16 |

View File

@@ -0,0 +1,976 @@
# CoreArchitectureBlueprint — shard-wiki
Status: **draft for review** · Date: 2026-06-15 · Owner: tegwick
The whole-system architecture for shard-wiki, synthesised from `INTENT.md`, the 84-entry
`UseCaseCatalog.md`, and the full research arc (`research/260608-*`, `research/260613-*`,
`research/260614-*` — ~23 wiki/knowledge systems plus two cross-dive syntheses). This is the
**core** blueprint: it defines the layers, the abstractions, and the load-bearing decisions
that everything else implements.
Scope relationship to the other specs:
- **`ArchitectureBlueprint.md`** (existing) is the **authorization & history sub-blueprint**
(the L0L4 ladder). This document references it as the design of the cross-cutting
authorization layer (§9) and does not restate it.
- **`SHARD-WP-0002`** is the workplan that turns §6§8 into
`spec/FederationArchitecture.md` + the adapter-contract section of
`spec/TechnicalSpecificationDocument.md`.
- **`UseCaseCatalog.md`** is the demand this architecture must satisfy; UC references below
are load tests, not decoration.
- **`WikiEngineCoreArchitecture.md`** designs shard-wiki's native, headless, API-first wiki
engine as a **canonical-mode shard backend** (one shard behind §6/§A — federation, union, and
projection stay here in the orchestrator, not in the engine). Added per the 2026-06-15 INTENT
amendment (decision `84ffdb48`, SHARD-WP-0013).
---
## 1. The thesis: *canonical vs derived* (three states)
Everything in shard-wiki follows from one organising decision — that state comes in exactly
**three kinds**, and only one of them is disposable:
> **1. Sharded-canonical** — content owned by each shard (shard sovereignty).
> **2. Coordination-canonical** — durable state *born inside shard-wiki* that encodes human
> or cross-shard decisions and exists nowhere else: overlays (the local truth against a
> read-only shard), curator equivalence bindings, alias tables, merge/reconciliation
> decisions. It is recorded as an **append-only decision log in the Git coordination
> journal** (event-sourced, §8.1); the *queryable current form* of that state (the effective
> alias table, the equivalence set) is a **derived fold** of the log — i.e. tier 3, not tier 2.
> What is canonical is the **log of decisions**, not any mutable snapshot of them.
> **3. Derived-disposable** — everything shard-wiki *computes* from (1)+(2): the union graph,
> equivalence index, query indexes, projections, views. It can be deleted and recomputed.
>
> **Canonical = sharded coordination. Derived = a pure function of canonical:**
> `derived = f(sharded, coordination)`.
This is the architectural form of "orchestrator, not engine." shard-wiki never *becomes* the
source of truth; it composes sources and records the decisions it makes about them. The
research earned the *files-canonical* half empirically — every serious system externalises its
durable truth to files+VCS and treats the rest as derived: Logseq (DataScript index over plain
files), ikiwiki (static HTML compiled from a git repo), Glamorous Toolkit / Lepiter (live
views over git-versioned JSON), Pharo (Tonel/Iceberg code as git text), Jupyter teams
(nbstripout — outputs are derived noise). The one tradition that refuses this — the Smalltalk
**image** — is exactly the one we record as a *boundary, not a backend*
(`research/260614-squeak-pharo-deep-dive`).
The earlier draft of this blueprint used a two-bucket framing ("canonical at the edges,
derived in the middle"). That was wrong by omission: it had no home for **coordination-
canonical** state, and so contradicted itself by listing curator bindings and alias tables as
"derived/rebuildable" when a human binding manifestly cannot be rebuilt. The three-state model
fixes that crack (review finding A-1) and makes `derived = f(canonical)` *literally* true.
Three consequences fall straight out, and they are the spine of the rest of this document:
1. **Graceful degradation is free.** If the derived tier is always recomputable, a backend
that can only be read is still a first-class participant — you just derive less from it.
2. **Provenance is tractable.** Because shard-wiki never claims to *be* the source, every
derived artifact can always point back to the canonical input it came from (union without
erasure is a structural property, not a feature bolted on).
3. **The derived tier is a pure function of canonical state.** `derived = f(sharded,
coordination)`. Bugs in the derived tier are recoverable by recompute; only the two
canonical tiers must be durably protected — sharded by each shard, coordination by the
Git journal (history). *Recomputability is a correctness property of the derived tier, not
a promise that a from-scratch rebuild is operationally cheap — see §8.4 and the
operational-envelope axis.*
### The dual narrow waist
Heterogeneity is mediated at exactly two interfaces, and nowhere else:
- **Bottom waist — the Shard Adapter Contract (§6).** Every backend, however weird, enters
through one versioned, capability-described interface.
- **Top waist — the Wiki Page Model (§7).** Every consumer, however demanding, sees one
backend-neutral, Markdown-first-but-stretchable page model.
Between the waists, core logic is written **once** against capabilities and the page model —
never against a specific backend. Adding TiddlyWiki or Notion or a git forge is writing an
adapter and declaring a capability profile, not editing core algorithms.
---
## 2. Architectural invariants
These are non-negotiable. Violating one is a design bug, not a tradeoff. They are INTENT's
principles fused with the research through-lines.
| # | Invariant | Source |
|---|-----------|--------|
| I-1 | **Orchestrator, not engine.** Core composes shards; it never replaces or homogenises them. | INTENT Stability Note |
| I-2 | **Three states; derived = f(canonical).** State is sharded-canonical, coordination-canonical (journal), or derived-disposable. The derived tier (union/index/projection) is a pure, recomputable function of the two canonical tiers; only canonical state is durably protected. | §1; Logseq/ikiwiki/GT through-line |
| I-3 | **Capability-awareness is data.** A binding's abilities are a *profile* (positions on spectra), read by generic core logic — not per-backend branches. | synthesis v3 §2; INTENT capability-aware adapters |
| I-4 | **Union without erasure.** Every page/revision/projection/overlay/view carries its provenance, freshness, liveness, and divergence. | INTENT; provenance-granularity spectrum (Wikibase) |
| I-5 | **Overlay before mutation.** Writes to anything below write-through land as drafts/patches/MRs first; no silent remote mutation. | INTENT |
| I-6 | **Git-addressable coordination.** Every information space has a Git-backed journal even when its shards are not git-native. | INTENT |
| I-7 | **Mechanism over policy.** Canonical-source, conflict, editorial, sync cadence are configurable presets, never hard-coded. | INTENT |
| I-8 | **Graceful degradation.** A limited backend is still usable as read-only / cache / projection / backup / patch target. | INTENT |
| I-9 | **Identity ≠ placement.** A page is an entity that may occupy N locations; address by identity, not by path. | Trilium note/branch; ZigZag |
| I-10 | **History is the floor.** Every write is a recoverable commit; recoverability, not gatekeeping, is the baseline protection. | ArchitectureBlueprint §2 |
| I-11 | **Authorization in core, authentication delegated.** Core decides who-may; an external provider says who-is. | INTENT; ArchitectureBlueprint |
| I-12 | **Not a file-sync daemon; not an execution platform.** Sync is wiki-page-semantic; computation is recognised+projected, not hosted. | INTENT; computational-page-model synthesis |
| I-13 | **Tenant-partitioned derived state.** Derived state is partitioned by tenant/root entity; no derived artifact spans tenants except via explicit, authorised cross-root federation. | §9.1; review B-3 |
---
## 3. The layered architecture
```
┌───────────────────────────────────────────────────────────────┐
│ L6 Consumers — Orchestrator API · CLI/agents · Web/Obsidian │
├───────────────────────────────────────────────────────────────┤
X-cut │ L5 Authorization (PEP/PDP, identity-provider iface) → │ X-cut
Prove- │ see ArchitectureBlueprint.md (L0L4 ladder) │ Capa-
nance ├───────────────────────────────────────────────────────────────┤ bility
▲ │ L4 Union & Projection (DERIVED · rebuild=fallback) │ ▲
│ │ identity resolution · equivalence/chorus · union graph · │ │
│ │ replication+derivation projections · moldable view registry│ │
│ │ · derived query index │ │
│ ├───────────────────────────────────────────────────────────────┤ │
│ │ L3 Coordination (Git journal · overlay/patch engine · │ │
│ │ federation-model strategies · reconciliation) │ │
│ ├───────────────────────────────────────────────────────────────┤ │
│ │ L2 Wiki Page Model ── TOP WAIST ── │ │
│ │ backend-neutral pages · identity≠placement · span address ·│ │
│ │ provenance envelope · the page shapes │ │
│ ├───────────────────────────────────────────────────────────────┤ │
│ │ L1 Shard Adapter Contract ── BOTTOM WAIST ── │ │
│ │ versioned iface · capability profile (orthogonal) · │ │
│ │ attachment-mode binding · operation verbs │ │
└──── ├───────────────────────────────────────────────────────────────┤ ──┘
│ L0 Backends (not ours): git repos, wiki/ subdirs, Gitea/ │
│ GitLab/GitHub wikis, folders, Obsidian, WebDAV, Notion, │
│ Coulomb spaces, notebooks, … │
└───────────────────────────────────────────────────────────────┘
```
**Provenance** and **Capability** are drawn as vertical rails because they are not layers —
they are present at every layer. A page object at L2 carries provenance; a projection at L4
carries provenance; an authz decision at L5 records the context under which content was read.
Likewise a capability profile declared at L1 is consulted at L3 (can we write-through?), L4
(can we delegate a query?), and L5 (can this principal even reach the op?).
The dependency rule is strict and downward, and it tracks the **three states (§1)**, not the
layer numbers: **the derived-disposable tier (the whole of L4) may be deleted and recomputed
from canonical state (sharded content at L1 + coordination-canonical state in the L3 journal).**
Nothing canonical may depend on derived state. Note the journal at L3 is *canonical* (it holds
overlays, bindings, aliases, merges); only L4 is disposable.
---
## 4. Core abstractions (the vocabulary code must use)
Straight from INTENT, sharpened by research. New code maps onto these; it does not invent
parallel terms.
- **Shard** — an independently meaningful page store attached to a root entity, with
*sovereignty*: its own backend, capability profile, history, identity model, limits.
- **Root entity / information space** — the joined space shards attach to; the unit of
Git coordination and of multi-tenancy (a tenant maps to a root entity, ArchitectureBlueprint).
- **Shard adapter contract** — the versioned L1 interface; the bottom waist.
- **Capability profile** — a shard binding's position on each of the 15 spectra (§6) plus its
supported verbs. *The* data structure that drives degradation.
- **Wiki page model** — the L2 backend-neutral page; the top waist.
- **Page identity vs placement vs equivalence** — a page is an entity with a *stable handle*
(identity); it may have N placements (paths/shards); **addressing and transclusion key on
identity, but equivalence keys on content fingerprint *across* identities** (§7.2, I-9). The
three are distinct mechanisms — never conflate identity with a fingerprint.
- **Provenance envelope** — the metadata each artifact carries (source shard, freshness,
liveness, authz context, overlay status, divergence, lineage), stored **layered**: a
page-level envelope + span-level *deltas*, so per-span cost is near-zero when uniform (§7.3).
- **Coordination journal** — the L3 Git-backed, **append-only decision log** for a space: the
durable home of all **coordination-canonical** state (§1, §8.1) as *events* (overlay-created,
binding-made, alias-set, merge-decided), plus the content change-flow record. It is event-
sourced — committed, never overwritten; the queryable current coordination state is a derived
fold of it (§8.1).
- **Overlay** — a non-destructive local edit against a remote/read-only/limited shard,
representable as draft/patch/commit/MR before destructive apply. Coordination-canonical: an
unapplied overlay is the local truth and lives in the journal.
- **Projection** — a derived view of shard content. The default is a **plain lazy
replication-projection** (a freshness-stamped cache); only *source* content needing
transform/evaluate uses the **derivation-projection** extension point with its two-axis
typing (kind × liveness) and the moldable view registry (§8.4§8.5).
- **Federation model** — the selected coordination strategy for a space (§ taxonomy, T17).
- **Shard mode** — read-only · write-through · mirrored · projected · cached · canonical
(a *policy* selection constrained by the capability profile).
---
## 5. Why "layered" and not "pipeline" or "plugin-bus"
Two rejected alternatives, recorded so the choice is legible:
- **A sync pipeline** (source → transform → sink) was rejected: it implies a privileged
direction and a canonical sink, which violates shard sovereignty (I-1) and union-without-
erasure (I-4). shard-wiki is a *star* (many shards ↔ one space), not a pipe.
- **A flat plugin bus** (every backend a peer plugin emitting events) was rejected as the
*top-level* shape: it has no narrow waist, so heterogeneity leaks into every consumer.
We keep the plugin idea but confine it to L1 (adapters) and L3 (federation strategies),
behind the waists.
The layered-with-rails shape is what makes I-2/I-3/I-4 hold simultaneously.
---
## 6. Bottom waist — the Shard Adapter Contract (L1)
The single most important design decision in the project: **the adapter contract models
positions on capability spectra, not a flat checklist of boolean verbs.** A backend is not
"can/can't merge"; it sits *somewhere* on the merge spectrum, and federation operations
degrade by position. This is the lesson of putting ~23 systems in one matrix
(`research/260614-shard-spectrum-synthesis`, v3).
### 6.1 The fifteen capability spectra
Each binding declares a position on each axis. Core algorithms read these positions; there is
no per-backend code in core (I-3).
1. **Addressing granularity** — none → path → page-level store-id → in-file span → in-file
block id (Logseq `id::`) → store-UUID → portable tumbler (Xanadu, the unreached ideal)
2. **Content identity** — none → path/title → fingerprint → span-set
3. **Identity vs placement** — path=identity → identity separated from placement (Trilium
note/branch = a DAG)
4. **Structure** — flat MD → frontmatter/`key::` → `%META%` → typed objects → DB schema+
relations → object-graph/ontology → computed (inherited+templated) → typed-graph statements
5. **History** — none → internal-only / CRDT-log → open-file → git-native
6. **Merge model** — none → git/text → conflict-notes/keep-both → native-CRDT → coexist-with-rank
7. **Native query** — none → text → build-your-own derived index → datalog/graph → DB query → SPARQL
8. **Translation** — native → lossless → lossy-with-fidelity-report (incl. HTML)
9. **Attachment mode** — file-store (native | interchange-mirror) → git-IS-store → in-engine-host
→ local-REST → external-API → direct-DB → CRDT-replica → P2P/no-central-endpoint
10. **Operational envelope** — local/unbounded → realtime CRDT/WebSocket → rate-limited/
eventually-consistent/paginated
11. **Access grant** — open → token → OAuth scoped+revocable → P2P key/invite → enterprise ACL
12. **Content opacity** — plaintext → structured re-evaluable value → encrypted whole-shard →
per-item → proprietary-lossy-exportable
13. **Write granularity** — whole-file (TiddlyWiki) → per-page → section/anchor → per-block → story-item
14. **Provenance granularity** — per-shard → per-page → per-edit → per-statement/value (Wikibase rank+refs)
15. **Computational / liveness** — static source → captured-output snapshot → live-over-files →
view-time render → irreducibly-live/temporal
### 6.2 Operation verbs
`read, write, diff, merge, lock, version, publish, notify, transclude-source,
translate-syntax, structured-payload, derive-projection, execute/evaluate`. The last two are
**gated, off by default** (§8, computational content). Verb support is part of the profile and
must reconcile with the federation-ops capability matrix (SHARD-WP-0002 T10).
### 6.3 Attachment-mode taxonomy (axis 9, expanded)
A backend may offer **several** modes; attach mode is a **per-binding, capability-gated
choice**, with one declared authoritative. Modes: file-store (native vault/folder *or* an
interchange/sync mirror), **git-IS-store** (the home case — forge wikis & ikiwiki: git is the
store *and* the journal at once, resolving the engine-mirror write-race), in-engine hosted
adapter (XWiki component, Obsidian/Logseq/Roam plugin, Trilium script), local-REST (Joplin
Data API, Trilium ETAPI), external-API-only (Notion), direct-DB (MojoMojo schema→model),
CRDT-replica (Anytype/AFFiNE/AppFlowy), P2P/no-central-endpoint. **Boundary:** a monolithic
live-memory blob (Smalltalk image, a kernel) is **never** an attach target — it participates
only via export→files (I-12).
### 6.4 Contract rules
- **Versioned interface** (Foswiki::Store + Foswiki::Meta is the proof that a stable
store-interface-with-swappable-backends works). Capability discovery is a static profile
with optional runtime negotiation.
- **Backend-swap tolerance** — shard identity/provenance survives a substrate change
(RCS↔PlainFile, folder→Git, Logseq file→SQLite): bind to *capabilities*, not to "it's files."
- **Absence is first-class** — the profile must express *can't* cleanly (Oddmuse floor), so
degradation paths are explicit, never guessed.
### 6.5 Orthogonal core, implied positions, and the interaction subset
Fifteen independent ordinal axes is *descriptively* right but would be *operationally* a mess
if treated as fifteen free dimensions: the axes are **not orthogonal**, and a degradation
function over all 15 jointly is the flat-checklist problem returning in higher dimensions
(review D-1). Three rules tame it.
**(a) A small orthogonal core; the rest are implied.** Most axes are *correlated* and collapse
to a few independent choices. The **core axes** an adapter must independently declare:
1. **Substrate** → drives attachment-mode, history, merge, and native-query positions together
(git-IS-store ⟹ history=git-native ⟹ merge=git/text ⟹ query=build-your-own-index;
relational-DB ⟹ direct-DB attach ⟹ DB-version-row history ⟹ DB query).
2. **Write granularity** → drives addressing granularity and the overlay/patch shape.
3. **Content opacity** → drives translation and (where encrypted) collapses native-query.
4. **Operational envelope** → drives freshness mode (§8.8) and rebuild expectations (§8.7).
5. **Access grant** → independent (authz, L5).
6. **Computational/liveness** → independent (projection kind, §8.5).
The remaining axes are **implied/derived** from these via published implication rules; an
adapter *may* override an implied position, but the default is computed, not hand-set. This
turns ~15 free dimensions into ~6 independent ones plus derivations — fewer things to get
wrong, and impossible combinations become unrepresentable.
**(b) Implication rules forbid impossible profiles.** E.g. `attachment=git-IS-store ⟹
history≥git-native`; `opacity=encrypted-whole-shard ⟹ native-query=none ∧ translation≤opaque`;
`merge=native-CRDT ⟹ history=CRDT-log ∧ envelope=realtime`. A profile that violates an
implication is rejected at registration — capability-as-data (I-3) with integrity constraints.
**(c) The degradation function reads a *named, small* interaction subset — not all pairs.**
"No per-backend code" is only credible if we say *which* axis interactions the generic logic
actually consults. They are:
| Operation | Axes consulted (jointly) |
|-----------|--------------------------|
| **write / overlay-apply** | write-granularity × merge-model × history × access-grant |
| **transclude / address a span** | addressing-granularity × write-granularity × identity-vs-placement |
| **project / cache** | operational-envelope × computational-liveness × content-opacity |
| **query** | native-query × content-opacity (encrypted ⇒ derive-index-or-none) |
| **translate** | translation × content-opacity × structure |
| **federate** | substrate × history × merge (per the §8.3 model) |
Everything else is a single-axis check. This table *is* the degradation contract: it is small,
enumerated, and testable — the proof obligation behind "core logic written once."
### 6.6 Conformance — profiles are verified, never self-asserted
Capability-as-data (I-3) and the entire degradation contract (§6.5) rest on one assumption:
**the profile tells the truth.** If an adapter declares `merge=git/text` but corrupts merges,
or claims `notify` and never emits, it silently poisons every degradation decision in core —
the failure is invisible because core *believed the data* (review B-2). So the profile is not
taken on trust:
- **The contract ships a versioned conformance suite.** A published battery that, given a live
binding, **exercises each declared verb and each declared spectrum position and checks that
observed behaviour matches the claim** (a `write` round-trips; a `diff` is real; `notify`
actually fires; an "encrypted/opaque" shard genuinely refuses plaintext query; an
implication-rule position, §6.5(b), holds). The suite is versioned *with* the contract, so an
adapter proves conformance against a known contract version.
- **Passing conformance is an admissibility precondition.** A binding that fails (declares a
capability it does not honour) is **rejected at registration**, not run in production with a
lying profile. Capability discovery (§6.4) therefore yields a *verified* profile.
- **Self-reported, then verified.** Adapters still *declare* their profile (discovery stays
cheap); conformance *verifies* the declaration. The two together are what make I-3 and §6.5
sound rather than aspirational — degradation logic acts on verified data.
- **Mismatch is data, not a crash.** A conformance gap is reported as a precise
capability-by-capability diff (what was claimed vs observed), so an adapter author fixes the
profile or the code; degraded-but-honest registration (drop the unsupported claim) is allowed.
This is the same discipline a versioned store interface needs in general (the `Foswiki::Store`
lineage that inspired the contract): a backend may only participate behind the interface if it
*demonstrably* behaves as the interface says.
---
## 7. Top waist — the Wiki Page Model (L2)
Backend-neutral, **Markdown-first but stretchable many ways at once**. The page model is the
lingua franca every consumer sees; an adapter's job is to project its backend into this model
(read) and accept overlays back (write), within its capabilities.
### 7.1 Page shapes the model must carry
- **Prose Markdown** — the baseline.
- **Typed / computed records** — frontmatter/`%META%`/XObjects/Notion DB rows; **computed
metadata** (Trilium inherited+templated) represented as *effective-vs-own with per-attribute
provenance*.
- **Typed-graph statements** — Wikibase claim + qualifiers + references + rank (structure
far-end).
- **Inline-embedded objects** — Quip/Notion spreadsheets & live apps inside prose.
- **Non-Markdown assets** — drawings, canvases, images: typed asset / opaque blob / pluggable
content-type registry, never silent-flattened.
- **The four computational shapes** (§8): one-source-many-projections, notebook (embedded
computed output), program-as-page, live/temporal.
All shapes reduce to a common skeleton: **`(content | source, structure, provenance envelope,
[derivation rule])`**. The page model stores the richest faithful form as canonical and treats
any Markdown rendering of a non-Markdown shape as a *lossy projection* (I-4 + fidelity report).
### 7.2 Identity, placement, addressing — three distinct concepts
The earlier draft used "identity" for two different things and (worse) suggested deriving page
identity from a content fingerprint — which would make *editing a page change its identity* and
break every reference to it (review bug B-1). They are pulled apart here:
- **Page identity — a *stable handle*.** A shard-scoped, durable key that **survives edits**:
the backend's native page/note id where one exists (Roam/Notion/Trilium uid, a git path
treated as a name, a wiki page name), wrapped in a shard scope so it survives projection and
never collides across shards. Identity is *assigned/minted, not computed from content*.
References, placement, transclusion targets, and overlays all key on identity.
- **Placement — *where* an identity sits.** One identity → N placements (paths/shards) = a DAG;
no single canonical path (I-9). Placement can change without changing identity.
- **Content equivalence — *detecting sameness*, never identity.** A **content fingerprint** (or
span-set overlap) identifies a *version / a piece of content*, used to detect that two
*distinct identities* hold the same or derived content (the equivalence/chorus mechanism,
§8.4). A fingerprint is never a page's identity: same page, edited → new fingerprint, **same
identity**; two pages, identical content → same fingerprint, **different identities**.
- **Span addressing** — a sub-page address within an identity: adopt native span IDs where
minted (Roam `:block/uid`, Logseq `id::`, Notion/CRDT UUID); else a *position* address
(path+range) or a *content-fingerprint* address for equivalence/transclusion. The Xanadu
tumbler is the portable ideal the scheme aims at without requiring.
- **Provenance envelope** rides on pages and spans (see §7.3 for its layered, low-cost form).
So the chain is: **identity (stable) → placements (N, mutable) → equivalence (cross-identity
sameness, fingerprint-based)** — three concepts, three mechanisms, never conflated.
### 7.3 Provenance is layered, not per-span-duplicated
A provenance envelope on *every span* (source shard, freshness, liveness, overlay status,
authz context, divergence, lineage) would, at block granularity, mean ~10k near-identical
envelopes for a 10k-block page — provenance dwarfing content (review D-2). The fix is the exact
pattern the page model already uses for Trilium's computed metadata: **effective-vs-own**.
- **Page-level envelope** holds the values that are uniform across the page (almost always:
source shard, observed-at, liveness, authz context).
- **Span-level deltas** record *only where a span differs* from its page envelope — a
transcluded span from another shard, an overlaid span, a span that diverges. A span with no
delta inherits the page envelope at zero storage cost.
- **Effective provenance** for any span = page envelope ⊕ span delta, computed on read.
Per-span cost is therefore **near-zero in the common (uniform) case** and pays only for genuine
heterogeneity — the same "carry only the difference" principle, applied to shard-wiki's own
metadata. Provenance remains complete (I-4); it is just not redundantly materialised.
---
## 8. Coordination, federation & projection
### 8.1 Coordination journal (L3) — Git as the spine
Every information space has a Git-backed coordination journal (I-6). It records cross-shard
operations (fork, import, reconcile, overlay-apply, space-branch) and **is** the history floor
(I-10). For git-IS-store shards the shard's own git log *is* this journal; for non-git shards
the journal supplements (begins-now / mirrors-forward / snapshots-replica) or imports
(backfill open file history). History portability is a spectrum, handled per profile (axis 5).
**The journal is an append-only decision log; current coordination state is a derived fold
(review B-3).** The first draft said coordination-canonical state "lives in the journal"
without saying how Git — excellent for history, poor for mutable structured state — represents
an alias table or an equivalence graph. Resolution: **event sourcing.** The journal stores
*decisions as events* (`overlay-created`, `binding-made`, `alias-set`, `merge-decided`,
`page-forked`), append-only and git-addressable (so history/patch/review/backup over
coordination state come for free — I-6 is *strengthened*, not bypassed). The **queryable
current state** (the effective alias table, the live equivalence set) is a **derived fold** of
the log — tier-3 disposable, indexed like any other derived structure (§8.7), rebuilt by
replaying the log. So "all equivalences touching X" is an index lookup, not an O(scan) of Git.
This is the clean form of the §1 three-state model: **the log is canonical; its folded current
state is derived.**
**Concurrency: who may append (review B-1).** A multi-tenant L4 deployment runs several
orchestrator instances, so "the journal is local Git, single writer" is not given. The model:
- **One *append authority* per information space.** Appends to a space's log are serialized
through a single logical writer (a per-space lease/leader; instances without the lease forward
their append intents to it). This makes the log a **totally-ordered event sequence** per space
— the ordering authority §8.6 relies on — without a distributed transaction. Spaces are
independent, so this scales horizontally *across* spaces (the unit of partition is the space /
root entity, matching the tenant partition, I-13); it is a per-space serialization point, not
a global one.
- **Git is the durable, addressable form; appends are commits** (or fast objects batched into
commits) under the lease — no concurrent-writer merge races because there is one writer at a
time per space.
- **Read-your-writes** holds within a space because every reader resolves current state from
the same ordered log (or its fold); across spaces there is no shared state to be inconsistent.
- **HA / failover:** the lease is time-bounded and re-grantable; a failed append-authority is
replaced and resumes from the log's head (the log is the recovery point). A partition that
splits the authority degrades that *space* to read-only until a single writer is re-elected —
it never forks the log (availability yields to log integrity; an explicit, stated trade).
- **Open residual (→ §12, O-3-adjacent):** whether very high append rates need per-space log
*sharding* (sub-logs merged by a deterministic order) is an implementation spike, not an
architectural change.
**History must stay recoverable *and* bounded (review C-3).** "Every write is a commit" + open
L0 means an unbounded, bot-/vandalism-amplified journal that eventually degrades Git itself.
Recoverability (I-10) is non-negotiable, so the answer is *compaction, not deletion*:
- **Routine git maintenance** — background `gc`/repack, commit-graph, and (for very large
spaces) partial-clone / sparse strategies; operational, no semantic change.
- **Squash-compaction of low-value churn (policy, §10)** — long runs of rapid same-author
edits or revert-pairs can be folded into checkpoint commits *while preserving the recoverable
endpoints*; what is squashed is configurable and always leaves the content recoverable (it
compacts the *path*, not the *reachable states*).
- **Per-shard history offload** — a git-IS-store shard keeps its own history in its own repo;
the coordination journal references it rather than duplicating it (the journal records
*coordination* events, not a second copy of every shard commit).
- **Anti-abuse hooks (policy)** — rate-limiting / quarantine for anonymous L0 writers feed the
authz/policy layer; they throttle *abuse*, never legitimate history. Recoverability is the
floor; bounding is how it survives at scale.
### 8.2 Overlay / patch engine (L3)
The default write path for anything below write-through capability (I-5): an edit becomes a
draft → patch/commit → MR, applied destructively only on explicit intent and only where the
profile + policy both permit. This is what lets a read-only or rate-limited or lossy backend
still be *edited* safely.
### 8.3 Federation is plural & composable (L3) — the model taxonomy
Federation is not one mechanism. shard-wiki selects a **federation model per space and
composes per shard** (mechanism over policy, I-7):
| Model | Anchor | Coordination shape |
|-------|--------|--------------------|
| **Fork + journal** (default home case) | Federated Wiki | copy-with-provenance + per-page action journal (story = replay) |
| **VCS-replication + ping** | ikiwiki | git clone/pull/push + change-ping |
| **Query-time graph-join** | Wikibase SPARQL `SERVICE` | join remote graphs at query time, no copy |
| **Feed aggregation** | RSS/Atom | inbound feed → pages |
| **Activity streams** | ActivityPub | Create/Update events, notify or content-bearing |
| **Engine-mirror** | Wiki.js DB↔Git | engine syncs its own store to a git mirror |
### 8.4 Union & projection (L4) — the derived cache
This whole layer is **derived-disposable**: recomputable from canonical state — sharded
content + the **coordination-canonical** inputs in the journal (I-2). Crucially, the *automatic*
equivalence results are derived, but the **human/curatorial inputs they consume — alias tables
and curator equivalence bindings — are coordination-canonical (they live in the journal), not
derived**; recompute reads them, never regenerates them. It comprises:
- **Identity resolution & equivalence** — detect "same topic / derived content" path-
independently from *derived* signals (content fingerprint, span-set overlap) **plus** the
*coordination-canonical* inputs (alias table, curator binding); present as
**chorus-of-voices** or designated-canonical (a *policy* preset). (Scaling: §8.7.)
- **Union graph** — the navigable join of pages, links, and dimensions (namespace, genealogy,
version, shard, equivalence). A *derived lens over canonical files+journal, never a new
store* (the ZigZag boundary).
- **Transclusion** — one **reference-not-copy** primitive unifying Xanadu transclusion, ZigZag
clone, Roam/Obsidian/Logseq embed, Notion synced block, Trilium note-cloning, and literate
named-chunk assembly, over the addressable union.
- **Projection — trivial by default, extensible for the tail.** The 95% case (Markdown in a
shard) must cost nothing conceptually, so:
- **Default = plain lazy replication-projection** — a freshness-stamped cache of remote
content (§8.8). This is *the* projection for ordinary pages; it needs no taxonomy, no
liveness reasoning, no registry. Most shards never touch anything below.
- **Extension point — derivation-projection** — invoked *only* for content that is a
*source* needing transform/compile/weave/evaluate (computational/typed content, §8.5). It
adds the liveness axis (static → captured → live-over-files → view-time → irreducibly-live)
and facets (materialization timing, multiplicity, continuity); the irreducibly-live far end
has no faithful static form (source + a marked recording). A binding that never serves such
content never instantiates any of this.
- Both kinds stamp freshness + provenance; only derivation carries the liveness machinery.
- **Moldable view registry — also an extension point, not a tax on every page.** Where a content
type offers multiple co-equal views (typed/computed/dimensional content), they are registered
as an **open, type-keyed set, none canonical-by-fact** (display-canonical is policy; GT prior
art, answers the "pluggable content-type registry" question). An ordinary Markdown page has
exactly one view and never consults the registry — the registry is queried only when a type
declares >1 view.
- **Derived query index** — delegate to a shard's native query engine where present
(Roam/Logseq Datalog, Notion DB query, XWiki XWQL, Wikibase SPARQL); else build a derived
index over the projection (the Logseq DataScript-over-files pattern). The index is
disposable (I-2).
### 8.5 Computational / executable content — the scope decision
**In scope as a page-model + projection concern; out of scope as an execution platform.**
shard-wiki *recognises* computational types, attaches the **canonical source**, and presents
derived forms as **provenance- and liveness-marked projections**. Driving a derivation
(tangle/weave, re-execute a notebook, render a sketch, evaluate a pattern) is a **gated
capability, off by default, with a trust/sandbox concern, degrading to a captured snapshot**.
One snapshot-provenance record (run id, source rev, timestamp, environment "unguaranteed")
serves notebooks, renders, and recordings alike. **No INTENT amendment is required** — this
lives inside the existing page model (L2) and projection model (L4).
### 8.6 Consistency, concurrency & conflict model
INTENT makes real-time cross-shard consistency a non-goal — but "no strong consistency" is not
the same as "no defined consistency." This is the guarantee shard-wiki *does* offer, and the
mechanism (not policy) that makes concurrent editing safe (review bug B-2).
**The consistency guarantee — causal, anchored on the journal:**
- **Read-your-writes for coordination-canonical state.** Once an overlay/binding/merge is
appended to the space's decision log, every reader of that space sees it — because the log is
a **single totally-ordered sequence per space** (one append authority, §8.1), and all readers
resolve current state from that one order. The guarantee holds across orchestrator instances,
not just within one process; it is cheap because ordering is per-space, never global.
- **Causal consistency across the derived tier.** The union/index/projections reflect a causal
cut of `(sharded inputs seen so far, journal)`. Effects never appear before their causes; a
projection that has seen journal commit *C* has seen everything *C* depends on.
- **Eventual convergence for sharded-canonical inputs.** Remote shard content is pulled
asynchronously (lazily or by notify/poll, §8.7); the union converges to each shard's latest
*as observed*, bounded by the shard's operational envelope. Freshness is always *shown*
(provenance envelope), never faked — a stale projection is labelled stale, not wrong.
So: **strong + read-your-writes for what shard-wiki owns (the journal); causal for what it
derives; eventual + freshness-labelled for what shards own.** No global clock, no distributed
transaction, no two-phase commit across shards — none is needed, because shard-wiki coordinates
rather than controls.
**Conflict detection & representation is core mechanism; only resolution is policy (I-7).**
The split the earlier draft elided:
- **Detection (core).** Divergence is detected structurally: two identities resolve as
equivalent (§8.4) but their content fingerprints differ, or an overlay's base revision no
longer matches the shard's current revision. Detection is always on; it is never optional.
- **Representation (core).** A detected conflict is **first-class data in the union**, not an
error: equivalent-but-divergent pages are presented as a **coexisting set** (the
chorus/keep-both representation), each fully attributed, with the divergence recorded in the
provenance envelope (union without erasure — a conflict is information, not a failure).
- **Resolution (policy).** *Which* version wins, or whether they stay coexisting, is a
configurable preset (§10): chorus / designated-canonical / git-merge / vote-to-merge /
overlay-only. Core never hard-codes one.
**Overlay-apply under source drift (the concurrent-write case).** An overlay carries the
**base revision** of the shard content it was authored against. On apply, core compares base to
the shard's current revision:
- *unchanged* → apply (fast-forward), commit to journal, propagate if the profile permits;
- *changed, non-overlapping* → three-way merge where the merge capability allows (axis 6),
else keep-both;
- *changed, overlapping* → **refuse + re-present** as a conflict (above); never silently
clobber (I-5, no silent remote mutation). The unapplied overlay remains coordination-
canonical and valid against its base.
**Ordering.** The journal commit is the ordering authority for coordination-canonical effects;
a shard-native write is only *acknowledged* in the journal after the adapter confirms it, so a
crash between journal-intent and shard-write is recoverable (the intent is replayable, the
write is idempotent-keyed on identity+base-rev). Cross-shard operations are ordered by their
journal commits, giving the causal cut above.
**Residual open items** (tracked in *Known scaling risks & open problems*, §12, not pretended
solved): the exact convergence bound for
high-write CRDT shards under partition, and whether per-equivalence-set divergence needs a
vector clock vs. a simple base-rev comparison, are deferred to implementation spikes.
### 8.7 Scaling the union — incremental-first, rebuild as fallback
The derived tier is *recomputable* (I-2) but recompute must never be the **operational**
mechanism. A from-scratch rebuild reads every page of every shard — including rate-limited,
paginated external APIs (Notion) and irreducibly-live sources — which can take hours to days
and directly fights the operational-envelope axis (review C-2). So:
**Incremental, change-driven maintenance is the primary mechanism.** Each shard's `notify`
capability (or a poll/ETag fallback where it has none, §8.8) emits **change events**; an event
drives a **delta update** to exactly the affected union nodes, equivalence candidates, indexes,
and projections. The derived tier is a continuously-maintained materialised view, not a
periodically-recomputed one. Steady-state cost is O(changes), not O(corpus).
**Full rebuild is a rare, bounded fallback** — for cold start, schema/algorithm change, or
suspected corruption — and it is **explicitly not required to be cheap**. It respects each
shard's envelope (it may be slow, throttled, or resumable for a rate-limited shard) and runs
*concurrently with serving the existing derived tier*; it swaps in atomically on completion.
I-2 guarantees rebuild is *possible and correct*, not instant.
**Equivalence detection is indexed, not pairwise (review C-1).** Naive fingerprint/span-set
comparison across all pages of all shards is O(N²) and is forbidden. Instead:
1. **Blocking / candidate generation** — cheap keys bucket pages that *could* be equivalent:
normalised title, normalised path tail, explicit alias-table entries (coordination-
canonical), and **MinHash/LSH bands over content shingles** for near-duplicate and
derived-content detection. Only within-bucket pairs are considered — turning O(N²) into
≈O(N) candidates.
2. **Verification** — candidate pairs are confirmed by full fingerprint / span-set overlap and
any curator binding. Confirmed equivalences become union edges.
3. **Incremental maintenance — the delta is *not* additive (review B-4).** A changed page may
*leave* buckets as well as *enter* them, and leaving a bucket can **break an existing
equivalence edge** another page relied on. So a change is processed as: (i) recompute the
page's bucket membership; (ii) for buckets it **left**, re-verify the pairs that depended on
the shared bucket and **retract** edges no longer supported; (iii) for buckets it **entered**,
verify the new candidate pairs and **add** edges; (iv) **propagate** to the equivalence
neighbours of any retracted/added edge (equivalence is transitive-ish via chorus sets, so a
retraction can split a set). Maintenance is per-change and bounded by the page's
neighbourhood, but it covers retraction and propagation — not just additions.
**The index is itself derived** (disposable, recomputable) and per-tenant-partitioned (§9).
Its parameters (LSH band/row counts, shingle size, precision/recall) are tunable; the accepted
**false-negative rate of blocking** is a known, tracked limitation (§12) — blocking trades a
small miss rate for tractability, and curator bindings are the escape hatch for misses.
**Verifying I-2 (`derived = f(canonical)`) — eventually, not on faith (review B-4).**
Incremental maintenance can drift from a from-scratch fold over time (a missed retraction, a
dropped event, a bug). I-2 is therefore an **eventually-verified** property, not a free one,
and the architecture names the mechanism that verifies it:
- **A digest of the derived tier.** Each partition's derived tier carries a rolling content
digest (a Merkle-style hash over union nodes/edges/index entries) maintained alongside the
incremental updates.
- **A background consistency-checker** periodically recomputes the digest over a *sampled* (or,
on a slow cadence, full) fold of canonical state and compares. A mismatch localises the drift
to a partition/region and triggers a **scoped recompute** of just that region — cheap relative
to a global rebuild, and self-healing.
- **So I-2 holds *eventually and verifiably*:** the incremental engine is the fast path, the
checker is the guarantee, and divergence is detected and repaired rather than silently
accumulating. The exact sampling rate / digest granularity is an implementation spike (§12).
### 8.8 Cache freshness & invalidation
Replication-projection caches remote shard content; cache invalidation is the actual hard part
and was missing from the first draft (review C-2). The protocol is **per-binding, driven by the
capability profile**, with one rule: **freshness is always represented, never assumed** — every
cached page's provenance envelope carries `(observed-at, source-rev-if-known, staleness-state)`,
so a consumer can always tell live from stale.
**Three invalidation modes, chosen by capability, not hard-coded:**
| Mode | When | Mechanism |
|------|------|-----------|
| **Event-driven (push)** | shard has `notify` | a change event invalidates exactly the affected entries and enqueues a delta refresh (§8.7); the preferred mode |
| **Validator poll** | shard exposes ETag / Last-Modified / rev | conditional fetch (`If-None-Match`); cheap "still fresh?" checks without transferring bodies |
| **TTL** | shard offers neither | time-bounded staleness; the floor mode (Oddmuse-class shards) |
Most real bindings are **hybrid**: event-driven for invalidation + a long TTL as a safety net
for missed events + validator polls on read when an entry is past a soft age.
**Operational-envelope coupling.** The mode is constrained by axis-10: a **rate-limited** shard
(Notion) *must* favour event-driven + long TTL and *must not* poll per-read — the freshness
policy is capability-gated like everything else. A local file shard can watch the filesystem
(near-instant invalidation, effectively event-driven for free).
**Thundering-herd / coalescing.** Concurrent reads of the same stale entry trigger a **single
in-flight refresh** (single-flight); other readers await it or are served the stale-but-labelled
value per policy. Bulk invalidations (a shard-wide event) are **batched and rate-shaped** to the
shard's envelope rather than fired as N concurrent fetches.
**Staleness is a policy knob, not a correctness bug.** Whether a reader gets *stale-but-fast* or
*blocks-for-fresh* is a §10 preset (per space or per request); either way the envelope tells the
truth about what was served. This is union-without-erasure applied to time.
---
## 9. Cross-cut — Authorization (L5)
Fully specified in **`ArchitectureBlueprint.md`** (the access & history sub-blueprint);
summarised here for completeness:
- **One core, a ladder of modes** L0 (open/c2, zero deps) → L1 (attributed) → L2
(authenticated) → L3 (role/group) → L4 (multi-tenant enterprise). Climbing is configuration,
not re-architecture.
- **PEP** wraps every adapter op; **PDP** decides `(principal, action, target)` over actions
`read/write/patch/merge/administer`, layered on the adapter's capability profile (a shard
that can't write can't be written regardless of policy — L5 consults the L1 rail).
- **Authentication delegated** to a pluggable IdentityProvider (null provider = L0 default);
real identity from `user-engine` over `net-kingdom` IAM.
- **Fail open only at L0, fail closed at L2+.** Authorization is pure/offline once a Principal
is resolved. Provenance carries authz context so the union never leaks unreadable content
(the L5↔provenance-rail interaction).
### 9.1 Tenant isolation of the derived tier (review B-3)
Read-time authz filtering is necessary but **not sufficient** when the derived tier is
*persisted*: a single cross-tenant union/index cache guarded only by a filter on read is a
standing leak surface (one filtering bug exposes another tenant's content). So isolation is
**structural, not just procedural**:
- **The derived tier is partitioned per tenant / root entity.** A tenant maps to a root entity
(§4); its union graph, equivalence index, projections, and caches live in a **separate
partition** keyed by that tenant. There is no shared cross-tenant derived store to leak from.
- **No cross-tenant equivalence by default.** Blocking/LSH (§8.7) operates *within* a partition;
cross-tenant equivalence is an explicit, authorised, opt-in federation between roots, never an
accident of a shared index.
- **Read-time filtering remains, as defence-in-depth** — the provenance envelope's authz context
is still checked, so even within a partition a principal sees only what it may; partitioning
removes the *blast radius*, filtering removes the *fine-grained* leak.
- **This reconciles I-2 with L5:** recomputability (a persisted-but-disposable derived tier) is
preserved *per partition* — each tenant's derived tier is independently rebuildable from that
tenant's canonical state — so isolation costs nothing in the rebuild model. At L0/L1 (single
tenant) there is one partition and the machinery is invisible.
**Isolation invariant (add to §2 as I-13):** *derived state is partitioned by tenant; no
derived artifact spans tenants except through an explicit, authorised cross-root federation.*
---
## 10. The policy surface (mechanism over policy, made concrete)
I-7 only means something if the policy knobs are enumerated and kept *out* of core algorithms.
The configurable presets are:
- **Canonical-source policy** — chorus / designated-canonical / git-merge / overlay-only /
vote-to-merge (per space or per equivalence set).
- **Federation model** — the §8.3 taxonomy, per space, composable per shard.
- **Shard mode** — read-only / write-through / mirrored / projected / cached / canonical
(constrained by the capability profile).
- **Reconciliation cadence & conflict exposure** — push/poll/manual; show-conflicts vs
auto-merge-when-supported.
- **Conflict-resolution preset** — chorus / designated-canonical / git-merge / vote-to-merge /
overlay-only (the *resolution* policy over §8.6's core detection; per space or equivalence set).
- **Freshness / invalidation mode** — event-driven / validator-poll / TTL / hybrid, and
stale-but-fast vs block-for-fresh on read (§8.8; constrained by the operational envelope).
- **History compaction** — squash policy for low-value churn, gc/repack cadence, per-shard
offload (§8.1), always preserving recoverable endpoints.
- **Tenant partition mapping** — tenant ↔ root-entity, and any explicit cross-root federation
(§9.1, I-13).
- **Execution policy** — derive/execute off (default) / sandboxed / per-shard-allowed.
- **Authorization mode** — the L0L4 ladder.
- **Projection materialization** — lazy/eager; snapshot vs view-time; recording retention.
Core ships sane defaults (L0 open; fork+journal; lazy replication-projection; event-driven+TTL
freshness; overlay-before-mutation; execution off; one tenant = one root) and never hard-codes
any of the above. (**Preset bundles** that package coherent knob-sets per persona are tracked
as O-8, §12 — flexibility without bundles is operator burden.)
---
## 11. Concrete module structure (bridge to implementation)
A proposed package layout for `src/shard_wiki/`, mapping 1:1 to the layers so the dependency
rule (downward only; the derived tier is incrementally maintained, rebuild = fallback) is
enforceable by import lint:
```
src/shard_wiki/
model/ # L2 top waist: Page, Identity, Placement, ProvenanceEnvelope,
# Span, the page-shape types; capability-spectrum value types
adapters/ # L1 bottom waist: AdapterContract (versioned iface), CapabilityProfile,
# attachment-mode binding; concrete adapters:
git/ folder/ gitea/ obsidian/ webdav/ notion/ … # each: profile + verbs
coordination/ # L3: DecisionLog (append-only, git-backed, per-space append authority/
# lease), OverlayEngine (draft→patch→MR), reconcile
# (current coordination state = a derived fold → lives in union/)
federation/ # L3: FederationModel strategies (fork_journal, vcs_ping,
# graph_join, feed, activitypub, engine_mirror)
union/ # L4 (derived): IdentityResolver, EquivalenceGraph, UnionGraph,
# Transclusion (reference-not-copy)
projection/ # L4 (derived): ReplicationProjection, DerivationProjection,
# ViewRegistry (moldable), QueryIndex (delegate|derive)
authz/ # L5 cross-cut: PDP, PEP, IdentityProvider iface, NullProvider
provenance/ # cross-cut LEAF: ProvenanceEnvelope type + ⊕ (effective) only — pure data
policy/ # cross-cut LEAF: the §10 policy surface (presets + a resolve() read by
# coordination/federation/projection/authz); owns NO mechanism
api/ # L6: orchestrator API (server-side union for agents/CLI)
```
**The cross-cutting rails are leaves, not god-modules (review D-4).** `provenance/` and
`policy/` are imported widely, so they are the highest coupling risk; the discipline that caps
it is: **they may import *nothing* in the tree and contain *only* stable data types + pure
functions** (the envelope and its ``; the policy presets and a `resolve(question) → choice`).
Mechanism never lives in a rail — `policy/` says *what* the preset is, `coordination/`/
`projection/` decide *how* to honour it. A change to a rail is then a change to a small, stable,
dependency-free leaf, not a ripple through every layer. Capability-spectrum value types live in
`model/` (also leaf-like) for the same reason.
Hard import rules (enforced by import lint):
- `union/` and `projection/` may import `model/`, `adapters/`, `coordination/`, `policy/`,
`provenance/` — but **nothing may import them** (they are the disposable derived tier).
- `model/`, `adapters/`, `provenance/`, `policy/` import nothing else in the tree (the waists
and rails stay thin); `provenance/` and `policy/` import nothing at all.
- `coordination/` and `federation/` may import the waists + rails, never the derived tier.
---
## 12. Known scaling risks & open problems
Tracked honestly rather than pretend-solved (review disposition F). Each has a **chosen
direction** and a **revisit trigger** — the thing that, if observed, forces a redesign.
| # | Risk / open problem | Chosen direction | Revisit trigger |
|---|---------------------|------------------|-----------------|
| O-1 | **Equivalence blocking misses true matches** (LSH false negatives, §8.7) | accept a small miss rate; curator bindings are the escape hatch | measured recall below an agreed threshold on real corpora |
| O-2 | **Convergence bound for high-write CRDT shards under partition** (§8.6) | causal via journal + CRDT-native merge at the shard; no global bound promised | user-visible divergence that outlives a partition |
| O-3 | **Per-equivalence-set divergence tracking** (§8.6) | start with base-rev comparison; add vector clocks only if needed | 3-way concurrent divergence that base-rev mis-orders |
| O-4 | **Persisted derived-tier cost ceiling** (§8.7/§9.1) | per-tenant partition, incremental-maintained, rebuild is fallback | a tenant whose incremental cost still exceeds budget |
| O-5 | **Axis-interaction completeness** (§6.5) | the named interaction table is the contract; extend deliberately | a real adapter needing an interaction not in the table |
| O-6 | **Span-address portability across projection** (§7.2) | shard-scoped native-id wrapping now; tumbler later | cross-shard transclusion that native ids can't satisfy |
| O-7 | **Squash-compaction vs. perfect auditability** (§8.1) | compact the *path*, preserve reachable states; configurable | a compliance need for every intermediate keystroke |
| O-8 | **Policy-knob proliferation → operator burden** (§10) | ship named **preset bundles** ("personal vault" / "team wiki" / "enterprise federation") over the policy surface | operators mis-configuring interacting knobs |
| O-9 | **Shard sharing across roots vs tenant partition** (§9.1, I-13) | shard exclusive to one root by default; explicit shared-read binding otherwise (avoids double-caching a rate-limited shard) | a shard legitimately needed live in two tenants |
| O-10 | **Span-level authz under transclusion** (aggregation/inference leak; ⊕ across boundaries, §7.3/§9) | a transcluded span inherits the **stricter** of source & host authz; provenance ⊕ composes the source-page envelope under the host | a real cross-authz transclusion |
| O-11 | **Union under shard unavailability** (§8.8 covers stale, not down) | **partial union** + per-shard "unavailable" provenance + last-known-projection where policy allows | an SLA need on partial reads |
| O-12 | **Per-space append-log throughput ceiling** (§8.1 append authority) | single writer per space scales across spaces; per-space log sharding if needed | a single space exceeding one writer's append rate |
These are the spec-writing inputs for `SHARD-WP-0002`; none blocks the architecture, each
scopes an implementation spike.
---
## 13. Canonical data flows (the architecture exercised)
**A. Attach a shard.** Adapter binds (chosen attachment mode) → probes/declares a capability
profile → core registers the shard under a root entity → if not git-native, the coordination
journal is seeded (begin-now/mirror/import per axis 5). No union rebuild yet (lazy).
**B. Read a page through the union.** Consumer asks the union for an identity → Identity
resolver maps it to placements across shards → equivalence yields chorus or canonical →
replication-projection lazily fetches from each shard (cache + freshness) → page returned
wrapped in its provenance envelope → L5 filters anything the principal can't see at source.
**C. Edit a read-only / limited shard.** Write request → L5 PDP allows → capability profile
says < write-through → OverlayEngine records a draft → renders a patch/MR in the shard's native
syntax (lossless) or Markdown (lossy-with-report) → on explicit apply, commit to the journal
and (if the profile permits) propagate; otherwise the overlay stands as the local truth, fully
attributed.
**D. Attach a computational notebook.** Adapter declares profile (attachment=file-store,
opacity=mixed, computational=captured-output). Core attaches the `.ipynb` **source** as
canonical; presents cells + embedded outputs as **derivation-projection snapshots** marked
"run N, env unguaranteed"; offers a static render via the view registry; re-execution stays
gated off. History uses paired-text/nbdime per axis 5.
---
## 14. Key tradeoffs & decisions
Decided:
- **Capability spectra over a verb checklist** — richer contract for precise, uniform
degradation; tamed by an orthogonal core + implied positions + a named interaction table
(§6.5). (Decided.)
- **Three states; derived = f(canonical)** — sharded + coordination canonical, derived
disposable (§1). (Decided; supersedes the earlier "edges vs middle" framing.)
- **Event-sourced coordination, one append authority per space** — coordination-canonical state
is an append-only **decision log** in the git journal; current state is a derived fold; a
per-space append lease gives a totally-ordered log and read-your-writes across orchestrator
instances (§8.1). (Decided — resolves the single-vs-multi-writer keystone.)
- **Profiles are verified, not asserted** — a versioned **conformance suite** gates adapter
admission; capability-as-data acts on verified data (§6.6). (Decided.)
- **I-2 is eventually-verified** — incremental maintenance is the fast path; a digest +
background consistency-checker detects and self-heals drift (§8.7). (Decided.)
- **Incremental-first, rebuild-as-fallback** — the derived tier is continuously maintained from
change events; full rebuild is rare and need not be cheap (§8.7). (Decided — resolves the
earlier "union graph persistence" open item: **persisted, per-tenant, incrementally
maintained, rebuildable**, §9.1.)
- **Identity ≠ fingerprint** — page identity is a stable handle; fingerprints are for
equivalence (§7.2). (Decided.)
- **Consistency = read-your-writes (journal) + causal (derived) + eventual/freshness-labelled
(shards)**; conflict detection/representation is core, resolution is policy (§8.6). (Decided.)
- **Address scheme** — shard-scoped native-id wrapping now; portable tumbler later (§7.2, O-6).
(Decided.)
- **Default federation = fork+journal over Git**; other models opt-in (§8.3). (Decided.)
- **Execution off by default** — recognise+project always; execute only when gated (§8.5). (Decided.)
- **Derived tier is tenant-partitioned** (I-13, §9.1). (Decided.)
Still open (carried to §12 / policy):
1. **L1 "attributed-but-open" mode** — ship it or jump L0→L2? (Carried from ArchitectureBlueprint.)
2. **Per-page ACL default** — off (per-shard/namespace) confirmed; revisit only if demand appears.
3. The implementation spikes in **§12** (O-1…O-7).
---
## 15. What this architecture is *not*
- Not a wiki engine, UI, or rendering pipeline (those are consumers at L6).
- Not a canonical-source-of-truth — shards keep sovereignty; the middle is derived.
- Not a generic file-sync daemon — synchronisation is wiki-page-semantic.
- Not an execution platform — computation is recognised and projected, not hosted.
- Not a universal ontology — no single schema is imposed on all shards.
- Not an authentication/identity store — that is delegated (authorization is owned).
---
## 16. Traceability
- **INTENT** — every invariant in §2 (I-1…I-13) cites an INTENT principle or boundary; no
invariant contradicts the Stability Note.
- **Review & hardening** — this revision folds in
`history/260615-core-architecture-blueprint-review.md` via **`SHARD-WP-0005`**: A-1→§1/§3/§4
(three-state re-frame), B-1→§7.2 (identity vs equivalence), B-2→§8.6 (consistency/conflict),
B-3→§9.1+I-13 (tenant isolation), C-1/C-2→§8.7/§8.8 (incremental + indexed + invalidation),
C-3→§8.1 (history scaling), D-1→§6.5 (orthogonal core), D-2→§7.3 (layered provenance),
D-3→§8.4 (common-case projection), D-4→§11 (policy module + rail discipline); open items→§12.
- **Round-2 review & hardening II** — folds in
`history/260615-core-architecture-blueprint-review-2.md` via **`SHARD-WP-0006`**:
A-1…A-4→§3/§4/§10/§11 (overview reconciled to the body), B-1+B-3→§8.1 (event-sourced
coordination + per-space append authority), B-2→§6.6 (adapter conformance suite),
B-4→§8.7 (incremental retraction/propagation + I-2 digest/checker); C-1…C-4→§12 (O-8…O-11).
- **Research** — §6 (spectra) ← `260614-shard-spectrum-synthesis` v3; §8.3 (federation
taxonomy) ← v3 §2.5; §8.4§8.5 (two-axis projection, view registry, computational scope) ←
`260614-computational-page-model-synthesis`; §7 page shapes ← the engine + modern-tool +
computational dives; §1 thesis ← the files-canonical/index-derived through-line across
Logseq/ikiwiki/GT/Pharo/Jupyter.
- **Use cases** — the architecture is sized to UC-01UC-84: federation/coordination (UC-0107,
2633, 56, 7172, 79) → §8; attachment/adapter (UC-3443, 50, 53, 57, 6062, 6466, 6870,
7682) → §6; page model & fidelity (UC-34, 39, 42, 55, 5859, 67, 73, 80, 8384) → §7/§8.5;
addressing/identity/query (UC-32, 4449, 5152, 54, 63, 74) → §7.2/§8.4; provenance &
metadata (UC-2425, 75) → the provenance rail; collaboration & discovery (UC-0823) → L6
consumers over the union.
- **Workplans** — §6§8 are the design target of `SHARD-WP-0002` (T11T18); §9 is owned by
`ArchitectureBlueprint.md`; §1 (yawex-derived resolution/overlay) aligns with
`SHARD-WP-0001`.
---
## 17. Stability note
This document defines shard-wiki's **internal** architecture; it may evolve as the spec
workplans land. But the **thesis (§1)**, the **invariants (§2)**, and the **dual narrow waist
(§1, §6, §7)** are load-bearing — changing any of them is an architectural change in the sense
of INTENT's Stability Note and should be rare and deliberate.

View File

@@ -0,0 +1,220 @@
# FederationArchitecture
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0002** (T1T10, T17)
The federation **design decisions** for shard-wiki: *what the union does*. It records, per
topic, a decision with rationale, tradeoffs, and a **Decided / Deferred / Open** footer. It
**references** `spec/CoreArchitectureBlueprint.md` (the whole-system architecture) and
`spec/FederationRequirements.md` (yawex-derived ADRs) rather than restating them; the adapter
contract (*what a backend must expose*) is the companion deliverable in
`spec/TechnicalSpecificationDocument.md` §A (T11T16, T18). UC references → `UseCaseCatalog.md`.
Cross-cutting invariants assumed throughout (blueprint §2): orchestrator-not-engine (I-1),
three-state canonical/derived (I-2), capability-as-data (I-3), union-without-erasure (I-4),
overlay-before-mutation (I-5), git-addressable coordination (I-6), mechanism-over-policy (I-7),
graceful degradation (I-8), identity≠placement (I-9), history-as-floor (I-10), authz-in-core
(I-11), tenant-partitioned derived state (I-13).
---
## T1 — Orchestrator positioning & boundaries
**Decision.** shard-wiki is an **adapter-layer orchestrator** in a **star** shape (many
sovereign shards ↔ one information space), **not** a homogeneous federation network. It
**compares, does not equate** (Federated Wiki homogeneous JSON sites, ikiwiki homogeneous git
wikis, ActivityPub activity streams are *models it can speak*, §T17 — not shapes it imposes).
Core owns: the union, the coordination journal, projection, overlay, resolution, authorization.
Adapters own backend specifics; UI and editorial policy live outside core. (Blueprint §1, §5;
UC-02, UC-26.)
*Decided:* star orchestrator; compare-not-equate; core/adapter/UI/policy boundaries.
*Deferred:* the reference UI (L6) is out of scope here. *Open:* none.
## T2 — Remix primitives: reference / projection / overlay / import-fork
**Decision.** Four primitives, escalating in commitment; **overlay-before-mutation is the
default write path** (I-5), fork is *one federation model* (§T17) not the default for editing:
| Primitive | Trigger | Writes remote? | Coordination-canonical? |
|-----------|---------|----------------|--------------------------|
| **Reference** | link only | no | no (a link in content) |
| **Projection** | read remote page | no (cache) | no (derived) |
| **Overlay** | edit a sub-write-through shard | not until explicit apply | **yes** (decision log) |
| **Import / fork** | copy into a writable shard / fork-with-provenance | source unchanged | **yes** (fork event) |
(Blueprint §8.2; FederationRequirements ADR-05; UC-04, UC-26, UC-29.)
*Decided:* the four primitives + overlay-default. *Deferred:* fork-attribution portability
format. *Open:* whether "import" and "fork" are one primitive with a flag or two (impl spike).
## T3 — Equivalent-page identity & multi-version presentation
**Decision.** Separate **identity** (stable handle), **placement** (N paths/shards), and
**equivalence** (content-fingerprint / span-set overlap *across* identities), per
FederationRequirements ADR-01/ADR-02 (I-9). Equivalence signals: normalized title/path, alias
table (coordination-canonical), link-graph overlap, fingerprint, **curator binding**. Default
presentation = **chorus-of-voices** (all equivalent versions, attributed); **designated-
canonical** is a policy preset (§T9). Divergence is data in the provenance envelope (I-4),
detected as core mechanism (blueprint §8.6). (UC-07, UC-27; UC-46.)
*Decided:* identity/placement/equivalence split; chorus default. *Deferred:* fingerprint
algorithm + blocking params (blueprint O-1). *Open:* curator-binding UX (L6).
## T4 — History, attribution & the coordination journal
**Decision.** Each information space has a **git-addressable, event-sourced coordination
journal** (blueprint §8.1, I-6/I-10): coordination-canonical decisions (overlay/binding/alias/
merge/fork) are append-only events; current state is a derived fold. **One append authority per
space** gives a total order (read-your-writes across instances). Per-shard native history is
**adopted** (git-native), **supplemented** (DB/CRDT internal → journal begins-now/mirrors), or
**imported** (open file history backfilled) per the history-portability axis (TSD §A T13).
Attribution is portable on the event (UC-29). (UC-29, UC-33.)
*Decided:* event-sourced journal + per-space append authority + adopt/supplement/import.
*Deferred:* per-space log sharding for extreme write rates (blueprint O-12). *Open:* none.
## T5 — Union composition layer
**Decision.** The **server-side orchestrator union is primary** — the derived tier (union
graph, indexes, projections) is composed in core for agents/CLI/non-browser consumers
(blueprint §8.4), **incrementally maintained** (not recomputed, §8.7), **tenant-partitioned**
(I-13). **Client-side composition** (Federated-Wiki-style browser pull) is a supported
*consumer pattern over the same union*, not the only path. Freshness/caching per §8.8. (UC-05,
UC-27, UC-03, UC-31.)
*Decided:* server-primary, client-optional; incremental, partitioned. *Deferred:* client
SDK shape. *Open:* persisted-union digest granularity (blueprint O-4, B-4 checker).
## T6 — Change notification & subscription transports
**Decision.** Change-notification is an **optional adapter capability** (`notify`), and it is
the **primary driver of incremental maintenance** (blueprint §8.7/§8.8). Transports, per
profile: git hook / ikiwiki-style ping, ActivityPub Create/Update, webhook, WebDAV/HTTP ETag
or `Last-Modified` poll, plain polling fallback. A notification → invalidate affected entries
+ enqueue delta refresh + a RecentChanges union entry (FederationRequirements ADR-03; UC-31,
UC-17). Rate-limited shards favour event-driven + long TTL (operational-envelope axis).
*Decided:* notify = optional capability, drives incremental + RecentChanges; transport per
profile. *Deferred:* ActivityPub as content-bearing (vs notify-only) — start notify-only.
*Open:* fediverse-dependency posture for v1.
## T7 — Information-space lifecycle
**Decision.** Root entities have lifecycle states: **active → read-only-archived → retired
(detached)**, and **merged-into-successor**. **Carry-forward** is selective import from an
archived shard (UC-28, UC-30). **Space-fork / branch** (UC-33) = branch the coordination
journal + shard-config (a fork event, §T2/§T17 fork+journal model). Retirement **preserves
read-only union entries** by default (history-as-floor, I-10), never hard-deletes projections.
(UC-28, UC-30, UC-33.)
*Decided:* the lifecycle states + carry-forward + space-branch; retire-preserves. *Deferred:*
ephemeral "happenings" as a first-class mode. *Open:* GC policy for retired-space derived tier.
## T8 — Transclusion & projection depth
**Decision.** Transclusion is **one reference-not-copy primitive** over the addressable union
(blueprint §8.4), at three depths:
| Level | UC | Behaviour |
|-------|-----|-----------|
| Whole-page projection | UC-03 | lazy full page from a remote shard (replication-projection) |
| Block/span transclusion | UC-32 | inline embed by span address, with origin + freshness |
| Link reference | UC-08 | pointer only (ADR-06) |
Every transcluded artifact carries provenance + **staleness** (I-4, §8.8); live-transclusion
fragility (link rot / remote down) degrades to last-known + an `unavailable`/`stale` marker
(blueprint O-11). Span-level authz under transclusion = the stricter of source/host (O-10).
Reference-not-copy unifies Xanadu transclusion, ZigZag clone, embeds, synced blocks, Trilium
note-clone, literate named-chunk. (UC-03, UC-32.)
*Decided:* one primitive, three depths, provenance+staleness mandatory. *Deferred:* snapshot-
on-import vs live default (a policy). *Open:* span-authz composition (O-10), addressing (O-6).
## T9 — Consensus & reconciliation policy catalog
**Decision.** **Detection is core, resolution is policy** (I-7, blueprint §8.6). Core always
detects divergence and represents it as a coexisting (chorus) set; the **resolution preset** is
configurable per space / per equivalence set:
- **chorus-spread** (versions coexist; default) · **designated-canonical** (explicit write
target) · **git-merge** (where merge capability allows) · **vote-to-merge / editorial gate**
(optional, L6-assisted) · **overlay-only** (no destructive merge on read-only sources).
(FederationRequirements ADR-03/ADR-05; UC-07, UC-27.)
*Decided:* the preset catalog; detection-core/resolution-policy. *Deferred:* vote/editorial
gate mechanics (L6). *Open:* default preset per persona bundle (blueprint O-8).
## T10 — Federation-operations capability matrix
**Decision.** Each federation operation requires a minimum **verified capability profile**
(TSD §A T11/§6.6); below it, the op **degrades** along the named interaction axes (blueprint
§6.5) rather than failing. The matrix (consuming the T11 vocabulary):
| Federation op | Min capabilities | Degradation when absent |
|---------------|------------------|--------------------------|
| **read / project** | `read` | (floor — every shard) |
| **transclude span** | `read` + addressing≥span | whole-page projection only |
| **overlay** | `read` | always available (overlay is local) |
| **write-through** | `write` (+ `merge` for concurrent) | → overlay/patch target |
| **diff / 3-way merge** | `diff`,`merge` | → keep-both / chorus |
| **notify-driven freshness** | `notify` | → validator-poll → TTL (§8.8) |
| **native search delegate** | native-query≥index | → derived index over projection |
| **publish / mirror** | `publish` | → read-only / static projection |
| **fork+journal federation** | `read` + history (any) | → projection-only participant |
Every backend, down to the Oddmuse flat-file floor, is usable as **read-only / cache /
projection / backup / patch target** (I-8). This matrix and the T11 spectra are kept in sync
(it consumes that vocabulary). (UC-02UC-07.)
*Decided:* the op→capability matrix + degradation paths. *Deferred:* exhaustive per-op test
vectors → the conformance suite (§6.6). *Open:* axis-interaction completeness (blueprint O-5).
---
## T17 — Federation-model taxonomy (selectable / composable)
**Decision.** Federation is **plural and composable**, not one mechanism (blueprint §8.3). A
space selects a model (composing per shard); the default is **fork+journal over Git** (the home
case):
| Model | Anchor | Coordination shape | Capability floor | UCs |
|-------|--------|--------------------|------------------|-----|
| **Fork + journal** *(default)* | Federated Wiki | copy-with-provenance + per-page action journal (story = replay) | `read` + journal | UC-26, UC-71, UC-72 |
| **VCS-replication + ping** | ikiwiki | git clone/pull/push + change-ping | git-IS-store | UC-31, UC-33, UC-79 |
| **Query-time graph-join** | Wikibase `SERVICE` | join remote graphs at query, no copy | native-query≥graph | UC-74 |
| **Feed aggregation** | RSS/Atom | inbound feed → pages | `read` | UC-03 |
| **Activity streams** | ActivityPub | Create/Update events (notify or content) | `notify` | UC-31 |
| **Engine-mirror** | Wiki.js DB↔Git | engine mirrors its store to git | `publish`/mirror | UC-68, UC-69 |
The coordination journal (T4) is the fork+journal model's home; **git IS the journal** in
VCS-replication and engine-mirror (forge wikis make git the store *and* journal, resolving the
engine-mirror write-race). Models compose: e.g. fork+journal locally, query-join for a
typed-graph shard, feed-aggregation for an external source. (Mechanism over policy, I-7.)
*Decided:* the six models, selectable per space + composable per shard, default fork+journal.
*Deferred:* activity-streams content-bearing mode. *Open:* multi-model interaction edge cases
(when two models touch one shard).
---
## Consolidated decisions / deferred / open
- **Decided (architecture-settling):** star orchestrator (T1); overlay-default remix (T2);
identity/placement/equivalence + chorus (T3); event-sourced journal + append authority (T4);
server-primary incremental union (T5); notify-driven freshness (T6); space lifecycle (T7);
reference-not-copy transclusion (T8); detection-core/resolution-policy preset catalog (T9);
op→capability matrix (T10); selectable/composable federation models (T17).
- **Deferred (to impl / L6 / persona bundles):** reference UI; fork-attribution format; client
SDK; vote/editorial mechanics; ephemeral spaces; ActivityPub content-bearing; default-preset
bundles (O-8).
- **Open (tracked in blueprint §12):** O-1 equivalence blocking, O-4 union digest, O-5 axis
interactions, O-6 addressing, O-8 presets, O-10 span-authz, O-11 unavailability, O-12 log
throughput.
## Acceptance (SHARD-WP-0002 federation half)
T1T10 + T17 each have a decision record with a Decided/Deferred/Open footer; all honour INTENT;
UC-26UC-33, UC-56, UC-71UC-72, UC-74, UC-79 trace here; the adapter-contract companion is
`spec/TechnicalSpecificationDocument.md` §A (T11T16, T18). Conflicts with SHARD-WP-0001 are
none (FederationRequirements ADRs are consumed, not duplicated).

View File

@@ -0,0 +1,214 @@
# FederationRequirements — yawex-derived design notes
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0001**
Concrete, ADR-ready design decisions for the union/federation layer, distilled from the yawex
prior-art review (`research/260608-yawex-prior-art/findings.md`) and made **consistent with**
`spec/CoreArchitectureBlueprint.md` (the whole-system architecture) and `INTENT.md`. yawex is
*inspiration and a case checklist*, never an interface to inherit (decision 2026-06-08).
These are **requirements** (what the union must do); `spec/FederationArchitecture.md`
(SHARD-WP-0002) is the architecture that realises them. UC references are to
`spec/UseCaseCatalog.md`.
**Resolved precondition.** The yawex "minimal access model in core" thread is **settled**:
authorization-in-core / authentication-delegated, the L0L4 ladder — INTENT amended,
`ArchitectureBlueprint.md` ratified. Auth is therefore *not* re-litigated here; these ADRs
assume it (the resolver and overlay paths run behind the L5 PEP/PDP).
---
## ADR-01 — Cross-shard page resolution
**Status:** accepted · **CoreArchitectureBlueprint:** §8.4 (union), §7.2 (identity)
**Context.** yawex's most-tested component was `PageLookUp`, a context-relative resolver with
ordered states `LOCAL, CLIMB, DROP, GLOBAL, REMOTE, SWITCH, JUMP, VIRTUAL, FAILED`. Decision
(2026-06-08): **inspiration only** — use the states as a *checklist of cases*, design the
federation resolver fresh.
**Decision.** Resolution is a pure function over the derived union:
`resolve(name, from_context, policy) → ResolutionResult`, evaluated in a defined order and
keyed on **page identity** (the stable handle), not path (I-9):
1. **Same-namespace, same-shard** exact identity match — *(yawex LOCAL)*.
2. **Namespace walk** — climb/descend ancestor namespaces of `from_context`*(CLIMB/DROP)*.
3. **Union lookup** — match identity / alias-table entry across all attached shards — *(GLOBAL)*.
4. **Equivalence set** — if several shards hold an equivalent page, return the **chorus set**;
the *canonical-source* policy preset (chorus / designated-canonical) decides presentation —
never silently pick one (union without erasure, I-4). *(implicit in yawex; explicit here.)*
5. **Projection / virtual** — a page whose content lives in a remote shard (lazy replication,
*REMOTE*) or is computed/query-defined (derivation, *VIRTUAL*).
6. **Explicit address** — a shard- or space-qualified reference resolves directly — *(SWITCH/JUMP)*.
7. **Not found****red-link**: a resolvable *target* with no page yet, createable (UC-23) —
*(FAILED)*.
`ResolutionResult` carries the resolved identity (or red-link), the placement(s), the
provenance envelope (ADR-04), and — on ambiguity — the full chorus set. Resolution is a
**derived-tier read** (union graph + equivalence index, §8.4); it is incremental-maintained and
respects the L5 authz filter (a principal never resolves to content it can't see).
**Consequences.** Deterministic, policy-driven, no privileged shard; every yawex state has a
home without inheriting its structure. **Open:** cross-space `JUMP` addressing syntax ties to
the portable-address scheme (CoreArchitectureBlueprint O-6).
---
## ADR-02 — Namespace / path model & shard roles
**Status:** accepted · **CoreArchitectureBlueprint:** §7.2 (identity≠placement), §4 (shard mode)
**Context.** yawex modelled **topics as directories** (a topic's gateway page shares the dir
name) with relative (`../`) / absolute (`/`) paths and normalization, and page classes
`local / global / virtual`.
**Decision.**
- The union exposes a **namespace tree**; a **path is a placement coordinate, not an identity**
(I-9). One identity may have **N placements** (paths/shards) → a DAG, no single canonical path.
- **Path grammar:** relative (`../`, `./`) resolved against `from_context`; absolute (`/`)
against the space root; **normalization** (collapse `.`/`..`; case & separator handling) is a
per-space **policy** knob (§10) — defaulting to case-preserving, `/`-separated.
- **yawex page classes → shard roles / modes:** `local → canonical`, `global → cross-cutting`
(a shard whose pages augment/overlay across namespaces), `virtual → projected/computed`. These
are the INTENT shard modes (read-only · write-through · mirrored · projected · cached ·
canonical), selected by policy, constrained by the capability profile.
- Cross-shard name collisions are resolved by ADR-01's order + equivalence, not by a global
flat namespace.
**Consequences.** The namespace is a navigable view over placements (a dimension of the union),
not a storage layout. **Open:** per-space normalization policy defaults (case-sensitivity,
unicode) — a policy preset, bundled per persona (O-8).
---
## ADR-03 — Union-level derived views: core vs adapter
**Status:** accepted · **CoreArchitectureBlueprint:** §8.4 (derived query index), §8.7
(incremental) · **resolves findings open-Q #3**
**Context.** yawex shipped BackLinks, RecentChanges, AllPages, SiteMap, full-text Search.
**Decision.** Each is a **derived-tier** view (rebuildable, incrementally maintained); classify
by where it is computed:
| View | Tier | Rationale |
|------|------|-----------|
| **BackLinks** | **core** | the link-graph over the union — squarely shard-wiki's concern (UC-18); the strongest core view |
| **RecentChanges** | **core** | merge the coordination journal (§8.1) with shard change events (notify/poll, §8.8) across the union (UC-17) |
| **AllPages / SiteMap** | **core** | an enumeration/projection of the union graph; cheap |
| **Search (full-text)** | **hybrid** | **delegate** to a shard's native search where the native-query axis allows; **else** build a derived index over projections (the Logseq DataScript-over-files pattern, UC-19/UC-63) |
All are computed in the derived tier and carry provenance; **presentation is L6 (UI)**, never
hard-coded in core (mechanism over policy, I-7).
**Consequences.** BackLinks/RecentChanges/AllPages/SiteMap are core deliverables; Search is a
capability-gated delegate-or-derive. **Open:** cross-shard search ranking/relevance is a policy
concern, not core mechanism.
---
## ADR-04 — Provenance & freshness model (concrete fields)
**Status:** accepted · **CoreArchitectureBlueprint:** §7.3 (layered provenance), §8.8 (freshness)
**Context.** yawex `Page::info` exposed modtime (and TODO'd last-editor / hits / edits). INTENT
mandates explicit provenance & freshness (I-4).
**Decision.** Concretise the **provenance envelope** (layered: page-level + span-level deltas):
```
ProvenanceEnvelope (page-level):
source_shard # which shard this came from
source_rev? # shard-native revision id, if the shard exposes one
observed_at # when shard-wiki last read it
liveness # static | captured-snapshot | live-over-files | view-time | irreducibly-live
staleness_state # live | fresh | stale | unavailable
authz_context # the L5 context under which it was read (no-leak, §9)
overlay_state # none | draft | patch-pending | applied
divergence[] # equivalence-set peers that differ (the chorus, ADR-01 step 4)
derivation_lineage? # for derived/derivation-projection content (source → this view)
Span-level: only the fields that differ from the page envelope (effective = page ⊕ delta).
```
`freshness = (observed_at, source_rev?, staleness_state)`; `unavailable` is the dead-shard
state (feeds ADR-03 RecentChanges and the union-under-unavailability open item O-11). This
envelope is the data behind union-without-erasure and the input to conflict display.
**Consequences.** Provenance is queryable, layered (cheap per-span), and drives views/conflict
UI. **Open / DROP:** hits/edits *analytics* are an L6/analytics concern, out of core (dropped
from the yawex feature set).
---
## ADR-05 — Overlay / lightweight-patch model
**Status:** accepted · **CoreArchitectureBlueprint:** §8.2 (overlay engine), §8.1 (decision
log), §8.6 (apply-under-drift)
**Context.** yawex had an append + threaded-comment workflow (edit a page without a full
rewrite). INTENT mandates overlay-before-mutation (I-5).
**Decision.** The overlay lifecycle for any shard below write-through capability:
1. **Draft** — an edit becomes a draft recorded as a **coordination-canonical event** in the
space's decision log (§8.1); the unapplied overlay is the **local truth**, fully attributed
(`overlay_state`, ADR-04).
2. **Patch / MR** — rendered in the shard's **native syntax (lossless)** or **Markdown
(lossy-with-fidelity-report)** per the translation axis.
3. **Apply** — only on explicit intent, only where profile + policy permit, with
**apply-under-drift** semantics (§8.6: fast-forward / three-way / refuse-and-re-present).
4. **Append / comment** = a **constrained overlay subtype** (purely additive, low-conflict) —
the direct generalisation of yawex's workflow; safe even on the most limited shards.
This makes read-only, rate-limited, and lossy shards **editable** without silent remote
mutation (graceful degradation, I-8; no silent mutation).
**Consequences.** One overlay mechanism spans drafts, patches, MRs, and comments; storage is the
decision log (no separate store). **Open:** whether threaded-comment threading is a first-class
overlay subtype or a generic structured-append — defer to implementation.
---
## ADR-06 — Markdown link semantics (wikilink + red-link)
**Status:** accepted · **CoreArchitectureBlueprint:** §7 (page model), ADR-01 (resolution) ·
**resolves findings open-Q #2**
**Context.** yawex had CamelCase WikiLinks, `[[free links]]`, red-`?` links for nonexistent
pages, `::` labels. INTENT mandates Markdown-first; TRANSFORM the *semantics*, drop the bespoke
syntax.
**Decision.**
- Adopt a **CommonMark wikilink extension**: `[[Target]]` / `[[Target|label]]`, resolved via
ADR-01 (by identity/name across the union).
- **Red-link** = a wikilink whose target resolves to FAILED (ADR-01 step 7): a valid,
*createable* target with no page yet (the soft-create affordance, UC-23).
- **CamelCase auto-linking is OFF by default** (a legacy affordance); opt-in **per space** for
migrating CamelCase wikis (UseModWiki/c2 lineage, UC-25).
- Links are stored **as text** (git-diffable; structure/links federate iff in-text).
- **Boundary (resolves open-Q #2):** the link **model + resolution is core**; the **visual
rendering** of wikilinks/red-links is a **reference-UI (L6)** concern. Core decides *what a
link resolves to and whether it's a red-link*; the UI decides how it looks.
**Consequences.** Markdown-first, backend-neutral, resolution-unified with ADR-01. **Open:**
cross-shard wikilink target disambiguation syntax ties to the portable-address scheme (O-6).
---
## Coverage & open questions
- **Findings open-Q #1** (per-page vs per-shard ACL) — answered by the settled access model:
per-shard / per-namespace default, per-page ACL opt-in at L4 (`ArchitectureBlueprint.md`).
- **Findings open-Q #2** (wikilink core vs UI) — ADR-06: model/resolution core, presentation UI.
- **Findings open-Q #3** (derived views core vs adapter) — ADR-03.
- Carried to CoreArchitectureBlueprint §12: cross-space addressing syntax (O-6), namespace
normalization presets (O-8), cross-shard search ranking (policy), union-under-unavailability
(O-11).
## Acceptance (SHARD-WP-0001)
Each task T1T6 has an ADR (T1→ADR-01, T2→ADR-02, T3→ADR-03, T4→ADR-04, T5→ADR-05, T6→ADR-06);
all honour INTENT (mechanism over policy, union without erasure, overlay before mutation,
capability-aware adapters) and are consistent with `CoreArchitectureBlueprint.md`. The
access-model thread is ratified (not deferred). The next implementation workplan (domain model /
adapter contract) can proceed without unresolved yawex-derived design gaps.

View File

@@ -6,10 +6,15 @@ Background on document types: InfoTechPrimers on coulomb.social.
| File | Status | Role |
|------|--------|------|
| `CoreArchitectureBlueprint.md` | draft for review | **Whole-system architecture** — layers, abstractions, load-bearing decisions (synthesised from all research) |
| `FederationArchitecture.md` | draft for review | federation design — *what the union does*: T1T10 decision records + the federation-model taxonomy (SHARD-WP-0002) |
| `WikiEngineCoreArchitecture.md` | draft for review | the native **headless, API-first wiki engine** — small page-store kernel + typed-extension framework, as a canonical-mode shard backend (SHARD-WP-0013) |
| `adr/` | living | Architecture Decision Records (ADR-0001: engine activation via feature-control) |
| `FederationRequirements.md` | draft for review | yawex-derived union/federation design notes — resolution, namespace, derived views, provenance, overlay, links (ADR-01…06; SHARD-WP-0001) |
| `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver |
| `TechnicalSpecificationDocument.md` | draft scaffold | How the system is built |
| `UseCaseCatalog.md` | draft | 25 use cases promoted from c2 + yawex research |
| `ArchitectureBlueprint.md` | draft | Access, history, and identity architecture |
| `TechnicalSpecificationDocument.md` | draft + §A | How the system is built; **§A = the normative shard adapter contract** (T11T16, T18; SHARD-WP-0002) |
| `UseCaseCatalog.md` | draft | 84 use cases promoted from c2 + yawex + ~23-system research |
| `ArchitectureBlueprint.md` | draft | Access, history, and identity sub-blueprint (the L0L4 authorization ladder; referenced by CoreArchitectureBlueprint §9) |
Promote material from `research/` and reviewed items from `demand/` into spec
before treating it as implementation authority.

View File

@@ -33,7 +33,134 @@ per information space.
## 4. Architecture References
- `spec/ArchitectureBlueprint.md`access, history, identity delegation
- `spec/CoreArchitectureBlueprint.md`whole-system architecture (layers, the dual narrow
waist, the 15 capability spectra, projection, consistency); the normative source for §A.
- `spec/FederationArchitecture.md` — federation design (*what the union does*); §A is its
companion (*what a backend must expose*).
- `spec/FederationRequirements.md` — yawex-derived ADRs (resolution, page model, overlay).
- `spec/ArchitectureBlueprint.md` — access, history, identity delegation (L5).
---
# A. Shard Adapter Contract (normative)
Deliverable of **SHARD-WP-0002** (T11T16, T18): the versioned interface a backend implements
to participate as a shard, and the *normative* rules core relies on. It is the **bottom narrow
waist** (blueprint §6); the page model is the top waist (§7). This section is normative where it
says **MUST/SHOULD**; design rationale lives in the blueprint (cited, not restated).
## A.1 (T11) Capability model & versioned interface
- The contract is a **versioned interface** (`Foswiki::Store`/`Foswiki::Meta` is the proof a
swappable-backend-behind-a-stable-interface works). A binding declares the contract version
it implements; core checks compatibility at registration.
- **Operation verbs:** `read, write, diff, merge, lock, version, publish, notify,
transclude-source, translate-syntax, structured-payload, derive-projection, execute`. The
last two are **gated, OFF by default** (A.6/T18). Verb support is part of the profile and MUST
reconcile with the T10 federation-ops matrix.
- **Capability profile = a position on each capability spectrum**, not a boolean checklist. The
full set is the **fifteen spectra** (blueprint §6.1); operationally they reduce to a **small
orthogonal core** (substrate, write-granularity, content-opacity, operational-envelope,
access-grant, computational/liveness) with the rest **implied** via published rules that
**forbid impossible profiles** (blueprint §6.5). Degradation reads only the **named
axis-interaction subset** (blueprint §6.5 table) — that table *is* the degradation contract.
- A profile MUST express **absence** cleanly (the Oddmuse floor); core never guesses a missing
capability.
## A.2 (T11/§6.6) Conformance — profiles are verified, not asserted
- The contract ships a **versioned conformance suite**. A binding is **admissible only if it
passes**: the suite exercises each declared verb and spectrum position and checks observed
behaviour matches the claim (a `write` round-trips, `notify` actually fires, an opaque shard
refuses plaintext query, implied-position rules hold). A lying/buggy profile is **rejected at
registration**, not run in production (blueprint §6.6).
- Conformance makes capability-as-data (I-3) and the §6.5 degradation contract **sound**.
Mismatch is reported as a capability-by-capability diff; an adapter may register
*degraded-but-honest* (drop the unsupported claim).
## A.3 (T14) Adapter binding — attachment-mode taxonomy
A backend MAY offer several modes; attach mode is a **per-binding, capability-gated choice**
with one declared authoritative (blueprint §6.3):
- **file-store** (native vault/folder *or* an interchange/sync mirror) · **git-IS-store** (forge
wikis & ikiwiki — git is store *and* journal; the home case) · **in-engine-hosted** (XWiki
component, Obsidian/Logseq/Roam plugin, Trilium script) · **local-REST** (Joplin Data API,
Trilium ETAPI) · **external-API-only** (Notion) · **direct-DB** (MojoMojo schema→model) ·
**CRDT-replica** (Anytype/AFFiNE/AppFlowy) · **P2P / no-central-endpoint**.
- **Backend-swap tolerance:** identity/provenance MUST survive a substrate change (bind to
capabilities, not "it's files"). **Boundary:** an **image / live-memory blob is never an
attach target** (I-12) — participation is export→files only.
- (UC-38, UC-40, UC-43, UC-50, UC-53, UC-57, UC-60, UC-62, UC-64, UC-65, UC-76, UC-79, UC-81.)
## A.4 (T12) Page model — structured / computational payload
The backend-neutral, Markdown-first page model MUST carry, without lossy flattening, every
shape in blueprint §7.1: prose; typed/computed records (incl. effective-vs-own computed
metadata); typed-graph statements (Wikibase); inline-embedded objects (Quip/Notion);
non-Markdown assets (typed asset / opaque blob / content-type registry); and the **four
computational shapes** (one-source-many-projections UC-83; notebook-with-embedded-output UC-84;
program-as-page; live/temporal). All reduce to `(content|source, structure, provenance
envelope, [derivation rule])`. **Identity is a stable handle; placement is separate; equivalence
is fingerprint-based** (blueprint §7.2, FederationRequirements ADR-01/02, I-9). The **provenance
envelope is layered** (page + span deltas; effective = page ⊕ delta, §7.3). (UC-34, UC-39,
UC-55, UC-58, UC-54, UC-44, UC-66, UC-67, UC-73, UC-83, UC-84.)
## A.5 (T13) History portability — adopt / supplement / import
Per the history axis: **adopt** git-native history as-is; **supplement** non-portable internal
history (DB / Notion / Joplin / Trilium revisions, CRDT-log) — the journal begins-now / mirrors
/ snapshots; **import** open file history (RCS, PlainFile, MojoMojo DB-version-rows) backfilled
preserving author/timestamp. **Partial/truncated history MUST be reported honestly** ("history
begins at", UC-24), never implied complete. Embedded-output documents (notebooks) use
paired-text (Jupytext) / cell-aware merge (nbdime). The space's own coordination history is the
event-sourced decision log (blueprint §8.1). (UC-36, UC-41, UC-24.)
## A.6 (T15) Syntax translation & content fidelity
Translation is a spectrum: **native → lossless → lossy-with-fidelity-report**. Read native
markup (TML, XWiki syntax, HTML) into the page model; accept Markdown overlays back via
**lossless bidirectional translation** where possible (Foswiki WysiwygPlugin is the proof). For
non-round-tripping models (Notion blocks, Trilium HTML, CRDT, typed-graph, notebook JSON/MIME):
translate lossily but **make the fidelity loss visible** — a per-shard/per-page report of what
projects cleanly vs degrades, with non-mappable elements preserved as provenance/sidecar (I-4).
Add a **structured-re-evaluable-value** point to the opacity spectrum (Wolfram expression).
Where no acceptable translation exists, the shard is a **read-only/projection** participant
(I-8), never silently corrupted. (UC-42, UC-59, UC-03, UC-73.)
## A.7 (T16) Addressing, identity & navigation
- **Span addressing:** adopt native span IDs where minted (Roam `:block/uid`, Logseq `id::`,
Notion/CRDT UUID), shard-scoped so they survive projection and don't collide; else a
position address (path+range) or content-fingerprint address. Portable tumbler is the ideal
(blueprint O-6).
- **Transclusion** = one reference-not-copy primitive over the addressable union (FederationArch
T8). **Query/navigation:** delegate to a shard's native query where capable (Datalog, DB
query, XWQL, SPARQL), **else build a derived index over the projection** (Logseq pattern);
dimensional/query-defined views are derived-tier. (UC-4448, UC-51, UC-52, UC-54, UC-63,
UC-74.)
## A.8 (T18) Computational / executable content capability
**In scope as a page-model + projection concern; out of scope as an execution platform**
(blueprint §8.5). Core **recognises** computational types, attaches the **canonical source**,
and presents derived forms as **provenance- and liveness-marked projections**. `derive-
projection`/`execute` are **gated capabilities, OFF by default**, carrying a **trust/sandbox**
concern, **degrading to a captured snapshot / static render / recording**. One snapshot-
provenance record (run id, source rev, timestamp, environment "unguaranteed") serves notebooks,
renders, recordings. No INTENT amendment required. (UC-54, UC-55, UC-83, UC-84.)
## A.9 Conformance & module mapping
The contract maps to `src/shard_wiki/adapters/` (the bottom waist: `AdapterContract`,
`CapabilityProfile`, attachment-mode binding, the conformance suite) consuming `model/`
(page model + capability value types) and `provenance/` (blueprint §11). Each concrete adapter
ships its declared profile + a conformance pass.
*Decided:* A.1A.8 (versioned capability contract, verified conformance, binding taxonomy,
page model, history portability, translation, addressing, gated computational content).
*Deferred:* per-backend adapter specs (one per backend, later). *Open:* blueprint §12 O-items
(addressing O-6, axis interactions O-5, span-authz O-10).
## 5. Integration Boundaries
@@ -47,10 +174,13 @@ Package scaffold only (`__version__`, smoke tests). Domain model not yet coded.
## 7. Use Cases
`spec/UseCaseCatalog.md`25 use cases (UC-01UC-25) promoted from c2 wiki
origins and yawex prior-art research.
`spec/UseCaseCatalog.md` — 84 use cases (UC-01UC-84) from c2/yawex origins,
the wiki-engine + modern-tool + computational research, and the syntheses.
## 8. Next Specification Work
Outputs from `SHARD-WP-0001` tasks (page resolution, namespaces, derived views,
provenance, overlays, link semantics) will be incorporated here as they complete.
The design layer is complete: `SHARD-WP-0001` (→ `FederationRequirements.md`),
`SHARD-WP-0002` (→ `FederationArchitecture.md` + §A above), and the hardened
`CoreArchitectureBlueprint.md`. The next workplan is the **implementation** of the
domain model + adapter contract (starting from §A and blueprint §11's module layout),
likely with a first spike on the keystone (the event-sourced decision log + derived fold).

View File

@@ -45,6 +45,71 @@ of the shard-spectrum work; EG carry the c2/yawex authoring and reading patte
---
## Capability structure for the wiki engine (reuse-surface-aligned)
*Added by SHARD-WP-0013 T2 — a structural **layer over** the thematic catalog (UC IDs unchanged),
mapping the 84 UCs onto a small **engine core** plus **typed, per-shard-activatable extensions**.
This is the bridge from demand (UCs) to the engine's typed-extension model
(`WikiEngineCoreArchitecture.md`, T5) and is aligned with the shard-wiki capability entries on the
reuse surface (`capability.wiki.*`).*
**Reading rule:** the engine is a **small always-on core**; everything else is a **typed extension
a shard activates only if it needs it** (the activation mechanism itself reuses
`capability.feature-control.evaluate`). "Default" = on/off for a standalone single-shard wiki.
### Engine core (always on — the irreducible kernel)
| Core unit | What it is | UCs | reuse-surface |
|-----------|------------|-----|---------------|
| **EC-1 Page lifecycle & content** | author / read / edit a Markdown page; delete is a history event | UC-08, UC-09 | `capability.wiki.page-model` |
| **EC-2 Identity, naming & placement** | stable page identity ≠ placement; namespace/path | UC-22 (placement), page-model kernel | `capability.wiki.page-model` |
| **EC-3 History & recoverability** | every write recoverable; history is the floor | UC-24 | `capability.wiki.coordination-journal` |
| **EC-4 Links (wikilink + red-link)** | `[[Target]]` resolution; createable red-links | UC-23 | `capability.wiki.page-model` |
| **EC-5 Storage via the adapter contract** | the engine *is* a canonical-mode shard behind §A | — | `capability.wiki.adapter-contract` (minimal profile) |
EC-1…EC-5 are the Ward-Cunningham/c2 open-wiki minimum: write a page, link pages, never lose
an edit. A shard can run with **only** the core.
### Typed extensions (activate per shard)
| Ext | Cluster | UCs | reuse-surface capability | Activatable | Default (standalone) |
|-----|---------|-----|--------------------------|-------------|----------------------|
| **X-OVERLAY** | overlay / lightweight-patch write path | UC-04, 26, 29 | `capability.wiki.overlay` | yes | on (no-op without remote/limited targets) |
| **X-AUTHZ** | access-control ladder L0→L4 | UC-06 + the L-ladder | `capability.authorization.policy-evaluate` (reuse) | yes (tiered) | L0 (open) |
| **X-VIEW** | derived views (RecentChanges, BackLinks, AllPages, SiteMap, Search) | UC-17UC-21, UC-63 | (planned `capability.wiki.derived-views` — gap, T3) | yes | BackLinks/RecentChanges on; Search off |
| **X-STRUCT** | structured / typed / computed page data + typed-graph | UC-34, 39, 58, 67, 73 | `capability.wiki.page-model` (typed) | yes | off |
| **X-ADDR** | span addressing, transclusion, dimensional / query navigation | UC-32, 4449, 51, 52, 74 | `capability.wiki.page-model` + query | yes (transclusion is core-lite) | transclusion on; dimensional/query off |
| **X-COMP** | computational / executable content (literate, notebook, program-as-page, live) | UC-54, 55, 83, 84 | `capability.wiki.engine-typed-extensions` (computational) | yes (gated, trust/sandbox) | off |
| **X-PROV** | rich provenance & page metadata | UC-25, 75 | `capability.wiki.page-model` (provenance) | yes (base provenance is core) | base on; rich off |
| **X-COLLAB** | c2 collaboration / social patterns | UC-10UC-16 | (UI/convention; partial gap, T3) | yes | varies (mostly UI-layer) |
| **X-FED** | federation & union across shards | UC-0107, 27, 28, 3033, 56, 71, 72, 79 | `capability.wiki.shard-orchestration` + `…federation-models` | yes | off (single-shard engine) |
| **X-ATT** | attach *foreign* backends as shards | UC-3543, 50, 53, 57, 6062, 6466, 6870, 7682 | `capability.wiki.adapter-contract` | yes | off (orchestrator role, not the engine's own shard) |
Note: X-FED and X-ATT are the **orchestrator** capabilities — for a *standalone* engine they are
off; when shard-wiki orchestrates, the engine is simply one attached (canonical) shard among many.
That is the engine↔orchestrator boundary made concrete (engine = a canonical-mode shard).
### Conflict-mediation map (how the framework reconciles tensions into one whole)
The collected UCs embed genuinely conflicting requirements. The framework mediates each with a
**mechanism**, never a hard-coded policy — so one consistent featureset serves all:
| Tension (conflicting UCs) | Mediation mechanism |
|---------------------------|---------------------|
| **Open vs governed** (UC-01 open edit ↔ UC-06 enterprise ACL) | the **L0→L4 authz ladder** (X-AUTHZ, additive); **history is the floor** at L0 — recoverability, not gatekeeping |
| **Lossless vs lossy content** (UC-42 ↔ UC-59) | **translation spectrum + fidelity report** (TSD §A.6); native form stays canonical, loss is shown not hidden |
| **Live vs snapshot** (UC-32 live transclusion ↔ UC-84/UC-3 cached) | the **live↔snapshot axis** + projection kind; degrade to a **marked** snapshot, never imply live |
| **Canonical vs chorus** (UC-07 reconcile ↔ UC-27 coexist) | **conflict detection is core, resolution is a policy preset** (chorus / designated-canonical / …) |
| **Eager vs lazy** (UC-03 projection ↔ UC-05 union) | **projection materialization** policy (lazy default; eager opt-in) |
| **Single vs multi-writer** (concurrent edits) | **per-space append authority** on the coordination journal (total order, read-your-writes) |
| **Integrated whole vs only-what-you-need** | the **typed-extension framework** + **per-shard activation** (reuse `feature-control`) — *the headline mediation*; extensions compose against typed contracts, so the featureset is one integrated whole yet selectively enabled |
| **Minimal core vs feature-rich** | **small core (EC-1…EC-5)** + extensions; nothing beyond the c2 minimum is mandatory |
This table is the seed of the engine's design rationale (T5): every conflict has a *mechanism*
that lets a shard sit anywhere on the spectrum without forking the framework.
---
## A. Information space, federation & coordination
*Federation, union, overlay, projection, and the Git-backed coordination layer (INTENT + yawex/federation seeds).*

View File

@@ -0,0 +1,269 @@
# WikiEngineCoreArchitecture
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0013 T5**
The architecture of shard-wiki's **native reference wiki-engine**: a **headless, API-first**
engine — a **small core** plus a **stringent typed-extension framework** — that addresses the
whole use-case catalogue, mediates conflicting requirements into one integrated featureset, and
lets each shard **activate only what it needs**. Authoritative as of the ratified INTENT
amendment (2026-06-15, decision `84ffdb48`): the engine is **additive** and is shard-wiki's
**reference first-party shard backend (a canonical-mode shard)** — not a replacement for other
engines, not a UI.
Relation to other specs (referenced, not restated):
- `CoreArchitectureBlueprint.md` — the orchestrator/whole-system architecture. **The engine is
one shard behind §A; federation, union, projection, and cross-shard coordination are the
orchestrator's job, not the engine's.** That is what keeps the engine small.
- `TechnicalSpecificationDocument.md §A` — the shard adapter contract the engine implements.
- `FederationRequirements.md` — page resolution, overlay, link semantics (ADRs the engine reuses).
- `UseCaseCatalog.md` "Capability structure" layer (T2) — the core-vs-extension map + the
conflict-mediation map this document realizes.
- reuse surface (`capability.wiki.*`, plus consumed `feature-control` / `authorization`).
---
## 1. Thesis: a small page-store kernel; everything else is a typed extension
> **The engine is a page-store kernel with a typed-extension runtime. Every capability beyond
> the c2-minimum is a *typed extension* a shard activates only if it needs it — and a shard's
> externally-visible capability profile is *computed from its active extension set*.**
That single chain — **configuration (which extensions) → capability (what the shard can do) →
conformance (verified)** — is the whole design. It mirrors the orchestrator's discipline
(`CoreArchitectureBlueprint` §6.5: capability-as-data, verified, no per-backend code) and turns
"integrated whole, yet activate only what you need" from a slogan into a mechanism.
The engine stays small for a structural reason: it is **one shard**, not a federation layer.
Union, projection, equivalence, cross-shard overlay-orchestration, and the federation models all
live in shard-wiki's orchestrator (the blueprint). The engine implements `ShardAdapter` (§A) and
nothing above it. So "wiki engine" here means *a really good single canonical shard with a
typed-extension framework and a headless agent-first API* — not a re-implementation of shard-wiki.
---
## 2. Engine invariants
| # | Invariant | Why |
|---|-----------|-----|
| E-1 | **One shard, not a federation layer.** The engine implements `ShardAdapter` (§A); union/projection/federation are the orchestrator's. | Keeps the engine small; no duplication of the blueprint. |
| E-2 | **Small kernel.** The kernel is only: page store + history, the page model (reused), the extension runtime, the API. | Common case (a plain wiki) is trivial. |
| E-3 | **Everything else is a typed extension.** No feature beyond the c2-minimum is baked into the kernel. | Integrated-whole-yet-selective; testable boundary. |
| E-4 | **Per-shard activation.** A shard runs an *activation profile* (a set of extensions + config); unused features cost nothing. | "Activate only what you need." |
| E-5 | **Capability profile is derived from active extensions.** The §A profile the engine declares is computed from its activation profile, then conformance-verified. | One source of truth; honest, verified capabilities. |
| E-6 | **Headless & API-first.** The API is the only interface; no bundled UI/rendering (consumer concern, L6). | INTENT amendment; clean orchestrator/consumer split. |
| E-7 | **Agent-first ergonomics.** The API is typed, introspectable, batchable, low-round-trip. | INTENT: optimized for efficient agent/automation access. |
| E-8 | **Reuse over reinvent.** Page model, history/journal, activation, and authz are *consumed* (existing capabilities), not rebuilt. | Smallness; reuse-surface alignment. |
| E-9 | **Extensions are typed & verified.** An extension declares its types/hooks/deps; activation is rejected if types conflict or deps are unmet (impossible profiles forbidden). | Stringency; mirrors §6.5 + conformance. |
---
## 3. The kernel (four concepts)
The kernel is deliberately four things — nothing more is mandatory.
1. **Page** — the backend-neutral page model (`capability.wiki.page-model`, reused as-is):
stable identity ≠ placement, layered provenance, page shapes. The kernel does **not** redefine
it; extensions may *register additional shapes/types* (§4).
2. **Store + history** — a git-backed page store (the engine is the *git-IS-store* case from the
blueprint): a write is a commit; history is native and recoverable (E-3/I-10). Coordination
decisions reuse the event-sourced journal (`capability.wiki.coordination-journal`).
3. **Extension runtime** — the typed-extension registry, hook dispatcher, type checker, and
activation engine (§4). *This is the core innovation; it is the only “framework” in the kernel.*
4. **API** — the headless, typed, agent-first surface (§7). Kernel endpoints cover the c2-minimum
(page CRUD-as-history, links, history); extensions extend the surface through typed routes.
The **c2-minimum** a kernel-only shard delivers (no extensions): write a page, link pages
(`[[wikilink]]` + red-link), never lose an edit. That is a complete, useful headless wiki.
---
## 4. The typed-extension model (the framework)
An **Extension** is a typed unit declaring a contract the runtime enforces:
```
Extension:
id : reverse-domain id (e.g. ext.struct.typed-records)
provides : capability ids it realizes (reuse-surface; e.g. capability.wiki.page-model[typed])
types : page shapes / field schemas / content-types it introduces (typed, validated)
hooks : kernel lifecycle bindings it implements (see below)
api : typed routes it adds to the headless surface
depends_on : other extensions / consumed capabilities required
conflicts_with: extensions it cannot co-activate with
config : declared, schema-checked activation parameters
```
**Hooks (the kernel lifecycle the runtime dispatches):**
`on_resolve` (name→page), `on_read`, `on_write` (validate/transform a draft), `on_link`
(link/transclusion resolution), `on_history`, `on_query`, `on_render_request` (produce a derived
representation for a consumer), `on_profile` (contribute capability-spectrum positions, E-5).
Hooks are **typed** (typed inputs/outputs) and dispatched in a **declared, deterministic order**.
**Typing & composition (stringency):**
- At activation, the runtime builds the **dependency closure**, checks **type consistency** (no
two active extensions claim incompatible types for the same page shape/field; `conflicts_with`
honoured), and rejects an **impossible profile** — exactly the §6.5 implication-rule discipline,
applied to extensions. A rejected profile fails fast at boot, never silently.
- Composition is **deterministic**: hook order is declared; conflicts are resolved by explicit
precedence or rejection, never by accident.
- Extensions ship a **conformance check** (mirrors §6.6): an activated extension is exercised
against its declared types/hooks before the shard serves traffic — *typed contracts verified,
not trusted*.
**Per-shard activation (reuse, not reinvent):**
- A shard's **activation profile** = `{extension id → config}`. Activation/evaluation **reuses
`capability.feature-control.evaluate`** (helix_forge/feature-control) — shard-wiki does not
build a bespoke flagging system (T3 consumption).
- **E-5 in action:** the engine's `on_profile` hooks fold the active extensions into the §A
**capability profile** the shard advertises to the orchestrator (e.g. activate
`ext.struct.typed-records` → the `structure` spectrum rises and `structured-payload` is
declared). The profile is then conformance-verified (§A.2). *Configuration → capability →
conformance is one chain.*
---
## 5. Featureset map: core vs extensions, and conflict mediation
The engine realizes the T2 "Capability structure" layer (`UseCaseCatalog.md`). Mapping (the
*page/content-level* clusters; **X-FED and X-ATT are orchestrator concerns, not engine
extensions** — E-1):
| Engine kernel (always on) | T2 | reuse-surface |
|---------------------------|----|---------------|
| Page lifecycle, identity/placement, history, links, store | EC-1…EC-5 | `capability.wiki.page-model`, `…coordination-journal`, `…adapter-contract` |
| Built-in typed extension | T2 cluster | provides / consumes | default |
|--------------------------|-----------|---------------------|---------|
| `ext.overlay` | X-OVERLAY | `capability.wiki.overlay` | on (no-op locally) |
| `ext.authz` (L0→L4 tiers) | X-AUTHZ | consumes `capability.authorization.policy-evaluate` | L0 |
| `ext.views` (BackLinks/RecentChanges/…) | X-VIEW | `capability.wiki.derived-views` | BackLinks/RecentChanges on |
| `ext.struct` (typed/computed/graph) | X-STRUCT | `capability.wiki.page-model[typed]` | off |
| `ext.addr` (span addr / transclusion / query) | X-ADDR | `capability.wiki.page-model`+query | transclusion on |
| `ext.compute` (literate/notebook/program/live) | X-COMP | `capability.wiki.engine-typed-extensions` | off (gated, sandbox) |
| `ext.prov` (rich provenance/metadata) | X-PROV | `capability.wiki.page-model[provenance]` | base on |
| `ext.collab` (c2 social patterns) | X-COLLAB | (UI/convention; mostly consumer) | off |
**Conflict mediation (T2 map) realized by the framework** — every tension is a *mechanism*, not a
baked-in choice, so one featureset serves all:
| Tension | Realized by |
|---------|-------------|
| open vs governed | `ext.authz` tiers (additive); kernel history is the floor at L0 |
| lossless vs lossy | a `translate` hook + fidelity report (consumes the proposed `capability.content.translation-fidelity`, G2) |
| live vs snapshot | `ext.compute`/`ext.addr` mark liveness; degrade to snapshot (never imply live) |
| canonical vs chorus | detection in kernel; resolution is a policy preset (orchestrator) |
| integrated-whole vs only-what-you-need | **the activation profile** (E-4) + typed composition (§4) — the headline mediation |
| minimal vs feature-rich | small kernel (§3) + extensions; nothing beyond c2 is mandatory |
---
## 6. The engine as a canonical-mode shard
The engine exposes itself through an `EngineShardAdapter` implementing §A:
- **Substrate** git-IS-store; **history** git-native; **write** = commit; `current_rev` = sha
(apply-under-drift works out of the box). It is the **most capable shard** shard-wiki can
attach — it dogfoods the contract.
- Its **capability profile is computed from active extensions** (E-5) and **conformance-verified**
(§A.2) — so the orchestrator sees an honest profile, and federation ops degrade by the engine's
*actually-activated* capabilities.
- The orchestrator attaches it like any shard; **federation/union/projection are not in the
engine** (E-1). A standalone deployment is "the engine as the sole canonical shard"; a
federated deployment is "the engine as one shard among many." Same engine, no re-architecture.
This is the precise realization of the INTENT reconciliation: shard-wiki orchestrates; the engine
is the first-party shard it can attach.
---
## 7. Headless API surface & agent ergonomics (E-6/E-7)
API-first means the typed API is the product; there is no UI. Agent-first means it is designed
for cheap, deterministic machine consumption:
- **Typed resource API** over pages, links, history, spans — content-negotiated (raw Markdown,
the structured page model, or an extension-rendered representation via `on_render_request`).
- **Capability/extension introspection** — an endpoint returns the shard's **active extensions,
their types, and the derived §A capability profile**, so an agent can discover *what this shard
can do* before acting (no trial-and-error). This is the agent-facing twin of E-5.
- **Batch & query** — multi-page reads, link-graph and RecentChanges queries (via `ext.views`),
and `on_query` delegation — minimizing round-trips.
- **Write via overlay** — edits go through the overlay path (FederationRequirements ADR-05), so
agent writes are safe (draft → apply-under-drift) and attributable.
- **Deterministic & provenance-carrying** — every response carries the provenance envelope;
identical inputs yield identical outputs (no hidden state) — friendly to caching agents.
---
## 8. Implementation sketch (module layout)
The engine lives under the shard-wiki package as a backend (it sits at L0/L1 — a shard behind the
adapter; nothing in the orchestrator depends *up* on it):
```
src/shard_wiki/engine/
kernel.py # page store + history (git-IS-store), lifecycle; reuses model/, provenance/, coordination/
extension.py # Extension contract, registry, typed hook dispatcher, type checker
activation.py # activation profile; reuses capability.feature-control.evaluate
profile.py # derive the §A CapabilityProfile from active extensions (E-5) + conformance
api.py # headless, typed, agent-first surface (+ extension introspection)
adapter.py # EngineShardAdapter implements adapters/ ShardAdapter (canonical-mode shard)
extensions/ # built-ins: overlay/ authz/ views/ struct/ addr/ compute/ prov/ collab/
```
Dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`, `adapters/`
(contract), `policy/`; it is consumed *only* via its `EngineShardAdapter` (the orchestrator
attaches it as a shard). No orchestrator-tier (`union/`, `projection/`) import.
---
## 9. Reuse (what the engine consumes vs registers)
- **Consumes:** `capability.feature-control.evaluate` (activation), `capability.authorization.
policy-evaluate` (`ext.authz`), the proposed `capability.content.translation-fidelity` (G2,
lossy translation), and shard-wiki's own `capability.wiki.{page-model, coordination-journal,
adapter-contract, overlay, derived-views}`.
- **Registers / realizes:** `capability.wiki.engine-typed-extensions` (this document is its
Discovery evidence — D2→D3 on ratification). The cross-cutting **typed-extension framework**
pattern is proposed back to the reuse surface as **G1** (`capability.platform.typed-extension-
framework`); this engine is its first instance.
---
## 10. Traceability
- **INTENT** — realizes the 2026-06-15 amendment (decision `84ffdb48`): headless, API-first,
additive native engine = canonical-mode shard backend; honours all engine invariants and the
orchestrator boundary (E-1).
- **Use cases** — the kernel/extension split *is* the T2 "Capability structure" layer
(`UseCaseCatalog.md`); every UC is either kernel (EC-1…EC-5) or a named extension; conflicts
use the T2 mediation map (§5). The engine must ultimately cover UC-01UC-84 (per-shard subsets).
- **Architecture** — consistent with `CoreArchitectureBlueprint` (engine = canonical-mode shard,
§6 contract, §7 page model, §8.1 journal) and `TechnicalSpecificationDocument §A` (the contract
it implements). `FederationRequirements` ADR-05/06 supply overlay + link semantics.
- **Reuse surface** — §9; G1/G2 proposals from SHARD-WP-0013 T3.
## 11. Decisions / deferred / open
**Decided:** small page-store kernel + typed-extension runtime (E-2/E-3); engine is one shard,
not a federation layer (E-1); capability profile derived from active extensions (E-5); headless,
API-first, agent-first (E-6/E-7); activation reuses `feature-control` (E-8); extensions are
typed + conformance-verified (E-9).
**Deferred:** the concrete extension SDK/ABI and hook signatures; the API protocol (REST/GraphQL/
MCP) — agent-first introspection is required, the wire format is an implementation spike; the
built-in extensions' internal designs (each is a later workplan).
**Open (tracked):** does `ext.compute` ever execute in-process or strictly delegate/snapshot
(ties blueprint §8.5 + trust/sandbox); is the typed-extension framework promoted to the
reuse-surface platform capability (G1) and then *consumed* here rather than engine-owned;
introspection granularity vs. leaking internal structure to agents.
## 12. Stability note
The **thesis (§1)** and **invariants (§2)** — especially *engine-is-one-shard* (E-1),
*small-kernel/everything-else-typed-extension* (E-2/E-3), and *capability-profile-derived-from-
extensions* (E-5) — are load-bearing. Changing them (e.g. moving federation into the engine, or
baking a feature into the kernel) is an architectural change in the sense of INTENT's Stability
Note and should be rare and deliberate. The headless/API-first posture is fixed by the ratified
INTENT amendment.

View File

@@ -0,0 +1,79 @@
# ADR-0001 — Engine extension activation via feature-control (OpenFeature)
Status: **Accepted** · Date: 2026-06-15 · Deciders: tegwick · Source: SHARD-WP-0013 follow-up
(feature-control assessment)
> First repo-level ADR. (Note: `FederationRequirements.md` contains document-internal
> "ADR-01…06" design notes — those are scoped to that spec; this `spec/adr/` series is the
> repository's standalone architecture decision log, starting here.)
## Context
`WikiEngineCoreArchitecture.md` (SHARD-WP-0013 T5) defines the engine as a small kernel plus a
**typed-extension framework** where each shard **activates only the extensions it needs**
(invariants E-4 activation, E-8 reuse-not-reinvent). It needs a mechanism to decide, per shard
(and per tenant/context), which extensions/features are active — without baking a bespoke flag
system into the engine, and without breaking the **standalone, zero-external-dependency** L0
posture shard-wiki guarantees.
The helix_forge sibling **`feature-control`** (`capability.feature-control.evaluate`, registered
at **D5 / A4 / C3 / R3**) provides exactly this: an **OpenFeature**-based feature-availability
control plane with a working SDK (`feature_control_sdk`: `FeatureControlClient`, `Resolver`, a
static `LocalProvider`), context-scoped evaluation (`tenant_id`/scope), explainable decisions,
and graceful degradation when OpenFeature is absent. shard-wiki already proposed this as a T3
*consumption* (reuse, don't rebuild).
## Decision
**Adopt `feature-control` (via the OpenFeature standard) as the engine's per-shard extension/
feature activation mechanism** — *availability only* — with these constraints:
1. **OpenFeature-shaped, provider-pluggable.** The engine evaluates activation through an
OpenFeature-style client. A static **`LocalProvider`** is the **standalone/L0 default**
(zero external dependency); a `feature-control`/remote provider is plugged in for governed
deployments. This mirrors shard-wiki's existing **identity-provider ladder** (null/local
default → external when present).
2. **Availability ≠ authorization.** feature-control decides *which extensions are active*,
never *who may read/write*. Authorization stays in core (X-AUTHZ / `authorization.policy-
evaluate`). The two are composed but never conflated. (feature-control's own INTENT requires
this.)
3. **Engine layer, not the orchestrator foundation.** Integration lives in
`engine/activation.py`; the current `src/shard_wiki/` core stays dependency-free. OpenFeature/
feature-control is an optional extra, kept out of the standalone path by the `LocalProvider`.
4. **Thin slice only.** Consume `feature-control.evaluate` (mature, A4). Do **not** take a
dependency on the heavier control-plane governance / `rollout` / `visibility` (A2) until a
concrete need appears.
Activation keys = extension ids; evaluation context = `{tenant_id: root-entity, shard_id, …}`;
the resulting active-extension set then **derives** the shard's §A capability profile (E-5).
## Consequences
**Positive**
- No bespoke flag system; reuses a mature (D5/A4) capability — reuse-surface aligned.
- Standalone stays zero-dep (LocalProvider); governed deployments get real runtime control,
multi-tenant scoping, and **explainable** decisions that feed the engine's agent-introspection
API (E-7: "why is extension X off for this shard?").
- "Activate only what you need" + compute control become first-class and reversible at runtime.
- Clean layering: availability (feature-control) vs authorization (core) vs identity (provider).
**Negative / risks (mitigated by the constraints)**
- An optional OpenFeature dependency at the engine layer (mitigated: out of the standalone path).
- Coupling to an external control plane in governed mode (mitigated: provider-pluggable, degrade
to LocalProvider).
- Temptation to route authz through it (mitigated: constraint 2, hard boundary).
## Alternatives considered
- **Bespoke per-shard flag/config in the engine** — rejected: reinvents feature-control, no
standard, no multi-tenant/explainability, violates reuse-not-reinvent (E-8).
- **No activation (all extensions always on)** — rejected: defeats "small core + activate only
what you need" (E-2/E-4) and the compute-control goal.
- **Build on the heavier feature-control control-plane now** — deferred: over-scoping a single
engine's activation; revisit if rollout/governance needs emerge.
## Related
`WikiEngineCoreArchitecture.md` (E-4/E-8, §4 activation), `UseCaseCatalog.md` capability-structure
layer (X-AUTHZ vs activation), `history/260615-reuse-surface-contributions.md` (T3 consumption),
reuse-surface `capability.feature-control.evaluate`, INTENT amendment decision `84ffdb48`.

11
spec/adr/README.md Normal file
View File

@@ -0,0 +1,11 @@
# spec/adr/ — Architecture Decision Records
Repository-level ADRs: one decision per file, `ADR-NNNN-<slug>.md`, status
**Proposed / Accepted / Superseded**. Each records Context · Decision · Consequences ·
Alternatives. These are the standalone, numbered decision log; design-note "ADRs" embedded
inside a spec (e.g. `FederationRequirements.md` ADR-01…06) are scoped to that document and are
not part of this series.
| ADR | Status | Subject |
|-----|--------|---------|
| [ADR-0001](ADR-0001-engine-activation-via-feature-control.md) | Accepted | Engine extension activation via feature-control (OpenFeature), availability-only, LocalProvider standalone default |

View File

@@ -1,10 +1,16 @@
"""shard-wiki — Git-based Markdown wiki orchestrator and federation layer.
See INTENT.md for the authoritative specification of scope and boundaries.
This package orchestrates wiki-shaped content across heterogeneous *shards*;
it is not itself a wiki engine.
See INTENT.md for the authoritative specification of scope and boundaries, and
spec/CoreArchitectureBlueprint.md for the architecture. This package orchestrates
wiki-shaped content across heterogeneous *shards*; it is not itself a wiki engine.
Foundation slice (SHARD-WP-0007): attach folder shard(s) to an
:class:`~shard_wiki.space.InformationSpace`, resolve a name through the union, and
read a page with layered provenance (chorus on ambiguity).
"""
from shard_wiki.space import InformationSpace
__version__ = "0.0.0"
__all__ = ["__version__"]
__all__ = ["__version__", "InformationSpace"]

View File

@@ -0,0 +1,25 @@
"""adapters/ — the shard adapter contract (bottom waist) and concrete adapters."""
from shard_wiki.adapters.conformance import (
Check,
ConformanceError,
ConformanceReport,
assert_conformant,
run_conformance,
)
from shard_wiki.adapters.contract import CONTRACT_VERSION, ShardAdapter
from shard_wiki.adapters.folder import FolderAdapter
from shard_wiki.adapters.git import GitShardAdapter, PageRevision
__all__ = [
"ShardAdapter",
"FolderAdapter",
"GitShardAdapter",
"PageRevision",
"CONTRACT_VERSION",
"Check",
"ConformanceReport",
"ConformanceError",
"run_conformance",
"assert_conformant",
]

View File

@@ -0,0 +1,142 @@
"""Adapter conformance — profiles are verified, not self-asserted (TSD §A.2, blueprint §6.6).
Capability-as-data (I-3) is only sound if a binding's *declared* profile matches its *observed*
behaviour. This battery exercises a binding and reports, check by check, whether claim == reality;
``assert_conformant`` gates registration. A lying/buggy profile fails here instead of silently
poisoning degradation decisions downstream.
This slice verifies the read path + honest absence of unclaimed verbs. Positive probes for
claimed write/diff/merge are deferred (they mutate) to a later workplan.
"""
from __future__ import annotations
from dataclasses import dataclass
from shard_wiki.adapters.contract import ShardAdapter
from shard_wiki.model import NotSupported, Verb
__all__ = ["Check", "ConformanceReport", "ConformanceError", "run_conformance", "assert_conformant"]
# Optional verbs whose *absence* must be honest (calling an unclaimed one raises NotSupported).
_HONEST_ABSENCE_VERBS = (Verb.WRITE, Verb.DIFF, Verb.NOTIFY)
_PROBE = {
Verb.WRITE: lambda a: a.write("__probe__", ""),
Verb.DIFF: lambda a: a.diff("__probe__", "__probe2__"),
Verb.NOTIFY: lambda a: a.notify(),
}
@dataclass(frozen=True, slots=True)
class Check:
name: str
ok: bool
detail: str = ""
@dataclass(frozen=True, slots=True)
class ConformanceReport:
adapter: str
checks: tuple[Check, ...]
@property
def ok(self) -> bool:
return all(c.ok for c in self.checks)
@property
def failures(self) -> tuple[Check, ...]:
return tuple(c for c in self.checks if not c.ok)
def diff(self) -> str:
return "; ".join(f"{c.name}: {c.detail}" for c in self.failures) or "conformant"
class ConformanceError(Exception):
def __init__(self, report: ConformanceReport) -> None:
super().__init__(f"{report.adapter} not conformant — {report.diff()}")
self.report = report
def _safe(fn, name: str, ok_detail: str = "") -> Check:
try:
return fn()
except Exception as exc: # noqa: BLE001 — a check must never crash the battery
return Check(name, False, f"unexpected error: {exc!r}")
def run_conformance(adapter: ShardAdapter) -> ConformanceReport:
checks: list[Check] = []
profile = None
def _profile_validates() -> Check:
nonlocal profile
profile = adapter.profile()
profile.validate()
return Check("profile-validates", True)
checks.append(_safe(_profile_validates, "profile-validates"))
if profile is None: # profile() or validate() failed; can't probe further meaningfully
return ConformanceReport(type(adapter).__name__, tuple(checks))
# READ is the capability floor.
checks.append(Check("supports-read", profile.supports(Verb.READ),
"" if profile.supports(Verb.READ) else "READ not declared"))
# READ round-trips: a declared-readable shard must actually read its own keys.
def _read_round_trips() -> Check:
keys = list(adapter.keys())
if not keys:
return Check("read-round-trips", True, "empty shard")
page = adapter.read(keys[0])
if page.identity.shard != adapter.shard_id:
return Check("read-round-trips", False,
f"page shard {page.identity.shard!r} != {adapter.shard_id!r}")
if not isinstance(page.body, str):
return Check("read-round-trips", False, "body is not text")
return Check("read-round-trips", True)
if profile.supports(Verb.READ):
checks.append(_safe(_read_round_trips, "read-round-trips"))
# WRITE positive probe: a claimed-writable shard must actually round-trip a write. The probe
# is content-preserving (rewrite an existing page with its own body) so it is non-destructive.
def _write_round_trips() -> Check:
keys = list(adapter.keys())
if not keys:
return Check("write-round-trips", True, "empty shard")
k = keys[0]
original = adapter.read(k).body
adapter.write(k, original)
if adapter.read(k).body != original:
return Check("write-round-trips", False, "rewrite did not preserve body")
return Check("write-round-trips", True)
if profile.supports(Verb.WRITE):
checks.append(_safe(_write_round_trips, "write-round-trips"))
# Honest absence: an *unclaimed* optional verb must raise NotSupported when invoked.
for verb in _HONEST_ABSENCE_VERBS:
if profile.supports(verb):
continue # claimed → positive probe deferred (would mutate)
name = f"honest-absence:{verb.value}"
def _probe(v=verb, n=name) -> Check:
try:
_PROBE[v](adapter)
except NotSupported:
return Check(n, True)
except Exception as exc: # noqa: BLE001
return Check(n, False, f"raised {type(exc).__name__}, expected NotSupported")
return Check(n, False, "did not raise NotSupported though verb is unclaimed")
checks.append(_probe())
return ConformanceReport(type(adapter).__name__, tuple(checks))
def assert_conformant(adapter: ShardAdapter) -> ConformanceReport:
"""Run the battery; raise :class:`ConformanceError` if any check fails. Returns the report."""
report = run_conformance(adapter)
if not report.ok:
raise ConformanceError(report)
return report

View File

@@ -0,0 +1,52 @@
"""The shard adapter contract — the bottom narrow waist (CoreArchitectureBlueprint §6, TSD §A).
A backend participates by implementing :class:`ShardAdapter`. ``shard_id``, ``profile`` and
``read`` are mandatory; everything else is an optional capability that defaults to raising
:class:`~shard_wiki.model.NotSupported` — so a limited backend is honest about what it can't do
(graceful degradation, I-8) and core never assumes a capability it wasn't given (capability-as-
data, I-3). Declared profiles are verified by the conformance suite (T4), never taken on trust.
"""
from __future__ import annotations
import abc
from collections.abc import Iterable
from shard_wiki.model import CapabilityProfile, NotSupported, Page
__all__ = ["ShardAdapter", "CONTRACT_VERSION"]
CONTRACT_VERSION = "0.1"
class ShardAdapter(abc.ABC):
"""Versioned interface a backend implements to attach as a shard."""
contract_version: str = CONTRACT_VERSION
@property
@abc.abstractmethod
def shard_id(self) -> str:
"""Stable id scoping every Identity this shard mints."""
@abc.abstractmethod
def profile(self) -> CapabilityProfile:
"""The (to-be-verified) capability profile of this binding."""
@abc.abstractmethod
def keys(self) -> Iterable[str]:
"""The stable page keys this shard offers (the handle half of Identity)."""
@abc.abstractmethod
def read(self, key: str) -> Page:
"""Read one page by its stable key. Raises ``KeyError`` if absent."""
# --- optional capability verbs: honest NotSupported by default ---
def write(self, key: str, body: str) -> Page: # noqa: ARG002
raise NotSupported(f"{type(self).__name__} does not support write")
def diff(self, key: str, other: str) -> str: # noqa: ARG002
raise NotSupported(f"{type(self).__name__} does not support diff")
def notify(self):
raise NotSupported(f"{type(self).__name__} does not support notify")

View File

@@ -0,0 +1,111 @@
"""FolderAdapter — a read-only file-store shard over a directory of Markdown.
The home-case substrate: a plain folder of ``.md`` files. The relative path (sans extension,
``/``-separated) is the stable page **key**; the file is the page **body**; mtime gives a
freshness stamp. Read-only in this slice (overlay/write-through come later); declared profile
reflects exactly that (read-only, file-store, path addressing, no native history/query).
"""
from __future__ import annotations
from collections.abc import Iterable
from datetime import datetime, timezone
from pathlib import Path
from shard_wiki.adapters.contract import ShardAdapter
from shard_wiki.model import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
Identity,
MergeModel,
NativeQuery,
NotSupported,
OperationalEnvelope,
Page,
Placement,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
__all__ = ["FolderAdapter"]
class FolderAdapter(ShardAdapter):
def __init__(self, shard_id: str, root: str | Path, writable: bool = False) -> None:
self._shard_id = shard_id
self._root = Path(root)
self._writable = writable
@property
def shard_id(self) -> str:
return self._shard_id
def profile(self) -> CapabilityProfile:
verbs = {Verb.READ, Verb.WRITE} if self._writable else {Verb.READ}
granularity = WriteGranularity.PER_PAGE if self._writable else WriteGranularity.NONE
return CapabilityProfile(
substrate=Substrate.FILES,
attachment_mode=AttachmentMode.FILE_STORE,
write_granularity=granularity,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.NONE,
merge_model=MergeModel.NONE,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset(verbs),
).validate()
def _path_for(self, key: str) -> Path:
return self._root / f"{key}.md"
def current_rev(self, key: str) -> str | None:
"""The shard's current revision token for ``key`` (mtime iso), or ``None`` if absent.
Used for apply-under-drift comparison (blueprint §8.6)."""
path = self._path_for(key)
if not path.is_file():
return None
return datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc).isoformat()
def write(self, key: str, body: str) -> Page:
if not self._writable:
raise NotSupported(f"{type(self).__name__} is read-only")
path = self._path_for(key)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(body, encoding="utf-8")
return self.read(key)
def keys(self) -> Iterable[str]:
for p in sorted(self._root.rglob("*.md")):
yield p.relative_to(self._root).with_suffix("").as_posix()
def read(self, key: str) -> Page:
path = self._path_for(key)
if not path.is_file():
raise KeyError(key)
body = path.read_text(encoding="utf-8")
mtime = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
envelope = ProvenanceEnvelope(
source_shard=self._shard_id,
liveness=Liveness.STATIC,
staleness=Staleness.FRESH,
source_rev=mtime.isoformat(),
observed_at=datetime.now(tz=timezone.utc),
)
rel = path.relative_to(self._root).as_posix()
return Page(
identity=Identity(self._shard_id, key),
body=body,
envelope=envelope,
placements=(Placement(self._shard_id, rel),),
)

View File

@@ -0,0 +1,180 @@
"""GitShardAdapter — a second substrate: git-as-store (SHARD-WP-0012; TSD §A.3 git-IS-store).
The home case where **git is the store *and* the journal**. Tracked ``*.md`` paths are the page
keys; the working-tree file is the body; a page's ``source_rev`` is the **commit sha of the last
commit touching its path** (per-path, so an edit to one page never drifts another). The declared
profile is *git-IS-store ⟹ substrate=git ∧ history=git-native* — the implication rule the
capability model enforces (§6.5), validated at registration like any other binding.
This adapter adds **no core changes**: it implements the same :class:`ShardAdapter` contract the
folder adapter does, proving "write an adapter + declare a verified profile" is the whole cost of a
new substrate (capability-as-data, I-3). Built on the ``git`` CLI via subprocess — zero new deps.
"""
from __future__ import annotations
import os
import subprocess
from collections.abc import Iterable
from dataclasses import dataclass
from pathlib import Path
from shard_wiki.adapters.contract import ShardAdapter
from shard_wiki.model import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
Identity,
MergeModel,
NativeQuery,
NotSupported,
OperationalEnvelope,
Page,
Placement,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
__all__ = ["GitShardAdapter", "PageRevision"]
@dataclass(frozen=True, slots=True)
class PageRevision:
"""One adopted git-native revision of a page: the commit sha and its subject line."""
sha: str
message: str
_GIT_IDENTITY = {
"GIT_AUTHOR_NAME": "shard-wiki",
"GIT_AUTHOR_EMAIL": "shard@shard-wiki",
"GIT_COMMITTER_NAME": "shard-wiki",
"GIT_COMMITTER_EMAIL": "shard@shard-wiki",
}
class GitShardAdapter(ShardAdapter):
"""A shard whose store is a git repo: keys are tracked ``*.md`` paths, revs are commit shas."""
def __init__(self, shard_id: str, repo_path: str | Path, writable: bool = False) -> None:
self._shard_id = shard_id
self._repo = Path(repo_path)
self._writable = writable
self._repo.mkdir(parents=True, exist_ok=True)
if not (self._repo / ".git").exists():
self._git("init", "--quiet")
@property
def shard_id(self) -> str:
return self._shard_id
def profile(self) -> CapabilityProfile:
# VERSION is always available — a git-IS-store has git-native history to adopt (§A.5),
# read-only or not. WRITE (= commit, PER_PAGE) is added only in writable mode.
verbs = {Verb.READ, Verb.VERSION}
granularity = WriteGranularity.NONE
if self._writable:
verbs |= {Verb.WRITE}
granularity = WriteGranularity.PER_PAGE
return CapabilityProfile(
substrate=Substrate.GIT,
attachment_mode=AttachmentMode.GIT_IS_STORE,
write_granularity=granularity,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.GIT_NATIVE, # git-is-store ⟹ git-native (§6.5)
merge_model=MergeModel.GIT_TEXT,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset(verbs),
).validate()
def write(self, key: str, body: str) -> Page:
"""Write = **commit**: stage the file and commit it (skip a no-op so no empty commit),
returning the page at the new sha. Drift detection rides on ``current_rev`` = that sha."""
if not self._writable:
raise NotSupported(f"{type(self).__name__} is read-only")
rel = f"{key}.md"
path = self._path_for(key)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(body, encoding="utf-8")
self._git("add", "--", rel)
if self._run("diff", "--cached", "--quiet").returncode != 0: # staged changes present
self._git("commit", "-m", f"write {rel}", env=_GIT_IDENTITY)
return self.read(key)
def keys(self) -> Iterable[str]:
out = self._git("ls-files", "*.md").decode()
for line in out.splitlines():
yield line[: -len(".md")] if line.endswith(".md") else line
def read(self, key: str) -> Page:
path = self._path_for(key)
if not path.is_file():
raise KeyError(key)
rev = self.current_rev(key)
return Page(
identity=Identity(self._shard_id, key),
body=path.read_text(encoding="utf-8"),
envelope=ProvenanceEnvelope(
source_shard=self._shard_id,
liveness=Liveness.STATIC,
staleness=Staleness.FRESH,
source_rev=rev,
lineage="git-native",
),
placements=(Placement(self._shard_id, f"{key}.md"),),
)
def current_rev(self, key: str) -> str | None:
"""The sha of the last commit touching ``key``'s path (per-path drift token), or None."""
rel = f"{key}.md"
if not self._path_for(key).is_file():
return None
sha = self._git("log", "-1", "--format=%H", "--", rel).decode().strip()
return sha or None
def history(self, key: str) -> tuple[PageRevision, ...]:
"""Adopt git-native history (§A.5): the commit list for ``key``'s path, newest-first.
VERSION-gated; raises ``KeyError`` for an unknown page. Each revision is a commit sha +
subject — the native log surfaced through the contract, not re-implemented.
"""
if not self.profile().supports(Verb.VERSION):
raise NotSupported(f"{type(self).__name__} does not support version")
if not self._path_for(key).is_file():
raise KeyError(key)
out = self._git("log", "--format=%H%x00%s", "--", f"{key}.md").decode()
revisions = []
for line in out.splitlines():
sha, _, message = line.partition("\x00")
revisions.append(PageRevision(sha=sha, message=message))
return tuple(revisions)
# -- git plumbing --------------------------------------------------------
def _path_for(self, key: str) -> Path:
return self._repo / f"{key}.md"
def _git(self, *args: str, stdin: bytes | None = None, env: dict | None = None) -> bytes:
return self._run(*args, stdin=stdin, env=env, check=True).stdout
def _run(
self, *args: str, stdin: bytes | None = None, env: dict | None = None, check: bool = False
) -> subprocess.CompletedProcess:
return subprocess.run(
["git", "-C", str(self._repo), *args],
input=stdin,
capture_output=True,
env={**os.environ, **(env or {})},
check=check,
)

View File

@@ -0,0 +1,58 @@
"""coordination/ — the event-sourced decision log (L3, coordination-canonical state)."""
from shard_wiki.coordination.decision_log import (
CoordinationState,
DecisionEvent,
DecisionLog,
EventStore,
EventType,
InMemoryEventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.append_authority import (
AppendAuthority,
Lease,
LeaseHeld,
LeaseRegistry,
)
from shard_wiki.coordination.git_event_store import GitEventStore
from shard_wiki.coordination.migration import (
export_jsonl,
import_jsonl,
import_log,
migrate_space,
)
from shard_wiki.coordination.overlay import (
ApplyResult,
ApplyStatus,
Overlay,
OverlayEngine,
)
from shard_wiki.coordination.patch import Patch, render_patch
__all__ = [
"DecisionLog",
"DecisionEvent",
"EventType",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"GitEventStore",
"Lease",
"LeaseHeld",
"LeaseRegistry",
"AppendAuthority",
"import_log",
"migrate_space",
"export_jsonl",
"import_jsonl",
"serialize_event",
"deserialize_event",
"Overlay",
"OverlayEngine",
"ApplyStatus",
"ApplyResult",
"Patch",
"render_patch",
]

View File

@@ -0,0 +1,158 @@
"""Per-space append authority — the single-writer lease over the log (SHARD-WP-0009 T2).
The log is a *total order per space* (§8.6). :class:`~shard_wiki.coordination.git_event_store`
makes a fork physically impossible via compare-and-swap; this layer adds the **policy** that gives
the order a single designated writer: a **per-space lease**. At most one node holds a space's lease
at a time; only the holder writes to the store. A non-holder does not write — it **forwards** its
append intent to the current holder, so intents from anywhere still land in one serialized stream.
The lease is **time-bounded and re-grantable** (HA): if a holder dies, its lease expires and a new
node may take it, resuming appends from the log head (``seq`` stays contiguous across the hand-off).
A node holding a *stale* lease (already re-granted elsewhere) cannot write either — it discovers it
is no longer the holder and forwards instead, so a partitioned ex-holder can never fork the log.
Mechanism over policy (CLAUDE.md): this provides the leasing *primitive*; who acquires when, and
the TTL, are the caller's policy. Single-coordinator only — distributed multi-node leasing and log
sharding are explicit non-goals of this workplan.
"""
from __future__ import annotations
import uuid
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any
from shard_wiki.coordination.decision_log import DecisionEvent, EventStore, EventType
__all__ = ["Lease", "LeaseHeld", "LeaseRegistry", "AppendAuthority"]
def _utcnow() -> datetime:
return datetime.now(tz=timezone.utc)
@dataclass(frozen=True, slots=True)
class Lease:
"""A time-bounded grant of single-writer authority over one space."""
space: str
holder: str
token: str
expires_at: datetime
def valid_at(self, now: datetime) -> bool:
return now < self.expires_at
class LeaseHeld(Exception):
"""Raised when a space's lease is validly held by a different node."""
def __init__(self, lease: Lease) -> None:
super().__init__(
f"space {lease.space!r} leased to {lease.holder!r} until {lease.expires_at}"
)
self.lease = lease
class LeaseRegistry:
"""The single coordinator's grant table: at most one *valid* lease per space.
A lease that has expired is freely re-grantable to any node (the HA replacement path); a still
valid lease is exclusive to its holder (renewable by that holder). The registry also routes
forwarded append intents to the current holder node.
"""
def __init__(self, clock: Callable[[], datetime] = _utcnow) -> None:
self._clock = clock
self._leases: dict[str, Lease] = {}
self._nodes: dict[str, AppendAuthority] = {}
def register(self, node: AppendAuthority) -> None:
self._nodes[node.node_id] = node
def grant(self, space: str, holder: str, ttl_seconds: float) -> Lease:
"""Grant/renew the lease for ``space`` to ``holder``; raise :class:`LeaseHeld` if another
node still holds it validly. An expired lease is re-grantable to anyone."""
now = self._clock()
current = self._leases.get(space)
if current is not None and current.valid_at(now) and current.holder != holder:
raise LeaseHeld(current)
lease = Lease(
space=space,
holder=holder,
token=uuid.uuid4().hex,
expires_at=now + timedelta(seconds=ttl_seconds),
)
self._leases[space] = lease
return lease
def current(self, space: str) -> Lease | None:
"""The lease for ``space`` if one is currently valid, else None (expired/absent)."""
lease = self._leases.get(space)
return lease if lease is not None and lease.valid_at(self._clock()) else None
def holder_node(self, space: str) -> AppendAuthority | None:
lease = self.current(space)
return self._nodes.get(lease.holder) if lease is not None else None
class AppendAuthority:
"""A coordinator node that appends to the shared log only when it holds the space's lease.
Nodes share one :class:`EventStore` and one :class:`LeaseRegistry`. ``append`` routes itself:
the holder writes; a non-holder forwards to whoever holds the lease (acquiring it first if the
space is currently unleased). The append API mirrors :class:`EventStore` so the authority is a
drop-in single-writer guard.
"""
def __init__(
self,
node_id: str,
store: EventStore,
registry: LeaseRegistry,
ttl_seconds: float = 30.0,
) -> None:
self.node_id = node_id
self._store = store
self._registry = registry
self._ttl = ttl_seconds
registry.register(self)
def acquire(self, space: str) -> Lease:
"""Take (or renew) the lease for ``space``. Raises :class:`LeaseHeld` if another node holds
it validly."""
return self._registry.grant(space, self.node_id, self._ttl)
def holds(self, space: str) -> bool:
lease = self._registry.current(space)
return lease is not None and lease.holder == self.node_id
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append via the single authority. If we hold the lease, write; otherwise forward to the
holder. If the space is unleased, acquire it first. A node with a *stale* lease forwards
(it is not the current holder) rather than writing — so it cannot fork the log."""
holder_node = self._registry.holder_node(space)
if holder_node is None:
self.acquire(space) # unleased: take authority, then write below
holder_node = self
if holder_node is self:
return self._store.append(space, type, payload, actor=actor)
return holder_node._write(space, type, payload, actor=actor)
def _write(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None,
) -> DecisionEvent:
"""Apply a forwarded intent. Called only on the lease holder by a forwarding peer."""
return self._store.append(space, type, payload, actor=actor)

View File

@@ -0,0 +1,208 @@
"""The event-sourced coordination decision log — the keystone (CoreArchitectureBlueprint §8.1).
Coordination-canonical state (overlays, equivalence bindings, aliases, merges, forks) is an
**append-only decision log**, not a mutable file; the queryable *current* state is a **derived
fold** of the log (tier-3 disposable). The log is **totally ordered per space** via a single
**append authority**. That total order is what gives read-your-writes across readers (§8.6).
Storage lives behind :class:`EventStore`: :class:`InMemoryEventStore` is the default test double
(an in-process counter); :class:`~shard_wiki.coordination.git_event_store.GitEventStore` is the
git-addressable backend (SHARD-WP-0009). The :class:`DecisionLog` API and the :meth:`fold` are
identical across backends — only storage + the concurrency model differ.
`derived = f(canonical)`: :class:`CoordinationState` is always reproducible by replaying the log.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from types import MappingProxyType
from typing import Any, Protocol, runtime_checkable
__all__ = [
"EventType",
"DecisionEvent",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"DecisionLog",
"serialize_event",
"deserialize_event",
]
class EventType(Enum):
OVERLAY_CREATED = "overlay-created"
BINDING_MADE = "binding-made"
ALIAS_SET = "alias-set"
MERGE_DECIDED = "merge-decided"
PAGE_FORKED = "page-forked"
@dataclass(frozen=True, slots=True)
class DecisionEvent:
"""One immutable, ordered decision. ``seq`` is the per-space total order."""
seq: int
space: str
type: EventType
payload: Mapping[str, Any]
actor: str | None = None
timestamp: datetime = field(default_factory=lambda: datetime.now(tz=timezone.utc))
@dataclass(frozen=True, slots=True)
class CoordinationState:
"""The derived fold of a space's log: current aliases + equivalence groups + open overlays.
Disposable (tier-3): always recomputable from the log via :meth:`DecisionLog.fold`.
"""
aliases: Mapping[str, str]
equivalence_groups: tuple[frozenset[str], ...]
open_overlays: Mapping[str, Mapping[str, Any]]
def resolve_alias(self, name: str) -> str | None:
return self.aliases.get(name)
def equivalent_to(self, identity: str) -> frozenset[str]:
"""All identities equivalent to ``identity`` (including itself if bound), else just it."""
for group in self.equivalence_groups:
if identity in group:
return group
return frozenset({identity})
def serialize_event(event: DecisionEvent) -> bytes:
"""Deterministic, stable-JSON wire form of an event (same bytes for equal events, any process).
Sorted keys + compact separators make the serialization canonical, so a git object hashed from
it is reproducible — the basis for content-addressable, comparable logs across backends.
"""
obj = {
"seq": event.seq,
"space": event.space,
"type": event.type.value,
"payload": event.payload,
"actor": event.actor,
"timestamp": event.timestamp.isoformat(),
}
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
def deserialize_event(data: bytes | str) -> DecisionEvent:
"""Inverse of :func:`serialize_event` — round-trips an event byte-for-byte by field."""
obj = json.loads(data)
return DecisionEvent(
seq=obj["seq"],
space=obj["space"],
type=EventType(obj["type"]),
payload=obj["payload"],
actor=obj["actor"],
timestamp=datetime.fromisoformat(obj["timestamp"]),
)
@runtime_checkable
class EventStore(Protocol):
"""Append-only, per-space ordered storage behind :class:`DecisionLog`.
Two bindings exist: :class:`InMemoryEventStore` (default/test double) and
:class:`~shard_wiki.coordination.git_event_store.GitEventStore` (git-addressable). Both assign
a per-space monotonic ``seq`` at the log head and guarantee read-your-writes for their reach
(in-process for memory; cross-process for git).
"""
def append(
self, space: str, type: EventType, payload: Mapping[str, Any], actor: str | None = None
) -> DecisionEvent: ...
def events(self, space: str) -> tuple[DecisionEvent, ...]: ...
class InMemoryEventStore:
"""In-process append-only store, totally ordered per space (the append authority for a process).
The default test double; the git backend preserves this exact contract on durable storage.
"""
def __init__(self) -> None:
self._events: dict[str, list[DecisionEvent]] = {}
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
seq = len(self._events.get(space, ()))
event = DecisionEvent(seq=seq, space=space, type=type, payload=dict(payload), actor=actor)
self._events.setdefault(space, []).append(event)
return event
def events(self, space: str) -> tuple[DecisionEvent, ...]:
return tuple(self._events.get(space, ()))
class DecisionLog:
"""Append-only decision log, totally ordered per space, with a derived :meth:`fold`.
Storage is delegated to an :class:`EventStore` (default :class:`InMemoryEventStore`); swapping
in the git backend changes only durability + the concurrency model, not this API or the fold.
"""
def __init__(self, store: EventStore | None = None) -> None:
self._store: EventStore = store if store is not None else InMemoryEventStore()
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
return self._store.append(space, type, payload, actor=actor)
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events in append (total) order. Read-your-writes: a just-appended event
is present immediately."""
return self._store.events(space)
def fold(self, space: str) -> CoordinationState:
"""Replay the log into current coordination state (derived = f(log))."""
aliases: dict[str, str] = {}
overlays: dict[str, dict[str, Any]] = {}
groups: list[set[str]] = []
for event in self.events(space):
if event.type is EventType.ALIAS_SET:
aliases[event.payload["alias"]] = event.payload["target"]
elif event.type is EventType.BINDING_MADE:
_merge_group(groups, {str(m) for m in event.payload["members"]})
elif event.type is EventType.OVERLAY_CREATED:
overlays[event.payload["overlay_id"]] = dict(event.payload)
elif event.type is EventType.MERGE_DECIDED:
# A merge resolution may collapse an overlay; minimal handling for the slice.
overlays.pop(event.payload.get("overlay_id", ""), None)
elif event.type is EventType.PAGE_FORKED:
_merge_group(groups, {str(event.payload["source"]), str(event.payload["fork"])})
return CoordinationState(
aliases=MappingProxyType(dict(aliases)),
equivalence_groups=tuple(frozenset(g) for g in groups),
open_overlays=MappingProxyType({k: MappingProxyType(v) for k, v in overlays.items()}),
)
def _merge_group(groups: list[set[str]], members: set[str]) -> None:
"""Union-merge ``members`` into ``groups`` (any existing group sharing a member absorbs it)."""
touching = [g for g in groups if g & members]
for g in touching:
groups.remove(g)
members |= g
groups.append(members)

View File

@@ -0,0 +1,172 @@
"""GitEventStore — a git-addressable binding of :class:`EventStore` (SHARD-WP-0009 T1).
Each space is a ref (``refs/spaces/<sha1(space)>``); each ``append`` writes the event as an
immutable git object (a one-blob tree committed onto the ref) and advances the ref. The commit
chain *is* the totally ordered log: ``seq`` is the depth, ``events`` walks first-parent from the
head oldest→newest. Coordination-canonical state therefore inherits git's history / patch /
review / backup affordances (I-6) and is read-your-writes correct across processes.
The total order is enforced at storage by a **compare-and-swap** ref update
(``git update-ref <ref> <new> <old>``): two appenders racing off the same head — the loser's CAS
fails and it retries off the new head, so a non-holder can never fork the log. The lease layer
(T2) sits *above* this as the append-authority policy; CAS is the mechanism that makes it safe.
Implemented over the ``git`` CLI through :mod:`subprocess` — zero runtime dependencies.
"""
from __future__ import annotations
import hashlib
import os
import subprocess
from collections.abc import Mapping
from pathlib import Path
from typing import Any
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventType,
deserialize_event,
serialize_event,
)
__all__ = ["GitEventStore"]
# Fixed identity so commit objects are reproducible and never prompt for git config; the event's
# own timestamp/actor carry the real provenance, the commit is just the ordered container.
_GIT_IDENTITY = {
"GIT_AUTHOR_NAME": "shard-wiki",
"GIT_AUTHOR_EMAIL": "coordination@shard-wiki",
"GIT_COMMITTER_NAME": "shard-wiki",
"GIT_COMMITTER_EMAIL": "coordination@shard-wiki",
}
_EVENT_PATH = "event.json"
_MAX_CAS_RETRIES = 50
class GitEventStore:
"""Git-backed, append-only, per-space ordered event store (an :class:`EventStore`)."""
def __init__(self, repo_path: str | Path) -> None:
self.repo_path = Path(repo_path)
self.repo_path.mkdir(parents=True, exist_ok=True)
if not (self.repo_path / "HEAD").exists() and not (self.repo_path / ".git").exists():
self._git("init", "--quiet", str(self.repo_path), at_cwd=True)
# -- EventStore contract -------------------------------------------------
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append one event, advancing the space ref under compare-and-swap (retry-on-race)."""
ref = self._ref(space)
for _ in range(_MAX_CAS_RETRIES):
head = self._head(ref)
seq = self._count(ref, head)
event = DecisionEvent(
seq=seq, space=space, type=type, payload=dict(payload), actor=actor
)
commit = self._commit_event(event, parent=head)
if self._cas_update(ref, new=commit, old=head):
return event
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
def import_event(self, event: DecisionEvent) -> None:
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
"""
ref = self._ref(event.space)
head = self._head(ref)
expected = self._count(ref, head)
if event.seq != expected:
raise ValueError(
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
)
commit = self._commit_event(event, parent=head)
if not self._cas_update(ref, new=commit, old=head):
raise RuntimeError(f"import race on {ref}")
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events oldest→newest (append/total order)."""
ref = self._ref(space)
head = self._head(ref)
if head is None:
return ()
shas = self._git("rev-list", "--reverse", "--first-parent", ref).decode().split()
return tuple(
deserialize_event(self._git("cat-file", "blob", f"{sha}:{_EVENT_PATH}"))
for sha in shas
)
# -- git plumbing --------------------------------------------------------
def _commit_event(self, event: DecisionEvent, parent: str | None) -> str:
blob = self._git(
"hash-object", "-w", "--stdin", stdin=serialize_event(event)
).decode().strip()
tree = self._git(
"mktree", stdin=f"100644 blob {blob}\t{_EVENT_PATH}\n".encode()
).decode().strip()
args = ["commit-tree", tree, "-m", f"event {event.seq} {event.type.value}"]
if parent is not None:
args += ["-p", parent]
# Pin the commit date to the event's timestamp for reproducible objects.
date = event.timestamp.isoformat()
env = {**_GIT_IDENTITY, "GIT_AUTHOR_DATE": date, "GIT_COMMITTER_DATE": date}
return self._git(*args, env=env).decode().strip()
def _cas_update(self, ref: str, new: str, old: str | None) -> bool:
"""``git update-ref`` with the old value as a CAS guard (empty oldvalue == must-not-exist).
Returns False if the ref moved since we read ``old`` (lost the race) — the caller retries.
"""
result = self._run("update-ref", ref, new, old if old is not None else "")
return result.returncode == 0
def _head(self, ref: str) -> str | None:
result = self._run("rev-parse", "--verify", "--quiet", ref)
out = result.stdout.decode().strip()
return out or None
def _count(self, ref: str, head: str | None) -> int:
if head is None:
return 0
return int(self._git("rev-list", "--count", "--first-parent", ref).decode().strip())
@staticmethod
def _ref(space: str) -> str:
return f"refs/spaces/{hashlib.sha1(space.encode()).hexdigest()}"
def _git(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
) -> bytes:
result = self._run(*args, stdin=stdin, env=env, at_cwd=at_cwd, check=True)
return result.stdout
def _run(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
check: bool = False,
) -> subprocess.CompletedProcess:
base = ["git"] if at_cwd else ["git", "-C", str(self.repo_path)]
return subprocess.run(
[*base, *args],
input=stdin,
capture_output=True,
env={**os.environ, **(env or {})},
check=check,
)

View File

@@ -0,0 +1,53 @@
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
reproduces identical coordination state; only durability changes.
"""
from __future__ import annotations
from collections.abc import Iterable
from pathlib import Path
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.git_event_store import GitEventStore
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
count = 0
for event in events:
dest.import_event(event)
count += 1
return count
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
return import_log(source.events(space), dest)
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
count = 0
with open(path, "wb") as handle:
for event in events:
handle.write(serialize_event(event) + b"\n")
count += 1
return count
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
with open(path, "rb") as handle:
events = [deserialize_event(line) for line in handle if line.strip()]
return import_log(events, dest)

View File

@@ -0,0 +1,134 @@
"""Overlay engine — overlay-before-mutation (FederationRequirements ADR-05, blueprint §8.2).
An overlay is a non-destructive local edit against a page. It is **coordination-canonical**: an
``OVERLAY_CREATED`` event in the decision log (§8.1), not a mutable side file. The current set
of open overlays is the log fold. ``draft`` records one; ``apply`` (T4) resolves it under drift.
"""
from __future__ import annotations
import uuid
from collections.abc import Mapping
from dataclasses import dataclass
from enum import Enum
from typing import Any
from shard_wiki.adapters import ShardAdapter
from shard_wiki.coordination.decision_log import DecisionLog, EventType
from shard_wiki.model import Identity, Page, Verb
from shard_wiki.provenance import OverlayState
__all__ = ["Overlay", "OverlayEngine", "ApplyStatus", "ApplyResult"]
@dataclass(frozen=True, slots=True)
class Overlay:
"""A non-destructive edit: the proposed ``body`` for ``target``, authored against
``base_rev`` (the shard revision seen at draft time, for drift detection)."""
overlay_id: str
target: Identity
base_rev: str | None
body: str
state: OverlayState = OverlayState.DRAFT
def to_payload(self) -> dict[str, Any]:
return {
"overlay_id": self.overlay_id,
"target_shard": self.target.shard,
"target_key": self.target.key,
"base_rev": self.base_rev,
"body": self.body,
"state": self.state.value,
}
@classmethod
def from_payload(cls, payload: Mapping[str, Any]) -> Overlay:
return cls(
overlay_id=payload["overlay_id"],
target=Identity(payload["target_shard"], payload["target_key"]),
base_rev=payload["base_rev"],
body=payload["body"],
state=OverlayState(payload.get("state", OverlayState.DRAFT.value)),
)
class ApplyStatus(Enum):
APPLIED = "applied" # fast-forwarded and written through
REFUSED_DRIFT = "refused-drift" # source moved under the overlay; no clobber
KEPT_DRAFT = "kept-draft" # target read-only; overlay remains the local truth
@dataclass(frozen=True, slots=True)
class ApplyResult:
status: ApplyStatus
overlay_id: str
page: Page | None = None
detail: str = ""
class OverlayEngine:
def __init__(self, space: str, log: DecisionLog) -> None:
self.space = space
self.log = log
def draft(
self,
target: Identity,
body: str,
base_rev: str | None,
actor: str | None = None,
) -> Overlay:
"""Create a draft overlay and record it in the decision log (coordination-canonical)."""
overlay = Overlay(uuid.uuid4().hex, target, base_rev, body)
self.log.append(self.space, EventType.OVERLAY_CREATED, overlay.to_payload(), actor=actor)
return overlay
def get(self, overlay_id: str) -> Overlay | None:
payload = self.log.fold(self.space).open_overlays.get(overlay_id)
return Overlay.from_payload(payload) if payload is not None else None
def open_overlays(self) -> tuple[Overlay, ...]:
state = self.log.fold(self.space)
return tuple(Overlay.from_payload(p) for p in state.open_overlays.values())
def apply(self, overlay_id: str, adapter: ShardAdapter) -> ApplyResult:
"""Resolve an overlay against its target shard with apply-under-drift semantics (§8.6).
Read-only target → ``KEPT_DRAFT`` (the overlay stays the local truth). Otherwise compare
the overlay's ``base_rev`` to the shard's current rev: equal → fast-forward write-through
(``APPLIED``); changed → ``REFUSED_DRIFT`` (never clobber, I-5). Applying records a
``MERGE_DECIDED`` event that closes the overlay in the fold.
"""
overlay = self.get(overlay_id)
if overlay is None:
raise KeyError(overlay_id)
if adapter.shard_id != overlay.target.shard:
raise ValueError(f"adapter {adapter.shard_id!r} != target {overlay.target.shard!r}")
if not adapter.profile().supports(Verb.WRITE):
return ApplyResult(
ApplyStatus.KEPT_DRAFT, overlay_id, detail="target is read-only; overlay retained"
)
current = _current_rev(adapter, overlay.target.key)
if current != overlay.base_rev:
return ApplyResult(
ApplyStatus.REFUSED_DRIFT,
overlay_id,
detail=f"base_rev {overlay.base_rev!r} != current {current!r}",
)
page = adapter.write(overlay.target.key, overlay.body)
self.log.append(
self.space,
EventType.MERGE_DECIDED,
{"overlay_id": overlay_id, "outcome": "applied"},
)
return ApplyResult(ApplyStatus.APPLIED, overlay_id, page=page)
def _current_rev(adapter: ShardAdapter, key: str) -> str | None:
"""Best-effort current-revision probe; adapters without one are treated as no-rev."""
probe = getattr(adapter, "current_rev", None)
return probe(key) if callable(probe) else None

View File

@@ -0,0 +1,38 @@
"""Patch rendering — an overlay as a reviewable diff (FederationRequirements ADR-05).
A pure function over (base body, overlay body): the auditable change proposal that an overlay
becomes before it is applied. Markdown/native text in this slice (lossless); a lossy
native-syntax-with-fidelity-report variant is later (TSD §A.6).
"""
from __future__ import annotations
import difflib
from dataclasses import dataclass
from shard_wiki.coordination.overlay import Overlay
from shard_wiki.model import Identity
__all__ = ["Patch", "render_patch"]
@dataclass(frozen=True, slots=True)
class Patch:
target: Identity
diff: str
@property
def is_empty(self) -> bool:
return self.diff == ""
def render_patch(overlay: Overlay, base_body: str) -> Patch:
"""Render ``base_body`` → ``overlay.body`` as a unified diff against the overlay target."""
label = str(overlay.target)
lines = difflib.unified_diff(
base_body.splitlines(keepends=True),
overlay.body.splitlines(keepends=True),
fromfile=f"a/{label}",
tofile=f"b/{label}",
)
return Patch(target=overlay.target, diff="".join(lines))

View File

@@ -0,0 +1,49 @@
"""engine/ — shard-wiki's native, headless wiki engine (a canonical-mode shard backend).
A small page-store kernel + a typed-extension runtime (WikiEngineCoreArchitecture). The engine
is *one shard*: it is consumed by the orchestrator only via its `EngineShardAdapter`; it never
imports the derived tier (`union`/`projection`).
"""
from shard_wiki.engine.activation import (
ActivationContext,
ActivationProvider,
ActivationResolver,
StaticProvider,
feature_control_provider,
)
from shard_wiki.engine.extension import (
ActiveExtensions,
Extension,
ExtensionError,
ExtensionRuntime,
Hook,
)
from shard_wiki.engine.adapter import EngineShardAdapter, build_engine_shard
from shard_wiki.engine.kernel import EngineKernel
from shard_wiki.engine.links import extract_wikilinks
from shard_wiki.engine.profile import (
ProfileContribution,
derive_profile,
engine_base_profile,
)
__all__ = [
"EngineKernel",
"extract_wikilinks",
"Hook",
"Extension",
"ExtensionError",
"ExtensionRuntime",
"ActiveExtensions",
"ActivationContext",
"ActivationProvider",
"StaticProvider",
"ActivationResolver",
"feature_control_provider",
"engine_base_profile",
"ProfileContribution",
"derive_profile",
"EngineShardAdapter",
"build_engine_shard",
]

View File

@@ -0,0 +1,129 @@
"""Per-shard extension activation (WikiEngineCoreArchitecture E-4/E-8, ADR-0001).
Decides *which registered extensions are active* for a given shard — **availability only, never
authorization**. The mechanism is OpenFeature-shaped and **provider-pluggable**:
- :class:`StaticProvider` is the **standalone / L0 default** — zero external dependency, in-process
flags with optional per-shard scoping. A kernel-only or offline shard uses this.
- :func:`feature_control_provider` lazily wraps the helix_forge ``feature_control_sdk`` (over
OpenFeature) when it is installed; absent, it returns ``None`` and the caller falls back to the
static default. This mirrors shard-wiki's identity-provider ladder (local default → external
when present), and keeps the engine core pure-stdlib.
An *activation profile* is ``{extension id → config}``; the active id set then feeds the
extension runtime's `activate()` (T2) and the derived capability profile (T4, E-5).
"""
from __future__ import annotations
from collections.abc import Iterable, Mapping
from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable
__all__ = [
"ActivationContext",
"ActivationProvider",
"StaticProvider",
"ActivationResolver",
"feature_control_provider",
]
@dataclass(frozen=True, slots=True)
class ActivationContext:
"""Scope for an activation decision. Carries no principal/authz — availability only."""
shard_id: str
tenant_id: str | None = None
extra: Mapping[str, Any] = field(default_factory=dict)
def as_dict(self) -> dict[str, Any]:
d: dict[str, Any] = {"shard_id": self.shard_id}
if self.tenant_id is not None:
d["tenant_id"] = self.tenant_id
d.update(self.extra)
return d
@runtime_checkable
class ActivationProvider(Protocol):
"""Evaluates feature availability for an extension key in a context (OpenFeature-shaped)."""
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool: ...
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]: ...
@dataclass(frozen=True, slots=True)
class StaticProvider:
"""The standalone default: in-process flags, optionally overridden per shard. Zero deps.
``flags`` is the base availability map; ``per_shard[shard_id]`` overrides it for one shard;
``configs[feature_key]`` supplies per-extension config. Unknown keys → ``default``.
"""
flags: Mapping[str, bool] = field(default_factory=dict)
per_shard: Mapping[str, Mapping[str, bool]] = field(default_factory=dict)
configs: Mapping[str, Mapping[str, Any]] = field(default_factory=dict)
default: bool = False
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
shard = context.get("shard_id")
scoped = self.per_shard.get(shard, {}) if shard is not None else {}
if feature_key in scoped:
return scoped[feature_key]
return self.flags.get(feature_key, self.default)
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
return self.configs.get(feature_key, {})
class ActivationResolver:
"""Maps candidate extension ids → the active set / activation profile for a context."""
def __init__(self, provider: ActivationProvider) -> None:
self.provider = provider
def active_extensions(
self, candidate_ids: Iterable[str], context: ActivationContext
) -> set[str]:
ctx = context.as_dict()
return {eid for eid in candidate_ids if self.provider.is_active(eid, ctx)}
def activation_profile(
self, candidate_ids: Iterable[str], context: ActivationContext
) -> dict[str, Mapping[str, Any]]:
"""``{extension id → config}`` for the active subset."""
ctx = context.as_dict()
return {
eid: self.provider.config(eid, ctx)
for eid in candidate_ids
if self.provider.is_active(eid, ctx)
}
def feature_control_provider(domain: str | None = None) -> ActivationProvider | None:
"""Return a feature-control-backed provider if ``feature_control_sdk`` is importable, else
``None`` (caller falls back to :class:`StaticProvider`). Lazy import keeps the engine core
dependency-free (ADR-0001)."""
try: # optional engine extra — not a core dependency
from feature_control_sdk import FeatureControlClient # type: ignore
except ImportError:
return None
client = FeatureControlClient(domain=domain)
@dataclass(frozen=True, slots=True)
class _FeatureControlProvider:
_client: Any
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
return bool(
self._client.get_boolean_value(feature_key, False, context=dict(context))
)
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
getter = getattr(self._client, "get_object_value", None)
return dict(getter(feature_key, {}, context=dict(context))) if getter else {}
return _FeatureControlProvider(client)

View File

@@ -0,0 +1,82 @@
"""EngineShardAdapter — the engine exposed as a canonical-mode shard (WikiEngineCoreArchitecture
§6, E-1/E-5).
The engine is *one shard*: the orchestrator consumes it only through this `ShardAdapter`. The
adapter is backed by the kernel (T1) + a composed extension set (T2/T3); its §A capability
profile is **derived from the active extensions** (T4), so the orchestrator sees an honest,
conformance-verifiable profile that reflects exactly what is activated. Read/write run the
extensions' transform hooks; everything above this stays in the orchestrator (no union/projection
import).
"""
from __future__ import annotations
from collections.abc import Iterable
from shard_wiki.adapters import ShardAdapter
from shard_wiki.engine.activation import ActivationContext, ActivationProvider, ActivationResolver
from shard_wiki.engine.extension import ActiveExtensions, ExtensionRuntime, Hook
from shard_wiki.engine.kernel import EngineKernel
from shard_wiki.engine.profile import derive_profile
from shard_wiki.model import CapabilityProfile, NotSupported, Page, Verb
__all__ = ["EngineShardAdapter", "build_engine_shard"]
class EngineShardAdapter(ShardAdapter):
def __init__(
self,
kernel: EngineKernel,
active: ActiveExtensions,
base_profile: CapabilityProfile | None = None,
) -> None:
self._kernel = kernel
self._active = active
self._profile = derive_profile(active, base_profile) # validated (E-5)
@property
def shard_id(self) -> str:
return self._kernel.shard_id
def profile(self) -> CapabilityProfile:
return self._profile
def keys(self) -> Iterable[str]:
return self._kernel.keys()
def read(self, key: str) -> Page:
page = self._kernel.read(key)
return self._active.dispatch_transform(Hook.ON_READ, page, {"shard_id": self.shard_id})
def current_rev(self, key: str) -> str | None:
return self._kernel.current_rev(key)
def write(self, key: str, body: str) -> Page:
if not self._profile.supports(Verb.WRITE):
raise NotSupported(f"{type(self).__name__} ({self.shard_id}) is read-only")
body = self._active.dispatch_transform(
Hook.ON_WRITE, body, {"shard_id": self.shard_id, "key": key}
)
return self._kernel.write(key, body)
def build_engine_shard(
shard_id: str,
runtime: ExtensionRuntime,
*,
activate: Iterable[str] | None = None,
provider: ActivationProvider | None = None,
context: ActivationContext | None = None,
base_profile: CapabilityProfile | None = None,
) -> EngineShardAdapter:
"""Stand up an engine shard: resolve which extensions are active (explicit ``activate`` ids,
or via an activation ``provider`` over the runtime's available set), compose them, and wrap a
fresh kernel as a `ShardAdapter`.
"""
if provider is not None:
ctx = context or ActivationContext(shard_id)
ids = ActivationResolver(provider).active_extensions(runtime.available(), ctx)
else:
ids = set(activate or ())
active = runtime.activate(ids)
return EngineShardAdapter(EngineKernel(shard_id), active, base_profile)

View File

@@ -0,0 +1,165 @@
"""Typed-extension runtime — the engine framework (WikiEngineCoreArchitecture §4, E-3/E-9).
Everything beyond the kernel's c2-minimum is an :class:`Extension`: it declares a typed
contract (id, provided capabilities, declared types, bound hooks, dependencies, conflicts) and
the runtime **composes** an activation set deterministically, **rejecting impossible profiles**
(unmet deps / conflicts / type collisions) — the §6.5 capability-as-data discipline applied to
extensions. Extension structure is **verified at registration** (mirrors §6.6 conformance):
bad ids or non-callable hook handlers are refused, so the framework acts on verified data.
Hooks are dispatched in a declared, deterministic order (dependency-topological, ties by id):
*transform* hooks chain a payload through handlers; *collect* hooks gather contributions.
"""
from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping
from enum import Enum
from typing import Any, ClassVar
__all__ = ["Hook", "Extension", "ExtensionError", "ExtensionRuntime", "ActiveExtensions"]
class ExtensionError(ValueError):
"""Raised when an extension is malformed or an activation set is impossible (§6.5)."""
class Hook(Enum):
# transform hooks: each handler takes (payload, ctx) and returns the next payload
ON_WRITE = "on_write" # transform a draft before persist
ON_READ = "on_read" # transform a page on read
ON_RESOLVE = "on_resolve" # transform a name resolution
ON_RENDER = "on_render" # produce a derived representation
# collect hooks: each handler takes (payload, ctx) and returns a contribution
ON_LINK = "on_link" # contribute link/transclusion edges
ON_QUERY = "on_query" # answer a query
ON_PROFILE = "on_profile" # contribute capability-profile positions (E-5)
_TRANSFORM = frozenset({Hook.ON_WRITE, Hook.ON_READ, Hook.ON_RESOLVE, Hook.ON_RENDER})
_COLLECT = frozenset({Hook.ON_LINK, Hook.ON_QUERY, Hook.ON_PROFILE})
class Extension:
"""Base class for a typed extension. Subclasses set the class vars and override
:meth:`hooks` to bind handlers (signature ``handler(payload, ctx) -> result``)."""
id: ClassVar[str] = ""
provides: ClassVar[tuple[str, ...]] = ()
declares_types: ClassVar[tuple[str, ...]] = ()
depends_on: ClassVar[tuple[str, ...]] = ()
conflicts_with: ClassVar[tuple[str, ...]] = ()
def hooks(self) -> Mapping[Hook, Callable[[Any, Any], Any]]:
return {}
class ActiveExtensions:
"""A composed, ordered activation set with deterministic hook dispatch."""
def __init__(self, ordered: list[Extension]) -> None:
self._ordered = ordered
self.ids: tuple[str, ...] = tuple(e.id for e in ordered)
self._tables: dict[Hook, list[tuple[str, Callable[[Any, Any], Any]]]] = {}
for ext in ordered:
for hook, fn in ext.hooks().items():
self._tables.setdefault(hook, []).append((ext.id, fn))
def handlers(self, hook: Hook) -> tuple[str, ...]:
"""The extension ids bound to ``hook``, in dispatch order (for introspection)."""
return tuple(eid for eid, _ in self._tables.get(hook, ()))
def dispatch_transform(self, hook: Hook, payload: Any, ctx: Any = None) -> Any:
if hook not in _TRANSFORM:
raise ExtensionError(f"{hook} is not a transform hook")
for _eid, fn in self._tables.get(hook, ()):
payload = fn(payload, ctx)
return payload
def dispatch_collect(self, hook: Hook, payload: Any = None, ctx: Any = None) -> list[Any]:
if hook not in _COLLECT:
raise ExtensionError(f"{hook} is not a collect hook")
return [fn(payload, ctx) for _eid, fn in self._tables.get(hook, ())]
class ExtensionRuntime:
def __init__(self) -> None:
self._registered: dict[str, Extension] = {}
def available(self) -> frozenset[str]:
"""Ids of all registered extensions (the candidate set for activation)."""
return frozenset(self._registered)
def register(self, ext: Extension) -> Extension:
"""Register an extension after structural verification (mirrors §6.6)."""
if not ext.id or not ext.id.startswith("ext."):
raise ExtensionError(f"extension id must be 'ext.<name>', got {ext.id!r}")
if ext.id in self._registered:
raise ExtensionError(f"duplicate extension id: {ext.id}")
bound = ext.hooks()
for hook, fn in bound.items():
if not isinstance(hook, Hook):
raise ExtensionError(f"{ext.id}: hook key {hook!r} is not a Hook")
if not callable(fn):
raise ExtensionError(f"{ext.id}: handler for {hook} is not callable")
self._registered[ext.id] = ext
return ext
def activate(self, ids: Iterable[str]) -> ActiveExtensions:
"""Compose an activation set: dependency closure → conflict/type checks → deterministic
order. Raises :class:`ExtensionError` on an impossible profile."""
requested = set(ids)
unknown = requested - self._registered.keys()
if unknown:
raise ExtensionError(f"unknown extensions: {sorted(unknown)}")
# dependency closure
active: set[str] = set()
frontier = list(requested)
while frontier:
eid = frontier.pop()
if eid in active:
continue
ext = self._registered.get(eid)
if ext is None:
raise ExtensionError(f"unmet dependency: {eid}")
active.add(eid)
frontier.extend(d for d in ext.depends_on if d not in active)
exts = [self._registered[e] for e in active]
# conflicts
for ext in exts:
clash = active & set(ext.conflicts_with)
if clash:
raise ExtensionError(f"{ext.id} conflicts with active {sorted(clash)}")
# type collisions (two active extensions claiming the same type id)
owner: dict[str, str] = {}
for ext in exts:
for t in ext.declares_types:
if t in owner:
raise ExtensionError(
f"type collision on {t!r}: {owner[t]} and {ext.id}"
)
owner[t] = ext.id
return ActiveExtensions(self._topo_order(exts))
def _topo_order(self, exts: list[Extension]) -> list[Extension]:
"""Dependencies before dependents; ties broken by id (deterministic)."""
by_id = {e.id: e for e in exts}
ordered: list[Extension] = []
placed: set[str] = set()
def visit(ext: Extension) -> None:
if ext.id in placed:
return
for dep in sorted(d for d in ext.depends_on if d in by_id):
visit(by_id[dep])
placed.add(ext.id)
ordered.append(ext)
for ext in sorted(exts, key=lambda e: e.id):
visit(ext)
return ordered

View File

@@ -0,0 +1,10 @@
"""engine/extensions/ — built-in typed extensions for the wiki engine.
Each is a typed :class:`~shard_wiki.engine.extension.Extension` a shard activates only if needed.
``ext.struct`` (typed records) is the first; more (views, addressing, computational, authz) follow
the same pattern.
"""
from shard_wiki.engine.extensions.struct import StructExt, parse_frontmatter
__all__ = ["StructExt", "parse_frontmatter"]

View File

@@ -0,0 +1,81 @@
"""ext.struct — typed records, a first built-in extension (WikiEngineCoreArchitecture X-STRUCT).
Demonstrates the typed-extension framework end-to-end. A page may carry a leading in-text
frontmatter block (`---` … `---`, `key: value` lines — git-diffable structure, blueprint T12).
With this extension **active**, the engine:
- **ON_WRITE** validates the structured block (optionally against an allowed-field set) — a
malformed/disallowed structured page is rejected; the body is otherwise unchanged
(content-preserving, so write conformance holds);
- **ON_READ** tags such pages as `PageShape.TYPED_RECORD`;
- **ON_PROFILE** raises the shard's profile with the `structured-payload` verb (E-5).
With the extension **inactive**, the kernel treats the same page as opaque prose — the feature
is genuinely absent (honest profile). This is "activate only what you need" in action.
"""
from __future__ import annotations
import dataclasses
from collections.abc import Iterable, Mapping
from typing import Any
from shard_wiki.engine.extension import Extension, Hook
from shard_wiki.engine.profile import ProfileContribution
from shard_wiki.model import Page, PageShape, Verb
__all__ = ["StructExt", "parse_frontmatter"]
def parse_frontmatter(body: str) -> tuple[dict[str, str], bool]:
"""Parse a leading ``---`` … ``---`` block of ``key: value`` lines.
Returns ``(fields, has_block)``. An unterminated opening ``---`` is *not* a valid block.
"""
lines = body.splitlines()
if not lines or lines[0].strip() != "---":
return {}, False
fields: dict[str, str] = {}
for line in lines[1:]:
if line.strip() == "---":
return fields, True
if ":" in line:
key, _, value = line.partition(":")
fields[key.strip()] = value.strip()
return {}, False # no closing fence → not a frontmatter block
class StructExt(Extension):
id = "ext.struct"
declares_types = ("record",)
provides = ("capability.wiki.page-model",)
def __init__(self, allowed_fields: Iterable[str] | None = None) -> None:
self._allowed: set[str] | None = set(allowed_fields) if allowed_fields is not None else None
def hooks(self) -> Mapping[Hook, Any]:
return {
Hook.ON_WRITE: self._on_write,
Hook.ON_READ: self._on_read,
Hook.ON_PROFILE: self._on_profile,
}
def _on_write(self, body: str, ctx: Any) -> str:
fields, has_block = parse_frontmatter(body)
if has_block and self._allowed is not None:
disallowed = set(fields) - self._allowed
if disallowed:
raise ValueError(f"ext.struct: disallowed fields {sorted(disallowed)}")
return body # structure stays in-text (git-diffable); body unchanged
def _on_read(self, page: Page, ctx: Any) -> Page:
_, has_block = parse_frontmatter(page.body)
return dataclasses.replace(page, shape=PageShape.TYPED_RECORD) if has_block else page
def _on_profile(self, payload: Any, ctx: Any) -> ProfileContribution:
return ProfileContribution(verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD}))
@staticmethod
def fields(body: str) -> dict[str, str]:
"""Parsed structured fields of a page body (empty if it has no frontmatter block)."""
return parse_frontmatter(body)[0]

View File

@@ -0,0 +1,87 @@
"""Engine kernel — the small page-store core (WikiEngineCoreArchitecture §3, EC-1…EC-4).
The irreducible engine: author/read/edit pages (edit = a new version; delete = a recoverable
tombstone — history is the floor, I-10), enumerate keys, and resolve `[[wikilinks]]` (red-link =
an unresolved target). No feature beyond this c2-minimum lives in the kernel; everything else is
a typed extension (E-3).
Storage is intentionally simple here (in-memory version history); the git-IS-store backing
(SHARD-WP-0009/0012) slots in behind the same API later. The kernel reuses the page model and
provenance leaf; it does not redefine them.
"""
from __future__ import annotations
from collections.abc import Iterable
from datetime import datetime, timezone
from shard_wiki.engine.links import extract_wikilinks
from shard_wiki.model import Identity, Page, Placement
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
__all__ = ["EngineKernel"]
class EngineKernel:
"""An in-process page store with per-page version history for one engine shard."""
def __init__(self, shard_id: str) -> None:
self.shard_id = shard_id
self._versions: dict[str, list[Page]] = {}
self._deleted: set[str] = set()
# --- write path (create/edit are one operation; both append a version) ---
def write(self, key: str, body: str) -> Page:
versions = self._versions.setdefault(key, [])
rev = str(len(versions) + 1)
page = Page(
identity=Identity(self.shard_id, key),
body=body,
envelope=ProvenanceEnvelope(
source_shard=self.shard_id,
liveness=Liveness.STATIC,
staleness=Staleness.FRESH,
source_rev=rev,
observed_at=datetime.now(tz=timezone.utc),
),
placements=(Placement(self.shard_id, key),),
)
versions.append(page)
self._deleted.discard(key)
return page
# --- read path ---
def exists(self, key: str) -> bool:
return key in self._versions and key not in self._deleted
def read(self, key: str) -> Page:
"""Latest version of a live page. Raises ``KeyError`` if absent or deleted."""
if not self.exists(key):
raise KeyError(key)
return self._versions[key][-1]
def keys(self) -> Iterable[str]:
return (k for k in sorted(self._versions) if k not in self._deleted)
def current_rev(self, key: str) -> str | None:
return self._versions[key][-1].envelope.source_rev if self.exists(key) else None
# --- history & recoverability (I-10): versions are retained across delete ---
def history(self, key: str) -> tuple[Page, ...]:
"""All versions ever written for ``key`` (oldest→newest), even after delete."""
return tuple(self._versions.get(key, ()))
def delete(self, key: str) -> None:
"""Tombstone a page (history retained; restore by writing again)."""
if key not in self._versions:
raise KeyError(key)
self._deleted.add(key)
# --- links (EC-4): resolution + red-link detection within this shard ---
def links(self, key: str) -> list[str]:
"""Wikilink targets in a page's current body."""
return extract_wikilinks(self.read(key).body)
def resolve_link(self, target: str) -> Identity | None:
"""Resolve a wikilink target to a live page identity, or ``None`` (a **red-link**)."""
return self.read(target).identity if self.exists(target) else None

View File

@@ -0,0 +1,25 @@
"""Wikilink extraction — the kernel's link primitive (WikiEngineCoreArchitecture EC-4).
`[[Target]]` and `[[Target|label]]`. CamelCase auto-linking is intentionally NOT here (it is an
opt-in concern per FederationRequirements ADR-06); the kernel only knows explicit wikilinks.
Link *resolution* (and red-link detection) is the kernel's job (it knows which keys exist);
*rendering* is a consumer concern (headless engine, no UI).
"""
from __future__ import annotations
import re
__all__ = ["extract_wikilinks"]
_WIKILINK = re.compile(r"\[\[([^\]|]+?)(?:\|[^\]]*)?\]\]")
def extract_wikilinks(body: str) -> list[str]:
"""Return the ordered, de-duplicated wikilink targets in ``body`` (label part dropped)."""
seen: dict[str, None] = {}
for m in _WIKILINK.finditer(body):
target = m.group(1).strip()
if target:
seen.setdefault(target, None)
return list(seen)

View File

@@ -0,0 +1,112 @@
"""Capability profile derived from active extensions (WikiEngineCoreArchitecture E-5).
The engine's §A `CapabilityProfile` is **computed**, not hand-set: start from the kernel base
profile, then fold each active extension's `on_profile` contribution (in the runtime's
deterministic order), then `validate()`. This realizes the chain *configuration (which
extensions) → capability (the profile) → conformance* — activating an extension raises the
shard's advertised capabilities, and composition can never yield an impossible profile (validate
rejects it, §6.5).
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass
from shard_wiki.engine.extension import ActiveExtensions, Hook
from shard_wiki.model import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
MergeModel,
NativeQuery,
OperationalEnvelope,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.provenance import Liveness
__all__ = ["engine_base_profile", "ProfileContribution", "derive_profile"]
# Profile fields an extension may *raise* via on_profile (substrate/attachment are kernel-fixed).
_OVERRIDABLE = (
"write_granularity",
"content_opacity",
"liveness",
"history",
"merge_model",
"addressing",
"native_query",
"translation",
"access_grant",
)
def engine_base_profile() -> CapabilityProfile:
"""The kernel-only (no extensions) capability profile — the c2-minimum engine shard."""
return CapabilityProfile(
substrate=Substrate.FILES,
attachment_mode=AttachmentMode.IN_ENGINE_HOST,
write_granularity=WriteGranularity.PER_PAGE,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.INTERNAL_ONLY,
merge_model=MergeModel.NONE,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset({Verb.READ, Verb.WRITE}),
).validate()
@dataclass(frozen=True, slots=True)
class ProfileContribution:
"""An extension's contribution to the derived profile (returned from its ON_PROFILE hook).
A non-``None`` axis overrides that axis; ``verbs_add`` are unioned in. Order = the runtime's
deterministic dispatch order, so later extensions win on a contested axis."""
write_granularity: WriteGranularity | None = None
content_opacity: ContentOpacity | None = None
liveness: Liveness | None = None
history: History | None = None
merge_model: MergeModel | None = None
addressing: Addressing | None = None
native_query: NativeQuery | None = None
translation: Translation | None = None
access_grant: AccessGrant | None = None
verbs_add: frozenset[Verb] = frozenset()
def derive_profile(
active: ActiveExtensions, base: CapabilityProfile | None = None
) -> CapabilityProfile:
"""Fold active extensions' ON_PROFILE contributions onto ``base`` and validate the result.
Raises :class:`~shard_wiki.model.ProfileError` if the composed profile is impossible — so an
activation set can never advertise an invalid capability profile.
"""
profile = base or engine_base_profile()
contributions = active.dispatch_collect(Hook.ON_PROFILE)
overrides: dict[str, object] = {}
verbs: set[Verb] = set(profile.supported_verbs)
for contrib in contributions:
if not isinstance(contrib, ProfileContribution):
continue
for field_name in _OVERRIDABLE:
value = getattr(contrib, field_name)
if value is not None:
overrides[field_name] = value
verbs |= set(contrib.verbs_add)
return dataclasses.replace(
profile, supported_verbs=frozenset(verbs), **overrides
).validate()

View File

@@ -0,0 +1,46 @@
"""incremental/ — the incremental-first derived tier (CoreArchitectureBlueprint §8.7).
Equivalence is **indexed** (blocking/LSH + verify), not pairwise O(N²); maintenance is
**change-driven** (delta with retraction + propagation, review B-4), keeping the derived tier equal
to a from-scratch rebuild — which becomes a bounded fallback, not the operational path. A
Merkle-style **digest** plus a background **consistency-checker** make ``derived = f(canonical)``
verified rather than asserted (I-2), self-healing on detected drift.
In-memory only for this slice (no persisted index store); per-partition structure is honoured but
multi-tenant deployment is later. Per the dependency rule this imports down (model/provenance) and
is wired by the orchestrator.
"""
from shard_wiki.incremental.equivalence import (
EquivalenceEdge,
EquivalenceIndex,
normalized_title,
)
from shard_wiki.incremental.minhash import (
MinHasher,
band_keys,
jaccard,
shingles,
)
from shard_wiki.incremental.union_index import UnionIndex
from shard_wiki.incremental.verification import (
ConsistencyChecker,
ConsistencyReport,
derived_digest,
region_digest,
)
__all__ = [
"shingles",
"MinHasher",
"band_keys",
"jaccard",
"EquivalenceEdge",
"EquivalenceIndex",
"normalized_title",
"derived_digest",
"region_digest",
"ConsistencyReport",
"ConsistencyChecker",
"UnionIndex",
]

View File

@@ -0,0 +1,225 @@
"""Indexed equivalence — blocking + verify, incrementally maintained (SHARD-WP-0011 T1/T2).
Equivalence (two *distinct* identities holding the same page) is detected without pairwise O(N²):
1. **Blocking** generates candidate pairs — pages sharing a normalized-title bucket or an LSH band
(MinHash over content shingles).
2. **Verify** confirms a candidate — exact-body fingerprint match, or shingle Jaccard ≥ threshold —
plus **curator bindings** (explicit decision-log edges) which are always equivalence edges.
The index is **incrementally maintained** (T2): ``add`` / ``update`` / ``remove`` re-bucket the
changed page, **retract** the edges it leaves and **add** the edges it enters; equivalence groups
are the connected components of the current edge set, so a retraction that disconnects a component
**splits** a chorus automatically. A full :meth:`build` is just repeated ``add`` — the bounded
rebuild fallback. The invariant (and the test oracle): incremental state == a from-scratch rebuild.
"""
from __future__ import annotations
import hashlib
import re
from collections.abc import Iterable
from dataclasses import dataclass
from shard_wiki.incremental.minhash import MinHasher, band_keys, jaccard, shingles
from shard_wiki.model import Identity, Page
__all__ = ["EquivalenceEdge", "EquivalenceIndex", "normalized_title"]
_NONALNUM_RE = re.compile(r"[^a-z0-9]+")
def normalized_title(key: str) -> str:
"""A blocking bucket key: the last path segment, lowercased, stripped of non-alphanumerics."""
leaf = key.rsplit("/", 1)[-1]
return _NONALNUM_RE.sub("", leaf.lower())
@dataclass(frozen=True, slots=True)
class EquivalenceEdge:
"""A verified equivalence between two identities, tagged with why it was accepted."""
a: Identity
b: Identity
reason: str # "fingerprint" | "content" | "curator"
@dataclass(frozen=True, slots=True)
class _Entry:
shingle_set: frozenset[str]
bands: tuple[tuple[int, tuple[int, ...]], ...]
title: str
fingerprint: str
def _fingerprint(body: str) -> str:
return hashlib.blake2b(body.strip().encode("utf-8"), digest_size=16).hexdigest()
def _pair(a: Identity, b: Identity) -> frozenset[Identity]:
return frozenset((a, b))
class EquivalenceIndex:
"""An incrementally maintained, blocked-and-verified equivalence relation over union pages."""
def __init__(
self,
*,
num_perm: int = 64,
num_bands: int = 32,
threshold: float = 0.7,
hasher: MinHasher | None = None,
) -> None:
self.threshold = threshold
self.num_bands = num_bands
self._hasher = hasher or MinHasher(num_perm=num_perm)
self._entries: dict[Identity, _Entry] = {}
self._band_buckets: dict[tuple[int, tuple[int, ...]], set[Identity]] = {}
self._title_buckets: dict[str, set[Identity]] = {}
self._content_edges: dict[frozenset[Identity], str] = {}
self._curator_edges: set[frozenset[Identity]] = set()
# -- build / maintain ----------------------------------------------------
def build(
self,
pages: Iterable[Page],
curator_edges: Iterable[tuple[Identity, Identity]] = (),
) -> None:
"""Rebuild from scratch (the bounded fallback): add every page, then curator edges."""
self.__init__(
num_bands=self.num_bands, threshold=self.threshold, hasher=self._hasher
)
for page in pages:
self.add(page)
for a, b in curator_edges:
self.bind(a, b)
def add(self, page: Page) -> None:
"""Index a new (or, via :meth:`update`, refreshed) page and add its equivalence edges."""
identity = page.identity
entry = self._make_entry(page)
self._entries[identity] = entry
for key in entry.bands:
self._band_buckets.setdefault(key, set()).add(identity)
self._title_buckets.setdefault(entry.title, set()).add(identity)
for candidate in self._candidates(identity, entry):
reason = self._verify(identity, candidate)
if reason is not None:
self._content_edges[_pair(identity, candidate)] = reason
def remove(self, identity: Identity) -> None:
"""Drop a page: de-bucket it and retract every content edge incident to it."""
entry = self._entries.pop(identity, None)
if entry is None:
return
for key in entry.bands:
self._discard_bucket(self._band_buckets, key, identity)
self._discard_bucket(self._title_buckets, entry.title, identity)
for edge in [e for e in self._content_edges if identity in e]:
del self._content_edges[edge]
def update(self, page: Page) -> None:
"""Apply a change as retract-then-add: stale (bucket-exit) edges go, new edges arrive."""
self.remove(page.identity)
self.add(page)
def bind(self, a: Identity, b: Identity) -> None:
"""Record a curator equivalence (an explicit decision-log binding); always an edge."""
if a != b:
self._curator_edges.add(_pair(a, b))
def unbind(self, a: Identity, b: Identity) -> None:
self._curator_edges.discard(_pair(a, b))
def set_curator_edges(self, edges: Iterable[tuple[Identity, Identity]]) -> None:
"""Replace all curator edges at once (re-syncing from the decision-log fold)."""
self._curator_edges = {_pair(a, b) for a, b in edges if a != b}
# -- queries -------------------------------------------------------------
def identities(self) -> frozenset[Identity]:
"""All identities currently present in the index."""
return frozenset(self._entries)
def fingerprint(self, identity: Identity) -> str | None:
"""The content fingerprint indexed for ``identity`` (None if absent) — a digest leaf."""
entry = self._entries.get(identity)
return entry.fingerprint if entry is not None else None
def edges(self) -> frozenset[frozenset[Identity]]:
"""All equivalence edges (content + curator) among currently present identities."""
present = self._entries.keys()
curator = {e for e in self._curator_edges if e <= present}
return frozenset(set(self._content_edges) | curator)
def groups(self) -> tuple[frozenset[Identity], ...]:
"""Equivalence groups: connected components of size ≥ 2 (union-find over the edges)."""
parent: dict[Identity, Identity] = {}
def find(x: Identity) -> Identity:
parent.setdefault(x, x)
root = x
while parent[root] != root:
root = parent[root]
while parent[x] != root:
parent[x], x = root, parent[x]
return root
for edge in self.edges():
a, b = tuple(edge)
ra, rb = find(a), find(b)
if ra != rb:
parent[ra] = rb
comps: dict[Identity, set[Identity]] = {}
for node in parent:
comps.setdefault(find(node), set()).add(node)
return tuple(
frozenset(members) for members in comps.values() if len(members) > 1
)
def equivalent_to(self, identity: Identity) -> frozenset[Identity]:
"""The equivalence group containing ``identity`` (including itself), else just itself."""
for group in self.groups():
if identity in group:
return group
return frozenset({identity})
# -- internals -----------------------------------------------------------
def _make_entry(self, page: Page) -> _Entry:
shingle_set = shingles(page.body)
signature = self._hasher.signature(shingle_set)
return _Entry(
shingle_set=shingle_set,
bands=band_keys(signature, self.num_bands),
title=normalized_title(page.identity.key),
fingerprint=_fingerprint(page.body),
)
def _candidates(self, identity: Identity, entry: _Entry) -> set[Identity]:
candidates: set[Identity] = set()
for key in entry.bands:
candidates |= self._band_buckets.get(key, set())
candidates |= self._title_buckets.get(entry.title, set())
candidates.discard(identity)
return candidates
def _verify(self, a: Identity, b: Identity) -> str | None:
ea, eb = self._entries[a], self._entries[b]
if ea.fingerprint == eb.fingerprint:
return "fingerprint"
if jaccard(ea.shingle_set, eb.shingle_set) >= self.threshold:
return "content"
return None
@staticmethod
def _discard_bucket(buckets: dict, key, identity: Identity) -> None:
bucket = buckets.get(key)
if bucket is not None:
bucket.discard(identity)
if not bucket:
del buckets[key]

View File

@@ -0,0 +1,71 @@
"""MinHash + LSH banding primitives for content-similarity blocking (SHARD-WP-0011 T1).
Pure, deterministic functions (fixed hashing, no per-run randomness) so the derived tier and its
digest are reproducible. Shingle a body into k-grams, MinHash the shingle set into a signature,
split the signature into LSH bands; two pages sharing a band are *candidates* for equivalence —
the cheap pre-filter that replaces pairwise O(N²) comparison.
"""
from __future__ import annotations
import hashlib
import random
import re
from collections.abc import Iterable
__all__ = ["shingles", "MinHasher", "band_keys", "jaccard"]
_WORD_RE = re.compile(r"\w+")
# Largest Mersenne prime below 2**61 — the modulus for the universal-hash permutations.
_PRIME = (1 << 61) - 1
def shingles(text: str, k: int = 3) -> frozenset[str]:
"""The set of word k-grams in ``text`` (lowercased). Short texts fall back to their word set."""
words = _WORD_RE.findall(text.lower())
if len(words) < k:
return frozenset(words)
return frozenset(" ".join(words[i : i + k]) for i in range(len(words) - k + 1))
def _stable_hash(token: str) -> int:
return int.from_bytes(hashlib.blake2b(token.encode("utf-8"), digest_size=8).digest(), "big")
class MinHasher:
"""A bank of ``num_perm`` universal hash permutations producing a fixed-length signature."""
def __init__(self, num_perm: int = 64, seed: int = 1) -> None:
self.num_perm = num_perm
rng = random.Random(seed)
self._coeffs = [
(rng.randrange(1, _PRIME), rng.randrange(0, _PRIME)) for _ in range(num_perm)
]
def signature(self, shingle_set: Iterable[str]) -> tuple[int, ...]:
"""The MinHash signature of ``shingle_set`` (empty set → all-``_PRIME`` sentinel)."""
hashed = [_stable_hash(s) for s in shingle_set]
if not hashed:
return tuple(_PRIME for _ in self._coeffs)
return tuple(min((a * h + b) % _PRIME for h in hashed) for a, b in self._coeffs)
def band_keys(
signature: tuple[int, ...], num_bands: int
) -> tuple[tuple[int, tuple[int, ...]], ...]:
"""Split a signature into ``num_bands`` band keys; two pages sharing one are LSH candidates."""
if num_bands <= 0 or len(signature) % num_bands != 0:
raise ValueError(f"signature length {len(signature)} not divisible into {num_bands} bands")
rows = len(signature) // num_bands
return tuple(
(b, signature[b * rows : (b + 1) * rows]) for b in range(num_bands)
)
def jaccard(a: frozenset[str], b: frozenset[str]) -> float:
"""Jaccard similarity of two shingle sets; two empty sets are defined as identical (1.0)."""
if not a and not b:
return 1.0
if not a or not b:
return 0.0
return len(a & b) / len(a | b)

View File

@@ -0,0 +1,91 @@
"""UnionIndex — the maintained derived tier wired behind resolution + views (SHARD-WP-0011 T4).
Wraps a :class:`UnionGraph` + decision log with an incrementally maintained
:class:`EquivalenceIndex`. Content equivalence is kept fresh by deltas (``note_change`` /
``note_removed``); curator bindings are re-synced live from the log fold. A full :meth:`rebuild`
is the bounded fallback. :meth:`verify` runs the I-2 consistency-checker over the live source.
Consumer-visible results are unchanged — equivalence groups are exposed in the same string form the
decision-log fold uses, a *superset* that additionally collapses genuine content duplicates — only
freshness and cost differ (recompute-on-read becomes change-driven).
"""
from __future__ import annotations
from shard_wiki.coordination import DecisionLog
from shard_wiki.incremental.equivalence import EquivalenceIndex
from shard_wiki.incremental.verification import (
ConsistencyChecker,
ConsistencyReport,
derived_digest,
)
from shard_wiki.model import Identity, Page
from shard_wiki.union import UnionGraph
__all__ = ["UnionIndex"]
def _identity(token: str) -> Identity:
shard, _, key = token.partition(":")
return Identity(shard, key)
class UnionIndex:
"""An incrementally maintained equivalence index over a union, with a rebuild fallback."""
def __init__(self, union: UnionGraph, log: DecisionLog, space: str) -> None:
self._union = union
self._log = log
self._space = space
self._eq = EquivalenceIndex()
self.rebuild()
def rebuild(self) -> None:
"""The bounded fallback: re-derive the whole index from current union pages + bindings."""
self._eq.build(self._union.iter_pages())
self._sync_curator()
def note_change(self, page: Page) -> None:
"""Change-driven update for one added/edited page (the operational path)."""
self._eq.update(page)
def note_removed(self, identity: Identity) -> None:
self._eq.remove(identity)
def _sync_curator(self) -> None:
"""Re-sync curator equivalence from the live decision-log fold (cheap, always correct)."""
groups = self._log.fold(self._space).equivalence_groups
edges: list[tuple[Identity, Identity]] = []
for group in groups:
members = [_identity(m) for m in group]
edges.extend((members[0], other) for other in members[1:])
self._eq.set_curator_edges(edges)
def equivalence_groups(self) -> tuple[frozenset[str], ...]:
"""Equivalence groups in decision-log string form (curator content), for the views."""
self._sync_curator()
return tuple(
frozenset(str(identity) for identity in group) for group in self._eq.groups()
)
def digest(self) -> str:
"""The Merkle-style digest of the maintained derived tier (I-2)."""
self._sync_curator()
return derived_digest(self._eq)
def verify(self) -> ConsistencyReport:
"""Check the maintained index against a from-scratch fold of the live source; self-heal."""
self._sync_curator()
checker = ConsistencyChecker(
self._eq,
pages=lambda: list(self._union.iter_pages()),
curator_edges=self._curator_pairs,
)
return checker.check_and_repair()
def _curator_pairs(self) -> list[tuple[Identity, Identity]]:
pairs: list[tuple[Identity, Identity]] = []
for group in self._log.fold(self._space).equivalence_groups:
members = [_identity(m) for m in group]
pairs.extend((members[0], other) for other in members[1:])
return pairs

View File

@@ -0,0 +1,112 @@
"""I-2 verification — digest + background consistency-checker (SHARD-WP-0011 T3).
``derived = f(canonical)`` is made *verified*, not asserted. A **Merkle-style digest** summarizes
the derived tier (each identity's content fingerprint + its incident equivalence edges as a leaf,
order-independently combined into a root) so two derived states are equal iff their digests match.
A **consistency-checker** recomputes the authoritative fold from the current source, compares it to
the maintained index over a (sampled) region, and on mismatch performs a **scoped recompute** of
just the affected identities — self-healing drift from a missed delta or corrupted state.
The digest is a pure function of index state, so it is "maintained alongside deltas" for free and
is stable under equivalent event orders (leaves are sorted before combination).
"""
from __future__ import annotations
import hashlib
from collections.abc import Callable, Iterable
from dataclasses import dataclass
from shard_wiki.incremental.equivalence import EquivalenceIndex
from shard_wiki.model import Identity, Page
__all__ = ["region_digest", "derived_digest", "ConsistencyReport", "ConsistencyChecker"]
CuratorEdges = Iterable[tuple[Identity, Identity]]
def _leaf(index: EquivalenceIndex, identity: Identity) -> str:
"""A digest leaf for one identity: its fingerprint + its incident edges (as sorted peers)."""
fingerprint = index.fingerprint(identity) or ""
peers = sorted(
str(other)
for edge in index.edges()
if identity in edge
for other in edge
if other != identity
)
payload = f"{identity}|{fingerprint}|{','.join(peers)}"
return hashlib.blake2b(payload.encode("utf-8"), digest_size=16).hexdigest()
def region_digest(index: EquivalenceIndex, identities: Iterable[Identity]) -> str:
"""A Merkle-style root over the given identities' leaves (order-independent)."""
leaves = sorted(_leaf(index, identity) for identity in identities)
root = hashlib.blake2b(digest_size=16)
for leaf in leaves:
root.update(leaf.encode("utf-8"))
return root.hexdigest()
def derived_digest(index: EquivalenceIndex) -> str:
"""The digest of the whole maintained derived tier."""
return region_digest(index, index.identities())
@dataclass(frozen=True, slots=True)
class ConsistencyReport:
"""Outcome of a consistency check: what was examined, whether it drifted, and if it healed."""
checked: int
drifted: bool
repaired: bool
healthy: bool
class ConsistencyChecker:
"""Compares the maintained index against an authoritative rebuild and repairs drift in place."""
def __init__(
self,
index: EquivalenceIndex,
pages: Callable[[], Iterable[Page]],
curator_edges: Callable[[], CuratorEdges] = lambda: (),
) -> None:
self._index = index
self._pages = pages
self._curator = curator_edges
def _authoritative(self) -> EquivalenceIndex:
expected = EquivalenceIndex(
num_bands=self._index.num_bands, threshold=self._index.threshold
)
expected.build(list(self._pages()), list(self._curator()))
return expected
def check_and_repair(self, sample: Iterable[Identity] | None = None) -> ConsistencyReport:
"""Verify the (sampled) region against a from-scratch fold; scoped-recompute on mismatch."""
source = {p.identity: p for p in self._pages()}
expected = self._authoritative()
region = (
set(sample)
if sample is not None
else set(source) | set(self._index.identities())
)
drifted = region_digest(self._index, region) != region_digest(expected, region)
if not drifted:
return ConsistencyReport(len(region), drifted=False, repaired=False, healthy=True)
self._repair(region, source)
healthy = region_digest(self._index, region) == region_digest(expected, region)
return ConsistencyReport(len(region), drifted=True, repaired=True, healthy=healthy)
def _repair(self, region: set[Identity], source: dict[Identity, Page]) -> None:
"""Scoped recompute: reconcile each affected identity to the current source."""
present = self._index.identities()
for identity in region:
page = source.get(identity)
if page is not None:
self._index.update(page) if identity in present else self._index.add(page)
elif identity in present:
self._index.remove(identity)

View File

@@ -0,0 +1,47 @@
"""model/ — the backend-neutral page model and capability types (the top waist).
Imports only the ``provenance`` leaf; nothing backend-specific lives here.
"""
from shard_wiki.model.capability import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
MergeModel,
NativeQuery,
NotSupported,
OperationalEnvelope,
ProfileError,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.model.identity import Identity, Placement, Span
from shard_wiki.model.page import Page, PageShape
__all__ = [
"Identity",
"Placement",
"Span",
"Page",
"PageShape",
"CapabilityProfile",
"Verb",
"Substrate",
"AttachmentMode",
"WriteGranularity",
"ContentOpacity",
"OperationalEnvelope",
"AccessGrant",
"History",
"MergeModel",
"Addressing",
"NativeQuery",
"Translation",
"ProfileError",
"NotSupported",
]

View File

@@ -0,0 +1,220 @@
"""Capability profile — capability-as-data (CoreArchitectureBlueprint §6, I-3).
A binding's abilities are a *position on each capability spectrum*, not a boolean checklist.
Descriptively there are fifteen spectra (§6.1); operationally they reduce to a small
**orthogonal core** plus **implied** axes, with **implication rules that forbid impossible
profiles** (§6.5). ``CapabilityProfile.validate()`` enforces those rules; a profile that
violates one is rejected (the structural half of "capability-as-data is sound" — the behavioural
half is the conformance suite, T4).
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from shard_wiki.provenance import Liveness
__all__ = [
"Verb",
"Substrate",
"AttachmentMode",
"WriteGranularity",
"ContentOpacity",
"OperationalEnvelope",
"AccessGrant",
"History",
"MergeModel",
"Addressing",
"NativeQuery",
"Translation",
"CapabilityProfile",
"ProfileError",
"NotSupported",
]
CONTRACT_VERSION = "0.1"
class ProfileError(ValueError):
"""Raised when a capability profile is internally impossible (violates §6.5 rules)."""
class NotSupported(Exception):
"""Raised by an adapter when an operation verb is not in its capability profile."""
class Verb(Enum):
READ = "read"
WRITE = "write"
DIFF = "diff"
MERGE = "merge"
LOCK = "lock"
VERSION = "version"
PUBLISH = "publish"
NOTIFY = "notify"
TRANSCLUDE_SOURCE = "transclude-source"
TRANSLATE_SYNTAX = "translate-syntax"
STRUCTURED_PAYLOAD = "structured-payload"
DERIVE_PROJECTION = "derive-projection" # gated, off by default (T18)
EXECUTE = "execute" # gated, off by default (T18)
# --- orthogonal core axes (§6.5a) ---
class Substrate(Enum):
GIT = "git"
FILES = "files"
RELATIONAL_DB = "relational-db"
EXTERNAL_API = "external-api"
CRDT = "crdt"
class WriteGranularity(Enum):
NONE = "none" # read-only
WHOLE_FILE = "whole-file"
SECTION = "section"
PER_PAGE = "per-page"
PER_BLOCK = "per-block"
class ContentOpacity(Enum):
TRANSPARENT = "transparent"
STRUCTURED_REEVALUABLE = "structured-re-evaluable"
ENCRYPTED = "encrypted"
PER_ITEM = "per-item"
PROPRIETARY_LOSSY = "proprietary-lossy"
class OperationalEnvelope(Enum):
LOCAL_UNBOUNDED = "local-unbounded"
REALTIME = "realtime"
RATE_LIMITED = "rate-limited"
class AccessGrant(Enum):
OPEN = "open"
TOKEN = "token"
OAUTH_SCOPED = "oauth-scoped"
P2P_KEY = "p2p-key"
ENTERPRISE_ACL = "enterprise-acl"
# --- implied axes (§6.5b): may be derived from the core, validated for consistency ---
class History(Enum):
NONE = "none"
INTERNAL_ONLY = "internal-only"
CRDT_LOG = "crdt-log"
OPEN_FILE = "open-file"
GIT_NATIVE = "git-native"
class MergeModel(Enum):
NONE = "none"
GIT_TEXT = "git-text"
KEEP_BOTH = "keep-both"
NATIVE_CRDT = "native-crdt"
COEXIST_RANK = "coexist-rank"
class AttachmentMode(Enum):
# NB: there is deliberately no IMAGE mode — an image/live-memory blob is never an attach
# target (I-12). It participates only via export→files.
FILE_STORE = "file-store"
GIT_IS_STORE = "git-is-store"
IN_ENGINE_HOST = "in-engine-host"
LOCAL_REST = "local-rest"
EXTERNAL_API = "external-api"
DIRECT_DB = "direct-db"
CRDT_REPLICA = "crdt-replica"
P2P = "p2p"
class Addressing(Enum):
NONE = "none"
PATH = "path"
PAGE_ID = "page-id"
SPAN = "span"
BLOCK_ID = "block-id"
STORE_UUID = "store-uuid"
TUMBLER = "tumbler"
class NativeQuery(Enum):
NONE = "none"
TEXT = "text"
DERIVED_INDEX = "derived-index"
DATALOG_GRAPH = "datalog-graph"
DB_QUERY = "db-query"
SPARQL = "sparql"
class Translation(Enum):
NATIVE = "native"
LOSSLESS = "lossless"
LOSSY_WITH_REPORT = "lossy-with-report"
@dataclass(frozen=True, slots=True)
class CapabilityProfile:
"""A verified-at-registration position on each capability axis, plus supported verbs."""
# orthogonal core
substrate: Substrate
attachment_mode: AttachmentMode
write_granularity: WriteGranularity
content_opacity: ContentOpacity
operational_envelope: OperationalEnvelope
access_grant: AccessGrant
liveness: Liveness # computational/liveness axis (15th)
# implied
history: History
merge_model: MergeModel
addressing: Addressing
native_query: NativeQuery
translation: Translation
supported_verbs: frozenset[Verb] = field(default_factory=frozenset)
contract_version: str = CONTRACT_VERSION
def supports(self, verb: Verb) -> bool:
return verb in self.supported_verbs
def validate(self) -> CapabilityProfile:
"""Apply the §6.5 implication rules; raise :class:`ProfileError` on an impossible
combination. Returns ``self`` so it can be used fluently at construction."""
rules: list[tuple[bool, str]] = [
(
Verb.READ in self.supported_verbs,
"every shard must support READ (the capability floor)",
),
(
self.attachment_mode is not AttachmentMode.GIT_IS_STORE
or (self.substrate is Substrate.GIT and self.history is History.GIT_NATIVE),
"git-is-store ⟹ substrate=git ∧ history=git-native",
),
(
self.content_opacity is not ContentOpacity.ENCRYPTED
or self.native_query is NativeQuery.NONE,
"encrypted opacity ⟹ native-query=none",
),
(
self.merge_model is not MergeModel.NATIVE_CRDT
or (
self.history is History.CRDT_LOG
and self.operational_envelope is OperationalEnvelope.REALTIME
),
"native-crdt merge ⟹ history=crdt-log ∧ envelope=realtime",
),
(
(self.write_granularity is WriteGranularity.NONE)
== (Verb.WRITE not in self.supported_verbs),
"write-granularity=none ⟺ WRITE not supported (read-only consistency)",
),
(
Verb.MERGE not in self.supported_verbs or self.merge_model is not MergeModel.NONE,
"MERGE verb ⟹ merge-model ≠ none",
),
]
broken = [msg for ok, msg in rules if not ok]
if broken:
raise ProfileError("; ".join(broken))
return self

View File

@@ -0,0 +1,60 @@
"""Identity, Placement, Span — three distinct concepts (CoreArchitectureBlueprint §7.2).
- **Identity** is a *stable handle* assigned/minted by a shard; it survives edits. It is NEVER
derived from content (a content fingerprint identifies a *version*, not a *page*).
- **Placement** is *where* an identity sits (path in a shard); one identity may have many
placements (a DAG). Placement can change without changing identity.
- **Span** is a sub-page address within an identity, optionally carrying a provenance delta.
Equivalence (detecting that two *distinct* identities hold the same content) is a separate
mechanism and lives in ``union/`` — not here.
"""
from __future__ import annotations
from dataclasses import dataclass
from shard_wiki.provenance import SpanProvenanceDelta
__all__ = ["Identity", "Placement", "Span"]
@dataclass(frozen=True, slots=True)
class Identity:
"""A shard-scoped, stable page handle. Value-equal; stable across content edits.
``shard`` scopes the handle so native ids survive projection and never collide across
shards. ``key`` is the backend's stable handle (page name / native uid) — not a fingerprint.
"""
shard: str
key: str
def __str__(self) -> str:
return f"{self.shard}:{self.key}"
@dataclass(frozen=True, slots=True)
class Placement:
"""Where an identity sits: a path within a shard. Mutable over an identity's life."""
shard: str
path: str
def __str__(self) -> str:
return f"{self.shard}/{self.path}"
@dataclass(frozen=True, slots=True)
class Span:
"""A sub-page address within a page identity, with an optional provenance delta.
``ref`` is a native span id where the backend mints one (Roam ``:block/uid``, Logseq
``id::``), else a positional/fingerprint address. ``delta`` overrides the page envelope
only where this span genuinely differs (effective-vs-own, blueprint §7.3).
"""
ref: str
start: int | None = None
end: int | None = None
delta: SpanProvenanceDelta | None = None

View File

@@ -0,0 +1,43 @@
"""The backend-neutral wiki page model — the top narrow waist (CoreArchitectureBlueprint §7).
Markdown-first but stretchable: every shape reduces to
``(content|source, structure, provenance envelope, [derivation rule])``.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from shard_wiki.model.identity import Identity, Placement, Span
from shard_wiki.provenance import ProvenanceEnvelope
__all__ = ["PageShape", "Page"]
class PageShape(Enum):
"""The page-model shapes (blueprint §7.1). The slice handles PROSE; the rest are declared
so adapters can tag content faithfully (union without erasure)."""
PROSE = "prose"
TYPED_RECORD = "typed-record"
TYPED_GRAPH = "typed-graph"
INLINE_EMBEDDED = "inline-embedded"
NON_MARKDOWN_ASSET = "non-markdown-asset"
ONE_SOURCE_MANY_PROJECTIONS = "one-source-many-projections"
NOTEBOOK = "notebook"
PROGRAM_AS_PAGE = "program-as-page"
LIVE_TEMPORAL = "live-temporal"
@dataclass(frozen=True, slots=True)
class Page:
"""A page in the union: a stable identity, content, the page-level provenance envelope,
its placements, and its spans (whose deltas layer over the page envelope, §7.3)."""
identity: Identity
body: str
envelope: ProvenanceEnvelope
shape: PageShape = PageShape.PROSE
placements: tuple[Placement, ...] = ()
spans: tuple[Span, ...] = field(default_factory=tuple)

View File

@@ -0,0 +1,39 @@
"""policy/ — the configurable policy surface, a dependency-free leaf (blueprint §10, §11).
Mechanism over policy (I-7): core mechanisms read policy *choices* from here; they never
hard-code one. This leaf holds only the presets + a pure ``resolve``-style selector. Mechanism
(how a choice is honoured) lives in ``coordination``/``union``/``projection``, never here.
"""
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
__all__ = ["CanonicalSource", "Policy", "DEFAULT_POLICY"]
class CanonicalSource(Enum):
"""Resolution policy over a divergent/equivalent set (FederationArchitecture T9). Detection
is core; this is only the *resolution* choice."""
CHORUS = "chorus" # default: present all versions, none privileged
DESIGNATED_CANONICAL = "designated-canonical"
GIT_MERGE = "git-merge"
VOTE = "vote"
OVERLAY_ONLY = "overlay-only"
@dataclass(frozen=True, slots=True)
class Policy:
"""A space's policy choices. Extended as the policy surface grows (freshness, compaction,
execution, tenant mapping — blueprint §10); the slice needs canonical-source + designation."""
canonical_source: CanonicalSource = CanonicalSource.CHORUS
designated_shard: str | None = None
# Whether an unapplied overlay's body is projected over its canonical page on read. Either
# way the overlay is never *hidden* — overlay_state is always surfaced in provenance.
show_drafts: bool = True
DEFAULT_POLICY = Policy()

View File

@@ -0,0 +1,116 @@
"""Provenance — the cross-cutting leaf rail (CoreArchitectureBlueprint §7.3, §11).
Every artifact in the union carries a :class:`ProvenanceEnvelope`. To keep per-span cost near
zero when provenance is uniform, envelopes are **layered**: a page-level envelope plus optional
per-span deltas. The *effective* provenance of a span is ``page_envelope ⊕ span_delta`` — a span
with no delta inherits the page envelope at zero cost ("effective-vs-own", the same pattern the
page model uses for computed metadata).
This module is a **dependency-free leaf**: it imports nothing else in the package and contains
only stable data types plus the pure ``effective`` composition. Mechanism never lives here.
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass
from datetime import datetime
from enum import Enum
__all__ = [
"Liveness",
"Staleness",
"OverlayState",
"ProvenanceEnvelope",
"SpanProvenanceDelta",
"effective",
]
class Liveness(Enum):
"""Where a view sits on the live↔snapshot axis (blueprint §8, T18)."""
STATIC = "static"
CAPTURED_SNAPSHOT = "captured-snapshot"
LIVE_OVER_FILES = "live-over-files"
VIEW_TIME = "view-time"
IRREDUCIBLY_LIVE = "irreducibly-live"
class Staleness(Enum):
"""Freshness state of cached/projected content (blueprint §8.8). ``UNAVAILABLE`` is the
dead-shard state (FederationRequirements ADR-04)."""
LIVE = "live"
FRESH = "fresh"
STALE = "stale"
UNAVAILABLE = "unavailable"
class OverlayState(Enum):
"""Overlay lifecycle state of an artifact (FederationRequirements ADR-05)."""
NONE = "none"
DRAFT = "draft"
PATCH_PENDING = "patch-pending"
APPLIED = "applied"
@dataclass(frozen=True, slots=True)
class ProvenanceEnvelope:
"""The metadata every page/span carries — union without erasure (INTENT I-4).
This is the *page-level* (or fully-resolved *effective*) form. Per-span overrides are
expressed as a :class:`SpanProvenanceDelta` and composed via :func:`effective`.
"""
source_shard: str
liveness: Liveness = Liveness.STATIC
staleness: Staleness = Staleness.FRESH
overlay_state: OverlayState = OverlayState.NONE
source_rev: str | None = None
observed_at: datetime | None = None
authz_context: str | None = None
divergence: tuple[str, ...] = ()
lineage: str | None = None
@dataclass(frozen=True, slots=True)
class SpanProvenanceDelta:
"""A per-span override of a page envelope. Every field is optional; ``None`` (or, for
``divergence``, the sentinel default) means *inherit the page-level value*. A span with an
all-default delta — or no delta at all — costs nothing and resolves to the page envelope.
"""
source_shard: str | None = None
liveness: Liveness | None = None
staleness: Staleness | None = None
overlay_state: OverlayState | None = None
source_rev: str | None = None
observed_at: datetime | None = None
authz_context: str | None = None
# Sentinel: ``None`` means inherit; an explicit (possibly empty) tuple overrides.
divergence: tuple[str, ...] | None = None
lineage: str | None = None
def is_empty(self) -> bool:
"""True when the delta overrides nothing (the common, zero-cost case)."""
return all(getattr(self, f.name) is None for f in dataclasses.fields(self))
def effective(
base: ProvenanceEnvelope, delta: SpanProvenanceDelta | None = None
) -> ProvenanceEnvelope:
"""Compose a page envelope with an optional span delta → the span's effective provenance.
``effective(base, None)`` and ``effective(base, <empty delta>)`` both return ``base``
unchanged (zero-cost inheritance). Any non-``None`` delta field overrides ``base``.
"""
if delta is None or delta.is_empty():
return base
overrides = {
f.name: value
for f in dataclasses.fields(delta)
if (value := getattr(delta, f.name)) is not None
}
return dataclasses.replace(base, **overrides)

148
src/shard_wiki/space.py Normal file
View File

@@ -0,0 +1,148 @@
"""InformationSpace — the thin orchestrator entry tying the slice together (blueprint §3 L6).
A root entity / information space: shards attach to it (conformance-gated, TSD §A.2), a
coordination decision log records its canonical decisions, and a derived union resolves names to
pages. This is the L6 consumer surface for the foundation slice (attach → resolve → read);
a network API is a later workplan.
"""
from __future__ import annotations
from pathlib import Path
from shard_wiki.adapters import ShardAdapter, assert_conformant
from shard_wiki.coordination import (
ApplyResult,
DecisionLog,
EventStore,
EventType,
GitEventStore,
Overlay,
OverlayEngine,
)
from shard_wiki.incremental import ConsistencyReport, UnionIndex
from shard_wiki.model import Page
from shard_wiki.policy import DEFAULT_POLICY, Policy
from shard_wiki.union import Resolution, UnionGraph
from shard_wiki.views import (
AllPagesEntry,
BackLink,
ChangeEntry,
SiteMapNode,
all_pages,
build_backlinks,
recent_changes,
site_map,
)
__all__ = ["InformationSpace"]
class InformationSpace:
def __init__(
self,
space_id: str,
policy: Policy = DEFAULT_POLICY,
*,
store: EventStore | None = None,
) -> None:
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
self.space_id = space_id
self.log = DecisionLog(store)
self.union = UnionGraph(space_id, log=self.log, policy=policy)
self.overlays = OverlayEngine(space_id, self.log)
self._index: UnionIndex | None = None # maintained derived tier, built lazily
self._index_stale = True
@classmethod
def git_backed(
cls,
space_id: str,
repo_path: str | Path,
policy: Policy = DEFAULT_POLICY,
) -> InformationSpace:
"""An information space whose coordination log is git-addressable (history/patch/review/
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
return cls(space_id, policy, store=GitEventStore(repo_path))
def attach(self, adapter: ShardAdapter) -> None:
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
assert_conformant(adapter)
self.union.attach(adapter)
self._index_stale = True
def alias(self, name: str, target: str, actor: str | None = None) -> None:
"""Record a coordination-canonical alias (``name`` → ``"shard:key"``) in the log."""
self.log.append(
self.space_id, EventType.ALIAS_SET, {"alias": name, "target": target}, actor=actor
)
def resolve(self, name: str) -> Resolution:
return self.union.resolve(name)
def read(self, name: str) -> Page:
"""Resolve and return the page (or the canonical pick of a chorus). KeyError if red-link."""
return self.union.resolve(name).single()
def overlay(self, name: str, body: str, actor: str | None = None) -> Overlay:
"""Draft a non-destructive overlay against the resolved page (overlay-before-mutation)."""
page = self.read(name)
return self.overlays.draft(page.identity, body, page.envelope.source_rev, actor=actor)
def apply_overlay(self, overlay_id: str) -> ApplyResult:
"""Apply a draft overlay to its target shard (apply-under-drift, §8.6)."""
overlay = self.overlays.get(overlay_id)
if overlay is None:
raise KeyError(overlay_id)
adapter = self.union.shard(overlay.target.shard)
if adapter is None:
raise KeyError(overlay.target.shard)
return self.overlays.apply(overlay_id, adapter)
def edit(self, name: str, body: str, actor: str | None = None) -> ApplyResult:
"""Edit a page through the one principled path: draft an overlay, then apply it. A
write-through-capable target fast-forwards (write-through); a read-only target keeps the
draft as local truth (I-5: overlay before mutation, always)."""
overlay = self.overlay(name, body, actor=actor)
result = self.apply_overlay(overlay.overlay_id)
self._index_stale = True # the applied edit changes the derived tier
return result
# --- maintained derived tier (SHARD-WP-0011): incremental-first, rebuild as fallback ---
@property
def index(self) -> UnionIndex:
"""The maintained equivalence index (built lazily; rebuilt when the union has changed)."""
if self._index is None:
self._index = UnionIndex(self.union, self.log, self.space_id)
elif self._index_stale:
self._index.rebuild() # bounded fallback after a mutation
self._index_stale = False
return self._index
def reindex(self) -> None:
"""Force a full rebuild of the maintained derived tier (the explicit fallback path)."""
self.index.rebuild()
def verify_index(self) -> ConsistencyReport:
"""Run the I-2 consistency-checker over the maintained tier; self-heal any drift."""
return self.index.verify()
# --- derived views (SHARD-WP-0010): recomputable, provenance-carrying, presentation-free ---
def backlinks(self, name: str, *, camelcase: bool = False) -> tuple[BackLink, ...]:
"""Pages across the union that link to ``name`` (UC-18)."""
return build_backlinks(self.union, camelcase=camelcase).to(name)
def recent_changes(self, *, limit: int | None = None) -> tuple[ChangeEntry, ...]:
"""The merged newest-first change feed: coordination journal + shard signals (UC-17)."""
return recent_changes(self.union, self.log, self.space_id, limit=limit)
def all_pages(self) -> tuple[AllPagesEntry, ...]:
"""The union's distinct pages, collapsed via the maintained equivalence index."""
return all_pages(self.union, equivalence_groups=self.index.equivalence_groups())
def site_map(self) -> SiteMapNode:
"""The union namespace tree built from page placements."""
return site_map(self.union)

View File

@@ -0,0 +1,8 @@
"""union/ — the derived-tier union (resolution, equivalence, projection read path).
Disposable/recomputable (I-2); imports down only and is imported by nothing.
"""
from shard_wiki.union.resolver import Resolution, ResolutionKind, UnionGraph
__all__ = ["UnionGraph", "Resolution", "ResolutionKind"]

View File

@@ -0,0 +1,170 @@
"""Union resolution — the derived-tier read path (CoreArchitectureBlueprint §8.4, ADR-01).
A minimal :class:`UnionGraph` over ≥1 attached shard plus the decision-log fold. ``resolve``
keys on **identity** (FederationRequirements ADR-01): alias redirect → exact union lookup →
equivalence chorus → red-link. Ambiguity returns a **chorus set** (union without erasure, I-4):
divergent peers are recorded in each page's provenance envelope rather than silently dropped.
This is derived/disposable: it reads shards + the log fold; it stores nothing canonical. Per the
dependency rule it may import down (model/adapters/coordination/policy/provenance) and is
imported by nothing.
"""
from __future__ import annotations
import dataclasses
from collections.abc import Iterator
from dataclasses import dataclass
from enum import Enum
from shard_wiki.adapters import ShardAdapter
from shard_wiki.coordination import DecisionLog, Overlay
from shard_wiki.model import Identity, Page
from shard_wiki.policy import DEFAULT_POLICY, CanonicalSource, Policy
from shard_wiki.provenance import OverlayState, ProvenanceEnvelope, Staleness
__all__ = ["ResolutionKind", "Resolution", "UnionGraph"]
class ResolutionKind(Enum):
SINGLE = "single"
CHORUS = "chorus"
RED_LINK = "red-link"
@dataclass(frozen=True, slots=True)
class Resolution:
name: str
kind: ResolutionKind
pages: tuple[Page, ...] = ()
@property
def is_red_link(self) -> bool:
return self.kind is ResolutionKind.RED_LINK
def single(self) -> Page:
"""The one page (SINGLE), or the canonical pick of a CHORUS (first), else KeyError."""
if not self.pages:
raise KeyError(self.name)
return self.pages[0]
class UnionGraph:
"""Composes attached shards + the coordination-log fold into a resolvable union."""
def __init__(
self,
space: str,
log: DecisionLog | None = None,
policy: Policy = DEFAULT_POLICY,
) -> None:
self.space = space
self.log = log or DecisionLog()
self.policy = policy
self._shards: list[ShardAdapter] = []
def attach(self, adapter: ShardAdapter) -> None:
self._shards.append(adapter)
def shard(self, shard_id: str) -> ShardAdapter | None:
return next((s for s in self._shards if s.shard_id == shard_id), None)
@property
def shards(self) -> tuple[ShardAdapter, ...]:
return tuple(self._shards)
def iter_pages(self) -> Iterator[Page]:
"""Every page across attached shards, raw (per-shard, not chorus-collapsed). The
enumeration substrate for derived views — BackLinks, AllPages, SiteMap (§8.4)."""
for shard in self._shards:
for key in shard.keys():
try:
yield shard.read(key)
except KeyError:
continue
def _read_all(self, key: str) -> list[Page]:
pages: list[Page] = []
for shard in self._shards:
try:
pages.append(shard.read(key))
except KeyError:
continue
return pages
def resolve(self, name: str) -> Resolution:
state = self.log.fold(self.space)
overlays = {
Overlay.from_payload(p).target: Overlay.from_payload(p)
for p in state.open_overlays.values()
}
# 1. alias redirect (coordination-canonical, via the log fold)
target = state.resolve_alias(name)
if target is not None and ":" in target:
shard_id, _, key = target.partition(":")
shard = self.shard(shard_id)
if shard is not None:
try:
page = self._with_overlay(shard.read(key), overlays)
return Resolution(name, ResolutionKind.SINGLE, (page,))
except KeyError:
pass # alias dangles → fall through to normal resolution
# 2/3. union lookup by key across shards, then layer open overlays
pages = [self._with_overlay(p, overlays) for p in self._read_all(name)]
pages.extend(self._draft_only_pages(name, pages, overlays))
if not pages:
return Resolution(name, ResolutionKind.RED_LINK)
if len(pages) == 1:
return Resolution(name, ResolutionKind.SINGLE, (pages[0],))
# ambiguity → chorus, with divergence recorded (never erased, I-4)
ordered = self._order_for_policy(pages)
marked = tuple(_with_divergence(p, ordered) for p in ordered)
return Resolution(name, ResolutionKind.CHORUS, marked)
def _with_overlay(self, page: Page, overlays: dict[Identity, Overlay]) -> Page:
"""Mark a canonical page that has an open overlay (overlay_state DRAFT; project the
overlaid body where policy shows drafts) — never hides the overlay (I-4)."""
overlay = overlays.get(page.identity)
if overlay is None:
return page
env = dataclasses.replace(page.envelope, overlay_state=OverlayState.DRAFT)
body = overlay.body if self.policy.show_drafts else page.body
return dataclasses.replace(page, body=body, envelope=env)
def _draft_only_pages(
self, name: str, existing: list[Page], overlays: dict[Identity, Overlay]
) -> list[Page]:
"""Drafts that create a not-yet-existing page on an attached shard become resolvable."""
have = {p.identity for p in existing}
out: list[Page] = []
for identity, overlay in overlays.items():
if identity.key != name or identity in have or self.shard(identity.shard) is None:
continue
env = ProvenanceEnvelope(
source_shard=identity.shard,
staleness=Staleness.FRESH,
overlay_state=OverlayState.DRAFT,
)
body = overlay.body if self.policy.show_drafts else ""
out.append(Page(identity=identity, body=body, envelope=env))
return out
def _order_for_policy(self, pages: list[Page]) -> list[Page]:
if (
self.policy.canonical_source is CanonicalSource.DESIGNATED_CANONICAL
and self.policy.designated_shard is not None
):
pages = sorted(
pages, key=lambda p: p.identity.shard != self.policy.designated_shard
)
return pages
def _with_divergence(page: Page, group: list[Page]) -> Page:
peers = tuple(str(p.identity) for p in group if p.identity != page.identity)
new_env = dataclasses.replace(page.envelope, divergence=peers)
return dataclasses.replace(page, envelope=new_env)

View File

@@ -0,0 +1,33 @@
"""views/ — derived, recomputable, provenance-carrying read views over the union (§8.4).
All views here are *derived tier*: pure functions of the attached shards plus the coordination-log
fold, storing nothing canonical (SHARD-WP-0011 makes them incrementally maintainable). Presentation
stays out of core (L6) — these produce models, never rendered output. Per the dependency rule this
package imports down (union/model/coordination/provenance) and is imported only by the orchestrator.
"""
from shard_wiki.views.allpages import AllPagesEntry, SiteMapNode, all_pages, site_map
from shard_wiki.views.backlinks import BackLink, BackLinksIndex, build_backlinks
from shard_wiki.views.links import (
ResolvedLink,
WikiLink,
extract_links,
resolve_links,
)
from shard_wiki.views.recentchanges import ChangeEntry, recent_changes
__all__ = [
"WikiLink",
"ResolvedLink",
"extract_links",
"resolve_links",
"BackLink",
"BackLinksIndex",
"build_backlinks",
"ChangeEntry",
"recent_changes",
"AllPagesEntry",
"SiteMapNode",
"all_pages",
"site_map",
]

View File

@@ -0,0 +1,131 @@
"""AllPages + SiteMap — enumeration views over the union (SHARD-WP-0010 T4).
**AllPages** lists the union's distinct pages, collapsing identities that name the same page: a
*chorus* (same key across shards) and *equivalence-bound* identities (decision-log bindings) fold
into one entry, with divergence noted when the members' bodies differ (union without erasure — the
collapse is acknowledged, never silent). **SiteMap** is the namespace tree built from page
placements (paths), spanning shards.
Both are derived/recomputable and presentation-free (the tree is a model, not rendered HTML).
"""
from __future__ import annotations
from dataclasses import dataclass
from shard_wiki.model import Identity, Page
from shard_wiki.union import UnionGraph
__all__ = ["AllPagesEntry", "SiteMapNode", "all_pages", "site_map"]
@dataclass(frozen=True, slots=True)
class AllPagesEntry:
"""One union page: its representative ``name``, the ``members`` collapsed into it, and whether
those members' bodies ``diverge`` (a chorus with differing content)."""
name: str
members: tuple[Identity, ...]
diverges: bool
@dataclass(frozen=True, slots=True)
class SiteMapNode:
"""A namespace node: its path ``name``, child namespaces, and pages directly under it."""
name: str
children: tuple[SiteMapNode, ...]
pages: tuple[Identity, ...]
class _UnionFind:
def __init__(self) -> None:
self._parent: dict[str, str] = {}
def add(self, x: str) -> None:
self._parent.setdefault(x, x)
def find(self, x: str) -> str:
self.add(x)
root = x
while self._parent[root] != root:
root = self._parent[root]
while self._parent[x] != root:
self._parent[x], x = root, self._parent[x]
return root
def union(self, a: str, b: str) -> None:
self.add(a)
self.add(b)
ra, rb = self.find(a), self.find(b)
if ra != rb:
self._parent[max(ra, rb)] = min(ra, rb)
def all_pages(
union: UnionGraph,
equivalence_groups: tuple[frozenset[str], ...] | None = None,
) -> tuple[AllPagesEntry, ...]:
"""Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members.
``equivalence_groups`` (string identities, decision-log form) overrides the source of
equivalence — the orchestrator passes the maintained index's groups (SHARD-WP-0011 T4); the
default falls back to the decision-log fold, so direct callers are unaffected.
"""
pages: dict[str, Page] = {}
by_key: dict[str, list[str]] = {}
for page in union.iter_pages():
ident = str(page.identity)
pages[ident] = page
by_key.setdefault(page.identity.key, []).append(ident)
uf = _UnionFind()
for ident in pages:
uf.add(ident)
for idents in by_key.values(): # same key across shards → chorus
for other in idents[1:]:
uf.union(idents[0], other)
if equivalence_groups is None:
equivalence_groups = union.log.fold(union.space).equivalence_groups
for group in equivalence_groups: # curator bindings (+ maintained content edges)
present = [m for m in group if m in pages]
for other in present[1:]:
uf.union(present[0], other)
groups: dict[str, list[str]] = {}
for ident in pages:
groups.setdefault(uf.find(ident), []).append(ident)
entries: list[AllPagesEntry] = []
for members in groups.values():
member_pages = [pages[m] for m in members]
identities = tuple(p.identity for p in member_pages)
name = min(p.identity.key for p in member_pages)
diverges = len({p.body for p in member_pages}) > 1
entries.append(AllPagesEntry(name=name, members=identities, diverges=diverges))
return tuple(sorted(entries, key=lambda e: e.name))
def _segments(page: Page) -> list[str]:
path = page.placements[0].path if page.placements else page.identity.key
if path.endswith(".md"):
path = path[:-3]
return [seg for seg in path.split("/") if seg]
def site_map(union: UnionGraph) -> SiteMapNode:
"""The union namespace tree from page placements (directories nest; pages sit at their dir)."""
root: dict = {"children": {}, "pages": []}
for page in union.iter_pages():
segments = _segments(page)
node = root
for seg in segments[:-1]: # directory segments build the nesting
node = node["children"].setdefault(seg, {"children": {}, "pages": []})
node["pages"].append(page.identity)
return _freeze("", root)
def _freeze(name: str, node: dict) -> SiteMapNode:
children = tuple(_freeze(k, v) for k, v in sorted(node["children"].items()))
pages = tuple(sorted(node["pages"], key=str))
return SiteMapNode(name=name, children=children, pages=pages)

View File

@@ -0,0 +1,65 @@
"""BackLinks — the strongest core derived view (SHARD-WP-0010 T2; UC-18).
For any page name, the set of pages that link to it. Built by extracting wikilinks (T1) from every
page across the attached shards and resolving each through the union: only **resolved** links
create a backlink (a red-link points at nothing, so it contributes none). Entries carry their
**source provenance** (the linking page's identity / shard). Keying by the resolved *name* means a
chorus target aggregates the backlinks of all its members into one bucket (union without erasure).
Derived/recomputable — stores nothing canonical; SHARD-WP-0011 maintains it incrementally.
"""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from shard_wiki.model import Identity
from shard_wiki.union import UnionGraph
from shard_wiki.views.links import resolve_links
__all__ = ["BackLink", "BackLinksIndex", "build_backlinks"]
@dataclass(frozen=True, slots=True)
class BackLink:
"""One inbound link: ``source`` (the linking page) references ``target_name``."""
source: Identity
target_name: str
@property
def source_shard(self) -> str:
return self.source.shard
class BackLinksIndex:
"""An immutable name → inbound-links index over the union link graph."""
def __init__(self, edges: Mapping[str, tuple[BackLink, ...]]) -> None:
self._edges = dict(edges)
def to(self, name: str) -> tuple[BackLink, ...]:
"""The backlinks pointing at ``name`` (empty if none)."""
return self._edges.get(name, ())
def sources(self, name: str) -> frozenset[Identity]:
"""Just the identities linking to ``name`` — convenient for set assertions."""
return frozenset(bl.source for bl in self.to(name))
def names(self) -> frozenset[str]:
return frozenset(self._edges)
def build_backlinks(union: UnionGraph, *, camelcase: bool = False) -> BackLinksIndex:
"""Scan every union page's links and index the resolved ones by target name."""
edges: dict[str, set[BackLink]] = {}
for page in union.iter_pages():
for resolved in resolve_links(union, page.body, camelcase=camelcase):
if resolved.is_red_link:
continue # red-links don't create backlinks
backlink = BackLink(source=page.identity, target_name=resolved.link.target)
edges.setdefault(resolved.link.target, set()).add(backlink)
return BackLinksIndex(
{name: tuple(sorted(links, key=lambda bl: str(bl.source))) for name, links in edges.items()}
)

View File

@@ -0,0 +1,91 @@
"""Wikilink + red-link model (SHARD-WP-0010 T1; FederationRequirements ADR-06).
A CommonMark *wikilink extension*: ``[[Target]]`` and ``[[Target|label]]`` are extracted from a
page body and each target is resolved through the union (ADR-01). A target that resolves is a
**link**; one that does not is a **red-link** — a createable hole (UC-23), never a dropped
reference (union without erasure). CamelCase auto-linking (``WikiWord``) is **off by default** and
opt-in per space, since bare CamelCase is noisy and policy-laden.
The link *model and resolution* are core; turning a :class:`ResolvedLink` into an ``<a>`` (or a
red anchor) is L6 presentation and lives outside this package. Link spans are byte/char offsets in
the body so a later layer can address them precisely.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from shard_wiki.union import Resolution, UnionGraph
__all__ = ["WikiLink", "ResolvedLink", "extract_links", "resolve_links"]
_WIKILINK_RE = re.compile(r"\[\[\s*([^\]|]+?)\s*(?:\|\s*([^\]]+?)\s*)?\]\]")
# A WikiWord: ≥2 capitalized alphanumeric segments run together (e.g. FrontPage, WikiWord).
_CAMELCASE_RE = re.compile(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b")
_FENCED_RE = re.compile(r"```.*?```", re.DOTALL)
_INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
@dataclass(frozen=True, slots=True)
class WikiLink:
"""One extracted reference. ``target`` is the resolve key; ``label`` is the display text (or
None to use the target); ``span`` is the ``[start, end)`` offset of the whole token in the body;
``auto`` marks a CamelCase auto-link (vs an explicit ``[[...]]``)."""
target: str
label: str | None
span: tuple[int, int]
auto: bool = False
@property
def text(self) -> str:
return self.label or self.target
@dataclass(frozen=True, slots=True)
class ResolvedLink:
"""A :class:`WikiLink` paired with its union :class:`Resolution` (the link's truth status)."""
link: WikiLink
resolution: Resolution
@property
def is_red_link(self) -> bool:
return self.resolution.is_red_link
def _mask(body: str, pattern: re.Pattern[str]) -> str:
"""Blank out ``pattern`` matches with equal-length spaces so later scans skip them while every
surviving match keeps its true offset."""
return pattern.sub(lambda m: " " * len(m.group(0)), body)
def extract_links(body: str, *, camelcase: bool = False) -> tuple[WikiLink, ...]:
"""Extract wikilinks from ``body`` in document order, skipping fenced/inline code.
With ``camelcase=True`` (per-space opt-in), bare ``WikiWord`` tokens outside code and outside
existing ``[[...]]`` also become links.
"""
scan = _mask(_mask(body, _FENCED_RE), _INLINE_CODE_RE)
links: list[WikiLink] = []
for m in _WIKILINK_RE.finditer(scan):
links.append(WikiLink(target=m.group(1).strip(), label=m.group(2), span=m.span()))
if camelcase:
# Mask explicit-link spans too, so a CamelCase target inside [[...]] isn't double-counted.
cc_scan = _mask(scan, _WIKILINK_RE)
for m in _CAMELCASE_RE.finditer(cc_scan):
links.append(WikiLink(target=m.group(1), label=None, span=m.span(), auto=True))
return tuple(sorted(links, key=lambda link: link.span[0]))
def resolve_links(
union: UnionGraph, body: str, *, camelcase: bool = False
) -> tuple[ResolvedLink, ...]:
"""Extract and resolve every link in ``body`` against ``union`` (link vs red-link, ADR-01)."""
return tuple(
ResolvedLink(link, union.resolve(link.target))
for link in extract_links(body, camelcase=camelcase)
)

View File

@@ -0,0 +1,108 @@
"""RecentChanges — a merged change feed over the union (SHARD-WP-0010 T3; UC-17).
Two streams, one ordered feed (newest-first):
* the **coordination journal** — overlay/alias/fork/merge/binding decisions from the decision log,
each carrying its actor and the decision payload; and
* **shard change signals** — a page's current revision (folder mtime / ``source_rev``), i.e. the
backend's own "this changed" evidence.
Every entry carries provenance: which shard the edit came from, or that it was a coordination
decision (and by whom). Derived/recomputable — `notify`-driven streaming is a later binding.
"""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime
from shard_wiki.coordination import DecisionLog, EventType
from shard_wiki.union import UnionGraph
__all__ = ["ChangeEntry", "recent_changes"]
_COORDINATION = "coordination"
# How each journal event names the thing it touched + a human kind label.
_EVENT_KIND = {
EventType.ALIAS_SET: "alias",
EventType.OVERLAY_CREATED: "overlay",
EventType.MERGE_DECIDED: "merge",
EventType.PAGE_FORKED: "fork",
EventType.BINDING_MADE: "binding",
}
@dataclass(frozen=True, slots=True)
class ChangeEntry:
"""One change in the feed. ``source`` is the shard id (a shard edit) or ``"coordination"``."""
when: datetime
kind: str
ref: str
source: str
actor: str | None = None
detail: Mapping[str, object] = field(default_factory=dict)
def _event_ref(event_type: EventType, payload: Mapping[str, object]) -> str:
if event_type is EventType.ALIAS_SET:
return str(payload.get("alias", ""))
if event_type is EventType.OVERLAY_CREATED:
return f"{payload.get('target_shard')}:{payload.get('target_key')}"
if event_type is EventType.PAGE_FORKED:
return f"{payload.get('source')}{payload.get('fork')}"
if event_type is EventType.BINDING_MADE:
return ", ".join(str(m) for m in payload.get("members", ()))
return str(payload.get("overlay_id", "")) # MERGE_DECIDED
def recent_changes(
union: UnionGraph,
log: DecisionLog,
space: str,
*,
limit: int | None = None,
) -> tuple[ChangeEntry, ...]:
"""Merge the coordination journal and shard change signals into one newest-first feed."""
entries: list[ChangeEntry] = []
for event in log.events(space):
entries.append(
ChangeEntry(
when=event.timestamp,
kind=_EVENT_KIND.get(event.type, event.type.value),
ref=_event_ref(event.type, event.payload),
source=_COORDINATION,
actor=event.actor,
detail=dict(event.payload),
)
)
for page in union.iter_pages():
rev = page.envelope.source_rev
when = _parse_rev(rev)
if when is None:
continue # shard offers no change signal for this page — skip gracefully
entries.append(
ChangeEntry(
when=when,
kind="edit",
ref=str(page.identity),
source=page.identity.shard,
detail={"source_rev": rev},
)
)
entries.sort(key=lambda e: e.when, reverse=True)
return tuple(entries if limit is None else entries[:limit])
def _parse_rev(rev: str | None) -> datetime | None:
if rev is None:
return None
try:
return datetime.fromisoformat(rev)
except ValueError:
return None # non-temporal revision token (e.g. a content hash) — no feed timestamp

View File

@@ -0,0 +1,120 @@
"""Tests for the per-space append authority / lease (SHARD-WP-0009 T2).
A single append authority per space serializes appends into a total order; non-holders forward
intents to the holder; the lease is time-bounded and re-grantable (HA hand-off); a stale ex-holder
cannot fork the log.
"""
from datetime import datetime, timedelta, timezone
import pytest
from shard_wiki.coordination import (
AppendAuthority,
EventType,
GitEventStore,
InMemoryEventStore,
LeaseHeld,
LeaseRegistry,
)
class FakeClock:
def __init__(self):
self.now = datetime(2026, 1, 1, tzinfo=timezone.utc)
def __call__(self):
return self.now
def advance(self, seconds):
self.now += timedelta(seconds=seconds)
def test_only_one_node_holds_a_space_at_a_time():
reg = LeaseRegistry()
a = AppendAuthority("A", InMemoryEventStore(), reg)
b = AppendAuthority("B", InMemoryEventStore(), reg)
a.acquire("s")
with pytest.raises(LeaseHeld):
b.acquire("s") # B is refused while A's lease is valid
def test_concurrent_appends_serialize_into_one_total_order():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
# B is a non-holder: its append forwards to A, the holder. Interleave A and B writers.
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"}) # forwarded
a.append("s", EventType.ALIAS_SET, {"alias": "3", "target": "x:3"})
seqs = [e.seq for e in store.events("s")]
aliases = [e.payload["alias"] for e in store.events("s")]
assert seqs == [0, 1, 2] # contiguous total order despite two writers
assert aliases == ["1", "2", "3"]
def test_non_holder_forwards_rather_than_writing_directly():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
assert not b.holds("s")
b.append("s", EventType.ALIAS_SET, {"alias": "fwd", "target": "x:1"})
# The write landed on the shared store under A's authority, in one stream.
assert [e.payload["alias"] for e in store.events("s")] == ["fwd"]
def test_lease_handoff_resumes_from_head():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
clock.advance(20) # A's lease expires (A "dies")
b.acquire("s") # re-grantable: B takes over
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
assert [e.seq for e in store.events("s")] == [0, 1, 2] # contiguous across hand-off
def test_stale_ex_holder_cannot_fork_the_log():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
clock.advance(20)
b.acquire("s") # B is now the holder; A's lease is stale
b.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
# A still thinks it can write, but it's no longer the holder: its intent forwards to B.
assert not a.holds("s")
a.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
aliases = [e.payload["alias"] for e in store.events("s")]
assert aliases == ["0", "1", "2"] # one stream, no fork
def test_authority_over_git_store_keeps_total_order(tmp_path):
reg = LeaseRegistry()
store = GitEventStore(tmp_path / "coord")
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
a.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
b.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"}) # forwarded
assert [e.seq for e in store.events("s")] == [0, 1]
def test_unleased_space_self_acquires_on_append():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
a.append("s", EventType.ALIAS_SET, {"alias": "x", "target": "y:1"}) # no explicit acquire
assert a.holds("s")
assert len(store.events("s")) == 1

66
tests/test_apply.py Normal file
View File

@@ -0,0 +1,66 @@
"""Tests for apply-under-drift (SHARD-WP-0008 T4)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.coordination import ApplyStatus, DecisionLog, OverlayEngine
from shard_wiki.model import Identity
def _writable(tmp_path, files):
for rel, text in files.items():
(tmp_path / rel).write_text(text, encoding="utf-8")
return FolderAdapter("w", tmp_path, writable=True)
def test_fast_forward_apply_writes_through(tmp_path):
shard = _writable(tmp_path, {"Home.md": "old"})
eng = OverlayEngine("space", DecisionLog())
base = shard.current_rev("Home")
ov = eng.draft(Identity("w", "Home"), "new", base_rev=base)
result = eng.apply(ov.overlay_id, shard)
assert result.status is ApplyStatus.APPLIED
assert shard.read("Home").body == "new" # written through
assert eng.get(ov.overlay_id) is None # overlay closed in the fold
def test_drift_refuses_without_clobber(tmp_path):
shard = _writable(tmp_path, {"Home.md": "old"})
eng = OverlayEngine("space", DecisionLog())
ov = eng.draft(Identity("w", "Home"), "mine", base_rev="STALE-REV")
result = eng.apply(ov.overlay_id, shard)
assert result.status is ApplyStatus.REFUSED_DRIFT
assert shard.read("Home").body == "old" # not clobbered (I-5)
assert eng.get(ov.overlay_id) is not None # overlay still open
def test_read_only_target_keeps_draft(tmp_path):
(tmp_path / "Home.md").write_text("canon", encoding="utf-8")
ro = FolderAdapter("ro", tmp_path) # not writable
eng = OverlayEngine("space", DecisionLog())
ov = eng.draft(Identity("ro", "Home"), "my edit", base_rev=ro.current_rev("Home"))
result = eng.apply(ov.overlay_id, ro)
assert result.status is ApplyStatus.KEPT_DRAFT
assert ro.read("Home").body == "canon" # source untouched
assert eng.get(ov.overlay_id) is not None # local truth retained
def test_new_page_fast_forwards(tmp_path):
shard = _writable(tmp_path, {})
eng = OverlayEngine("space", DecisionLog())
ov = eng.draft(Identity("w", "Fresh"), "brand new", base_rev=None) # didn't exist
result = eng.apply(ov.overlay_id, shard)
assert result.status is ApplyStatus.APPLIED
assert shard.read("Fresh").body == "brand new"
def test_wrong_adapter_is_rejected(tmp_path):
shard = _writable(tmp_path, {"Home.md": "x"})
eng = OverlayEngine("space", DecisionLog())
ov = eng.draft(Identity("other", "Home"), "y", base_rev=None)
try:
eng.apply(ov.overlay_id, shard)
raise AssertionError("expected ValueError")
except ValueError:
pass

67
tests/test_conformance.py Normal file
View File

@@ -0,0 +1,67 @@
"""Tests for the adapter conformance suite (SHARD-WP-0007 T4)."""
from collections.abc import Iterable
import pytest
from shard_wiki.adapters import (
ConformanceError,
FolderAdapter,
ShardAdapter,
assert_conformant,
run_conformance,
)
from shard_wiki.model import CapabilityProfile, Identity, Page
from shard_wiki.provenance import ProvenanceEnvelope
def test_folder_adapter_is_conformant(tmp_path):
(tmp_path / "Home.md").write_text("# Home", encoding="utf-8")
report = assert_conformant(FolderAdapter("shardA", tmp_path))
assert report.ok
assert report.diff() == "conformant"
class _LyingReadAdapter(ShardAdapter):
"""Claims READ but read() is broken — must fail conformance."""
def __init__(self, profile: CapabilityProfile) -> None:
self._profile = profile
@property
def shard_id(self) -> str:
return "liar"
def profile(self) -> CapabilityProfile:
return self._profile
def keys(self) -> Iterable[str]:
return ["Home"]
def read(self, key: str) -> Page:
# Returns a page attributed to the WRONG shard — a lie about its own content.
return Page(Identity("someone-else", key), "x", ProvenanceEnvelope(source_shard="x"))
def test_lying_read_adapter_fails_with_precise_diff(tmp_path):
good = FolderAdapter("shardA", tmp_path).profile() # a valid read-only profile
liar = _LyingReadAdapter(good)
report = run_conformance(liar)
assert not report.ok
assert "read-round-trips" in report.diff()
with pytest.raises(ConformanceError, match="read-round-trips"):
assert_conformant(liar)
class _DishonestWriteAdapter(FolderAdapter):
"""Declares no WRITE (inherits read-only folder profile) but write() silently succeeds."""
def write(self, key: str, body: str) -> Page: # noqa: ARG002
return self.read(next(iter(self.keys())))
def test_dishonest_absence_fails(tmp_path):
(tmp_path / "Home.md").write_text("x", encoding="utf-8")
report = run_conformance(_DishonestWriteAdapter("shardA", tmp_path))
assert not report.ok
assert "honest-absence:write" in report.diff()

View File

@@ -0,0 +1,74 @@
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
export_jsonl,
import_jsonl,
migrate_space,
)
from shard_wiki.space import InformationSpace
def test_information_space_git_backed_uses_git_log(tmp_path):
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
assert isinstance(space.log._store, GitEventStore)
space.alias("Home", "shardA:Index")
# Read-your-writes through the orchestrator's git-backed log.
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
def test_default_information_space_stays_in_memory():
space = InformationSpace("space-1")
assert isinstance(space.log._store, InMemoryEventStore)
def test_migrate_space_preserves_order_and_provenance(tmp_path):
source = InMemoryEventStore()
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
dest = GitEventStore(tmp_path / "coord")
n = migrate_space(source, "s", dest)
assert n == 2
migrated = dest.events("s")
assert [e.seq for e in migrated] == [0, 1]
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
assert migrated[0].actor == "ana"
assert migrated[1].actor == "ben"
assert migrated[0].timestamp == e0.timestamp
def test_migration_yields_identical_fold(tmp_path):
source = DecisionLog(InMemoryEventStore())
for typ, payload in [
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
]:
source.append("s", typ, payload)
dest = GitEventStore(tmp_path / "coord")
migrate_space(source._store, "s", dest)
after = DecisionLog(dest)
assert after.fold("s").aliases == source.fold("s").aliases
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
def test_jsonl_round_trip_into_git(tmp_path):
source = InMemoryEventStore()
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
path = tmp_path / "log.jsonl"
assert export_jsonl(source.events("s"), path) == 2
dest = GitEventStore(tmp_path / "coord")
assert import_jsonl(path, dest) == 2
state = DecisionLog(dest).fold("s")
assert state.resolve_alias("Home") == "x:1"
assert state.equivalent_to("p") == frozenset({"p", "q"})

View File

@@ -0,0 +1,83 @@
"""Cross-process read-your-writes over the git log + fold parity (SHARD-WP-0009 T3).
The git backend's value over the in-memory double is that the totally ordered log is durable and
shared: a write by one process/handle is immediately visible to another opening the same ref, and
the derived fold is identical to the in-memory fold of the same event sequence (derived = f(log)).
"""
import os
import subprocess
import sys
import textwrap
from pathlib import Path
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
)
_SRC = str(Path(__file__).resolve().parents[1] / "src")
def test_new_handle_sees_prior_writes(tmp_path):
repo = tmp_path / "coord"
writer = DecisionLog(GitEventStore(repo))
writer.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
writer.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
# A second, independent handle on the same repo — read-your-writes across handles.
reader = DecisionLog(GitEventStore(repo))
assert [e.seq for e in reader.events("s")] == [0, 1]
assert reader.fold("s").resolve_alias("Home") == "shardA:Index"
def test_append_in_separate_process_is_visible(tmp_path):
repo = tmp_path / "coord"
# Seed from this process so the repo exists.
DecisionLog(GitEventStore(repo)).append(
"s", EventType.ALIAS_SET, {"alias": "A", "target": "x:1"}
)
child = textwrap.dedent(
f"""
from shard_wiki.coordination import DecisionLog, EventType, GitEventStore
log = DecisionLog(GitEventStore({str(repo)!r}))
log.append("s", EventType.ALIAS_SET, {{"alias": "B", "target": "x:2"}})
"""
)
result = subprocess.run(
[sys.executable, "-c", child],
capture_output=True,
text=True,
env={"PYTHONPATH": _SRC, "PATH": os.environ.get("PATH", "")},
)
assert result.returncode == 0, result.stderr
# This process, with a fresh handle, sees the child's append in order.
reader = DecisionLog(GitEventStore(repo))
assert [e.payload["alias"] for e in reader.events("s")] == ["A", "B"]
assert [e.seq for e in reader.events("s")] == [0, 1]
def test_cross_process_fold_equals_in_memory_fold(tmp_path):
sequence = [
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.PAGE_FORKED, {"source": "p", "fork": "q"}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
]
mem = DecisionLog(InMemoryEventStore())
for typ, payload in sequence:
mem.append("s", typ, payload)
repo = tmp_path / "coord"
DecisionLog(GitEventStore(repo)) # init repo
for typ, payload in sequence:
# Each append from a fresh handle to simulate distinct writers over time.
DecisionLog(GitEventStore(repo)).append("s", typ, payload)
git_state = DecisionLog(GitEventStore(repo)).fold("s")
mem_state = mem.fold("s")
assert git_state.aliases == mem_state.aliases
assert git_state.equivalence_groups == mem_state.equivalence_groups
assert git_state.equivalent_to("a") == frozenset({"a", "b", "c"})

View File

@@ -0,0 +1,51 @@
"""Tests for the event-sourced DecisionLog (SHARD-WP-0007 T5)."""
from shard_wiki.coordination import DecisionLog, EventType
def test_append_is_totally_ordered_per_space():
log = DecisionLog()
log.append("spaceA", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
log.append("spaceA", EventType.ALIAS_SET, {"alias": "Start", "target": "shardA:Index"})
log.append("spaceB", EventType.ALIAS_SET, {"alias": "X", "target": "shardB:Y"})
seqs = [e.seq for e in log.events("spaceA")]
assert seqs == [0, 1] # per-space monotonic
assert [e.seq for e in log.events("spaceB")] == [0] # independent ordering
def test_read_your_writes():
log = DecisionLog()
ev = log.append("s", EventType.ALIAS_SET, {"alias": "a", "target": "shardA:b"})
assert log.events("s")[-1] is ev
def test_fold_reproduces_current_state():
log = DecisionLog()
log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}) # LWW
state = log.fold("s")
assert state.resolve_alias("Home") == "shardB:Main"
assert state.resolve_alias("missing") is None
def test_fold_is_pure_function_of_log():
log = DecisionLog()
log.append("s", EventType.BINDING_MADE, {"members": ["shardA:Home", "shardB:Home"]})
first = log.fold("s")
second = log.fold("s")
assert first.equivalence_groups == second.equivalence_groups # derived = f(log)
def test_equivalence_groups_merge_transitively():
log = DecisionLog()
log.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
log.append("s", EventType.BINDING_MADE, {"members": ["b", "c"]})
state = log.fold("s")
assert state.equivalent_to("a") == frozenset({"a", "b", "c"})
assert state.equivalent_to("lonely") == frozenset({"lonely"})
def test_page_fork_creates_equivalence():
log = DecisionLog()
log.append("s", EventType.PAGE_FORKED, {"source": "shardA:Doc", "fork": "shardB:Doc"})
assert log.fold("s").equivalent_to("shardA:Doc") == frozenset({"shardA:Doc", "shardB:Doc"})

View File

@@ -0,0 +1,61 @@
"""Tests for per-shard extension activation (SHARD-WP-0014 T3, ADR-0001)."""
from shard_wiki.engine import (
ActivationContext,
ActivationResolver,
StaticProvider,
feature_control_provider,
)
CANDIDATES = ["ext.overlay", "ext.views", "ext.struct", "ext.compute"]
def test_static_provider_default_off():
r = ActivationResolver(StaticProvider()) # nothing enabled
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == set()
def test_static_provider_global_flags():
r = ActivationResolver(StaticProvider(flags={"ext.overlay": True, "ext.views": True}))
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == {"ext.overlay", "ext.views"}
def test_per_shard_scoping_overrides_global():
provider = StaticProvider(
flags={"ext.views": True},
per_shard={"engB": {"ext.struct": True, "ext.views": False}},
)
r = ActivationResolver(provider)
assert r.active_extensions(CANDIDATES, ActivationContext("engA")) == {"ext.views"}
assert r.active_extensions(CANDIDATES, ActivationContext("engB")) == {"ext.struct"}
def test_context_carries_tenant():
captured = {}
class Spy(StaticProvider):
def is_active(self, feature_key, context):
captured.update(context)
return super().is_active(feature_key, context)
ActivationResolver(Spy(flags={"ext.views": True})).active_extensions(
["ext.views"], ActivationContext("s1", tenant_id="acme")
)
assert captured["shard_id"] == "s1" and captured["tenant_id"] == "acme"
def test_activation_profile_returns_config_for_active():
provider = StaticProvider(
flags={"ext.struct": True},
configs={"ext.struct": {"max_fields": 50}},
)
profile = ActivationResolver(provider).activation_profile(CANDIDATES, ActivationContext("s"))
assert profile == {"ext.struct": {"max_fields": 50}}
def test_feature_control_provider_degrades_gracefully():
# feature_control_sdk is not a dependency of shard-wiki: when absent the factory returns
# None (standalone path stays dependency-free, ADR-0001); when present it yields a usable
# ActivationProvider. Either way it must not raise.
provider = feature_control_provider()
assert provider is None or hasattr(provider, "is_active")

View File

@@ -0,0 +1,80 @@
"""Tests for EngineShardAdapter (SHARD-WP-0014 T5): engine as a canonical-mode shard."""
from shard_wiki import InformationSpace
from shard_wiki.adapters import assert_conformant
from shard_wiki.engine import (
Extension,
ExtensionRuntime,
Hook,
ProfileContribution,
StaticProvider,
build_engine_shard,
)
from shard_wiki.engine.activation import ActivationContext
from shard_wiki.model import Verb
class StructProfileExt(Extension):
"""Profile-only extension (no body transform → write stays content-preserving)."""
id = "ext.struct"
def hooks(self):
return {
Hook.ON_PROFILE: lambda p, c: ProfileContribution(
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
)
}
def _runtime():
rt = ExtensionRuntime()
rt.register(StructProfileExt())
return rt
def test_kernel_only_engine_shard_is_conformant():
shard = build_engine_shard("eng", ExtensionRuntime(), activate=set())
shard.write("Home", "hi")
report = assert_conformant(shard) # read + positive write probe
assert report.ok
assert shard.profile().supports(Verb.WRITE)
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
def test_profile_reflects_activated_extensions():
off = build_engine_shard("a", _runtime(), activate=set())
on = build_engine_shard("b", _runtime(), activate={"ext.struct"})
assert not off.profile().supports(Verb.STRUCTURED_PAYLOAD)
assert on.profile().supports(Verb.STRUCTURED_PAYLOAD) # E-5
assert_conformant(on)
def test_activation_via_provider():
provider = StaticProvider(flags={"ext.struct": True})
shard = build_engine_shard("c", _runtime(), provider=provider, context=ActivationContext("c"))
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
def test_attach_resolve_edit_through_engine_shard(tmp_path):
space = InformationSpace("team")
space.attach(build_engine_shard("wikiE", ExtensionRuntime(), activate=set()))
# seed a page directly via the shard, then read + edit through the orchestrator
space.union.shard("wikiE").write("Home", "v1")
assert space.read("Home").body == "v1"
result = space.edit("Home", "v2") # overlay -> apply-under-drift -> write-through
assert result.status.value == "applied"
assert space.read("Home").body == "v2"
def test_on_write_transform_runs():
class Upper(Extension):
id = "ext.upper"
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
rt = ExtensionRuntime()
rt.register(Upper())
shard = build_engine_shard("u", rt, activate={"ext.upper"})
shard.write("P", "quiet")
assert shard.read("P").body == "QUIET" # extension transformed the write

View File

@@ -0,0 +1,109 @@
"""Tests for the typed-extension runtime (SHARD-WP-0014 T2)."""
import pytest
from shard_wiki.engine import Extension, ExtensionError, ExtensionRuntime, Hook
class Upper(Extension):
id = "ext.upper"
declares_types = ("upper",)
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
class Bang(Extension):
id = "ext.bang"
depends_on = ("ext.upper",)
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body + "!"}
class Profiler(Extension):
id = "ext.profiler"
def hooks(self):
return {Hook.ON_PROFILE: lambda payload, ctx: {"structure": "typed"}}
def _runtime(*exts):
rt = ExtensionRuntime()
for e in exts:
rt.register(e)
return rt
def test_register_rejects_bad_id_and_noncallable_hook():
rt = ExtensionRuntime()
class BadId(Extension):
id = "upper" # missing 'ext.' prefix
with pytest.raises(ExtensionError, match="ext."):
rt.register(BadId())
class Liar(Extension):
id = "ext.liar"
def hooks(self):
return {Hook.ON_WRITE: "not-callable"} # type: ignore[dict-item]
with pytest.raises(ExtensionError, match="not callable"):
rt.register(Liar())
def test_transform_hook_chains_in_dependency_order():
rt = _runtime(Upper(), Bang())
active = rt.activate({"ext.bang"}) # pulls ext.upper via depends_on
assert set(active.ids) == {"ext.upper", "ext.bang"}
# upper (dependency) runs before bang (dependent): "hi" -> "HI" -> "HI!"
assert active.handlers(Hook.ON_WRITE) == ("ext.upper", "ext.bang")
assert active.dispatch_transform(Hook.ON_WRITE, "hi") == "HI!"
def test_dependency_closure_is_automatic():
rt = _runtime(Upper(), Bang())
assert set(rt.activate({"ext.bang"}).ids) == {"ext.upper", "ext.bang"}
def test_unknown_extension_rejected():
with pytest.raises(ExtensionError, match="unknown"):
ExtensionRuntime().activate({"ext.ghost"})
def test_conflict_rejected():
class A(Extension):
id = "ext.a"
conflicts_with = ("ext.b",)
class B(Extension):
id = "ext.b"
with pytest.raises(ExtensionError, match="conflicts"):
_runtime(A(), B()).activate({"ext.a", "ext.b"})
def test_type_collision_rejected():
class S1(Extension):
id = "ext.s1"
declares_types = ("record",)
class S2(Extension):
id = "ext.s2"
declares_types = ("record",)
with pytest.raises(ExtensionError, match="type collision"):
_runtime(S1(), S2()).activate({"ext.s1", "ext.s2"})
def test_collect_hook_gathers_contributions():
active = _runtime(Profiler()).activate({"ext.profiler"})
assert active.dispatch_collect(Hook.ON_PROFILE) == [{"structure": "typed"}]
def test_wrong_dispatch_kind_errors():
active = _runtime(Profiler()).activate({"ext.profiler"})
with pytest.raises(ExtensionError, match="not a transform hook"):
active.dispatch_transform(Hook.ON_PROFILE, "x")
def test_unmet_dependency_rejected():
# Bang depends on ext.upper, but only Bang is registered.
rt = ExtensionRuntime()
rt.register(Bang())
with pytest.raises(ExtensionError, match="unmet dependency|unknown"):
rt.activate({"ext.bang"})

View File

@@ -0,0 +1,59 @@
"""Tests for the engine kernel (SHARD-WP-0014 T1)."""
import pytest
from shard_wiki.engine import EngineKernel, extract_wikilinks
from shard_wiki.model import Identity
def test_write_creates_then_edits_as_history():
k = EngineKernel("eng")
p1 = k.write("Home", "first")
assert p1.identity == Identity("eng", "Home")
assert p1.envelope.source_rev == "1"
p2 = k.write("Home", "second")
assert p2.envelope.source_rev == "2"
assert k.read("Home").body == "second" # latest
assert [v.body for v in k.history("Home")] == ["first", "second"] # recoverable history
def test_read_missing_raises():
k = EngineKernel("eng")
with pytest.raises(KeyError):
k.read("Nope")
def test_delete_is_recoverable():
k = EngineKernel("eng")
k.write("Doc", "v1")
k.delete("Doc")
assert not k.exists("Doc")
with pytest.raises(KeyError):
k.read("Doc")
assert [v.body for v in k.history("Doc")] == ["v1"] # history retained
k.write("Doc", "v2") # restore by writing
assert k.exists("Doc") and k.read("Doc").body == "v2"
def test_keys_and_current_rev():
k = EngineKernel("eng")
k.write("A", "a")
k.write("B", "b")
k.write("A", "a2")
assert set(k.keys()) == {"A", "B"}
assert k.current_rev("A") == "2"
assert k.current_rev("Missing") is None
def test_links_and_red_link_resolution():
k = EngineKernel("eng")
k.write("Home", "see [[Target]] and [[Other|labelled]] and [[Target]] again")
k.write("Target", "exists")
assert k.links("Home") == ["Target", "Other"] # ordered, de-duped, label dropped
assert k.resolve_link("Target") == Identity("eng", "Target")
assert k.resolve_link("Other") is None # red-link (not yet created)
def test_extract_wikilinks_helper():
assert extract_wikilinks("none here") == []
assert extract_wikilinks("[[A]] [[B|x]] [[A]]") == ["A", "B"]

View File

@@ -0,0 +1,98 @@
"""Tests for capability-profile-derived-from-extensions (SHARD-WP-0014 T4, E-5)."""
import pytest
from shard_wiki.engine import (
Extension,
ExtensionRuntime,
Hook,
ProfileContribution,
derive_profile,
engine_base_profile,
)
from shard_wiki.model import (
Addressing,
ContentOpacity,
NativeQuery,
ProfileError,
Verb,
)
class StructExt(Extension):
id = "ext.struct"
def hooks(self):
return {
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
)
}
class AddrExt(Extension):
id = "ext.addr"
def hooks(self):
return {
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
addressing=Addressing.SPAN, verbs_add=frozenset({Verb.TRANSCLUDE_SOURCE})
)
}
class EncryptExt(Extension):
id = "ext.encrypt"
def hooks(self):
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(content_opacity=ContentOpacity.ENCRYPTED)}
class QueryExt(Extension):
id = "ext.query"
def hooks(self):
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(native_query=NativeQuery.DB_QUERY)}
def _active(*exts, ids=None):
rt = ExtensionRuntime()
for e in exts:
rt.register(e)
return rt.activate(ids if ids is not None else {e.id for e in exts})
def test_base_profile_is_valid_kernel_minimum():
p = engine_base_profile()
assert p.supports(Verb.READ) and p.supports(Verb.WRITE)
assert not p.supports(Verb.STRUCTURED_PAYLOAD)
assert p.addressing is Addressing.PATH
def test_no_extensions_yields_base():
rt = ExtensionRuntime()
active = rt.activate(set())
assert derive_profile(active) == engine_base_profile()
def test_activating_extension_raises_the_profile():
active = _active(StructExt(), AddrExt())
p = derive_profile(active)
assert p.supports(Verb.STRUCTURED_PAYLOAD) # from ext.struct
assert p.supports(Verb.TRANSCLUDE_SOURCE) # from ext.addr
assert p.addressing is Addressing.SPAN # raised by ext.addr
assert p.supports(Verb.READ) # base preserved
def test_profile_changes_with_active_set():
only_struct = derive_profile(_active(StructExt(), AddrExt(), ids={"ext.struct"}))
both = derive_profile(_active(StructExt(), AddrExt()))
assert not only_struct.supports(Verb.TRANSCLUDE_SOURCE)
assert both.supports(Verb.TRANSCLUDE_SOURCE) # E-5: profile reflects what's active
def test_composition_cannot_yield_an_impossible_profile():
# encrypted opacity + native query violates §6.5 implication rules -> derive must reject.
active = _active(EncryptExt(), QueryExt())
with pytest.raises(ProfileError):
derive_profile(active)

76
tests/test_error_paths.py Normal file
View File

@@ -0,0 +1,76 @@
"""Error-path / contract tests across modules (keeps the suite honest about failure behaviour)."""
from collections.abc import Iterable
import pytest
from shard_wiki import InformationSpace
from shard_wiki.adapters import FolderAdapter, ShardAdapter, run_conformance
from shard_wiki.engine import EngineKernel
from shard_wiki.model import CapabilityProfile, Identity, Page, Placement
from shard_wiki.provenance import ProvenanceEnvelope
from shard_wiki.union import ResolutionKind, UnionGraph
def _folder(tmp_path, name, files, writable=False):
root = tmp_path / name
root.mkdir(parents=True, exist_ok=True)
for rel, text in files.items():
(root / rel).write_text(text, encoding="utf-8")
return FolderAdapter(name, root, writable=writable)
def test_resolution_single_on_red_link_raises():
u = UnionGraph("s")
res = u.resolve("ghost")
assert res.kind is ResolutionKind.RED_LINK
with pytest.raises(KeyError):
res.single()
def test_apply_unknown_overlay_raises(tmp_path):
space = InformationSpace("t")
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
with pytest.raises(KeyError):
space.apply_overlay("does-not-exist")
def test_apply_overlay_for_unattached_shard_raises(tmp_path):
space = InformationSpace("t")
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
# draft an overlay whose target shard is not attached -> apply can't find an adapter
ov = space.overlays.draft(Identity("ghost", "X"), "body", base_rev=None)
with pytest.raises(KeyError):
space.apply_overlay(ov.overlay_id)
def test_kernel_delete_missing_raises():
with pytest.raises(KeyError):
EngineKernel("eng").delete("nope")
def test_placement_str():
assert str(Placement("shardA", "sub/Page")) == "shardA/sub/Page"
class _BrokenProfileAdapter(ShardAdapter):
"""profile() raises — the conformance battery must report failure, not crash."""
@property
def shard_id(self) -> str:
return "broken"
def profile(self) -> CapabilityProfile:
raise RuntimeError("profile blew up")
def keys(self) -> Iterable[str]:
return []
def read(self, key: str) -> Page:
return Page(Identity("broken", key), "x", ProvenanceEnvelope(source_shard="broken"))
def test_conformance_survives_a_broken_profile():
report = run_conformance(_BrokenProfileAdapter())
assert not report.ok
assert any(c.name == "profile-validates" and not c.ok for c in report.checks)

64
tests/test_ext_struct.py Normal file
View File

@@ -0,0 +1,64 @@
"""Tests for the ext.struct built-in extension (SHARD-WP-0014 T6)."""
import pytest
from shard_wiki import InformationSpace
from shard_wiki.adapters import assert_conformant
from shard_wiki.engine import ExtensionRuntime, build_engine_shard
from shard_wiki.engine.extensions import StructExt, parse_frontmatter
from shard_wiki.model import PageShape, Verb
_STRUCT_PAGE = "---\ntitle: Spec\nstatus: draft\n---\nbody text"
def test_parse_frontmatter():
fields, has = parse_frontmatter(_STRUCT_PAGE)
assert has and fields == {"title": "Spec", "status": "draft"}
assert parse_frontmatter("just prose") == ({}, False)
assert parse_frontmatter("---\nunterminated") == ({}, False)
def _runtime(allowed=None):
rt = ExtensionRuntime()
rt.register(StructExt(allowed_fields=allowed))
return rt
def test_feature_absent_when_extension_off():
shard = build_engine_shard("off", ExtensionRuntime(), activate=set())
shard.write("Spec", _STRUCT_PAGE)
assert shard.read("Spec").shape is PageShape.PROSE # kernel: opaque prose
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # honest absence
def test_feature_present_when_extension_on():
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
shard.write("Spec", _STRUCT_PAGE)
assert shard.read("Spec").shape is PageShape.TYPED_RECORD # tagged by ext.struct
assert shard.read("Spec").body.endswith("body text") # content preserved (in-text)
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # profile reflects activation (E-5)
assert_conformant(shard) # still conformant
def test_plain_page_is_not_tagged_even_when_on():
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
shard.write("Plain", "no frontmatter here")
assert shard.read("Plain").shape is PageShape.PROSE
def test_allowed_fields_validation_rejects_disallowed():
shard = build_engine_shard("v", _runtime(allowed={"title"}), activate={"ext.struct"})
with pytest.raises(ValueError, match="disallowed fields"):
shard.write("Bad", "---\ntitle: ok\nsecret: no\n---\nx")
shard.write("Good", "---\ntitle: ok\n---\nx") # allowed field passes
assert shard.read("Good").shape is PageShape.TYPED_RECORD
def test_through_information_space_edit():
space = InformationSpace("team")
space.attach(build_engine_shard("wikiE", _runtime(), activate={"ext.struct"}))
space.union.shard("wikiE").write("Doc", "---\ntitle: T\n---\nv1")
res = space.edit("Doc", "---\ntitle: T2\n---\nv2") # overlay→apply→write-through
assert res.status.value == "applied"
page = space.read("Doc")
assert page.shape is PageShape.TYPED_RECORD and "v2" in page.body

View File

@@ -0,0 +1,44 @@
"""Tests for the FolderAdapter (SHARD-WP-0007 T3)."""
import pytest
from shard_wiki.adapters import FolderAdapter
from shard_wiki.model import Identity, NotSupported, Verb
def _make_shard(tmp_path, files: dict[str, str]) -> FolderAdapter:
for rel, text in files.items():
p = tmp_path / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter("shardA", tmp_path)
def test_keys_and_read(tmp_path):
shard = _make_shard(tmp_path, {"Home.md": "# Home", "sub/Topic.md": "topic body"})
assert set(shard.keys()) == {"Home", "sub/Topic"}
page = shard.read("sub/Topic")
assert page.identity == Identity("shardA", "sub/Topic")
assert page.body == "topic body"
assert page.envelope.source_shard == "shardA"
assert page.envelope.source_rev is not None # mtime stamp
assert page.placements[0].path == "sub/Topic.md"
def test_read_missing_raises_keyerror(tmp_path):
shard = _make_shard(tmp_path, {"Home.md": "x"})
with pytest.raises(KeyError):
shard.read("Nope")
def test_profile_is_valid_and_read_only(tmp_path):
shard = _make_shard(tmp_path, {"Home.md": "x"})
prof = shard.profile() # .validate() already called inside
assert prof.supports(Verb.READ)
assert not prof.supports(Verb.WRITE)
def test_unsupported_write_is_honest(tmp_path):
shard = _make_shard(tmp_path, {"Home.md": "x"})
with pytest.raises(NotSupported):
shard.write("Home", "new")

131
tests/test_git_adapter.py Normal file
View File

@@ -0,0 +1,131 @@
"""Tests for the GitShardAdapter read path + profile (SHARD-WP-0012 T1)."""
import subprocess
import pytest
from shard_wiki.adapters import GitShardAdapter, run_conformance
from shard_wiki.model import (
AttachmentMode,
History,
NotSupported,
ProfileError,
Substrate,
Verb,
)
def _git(repo, *args):
subprocess.run(
["git", "-C", str(repo), *args],
check=True,
capture_output=True,
env={"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
"PATH": __import__("os").environ.get("PATH", "")},
)
def _repo(tmp_path, files, name="repo"):
repo = tmp_path / name
repo.mkdir()
_git(repo, "init", "--quiet")
for rel, text in files.items():
p = repo / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
_git(repo, "add", rel)
_git(repo, "commit", "-m", "seed")
return repo
def test_keys_are_tracked_md_paths(tmp_path):
repo = _repo(tmp_path, {"Home.md": "h", "docs/Guide.md": "g", "ignore.txt": "x"})
adapter = GitShardAdapter("git", repo)
assert set(adapter.keys()) == {"Home", "docs/Guide"} # only tracked *.md
def test_read_returns_page_with_commit_sha_rev(tmp_path):
repo = _repo(tmp_path, {"Home.md": "welcome"})
adapter = GitShardAdapter("git", repo)
page = adapter.read("Home")
assert page.identity.shard == "git"
assert page.body == "welcome"
head = subprocess.run(
["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True, check=True
).stdout.strip()
assert page.envelope.source_rev == head # source_rev is the commit sha
assert page.envelope.lineage == "git-native"
def test_read_missing_key_raises(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(KeyError):
adapter.read("Nope")
def test_profile_validates_implication_rules(tmp_path):
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
assert profile.substrate is Substrate.GIT
assert profile.attachment_mode is AttachmentMode.GIT_IS_STORE
assert profile.history is History.GIT_NATIVE # git-is-store ⟹ git-native
profile.validate() # raises if the implication rule were violated
def test_profile_is_read_only_in_t1(tmp_path):
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
assert profile.supports(Verb.READ)
assert not profile.supports(Verb.WRITE)
def test_conformance_read_path_passes(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h", "Other.md": "o"}))
report = run_conformance(adapter)
assert report.ok, report.diff()
def test_unclaimed_write_raises_not_supported(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(NotSupported):
adapter.write("Home", "new") # read-only: honest absence
def test_empty_repo_has_no_keys(tmp_path):
repo = tmp_path / "empty"
repo.mkdir()
_git(repo, "init", "--quiet")
adapter = GitShardAdapter("git", repo)
assert list(adapter.keys()) == []
def test_bad_profile_combo_is_rejected():
# Sanity: the implication rule that backs the git profile actually bites when violated.
from shard_wiki.model import (
AccessGrant,
Addressing,
CapabilityProfile,
ContentOpacity,
MergeModel,
NativeQuery,
OperationalEnvelope,
Translation,
WriteGranularity,
)
from shard_wiki.provenance import Liveness
with pytest.raises(ProfileError):
CapabilityProfile(
substrate=Substrate.FILES, # not git, but claims git-is-store
attachment_mode=AttachmentMode.GIT_IS_STORE,
write_granularity=WriteGranularity.NONE,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.NONE,
merge_model=MergeModel.NONE,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset({Verb.READ}),
).validate()

View File

@@ -0,0 +1,116 @@
"""GitShardAdapter history adopt + cross-substrate integration (SHARD-WP-0012 T3)."""
import os
import subprocess
import pytest
from shard_wiki.adapters import FolderAdapter, GitShardAdapter
from shard_wiki.coordination import ApplyStatus
from shard_wiki.space import InformationSpace
_ENV = {
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
"PATH": os.environ.get("PATH", ""),
}
def _git(repo, *args):
return subprocess.run(
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
).stdout.strip()
def _git_repo(tmp_path, files, name="git"):
repo = tmp_path / name
repo.mkdir()
_git(repo, "init", "--quiet")
for rel, text in files.items():
(repo / rel).parent.mkdir(parents=True, exist_ok=True)
(repo / rel).write_text(text, encoding="utf-8")
_git(repo, "add", rel)
_git(repo, "commit", "-m", "seed")
return repo
def _folder(tmp_path, name, files, writable=False):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root, writable=writable)
# -- history adopt -------------------------------------------------------------
def test_history_lists_commits_newest_first(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "v1"})
adapter = GitShardAdapter("git", repo, writable=True)
adapter.write("Home", "v2")
history = adapter.history("Home")
assert len(history) == 2
assert history[0].message == "write Home.md" # newest first
assert history[-1].message == "seed"
assert all(rev.sha for rev in history)
def test_history_unknown_key_raises(tmp_path):
adapter = GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(KeyError):
adapter.history("Nope")
# -- cross-substrate integration ----------------------------------------------
def test_resolve_across_git_and_folder(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "git home"})))
space.attach(_folder(tmp_path, "notes", {"Daily.md": "folder daily"}))
assert space.read("Home").body == "git home" # resolved from the git shard
assert space.read("Daily").body == "folder daily" # resolved from the folder shard
def test_chorus_spans_substrates_with_divergence(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Shared.md": "from git"})))
space.attach(_folder(tmp_path, "notes", {"Shared.md": "from folder"}))
res = space.resolve("Shared")
assert {p.body for p in res.pages} == {"from git", "from folder"} # chorus across substrates
git_page = next(p for p in res.pages if p.identity.shard == "git")
assert git_page.envelope.divergence # divergence recorded, not erased
def test_edit_through_git_shard_commits(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "original"})
space = InformationSpace("space")
space.attach(GitShardAdapter("git", repo, writable=True))
result = space.edit("Home", "edited via overlay")
assert result.status is ApplyStatus.APPLIED # write-through fast-forward on a git shard
assert space.read("Home").body == "edited via overlay"
assert int(_git(repo, "rev-list", "--count", "HEAD")) == 2 # the edit became a commit
def test_apply_under_drift_refuses_on_external_commit(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "original"})
space = InformationSpace("space")
space.attach(GitShardAdapter("git", repo, writable=True))
overlay = space.overlay("Home", "my draft") # base_rev = current git sha
# Another writer commits to the same path → the sha moves underneath the draft.
(repo / "Home.md").write_text("someone else", encoding="utf-8")
_git(repo, "add", "Home.md")
_git(repo, "commit", "-m", "external")
result = space.apply_overlay(overlay.overlay_id)
assert result.status is ApplyStatus.REFUSED_DRIFT # never clobber (sha drift detected)
# The shard itself is untouched — the external commit stands; the draft remains a draft.
assert space.union.shard("git").read("Home").body == "someone else"
def test_overlay_on_read_only_git_shard_kept_as_draft(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "ro"}), writable=False))
result = space.edit("Home", "wanted change")
assert result.status is ApplyStatus.KEPT_DRAFT # read-only target → overlay retained

Some files were not shown because too many files have changed in this diff Show More