Compare commits

...

4 Commits

Author SHA1 Message Date
c731c96634 feat(coordination): git backend wiring + verbatim log migration (WP-0009 T4)
InformationSpace.git_backed(space_id, repo_path) wires the git coordination log;
the default constructor stays in-memory for tests (new keyword-only store=). A
one-time importer (migrate_space / import_log / JSONL export+import) replays an
existing in-memory or JSON log into git verbatim — preserving seq, timestamp and
actor (union-without-erasure) and refusing out-of-order import. Same fold after
migration; no behavioural change to overlay/union. SCOPE updated; WP-0009 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:49:55 +02:00
f0fee65cc0 test(coordination): cross-process read-your-writes + fold parity (WP-0009 T3)
Verify the git backend's fold reads the durable log into CoordinationState with
unchanged semantics, and that read-your-writes holds across separate handles and
separate OS processes against the same space ref (one test spawns a real
subprocess that appends, then reads it back). Cross-process fold equals the
in-memory fold for the same event sequence (derived = f(log)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:47:08 +02:00
34432c2e15 feat(coordination): per-space append authority (lease) (WP-0009 T2)
A single append authority per space serializes appends into a total order: at
most one node holds a space's lease; only the holder writes, non-holders forward
their append intent to the holder. Leases are time-bounded and re-grantable, so
a dead holder's lease expires and a new node resumes from the log head (seq stays
contiguous). A stale ex-holder discovers it is no longer the holder and forwards
rather than writing, so a partitioned node cannot fork the log. Works over both
in-memory and git stores. Single-coordinator only (distributed leasing out of scope).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:45:52 +02:00
45a858ead0 feat(coordination): git-backed DecisionLog event store (WP-0009 T1)
Factor DecisionLog storage behind an EventStore abstraction: InMemoryEventStore
stays the default/test double, GitEventStore makes the coordination log
git-addressable. Each space is a ref (refs/spaces/<sha1>); append writes an
immutable one-blob commit and advances the ref under compare-and-swap, so the
commit chain is the per-space total order and a racing appender can never fork
the log. Deterministic stable-JSON event serialization. Zero runtime deps
(git CLI via subprocess). API and fold unchanged across backends.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:41:27 +02:00
12 changed files with 897 additions and 16 deletions

View File

@@ -17,7 +17,7 @@ Learnings update both SCOPE and INTENT where necessary.
| Layer | State |
|-------|-------|
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. 107 tests green, ~97% coverage |
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. 128 tests green, ~97% coverage |
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
| Demand | NetKingdom integration asks captured, not yet negotiated |

View File

@@ -4,7 +4,24 @@ from shard_wiki.coordination.decision_log import (
CoordinationState,
DecisionEvent,
DecisionLog,
EventStore,
EventType,
InMemoryEventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.append_authority import (
AppendAuthority,
Lease,
LeaseHeld,
LeaseRegistry,
)
from shard_wiki.coordination.git_event_store import GitEventStore
from shard_wiki.coordination.migration import (
export_jsonl,
import_jsonl,
import_log,
migrate_space,
)
from shard_wiki.coordination.overlay import (
ApplyResult,
@@ -19,6 +36,19 @@ __all__ = [
"DecisionEvent",
"EventType",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"GitEventStore",
"Lease",
"LeaseHeld",
"LeaseRegistry",
"AppendAuthority",
"import_log",
"migrate_space",
"export_jsonl",
"import_jsonl",
"serialize_event",
"deserialize_event",
"Overlay",
"OverlayEngine",
"ApplyStatus",

View File

@@ -0,0 +1,158 @@
"""Per-space append authority — the single-writer lease over the log (SHARD-WP-0009 T2).
The log is a *total order per space* (§8.6). :class:`~shard_wiki.coordination.git_event_store`
makes a fork physically impossible via compare-and-swap; this layer adds the **policy** that gives
the order a single designated writer: a **per-space lease**. At most one node holds a space's lease
at a time; only the holder writes to the store. A non-holder does not write — it **forwards** its
append intent to the current holder, so intents from anywhere still land in one serialized stream.
The lease is **time-bounded and re-grantable** (HA): if a holder dies, its lease expires and a new
node may take it, resuming appends from the log head (``seq`` stays contiguous across the hand-off).
A node holding a *stale* lease (already re-granted elsewhere) cannot write either — it discovers it
is no longer the holder and forwards instead, so a partitioned ex-holder can never fork the log.
Mechanism over policy (CLAUDE.md): this provides the leasing *primitive*; who acquires when, and
the TTL, are the caller's policy. Single-coordinator only — distributed multi-node leasing and log
sharding are explicit non-goals of this workplan.
"""
from __future__ import annotations
import uuid
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any
from shard_wiki.coordination.decision_log import DecisionEvent, EventStore, EventType
__all__ = ["Lease", "LeaseHeld", "LeaseRegistry", "AppendAuthority"]
def _utcnow() -> datetime:
return datetime.now(tz=timezone.utc)
@dataclass(frozen=True, slots=True)
class Lease:
"""A time-bounded grant of single-writer authority over one space."""
space: str
holder: str
token: str
expires_at: datetime
def valid_at(self, now: datetime) -> bool:
return now < self.expires_at
class LeaseHeld(Exception):
"""Raised when a space's lease is validly held by a different node."""
def __init__(self, lease: Lease) -> None:
super().__init__(
f"space {lease.space!r} leased to {lease.holder!r} until {lease.expires_at}"
)
self.lease = lease
class LeaseRegistry:
"""The single coordinator's grant table: at most one *valid* lease per space.
A lease that has expired is freely re-grantable to any node (the HA replacement path); a still
valid lease is exclusive to its holder (renewable by that holder). The registry also routes
forwarded append intents to the current holder node.
"""
def __init__(self, clock: Callable[[], datetime] = _utcnow) -> None:
self._clock = clock
self._leases: dict[str, Lease] = {}
self._nodes: dict[str, AppendAuthority] = {}
def register(self, node: AppendAuthority) -> None:
self._nodes[node.node_id] = node
def grant(self, space: str, holder: str, ttl_seconds: float) -> Lease:
"""Grant/renew the lease for ``space`` to ``holder``; raise :class:`LeaseHeld` if another
node still holds it validly. An expired lease is re-grantable to anyone."""
now = self._clock()
current = self._leases.get(space)
if current is not None and current.valid_at(now) and current.holder != holder:
raise LeaseHeld(current)
lease = Lease(
space=space,
holder=holder,
token=uuid.uuid4().hex,
expires_at=now + timedelta(seconds=ttl_seconds),
)
self._leases[space] = lease
return lease
def current(self, space: str) -> Lease | None:
"""The lease for ``space`` if one is currently valid, else None (expired/absent)."""
lease = self._leases.get(space)
return lease if lease is not None and lease.valid_at(self._clock()) else None
def holder_node(self, space: str) -> AppendAuthority | None:
lease = self.current(space)
return self._nodes.get(lease.holder) if lease is not None else None
class AppendAuthority:
"""A coordinator node that appends to the shared log only when it holds the space's lease.
Nodes share one :class:`EventStore` and one :class:`LeaseRegistry`. ``append`` routes itself:
the holder writes; a non-holder forwards to whoever holds the lease (acquiring it first if the
space is currently unleased). The append API mirrors :class:`EventStore` so the authority is a
drop-in single-writer guard.
"""
def __init__(
self,
node_id: str,
store: EventStore,
registry: LeaseRegistry,
ttl_seconds: float = 30.0,
) -> None:
self.node_id = node_id
self._store = store
self._registry = registry
self._ttl = ttl_seconds
registry.register(self)
def acquire(self, space: str) -> Lease:
"""Take (or renew) the lease for ``space``. Raises :class:`LeaseHeld` if another node holds
it validly."""
return self._registry.grant(space, self.node_id, self._ttl)
def holds(self, space: str) -> bool:
lease = self._registry.current(space)
return lease is not None and lease.holder == self.node_id
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append via the single authority. If we hold the lease, write; otherwise forward to the
holder. If the space is unleased, acquire it first. A node with a *stale* lease forwards
(it is not the current holder) rather than writing — so it cannot fork the log."""
holder_node = self._registry.holder_node(space)
if holder_node is None:
self.acquire(space) # unleased: take authority, then write below
holder_node = self
if holder_node is self:
return self._store.append(space, type, payload, actor=actor)
return holder_node._write(space, type, payload, actor=actor)
def _write(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None,
) -> DecisionEvent:
"""Apply a forwarded intent. Called only on the lease holder by a forwarding peer."""
return self._store.append(space, type, payload, actor=actor)

View File

@@ -3,22 +3,36 @@
Coordination-canonical state (overlays, equivalence bindings, aliases, merges, forks) is an
**append-only decision log**, not a mutable file; the queryable *current* state is a **derived
fold** of the log (tier-3 disposable). The log is **totally ordered per space** via a single
**append authority** — here an in-process counter; a git-backed, lease-held authority is a later
binding. That total order is what gives read-your-writes across readers (§8.6).
**append authority**. That total order is what gives read-your-writes across readers (§8.6).
Storage lives behind :class:`EventStore`: :class:`InMemoryEventStore` is the default test double
(an in-process counter); :class:`~shard_wiki.coordination.git_event_store.GitEventStore` is the
git-addressable backend (SHARD-WP-0009). The :class:`DecisionLog` API and the :meth:`fold` are
identical across backends — only storage + the concurrency model differ.
`derived = f(canonical)`: :class:`CoordinationState` is always reproducible by replaying the log.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from types import MappingProxyType
from typing import Any
from typing import Any, Protocol, runtime_checkable
__all__ = ["EventType", "DecisionEvent", "CoordinationState", "DecisionLog"]
__all__ = [
"EventType",
"DecisionEvent",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"DecisionLog",
"serialize_event",
"deserialize_event",
]
class EventType(Enum):
@@ -63,10 +77,57 @@ class CoordinationState:
return frozenset({identity})
class DecisionLog:
"""In-memory append-only log, totally ordered per space (the append authority for a process).
def serialize_event(event: DecisionEvent) -> bytes:
"""Deterministic, stable-JSON wire form of an event (same bytes for equal events, any process).
A later binding swaps the storage for git + a per-space lease without changing this API.
Sorted keys + compact separators make the serialization canonical, so a git object hashed from
it is reproducible — the basis for content-addressable, comparable logs across backends.
"""
obj = {
"seq": event.seq,
"space": event.space,
"type": event.type.value,
"payload": event.payload,
"actor": event.actor,
"timestamp": event.timestamp.isoformat(),
}
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
def deserialize_event(data: bytes | str) -> DecisionEvent:
"""Inverse of :func:`serialize_event` — round-trips an event byte-for-byte by field."""
obj = json.loads(data)
return DecisionEvent(
seq=obj["seq"],
space=obj["space"],
type=EventType(obj["type"]),
payload=obj["payload"],
actor=obj["actor"],
timestamp=datetime.fromisoformat(obj["timestamp"]),
)
@runtime_checkable
class EventStore(Protocol):
"""Append-only, per-space ordered storage behind :class:`DecisionLog`.
Two bindings exist: :class:`InMemoryEventStore` (default/test double) and
:class:`~shard_wiki.coordination.git_event_store.GitEventStore` (git-addressable). Both assign
a per-space monotonic ``seq`` at the log head and guarantee read-your-writes for their reach
(in-process for memory; cross-process for git).
"""
def append(
self, space: str, type: EventType, payload: Mapping[str, Any], actor: str | None = None
) -> DecisionEvent: ...
def events(self, space: str) -> tuple[DecisionEvent, ...]: ...
class InMemoryEventStore:
"""In-process append-only store, totally ordered per space (the append authority for a process).
The default test double; the git backend preserves this exact contract on durable storage.
"""
def __init__(self) -> None:
@@ -84,10 +145,33 @@ class DecisionLog:
self._events.setdefault(space, []).append(event)
return event
def events(self, space: str) -> tuple[DecisionEvent, ...]:
return tuple(self._events.get(space, ()))
class DecisionLog:
"""Append-only decision log, totally ordered per space, with a derived :meth:`fold`.
Storage is delegated to an :class:`EventStore` (default :class:`InMemoryEventStore`); swapping
in the git backend changes only durability + the concurrency model, not this API or the fold.
"""
def __init__(self, store: EventStore | None = None) -> None:
self._store: EventStore = store if store is not None else InMemoryEventStore()
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
return self._store.append(space, type, payload, actor=actor)
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events in append (total) order. Read-your-writes: a just-appended event
is present immediately."""
return tuple(self._events.get(space, ()))
return self._store.events(space)
def fold(self, space: str) -> CoordinationState:
"""Replay the log into current coordination state (derived = f(log))."""

View File

@@ -0,0 +1,172 @@
"""GitEventStore — a git-addressable binding of :class:`EventStore` (SHARD-WP-0009 T1).
Each space is a ref (``refs/spaces/<sha1(space)>``); each ``append`` writes the event as an
immutable git object (a one-blob tree committed onto the ref) and advances the ref. The commit
chain *is* the totally ordered log: ``seq`` is the depth, ``events`` walks first-parent from the
head oldest→newest. Coordination-canonical state therefore inherits git's history / patch /
review / backup affordances (I-6) and is read-your-writes correct across processes.
The total order is enforced at storage by a **compare-and-swap** ref update
(``git update-ref <ref> <new> <old>``): two appenders racing off the same head — the loser's CAS
fails and it retries off the new head, so a non-holder can never fork the log. The lease layer
(T2) sits *above* this as the append-authority policy; CAS is the mechanism that makes it safe.
Implemented over the ``git`` CLI through :mod:`subprocess` — zero runtime dependencies.
"""
from __future__ import annotations
import hashlib
import os
import subprocess
from collections.abc import Mapping
from pathlib import Path
from typing import Any
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventType,
deserialize_event,
serialize_event,
)
__all__ = ["GitEventStore"]
# Fixed identity so commit objects are reproducible and never prompt for git config; the event's
# own timestamp/actor carry the real provenance, the commit is just the ordered container.
_GIT_IDENTITY = {
"GIT_AUTHOR_NAME": "shard-wiki",
"GIT_AUTHOR_EMAIL": "coordination@shard-wiki",
"GIT_COMMITTER_NAME": "shard-wiki",
"GIT_COMMITTER_EMAIL": "coordination@shard-wiki",
}
_EVENT_PATH = "event.json"
_MAX_CAS_RETRIES = 50
class GitEventStore:
"""Git-backed, append-only, per-space ordered event store (an :class:`EventStore`)."""
def __init__(self, repo_path: str | Path) -> None:
self.repo_path = Path(repo_path)
self.repo_path.mkdir(parents=True, exist_ok=True)
if not (self.repo_path / "HEAD").exists() and not (self.repo_path / ".git").exists():
self._git("init", "--quiet", str(self.repo_path), at_cwd=True)
# -- EventStore contract -------------------------------------------------
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append one event, advancing the space ref under compare-and-swap (retry-on-race)."""
ref = self._ref(space)
for _ in range(_MAX_CAS_RETRIES):
head = self._head(ref)
seq = self._count(ref, head)
event = DecisionEvent(
seq=seq, space=space, type=type, payload=dict(payload), actor=actor
)
commit = self._commit_event(event, parent=head)
if self._cas_update(ref, new=commit, old=head):
return event
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
def import_event(self, event: DecisionEvent) -> None:
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
"""
ref = self._ref(event.space)
head = self._head(ref)
expected = self._count(ref, head)
if event.seq != expected:
raise ValueError(
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
)
commit = self._commit_event(event, parent=head)
if not self._cas_update(ref, new=commit, old=head):
raise RuntimeError(f"import race on {ref}")
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events oldest→newest (append/total order)."""
ref = self._ref(space)
head = self._head(ref)
if head is None:
return ()
shas = self._git("rev-list", "--reverse", "--first-parent", ref).decode().split()
return tuple(
deserialize_event(self._git("cat-file", "blob", f"{sha}:{_EVENT_PATH}"))
for sha in shas
)
# -- git plumbing --------------------------------------------------------
def _commit_event(self, event: DecisionEvent, parent: str | None) -> str:
blob = self._git(
"hash-object", "-w", "--stdin", stdin=serialize_event(event)
).decode().strip()
tree = self._git(
"mktree", stdin=f"100644 blob {blob}\t{_EVENT_PATH}\n".encode()
).decode().strip()
args = ["commit-tree", tree, "-m", f"event {event.seq} {event.type.value}"]
if parent is not None:
args += ["-p", parent]
# Pin the commit date to the event's timestamp for reproducible objects.
date = event.timestamp.isoformat()
env = {**_GIT_IDENTITY, "GIT_AUTHOR_DATE": date, "GIT_COMMITTER_DATE": date}
return self._git(*args, env=env).decode().strip()
def _cas_update(self, ref: str, new: str, old: str | None) -> bool:
"""``git update-ref`` with the old value as a CAS guard (empty oldvalue == must-not-exist).
Returns False if the ref moved since we read ``old`` (lost the race) — the caller retries.
"""
result = self._run("update-ref", ref, new, old if old is not None else "")
return result.returncode == 0
def _head(self, ref: str) -> str | None:
result = self._run("rev-parse", "--verify", "--quiet", ref)
out = result.stdout.decode().strip()
return out or None
def _count(self, ref: str, head: str | None) -> int:
if head is None:
return 0
return int(self._git("rev-list", "--count", "--first-parent", ref).decode().strip())
@staticmethod
def _ref(space: str) -> str:
return f"refs/spaces/{hashlib.sha1(space.encode()).hexdigest()}"
def _git(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
) -> bytes:
result = self._run(*args, stdin=stdin, env=env, at_cwd=at_cwd, check=True)
return result.stdout
def _run(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
check: bool = False,
) -> subprocess.CompletedProcess:
base = ["git"] if at_cwd else ["git", "-C", str(self.repo_path)]
return subprocess.run(
[*base, *args],
input=stdin,
capture_output=True,
env={**os.environ, **(env or {})},
check=check,
)

View File

@@ -0,0 +1,53 @@
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
reproduces identical coordination state; only durability changes.
"""
from __future__ import annotations
from collections.abc import Iterable
from pathlib import Path
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.git_event_store import GitEventStore
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
count = 0
for event in events:
dest.import_event(event)
count += 1
return count
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
return import_log(source.events(space), dest)
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
count = 0
with open(path, "wb") as handle:
for event in events:
handle.write(serialize_event(event) + b"\n")
count += 1
return count
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
with open(path, "rb") as handle:
events = [deserialize_event(line) for line in handle if line.strip()]
return import_log(events, dest)

View File

@@ -8,11 +8,15 @@ a network API is a later workplan.
from __future__ import annotations
from pathlib import Path
from shard_wiki.adapters import ShardAdapter, assert_conformant
from shard_wiki.coordination import (
ApplyResult,
DecisionLog,
EventStore,
EventType,
GitEventStore,
Overlay,
OverlayEngine,
)
@@ -24,12 +28,31 @@ __all__ = ["InformationSpace"]
class InformationSpace:
def __init__(self, space_id: str, policy: Policy = DEFAULT_POLICY) -> None:
def __init__(
self,
space_id: str,
policy: Policy = DEFAULT_POLICY,
*,
store: EventStore | None = None,
) -> None:
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
self.space_id = space_id
self.log = DecisionLog()
self.log = DecisionLog(store)
self.union = UnionGraph(space_id, log=self.log, policy=policy)
self.overlays = OverlayEngine(space_id, self.log)
@classmethod
def git_backed(
cls,
space_id: str,
repo_path: str | Path,
policy: Policy = DEFAULT_POLICY,
) -> InformationSpace:
"""An information space whose coordination log is git-addressable (history/patch/review/
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
return cls(space_id, policy, store=GitEventStore(repo_path))
def attach(self, adapter: ShardAdapter) -> None:
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
assert_conformant(adapter)

View File

@@ -0,0 +1,120 @@
"""Tests for the per-space append authority / lease (SHARD-WP-0009 T2).
A single append authority per space serializes appends into a total order; non-holders forward
intents to the holder; the lease is time-bounded and re-grantable (HA hand-off); a stale ex-holder
cannot fork the log.
"""
from datetime import datetime, timedelta, timezone
import pytest
from shard_wiki.coordination import (
AppendAuthority,
EventType,
GitEventStore,
InMemoryEventStore,
LeaseHeld,
LeaseRegistry,
)
class FakeClock:
def __init__(self):
self.now = datetime(2026, 1, 1, tzinfo=timezone.utc)
def __call__(self):
return self.now
def advance(self, seconds):
self.now += timedelta(seconds=seconds)
def test_only_one_node_holds_a_space_at_a_time():
reg = LeaseRegistry()
a = AppendAuthority("A", InMemoryEventStore(), reg)
b = AppendAuthority("B", InMemoryEventStore(), reg)
a.acquire("s")
with pytest.raises(LeaseHeld):
b.acquire("s") # B is refused while A's lease is valid
def test_concurrent_appends_serialize_into_one_total_order():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
# B is a non-holder: its append forwards to A, the holder. Interleave A and B writers.
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"}) # forwarded
a.append("s", EventType.ALIAS_SET, {"alias": "3", "target": "x:3"})
seqs = [e.seq for e in store.events("s")]
aliases = [e.payload["alias"] for e in store.events("s")]
assert seqs == [0, 1, 2] # contiguous total order despite two writers
assert aliases == ["1", "2", "3"]
def test_non_holder_forwards_rather_than_writing_directly():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
assert not b.holds("s")
b.append("s", EventType.ALIAS_SET, {"alias": "fwd", "target": "x:1"})
# The write landed on the shared store under A's authority, in one stream.
assert [e.payload["alias"] for e in store.events("s")] == ["fwd"]
def test_lease_handoff_resumes_from_head():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
clock.advance(20) # A's lease expires (A "dies")
b.acquire("s") # re-grantable: B takes over
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
assert [e.seq for e in store.events("s")] == [0, 1, 2] # contiguous across hand-off
def test_stale_ex_holder_cannot_fork_the_log():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
clock.advance(20)
b.acquire("s") # B is now the holder; A's lease is stale
b.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
# A still thinks it can write, but it's no longer the holder: its intent forwards to B.
assert not a.holds("s")
a.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
aliases = [e.payload["alias"] for e in store.events("s")]
assert aliases == ["0", "1", "2"] # one stream, no fork
def test_authority_over_git_store_keeps_total_order(tmp_path):
reg = LeaseRegistry()
store = GitEventStore(tmp_path / "coord")
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
a.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
b.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"}) # forwarded
assert [e.seq for e in store.events("s")] == [0, 1]
def test_unleased_space_self_acquires_on_append():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
a.append("s", EventType.ALIAS_SET, {"alias": "x", "target": "y:1"}) # no explicit acquire
assert a.holds("s")
assert len(store.events("s")) == 1

View File

@@ -0,0 +1,74 @@
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
export_jsonl,
import_jsonl,
migrate_space,
)
from shard_wiki.space import InformationSpace
def test_information_space_git_backed_uses_git_log(tmp_path):
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
assert isinstance(space.log._store, GitEventStore)
space.alias("Home", "shardA:Index")
# Read-your-writes through the orchestrator's git-backed log.
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
def test_default_information_space_stays_in_memory():
space = InformationSpace("space-1")
assert isinstance(space.log._store, InMemoryEventStore)
def test_migrate_space_preserves_order_and_provenance(tmp_path):
source = InMemoryEventStore()
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
dest = GitEventStore(tmp_path / "coord")
n = migrate_space(source, "s", dest)
assert n == 2
migrated = dest.events("s")
assert [e.seq for e in migrated] == [0, 1]
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
assert migrated[0].actor == "ana"
assert migrated[1].actor == "ben"
assert migrated[0].timestamp == e0.timestamp
def test_migration_yields_identical_fold(tmp_path):
source = DecisionLog(InMemoryEventStore())
for typ, payload in [
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
]:
source.append("s", typ, payload)
dest = GitEventStore(tmp_path / "coord")
migrate_space(source._store, "s", dest)
after = DecisionLog(dest)
assert after.fold("s").aliases == source.fold("s").aliases
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
def test_jsonl_round_trip_into_git(tmp_path):
source = InMemoryEventStore()
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
path = tmp_path / "log.jsonl"
assert export_jsonl(source.events("s"), path) == 2
dest = GitEventStore(tmp_path / "coord")
assert import_jsonl(path, dest) == 2
state = DecisionLog(dest).fold("s")
assert state.resolve_alias("Home") == "x:1"
assert state.equivalent_to("p") == frozenset({"p", "q"})

View File

@@ -0,0 +1,83 @@
"""Cross-process read-your-writes over the git log + fold parity (SHARD-WP-0009 T3).
The git backend's value over the in-memory double is that the totally ordered log is durable and
shared: a write by one process/handle is immediately visible to another opening the same ref, and
the derived fold is identical to the in-memory fold of the same event sequence (derived = f(log)).
"""
import os
import subprocess
import sys
import textwrap
from pathlib import Path
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
)
_SRC = str(Path(__file__).resolve().parents[1] / "src")
def test_new_handle_sees_prior_writes(tmp_path):
repo = tmp_path / "coord"
writer = DecisionLog(GitEventStore(repo))
writer.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
writer.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
# A second, independent handle on the same repo — read-your-writes across handles.
reader = DecisionLog(GitEventStore(repo))
assert [e.seq for e in reader.events("s")] == [0, 1]
assert reader.fold("s").resolve_alias("Home") == "shardA:Index"
def test_append_in_separate_process_is_visible(tmp_path):
repo = tmp_path / "coord"
# Seed from this process so the repo exists.
DecisionLog(GitEventStore(repo)).append(
"s", EventType.ALIAS_SET, {"alias": "A", "target": "x:1"}
)
child = textwrap.dedent(
f"""
from shard_wiki.coordination import DecisionLog, EventType, GitEventStore
log = DecisionLog(GitEventStore({str(repo)!r}))
log.append("s", EventType.ALIAS_SET, {{"alias": "B", "target": "x:2"}})
"""
)
result = subprocess.run(
[sys.executable, "-c", child],
capture_output=True,
text=True,
env={"PYTHONPATH": _SRC, "PATH": os.environ.get("PATH", "")},
)
assert result.returncode == 0, result.stderr
# This process, with a fresh handle, sees the child's append in order.
reader = DecisionLog(GitEventStore(repo))
assert [e.payload["alias"] for e in reader.events("s")] == ["A", "B"]
assert [e.seq for e in reader.events("s")] == [0, 1]
def test_cross_process_fold_equals_in_memory_fold(tmp_path):
sequence = [
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.PAGE_FORKED, {"source": "p", "fork": "q"}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
]
mem = DecisionLog(InMemoryEventStore())
for typ, payload in sequence:
mem.append("s", typ, payload)
repo = tmp_path / "coord"
DecisionLog(GitEventStore(repo)) # init repo
for typ, payload in sequence:
# Each append from a fresh handle to simulate distinct writers over time.
DecisionLog(GitEventStore(repo)).append("s", typ, payload)
git_state = DecisionLog(GitEventStore(repo)).fold("s")
mem_state = mem.fold("s")
assert git_state.aliases == mem_state.aliases
assert git_state.equivalence_groups == mem_state.equivalence_groups
assert git_state.equivalent_to("a") == frozenset({"a", "b", "c"})

View File

@@ -0,0 +1,84 @@
"""Tests for the git-backed event store (SHARD-WP-0009 T1).
The git backend must satisfy the same EventStore contract as the in-memory one (round-trip,
ordering, determinism) while making the log git-addressable.
"""
import subprocess
import pytest
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
deserialize_event,
serialize_event,
)
@pytest.fixture
def git_store(tmp_path):
return GitEventStore(tmp_path / "coord")
def test_append_git_read_round_trips(git_store):
log = DecisionLog(git_store)
ev = log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
(read,) = log.events("s")
assert read.seq == ev.seq == 0
assert read.space == "s"
assert read.type is EventType.ALIAS_SET
assert read.payload == {"alias": "Home", "target": "shardA:Index"}
def test_ordering_preserved_and_per_space_monotonic(git_store):
log = DecisionLog(git_store)
log.append("a", EventType.ALIAS_SET, {"alias": "X", "target": "s:1"})
log.append("a", EventType.ALIAS_SET, {"alias": "Y", "target": "s:2"})
log.append("b", EventType.ALIAS_SET, {"alias": "Z", "target": "s:3"})
assert [e.seq for e in log.events("a")] == [0, 1]
assert [e.payload["alias"] for e in log.events("a")] == ["X", "Y"]
assert [e.seq for e in log.events("b")] == [0] # independent ref/ordering
def test_each_append_is_a_git_commit(git_store):
log = DecisionLog(git_store)
log.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
log.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"})
ref = GitEventStore._ref("s")
count = subprocess.run(
["git", "-C", str(git_store.repo_path), "rev-list", "--count", ref],
capture_output=True, text=True, check=True,
).stdout.strip()
assert count == "2" # one immutable commit object per append
def test_deterministic_serialization_is_stable_and_sorted():
log = InMemoryEventStore()
ev = log.append("s", EventType.ALIAS_SET, {"target": "z", "alias": "a"})
blob = serialize_event(ev)
assert serialize_event(ev) == blob # stable across calls
assert blob.index(b'"alias"') < blob.index(b'"target"') # payload keys sorted, not insertion
assert deserialize_event(blob).payload == {"alias": "a", "target": "z"}
def test_git_fold_matches_in_memory_fold(git_store):
events = [
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
]
mem = DecisionLog(InMemoryEventStore())
git = DecisionLog(git_store)
for typ, payload in events:
mem.append("s", typ, payload)
git.append("s", typ, payload)
assert git.fold("s").aliases == mem.fold("s").aliases
assert git.fold("s").equivalence_groups == mem.fold("s").equivalence_groups
def test_default_decisionlog_is_in_memory():
assert isinstance(DecisionLog()._store, InMemoryEventStore)

View File

@@ -4,7 +4,7 @@ type: workplan
title: "git-backed DecisionLog + per-space append authority"
domain: whynot
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -39,7 +39,7 @@ sharding (blueprint O-12). Single append authority per space is the target.
```task
id: SHARD-WP-0009-T1
status: todo
status: done
priority: high
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
```
@@ -54,7 +54,7 @@ ordering preserved; deterministic serialization.
```task
id: SHARD-WP-0009-T2
status: todo
status: done
priority: high
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
```
@@ -69,7 +69,7 @@ hand-off resumes from head; a partitioned non-holder cannot fork the log.
```task
id: SHARD-WP-0009-T3
status: todo
status: done
priority: high
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
```
@@ -82,7 +82,7 @@ process B (new handle) sees it; fold equals the in-memory fold for the same even
```task
id: SHARD-WP-0009-T4
status: todo
status: done
priority: medium
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
```