generated from coulomb/repo-seed
feat(coordination): git backend wiring + verbatim log migration (WP-0009 T4)
InformationSpace.git_backed(space_id, repo_path) wires the git coordination log; the default constructor stays in-memory for tests (new keyword-only store=). A one-time importer (migrate_space / import_log / JSONL export+import) replays an existing in-memory or JSON log into git verbatim — preserving seq, timestamp and actor (union-without-erasure) and refusing out-of-order import. Same fold after migration; no behavioural change to overlay/union. SCOPE updated; WP-0009 done. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2
SCOPE.md
2
SCOPE.md
@@ -17,7 +17,7 @@ Learnings update both SCOPE and INTENT where necessary.
|
||||
|
||||
| Layer | State |
|
||||
|-------|-------|
|
||||
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. 107 tests green, ~97% coverage |
|
||||
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. 128 tests green, ~97% coverage |
|
||||
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
||||
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
||||
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
||||
|
||||
@@ -17,6 +17,12 @@ from shard_wiki.coordination.append_authority import (
|
||||
LeaseRegistry,
|
||||
)
|
||||
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||
from shard_wiki.coordination.migration import (
|
||||
export_jsonl,
|
||||
import_jsonl,
|
||||
import_log,
|
||||
migrate_space,
|
||||
)
|
||||
from shard_wiki.coordination.overlay import (
|
||||
ApplyResult,
|
||||
ApplyStatus,
|
||||
@@ -37,6 +43,10 @@ __all__ = [
|
||||
"LeaseHeld",
|
||||
"LeaseRegistry",
|
||||
"AppendAuthority",
|
||||
"import_log",
|
||||
"migrate_space",
|
||||
"export_jsonl",
|
||||
"import_jsonl",
|
||||
"serialize_event",
|
||||
"deserialize_event",
|
||||
"Overlay",
|
||||
|
||||
@@ -75,6 +75,24 @@ class GitEventStore:
|
||||
return event
|
||||
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
|
||||
|
||||
def import_event(self, event: DecisionEvent) -> None:
|
||||
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
|
||||
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
|
||||
|
||||
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
|
||||
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
|
||||
"""
|
||||
ref = self._ref(event.space)
|
||||
head = self._head(ref)
|
||||
expected = self._count(ref, head)
|
||||
if event.seq != expected:
|
||||
raise ValueError(
|
||||
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
|
||||
)
|
||||
commit = self._commit_event(event, parent=head)
|
||||
if not self._cas_update(ref, new=commit, old=head):
|
||||
raise RuntimeError(f"import race on {ref}")
|
||||
|
||||
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||
"""The space's events oldest→newest (append/total order)."""
|
||||
ref = self._ref(space)
|
||||
|
||||
53
src/shard_wiki/coordination/migration.py
Normal file
53
src/shard_wiki/coordination/migration.py
Normal file
@@ -0,0 +1,53 @@
|
||||
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
|
||||
|
||||
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
|
||||
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
|
||||
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
|
||||
reproduces identical coordination state; only durability changes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.coordination.decision_log import (
|
||||
DecisionEvent,
|
||||
EventStore,
|
||||
deserialize_event,
|
||||
serialize_event,
|
||||
)
|
||||
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||
|
||||
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
|
||||
|
||||
|
||||
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
|
||||
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
|
||||
count = 0
|
||||
for event in events:
|
||||
dest.import_event(event)
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
|
||||
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
|
||||
return import_log(source.events(space), dest)
|
||||
|
||||
|
||||
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
|
||||
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
|
||||
count = 0
|
||||
with open(path, "wb") as handle:
|
||||
for event in events:
|
||||
handle.write(serialize_event(event) + b"\n")
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
|
||||
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
|
||||
with open(path, "rb") as handle:
|
||||
events = [deserialize_event(line) for line in handle if line.strip()]
|
||||
return import_log(events, dest)
|
||||
@@ -8,11 +8,15 @@ a network API is a later workplan.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.adapters import ShardAdapter, assert_conformant
|
||||
from shard_wiki.coordination import (
|
||||
ApplyResult,
|
||||
DecisionLog,
|
||||
EventStore,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
Overlay,
|
||||
OverlayEngine,
|
||||
)
|
||||
@@ -24,12 +28,31 @@ __all__ = ["InformationSpace"]
|
||||
|
||||
|
||||
class InformationSpace:
|
||||
def __init__(self, space_id: str, policy: Policy = DEFAULT_POLICY) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
space_id: str,
|
||||
policy: Policy = DEFAULT_POLICY,
|
||||
*,
|
||||
store: EventStore | None = None,
|
||||
) -> None:
|
||||
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
|
||||
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
|
||||
self.space_id = space_id
|
||||
self.log = DecisionLog()
|
||||
self.log = DecisionLog(store)
|
||||
self.union = UnionGraph(space_id, log=self.log, policy=policy)
|
||||
self.overlays = OverlayEngine(space_id, self.log)
|
||||
|
||||
@classmethod
|
||||
def git_backed(
|
||||
cls,
|
||||
space_id: str,
|
||||
repo_path: str | Path,
|
||||
policy: Policy = DEFAULT_POLICY,
|
||||
) -> InformationSpace:
|
||||
"""An information space whose coordination log is git-addressable (history/patch/review/
|
||||
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
|
||||
return cls(space_id, policy, store=GitEventStore(repo_path))
|
||||
|
||||
def attach(self, adapter: ShardAdapter) -> None:
|
||||
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
|
||||
assert_conformant(adapter)
|
||||
|
||||
74
tests/test_coordination_migration.py
Normal file
74
tests/test_coordination_migration.py
Normal file
@@ -0,0 +1,74 @@
|
||||
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
|
||||
|
||||
from shard_wiki.coordination import (
|
||||
DecisionLog,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
InMemoryEventStore,
|
||||
export_jsonl,
|
||||
import_jsonl,
|
||||
migrate_space,
|
||||
)
|
||||
from shard_wiki.space import InformationSpace
|
||||
|
||||
|
||||
def test_information_space_git_backed_uses_git_log(tmp_path):
|
||||
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
|
||||
assert isinstance(space.log._store, GitEventStore)
|
||||
space.alias("Home", "shardA:Index")
|
||||
# Read-your-writes through the orchestrator's git-backed log.
|
||||
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
|
||||
|
||||
|
||||
def test_default_information_space_stays_in_memory():
|
||||
space = InformationSpace("space-1")
|
||||
assert isinstance(space.log._store, InMemoryEventStore)
|
||||
|
||||
|
||||
def test_migrate_space_preserves_order_and_provenance(tmp_path):
|
||||
source = InMemoryEventStore()
|
||||
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
|
||||
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
n = migrate_space(source, "s", dest)
|
||||
assert n == 2
|
||||
|
||||
migrated = dest.events("s")
|
||||
assert [e.seq for e in migrated] == [0, 1]
|
||||
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
|
||||
assert migrated[0].actor == "ana"
|
||||
assert migrated[1].actor == "ben"
|
||||
assert migrated[0].timestamp == e0.timestamp
|
||||
|
||||
|
||||
def test_migration_yields_identical_fold(tmp_path):
|
||||
source = DecisionLog(InMemoryEventStore())
|
||||
for typ, payload in [
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
|
||||
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
|
||||
]:
|
||||
source.append("s", typ, payload)
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
migrate_space(source._store, "s", dest)
|
||||
after = DecisionLog(dest)
|
||||
assert after.fold("s").aliases == source.fold("s").aliases
|
||||
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
|
||||
|
||||
|
||||
def test_jsonl_round_trip_into_git(tmp_path):
|
||||
source = InMemoryEventStore()
|
||||
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
|
||||
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
|
||||
|
||||
path = tmp_path / "log.jsonl"
|
||||
assert export_jsonl(source.events("s"), path) == 2
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
assert import_jsonl(path, dest) == 2
|
||||
state = DecisionLog(dest).fold("s")
|
||||
assert state.resolve_alias("Home") == "x:1"
|
||||
assert state.equivalent_to("p") == frozenset({"p", "q"})
|
||||
@@ -4,7 +4,7 @@ type: workplan
|
||||
title: "git-backed DecisionLog + per-space append authority"
|
||||
domain: whynot
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -39,7 +39,7 @@ sharding (blueprint O-12). Single append authority per space is the target.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T1
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
|
||||
```
|
||||
@@ -54,7 +54,7 @@ ordering preserved; deterministic serialization.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T2
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
|
||||
```
|
||||
@@ -69,7 +69,7 @@ hand-off resumes from head; a partitioned non-holder cannot fork the log.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T3
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
|
||||
```
|
||||
@@ -82,7 +82,7 @@ process B (new handle) sees it; fold equals the in-memory fold for the same even
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T4
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user