generated from coulomb/repo-seed
feat(coordination): git backend wiring + verbatim log migration (WP-0009 T4)
InformationSpace.git_backed(space_id, repo_path) wires the git coordination log; the default constructor stays in-memory for tests (new keyword-only store=). A one-time importer (migrate_space / import_log / JSONL export+import) replays an existing in-memory or JSON log into git verbatim — preserving seq, timestamp and actor (union-without-erasure) and refusing out-of-order import. Same fold after migration; no behavioural change to overlay/union. SCOPE updated; WP-0009 done. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2
SCOPE.md
2
SCOPE.md
@@ -17,7 +17,7 @@ Learnings update both SCOPE and INTENT where necessary.
|
|||||||
|
|
||||||
| Layer | State |
|
| Layer | State |
|
||||||
|-------|-------|
|
|-------|-------|
|
||||||
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. 107 tests green, ~97% coverage |
|
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. 128 tests green, ~97% coverage |
|
||||||
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
||||||
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
||||||
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
||||||
|
|||||||
@@ -17,6 +17,12 @@ from shard_wiki.coordination.append_authority import (
|
|||||||
LeaseRegistry,
|
LeaseRegistry,
|
||||||
)
|
)
|
||||||
from shard_wiki.coordination.git_event_store import GitEventStore
|
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||||
|
from shard_wiki.coordination.migration import (
|
||||||
|
export_jsonl,
|
||||||
|
import_jsonl,
|
||||||
|
import_log,
|
||||||
|
migrate_space,
|
||||||
|
)
|
||||||
from shard_wiki.coordination.overlay import (
|
from shard_wiki.coordination.overlay import (
|
||||||
ApplyResult,
|
ApplyResult,
|
||||||
ApplyStatus,
|
ApplyStatus,
|
||||||
@@ -37,6 +43,10 @@ __all__ = [
|
|||||||
"LeaseHeld",
|
"LeaseHeld",
|
||||||
"LeaseRegistry",
|
"LeaseRegistry",
|
||||||
"AppendAuthority",
|
"AppendAuthority",
|
||||||
|
"import_log",
|
||||||
|
"migrate_space",
|
||||||
|
"export_jsonl",
|
||||||
|
"import_jsonl",
|
||||||
"serialize_event",
|
"serialize_event",
|
||||||
"deserialize_event",
|
"deserialize_event",
|
||||||
"Overlay",
|
"Overlay",
|
||||||
|
|||||||
@@ -75,6 +75,24 @@ class GitEventStore:
|
|||||||
return event
|
return event
|
||||||
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
|
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
|
||||||
|
|
||||||
|
def import_event(self, event: DecisionEvent) -> None:
|
||||||
|
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
|
||||||
|
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
|
||||||
|
|
||||||
|
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
|
||||||
|
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
|
||||||
|
"""
|
||||||
|
ref = self._ref(event.space)
|
||||||
|
head = self._head(ref)
|
||||||
|
expected = self._count(ref, head)
|
||||||
|
if event.seq != expected:
|
||||||
|
raise ValueError(
|
||||||
|
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
|
||||||
|
)
|
||||||
|
commit = self._commit_event(event, parent=head)
|
||||||
|
if not self._cas_update(ref, new=commit, old=head):
|
||||||
|
raise RuntimeError(f"import race on {ref}")
|
||||||
|
|
||||||
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||||
"""The space's events oldest→newest (append/total order)."""
|
"""The space's events oldest→newest (append/total order)."""
|
||||||
ref = self._ref(space)
|
ref = self._ref(space)
|
||||||
|
|||||||
53
src/shard_wiki/coordination/migration.py
Normal file
53
src/shard_wiki/coordination/migration.py
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
|
||||||
|
|
||||||
|
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
|
||||||
|
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
|
||||||
|
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
|
||||||
|
reproduces identical coordination state; only durability changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.coordination.decision_log import (
|
||||||
|
DecisionEvent,
|
||||||
|
EventStore,
|
||||||
|
deserialize_event,
|
||||||
|
serialize_event,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||||
|
|
||||||
|
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
|
||||||
|
|
||||||
|
|
||||||
|
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
|
||||||
|
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
|
||||||
|
count = 0
|
||||||
|
for event in events:
|
||||||
|
dest.import_event(event)
|
||||||
|
count += 1
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
|
||||||
|
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
|
||||||
|
return import_log(source.events(space), dest)
|
||||||
|
|
||||||
|
|
||||||
|
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
|
||||||
|
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
|
||||||
|
count = 0
|
||||||
|
with open(path, "wb") as handle:
|
||||||
|
for event in events:
|
||||||
|
handle.write(serialize_event(event) + b"\n")
|
||||||
|
count += 1
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
|
||||||
|
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
|
||||||
|
with open(path, "rb") as handle:
|
||||||
|
events = [deserialize_event(line) for line in handle if line.strip()]
|
||||||
|
return import_log(events, dest)
|
||||||
@@ -8,11 +8,15 @@ a network API is a later workplan.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
from shard_wiki.adapters import ShardAdapter, assert_conformant
|
from shard_wiki.adapters import ShardAdapter, assert_conformant
|
||||||
from shard_wiki.coordination import (
|
from shard_wiki.coordination import (
|
||||||
ApplyResult,
|
ApplyResult,
|
||||||
DecisionLog,
|
DecisionLog,
|
||||||
|
EventStore,
|
||||||
EventType,
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
Overlay,
|
Overlay,
|
||||||
OverlayEngine,
|
OverlayEngine,
|
||||||
)
|
)
|
||||||
@@ -24,12 +28,31 @@ __all__ = ["InformationSpace"]
|
|||||||
|
|
||||||
|
|
||||||
class InformationSpace:
|
class InformationSpace:
|
||||||
def __init__(self, space_id: str, policy: Policy = DEFAULT_POLICY) -> None:
|
def __init__(
|
||||||
|
self,
|
||||||
|
space_id: str,
|
||||||
|
policy: Policy = DEFAULT_POLICY,
|
||||||
|
*,
|
||||||
|
store: EventStore | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
|
||||||
|
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
|
||||||
self.space_id = space_id
|
self.space_id = space_id
|
||||||
self.log = DecisionLog()
|
self.log = DecisionLog(store)
|
||||||
self.union = UnionGraph(space_id, log=self.log, policy=policy)
|
self.union = UnionGraph(space_id, log=self.log, policy=policy)
|
||||||
self.overlays = OverlayEngine(space_id, self.log)
|
self.overlays = OverlayEngine(space_id, self.log)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def git_backed(
|
||||||
|
cls,
|
||||||
|
space_id: str,
|
||||||
|
repo_path: str | Path,
|
||||||
|
policy: Policy = DEFAULT_POLICY,
|
||||||
|
) -> InformationSpace:
|
||||||
|
"""An information space whose coordination log is git-addressable (history/patch/review/
|
||||||
|
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
|
||||||
|
return cls(space_id, policy, store=GitEventStore(repo_path))
|
||||||
|
|
||||||
def attach(self, adapter: ShardAdapter) -> None:
|
def attach(self, adapter: ShardAdapter) -> None:
|
||||||
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
|
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
|
||||||
assert_conformant(adapter)
|
assert_conformant(adapter)
|
||||||
|
|||||||
74
tests/test_coordination_migration.py
Normal file
74
tests/test_coordination_migration.py
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
|
||||||
|
|
||||||
|
from shard_wiki.coordination import (
|
||||||
|
DecisionLog,
|
||||||
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
|
InMemoryEventStore,
|
||||||
|
export_jsonl,
|
||||||
|
import_jsonl,
|
||||||
|
migrate_space,
|
||||||
|
)
|
||||||
|
from shard_wiki.space import InformationSpace
|
||||||
|
|
||||||
|
|
||||||
|
def test_information_space_git_backed_uses_git_log(tmp_path):
|
||||||
|
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
|
||||||
|
assert isinstance(space.log._store, GitEventStore)
|
||||||
|
space.alias("Home", "shardA:Index")
|
||||||
|
# Read-your-writes through the orchestrator's git-backed log.
|
||||||
|
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_information_space_stays_in_memory():
|
||||||
|
space = InformationSpace("space-1")
|
||||||
|
assert isinstance(space.log._store, InMemoryEventStore)
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_space_preserves_order_and_provenance(tmp_path):
|
||||||
|
source = InMemoryEventStore()
|
||||||
|
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
|
||||||
|
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
n = migrate_space(source, "s", dest)
|
||||||
|
assert n == 2
|
||||||
|
|
||||||
|
migrated = dest.events("s")
|
||||||
|
assert [e.seq for e in migrated] == [0, 1]
|
||||||
|
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
|
||||||
|
assert migrated[0].actor == "ana"
|
||||||
|
assert migrated[1].actor == "ben"
|
||||||
|
assert migrated[0].timestamp == e0.timestamp
|
||||||
|
|
||||||
|
|
||||||
|
def test_migration_yields_identical_fold(tmp_path):
|
||||||
|
source = DecisionLog(InMemoryEventStore())
|
||||||
|
for typ, payload in [
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
|
||||||
|
]:
|
||||||
|
source.append("s", typ, payload)
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
migrate_space(source._store, "s", dest)
|
||||||
|
after = DecisionLog(dest)
|
||||||
|
assert after.fold("s").aliases == source.fold("s").aliases
|
||||||
|
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
|
||||||
|
|
||||||
|
|
||||||
|
def test_jsonl_round_trip_into_git(tmp_path):
|
||||||
|
source = InMemoryEventStore()
|
||||||
|
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
|
||||||
|
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
|
||||||
|
|
||||||
|
path = tmp_path / "log.jsonl"
|
||||||
|
assert export_jsonl(source.events("s"), path) == 2
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
assert import_jsonl(path, dest) == 2
|
||||||
|
state = DecisionLog(dest).fold("s")
|
||||||
|
assert state.resolve_alias("Home") == "x:1"
|
||||||
|
assert state.equivalent_to("p") == frozenset({"p", "q"})
|
||||||
@@ -4,7 +4,7 @@ type: workplan
|
|||||||
title: "git-backed DecisionLog + per-space append authority"
|
title: "git-backed DecisionLog + per-space append authority"
|
||||||
domain: whynot
|
domain: whynot
|
||||||
repo: shard-wiki
|
repo: shard-wiki
|
||||||
status: active
|
status: done
|
||||||
owner: tegwick
|
owner: tegwick
|
||||||
topic_slug: whynot
|
topic_slug: whynot
|
||||||
created: "2026-06-15"
|
created: "2026-06-15"
|
||||||
@@ -39,7 +39,7 @@ sharding (blueprint O-12). Single append authority per space is the target.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: SHARD-WP-0009-T1
|
id: SHARD-WP-0009-T1
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
|
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
|
||||||
```
|
```
|
||||||
@@ -54,7 +54,7 @@ ordering preserved; deterministic serialization.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: SHARD-WP-0009-T2
|
id: SHARD-WP-0009-T2
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
|
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
|
||||||
```
|
```
|
||||||
@@ -69,7 +69,7 @@ hand-off resumes from head; a partitioned non-holder cannot fork the log.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: SHARD-WP-0009-T3
|
id: SHARD-WP-0009-T3
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
|
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
|
||||||
```
|
```
|
||||||
@@ -82,7 +82,7 @@ process B (new handle) sees it; fold equals the in-memory fold for the same even
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: SHARD-WP-0009-T4
|
id: SHARD-WP-0009-T4
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
|
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in New Issue
Block a user