Compare commits

..

42 Commits

Author SHA1 Message Date
776f5af5a7 Normalize agent instructions and workplan frontmatter (STATE-WP-0067)
- Align agent files with on-disk workplan prefixes (infer from workplan ids)
- Set workplan domain to registered domain_slug; add topic_slug where applicable
- Repair frontmatter delimiter formatting; migrate legacy task status literals
- Regenerate AGENTS.md, CLAUDE.md, and .claude/rules from State Hub templates
2026-06-22 23:16:28 +02:00
fd961c83b4 Add .repo-classification.yaml (CUST-WP-0050 T11 agent first-pass) 2026-06-22 17:47:43 +02:00
cca5bf83c3 Add credential routing instructions for all agent runtimes
Propagate shared credential-routing section (Codex, Claude, Grok, llm-connect)
from state-hub template via scripts/propagate_credential_routing.py.
2026-06-18 22:48:39 +02:00
def699c1eb feat(adapters): GitShardAdapter history adopt + cross-substrate integration (WP-0012 T3)
Adopt git-native history (TSD §A.5): a VERSION-gated history(key) surfaces the
commit list for a path (newest-first sha + subject) — declared by every git-IS-store
shard, read-only or not. Integration proves the union/overlay/edit machinery works
unchanged across folder + git substrates: resolve/chorus span both, edit through a
git shard fast-forwards as a commit, apply-under-drift refuses on an external commit
(sha drift) without clobbering, and a read-only git target keeps the overlay as a
draft. SCOPE updated; WP-0012 done. 196 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:41:19 +02:00
a4e0f52ec1 feat(adapters): GitShardAdapter write=commit + current_rev drift (WP-0012 T2)
Writable mode: write(key, body) stages and commits the file (skipping a no-op so
no empty commit is created), returning the page at the new commit sha. The
writable profile declares WRITE + VERSION with PER_PAGE granularity. current_rev
is the per-path commit sha, so a write — or an external commit to the same path —
moves it, driving apply-under-drift. Passes the conformance positive-write probe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:38:41 +02:00
4231daf94f feat(adapters): GitShardAdapter read path + git-IS-store profile (WP-0012 T1)
A second substrate validating the contract beyond plain folders: a git-IS-store
shard reading Markdown from a git repo. Keys are tracked *.md paths; read returns
a Page whose source_rev is the per-path last-commit sha (so an edit to one page
never drifts another); profile is git-IS-store / substrate=git / history=git-native
/ addressing=path, validated against the §6.5 implication rules. Passes the
conformance read path with honest absence of unclaimed verbs. Zero new deps
(git CLI via subprocess). No core changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:36:28 +02:00
37681d89b6 feat(incremental): wire maintained tier behind views; rebuild fallback (WP-0011 T4)
Route InformationSpace.all_pages through a maintained UnionIndex: equivalence is
served from the incrementally maintained index (curator bindings re-synced live
from the log fold + detected content edges), exposed in decision-log string form
so results are a behaviour-preserving superset. The index is built lazily and
rebuilt (bounded fallback) when the union mutates (attach/edit invalidate it);
reindex() forces a rebuild and verify_index() runs the I-2 self-healing checker.
all_pages() gains an optional equivalence_groups source (default = fold) so
direct callers are unaffected. SCOPE updated; WP-0011 done. 173 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:21:39 +02:00
a8e65235a8 feat(incremental): I-2 digest + consistency-checker (WP-0011 T3)
A Merkle-style digest summarizes the derived tier (per-identity fingerprint +
incident edges as order-independent leaves) so equal states have equal digests
and the digest is stable under equivalent event orders. A ConsistencyChecker
recomputes the authoritative fold from the current source, compares it over a
sampled region, and on mismatch scoped-recomputes just the affected identities —
self-healing missed-delta drift, corrupted internal state, and vanished pages.
Makes derived = f(canonical) verified, not asserted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:16:50 +02:00
d7d046cac0 test(incremental): delta maintenance == rebuild, retraction + split (WP-0011 T2)
Verify change-driven maintenance keeps the equivalence index equal to a
from-scratch rebuild under add / edit / remove: an edit into a new bucket
retracts the stale edge, an edit into equivalence adds one, and removing a
connector node propagates a retraction that splits a chorus. Equality checked
against a fresh build() oracle on every operation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:14:32 +02:00
0b3ab2086f feat(incremental): indexed equivalence — blocking + verify (WP-0011 T1)
Detect equivalence (distinct identities holding the same page) without pairwise
O(N²): MinHash/LSH bands over content shingles + normalized-title buckets
generate candidates (blocking), then exact-fingerprint or Jaccard>=threshold
confirm them (verify), with curator decision-log bindings always forming edges.
Groups are the connected components of the edge set. Includes the incremental
add/update/remove internals used by T2. Matches a brute-force oracle. New
incremental/ package (minhash primitives + EquivalenceIndex).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:13:06 +02:00
d85d019543 feat(views): wire derived views onto InformationSpace + integration (WP-0010 T5)
Expose backlinks(name), recent_changes(), all_pages(), site_map() on
InformationSpace. Integration test exercises all four over two shards (BackLinks
aggregate across shards, AllPages/SiteMap span the union, RecentChanges merges an
alias decision with shard edits). SCOPE updated; WP-0010 done. 152 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:05:12 +02:00
3a5acdcb28 feat(views): AllPages + SiteMap enumeration views (WP-0010 T4)
AllPages enumerates the union's distinct pages, collapsing chorus (same key
across shards) and equivalence-bound identities into one entry via union-find,
noting divergence when members' bodies differ (collapse acknowledged, not
silent). SiteMap builds the namespace tree from page placements, spanning shards.
Both derived/recomputable and presentation-free.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 02:03:15 +02:00
34b0c539f3 feat(views): RecentChanges merged change feed (WP-0010 T3)
One newest-first feed merging the coordination journal (overlay/alias/fork/merge/
binding decisions, with actor + payload) and shard change signals (page
source_rev / mtime). Each entry carries provenance: the originating shard for an
edit, or 'coordination' (and the actor) for a decision. Non-temporal revision
tokens are skipped gracefully. Derived/recomputable; notify-streaming later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:59:11 +02:00
da540d4eea feat(views): BackLinks derived view over the union link graph (WP-0010 T2)
For any page name, the set of pages that link to it: extract wikilinks from every
union page (new UnionGraph.iter_pages enumeration) and index the resolved ones by
target name. Red-links create no backlinks; entries carry source provenance; a
chorus target aggregates the backlinks of all members under one name. Derived/
recomputable, stores nothing canonical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:56:48 +02:00
951b24300d feat(views): wikilink + red-link model (WP-0010 T1)
A CommonMark wikilink extension: extract [[Target]] / [[Target|label]] from a
page body (skipping fenced + inline code, preserving offsets), and resolve each
target through the union — resolved is a link, unresolved is a createable
red-link (never a dropped reference). CamelCase auto-linking is off by default,
opt-in per space, and never double-counts a target already inside [[...]]. Link
model + resolution are core; rendering stays L6. New views/ package.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:55:06 +02:00
c731c96634 feat(coordination): git backend wiring + verbatim log migration (WP-0009 T4)
InformationSpace.git_backed(space_id, repo_path) wires the git coordination log;
the default constructor stays in-memory for tests (new keyword-only store=). A
one-time importer (migrate_space / import_log / JSONL export+import) replays an
existing in-memory or JSON log into git verbatim — preserving seq, timestamp and
actor (union-without-erasure) and refusing out-of-order import. Same fold after
migration; no behavioural change to overlay/union. SCOPE updated; WP-0009 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:49:55 +02:00
f0fee65cc0 test(coordination): cross-process read-your-writes + fold parity (WP-0009 T3)
Verify the git backend's fold reads the durable log into CoordinationState with
unchanged semantics, and that read-your-writes holds across separate handles and
separate OS processes against the same space ref (one test spawns a real
subprocess that appends, then reads it back). Cross-process fold equals the
in-memory fold for the same event sequence (derived = f(log)).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:47:08 +02:00
34432c2e15 feat(coordination): per-space append authority (lease) (WP-0009 T2)
A single append authority per space serializes appends into a total order: at
most one node holds a space's lease; only the holder writes, non-holders forward
their append intent to the holder. Leases are time-bounded and re-grantable, so
a dead holder's lease expires and a new node resumes from the log head (seq stays
contiguous). A stale ex-holder discovers it is no longer the holder and forwards
rather than writing, so a partitioned node cannot fork the log. Works over both
in-memory and git stores. Single-coordinator only (distributed leasing out of scope).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:45:52 +02:00
45a858ead0 feat(coordination): git-backed DecisionLog event store (WP-0009 T1)
Factor DecisionLog storage behind an EventStore abstraction: InMemoryEventStore
stays the default/test double, GitEventStore makes the coordination log
git-addressable. Each space is a ref (refs/spaces/<sha1>); append writes an
immutable one-blob commit and advances the ref under compare-and-swap, so the
commit chain is the per-space total order and a racing appender can never fork
the log. Deterministic stable-JSON event serialization. Zero runtime deps
(git CLI via subprocess). API and fold unchanged across backends.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:41:27 +02:00
b31e9bc337 Add capability registry scaffold and seed entries from reuse-surface
Bootstrap registry/indexes/capabilities.yaml and migrate helix_forge
capability entries owned by this repository for federation publishing.
2026-06-16 01:34:23 +02:00
e50dcc6b5c chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:57:09 +02:00
a165cced33 feat(engine): ext.struct typed-records built-in; close engine implementation (WP-0014 T6)
engine/extensions/struct.py: ext.struct (typed records) — in-text frontmatter
parse + ON_WRITE validation (allowed-fields, content-preserving), ON_READ tags
PageShape.TYPED_RECORD, ON_PROFILE raises structured-payload. Proves the framework:
feature absent when off (opaque prose, honest profile), present + profile-reflected
when on; works through InformationSpace edit. SCOPE updated. 6 tests, 107 total,
~97% coverage, pyflakes clean. Marks T6 + SHARD-WP-0014 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:56:22 +02:00
8393a9c55d chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:36:55 +02:00
ff96ee0c48 feat(engine): EngineShardAdapter — engine as a canonical-mode shard (WP-0014 T5)
engine/adapter.py: EngineShardAdapter implements adapters.ShardAdapter (read/write
run extension transform hooks; profile derived from active extensions, E-5;
current_rev for apply-under-drift) + build_engine_shard() helper (explicit ids or
activation provider). runtime.available() added. Engine shard passes assert_conformant
and attaches to an InformationSpace — resolve + edit (overlay->apply->write-through)
work, and the declared profile reflects the active extensions. 5 tests green, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:36:24 +02:00
8b353f1077 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:27:57 +02:00
b9bb1f7d10 feat(engine): capability profile derived from active extensions (WP-0014 T4, E-5)
engine/profile.py: engine_base_profile() (kernel-only c2-minimum profile),
ProfileContribution (an extension's ON_PROFILE contribution: axis raises + verbs),
derive_profile() folds active extensions' contributions onto the base in deterministic
order then validate() — so configuration->capability is one chain and composition can
never yield an impossible profile (encrypted+native-query rejected). 5 tests green,
96 total, pyflakes clean. Marks T4 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:27:11 +02:00
c40fa3c934 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:16:54 +02:00
54c2bf2ae5 feat(engine): per-shard extension activation (WP-0014 T3, ADR-0001)
engine/activation.py: ActivationContext (shard/tenant, no authz), pluggable
ActivationProvider protocol, StaticProvider standalone default (zero-dep, global
flags + per-shard scoping + per-ext config), ActivationResolver (candidate ids ->
active set / activation profile), and feature_control_provider() lazy factory
(returns None when feature_control_sdk absent -> degrade to static; OpenFeature-
shaped when present). Availability only. 6 tests green, coverage held, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:15:33 +02:00
6d8bd837a4 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-16:
  - update .custodian-brief.md for shard-wiki
2026-06-16 00:12:18 +02:00
b48a99d3c2 feat(engine): typed-extension runtime (WP-0014 T2)
engine/extension.py: Extension contract (id/provides/declares_types/depends_on/
conflicts_with + bound hooks), ExtensionRuntime (register-with-verification,
activate = dependency closure + conflict + type-collision checks rejecting
impossible profiles), and ActiveExtensions with deterministic (topological, id-tie)
hook dispatch — transform hooks chain, collect hooks gather. 9 tests green,
coverage floor held, pyflakes clean. Marks T2 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:11:18 +02:00
9b7f86ba69 test: harden suite with error-path contracts + coverage floor (98%)
Adds tests/test_error_paths.py covering real failure contracts (red-link
single() KeyError, unknown/unattached-shard apply_overlay, kernel.delete
missing, conformance survives a broken profile, Placement str). Adds a
[tool.coverage.report] fail_under=90 floor (engages on pytest --cov, not bare
pytest). 76 tests, 98% coverage, pyflakes clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 00:05:16 +02:00
74142096d0 chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:58:54 +02:00
2100e956aa feat(engine): page-store kernel skeleton (WP-0014 T1)
engine/ package: EngineKernel (in-process page store with per-page version
history; create/edit-as-version, recoverable delete-tombstone, keys, current_rev)
+ wikilink extraction + in-shard link resolution / red-link detection (EC-1..EC-4).
Reuses model/provenance; git-IS-store backing slots in later. 6 tests green,
pyflakes clean, full suite green. Marks T1 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:57:31 +02:00
e62560eb5a chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:52:14 +02:00
b147d3e831 workplan: SHARD-WP-0014 wiki-engine implementation (kernel + typed-extension runtime + activation)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:51:12 +02:00
cdcf4b09aa adr: ADR-0001 engine activation via feature-control (OpenFeature, availability-only)
Records the accepted decision: shard-wiki's native engine uses feature-control via
OpenFeature for per-shard extension activation (availability only, never authz),
provider-pluggable with a LocalProvider standalone default (mirrors the identity
ladder), at the engine layer, consuming the mature feature-control.evaluate slice.
Adds spec/adr/ series + README; hub decision abf7830f recorded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:48:49 +02:00
b21efe307b chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 23:05:45 +02:00
e18397272a spec(SHARD-WP-0013 T6): wire-up + close-out
spec/README + SCOPE list WikiEngineCoreArchitecture.md; CoreArchitectureBlueprint
cross-links the engine as a canonical-mode shard (federation/union stay in the
orchestrator). reuse-surface engine capability promoted D2->D3 (4204255).
Marks T6 + SHARD-WP-0013 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:02:15 +02:00
0ee972f2e2 spec(SHARD-WP-0013 T5): WikiEngineCoreArchitecture.md — small core + typed extensions
Headless, API-first, agent-optimized native engine = canonical-mode shard backend.
Thesis: a page-store kernel with a typed-extension runtime; everything beyond the
c2-minimum is a typed extension activated per shard, and the shard's §A capability
profile is DERIVED from its active extensions (configuration->capability->conformance).
9 engine invariants (engine-is-one-shard, small kernel, per-shard activation,
profile-from-extensions, headless/agent-first, reuse-not-reinvent, typed+verified).
Kernel (4 concepts), typed-extension model (typed hooks + deterministic composition +
feature-control activation), T2 featureset/conflict-mediation realized, engine-as-shard,
agent-first API surface, module sketch, reuse (consumes feature-control/authorization;
G1 framework proposal), traceability, decisions/open, stability note. Marks T5 done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:54:40 +02:00
bb1b54e0af chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 22:45:17 +02:00
b70f1c9acc intent(SHARD-WP-0013 T4): ratify additive native engine — headless, API-first, agent-optimized
Ratified by tegwick (decision 84ffdb48 resolved). INTENT.md amended: native
reference wiki-engine as a canonical-mode shard backend, reframed as HEADLESS &
API-FIRST — small typed-extension core optimized for integrating heterogeneous
data sources and efficient agent/automation access; no bundled UI. Edits:
Primary Utility framing + bullet, non-goal clarifier, new Design Principle,
Stability Note amendment. SCOPE.md: engine core in scope, UI/rendering out.
Orchestrator role unchanged; union-without-erasure + shard sovereignty preserved.
Marks T4 done; unblocks T5 (WikiEngineCoreArchitecture.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 22:43:46 +02:00
8de044bbde chore(consistency): sync task status from DB [auto]
Updated by fix-consistency on 2026-06-15:
  - update .custodian-brief.md for shard-wiki
2026-06-15 22:29:53 +02:00
96 changed files with 6453 additions and 158 deletions

20
.claude/rules/agents.md Normal file
View File

@@ -0,0 +1,20 @@
## Kaizen Agents
Specialized agent personas available on demand via the state-hub MCP.
**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category
**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them
Common agents:
| Agent | Category | When to use |
|-------|----------|-------------|
| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature |
| `code-refactoring` | quality | Code quality analysis and safe refactoring |
| `test-maintenance` | testing | Diagnose and fix failing tests |
| `requirements-engineering` | process | Prevent interface/mock mismatches upfront |
| `keepaTodofile` | process | Maintain TODO.md during work |
| `project-management` | process | Track status, determine next steps |
| `datamodel-optimization` | quality | Optimize dataclasses and data structures |
All 17 agents: call `list_kaizen_agents()` for the full list.

View File

@@ -0,0 +1,8 @@
## Architecture
<!-- TODO: Describe the key design decisions and component structure.
Key modules, data flows, external integrations, state machines, etc. -->
## Quick Reference
`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference

View File

@@ -0,0 +1,50 @@
# Credential and access routing
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
for inference. Run this check **before** requesting secrets, API keys, SSH access,
login tokens, or database passwords — in any repo, not only `ops-warden`.
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
other credential need belongs to another subsystem. **Do not** message
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
### Lookup (do this first)
```bash
warden route find "<describe your need>" --json
warden route show <catalog-id> --json
```
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
| Agent runtime | How to orient |
| --- | --- |
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
### Quick routing table
| I need… | Owner | ops-warden executes? |
| --- | --- | --- |
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes**`warden sign` |
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
| Authorization decision | flex-auth | No — route only |
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
### Anti-patterns (do not do these)
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
- Pasting secrets into Git, State Hub, workplans, logs, or chat
### Other capabilities (reuse-surface)
Non-credential capabilities are usually discovered through **reuse-surface** federation
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
every repo's agent instructions because it is high-frequency, high-risk, and easy to
get wrong.
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`

View File

@@ -0,0 +1,38 @@
## First Session Protocol
Triggered when `get_domain_summary("consumer")` shows **no workstreams**.
The project is registered but work has not yet been structured.
**Step 1 — Read, don't write**
- `~/the-custodian/canon/projects/consumer/project_charter_v0.1.md` — purpose, scope
- `~/the-custodian/canon/projects/consumer/roadmap_v0.1.md` — planned phases
- Scan repo root: README, directory structure, existing code or docs
**Step 2 — Survey in-progress work**
Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete.
**Step 3 — Propose workstreams to Bernd**
Propose 13 workstreams — each a coherent strand, weeks to months, anchored to a
roadmap phase. **Wait for approval before creating.**
**Step 4 — Create workplan file first, then DB record (ADR-001)**
```
workplans/SHARD-WP-NNNN-<slug>.md ← write this first
```
Then register in the hub:
```
create_workstream(topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", title="...", owner="...", description="...")
create_task(workstream_id="<id>", title="...", priority="high|medium|low")
```
**Step 5 — Record the setup**
```
add_progress_event(
summary="First session: structured consumer into N workstreams, M tasks",
event_type="milestone",
topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0",
detail={"workstreams": [...], "tasks_created": M}
)
```
<!-- Delete or archive this file once past first session -->

View File

@@ -0,0 +1,8 @@
## Repo boundary
This repo owns **shard-wiki** only. It does not own:
<!-- TODO: List what belongs in adjacent repos, e.g.:
- SSH key management → railiance-infra/
- State hub code → state-hub/
-->

View File

@@ -0,0 +1,5 @@
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
**Domain:** consumer
**Repo slug:** shard-wiki
**Topic ID:** 4c2e5315-2cb9-447c-9d16-a39bdb0aabd0

View File

@@ -0,0 +1,85 @@
## Session Protocol
Dev Hub (State Hub API): http://127.0.0.1:8000
MCP server name in `~/.claude.json`: `dev-hub`
**Step 1 — Orient**
Read the offline-safe brief first — it works without a live hub connection:
```bash
cat .custodian-brief.md
```
Then call the MCP tool for richer cross-domain context when MCP tools are exposed:
```
get_domain_summary("consumer")
```
If MCP tools are unavailable in the current agent session, use the REST API:
```bash
curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool
```
If the hub is offline: `cd ~/state-hub && make api`
**Step 2 — Check inbox**
With MCP tools:
```
get_messages(to_agent="shard-wiki", unread_only=True)
```
Mark read with `mark_message_read(message_id)`. Reply or act on coordination
requests before proceeding.
Without MCP tools:
```bash
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
| python3 -m json.tool
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
-H "Content-Type: application/json" -d '{}'
```
**Step 3 — Scan workplans**
```bash
ls workplans/
```
For each file with `status: ready`, `active`, or `blocked`, note pending
`wait`/`todo`/`progress` tasks.
**Step 4 — Present brief**
1. **Active workstreams** for `consumer` — title, task counts, blocking decisions
2. **Pending tasks** from `workplans/` + any `[repo:shard-wiki]` hub tasks
3. **Goal guidance** — if `goal_guidance` in summary:
- `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"*
- `alignment_warnings`: flag if active work is not aligned with current goal
4. **Suggested next action** — highest-priority open item
5. **SBOM status** — flag if `last_sbom_at` is unset for this repo
If no workstreams: follow First Session Protocol (`first-session.md`).
**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()`
> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`)
> are First Session Protocol only. Work structure belongs in repo files (ADR-001).
**Session close:**
With MCP tools:
```
add_progress_event(summary="...", topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", workstream_id="<uuid>")
```
Without MCP tools:
```bash
curl -s -X POST http://127.0.0.1:8000/progress/ \
-H "Content-Type: application/json" \
-d '{"topic_id":"4c2e5315-2cb9-447c-9d16-a39bdb0aabd0","workstream_id":"<uuid>","event_type":"note","summary":"what changed","author":"codex"}'
```
If workplan files were modified, ensure the local copy is up to date first:
```bash
git -C <repo_path> pull --ff-only
cd ~/state-hub && make fix-consistency REPO=shard-wiki
```
For repos where implementation runs on a remote machine (e.g. CoulombCore),
use the combined target which pulls before fixing:
```bash
cd ~/state-hub && make fix-consistency-remote REPO=shard-wiki
```
**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback
will sync the file to match DB. **C-16** (repo behind remote) blocks all writes
until you pull — intentional to prevent clobbering remote progress.

View File

@@ -0,0 +1,19 @@
## Stack
<!-- TODO: Fill in language, frameworks, and key dependencies -->
- **Language:**
- **Key deps:**
## Dev Commands
```bash
# TODO: Fill in the standard commands for this repo
# Install dependencies
# Run tests
# Lint / type check
# Build / package (if applicable)
```

View File

@@ -0,0 +1,40 @@
## Workplan Convention (ADR-001)
File location: `workplans/SHARD-WP-NNNN-<slug>.md`
ID prefix: `SHARD-WP-`
Work items originate as files in this repo **before** being registered in the hub.
Canonical workplan/workstream frontmatter statuses are:
`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`.
Use `proposed` for a newly drafted plan, `ready` after review against current
repo state, and `finished` when implementation is complete. `stalled` and
`needs_review` are derived health labels, not stored statuses.
Closed workplans may be moved to `workplans/archived/` with a completion-date
prefix: `YYMMDD-SHARD-WP-NNNN-<slug>.md`. The frontmatter id remains
unchanged; the prefix is only for quick visual reference.
Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**:
`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids
`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed
directly. Promote anything requiring analysis, design, approval, dependencies, or
multiple planned phases into a normal workplan.
Ecosystem todos from other agents arrive as `[repo:shard-wiki]` hub tasks —
visible at session start. Pick one up by creating the workplan file, then registering
the workstream.
Task blocks use this shape:
```task
id: SHARD-WP-NNNN-T01
status: wait | todo | progress | done | cancel
priority: high | medium | low
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
```
Status progression is `todo``progress``done`; use `wait` for waiting or
blocked work and `cancel` for stopped work.
<!-- Ralph Loop rules and HEUREKA sequence: ~/.claude/CLAUDE.md — do not duplicate here -->

View File

@@ -2,21 +2,11 @@
# Custodian Brief — shard-wiki
**Domain:** whynot
**Last synced:** 2026-06-15 20:11 UTC
**Last synced:** 2026-06-15 22:57 UTC
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
## Active Workstreams
### wiki-engine prep — reuse-surface registration, UC-catalog systematization, WikiEngineCoreArchitecture
Progress: 1/6 done | workstream_id: `04a25a53-2169-41d4-88c0-5035a06e72ef`
**Open tasks:**
- · Systematize the UseCaseCatalog around a reuse-surface capability structure `d83d0f96`
- · Surface capability gaps / suggestions to the reuse surface `06b62406`
- · INTENT amendment + ratified decision (additive engine = reference shard backend) `1d0ef72b`
- · Author spec/WikiEngineCoreArchitecture.md `4712bbfe`
- · Wire-up & close-out `1c383414`
### second adapter — git-IS-store shard (contract validation on a new substrate)
Progress: 0/3 done | workstream_id: `9e24eeb0-c0f0-41e6-a1ca-88d71e4139ea`

17
.repo-classification.yaml Normal file
View File

@@ -0,0 +1,17 @@
repo_classification:
standard: Repo Classification Standard
version: '1.0'
classified_at: '2026-06-22'
classified_by: agent
category: project
domain: consumer
secondary_domains: []
capability_tags:
- knowledge
- documentation
business_stake:
- product
- experience
business_mechanics:
- coordination
- operation

243
AGENTS.md
View File

@@ -1,62 +1,219 @@
# AGENTS.md
# shard-wiki — Agent Instructions
Guidance for agents working in `shard-wiki`.
## Repo Identity
## Read First
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
1. `INTENT.md` — aspiration and boundaries (stable; architectural changes are rare).
2. `SCOPE.md` — what we are achieving now and current maturity.
3. `.custodian-brief.md` — State Hub snapshot (generated; do not edit manually).
**Domain:** consumer
**Repo slug:** shard-wiki
**Topic ID:** `4c2e5315-2cb9-447c-9d16-a39bdb0aabd0`
**Workplan prefix:** `SHARD-WP-`
## Documentation Layout
---
This repo follows the CoulombSocial / HelixForge / MarkiTect documentation
layout (recommendation, not strict law). Efficient retrieval by purpose:
## State Hub Integration
| Path | Purpose |
|------|---------|
| `INTENT.md` | Aspiration and boundaries |
| `SCOPE.md` | Top-level view of current achievement; closes gap to INTENT |
| `research/` | Exploration results (`yymmdd-` prefix on files or subdirs) |
| `demand/` | Inbound requests not yet reviewed into spec or workplans |
| `spec/` | Implementation guardrails (PRD, TSD, use cases, architecture) |
| `workplans/` | State Hubregistered implementation tasks |
| `docs/` | Stakeholder documentation (users, developers, humans, agents) |
| `wiki/` | Perspective-free interconnected knowledge (wiki UI when connected) |
| `issues/` | Mirror of relevant open tickets when ticket systems are in use |
| `history/` | Archived material (`yymmdd-` prefix); out of scope for daily work |
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
there is no MCP server for Codex agents.
**Mode of operation:** close SCOPE → INTENT while learning; refine both as needed.
| Context | URL |
|---------|-----|
| Local workstation | `http://127.0.0.1:8000` |
| Remote via tunnel | `http://127.0.0.1:18000` |
## Domain Vocabulary
Honor terms from `INTENT.md`: shard, root entity, adapter contract, projection,
overlay, coordination journal, shard modes. Do not invent parallel vocabulary.
## Build And Test
### Orient at session start
```bash
pip install -e ".[dev]"
pytest
ruff check
ruff format
# Offline brief — works without hub connection
cat .custodian-brief.md
# Active workstreams for this domain
curl -s "http://127.0.0.1:8000/workstreams/?topic_id=4c2e5315-2cb9-447c-9d16-a39bdb0aabd0&status=active" \
| python3 -m json.tool
# Check inbox
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
| python3 -m json.tool
```
## State Hub
Mark a message read:
```bash
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
-H "Content-Type: application/json" -d '{}'
```
Workplans register with State Hub. After workplan changes:
### Log progress (required at session close)
```bash
cd ~/state-hub && make fix-consistency REPO=shard-wiki
curl -s -X POST http://127.0.0.1:8000/progress/ \
-H "Content-Type: application/json" \
-d '{
"summary": "what was done",
"event_type": "note",
"author": "codex",
"workstream_id": "<uuid>",
"task_id": "<uuid>"
}'
```
Finished or canceled workplans move to `history/` with a `yymmdd-` archive prefix.
Omit `workstream_id` / `task_id` when not applicable.
## Where To Put New Material
### Update task status
- Exploratory analysis → `research/yymmdd-<topic>/`
- Raw feature ask or external requirement → `demand/`
- Reviewed design ready to guide code → `spec/`
- Implementation tasks → `workplans/`
- User/dev/agent how-to → `docs/`
- Collaborative unstructured notes → `wiki/`
```bash
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
-H "Content-Type: application/json" \
-d '{"status": "progress"}'
# values: wait | todo | progress | done | cancel
```
### Flag a task for human review
```bash
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
-H "Content-Type: application/json" \
-d '{"needs_human": true, "intervention_note": "reason"}'
```
---
## Session Protocol
**Start:**
1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe)
2. Check inbox: `GET /messages/?to_agent=shard-wiki&unread_only=true`; mark read
3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks
4. Check human-needed tasks: `GET /tasks/?needs_human=true`
**During work:**
- Update task statuses in workplan files as tasks progress
- Record significant decisions via `POST /decisions/`
**Close:**
1. Update workplan file task statuses to reflect progress
2. Log: `POST /progress/` with a summary of what changed
3. Note for the custodian operator: after workplan file changes, run from
`~/state-hub`:
```bash
make fix-consistency REPO=shard-wiki
```
This syncs task status from files into the hub DB.
---
## Credential and access routing
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
for inference. Run this check **before** requesting secrets, API keys, SSH access,
login tokens, or database passwords — in any repo, not only `ops-warden`.
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
other credential need belongs to another subsystem. **Do not** message
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
### Lookup (do this first)
```bash
warden route find "<describe your need>" --json
warden route show <catalog-id> --json
```
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
| Agent runtime | How to orient |
| --- | --- |
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
### Quick routing table
| I need… | Owner | ops-warden executes? |
| --- | --- | --- |
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
| Authorization decision | flex-auth | No — route only |
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
### Anti-patterns (do not do these)
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
- Pasting secrets into Git, State Hub, workplans, logs, or chat
### Other capabilities (reuse-surface)
Non-credential capabilities are usually discovered through **reuse-surface** federation
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
every repo's agent instructions because it is high-frequency, high-risk, and easy to
get wrong.
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
<!-- REPO-AGENTS-EXTENSIONS -->
<!-- Append repo-specific agent instructions below this marker.
The state-hub template sync preserves content after this line. -->
---
## Workplan Convention (ADR-001)
Work items originate as files in this repo — not in the hub. The hub is a
read/cache/index layer that rebuilds from files.
**File location:** `workplans/SHARD-WP-NNNN-<slug>.md`
**Archived location:** finished workplans may move to
`workplans/archived/YYMMDD-SHARD-WP-NNNN-<slug>.md`. The `YYMMDD` prefix is
the completion/archive date; the frontmatter `id` does not change.
**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use
`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use
this only for low-risk work completed directly; create a normal workplan for
anything needing analysis, design, approval, dependencies, or multiple phases.
**Frontmatter:**
```yaml
---
id: SHARD-WP-NNNN
type: workplan
title: "..."
domain: consumer
repo: shard-wiki
status: proposed | ready | active | blocked | backlog | finished | archived
owner: codex
topic_slug: ...
created: "YYYY-MM-DD"
updated: "YYYY-MM-DD"
state_hub_workstream_id: "<uuid>" # written by fix-consistency — do not edit
---
```
Use `proposed` for a new draft, `ready` after review against current repo
state, and `finished` after implementation. `stalled` and `needs_review` are
derived health labels, not frontmatter statuses.
**Task block format** (one per `##` section):
```
## Task Title
` ` `task
id: SHARD-WP-NNNN-T01
status: wait | todo | progress | done | cancel
priority: high | medium | low
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
` ` `
Task description text.
```
Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work.
To create a new workplan:
1. Write the file following the format above
2. Notify the custodian operator to run `make fix-consistency REPO=shard-wiki`
(or send a message to the hub agent via `POST /messages/`)

View File

@@ -1,53 +1,12 @@
# CLAUDE.md
# shard-wiki — Claude Code Instructions
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Repository status
This is an **early-stage Python repository**. The package scaffold (`src/shard_wiki/`, `tests/`, `pyproject.toml`) exists with only smoke tests — the domain model is not yet implemented. Read `INTENT.md` (aspiration), `SCOPE.md` (current achievement), and `AGENTS.md` (layout and conventions) before designing anything. Close the gap from SCOPE to INTENT via `research/`, `spec/`, and `workplans/`.
## What this project is
`shard-wiki` is a **Git-based Markdown wiki orchestrator and federation layer**, not a wiki engine. It lets multiple heterogeneous wiki-shaped page stores (**shards**) attach to a shared root entity and be presented as a **union of pages**, while preserving each shard's separate storage, provenance, capabilities, and history.
The core job is orchestration across backends — Git repos, repo subdirectories (`wiki/`), Gitea wikis, local folders, Obsidian vaults, WebDAV/Nextcloud directories, Coulomb spaces — never replacing or homogenizing them.
## Core domain model (the concepts code must honor)
These abstractions come from `INTENT.md` and define the architecture. New code should map onto them rather than inventing parallel vocabulary:
- **Shard** — an independently meaningful page store attached to a root entity. Shards have *sovereignty*: their own backend, capabilities, limits, history, and identity model. Not all shards are Git-native.
- **Root entity / information space** — the joined space that shards attach to. Each information space should have a **Git-addressable coordination layer** (history, patches, review, backup, reconciliation) even when individual shards are not Git-native.
- **Shard adapter contract** — the versioned interface a backend implements to participate. Adapters are **capability-aware**: the core must model explicitly which operations a shard supports (read, write, diff, merge, lock, version, publish, accept patches) rather than assuming uniformity.
- **Wiki page model** — a stable, versioned, Markdown-first but backend-neutral representation of pages, paths, links, metadata, revisions.
- **Projection** — a lazy, cache-like local view of remote/external shard content. Prefer lazy projection over eager copying.
- **Overlay** — a non-destructive local edit against a remote, read-only, or capability-limited shard, representable as drafts/patches/commits/merge requests *before* destructive application ("overlay before mutation").
- **Coordination journal** — the Git-backed record of change flows for an information space.
- **Shard modes** — read-only, write-through, mirrored, projected, cached, canonical.
## Design constraints to enforce in code
These are hard boundaries from `INTENT.md`; treat violations as design bugs:
- **Mechanism over policy.** Provide primitives for federation, sync, overlays, patching, conflict detection, projection, reconciliation. Do *not* hard-code one editorial/sync/conflict/canonical-source policy — keep those configurable.
- **Union without erasure.** Always preserve provenance: which shard a page came from, its freshness, whether it is cached, whether it has overlays, whether it diverges from an equivalent page elsewhere. Never hide authorship, conflicts, freshness, or backend limitations.
- **No silent remote mutation.** Do not mutate remote systems without explicit adapter support and user intent.
- **Graceful degradation.** Limited backends must still be usable as read-only/cache/projection/backup/patch targets.
- **Not a file-sync daemon.** Synchronization is wiki-page-semantic, not generic file mirroring.
`INTENT.md` has a "Stability Note": changes that redefine what a shard is, Git's role, how root entities are modeled, or whether this is an orchestrator vs. an engine are **architectural changes** and should be rare and deliberate.
## Build, test, run
Python with a `src/` layout, built via hatchling, tested with pytest. Tests run against the source tree directly (`pythonpath = ["src"]` in `pyproject.toml`), so no install/editable step is required to run them.
```bash
pip install -e ".[dev]" # one-time: install dev tooling (pytest, pytest-cov, ruff)
pytest # run the full test suite
pytest tests/test_package.py::test_version_is_exposed # run a single test
pytest --cov # run with coverage
ruff check # lint
ruff format # format
```
Note: the system `pytest` is 7.4.x; `minversion` in `pyproject.toml` is pinned to `7.0` to match. Bump it if a newer pytest is installed into the dev environment.
@SCOPE.md
@.claude/rules/repo-identity.md
@.claude/rules/session-protocol.md
@.claude/rules/first-session.md
@.claude/rules/workplan-convention.md
@.claude/rules/stack-and-commands.md
@.claude/rules/architecture.md
@.claude/rules/repo-boundary.md
@.claude/rules/credential-routing.md
@.claude/rules/agents.md

View File

@@ -16,6 +16,8 @@ The goal is to allow independently stored and differently implemented wikis, pag
The repository provides a **shard orchestration layer** for interconnected Markdown and markup-based wiki content.
Equivalently, shard-wiki can be used as a **headless, API-first wiki engine** — optimized for **integrating heterogeneous data sources** and for **efficient access by agents and automation** — that ships its own native engine as one (canonical-mode) shard among many. There is no bundled UI: presentation and rendering are consumer concerns.
It allows wiki-like systems to:
* Attach heterogeneous page stores as shards of a shared information space
@@ -30,6 +32,7 @@ It allows wiki-like systems to:
* Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
* Allow existing wiki engines to become federation-capable through a shared API
* Allow non-federation-aware systems to participate through adapters and projections
* Serve as a **headless, API-first wiki engine** (a small typed-extension core) that integrates heterogeneous data sources and is consumed efficiently by agents and automation
It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a **composable federated wiki space**.
@@ -85,7 +88,7 @@ A mature `shard-wiki` should allow each participating shard to see the others as
This repository is **not** intended to:
* Replace all wiki engines with a single canonical wiki implementation
* Replace all wiki engines with a single canonical wiki implementation *(shard-wiki MAY still provide its own native, headless, API-first engine as one optional shard backend — see Design Principles — but never as a mandated or universal replacement)*
* Force every shard to use the same backend, database, directory layout, or storage format
* Require every participating system to become federation-aware
* Require every participating shard to be Git-native
@@ -148,6 +151,9 @@ Policy decisions such as conflict preference, canonical source selection, public
* **Composable integration**
Wiki engines should be able to use the `shard-wiki` API to become federation-enabled without reimplementing federation internally.
* **Native reference engine (additive, headless & API-first)**
shard-wiki MAY provide its own native wiki-engine as a **canonical-mode shard backend** — a **small core** with a **typed-extension framework**, activated **per shard** (only what you need). It is **headless and API-first** (no bundled UI; presentation/rendering are consumer concerns) and tuned for **integrating heterogeneous data sources** and **efficient agent/automation access**. It is *one shard type among many*, implemented against shard-wiki's own adapter contract; it does **not** replace other engines, mandate a single implementation, or change shard-wiki's role as an orchestrator. Shard sovereignty and union-without-erasure are preserved.
* **Open by default, progressively governed**
A standalone `shard-wiki` must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an *additive capability*, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).
@@ -201,3 +207,5 @@ Such changes should be rare, because they affect all downstream systems relying
In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether `shard-wiki` is an orchestrator rather than a wiki engine should be treated as architectural changes.
**Amendment — 2026-06-15 (SHARD-WP-0013 T4, decision `84ffdb48`):** admits an **additive** native reference wiki-engine — **headless, API-first**, a small typed-extension core — as a **canonical-mode shard backend** optimized for data-source integration and agent access. Deliberate, narrow scope change; shard-wiki remains an orchestrator and neither mandates nor replaces other engines. (Mirrors the earlier auth-in-core amendment precedent.)

View File

@@ -17,11 +17,11 @@ Learnings update both SCOPE and INTENT where necessary.
| Layer | State |
|-------|-------|
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. attach→resolve→read + edit/overlay/apply work; 64 tests green |
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. Incremental-first derived tier (SHARD-WP-0011): `incremental` (indexed equivalence via MinHash/LSH blocking + verify, change-driven delta maintenance with retraction/propagation, Merkle-style digest + self-healing I-2 consistency-checker, `UnionIndex` routed behind `InformationSpace.all_pages` with rebuild as explicit fallback). Second adapter (SHARD-WP-0012): `GitShardAdapter` — git-IS-store substrate (read=tracked *.md, write=commit, current_rev=per-path sha for drift, adopted git-native history), passes conformance, works across folder+git shards in union/overlay/edit with no core change (capability-as-data proven on a second substrate). 196 tests green, ~97% coverage |
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
| Demand | NetKingdom integration asks captured, not yet negotiated |
| Spec | CoreArchitectureBlueprint (whole-system architecture, hardened via SHARD-WP-0005) + ArchitectureBlueprint (auth/history) drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds |
| Spec | CoreArchitectureBlueprint (whole-system, hardened via SHARD-WP-0005/0006) + FederationArchitecture + FederationRequirements + TSD §A adapter contract + ArchitectureBlueprint (auth/history) + WikiEngineCoreArchitecture (headless API-first engine, SHARD-WP-0013) drafted; UseCaseCatalog 84 UCs (+ engine capability-structure layer); PRD scaffold |
| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` **done** (18 tasks → `FederationArchitecture.md` [T1T10, T17] + `TechnicalSpecificationDocument.md` §A adapter contract [T11T16, T18]); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) |
## In Scope (today)
@@ -32,11 +32,15 @@ Learnings update both SCOPE and INTENT where necessary.
- Authorization model design (delegated authentication, core authorization).
- Shard adapter contract and wiki page model (to be specified, then implemented).
- Git-backed coordination journal for information spaces.
- A **native, headless, API-first wiki-engine core** (small typed-extension core, as a
canonical-mode shard backend) — design via SHARD-WP-0013; optimized for data-source
integration and agent access.
- State Hub workplan registration and consistency sync.
## Out Of Scope (today)
- A standalone wiki engine UI or rendering pipeline.
- A wiki-engine **UI or rendering pipeline** (the engine is headless/API-first; presentation
is a consumer concern). A bundled standalone UI is not provided.
- Authentication, credential storage, or user directory implementation.
- Hard-coded editorial, sync, or conflict-resolution policy.
- Generic file mirroring independent of wiki-page semantics.

View File

@@ -36,6 +36,11 @@ pythonpath = ["src"]
branch = true
source = ["shard_wiki"]
[tool.coverage.report]
show_missing = true
# Quality floor for `pytest --cov` / `coverage report` (not forced on a bare `pytest` run).
fail_under = 90
[tool.ruff]
src = ["src", "tests"]
target-version = "py311"

12
registry/README.md Normal file
View File

@@ -0,0 +1,12 @@
# Capability Registry
Markdown-first capability index for federation and reuse planning.
## Authoring
1. Copy a capability entry template (see reuse-surface `templates/capability-entry.template.md`).
2. Add the row to `indexes/capabilities.yaml`.
3. Run `reuse-surface validate` from a checkout with the CLI installed.
4. Merge to `main` and verify publish with `reuse-surface establish --publish-check`.
Federation contract: reuse-surface `docs/RegistryFederation.md`.

View File

@@ -0,0 +1,103 @@
---
id: capability.wiki.adapter-contract
name: Capability-Aware Shard Adapter Contract
summary: A versioned backend interface where each binding declares a verified capability profile (positions on capability spectra), so federation ops degrade by capability.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, adapter, capability, contract, conformance, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Fifteen capability spectra with an orthogonal core + implication rules, plus
a normative contract spec (TSD Section A); derived from a ~23-system synthesis.
availability:
current: A2
target: A5
confidence: medium
rationale: >
AdapterContract + a read/write FolderAdapter + a conformance suite that
verifies declared profile == observed behaviour exist as a source module.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- versioned interface with declared, conformance-verified capability profiles
- one concrete adapter (file-store) passes the conformance suite
broken_expectations:
- only one substrate implemented (git-IS-store, REST, CRDT adapters planned)
out_of_scope_expectations:
- hosting backends
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- single adapter implemented so far
discovery:
intent: >
Mediate heterogeneity at one narrow waist: a backend participates by implementing a
versioned interface and declaring a verified position on each capability spectrum.
includes:
- capability profile as data (orthogonal-core spectra + implied positions)
- operation verbs (read/write/diff/merge/notify/.../derive-projection/execute)
- a conformance suite (profiles verified, not self-asserted)
excludes:
- assuming uniform backend capabilities
use_cases:
- "shard-wiki UseCaseCatalog UC-34..UC-43, UC-50, UC-57, UC-60..UC-69 (shard attachment & adapter binding)"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/adapters/"
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.page-model
supports:
- capability.wiki.shard-orchestration
evidence:
documentation:
- "shard-wiki/spec/TechnicalSpecificationDocument.md (Section A)"
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 6)"
tests:
- "shard-wiki/tests/test_folder_adapter.py"
- "shard-wiki/tests/test_conformance.py"
consumer_guidance:
recommended_for:
- exposing any page store as a capability-described, conformance-checked shard
not_recommended_for:
- backends that cannot honestly describe their capabilities
known_limitations:
- reference implementation covers the file-store substrate only so far
---
# Capability-Aware Shard Adapter Contract
The bottom narrow waist of shard-wiki: a versioned interface plus a **verified** capability
profile per binding. Core logic is written once against capabilities (not per-backend), and
the conformance suite rejects profiles whose declared abilities don't match observed behaviour.
## Assessment notes
### Discovery
Fifteen spectra reduced to an orthogonal core with implication rules (CoreArchitectureBlueprint
Section 6.5); normative in TSD Section A.
### Availability
`adapters/` ships the contract, a folder adapter, and `assert_conformant`.

View File

@@ -0,0 +1,103 @@
---
id: capability.wiki.coordination-journal
name: Event-Sourced Coordination Journal
summary: An append-only, totally-ordered-per-space decision log (overlays, bindings, aliases, merges, forks) whose current state is a derived fold; git-addressable history.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, event-sourcing, coordination, git, journal, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Keystone resolved across two architecture reviews: coordination-canonical state
as an append-only decision log with a per-space append authority; current state
is a derived fold (derived = f(log)).
availability:
current: A2
target: A4
confidence: medium
rationale: >
In-memory DecisionLog + fold work as a source module; the git-backed store with a
per-space lease (the production backing) is planned.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- append-only, totally-ordered-per-space log with read-your-writes
- derived fold to aliases + transitively-merged equivalence groups
broken_expectations:
- git-backed storage and per-space lease/append-authority not yet implemented
out_of_scope_expectations:
- general-purpose event bus
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- in-memory backing only; cross-process durability pending
discovery:
intent: >
Make coordination-canonical decisions durable and git-addressable as events, with the
queryable current state always recomputable by replay.
includes:
- append-only decision log, totally ordered per information space
- derived fold to current coordination state (aliases, equivalence groups, overlays)
- per-space append authority (concurrency model)
excludes:
- storing derived/disposable union state
use_cases:
- "shard-wiki UseCaseCatalog UC-29, UC-33 (history, attribution, coordination journal)"
availability:
current_level: A2
target_level: A4
current_artifacts:
- "shard-wiki/src/shard_wiki/coordination/decision_log.py"
target_artifacts:
- git-backed log store with per-space lease
consumption_modes:
- source module
relations:
supports:
- capability.wiki.shard-orchestration
- capability.wiki.overlay
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.1)"
tests:
- "shard-wiki/tests/test_decision_log.py"
consumer_guidance:
recommended_for:
- durable, replayable, git-addressable coordination state for a federated space
not_recommended_for:
- high-frequency general event streaming
known_limitations:
- production git backing + lease are still on the roadmap (SHARD-WP-0009)
---
# Event-Sourced Coordination Journal
The keystone: coordination-canonical state (overlays, equivalence bindings, aliases, merges,
forks) is an append-only **decision log**, totally ordered per information space; the queryable
current state is a derived **fold** of the log (`derived = f(log)`). The log is git-addressable,
giving history/patch/review/backup for coordination decisions for free.
## Assessment notes
### Discovery
Resolved across the round-1/round-2 architecture reviews (CoreArchitectureBlueprint Section 8.1).
### Availability
`decision_log.py` ships an in-memory, totally-ordered log + fold; git+lease backing is planned.

View File

@@ -0,0 +1,87 @@
---
id: capability.wiki.derived-views
name: Wiki Derived Views
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages, SiteMap, and (delegate-or-derive) Search — carrying provenance.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, derived-views, backlinks, recentchanges, search, shard-wiki]
maturity:
discovery:
current: D3
target: D5
confidence: medium
rationale: >
Core-vs-adapter classification and behaviours are decided (FederationRequirements ADR-03);
implementation is planned (SHARD-WP-0010), not built.
availability:
current: A0
target: A4
confidence: low
rationale: >
Designed; no implementation yet. Informational/planning reuse only today.
external_evidence:
completeness:
level: C0
name: Absent
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations: []
broken_expectations:
- no derived view is implemented yet
out_of_scope_expectations:
- presentation / rendering of views
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- planning-stage
discovery:
intent: >
Provide recomputable, provenance-carrying views over the union (link graph, change feed,
enumeration, search) without introducing canonical state.
includes:
- BackLinks (link graph), RecentChanges (journal + shard signals), AllPages, SiteMap
- Search as delegate-to-native-or-derive-index
excludes:
- view presentation / UI
use_cases:
- "shard-wiki UseCaseCatalog UC-17..UC-21, UC-63"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/workplans/SHARD-WP-0010-derived-views.md"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.shard-orchestration
- capability.wiki.page-model
related_to:
- capability.wiki.engine-typed-extensions
evidence:
documentation:
- "shard-wiki/spec/FederationRequirements.md (ADR-03)"
consumer_guidance:
recommended_for:
- planning derived navigation/discovery over a federated wiki union
not_recommended_for:
- implementation reuse today (planning-stage)
known_limitations:
- not implemented; Search ranking policy undecided
---
# Wiki Derived Views
Recomputable views over the union (BackLinks, RecentChanges, AllPages, SiteMap, Search). All
are derived/disposable (no canonical state) and carry provenance; Search is delegate-to-native
where a shard's query capability allows, else a derived index. Planned in SHARD-WP-0010.

View File

@@ -0,0 +1,115 @@
---
id: capability.wiki.engine-typed-extensions
name: Wiki Engine with Typed Extensions
summary: A small-core wiki engine realizing a stringent typed-extension framework that addresses all wiki use cases and lets each shard activate only the features it needs.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, engine, typed-extensions, feature-activation, shard-wiki]
maturity:
discovery:
current: D3
target: D5
confidence: medium
rationale: >
Architecture authored (shard-wiki/spec/WikiEngineCoreArchitecture.md): small page-store
kernel + typed-extension framework, per-shard activation, engine-as-canonical-mode-shard,
and a conflict-mediation realization are explored. Detailed extension SDK/ABI and the API
protocol remain (so D3 Explored, not yet D4/D5).
availability:
current: A0
target: A4
confidence: low
rationale: >
Planned. No engine kernel or extensions exist yet; informational/planning reuse only.
external_evidence:
completeness:
level: C0
name: Absent
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations: []
broken_expectations:
- engine core and typed-extension mechanism not yet designed in detail
out_of_scope_expectations:
- replacing other wiki engines or mandating one implementation
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- planning-stage capability
discovery:
intent: >
Provide shard-wiki's reference first-party shard backend: a small core + a stringent
typed-extension framework covering all collected use cases, mediating conflicting
requirements into an integrated whole, with per-shard activation (only what you need).
includes:
- a minimal engine kernel (page lifecycle, storage via the adapter contract, the typing mechanism)
- typed extensions that declare contracts and compose
- per-shard feature activation
excludes:
- replacing or mandating other wiki engines (it is one shard type among many)
- a single canonical implementation for all wikis
use_cases:
- "shard-wiki UseCaseCatalog UC-08..UC-25 and the full catalog (the engine must cover all)"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
- "shard-wiki/spec/WikiEngineCoreArchitecture.md"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.adapter-contract
- capability.wiki.page-model
related_to:
- capability.feature-control.evaluate
- capability.authorization.policy-evaluate
evidence:
documentation:
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
consumer_guidance:
recommended_for:
- planning a composable, feature-activatable native wiki engine
not_recommended_for:
- implementation reuse today (planning-stage)
known_limitations:
- architecture authored; extension SDK/ABI + API protocol still to design; not yet built
promotion_history:
- date: "2026-06-15"
dimension: discovery
from: D2
to: D3
rationale: WikiEngineCoreArchitecture.md authored (kernel + typed-extension framework explored); INTENT amendment ratified.
author: shard-wiki
---
# Wiki Engine with Typed Extensions
shard-wiki's planned reference first-party shard backend — a *canonical-mode shard* it
implements natively: a small core plus a stringent typed-extension framework addressing all
collected use cases, mediating conflicting requirements into a consistent whole, with per-shard
activation (activate only what you need). It is one shard type among many — not a replacement
for other engines. Per-shard activation is a candidate consumer of
`capability.feature-control.evaluate`.
## Assessment notes
### Discovery
Architecture authored: `shard-wiki/spec/WikiEngineCoreArchitecture.md` (small kernel +
typed-extension framework; engine = canonical-mode shard). INTENT amendment ratified
(2026-06-15, decision 84ffdb48). Extension SDK/ABI + API protocol are the next deliverables.
### Availability
Planning-stage; informational reuse only.

View File

@@ -0,0 +1,97 @@
---
id: capability.wiki.federation-models
name: Selectable Federation-Model Taxonomy
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication+ping, query-time graph-join, feed, activity-streams, engine-mirror) selected per space.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, federation, taxonomy, composable, shard-wiki]
maturity:
discovery:
current: D4
target: D6
confidence: high
rationale: >
A six-model taxonomy distilled from a ~23-system synthesis, each model anchored in a
real system, with capability prerequisites and per-space/per-shard composition rules.
availability:
current: A0
target: A4
confidence: low
rationale: >
Designed and specified (FederationArchitecture T17) but not implemented; informational
reuse only today.
external_evidence:
completeness:
level: C1
name: Sparse
confidence: low
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- the model taxonomy and selection/composition rules are documented
broken_expectations:
- no federation transport is implemented yet
out_of_scope_expectations:
- mandating a single federation mechanism
reliability:
level: R0
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- design-stage; no runtime evidence
discovery:
intent: >
Treat federation as selectable and composable rather than one mechanism, so each space
picks fork+journal, VCS-replication, query-join, feed, activity-streams, or engine-mirror.
includes:
- the six federation models + their capability floors
- per-space selection and per-shard composition
excludes:
- imposing one homogeneous federation network
use_cases:
- "shard-wiki UseCaseCatalog UC-26, UC-31, UC-33, UC-71, UC-72, UC-74, UC-79"
availability:
current_level: A0
target_level: A4
current_artifacts:
- "shard-wiki/spec/FederationArchitecture.md (T17)"
consumption_modes:
- informational
relations:
depends_on:
- capability.wiki.shard-orchestration
- capability.wiki.coordination-journal
evidence:
documentation:
- "shard-wiki/spec/FederationArchitecture.md"
- "shard-wiki/research/260614-shard-spectrum-synthesis/findings.md"
consumer_guidance:
recommended_for:
- planning a federation strategy that mixes models per source
not_recommended_for:
- implementation reuse today (design-stage)
known_limitations:
- no transport implemented; informational planning reuse only
---
# Selectable Federation-Model Taxonomy
Federation is plural and composable: fork+journal (Federated Wiki), VCS-replication+ping
(ikiwiki), query-time graph-join (Wikibase SERVICE), feed aggregation, activity streams
(ActivityPub), and engine-mirror (Wiki.js). A space selects a model and composes per shard;
the default is fork+journal over git. Design-stage capability — strong for planning reuse.
## Assessment notes
### Discovery
FederationArchitecture T17, distilled from the shard-spectrum synthesis (v3).
### Availability
Specified, not implemented — informational reuse only.

View File

@@ -0,0 +1,102 @@
---
id: capability.wiki.overlay
name: Overlay-Before-Mutation Write Path
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only, rate-limited, or lossy backends be edited safely without silent remote mutation.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, overlay, patch, write-path, conflict, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Overlay lifecycle and apply-under-drift semantics are specified (ADR-05, blueprint
Section 8.6) and implemented as a single principled write path.
availability:
current: A2
target: A4
confidence: medium
rationale: >
OverlayEngine (draft/patch/apply), writable adapter, and InformationSpace.edit
exist as a source module; three-way merge is not (refuse-on-drift only).
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- draft -> patch -> apply with fast-forward / refuse-on-drift / keep-draft outcomes
- no silent remote mutation; overlay_state surfaced in provenance
broken_expectations:
- three-way / auto merge not implemented (refuse-on-conflict only)
out_of_scope_expectations:
- federation propagation of applied overlays
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- early implementation; conflict handling is detect-and-refuse only
discovery:
intent: >
Make any sub-write-through backend editable safely: an edit is an overlay first, applied
only on explicit intent and only when the source has not drifted.
includes:
- overlay drafts recorded as coordination-canonical events
- patch rendering (unified diff)
- apply-under-drift (fast-forward / refuse / keep-draft)
excludes:
- destructive write without drift check
use_cases:
- "shard-wiki UseCaseCatalog UC-04, UC-26, UC-29 (remix primitives, overlay)"
availability:
current_level: A2
target_level: A4
current_artifacts:
- "shard-wiki/src/shard_wiki/coordination/overlay.py"
- "shard-wiki/src/shard_wiki/coordination/patch.py"
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.coordination-journal
- capability.wiki.adapter-contract
evidence:
documentation:
- "shard-wiki/spec/FederationRequirements.md (ADR-05)"
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.2, 8.6)"
tests:
- "shard-wiki/tests/test_apply.py"
- "shard-wiki/tests/test_write_path_integration.py"
consumer_guidance:
recommended_for:
- safe editing over read-only / rate-limited / lossy backends
not_recommended_for:
- workflows needing automatic conflict resolution today
known_limitations:
- merge is detect-and-refuse; three-way merge is future work
---
# Overlay-Before-Mutation Write Path
One principled write path: every edit drafts an overlay (a coordination-canonical event),
renders as a patch, and applies under drift checks — fast-forwarding a writable target,
keeping a local draft on a read-only target, and refusing (never clobbering) on external drift.
## Assessment notes
### Discovery
Specified in FederationRequirements ADR-05 and CoreArchitectureBlueprint Section 8.2/8.6.
### Availability
`overlay.py` + `patch.py` + `InformationSpace.edit` ship the path; built in SHARD-WP-0008.

View File

@@ -0,0 +1,104 @@
---
id: capability.wiki.page-model
name: Backend-Neutral Wiki Page Model
summary: A Markdown-first but stretchable page model with stable identity separate from placement and layered provenance, spanning prose to typed-graph and computational shapes.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, page-model, identity, provenance, markdown, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Page shapes (prose, typed records, typed-graph, inline-embedded, non-Markdown,
and four computational shapes) plus identity != placement and layered provenance
are specified and grounded in the dive research.
availability:
current: A2
target: A5
confidence: medium
rationale: >
Identity/Placement/Span/Page and layered ProvenanceEnvelope exist as a source
module; richer shapes (typed-graph, notebook) are modeled but not all built.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- stable identity distinct from placement and from content fingerprint
- layered (effective-vs-own) provenance with near-zero per-span cost
broken_expectations:
- non-prose shapes (typed-graph, notebook, inline-embedded) not fully realized
out_of_scope_expectations:
- rendering / presentation
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- prose shape is the only exercised path so far
discovery:
intent: >
One backend-neutral lingua franca every consumer sees; every shape reduces to
(content|source, structure, provenance envelope, optional derivation rule).
includes:
- page identity (stable handle) vs placement (N paths/shards) vs equivalence (fingerprint)
- layered provenance envelope (page + span deltas)
- page-shape taxonomy incl. computational shapes
excludes:
- deriving identity from content (a fingerprint identifies a version, not a page)
use_cases:
- "shard-wiki UseCaseCatalog UC-34, UC-39, UC-44..UC-49, UC-55, UC-73, UC-83, UC-84"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/model/"
- "shard-wiki/src/shard_wiki/provenance/"
consumption_modes:
- source module
relations:
supports:
- capability.wiki.adapter-contract
- capability.wiki.shard-orchestration
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 7)"
- "shard-wiki/spec/FederationRequirements.md (ADR-02, ADR-04)"
tests:
- "shard-wiki/tests/test_model.py"
- "shard-wiki/tests/test_provenance.py"
consumer_guidance:
recommended_for:
- a portable, provenance-carrying representation of wiki pages across backends
not_recommended_for:
- cases needing a single canonical path per page (use identity, not path)
known_limitations:
- non-prose shapes specified ahead of implementation
---
# Backend-Neutral Wiki Page Model
The top narrow waist: a Markdown-first model that stretches to typed records, typed-graph
statements, inline-embedded objects, non-Markdown assets, and computational shapes. Identity
is a stable handle; placement and equivalence are separate mechanisms; provenance is layered
(effective = page envelope + span delta).
## Assessment notes
### Discovery
Specified in CoreArchitectureBlueprint Section 7 and FederationRequirements ADR-02/04.
### Availability
`model/` + `provenance/` ship the prose path and the layered envelope today.

View File

@@ -0,0 +1,114 @@
---
id: capability.wiki.shard-orchestration
name: Wiki Shard Orchestration
summary: Present a union of pages across heterogeneous wiki-shaped shards while preserving each shard's provenance, capabilities, and history.
owner: shard-wiki
status: draft
domain: helix_forge
tags: [wiki, federation, orchestration, union, shard-wiki]
maturity:
discovery:
current: D5
target: D6
confidence: high
rationale: >
Grounded in 84 documented use cases and a twice-reviewed whole-system
architecture (CoreArchitectureBlueprint) derived from ~23 prior-art systems.
availability:
current: A2
target: A5
confidence: medium
rationale: >
InformationSpace orchestrator (attach -> resolve -> read, chorus on
ambiguity) works as a Python source module; network API and incremental
union are planned.
external_evidence:
completeness:
level: C2
name: Partial
confidence: medium
basis: scope_vs_intent_and_consumer_expectations
satisfied_expectations:
- attach folder shards and read a union page with layered provenance
- chorus presentation of equivalent-but-divergent pages (union without erasure)
broken_expectations:
- incremental union maintenance and equivalence index not yet built
- write-through federation transports not yet built
out_of_scope_expectations:
- hosting or replacing the underlying wiki engines
reliability:
level: R1
confidence: low
basis: consumer_quality_signals
known_reliability_risks:
- early implementation; 64 tests but no production exposure
discovery:
intent: >
Let independently stored, differently implemented wikis behave as one
coherent, versionable, inspectable information space without homogenizing them.
includes:
- union resolution across shards (identity-keyed)
- chorus / designated-canonical presentation of equivalent pages
- lazy replication projection of remote content with freshness
excludes:
- implementing a backend wiki engine (see capability.wiki.engine-typed-extensions)
- silent remote mutation
assumptions:
- canonical truth lives in shards + a git coordination journal; the union is derived
use_cases:
- "shard-wiki UseCaseCatalog UC-01..UC-07, UC-26..UC-33 (information space, federation, coordination)"
availability:
current_level: A2
target_level: A5
current_artifacts:
- "shard-wiki/src/shard_wiki/union/"
- "shard-wiki/src/shard_wiki/space.py"
target_artifacts:
- orchestrator network API
consumption_modes:
- source module
relations:
depends_on:
- capability.wiki.adapter-contract
- capability.wiki.page-model
- capability.wiki.coordination-journal
supports:
- capability.wiki.federation-models
evidence:
documentation:
- "shard-wiki/spec/CoreArchitectureBlueprint.md"
- "shard-wiki/spec/FederationArchitecture.md"
tests:
- "shard-wiki/tests/test_union.py"
- "shard-wiki/tests/test_integration.py"
consumer_guidance:
recommended_for:
- composing multiple Markdown/wiki stores into one provenance-preserving view
not_recommended_for:
- replacing a single wiki engine
known_limitations:
- resolution is recompute-on-read until the incremental tier lands
---
# Wiki Shard Orchestration
shard-wiki's core capability: orchestrate wiki-shaped content across heterogeneous *shards*
as a union of pages, preserving provenance, capabilities, and history per shard. Canonical
truth stays at the edges (shards + the git coordination journal); the union is a derived,
recomputable view (orchestrator, not engine).
## Assessment notes
### Discovery
Grounded by `UseCaseCatalog.md` (84 UCs) and the hardened `CoreArchitectureBlueprint.md`.
### Availability
`InformationSpace` provides attach/resolve/read today (source module); a network API is the
target availability step.

View File

@@ -0,0 +1,145 @@
version: 1
updated: '2026-06-16'
domain: helix_forge
capabilities:
- id: capability.wiki.shard-orchestration
name: Wiki Shard Orchestration
summary: Present a union of pages across heterogeneous wiki-shaped shards while
preserving each shard's provenance, capabilities, and history.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.shard-orchestration.md
tags:
- wiki
- federation
- orchestration
- union
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.adapter-contract
name: Capability-Aware Shard Adapter Contract
summary: A versioned backend interface where each binding declares a verified capability
profile, so federation ops degrade by capability.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.adapter-contract.md
tags:
- wiki
- adapter
- capability
- contract
- conformance
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.page-model
name: Backend-Neutral Wiki Page Model
summary: A Markdown-first but stretchable page model with stable identity separate
from placement and layered provenance.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.page-model.md
tags:
- wiki
- page-model
- identity
- provenance
- markdown
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.coordination-journal
name: Event-Sourced Coordination Journal
summary: An append-only, totally-ordered-per-space decision log whose current state
is a derived fold; git-addressable history.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.coordination-journal.md
tags:
- wiki
- event-sourcing
- coordination
- git
- journal
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.overlay
name: Overlay-Before-Mutation Write Path
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only
or limited backends be edited safely without silent remote mutation.
vector: D5 / A2 / C2 / R1
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.overlay.md
tags:
- wiki
- overlay
- patch
- write-path
- conflict
- shard-wiki
consumption_modes:
- source module
- id: capability.wiki.federation-models
name: Selectable Federation-Model Taxonomy
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication,
query-join, feed, activity-streams, engine-mirror) selected per space.
vector: D4 / A0 / C1 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.federation-models.md
tags:
- wiki
- federation
- taxonomy
- composable
- shard-wiki
consumption_modes:
- informational
- id: capability.wiki.engine-typed-extensions
name: Wiki Engine with Typed Extensions
summary: A small-core wiki engine realizing a typed-extension framework that addresses
all wiki use cases and lets each shard activate only the features it needs.
vector: D3 / A0 / C0 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.engine-typed-extensions.md
tags:
- wiki
- engine
- typed-extensions
- feature-activation
- shard-wiki
consumption_modes:
- informational
- id: capability.wiki.derived-views
name: Wiki Derived Views
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages,
SiteMap, and (delegate-or-derive) Search — carrying provenance.
vector: D3 / A0 / C0 / R0
domain: helix_forge
status: draft
owner: shard-wiki
path: registry/capabilities/capability.wiki.derived-views.md
tags:
- wiki
- derived-views
- backlinks
- recentchanges
- search
- shard-wiki
consumption_modes:
- informational

View File

@@ -18,6 +18,10 @@ Scope relationship to the other specs:
`spec/TechnicalSpecificationDocument.md`.
- **`UseCaseCatalog.md`** is the demand this architecture must satisfy; UC references below
are load tests, not decoration.
- **`WikiEngineCoreArchitecture.md`** designs shard-wiki's native, headless, API-first wiki
engine as a **canonical-mode shard backend** (one shard behind §6/§A — federation, union, and
projection stay here in the orchestrator, not in the engine). Added per the 2026-06-15 INTENT
amendment (decision `84ffdb48`, SHARD-WP-0013).
---

View File

@@ -8,6 +8,8 @@ Background on document types: InfoTechPrimers on coulomb.social.
|------|--------|------|
| `CoreArchitectureBlueprint.md` | draft for review | **Whole-system architecture** — layers, abstractions, load-bearing decisions (synthesised from all research) |
| `FederationArchitecture.md` | draft for review | federation design — *what the union does*: T1T10 decision records + the federation-model taxonomy (SHARD-WP-0002) |
| `WikiEngineCoreArchitecture.md` | draft for review | the native **headless, API-first wiki engine** — small page-store kernel + typed-extension framework, as a canonical-mode shard backend (SHARD-WP-0013) |
| `adr/` | living | Architecture Decision Records (ADR-0001: engine activation via feature-control) |
| `FederationRequirements.md` | draft for review | yawex-derived union/federation design notes — resolution, namespace, derived views, provenance, overlay, links (ADR-01…06; SHARD-WP-0001) |
| `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver |
| `TechnicalSpecificationDocument.md` | draft + §A | How the system is built; **§A = the normative shard adapter contract** (T11T16, T18; SHARD-WP-0002) |

View File

@@ -0,0 +1,269 @@
# WikiEngineCoreArchitecture
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0013 T5**
The architecture of shard-wiki's **native reference wiki-engine**: a **headless, API-first**
engine — a **small core** plus a **stringent typed-extension framework** — that addresses the
whole use-case catalogue, mediates conflicting requirements into one integrated featureset, and
lets each shard **activate only what it needs**. Authoritative as of the ratified INTENT
amendment (2026-06-15, decision `84ffdb48`): the engine is **additive** and is shard-wiki's
**reference first-party shard backend (a canonical-mode shard)** — not a replacement for other
engines, not a UI.
Relation to other specs (referenced, not restated):
- `CoreArchitectureBlueprint.md` — the orchestrator/whole-system architecture. **The engine is
one shard behind §A; federation, union, projection, and cross-shard coordination are the
orchestrator's job, not the engine's.** That is what keeps the engine small.
- `TechnicalSpecificationDocument.md §A` — the shard adapter contract the engine implements.
- `FederationRequirements.md` — page resolution, overlay, link semantics (ADRs the engine reuses).
- `UseCaseCatalog.md` "Capability structure" layer (T2) — the core-vs-extension map + the
conflict-mediation map this document realizes.
- reuse surface (`capability.wiki.*`, plus consumed `feature-control` / `authorization`).
---
## 1. Thesis: a small page-store kernel; everything else is a typed extension
> **The engine is a page-store kernel with a typed-extension runtime. Every capability beyond
> the c2-minimum is a *typed extension* a shard activates only if it needs it — and a shard's
> externally-visible capability profile is *computed from its active extension set*.**
That single chain — **configuration (which extensions) → capability (what the shard can do) →
conformance (verified)** — is the whole design. It mirrors the orchestrator's discipline
(`CoreArchitectureBlueprint` §6.5: capability-as-data, verified, no per-backend code) and turns
"integrated whole, yet activate only what you need" from a slogan into a mechanism.
The engine stays small for a structural reason: it is **one shard**, not a federation layer.
Union, projection, equivalence, cross-shard overlay-orchestration, and the federation models all
live in shard-wiki's orchestrator (the blueprint). The engine implements `ShardAdapter` (§A) and
nothing above it. So "wiki engine" here means *a really good single canonical shard with a
typed-extension framework and a headless agent-first API* — not a re-implementation of shard-wiki.
---
## 2. Engine invariants
| # | Invariant | Why |
|---|-----------|-----|
| E-1 | **One shard, not a federation layer.** The engine implements `ShardAdapter` (§A); union/projection/federation are the orchestrator's. | Keeps the engine small; no duplication of the blueprint. |
| E-2 | **Small kernel.** The kernel is only: page store + history, the page model (reused), the extension runtime, the API. | Common case (a plain wiki) is trivial. |
| E-3 | **Everything else is a typed extension.** No feature beyond the c2-minimum is baked into the kernel. | Integrated-whole-yet-selective; testable boundary. |
| E-4 | **Per-shard activation.** A shard runs an *activation profile* (a set of extensions + config); unused features cost nothing. | "Activate only what you need." |
| E-5 | **Capability profile is derived from active extensions.** The §A profile the engine declares is computed from its activation profile, then conformance-verified. | One source of truth; honest, verified capabilities. |
| E-6 | **Headless & API-first.** The API is the only interface; no bundled UI/rendering (consumer concern, L6). | INTENT amendment; clean orchestrator/consumer split. |
| E-7 | **Agent-first ergonomics.** The API is typed, introspectable, batchable, low-round-trip. | INTENT: optimized for efficient agent/automation access. |
| E-8 | **Reuse over reinvent.** Page model, history/journal, activation, and authz are *consumed* (existing capabilities), not rebuilt. | Smallness; reuse-surface alignment. |
| E-9 | **Extensions are typed & verified.** An extension declares its types/hooks/deps; activation is rejected if types conflict or deps are unmet (impossible profiles forbidden). | Stringency; mirrors §6.5 + conformance. |
---
## 3. The kernel (four concepts)
The kernel is deliberately four things — nothing more is mandatory.
1. **Page** — the backend-neutral page model (`capability.wiki.page-model`, reused as-is):
stable identity ≠ placement, layered provenance, page shapes. The kernel does **not** redefine
it; extensions may *register additional shapes/types* (§4).
2. **Store + history** — a git-backed page store (the engine is the *git-IS-store* case from the
blueprint): a write is a commit; history is native and recoverable (E-3/I-10). Coordination
decisions reuse the event-sourced journal (`capability.wiki.coordination-journal`).
3. **Extension runtime** — the typed-extension registry, hook dispatcher, type checker, and
activation engine (§4). *This is the core innovation; it is the only “framework” in the kernel.*
4. **API** — the headless, typed, agent-first surface (§7). Kernel endpoints cover the c2-minimum
(page CRUD-as-history, links, history); extensions extend the surface through typed routes.
The **c2-minimum** a kernel-only shard delivers (no extensions): write a page, link pages
(`[[wikilink]]` + red-link), never lose an edit. That is a complete, useful headless wiki.
---
## 4. The typed-extension model (the framework)
An **Extension** is a typed unit declaring a contract the runtime enforces:
```
Extension:
id : reverse-domain id (e.g. ext.struct.typed-records)
provides : capability ids it realizes (reuse-surface; e.g. capability.wiki.page-model[typed])
types : page shapes / field schemas / content-types it introduces (typed, validated)
hooks : kernel lifecycle bindings it implements (see below)
api : typed routes it adds to the headless surface
depends_on : other extensions / consumed capabilities required
conflicts_with: extensions it cannot co-activate with
config : declared, schema-checked activation parameters
```
**Hooks (the kernel lifecycle the runtime dispatches):**
`on_resolve` (name→page), `on_read`, `on_write` (validate/transform a draft), `on_link`
(link/transclusion resolution), `on_history`, `on_query`, `on_render_request` (produce a derived
representation for a consumer), `on_profile` (contribute capability-spectrum positions, E-5).
Hooks are **typed** (typed inputs/outputs) and dispatched in a **declared, deterministic order**.
**Typing & composition (stringency):**
- At activation, the runtime builds the **dependency closure**, checks **type consistency** (no
two active extensions claim incompatible types for the same page shape/field; `conflicts_with`
honoured), and rejects an **impossible profile** — exactly the §6.5 implication-rule discipline,
applied to extensions. A rejected profile fails fast at boot, never silently.
- Composition is **deterministic**: hook order is declared; conflicts are resolved by explicit
precedence or rejection, never by accident.
- Extensions ship a **conformance check** (mirrors §6.6): an activated extension is exercised
against its declared types/hooks before the shard serves traffic — *typed contracts verified,
not trusted*.
**Per-shard activation (reuse, not reinvent):**
- A shard's **activation profile** = `{extension id → config}`. Activation/evaluation **reuses
`capability.feature-control.evaluate`** (helix_forge/feature-control) — shard-wiki does not
build a bespoke flagging system (T3 consumption).
- **E-5 in action:** the engine's `on_profile` hooks fold the active extensions into the §A
**capability profile** the shard advertises to the orchestrator (e.g. activate
`ext.struct.typed-records` → the `structure` spectrum rises and `structured-payload` is
declared). The profile is then conformance-verified (§A.2). *Configuration → capability →
conformance is one chain.*
---
## 5. Featureset map: core vs extensions, and conflict mediation
The engine realizes the T2 "Capability structure" layer (`UseCaseCatalog.md`). Mapping (the
*page/content-level* clusters; **X-FED and X-ATT are orchestrator concerns, not engine
extensions** — E-1):
| Engine kernel (always on) | T2 | reuse-surface |
|---------------------------|----|---------------|
| Page lifecycle, identity/placement, history, links, store | EC-1…EC-5 | `capability.wiki.page-model`, `…coordination-journal`, `…adapter-contract` |
| Built-in typed extension | T2 cluster | provides / consumes | default |
|--------------------------|-----------|---------------------|---------|
| `ext.overlay` | X-OVERLAY | `capability.wiki.overlay` | on (no-op locally) |
| `ext.authz` (L0→L4 tiers) | X-AUTHZ | consumes `capability.authorization.policy-evaluate` | L0 |
| `ext.views` (BackLinks/RecentChanges/…) | X-VIEW | `capability.wiki.derived-views` | BackLinks/RecentChanges on |
| `ext.struct` (typed/computed/graph) | X-STRUCT | `capability.wiki.page-model[typed]` | off |
| `ext.addr` (span addr / transclusion / query) | X-ADDR | `capability.wiki.page-model`+query | transclusion on |
| `ext.compute` (literate/notebook/program/live) | X-COMP | `capability.wiki.engine-typed-extensions` | off (gated, sandbox) |
| `ext.prov` (rich provenance/metadata) | X-PROV | `capability.wiki.page-model[provenance]` | base on |
| `ext.collab` (c2 social patterns) | X-COLLAB | (UI/convention; mostly consumer) | off |
**Conflict mediation (T2 map) realized by the framework** — every tension is a *mechanism*, not a
baked-in choice, so one featureset serves all:
| Tension | Realized by |
|---------|-------------|
| open vs governed | `ext.authz` tiers (additive); kernel history is the floor at L0 |
| lossless vs lossy | a `translate` hook + fidelity report (consumes the proposed `capability.content.translation-fidelity`, G2) |
| live vs snapshot | `ext.compute`/`ext.addr` mark liveness; degrade to snapshot (never imply live) |
| canonical vs chorus | detection in kernel; resolution is a policy preset (orchestrator) |
| integrated-whole vs only-what-you-need | **the activation profile** (E-4) + typed composition (§4) — the headline mediation |
| minimal vs feature-rich | small kernel (§3) + extensions; nothing beyond c2 is mandatory |
---
## 6. The engine as a canonical-mode shard
The engine exposes itself through an `EngineShardAdapter` implementing §A:
- **Substrate** git-IS-store; **history** git-native; **write** = commit; `current_rev` = sha
(apply-under-drift works out of the box). It is the **most capable shard** shard-wiki can
attach — it dogfoods the contract.
- Its **capability profile is computed from active extensions** (E-5) and **conformance-verified**
(§A.2) — so the orchestrator sees an honest profile, and federation ops degrade by the engine's
*actually-activated* capabilities.
- The orchestrator attaches it like any shard; **federation/union/projection are not in the
engine** (E-1). A standalone deployment is "the engine as the sole canonical shard"; a
federated deployment is "the engine as one shard among many." Same engine, no re-architecture.
This is the precise realization of the INTENT reconciliation: shard-wiki orchestrates; the engine
is the first-party shard it can attach.
---
## 7. Headless API surface & agent ergonomics (E-6/E-7)
API-first means the typed API is the product; there is no UI. Agent-first means it is designed
for cheap, deterministic machine consumption:
- **Typed resource API** over pages, links, history, spans — content-negotiated (raw Markdown,
the structured page model, or an extension-rendered representation via `on_render_request`).
- **Capability/extension introspection** — an endpoint returns the shard's **active extensions,
their types, and the derived §A capability profile**, so an agent can discover *what this shard
can do* before acting (no trial-and-error). This is the agent-facing twin of E-5.
- **Batch & query** — multi-page reads, link-graph and RecentChanges queries (via `ext.views`),
and `on_query` delegation — minimizing round-trips.
- **Write via overlay** — edits go through the overlay path (FederationRequirements ADR-05), so
agent writes are safe (draft → apply-under-drift) and attributable.
- **Deterministic & provenance-carrying** — every response carries the provenance envelope;
identical inputs yield identical outputs (no hidden state) — friendly to caching agents.
---
## 8. Implementation sketch (module layout)
The engine lives under the shard-wiki package as a backend (it sits at L0/L1 — a shard behind the
adapter; nothing in the orchestrator depends *up* on it):
```
src/shard_wiki/engine/
kernel.py # page store + history (git-IS-store), lifecycle; reuses model/, provenance/, coordination/
extension.py # Extension contract, registry, typed hook dispatcher, type checker
activation.py # activation profile; reuses capability.feature-control.evaluate
profile.py # derive the §A CapabilityProfile from active extensions (E-5) + conformance
api.py # headless, typed, agent-first surface (+ extension introspection)
adapter.py # EngineShardAdapter implements adapters/ ShardAdapter (canonical-mode shard)
extensions/ # built-ins: overlay/ authz/ views/ struct/ addr/ compute/ prov/ collab/
```
Dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`, `adapters/`
(contract), `policy/`; it is consumed *only* via its `EngineShardAdapter` (the orchestrator
attaches it as a shard). No orchestrator-tier (`union/`, `projection/`) import.
---
## 9. Reuse (what the engine consumes vs registers)
- **Consumes:** `capability.feature-control.evaluate` (activation), `capability.authorization.
policy-evaluate` (`ext.authz`), the proposed `capability.content.translation-fidelity` (G2,
lossy translation), and shard-wiki's own `capability.wiki.{page-model, coordination-journal,
adapter-contract, overlay, derived-views}`.
- **Registers / realizes:** `capability.wiki.engine-typed-extensions` (this document is its
Discovery evidence — D2→D3 on ratification). The cross-cutting **typed-extension framework**
pattern is proposed back to the reuse surface as **G1** (`capability.platform.typed-extension-
framework`); this engine is its first instance.
---
## 10. Traceability
- **INTENT** — realizes the 2026-06-15 amendment (decision `84ffdb48`): headless, API-first,
additive native engine = canonical-mode shard backend; honours all engine invariants and the
orchestrator boundary (E-1).
- **Use cases** — the kernel/extension split *is* the T2 "Capability structure" layer
(`UseCaseCatalog.md`); every UC is either kernel (EC-1…EC-5) or a named extension; conflicts
use the T2 mediation map (§5). The engine must ultimately cover UC-01UC-84 (per-shard subsets).
- **Architecture** — consistent with `CoreArchitectureBlueprint` (engine = canonical-mode shard,
§6 contract, §7 page model, §8.1 journal) and `TechnicalSpecificationDocument §A` (the contract
it implements). `FederationRequirements` ADR-05/06 supply overlay + link semantics.
- **Reuse surface** — §9; G1/G2 proposals from SHARD-WP-0013 T3.
## 11. Decisions / deferred / open
**Decided:** small page-store kernel + typed-extension runtime (E-2/E-3); engine is one shard,
not a federation layer (E-1); capability profile derived from active extensions (E-5); headless,
API-first, agent-first (E-6/E-7); activation reuses `feature-control` (E-8); extensions are
typed + conformance-verified (E-9).
**Deferred:** the concrete extension SDK/ABI and hook signatures; the API protocol (REST/GraphQL/
MCP) — agent-first introspection is required, the wire format is an implementation spike; the
built-in extensions' internal designs (each is a later workplan).
**Open (tracked):** does `ext.compute` ever execute in-process or strictly delegate/snapshot
(ties blueprint §8.5 + trust/sandbox); is the typed-extension framework promoted to the
reuse-surface platform capability (G1) and then *consumed* here rather than engine-owned;
introspection granularity vs. leaking internal structure to agents.
## 12. Stability note
The **thesis (§1)** and **invariants (§2)** — especially *engine-is-one-shard* (E-1),
*small-kernel/everything-else-typed-extension* (E-2/E-3), and *capability-profile-derived-from-
extensions* (E-5) — are load-bearing. Changing them (e.g. moving federation into the engine, or
baking a feature into the kernel) is an architectural change in the sense of INTENT's Stability
Note and should be rare and deliberate. The headless/API-first posture is fixed by the ratified
INTENT amendment.

View File

@@ -0,0 +1,79 @@
# ADR-0001 — Engine extension activation via feature-control (OpenFeature)
Status: **Accepted** · Date: 2026-06-15 · Deciders: tegwick · Source: SHARD-WP-0013 follow-up
(feature-control assessment)
> First repo-level ADR. (Note: `FederationRequirements.md` contains document-internal
> "ADR-01…06" design notes — those are scoped to that spec; this `spec/adr/` series is the
> repository's standalone architecture decision log, starting here.)
## Context
`WikiEngineCoreArchitecture.md` (SHARD-WP-0013 T5) defines the engine as a small kernel plus a
**typed-extension framework** where each shard **activates only the extensions it needs**
(invariants E-4 activation, E-8 reuse-not-reinvent). It needs a mechanism to decide, per shard
(and per tenant/context), which extensions/features are active — without baking a bespoke flag
system into the engine, and without breaking the **standalone, zero-external-dependency** L0
posture shard-wiki guarantees.
The helix_forge sibling **`feature-control`** (`capability.feature-control.evaluate`, registered
at **D5 / A4 / C3 / R3**) provides exactly this: an **OpenFeature**-based feature-availability
control plane with a working SDK (`feature_control_sdk`: `FeatureControlClient`, `Resolver`, a
static `LocalProvider`), context-scoped evaluation (`tenant_id`/scope), explainable decisions,
and graceful degradation when OpenFeature is absent. shard-wiki already proposed this as a T3
*consumption* (reuse, don't rebuild).
## Decision
**Adopt `feature-control` (via the OpenFeature standard) as the engine's per-shard extension/
feature activation mechanism** — *availability only* — with these constraints:
1. **OpenFeature-shaped, provider-pluggable.** The engine evaluates activation through an
OpenFeature-style client. A static **`LocalProvider`** is the **standalone/L0 default**
(zero external dependency); a `feature-control`/remote provider is plugged in for governed
deployments. This mirrors shard-wiki's existing **identity-provider ladder** (null/local
default → external when present).
2. **Availability ≠ authorization.** feature-control decides *which extensions are active*,
never *who may read/write*. Authorization stays in core (X-AUTHZ / `authorization.policy-
evaluate`). The two are composed but never conflated. (feature-control's own INTENT requires
this.)
3. **Engine layer, not the orchestrator foundation.** Integration lives in
`engine/activation.py`; the current `src/shard_wiki/` core stays dependency-free. OpenFeature/
feature-control is an optional extra, kept out of the standalone path by the `LocalProvider`.
4. **Thin slice only.** Consume `feature-control.evaluate` (mature, A4). Do **not** take a
dependency on the heavier control-plane governance / `rollout` / `visibility` (A2) until a
concrete need appears.
Activation keys = extension ids; evaluation context = `{tenant_id: root-entity, shard_id, …}`;
the resulting active-extension set then **derives** the shard's §A capability profile (E-5).
## Consequences
**Positive**
- No bespoke flag system; reuses a mature (D5/A4) capability — reuse-surface aligned.
- Standalone stays zero-dep (LocalProvider); governed deployments get real runtime control,
multi-tenant scoping, and **explainable** decisions that feed the engine's agent-introspection
API (E-7: "why is extension X off for this shard?").
- "Activate only what you need" + compute control become first-class and reversible at runtime.
- Clean layering: availability (feature-control) vs authorization (core) vs identity (provider).
**Negative / risks (mitigated by the constraints)**
- An optional OpenFeature dependency at the engine layer (mitigated: out of the standalone path).
- Coupling to an external control plane in governed mode (mitigated: provider-pluggable, degrade
to LocalProvider).
- Temptation to route authz through it (mitigated: constraint 2, hard boundary).
## Alternatives considered
- **Bespoke per-shard flag/config in the engine** — rejected: reinvents feature-control, no
standard, no multi-tenant/explainability, violates reuse-not-reinvent (E-8).
- **No activation (all extensions always on)** — rejected: defeats "small core + activate only
what you need" (E-2/E-4) and the compute-control goal.
- **Build on the heavier feature-control control-plane now** — deferred: over-scoping a single
engine's activation; revisit if rollout/governance needs emerge.
## Related
`WikiEngineCoreArchitecture.md` (E-4/E-8, §4 activation), `UseCaseCatalog.md` capability-structure
layer (X-AUTHZ vs activation), `history/260615-reuse-surface-contributions.md` (T3 consumption),
reuse-surface `capability.feature-control.evaluate`, INTENT amendment decision `84ffdb48`.

11
spec/adr/README.md Normal file
View File

@@ -0,0 +1,11 @@
# spec/adr/ — Architecture Decision Records
Repository-level ADRs: one decision per file, `ADR-NNNN-<slug>.md`, status
**Proposed / Accepted / Superseded**. Each records Context · Decision · Consequences ·
Alternatives. These are the standalone, numbered decision log; design-note "ADRs" embedded
inside a spec (e.g. `FederationRequirements.md` ADR-01…06) are scoped to that document and are
not part of this series.
| ADR | Status | Subject |
|-----|--------|---------|
| [ADR-0001](ADR-0001-engine-activation-via-feature-control.md) | Accepted | Engine extension activation via feature-control (OpenFeature), availability-only, LocalProvider standalone default |

View File

@@ -9,10 +9,13 @@ from shard_wiki.adapters.conformance import (
)
from shard_wiki.adapters.contract import CONTRACT_VERSION, ShardAdapter
from shard_wiki.adapters.folder import FolderAdapter
from shard_wiki.adapters.git import GitShardAdapter, PageRevision
__all__ = [
"ShardAdapter",
"FolderAdapter",
"GitShardAdapter",
"PageRevision",
"CONTRACT_VERSION",
"Check",
"ConformanceReport",

View File

@@ -0,0 +1,180 @@
"""GitShardAdapter — a second substrate: git-as-store (SHARD-WP-0012; TSD §A.3 git-IS-store).
The home case where **git is the store *and* the journal**. Tracked ``*.md`` paths are the page
keys; the working-tree file is the body; a page's ``source_rev`` is the **commit sha of the last
commit touching its path** (per-path, so an edit to one page never drifts another). The declared
profile is *git-IS-store ⟹ substrate=git ∧ history=git-native* — the implication rule the
capability model enforces (§6.5), validated at registration like any other binding.
This adapter adds **no core changes**: it implements the same :class:`ShardAdapter` contract the
folder adapter does, proving "write an adapter + declare a verified profile" is the whole cost of a
new substrate (capability-as-data, I-3). Built on the ``git`` CLI via subprocess — zero new deps.
"""
from __future__ import annotations
import os
import subprocess
from collections.abc import Iterable
from dataclasses import dataclass
from pathlib import Path
from shard_wiki.adapters.contract import ShardAdapter
from shard_wiki.model import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
Identity,
MergeModel,
NativeQuery,
NotSupported,
OperationalEnvelope,
Page,
Placement,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
__all__ = ["GitShardAdapter", "PageRevision"]
@dataclass(frozen=True, slots=True)
class PageRevision:
"""One adopted git-native revision of a page: the commit sha and its subject line."""
sha: str
message: str
_GIT_IDENTITY = {
"GIT_AUTHOR_NAME": "shard-wiki",
"GIT_AUTHOR_EMAIL": "shard@shard-wiki",
"GIT_COMMITTER_NAME": "shard-wiki",
"GIT_COMMITTER_EMAIL": "shard@shard-wiki",
}
class GitShardAdapter(ShardAdapter):
"""A shard whose store is a git repo: keys are tracked ``*.md`` paths, revs are commit shas."""
def __init__(self, shard_id: str, repo_path: str | Path, writable: bool = False) -> None:
self._shard_id = shard_id
self._repo = Path(repo_path)
self._writable = writable
self._repo.mkdir(parents=True, exist_ok=True)
if not (self._repo / ".git").exists():
self._git("init", "--quiet")
@property
def shard_id(self) -> str:
return self._shard_id
def profile(self) -> CapabilityProfile:
# VERSION is always available — a git-IS-store has git-native history to adopt (§A.5),
# read-only or not. WRITE (= commit, PER_PAGE) is added only in writable mode.
verbs = {Verb.READ, Verb.VERSION}
granularity = WriteGranularity.NONE
if self._writable:
verbs |= {Verb.WRITE}
granularity = WriteGranularity.PER_PAGE
return CapabilityProfile(
substrate=Substrate.GIT,
attachment_mode=AttachmentMode.GIT_IS_STORE,
write_granularity=granularity,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.GIT_NATIVE, # git-is-store ⟹ git-native (§6.5)
merge_model=MergeModel.GIT_TEXT,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset(verbs),
).validate()
def write(self, key: str, body: str) -> Page:
"""Write = **commit**: stage the file and commit it (skip a no-op so no empty commit),
returning the page at the new sha. Drift detection rides on ``current_rev`` = that sha."""
if not self._writable:
raise NotSupported(f"{type(self).__name__} is read-only")
rel = f"{key}.md"
path = self._path_for(key)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(body, encoding="utf-8")
self._git("add", "--", rel)
if self._run("diff", "--cached", "--quiet").returncode != 0: # staged changes present
self._git("commit", "-m", f"write {rel}", env=_GIT_IDENTITY)
return self.read(key)
def keys(self) -> Iterable[str]:
out = self._git("ls-files", "*.md").decode()
for line in out.splitlines():
yield line[: -len(".md")] if line.endswith(".md") else line
def read(self, key: str) -> Page:
path = self._path_for(key)
if not path.is_file():
raise KeyError(key)
rev = self.current_rev(key)
return Page(
identity=Identity(self._shard_id, key),
body=path.read_text(encoding="utf-8"),
envelope=ProvenanceEnvelope(
source_shard=self._shard_id,
liveness=Liveness.STATIC,
staleness=Staleness.FRESH,
source_rev=rev,
lineage="git-native",
),
placements=(Placement(self._shard_id, f"{key}.md"),),
)
def current_rev(self, key: str) -> str | None:
"""The sha of the last commit touching ``key``'s path (per-path drift token), or None."""
rel = f"{key}.md"
if not self._path_for(key).is_file():
return None
sha = self._git("log", "-1", "--format=%H", "--", rel).decode().strip()
return sha or None
def history(self, key: str) -> tuple[PageRevision, ...]:
"""Adopt git-native history (§A.5): the commit list for ``key``'s path, newest-first.
VERSION-gated; raises ``KeyError`` for an unknown page. Each revision is a commit sha +
subject — the native log surfaced through the contract, not re-implemented.
"""
if not self.profile().supports(Verb.VERSION):
raise NotSupported(f"{type(self).__name__} does not support version")
if not self._path_for(key).is_file():
raise KeyError(key)
out = self._git("log", "--format=%H%x00%s", "--", f"{key}.md").decode()
revisions = []
for line in out.splitlines():
sha, _, message = line.partition("\x00")
revisions.append(PageRevision(sha=sha, message=message))
return tuple(revisions)
# -- git plumbing --------------------------------------------------------
def _path_for(self, key: str) -> Path:
return self._repo / f"{key}.md"
def _git(self, *args: str, stdin: bytes | None = None, env: dict | None = None) -> bytes:
return self._run(*args, stdin=stdin, env=env, check=True).stdout
def _run(
self, *args: str, stdin: bytes | None = None, env: dict | None = None, check: bool = False
) -> subprocess.CompletedProcess:
return subprocess.run(
["git", "-C", str(self._repo), *args],
input=stdin,
capture_output=True,
env={**os.environ, **(env or {})},
check=check,
)

View File

@@ -4,7 +4,24 @@ from shard_wiki.coordination.decision_log import (
CoordinationState,
DecisionEvent,
DecisionLog,
EventStore,
EventType,
InMemoryEventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.append_authority import (
AppendAuthority,
Lease,
LeaseHeld,
LeaseRegistry,
)
from shard_wiki.coordination.git_event_store import GitEventStore
from shard_wiki.coordination.migration import (
export_jsonl,
import_jsonl,
import_log,
migrate_space,
)
from shard_wiki.coordination.overlay import (
ApplyResult,
@@ -19,6 +36,19 @@ __all__ = [
"DecisionEvent",
"EventType",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"GitEventStore",
"Lease",
"LeaseHeld",
"LeaseRegistry",
"AppendAuthority",
"import_log",
"migrate_space",
"export_jsonl",
"import_jsonl",
"serialize_event",
"deserialize_event",
"Overlay",
"OverlayEngine",
"ApplyStatus",

View File

@@ -0,0 +1,158 @@
"""Per-space append authority — the single-writer lease over the log (SHARD-WP-0009 T2).
The log is a *total order per space* (§8.6). :class:`~shard_wiki.coordination.git_event_store`
makes a fork physically impossible via compare-and-swap; this layer adds the **policy** that gives
the order a single designated writer: a **per-space lease**. At most one node holds a space's lease
at a time; only the holder writes to the store. A non-holder does not write — it **forwards** its
append intent to the current holder, so intents from anywhere still land in one serialized stream.
The lease is **time-bounded and re-grantable** (HA): if a holder dies, its lease expires and a new
node may take it, resuming appends from the log head (``seq`` stays contiguous across the hand-off).
A node holding a *stale* lease (already re-granted elsewhere) cannot write either — it discovers it
is no longer the holder and forwards instead, so a partitioned ex-holder can never fork the log.
Mechanism over policy (CLAUDE.md): this provides the leasing *primitive*; who acquires when, and
the TTL, are the caller's policy. Single-coordinator only — distributed multi-node leasing and log
sharding are explicit non-goals of this workplan.
"""
from __future__ import annotations
import uuid
from collections.abc import Callable, Mapping
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from typing import Any
from shard_wiki.coordination.decision_log import DecisionEvent, EventStore, EventType
__all__ = ["Lease", "LeaseHeld", "LeaseRegistry", "AppendAuthority"]
def _utcnow() -> datetime:
return datetime.now(tz=timezone.utc)
@dataclass(frozen=True, slots=True)
class Lease:
"""A time-bounded grant of single-writer authority over one space."""
space: str
holder: str
token: str
expires_at: datetime
def valid_at(self, now: datetime) -> bool:
return now < self.expires_at
class LeaseHeld(Exception):
"""Raised when a space's lease is validly held by a different node."""
def __init__(self, lease: Lease) -> None:
super().__init__(
f"space {lease.space!r} leased to {lease.holder!r} until {lease.expires_at}"
)
self.lease = lease
class LeaseRegistry:
"""The single coordinator's grant table: at most one *valid* lease per space.
A lease that has expired is freely re-grantable to any node (the HA replacement path); a still
valid lease is exclusive to its holder (renewable by that holder). The registry also routes
forwarded append intents to the current holder node.
"""
def __init__(self, clock: Callable[[], datetime] = _utcnow) -> None:
self._clock = clock
self._leases: dict[str, Lease] = {}
self._nodes: dict[str, AppendAuthority] = {}
def register(self, node: AppendAuthority) -> None:
self._nodes[node.node_id] = node
def grant(self, space: str, holder: str, ttl_seconds: float) -> Lease:
"""Grant/renew the lease for ``space`` to ``holder``; raise :class:`LeaseHeld` if another
node still holds it validly. An expired lease is re-grantable to anyone."""
now = self._clock()
current = self._leases.get(space)
if current is not None and current.valid_at(now) and current.holder != holder:
raise LeaseHeld(current)
lease = Lease(
space=space,
holder=holder,
token=uuid.uuid4().hex,
expires_at=now + timedelta(seconds=ttl_seconds),
)
self._leases[space] = lease
return lease
def current(self, space: str) -> Lease | None:
"""The lease for ``space`` if one is currently valid, else None (expired/absent)."""
lease = self._leases.get(space)
return lease if lease is not None and lease.valid_at(self._clock()) else None
def holder_node(self, space: str) -> AppendAuthority | None:
lease = self.current(space)
return self._nodes.get(lease.holder) if lease is not None else None
class AppendAuthority:
"""A coordinator node that appends to the shared log only when it holds the space's lease.
Nodes share one :class:`EventStore` and one :class:`LeaseRegistry`. ``append`` routes itself:
the holder writes; a non-holder forwards to whoever holds the lease (acquiring it first if the
space is currently unleased). The append API mirrors :class:`EventStore` so the authority is a
drop-in single-writer guard.
"""
def __init__(
self,
node_id: str,
store: EventStore,
registry: LeaseRegistry,
ttl_seconds: float = 30.0,
) -> None:
self.node_id = node_id
self._store = store
self._registry = registry
self._ttl = ttl_seconds
registry.register(self)
def acquire(self, space: str) -> Lease:
"""Take (or renew) the lease for ``space``. Raises :class:`LeaseHeld` if another node holds
it validly."""
return self._registry.grant(space, self.node_id, self._ttl)
def holds(self, space: str) -> bool:
lease = self._registry.current(space)
return lease is not None and lease.holder == self.node_id
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append via the single authority. If we hold the lease, write; otherwise forward to the
holder. If the space is unleased, acquire it first. A node with a *stale* lease forwards
(it is not the current holder) rather than writing — so it cannot fork the log."""
holder_node = self._registry.holder_node(space)
if holder_node is None:
self.acquire(space) # unleased: take authority, then write below
holder_node = self
if holder_node is self:
return self._store.append(space, type, payload, actor=actor)
return holder_node._write(space, type, payload, actor=actor)
def _write(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None,
) -> DecisionEvent:
"""Apply a forwarded intent. Called only on the lease holder by a forwarding peer."""
return self._store.append(space, type, payload, actor=actor)

View File

@@ -3,22 +3,36 @@
Coordination-canonical state (overlays, equivalence bindings, aliases, merges, forks) is an
**append-only decision log**, not a mutable file; the queryable *current* state is a **derived
fold** of the log (tier-3 disposable). The log is **totally ordered per space** via a single
**append authority** — here an in-process counter; a git-backed, lease-held authority is a later
binding. That total order is what gives read-your-writes across readers (§8.6).
**append authority**. That total order is what gives read-your-writes across readers (§8.6).
Storage lives behind :class:`EventStore`: :class:`InMemoryEventStore` is the default test double
(an in-process counter); :class:`~shard_wiki.coordination.git_event_store.GitEventStore` is the
git-addressable backend (SHARD-WP-0009). The :class:`DecisionLog` API and the :meth:`fold` are
identical across backends — only storage + the concurrency model differ.
`derived = f(canonical)`: :class:`CoordinationState` is always reproducible by replaying the log.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from types import MappingProxyType
from typing import Any
from typing import Any, Protocol, runtime_checkable
__all__ = ["EventType", "DecisionEvent", "CoordinationState", "DecisionLog"]
__all__ = [
"EventType",
"DecisionEvent",
"CoordinationState",
"EventStore",
"InMemoryEventStore",
"DecisionLog",
"serialize_event",
"deserialize_event",
]
class EventType(Enum):
@@ -63,10 +77,57 @@ class CoordinationState:
return frozenset({identity})
class DecisionLog:
"""In-memory append-only log, totally ordered per space (the append authority for a process).
def serialize_event(event: DecisionEvent) -> bytes:
"""Deterministic, stable-JSON wire form of an event (same bytes for equal events, any process).
A later binding swaps the storage for git + a per-space lease without changing this API.
Sorted keys + compact separators make the serialization canonical, so a git object hashed from
it is reproducible — the basis for content-addressable, comparable logs across backends.
"""
obj = {
"seq": event.seq,
"space": event.space,
"type": event.type.value,
"payload": event.payload,
"actor": event.actor,
"timestamp": event.timestamp.isoformat(),
}
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
def deserialize_event(data: bytes | str) -> DecisionEvent:
"""Inverse of :func:`serialize_event` — round-trips an event byte-for-byte by field."""
obj = json.loads(data)
return DecisionEvent(
seq=obj["seq"],
space=obj["space"],
type=EventType(obj["type"]),
payload=obj["payload"],
actor=obj["actor"],
timestamp=datetime.fromisoformat(obj["timestamp"]),
)
@runtime_checkable
class EventStore(Protocol):
"""Append-only, per-space ordered storage behind :class:`DecisionLog`.
Two bindings exist: :class:`InMemoryEventStore` (default/test double) and
:class:`~shard_wiki.coordination.git_event_store.GitEventStore` (git-addressable). Both assign
a per-space monotonic ``seq`` at the log head and guarantee read-your-writes for their reach
(in-process for memory; cross-process for git).
"""
def append(
self, space: str, type: EventType, payload: Mapping[str, Any], actor: str | None = None
) -> DecisionEvent: ...
def events(self, space: str) -> tuple[DecisionEvent, ...]: ...
class InMemoryEventStore:
"""In-process append-only store, totally ordered per space (the append authority for a process).
The default test double; the git backend preserves this exact contract on durable storage.
"""
def __init__(self) -> None:
@@ -84,10 +145,33 @@ class DecisionLog:
self._events.setdefault(space, []).append(event)
return event
def events(self, space: str) -> tuple[DecisionEvent, ...]:
return tuple(self._events.get(space, ()))
class DecisionLog:
"""Append-only decision log, totally ordered per space, with a derived :meth:`fold`.
Storage is delegated to an :class:`EventStore` (default :class:`InMemoryEventStore`); swapping
in the git backend changes only durability + the concurrency model, not this API or the fold.
"""
def __init__(self, store: EventStore | None = None) -> None:
self._store: EventStore = store if store is not None else InMemoryEventStore()
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
return self._store.append(space, type, payload, actor=actor)
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events in append (total) order. Read-your-writes: a just-appended event
is present immediately."""
return tuple(self._events.get(space, ()))
return self._store.events(space)
def fold(self, space: str) -> CoordinationState:
"""Replay the log into current coordination state (derived = f(log))."""

View File

@@ -0,0 +1,172 @@
"""GitEventStore — a git-addressable binding of :class:`EventStore` (SHARD-WP-0009 T1).
Each space is a ref (``refs/spaces/<sha1(space)>``); each ``append`` writes the event as an
immutable git object (a one-blob tree committed onto the ref) and advances the ref. The commit
chain *is* the totally ordered log: ``seq`` is the depth, ``events`` walks first-parent from the
head oldest→newest. Coordination-canonical state therefore inherits git's history / patch /
review / backup affordances (I-6) and is read-your-writes correct across processes.
The total order is enforced at storage by a **compare-and-swap** ref update
(``git update-ref <ref> <new> <old>``): two appenders racing off the same head — the loser's CAS
fails and it retries off the new head, so a non-holder can never fork the log. The lease layer
(T2) sits *above* this as the append-authority policy; CAS is the mechanism that makes it safe.
Implemented over the ``git`` CLI through :mod:`subprocess` — zero runtime dependencies.
"""
from __future__ import annotations
import hashlib
import os
import subprocess
from collections.abc import Mapping
from pathlib import Path
from typing import Any
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventType,
deserialize_event,
serialize_event,
)
__all__ = ["GitEventStore"]
# Fixed identity so commit objects are reproducible and never prompt for git config; the event's
# own timestamp/actor carry the real provenance, the commit is just the ordered container.
_GIT_IDENTITY = {
"GIT_AUTHOR_NAME": "shard-wiki",
"GIT_AUTHOR_EMAIL": "coordination@shard-wiki",
"GIT_COMMITTER_NAME": "shard-wiki",
"GIT_COMMITTER_EMAIL": "coordination@shard-wiki",
}
_EVENT_PATH = "event.json"
_MAX_CAS_RETRIES = 50
class GitEventStore:
"""Git-backed, append-only, per-space ordered event store (an :class:`EventStore`)."""
def __init__(self, repo_path: str | Path) -> None:
self.repo_path = Path(repo_path)
self.repo_path.mkdir(parents=True, exist_ok=True)
if not (self.repo_path / "HEAD").exists() and not (self.repo_path / ".git").exists():
self._git("init", "--quiet", str(self.repo_path), at_cwd=True)
# -- EventStore contract -------------------------------------------------
def append(
self,
space: str,
type: EventType,
payload: Mapping[str, Any],
actor: str | None = None,
) -> DecisionEvent:
"""Append one event, advancing the space ref under compare-and-swap (retry-on-race)."""
ref = self._ref(space)
for _ in range(_MAX_CAS_RETRIES):
head = self._head(ref)
seq = self._count(ref, head)
event = DecisionEvent(
seq=seq, space=space, type=type, payload=dict(payload), actor=actor
)
commit = self._commit_event(event, parent=head)
if self._cas_update(ref, new=commit, old=head):
return event
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
def import_event(self, event: DecisionEvent) -> None:
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
"""
ref = self._ref(event.space)
head = self._head(ref)
expected = self._count(ref, head)
if event.seq != expected:
raise ValueError(
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
)
commit = self._commit_event(event, parent=head)
if not self._cas_update(ref, new=commit, old=head):
raise RuntimeError(f"import race on {ref}")
def events(self, space: str) -> tuple[DecisionEvent, ...]:
"""The space's events oldest→newest (append/total order)."""
ref = self._ref(space)
head = self._head(ref)
if head is None:
return ()
shas = self._git("rev-list", "--reverse", "--first-parent", ref).decode().split()
return tuple(
deserialize_event(self._git("cat-file", "blob", f"{sha}:{_EVENT_PATH}"))
for sha in shas
)
# -- git plumbing --------------------------------------------------------
def _commit_event(self, event: DecisionEvent, parent: str | None) -> str:
blob = self._git(
"hash-object", "-w", "--stdin", stdin=serialize_event(event)
).decode().strip()
tree = self._git(
"mktree", stdin=f"100644 blob {blob}\t{_EVENT_PATH}\n".encode()
).decode().strip()
args = ["commit-tree", tree, "-m", f"event {event.seq} {event.type.value}"]
if parent is not None:
args += ["-p", parent]
# Pin the commit date to the event's timestamp for reproducible objects.
date = event.timestamp.isoformat()
env = {**_GIT_IDENTITY, "GIT_AUTHOR_DATE": date, "GIT_COMMITTER_DATE": date}
return self._git(*args, env=env).decode().strip()
def _cas_update(self, ref: str, new: str, old: str | None) -> bool:
"""``git update-ref`` with the old value as a CAS guard (empty oldvalue == must-not-exist).
Returns False if the ref moved since we read ``old`` (lost the race) — the caller retries.
"""
result = self._run("update-ref", ref, new, old if old is not None else "")
return result.returncode == 0
def _head(self, ref: str) -> str | None:
result = self._run("rev-parse", "--verify", "--quiet", ref)
out = result.stdout.decode().strip()
return out or None
def _count(self, ref: str, head: str | None) -> int:
if head is None:
return 0
return int(self._git("rev-list", "--count", "--first-parent", ref).decode().strip())
@staticmethod
def _ref(space: str) -> str:
return f"refs/spaces/{hashlib.sha1(space.encode()).hexdigest()}"
def _git(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
) -> bytes:
result = self._run(*args, stdin=stdin, env=env, at_cwd=at_cwd, check=True)
return result.stdout
def _run(
self,
*args: str,
stdin: bytes | None = None,
env: dict | None = None,
at_cwd: bool = False,
check: bool = False,
) -> subprocess.CompletedProcess:
base = ["git"] if at_cwd else ["git", "-C", str(self.repo_path)]
return subprocess.run(
[*base, *args],
input=stdin,
capture_output=True,
env={**os.environ, **(env or {})},
check=check,
)

View File

@@ -0,0 +1,53 @@
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
reproduces identical coordination state; only durability changes.
"""
from __future__ import annotations
from collections.abc import Iterable
from pathlib import Path
from shard_wiki.coordination.decision_log import (
DecisionEvent,
EventStore,
deserialize_event,
serialize_event,
)
from shard_wiki.coordination.git_event_store import GitEventStore
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
count = 0
for event in events:
dest.import_event(event)
count += 1
return count
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
return import_log(source.events(space), dest)
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
count = 0
with open(path, "wb") as handle:
for event in events:
handle.write(serialize_event(event) + b"\n")
count += 1
return count
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
with open(path, "rb") as handle:
events = [deserialize_event(line) for line in handle if line.strip()]
return import_log(events, dest)

View File

@@ -0,0 +1,49 @@
"""engine/ — shard-wiki's native, headless wiki engine (a canonical-mode shard backend).
A small page-store kernel + a typed-extension runtime (WikiEngineCoreArchitecture). The engine
is *one shard*: it is consumed by the orchestrator only via its `EngineShardAdapter`; it never
imports the derived tier (`union`/`projection`).
"""
from shard_wiki.engine.activation import (
ActivationContext,
ActivationProvider,
ActivationResolver,
StaticProvider,
feature_control_provider,
)
from shard_wiki.engine.extension import (
ActiveExtensions,
Extension,
ExtensionError,
ExtensionRuntime,
Hook,
)
from shard_wiki.engine.adapter import EngineShardAdapter, build_engine_shard
from shard_wiki.engine.kernel import EngineKernel
from shard_wiki.engine.links import extract_wikilinks
from shard_wiki.engine.profile import (
ProfileContribution,
derive_profile,
engine_base_profile,
)
__all__ = [
"EngineKernel",
"extract_wikilinks",
"Hook",
"Extension",
"ExtensionError",
"ExtensionRuntime",
"ActiveExtensions",
"ActivationContext",
"ActivationProvider",
"StaticProvider",
"ActivationResolver",
"feature_control_provider",
"engine_base_profile",
"ProfileContribution",
"derive_profile",
"EngineShardAdapter",
"build_engine_shard",
]

View File

@@ -0,0 +1,129 @@
"""Per-shard extension activation (WikiEngineCoreArchitecture E-4/E-8, ADR-0001).
Decides *which registered extensions are active* for a given shard — **availability only, never
authorization**. The mechanism is OpenFeature-shaped and **provider-pluggable**:
- :class:`StaticProvider` is the **standalone / L0 default** — zero external dependency, in-process
flags with optional per-shard scoping. A kernel-only or offline shard uses this.
- :func:`feature_control_provider` lazily wraps the helix_forge ``feature_control_sdk`` (over
OpenFeature) when it is installed; absent, it returns ``None`` and the caller falls back to the
static default. This mirrors shard-wiki's identity-provider ladder (local default → external
when present), and keeps the engine core pure-stdlib.
An *activation profile* is ``{extension id → config}``; the active id set then feeds the
extension runtime's `activate()` (T2) and the derived capability profile (T4, E-5).
"""
from __future__ import annotations
from collections.abc import Iterable, Mapping
from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable
__all__ = [
"ActivationContext",
"ActivationProvider",
"StaticProvider",
"ActivationResolver",
"feature_control_provider",
]
@dataclass(frozen=True, slots=True)
class ActivationContext:
"""Scope for an activation decision. Carries no principal/authz — availability only."""
shard_id: str
tenant_id: str | None = None
extra: Mapping[str, Any] = field(default_factory=dict)
def as_dict(self) -> dict[str, Any]:
d: dict[str, Any] = {"shard_id": self.shard_id}
if self.tenant_id is not None:
d["tenant_id"] = self.tenant_id
d.update(self.extra)
return d
@runtime_checkable
class ActivationProvider(Protocol):
"""Evaluates feature availability for an extension key in a context (OpenFeature-shaped)."""
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool: ...
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]: ...
@dataclass(frozen=True, slots=True)
class StaticProvider:
"""The standalone default: in-process flags, optionally overridden per shard. Zero deps.
``flags`` is the base availability map; ``per_shard[shard_id]`` overrides it for one shard;
``configs[feature_key]`` supplies per-extension config. Unknown keys → ``default``.
"""
flags: Mapping[str, bool] = field(default_factory=dict)
per_shard: Mapping[str, Mapping[str, bool]] = field(default_factory=dict)
configs: Mapping[str, Mapping[str, Any]] = field(default_factory=dict)
default: bool = False
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
shard = context.get("shard_id")
scoped = self.per_shard.get(shard, {}) if shard is not None else {}
if feature_key in scoped:
return scoped[feature_key]
return self.flags.get(feature_key, self.default)
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
return self.configs.get(feature_key, {})
class ActivationResolver:
"""Maps candidate extension ids → the active set / activation profile for a context."""
def __init__(self, provider: ActivationProvider) -> None:
self.provider = provider
def active_extensions(
self, candidate_ids: Iterable[str], context: ActivationContext
) -> set[str]:
ctx = context.as_dict()
return {eid for eid in candidate_ids if self.provider.is_active(eid, ctx)}
def activation_profile(
self, candidate_ids: Iterable[str], context: ActivationContext
) -> dict[str, Mapping[str, Any]]:
"""``{extension id → config}`` for the active subset."""
ctx = context.as_dict()
return {
eid: self.provider.config(eid, ctx)
for eid in candidate_ids
if self.provider.is_active(eid, ctx)
}
def feature_control_provider(domain: str | None = None) -> ActivationProvider | None:
"""Return a feature-control-backed provider if ``feature_control_sdk`` is importable, else
``None`` (caller falls back to :class:`StaticProvider`). Lazy import keeps the engine core
dependency-free (ADR-0001)."""
try: # optional engine extra — not a core dependency
from feature_control_sdk import FeatureControlClient # type: ignore
except ImportError:
return None
client = FeatureControlClient(domain=domain)
@dataclass(frozen=True, slots=True)
class _FeatureControlProvider:
_client: Any
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
return bool(
self._client.get_boolean_value(feature_key, False, context=dict(context))
)
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
getter = getattr(self._client, "get_object_value", None)
return dict(getter(feature_key, {}, context=dict(context))) if getter else {}
return _FeatureControlProvider(client)

View File

@@ -0,0 +1,82 @@
"""EngineShardAdapter — the engine exposed as a canonical-mode shard (WikiEngineCoreArchitecture
§6, E-1/E-5).
The engine is *one shard*: the orchestrator consumes it only through this `ShardAdapter`. The
adapter is backed by the kernel (T1) + a composed extension set (T2/T3); its §A capability
profile is **derived from the active extensions** (T4), so the orchestrator sees an honest,
conformance-verifiable profile that reflects exactly what is activated. Read/write run the
extensions' transform hooks; everything above this stays in the orchestrator (no union/projection
import).
"""
from __future__ import annotations
from collections.abc import Iterable
from shard_wiki.adapters import ShardAdapter
from shard_wiki.engine.activation import ActivationContext, ActivationProvider, ActivationResolver
from shard_wiki.engine.extension import ActiveExtensions, ExtensionRuntime, Hook
from shard_wiki.engine.kernel import EngineKernel
from shard_wiki.engine.profile import derive_profile
from shard_wiki.model import CapabilityProfile, NotSupported, Page, Verb
__all__ = ["EngineShardAdapter", "build_engine_shard"]
class EngineShardAdapter(ShardAdapter):
def __init__(
self,
kernel: EngineKernel,
active: ActiveExtensions,
base_profile: CapabilityProfile | None = None,
) -> None:
self._kernel = kernel
self._active = active
self._profile = derive_profile(active, base_profile) # validated (E-5)
@property
def shard_id(self) -> str:
return self._kernel.shard_id
def profile(self) -> CapabilityProfile:
return self._profile
def keys(self) -> Iterable[str]:
return self._kernel.keys()
def read(self, key: str) -> Page:
page = self._kernel.read(key)
return self._active.dispatch_transform(Hook.ON_READ, page, {"shard_id": self.shard_id})
def current_rev(self, key: str) -> str | None:
return self._kernel.current_rev(key)
def write(self, key: str, body: str) -> Page:
if not self._profile.supports(Verb.WRITE):
raise NotSupported(f"{type(self).__name__} ({self.shard_id}) is read-only")
body = self._active.dispatch_transform(
Hook.ON_WRITE, body, {"shard_id": self.shard_id, "key": key}
)
return self._kernel.write(key, body)
def build_engine_shard(
shard_id: str,
runtime: ExtensionRuntime,
*,
activate: Iterable[str] | None = None,
provider: ActivationProvider | None = None,
context: ActivationContext | None = None,
base_profile: CapabilityProfile | None = None,
) -> EngineShardAdapter:
"""Stand up an engine shard: resolve which extensions are active (explicit ``activate`` ids,
or via an activation ``provider`` over the runtime's available set), compose them, and wrap a
fresh kernel as a `ShardAdapter`.
"""
if provider is not None:
ctx = context or ActivationContext(shard_id)
ids = ActivationResolver(provider).active_extensions(runtime.available(), ctx)
else:
ids = set(activate or ())
active = runtime.activate(ids)
return EngineShardAdapter(EngineKernel(shard_id), active, base_profile)

View File

@@ -0,0 +1,165 @@
"""Typed-extension runtime — the engine framework (WikiEngineCoreArchitecture §4, E-3/E-9).
Everything beyond the kernel's c2-minimum is an :class:`Extension`: it declares a typed
contract (id, provided capabilities, declared types, bound hooks, dependencies, conflicts) and
the runtime **composes** an activation set deterministically, **rejecting impossible profiles**
(unmet deps / conflicts / type collisions) — the §6.5 capability-as-data discipline applied to
extensions. Extension structure is **verified at registration** (mirrors §6.6 conformance):
bad ids or non-callable hook handlers are refused, so the framework acts on verified data.
Hooks are dispatched in a declared, deterministic order (dependency-topological, ties by id):
*transform* hooks chain a payload through handlers; *collect* hooks gather contributions.
"""
from __future__ import annotations
from collections.abc import Callable, Iterable, Mapping
from enum import Enum
from typing import Any, ClassVar
__all__ = ["Hook", "Extension", "ExtensionError", "ExtensionRuntime", "ActiveExtensions"]
class ExtensionError(ValueError):
"""Raised when an extension is malformed or an activation set is impossible (§6.5)."""
class Hook(Enum):
# transform hooks: each handler takes (payload, ctx) and returns the next payload
ON_WRITE = "on_write" # transform a draft before persist
ON_READ = "on_read" # transform a page on read
ON_RESOLVE = "on_resolve" # transform a name resolution
ON_RENDER = "on_render" # produce a derived representation
# collect hooks: each handler takes (payload, ctx) and returns a contribution
ON_LINK = "on_link" # contribute link/transclusion edges
ON_QUERY = "on_query" # answer a query
ON_PROFILE = "on_profile" # contribute capability-profile positions (E-5)
_TRANSFORM = frozenset({Hook.ON_WRITE, Hook.ON_READ, Hook.ON_RESOLVE, Hook.ON_RENDER})
_COLLECT = frozenset({Hook.ON_LINK, Hook.ON_QUERY, Hook.ON_PROFILE})
class Extension:
"""Base class for a typed extension. Subclasses set the class vars and override
:meth:`hooks` to bind handlers (signature ``handler(payload, ctx) -> result``)."""
id: ClassVar[str] = ""
provides: ClassVar[tuple[str, ...]] = ()
declares_types: ClassVar[tuple[str, ...]] = ()
depends_on: ClassVar[tuple[str, ...]] = ()
conflicts_with: ClassVar[tuple[str, ...]] = ()
def hooks(self) -> Mapping[Hook, Callable[[Any, Any], Any]]:
return {}
class ActiveExtensions:
"""A composed, ordered activation set with deterministic hook dispatch."""
def __init__(self, ordered: list[Extension]) -> None:
self._ordered = ordered
self.ids: tuple[str, ...] = tuple(e.id for e in ordered)
self._tables: dict[Hook, list[tuple[str, Callable[[Any, Any], Any]]]] = {}
for ext in ordered:
for hook, fn in ext.hooks().items():
self._tables.setdefault(hook, []).append((ext.id, fn))
def handlers(self, hook: Hook) -> tuple[str, ...]:
"""The extension ids bound to ``hook``, in dispatch order (for introspection)."""
return tuple(eid for eid, _ in self._tables.get(hook, ()))
def dispatch_transform(self, hook: Hook, payload: Any, ctx: Any = None) -> Any:
if hook not in _TRANSFORM:
raise ExtensionError(f"{hook} is not a transform hook")
for _eid, fn in self._tables.get(hook, ()):
payload = fn(payload, ctx)
return payload
def dispatch_collect(self, hook: Hook, payload: Any = None, ctx: Any = None) -> list[Any]:
if hook not in _COLLECT:
raise ExtensionError(f"{hook} is not a collect hook")
return [fn(payload, ctx) for _eid, fn in self._tables.get(hook, ())]
class ExtensionRuntime:
def __init__(self) -> None:
self._registered: dict[str, Extension] = {}
def available(self) -> frozenset[str]:
"""Ids of all registered extensions (the candidate set for activation)."""
return frozenset(self._registered)
def register(self, ext: Extension) -> Extension:
"""Register an extension after structural verification (mirrors §6.6)."""
if not ext.id or not ext.id.startswith("ext."):
raise ExtensionError(f"extension id must be 'ext.<name>', got {ext.id!r}")
if ext.id in self._registered:
raise ExtensionError(f"duplicate extension id: {ext.id}")
bound = ext.hooks()
for hook, fn in bound.items():
if not isinstance(hook, Hook):
raise ExtensionError(f"{ext.id}: hook key {hook!r} is not a Hook")
if not callable(fn):
raise ExtensionError(f"{ext.id}: handler for {hook} is not callable")
self._registered[ext.id] = ext
return ext
def activate(self, ids: Iterable[str]) -> ActiveExtensions:
"""Compose an activation set: dependency closure → conflict/type checks → deterministic
order. Raises :class:`ExtensionError` on an impossible profile."""
requested = set(ids)
unknown = requested - self._registered.keys()
if unknown:
raise ExtensionError(f"unknown extensions: {sorted(unknown)}")
# dependency closure
active: set[str] = set()
frontier = list(requested)
while frontier:
eid = frontier.pop()
if eid in active:
continue
ext = self._registered.get(eid)
if ext is None:
raise ExtensionError(f"unmet dependency: {eid}")
active.add(eid)
frontier.extend(d for d in ext.depends_on if d not in active)
exts = [self._registered[e] for e in active]
# conflicts
for ext in exts:
clash = active & set(ext.conflicts_with)
if clash:
raise ExtensionError(f"{ext.id} conflicts with active {sorted(clash)}")
# type collisions (two active extensions claiming the same type id)
owner: dict[str, str] = {}
for ext in exts:
for t in ext.declares_types:
if t in owner:
raise ExtensionError(
f"type collision on {t!r}: {owner[t]} and {ext.id}"
)
owner[t] = ext.id
return ActiveExtensions(self._topo_order(exts))
def _topo_order(self, exts: list[Extension]) -> list[Extension]:
"""Dependencies before dependents; ties broken by id (deterministic)."""
by_id = {e.id: e for e in exts}
ordered: list[Extension] = []
placed: set[str] = set()
def visit(ext: Extension) -> None:
if ext.id in placed:
return
for dep in sorted(d for d in ext.depends_on if d in by_id):
visit(by_id[dep])
placed.add(ext.id)
ordered.append(ext)
for ext in sorted(exts, key=lambda e: e.id):
visit(ext)
return ordered

View File

@@ -0,0 +1,10 @@
"""engine/extensions/ — built-in typed extensions for the wiki engine.
Each is a typed :class:`~shard_wiki.engine.extension.Extension` a shard activates only if needed.
``ext.struct`` (typed records) is the first; more (views, addressing, computational, authz) follow
the same pattern.
"""
from shard_wiki.engine.extensions.struct import StructExt, parse_frontmatter
__all__ = ["StructExt", "parse_frontmatter"]

View File

@@ -0,0 +1,81 @@
"""ext.struct — typed records, a first built-in extension (WikiEngineCoreArchitecture X-STRUCT).
Demonstrates the typed-extension framework end-to-end. A page may carry a leading in-text
frontmatter block (`---` … `---`, `key: value` lines — git-diffable structure, blueprint T12).
With this extension **active**, the engine:
- **ON_WRITE** validates the structured block (optionally against an allowed-field set) — a
malformed/disallowed structured page is rejected; the body is otherwise unchanged
(content-preserving, so write conformance holds);
- **ON_READ** tags such pages as `PageShape.TYPED_RECORD`;
- **ON_PROFILE** raises the shard's profile with the `structured-payload` verb (E-5).
With the extension **inactive**, the kernel treats the same page as opaque prose — the feature
is genuinely absent (honest profile). This is "activate only what you need" in action.
"""
from __future__ import annotations
import dataclasses
from collections.abc import Iterable, Mapping
from typing import Any
from shard_wiki.engine.extension import Extension, Hook
from shard_wiki.engine.profile import ProfileContribution
from shard_wiki.model import Page, PageShape, Verb
__all__ = ["StructExt", "parse_frontmatter"]
def parse_frontmatter(body: str) -> tuple[dict[str, str], bool]:
"""Parse a leading ``---`` … ``---`` block of ``key: value`` lines.
Returns ``(fields, has_block)``. An unterminated opening ``---`` is *not* a valid block.
"""
lines = body.splitlines()
if not lines or lines[0].strip() != "---":
return {}, False
fields: dict[str, str] = {}
for line in lines[1:]:
if line.strip() == "---":
return fields, True
if ":" in line:
key, _, value = line.partition(":")
fields[key.strip()] = value.strip()
return {}, False # no closing fence → not a frontmatter block
class StructExt(Extension):
id = "ext.struct"
declares_types = ("record",)
provides = ("capability.wiki.page-model",)
def __init__(self, allowed_fields: Iterable[str] | None = None) -> None:
self._allowed: set[str] | None = set(allowed_fields) if allowed_fields is not None else None
def hooks(self) -> Mapping[Hook, Any]:
return {
Hook.ON_WRITE: self._on_write,
Hook.ON_READ: self._on_read,
Hook.ON_PROFILE: self._on_profile,
}
def _on_write(self, body: str, ctx: Any) -> str:
fields, has_block = parse_frontmatter(body)
if has_block and self._allowed is not None:
disallowed = set(fields) - self._allowed
if disallowed:
raise ValueError(f"ext.struct: disallowed fields {sorted(disallowed)}")
return body # structure stays in-text (git-diffable); body unchanged
def _on_read(self, page: Page, ctx: Any) -> Page:
_, has_block = parse_frontmatter(page.body)
return dataclasses.replace(page, shape=PageShape.TYPED_RECORD) if has_block else page
def _on_profile(self, payload: Any, ctx: Any) -> ProfileContribution:
return ProfileContribution(verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD}))
@staticmethod
def fields(body: str) -> dict[str, str]:
"""Parsed structured fields of a page body (empty if it has no frontmatter block)."""
return parse_frontmatter(body)[0]

View File

@@ -0,0 +1,87 @@
"""Engine kernel — the small page-store core (WikiEngineCoreArchitecture §3, EC-1…EC-4).
The irreducible engine: author/read/edit pages (edit = a new version; delete = a recoverable
tombstone — history is the floor, I-10), enumerate keys, and resolve `[[wikilinks]]` (red-link =
an unresolved target). No feature beyond this c2-minimum lives in the kernel; everything else is
a typed extension (E-3).
Storage is intentionally simple here (in-memory version history); the git-IS-store backing
(SHARD-WP-0009/0012) slots in behind the same API later. The kernel reuses the page model and
provenance leaf; it does not redefine them.
"""
from __future__ import annotations
from collections.abc import Iterable
from datetime import datetime, timezone
from shard_wiki.engine.links import extract_wikilinks
from shard_wiki.model import Identity, Page, Placement
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
__all__ = ["EngineKernel"]
class EngineKernel:
"""An in-process page store with per-page version history for one engine shard."""
def __init__(self, shard_id: str) -> None:
self.shard_id = shard_id
self._versions: dict[str, list[Page]] = {}
self._deleted: set[str] = set()
# --- write path (create/edit are one operation; both append a version) ---
def write(self, key: str, body: str) -> Page:
versions = self._versions.setdefault(key, [])
rev = str(len(versions) + 1)
page = Page(
identity=Identity(self.shard_id, key),
body=body,
envelope=ProvenanceEnvelope(
source_shard=self.shard_id,
liveness=Liveness.STATIC,
staleness=Staleness.FRESH,
source_rev=rev,
observed_at=datetime.now(tz=timezone.utc),
),
placements=(Placement(self.shard_id, key),),
)
versions.append(page)
self._deleted.discard(key)
return page
# --- read path ---
def exists(self, key: str) -> bool:
return key in self._versions and key not in self._deleted
def read(self, key: str) -> Page:
"""Latest version of a live page. Raises ``KeyError`` if absent or deleted."""
if not self.exists(key):
raise KeyError(key)
return self._versions[key][-1]
def keys(self) -> Iterable[str]:
return (k for k in sorted(self._versions) if k not in self._deleted)
def current_rev(self, key: str) -> str | None:
return self._versions[key][-1].envelope.source_rev if self.exists(key) else None
# --- history & recoverability (I-10): versions are retained across delete ---
def history(self, key: str) -> tuple[Page, ...]:
"""All versions ever written for ``key`` (oldest→newest), even after delete."""
return tuple(self._versions.get(key, ()))
def delete(self, key: str) -> None:
"""Tombstone a page (history retained; restore by writing again)."""
if key not in self._versions:
raise KeyError(key)
self._deleted.add(key)
# --- links (EC-4): resolution + red-link detection within this shard ---
def links(self, key: str) -> list[str]:
"""Wikilink targets in a page's current body."""
return extract_wikilinks(self.read(key).body)
def resolve_link(self, target: str) -> Identity | None:
"""Resolve a wikilink target to a live page identity, or ``None`` (a **red-link**)."""
return self.read(target).identity if self.exists(target) else None

View File

@@ -0,0 +1,25 @@
"""Wikilink extraction — the kernel's link primitive (WikiEngineCoreArchitecture EC-4).
`[[Target]]` and `[[Target|label]]`. CamelCase auto-linking is intentionally NOT here (it is an
opt-in concern per FederationRequirements ADR-06); the kernel only knows explicit wikilinks.
Link *resolution* (and red-link detection) is the kernel's job (it knows which keys exist);
*rendering* is a consumer concern (headless engine, no UI).
"""
from __future__ import annotations
import re
__all__ = ["extract_wikilinks"]
_WIKILINK = re.compile(r"\[\[([^\]|]+?)(?:\|[^\]]*)?\]\]")
def extract_wikilinks(body: str) -> list[str]:
"""Return the ordered, de-duplicated wikilink targets in ``body`` (label part dropped)."""
seen: dict[str, None] = {}
for m in _WIKILINK.finditer(body):
target = m.group(1).strip()
if target:
seen.setdefault(target, None)
return list(seen)

View File

@@ -0,0 +1,112 @@
"""Capability profile derived from active extensions (WikiEngineCoreArchitecture E-5).
The engine's §A `CapabilityProfile` is **computed**, not hand-set: start from the kernel base
profile, then fold each active extension's `on_profile` contribution (in the runtime's
deterministic order), then `validate()`. This realizes the chain *configuration (which
extensions) → capability (the profile) → conformance* — activating an extension raises the
shard's advertised capabilities, and composition can never yield an impossible profile (validate
rejects it, §6.5).
"""
from __future__ import annotations
import dataclasses
from dataclasses import dataclass
from shard_wiki.engine.extension import ActiveExtensions, Hook
from shard_wiki.model import (
AccessGrant,
Addressing,
AttachmentMode,
CapabilityProfile,
ContentOpacity,
History,
MergeModel,
NativeQuery,
OperationalEnvelope,
Substrate,
Translation,
Verb,
WriteGranularity,
)
from shard_wiki.provenance import Liveness
__all__ = ["engine_base_profile", "ProfileContribution", "derive_profile"]
# Profile fields an extension may *raise* via on_profile (substrate/attachment are kernel-fixed).
_OVERRIDABLE = (
"write_granularity",
"content_opacity",
"liveness",
"history",
"merge_model",
"addressing",
"native_query",
"translation",
"access_grant",
)
def engine_base_profile() -> CapabilityProfile:
"""The kernel-only (no extensions) capability profile — the c2-minimum engine shard."""
return CapabilityProfile(
substrate=Substrate.FILES,
attachment_mode=AttachmentMode.IN_ENGINE_HOST,
write_granularity=WriteGranularity.PER_PAGE,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.INTERNAL_ONLY,
merge_model=MergeModel.NONE,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset({Verb.READ, Verb.WRITE}),
).validate()
@dataclass(frozen=True, slots=True)
class ProfileContribution:
"""An extension's contribution to the derived profile (returned from its ON_PROFILE hook).
A non-``None`` axis overrides that axis; ``verbs_add`` are unioned in. Order = the runtime's
deterministic dispatch order, so later extensions win on a contested axis."""
write_granularity: WriteGranularity | None = None
content_opacity: ContentOpacity | None = None
liveness: Liveness | None = None
history: History | None = None
merge_model: MergeModel | None = None
addressing: Addressing | None = None
native_query: NativeQuery | None = None
translation: Translation | None = None
access_grant: AccessGrant | None = None
verbs_add: frozenset[Verb] = frozenset()
def derive_profile(
active: ActiveExtensions, base: CapabilityProfile | None = None
) -> CapabilityProfile:
"""Fold active extensions' ON_PROFILE contributions onto ``base`` and validate the result.
Raises :class:`~shard_wiki.model.ProfileError` if the composed profile is impossible — so an
activation set can never advertise an invalid capability profile.
"""
profile = base or engine_base_profile()
contributions = active.dispatch_collect(Hook.ON_PROFILE)
overrides: dict[str, object] = {}
verbs: set[Verb] = set(profile.supported_verbs)
for contrib in contributions:
if not isinstance(contrib, ProfileContribution):
continue
for field_name in _OVERRIDABLE:
value = getattr(contrib, field_name)
if value is not None:
overrides[field_name] = value
verbs |= set(contrib.verbs_add)
return dataclasses.replace(
profile, supported_verbs=frozenset(verbs), **overrides
).validate()

View File

@@ -0,0 +1,46 @@
"""incremental/ — the incremental-first derived tier (CoreArchitectureBlueprint §8.7).
Equivalence is **indexed** (blocking/LSH + verify), not pairwise O(N²); maintenance is
**change-driven** (delta with retraction + propagation, review B-4), keeping the derived tier equal
to a from-scratch rebuild — which becomes a bounded fallback, not the operational path. A
Merkle-style **digest** plus a background **consistency-checker** make ``derived = f(canonical)``
verified rather than asserted (I-2), self-healing on detected drift.
In-memory only for this slice (no persisted index store); per-partition structure is honoured but
multi-tenant deployment is later. Per the dependency rule this imports down (model/provenance) and
is wired by the orchestrator.
"""
from shard_wiki.incremental.equivalence import (
EquivalenceEdge,
EquivalenceIndex,
normalized_title,
)
from shard_wiki.incremental.minhash import (
MinHasher,
band_keys,
jaccard,
shingles,
)
from shard_wiki.incremental.union_index import UnionIndex
from shard_wiki.incremental.verification import (
ConsistencyChecker,
ConsistencyReport,
derived_digest,
region_digest,
)
__all__ = [
"shingles",
"MinHasher",
"band_keys",
"jaccard",
"EquivalenceEdge",
"EquivalenceIndex",
"normalized_title",
"derived_digest",
"region_digest",
"ConsistencyReport",
"ConsistencyChecker",
"UnionIndex",
]

View File

@@ -0,0 +1,225 @@
"""Indexed equivalence — blocking + verify, incrementally maintained (SHARD-WP-0011 T1/T2).
Equivalence (two *distinct* identities holding the same page) is detected without pairwise O(N²):
1. **Blocking** generates candidate pairs — pages sharing a normalized-title bucket or an LSH band
(MinHash over content shingles).
2. **Verify** confirms a candidate — exact-body fingerprint match, or shingle Jaccard ≥ threshold —
plus **curator bindings** (explicit decision-log edges) which are always equivalence edges.
The index is **incrementally maintained** (T2): ``add`` / ``update`` / ``remove`` re-bucket the
changed page, **retract** the edges it leaves and **add** the edges it enters; equivalence groups
are the connected components of the current edge set, so a retraction that disconnects a component
**splits** a chorus automatically. A full :meth:`build` is just repeated ``add`` — the bounded
rebuild fallback. The invariant (and the test oracle): incremental state == a from-scratch rebuild.
"""
from __future__ import annotations
import hashlib
import re
from collections.abc import Iterable
from dataclasses import dataclass
from shard_wiki.incremental.minhash import MinHasher, band_keys, jaccard, shingles
from shard_wiki.model import Identity, Page
__all__ = ["EquivalenceEdge", "EquivalenceIndex", "normalized_title"]
_NONALNUM_RE = re.compile(r"[^a-z0-9]+")
def normalized_title(key: str) -> str:
"""A blocking bucket key: the last path segment, lowercased, stripped of non-alphanumerics."""
leaf = key.rsplit("/", 1)[-1]
return _NONALNUM_RE.sub("", leaf.lower())
@dataclass(frozen=True, slots=True)
class EquivalenceEdge:
"""A verified equivalence between two identities, tagged with why it was accepted."""
a: Identity
b: Identity
reason: str # "fingerprint" | "content" | "curator"
@dataclass(frozen=True, slots=True)
class _Entry:
shingle_set: frozenset[str]
bands: tuple[tuple[int, tuple[int, ...]], ...]
title: str
fingerprint: str
def _fingerprint(body: str) -> str:
return hashlib.blake2b(body.strip().encode("utf-8"), digest_size=16).hexdigest()
def _pair(a: Identity, b: Identity) -> frozenset[Identity]:
return frozenset((a, b))
class EquivalenceIndex:
"""An incrementally maintained, blocked-and-verified equivalence relation over union pages."""
def __init__(
self,
*,
num_perm: int = 64,
num_bands: int = 32,
threshold: float = 0.7,
hasher: MinHasher | None = None,
) -> None:
self.threshold = threshold
self.num_bands = num_bands
self._hasher = hasher or MinHasher(num_perm=num_perm)
self._entries: dict[Identity, _Entry] = {}
self._band_buckets: dict[tuple[int, tuple[int, ...]], set[Identity]] = {}
self._title_buckets: dict[str, set[Identity]] = {}
self._content_edges: dict[frozenset[Identity], str] = {}
self._curator_edges: set[frozenset[Identity]] = set()
# -- build / maintain ----------------------------------------------------
def build(
self,
pages: Iterable[Page],
curator_edges: Iterable[tuple[Identity, Identity]] = (),
) -> None:
"""Rebuild from scratch (the bounded fallback): add every page, then curator edges."""
self.__init__(
num_bands=self.num_bands, threshold=self.threshold, hasher=self._hasher
)
for page in pages:
self.add(page)
for a, b in curator_edges:
self.bind(a, b)
def add(self, page: Page) -> None:
"""Index a new (or, via :meth:`update`, refreshed) page and add its equivalence edges."""
identity = page.identity
entry = self._make_entry(page)
self._entries[identity] = entry
for key in entry.bands:
self._band_buckets.setdefault(key, set()).add(identity)
self._title_buckets.setdefault(entry.title, set()).add(identity)
for candidate in self._candidates(identity, entry):
reason = self._verify(identity, candidate)
if reason is not None:
self._content_edges[_pair(identity, candidate)] = reason
def remove(self, identity: Identity) -> None:
"""Drop a page: de-bucket it and retract every content edge incident to it."""
entry = self._entries.pop(identity, None)
if entry is None:
return
for key in entry.bands:
self._discard_bucket(self._band_buckets, key, identity)
self._discard_bucket(self._title_buckets, entry.title, identity)
for edge in [e for e in self._content_edges if identity in e]:
del self._content_edges[edge]
def update(self, page: Page) -> None:
"""Apply a change as retract-then-add: stale (bucket-exit) edges go, new edges arrive."""
self.remove(page.identity)
self.add(page)
def bind(self, a: Identity, b: Identity) -> None:
"""Record a curator equivalence (an explicit decision-log binding); always an edge."""
if a != b:
self._curator_edges.add(_pair(a, b))
def unbind(self, a: Identity, b: Identity) -> None:
self._curator_edges.discard(_pair(a, b))
def set_curator_edges(self, edges: Iterable[tuple[Identity, Identity]]) -> None:
"""Replace all curator edges at once (re-syncing from the decision-log fold)."""
self._curator_edges = {_pair(a, b) for a, b in edges if a != b}
# -- queries -------------------------------------------------------------
def identities(self) -> frozenset[Identity]:
"""All identities currently present in the index."""
return frozenset(self._entries)
def fingerprint(self, identity: Identity) -> str | None:
"""The content fingerprint indexed for ``identity`` (None if absent) — a digest leaf."""
entry = self._entries.get(identity)
return entry.fingerprint if entry is not None else None
def edges(self) -> frozenset[frozenset[Identity]]:
"""All equivalence edges (content + curator) among currently present identities."""
present = self._entries.keys()
curator = {e for e in self._curator_edges if e <= present}
return frozenset(set(self._content_edges) | curator)
def groups(self) -> tuple[frozenset[Identity], ...]:
"""Equivalence groups: connected components of size ≥ 2 (union-find over the edges)."""
parent: dict[Identity, Identity] = {}
def find(x: Identity) -> Identity:
parent.setdefault(x, x)
root = x
while parent[root] != root:
root = parent[root]
while parent[x] != root:
parent[x], x = root, parent[x]
return root
for edge in self.edges():
a, b = tuple(edge)
ra, rb = find(a), find(b)
if ra != rb:
parent[ra] = rb
comps: dict[Identity, set[Identity]] = {}
for node in parent:
comps.setdefault(find(node), set()).add(node)
return tuple(
frozenset(members) for members in comps.values() if len(members) > 1
)
def equivalent_to(self, identity: Identity) -> frozenset[Identity]:
"""The equivalence group containing ``identity`` (including itself), else just itself."""
for group in self.groups():
if identity in group:
return group
return frozenset({identity})
# -- internals -----------------------------------------------------------
def _make_entry(self, page: Page) -> _Entry:
shingle_set = shingles(page.body)
signature = self._hasher.signature(shingle_set)
return _Entry(
shingle_set=shingle_set,
bands=band_keys(signature, self.num_bands),
title=normalized_title(page.identity.key),
fingerprint=_fingerprint(page.body),
)
def _candidates(self, identity: Identity, entry: _Entry) -> set[Identity]:
candidates: set[Identity] = set()
for key in entry.bands:
candidates |= self._band_buckets.get(key, set())
candidates |= self._title_buckets.get(entry.title, set())
candidates.discard(identity)
return candidates
def _verify(self, a: Identity, b: Identity) -> str | None:
ea, eb = self._entries[a], self._entries[b]
if ea.fingerprint == eb.fingerprint:
return "fingerprint"
if jaccard(ea.shingle_set, eb.shingle_set) >= self.threshold:
return "content"
return None
@staticmethod
def _discard_bucket(buckets: dict, key, identity: Identity) -> None:
bucket = buckets.get(key)
if bucket is not None:
bucket.discard(identity)
if not bucket:
del buckets[key]

View File

@@ -0,0 +1,71 @@
"""MinHash + LSH banding primitives for content-similarity blocking (SHARD-WP-0011 T1).
Pure, deterministic functions (fixed hashing, no per-run randomness) so the derived tier and its
digest are reproducible. Shingle a body into k-grams, MinHash the shingle set into a signature,
split the signature into LSH bands; two pages sharing a band are *candidates* for equivalence —
the cheap pre-filter that replaces pairwise O(N²) comparison.
"""
from __future__ import annotations
import hashlib
import random
import re
from collections.abc import Iterable
__all__ = ["shingles", "MinHasher", "band_keys", "jaccard"]
_WORD_RE = re.compile(r"\w+")
# Largest Mersenne prime below 2**61 — the modulus for the universal-hash permutations.
_PRIME = (1 << 61) - 1
def shingles(text: str, k: int = 3) -> frozenset[str]:
"""The set of word k-grams in ``text`` (lowercased). Short texts fall back to their word set."""
words = _WORD_RE.findall(text.lower())
if len(words) < k:
return frozenset(words)
return frozenset(" ".join(words[i : i + k]) for i in range(len(words) - k + 1))
def _stable_hash(token: str) -> int:
return int.from_bytes(hashlib.blake2b(token.encode("utf-8"), digest_size=8).digest(), "big")
class MinHasher:
"""A bank of ``num_perm`` universal hash permutations producing a fixed-length signature."""
def __init__(self, num_perm: int = 64, seed: int = 1) -> None:
self.num_perm = num_perm
rng = random.Random(seed)
self._coeffs = [
(rng.randrange(1, _PRIME), rng.randrange(0, _PRIME)) for _ in range(num_perm)
]
def signature(self, shingle_set: Iterable[str]) -> tuple[int, ...]:
"""The MinHash signature of ``shingle_set`` (empty set → all-``_PRIME`` sentinel)."""
hashed = [_stable_hash(s) for s in shingle_set]
if not hashed:
return tuple(_PRIME for _ in self._coeffs)
return tuple(min((a * h + b) % _PRIME for h in hashed) for a, b in self._coeffs)
def band_keys(
signature: tuple[int, ...], num_bands: int
) -> tuple[tuple[int, tuple[int, ...]], ...]:
"""Split a signature into ``num_bands`` band keys; two pages sharing one are LSH candidates."""
if num_bands <= 0 or len(signature) % num_bands != 0:
raise ValueError(f"signature length {len(signature)} not divisible into {num_bands} bands")
rows = len(signature) // num_bands
return tuple(
(b, signature[b * rows : (b + 1) * rows]) for b in range(num_bands)
)
def jaccard(a: frozenset[str], b: frozenset[str]) -> float:
"""Jaccard similarity of two shingle sets; two empty sets are defined as identical (1.0)."""
if not a and not b:
return 1.0
if not a or not b:
return 0.0
return len(a & b) / len(a | b)

View File

@@ -0,0 +1,91 @@
"""UnionIndex — the maintained derived tier wired behind resolution + views (SHARD-WP-0011 T4).
Wraps a :class:`UnionGraph` + decision log with an incrementally maintained
:class:`EquivalenceIndex`. Content equivalence is kept fresh by deltas (``note_change`` /
``note_removed``); curator bindings are re-synced live from the log fold. A full :meth:`rebuild`
is the bounded fallback. :meth:`verify` runs the I-2 consistency-checker over the live source.
Consumer-visible results are unchanged — equivalence groups are exposed in the same string form the
decision-log fold uses, a *superset* that additionally collapses genuine content duplicates — only
freshness and cost differ (recompute-on-read becomes change-driven).
"""
from __future__ import annotations
from shard_wiki.coordination import DecisionLog
from shard_wiki.incremental.equivalence import EquivalenceIndex
from shard_wiki.incremental.verification import (
ConsistencyChecker,
ConsistencyReport,
derived_digest,
)
from shard_wiki.model import Identity, Page
from shard_wiki.union import UnionGraph
__all__ = ["UnionIndex"]
def _identity(token: str) -> Identity:
shard, _, key = token.partition(":")
return Identity(shard, key)
class UnionIndex:
"""An incrementally maintained equivalence index over a union, with a rebuild fallback."""
def __init__(self, union: UnionGraph, log: DecisionLog, space: str) -> None:
self._union = union
self._log = log
self._space = space
self._eq = EquivalenceIndex()
self.rebuild()
def rebuild(self) -> None:
"""The bounded fallback: re-derive the whole index from current union pages + bindings."""
self._eq.build(self._union.iter_pages())
self._sync_curator()
def note_change(self, page: Page) -> None:
"""Change-driven update for one added/edited page (the operational path)."""
self._eq.update(page)
def note_removed(self, identity: Identity) -> None:
self._eq.remove(identity)
def _sync_curator(self) -> None:
"""Re-sync curator equivalence from the live decision-log fold (cheap, always correct)."""
groups = self._log.fold(self._space).equivalence_groups
edges: list[tuple[Identity, Identity]] = []
for group in groups:
members = [_identity(m) for m in group]
edges.extend((members[0], other) for other in members[1:])
self._eq.set_curator_edges(edges)
def equivalence_groups(self) -> tuple[frozenset[str], ...]:
"""Equivalence groups in decision-log string form (curator content), for the views."""
self._sync_curator()
return tuple(
frozenset(str(identity) for identity in group) for group in self._eq.groups()
)
def digest(self) -> str:
"""The Merkle-style digest of the maintained derived tier (I-2)."""
self._sync_curator()
return derived_digest(self._eq)
def verify(self) -> ConsistencyReport:
"""Check the maintained index against a from-scratch fold of the live source; self-heal."""
self._sync_curator()
checker = ConsistencyChecker(
self._eq,
pages=lambda: list(self._union.iter_pages()),
curator_edges=self._curator_pairs,
)
return checker.check_and_repair()
def _curator_pairs(self) -> list[tuple[Identity, Identity]]:
pairs: list[tuple[Identity, Identity]] = []
for group in self._log.fold(self._space).equivalence_groups:
members = [_identity(m) for m in group]
pairs.extend((members[0], other) for other in members[1:])
return pairs

View File

@@ -0,0 +1,112 @@
"""I-2 verification — digest + background consistency-checker (SHARD-WP-0011 T3).
``derived = f(canonical)`` is made *verified*, not asserted. A **Merkle-style digest** summarizes
the derived tier (each identity's content fingerprint + its incident equivalence edges as a leaf,
order-independently combined into a root) so two derived states are equal iff their digests match.
A **consistency-checker** recomputes the authoritative fold from the current source, compares it to
the maintained index over a (sampled) region, and on mismatch performs a **scoped recompute** of
just the affected identities — self-healing drift from a missed delta or corrupted state.
The digest is a pure function of index state, so it is "maintained alongside deltas" for free and
is stable under equivalent event orders (leaves are sorted before combination).
"""
from __future__ import annotations
import hashlib
from collections.abc import Callable, Iterable
from dataclasses import dataclass
from shard_wiki.incremental.equivalence import EquivalenceIndex
from shard_wiki.model import Identity, Page
__all__ = ["region_digest", "derived_digest", "ConsistencyReport", "ConsistencyChecker"]
CuratorEdges = Iterable[tuple[Identity, Identity]]
def _leaf(index: EquivalenceIndex, identity: Identity) -> str:
"""A digest leaf for one identity: its fingerprint + its incident edges (as sorted peers)."""
fingerprint = index.fingerprint(identity) or ""
peers = sorted(
str(other)
for edge in index.edges()
if identity in edge
for other in edge
if other != identity
)
payload = f"{identity}|{fingerprint}|{','.join(peers)}"
return hashlib.blake2b(payload.encode("utf-8"), digest_size=16).hexdigest()
def region_digest(index: EquivalenceIndex, identities: Iterable[Identity]) -> str:
"""A Merkle-style root over the given identities' leaves (order-independent)."""
leaves = sorted(_leaf(index, identity) for identity in identities)
root = hashlib.blake2b(digest_size=16)
for leaf in leaves:
root.update(leaf.encode("utf-8"))
return root.hexdigest()
def derived_digest(index: EquivalenceIndex) -> str:
"""The digest of the whole maintained derived tier."""
return region_digest(index, index.identities())
@dataclass(frozen=True, slots=True)
class ConsistencyReport:
"""Outcome of a consistency check: what was examined, whether it drifted, and if it healed."""
checked: int
drifted: bool
repaired: bool
healthy: bool
class ConsistencyChecker:
"""Compares the maintained index against an authoritative rebuild and repairs drift in place."""
def __init__(
self,
index: EquivalenceIndex,
pages: Callable[[], Iterable[Page]],
curator_edges: Callable[[], CuratorEdges] = lambda: (),
) -> None:
self._index = index
self._pages = pages
self._curator = curator_edges
def _authoritative(self) -> EquivalenceIndex:
expected = EquivalenceIndex(
num_bands=self._index.num_bands, threshold=self._index.threshold
)
expected.build(list(self._pages()), list(self._curator()))
return expected
def check_and_repair(self, sample: Iterable[Identity] | None = None) -> ConsistencyReport:
"""Verify the (sampled) region against a from-scratch fold; scoped-recompute on mismatch."""
source = {p.identity: p for p in self._pages()}
expected = self._authoritative()
region = (
set(sample)
if sample is not None
else set(source) | set(self._index.identities())
)
drifted = region_digest(self._index, region) != region_digest(expected, region)
if not drifted:
return ConsistencyReport(len(region), drifted=False, repaired=False, healthy=True)
self._repair(region, source)
healthy = region_digest(self._index, region) == region_digest(expected, region)
return ConsistencyReport(len(region), drifted=True, repaired=True, healthy=healthy)
def _repair(self, region: set[Identity], source: dict[Identity, Page]) -> None:
"""Scoped recompute: reconcile each affected identity to the current source."""
present = self._index.identities()
for identity in region:
page = source.get(identity)
if page is not None:
self._index.update(page) if identity in present else self._index.add(page)
elif identity in present:
self._index.remove(identity)

View File

@@ -8,32 +8,69 @@ a network API is a later workplan.
from __future__ import annotations
from pathlib import Path
from shard_wiki.adapters import ShardAdapter, assert_conformant
from shard_wiki.coordination import (
ApplyResult,
DecisionLog,
EventStore,
EventType,
GitEventStore,
Overlay,
OverlayEngine,
)
from shard_wiki.incremental import ConsistencyReport, UnionIndex
from shard_wiki.model import Page
from shard_wiki.policy import DEFAULT_POLICY, Policy
from shard_wiki.union import Resolution, UnionGraph
from shard_wiki.views import (
AllPagesEntry,
BackLink,
ChangeEntry,
SiteMapNode,
all_pages,
build_backlinks,
recent_changes,
site_map,
)
__all__ = ["InformationSpace"]
class InformationSpace:
def __init__(self, space_id: str, policy: Policy = DEFAULT_POLICY) -> None:
def __init__(
self,
space_id: str,
policy: Policy = DEFAULT_POLICY,
*,
store: EventStore | None = None,
) -> None:
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
self.space_id = space_id
self.log = DecisionLog()
self.log = DecisionLog(store)
self.union = UnionGraph(space_id, log=self.log, policy=policy)
self.overlays = OverlayEngine(space_id, self.log)
self._index: UnionIndex | None = None # maintained derived tier, built lazily
self._index_stale = True
@classmethod
def git_backed(
cls,
space_id: str,
repo_path: str | Path,
policy: Policy = DEFAULT_POLICY,
) -> InformationSpace:
"""An information space whose coordination log is git-addressable (history/patch/review/
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
return cls(space_id, policy, store=GitEventStore(repo_path))
def attach(self, adapter: ShardAdapter) -> None:
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
assert_conformant(adapter)
self.union.attach(adapter)
self._index_stale = True
def alias(self, name: str, target: str, actor: str | None = None) -> None:
"""Record a coordination-canonical alias (``name`` → ``"shard:key"``) in the log."""
@@ -68,4 +105,44 @@ class InformationSpace:
write-through-capable target fast-forwards (write-through); a read-only target keeps the
draft as local truth (I-5: overlay before mutation, always)."""
overlay = self.overlay(name, body, actor=actor)
return self.apply_overlay(overlay.overlay_id)
result = self.apply_overlay(overlay.overlay_id)
self._index_stale = True # the applied edit changes the derived tier
return result
# --- maintained derived tier (SHARD-WP-0011): incremental-first, rebuild as fallback ---
@property
def index(self) -> UnionIndex:
"""The maintained equivalence index (built lazily; rebuilt when the union has changed)."""
if self._index is None:
self._index = UnionIndex(self.union, self.log, self.space_id)
elif self._index_stale:
self._index.rebuild() # bounded fallback after a mutation
self._index_stale = False
return self._index
def reindex(self) -> None:
"""Force a full rebuild of the maintained derived tier (the explicit fallback path)."""
self.index.rebuild()
def verify_index(self) -> ConsistencyReport:
"""Run the I-2 consistency-checker over the maintained tier; self-heal any drift."""
return self.index.verify()
# --- derived views (SHARD-WP-0010): recomputable, provenance-carrying, presentation-free ---
def backlinks(self, name: str, *, camelcase: bool = False) -> tuple[BackLink, ...]:
"""Pages across the union that link to ``name`` (UC-18)."""
return build_backlinks(self.union, camelcase=camelcase).to(name)
def recent_changes(self, *, limit: int | None = None) -> tuple[ChangeEntry, ...]:
"""The merged newest-first change feed: coordination journal + shard signals (UC-17)."""
return recent_changes(self.union, self.log, self.space_id, limit=limit)
def all_pages(self) -> tuple[AllPagesEntry, ...]:
"""The union's distinct pages, collapsed via the maintained equivalence index."""
return all_pages(self.union, equivalence_groups=self.index.equivalence_groups())
def site_map(self) -> SiteMapNode:
"""The union namespace tree built from page placements."""
return site_map(self.union)

View File

@@ -13,6 +13,7 @@ imported by nothing.
from __future__ import annotations
import dataclasses
from collections.abc import Iterator
from dataclasses import dataclass
from enum import Enum
@@ -68,6 +69,20 @@ class UnionGraph:
def shard(self, shard_id: str) -> ShardAdapter | None:
return next((s for s in self._shards if s.shard_id == shard_id), None)
@property
def shards(self) -> tuple[ShardAdapter, ...]:
return tuple(self._shards)
def iter_pages(self) -> Iterator[Page]:
"""Every page across attached shards, raw (per-shard, not chorus-collapsed). The
enumeration substrate for derived views — BackLinks, AllPages, SiteMap (§8.4)."""
for shard in self._shards:
for key in shard.keys():
try:
yield shard.read(key)
except KeyError:
continue
def _read_all(self, key: str) -> list[Page]:
pages: list[Page] = []
for shard in self._shards:

View File

@@ -0,0 +1,33 @@
"""views/ — derived, recomputable, provenance-carrying read views over the union (§8.4).
All views here are *derived tier*: pure functions of the attached shards plus the coordination-log
fold, storing nothing canonical (SHARD-WP-0011 makes them incrementally maintainable). Presentation
stays out of core (L6) — these produce models, never rendered output. Per the dependency rule this
package imports down (union/model/coordination/provenance) and is imported only by the orchestrator.
"""
from shard_wiki.views.allpages import AllPagesEntry, SiteMapNode, all_pages, site_map
from shard_wiki.views.backlinks import BackLink, BackLinksIndex, build_backlinks
from shard_wiki.views.links import (
ResolvedLink,
WikiLink,
extract_links,
resolve_links,
)
from shard_wiki.views.recentchanges import ChangeEntry, recent_changes
__all__ = [
"WikiLink",
"ResolvedLink",
"extract_links",
"resolve_links",
"BackLink",
"BackLinksIndex",
"build_backlinks",
"ChangeEntry",
"recent_changes",
"AllPagesEntry",
"SiteMapNode",
"all_pages",
"site_map",
]

View File

@@ -0,0 +1,131 @@
"""AllPages + SiteMap — enumeration views over the union (SHARD-WP-0010 T4).
**AllPages** lists the union's distinct pages, collapsing identities that name the same page: a
*chorus* (same key across shards) and *equivalence-bound* identities (decision-log bindings) fold
into one entry, with divergence noted when the members' bodies differ (union without erasure — the
collapse is acknowledged, never silent). **SiteMap** is the namespace tree built from page
placements (paths), spanning shards.
Both are derived/recomputable and presentation-free (the tree is a model, not rendered HTML).
"""
from __future__ import annotations
from dataclasses import dataclass
from shard_wiki.model import Identity, Page
from shard_wiki.union import UnionGraph
__all__ = ["AllPagesEntry", "SiteMapNode", "all_pages", "site_map"]
@dataclass(frozen=True, slots=True)
class AllPagesEntry:
"""One union page: its representative ``name``, the ``members`` collapsed into it, and whether
those members' bodies ``diverge`` (a chorus with differing content)."""
name: str
members: tuple[Identity, ...]
diverges: bool
@dataclass(frozen=True, slots=True)
class SiteMapNode:
"""A namespace node: its path ``name``, child namespaces, and pages directly under it."""
name: str
children: tuple[SiteMapNode, ...]
pages: tuple[Identity, ...]
class _UnionFind:
def __init__(self) -> None:
self._parent: dict[str, str] = {}
def add(self, x: str) -> None:
self._parent.setdefault(x, x)
def find(self, x: str) -> str:
self.add(x)
root = x
while self._parent[root] != root:
root = self._parent[root]
while self._parent[x] != root:
self._parent[x], x = root, self._parent[x]
return root
def union(self, a: str, b: str) -> None:
self.add(a)
self.add(b)
ra, rb = self.find(a), self.find(b)
if ra != rb:
self._parent[max(ra, rb)] = min(ra, rb)
def all_pages(
union: UnionGraph,
equivalence_groups: tuple[frozenset[str], ...] | None = None,
) -> tuple[AllPagesEntry, ...]:
"""Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members.
``equivalence_groups`` (string identities, decision-log form) overrides the source of
equivalence — the orchestrator passes the maintained index's groups (SHARD-WP-0011 T4); the
default falls back to the decision-log fold, so direct callers are unaffected.
"""
pages: dict[str, Page] = {}
by_key: dict[str, list[str]] = {}
for page in union.iter_pages():
ident = str(page.identity)
pages[ident] = page
by_key.setdefault(page.identity.key, []).append(ident)
uf = _UnionFind()
for ident in pages:
uf.add(ident)
for idents in by_key.values(): # same key across shards → chorus
for other in idents[1:]:
uf.union(idents[0], other)
if equivalence_groups is None:
equivalence_groups = union.log.fold(union.space).equivalence_groups
for group in equivalence_groups: # curator bindings (+ maintained content edges)
present = [m for m in group if m in pages]
for other in present[1:]:
uf.union(present[0], other)
groups: dict[str, list[str]] = {}
for ident in pages:
groups.setdefault(uf.find(ident), []).append(ident)
entries: list[AllPagesEntry] = []
for members in groups.values():
member_pages = [pages[m] for m in members]
identities = tuple(p.identity for p in member_pages)
name = min(p.identity.key for p in member_pages)
diverges = len({p.body for p in member_pages}) > 1
entries.append(AllPagesEntry(name=name, members=identities, diverges=diverges))
return tuple(sorted(entries, key=lambda e: e.name))
def _segments(page: Page) -> list[str]:
path = page.placements[0].path if page.placements else page.identity.key
if path.endswith(".md"):
path = path[:-3]
return [seg for seg in path.split("/") if seg]
def site_map(union: UnionGraph) -> SiteMapNode:
"""The union namespace tree from page placements (directories nest; pages sit at their dir)."""
root: dict = {"children": {}, "pages": []}
for page in union.iter_pages():
segments = _segments(page)
node = root
for seg in segments[:-1]: # directory segments build the nesting
node = node["children"].setdefault(seg, {"children": {}, "pages": []})
node["pages"].append(page.identity)
return _freeze("", root)
def _freeze(name: str, node: dict) -> SiteMapNode:
children = tuple(_freeze(k, v) for k, v in sorted(node["children"].items()))
pages = tuple(sorted(node["pages"], key=str))
return SiteMapNode(name=name, children=children, pages=pages)

View File

@@ -0,0 +1,65 @@
"""BackLinks — the strongest core derived view (SHARD-WP-0010 T2; UC-18).
For any page name, the set of pages that link to it. Built by extracting wikilinks (T1) from every
page across the attached shards and resolving each through the union: only **resolved** links
create a backlink (a red-link points at nothing, so it contributes none). Entries carry their
**source provenance** (the linking page's identity / shard). Keying by the resolved *name* means a
chorus target aggregates the backlinks of all its members into one bucket (union without erasure).
Derived/recomputable — stores nothing canonical; SHARD-WP-0011 maintains it incrementally.
"""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from shard_wiki.model import Identity
from shard_wiki.union import UnionGraph
from shard_wiki.views.links import resolve_links
__all__ = ["BackLink", "BackLinksIndex", "build_backlinks"]
@dataclass(frozen=True, slots=True)
class BackLink:
"""One inbound link: ``source`` (the linking page) references ``target_name``."""
source: Identity
target_name: str
@property
def source_shard(self) -> str:
return self.source.shard
class BackLinksIndex:
"""An immutable name → inbound-links index over the union link graph."""
def __init__(self, edges: Mapping[str, tuple[BackLink, ...]]) -> None:
self._edges = dict(edges)
def to(self, name: str) -> tuple[BackLink, ...]:
"""The backlinks pointing at ``name`` (empty if none)."""
return self._edges.get(name, ())
def sources(self, name: str) -> frozenset[Identity]:
"""Just the identities linking to ``name`` — convenient for set assertions."""
return frozenset(bl.source for bl in self.to(name))
def names(self) -> frozenset[str]:
return frozenset(self._edges)
def build_backlinks(union: UnionGraph, *, camelcase: bool = False) -> BackLinksIndex:
"""Scan every union page's links and index the resolved ones by target name."""
edges: dict[str, set[BackLink]] = {}
for page in union.iter_pages():
for resolved in resolve_links(union, page.body, camelcase=camelcase):
if resolved.is_red_link:
continue # red-links don't create backlinks
backlink = BackLink(source=page.identity, target_name=resolved.link.target)
edges.setdefault(resolved.link.target, set()).add(backlink)
return BackLinksIndex(
{name: tuple(sorted(links, key=lambda bl: str(bl.source))) for name, links in edges.items()}
)

View File

@@ -0,0 +1,91 @@
"""Wikilink + red-link model (SHARD-WP-0010 T1; FederationRequirements ADR-06).
A CommonMark *wikilink extension*: ``[[Target]]`` and ``[[Target|label]]`` are extracted from a
page body and each target is resolved through the union (ADR-01). A target that resolves is a
**link**; one that does not is a **red-link** — a createable hole (UC-23), never a dropped
reference (union without erasure). CamelCase auto-linking (``WikiWord``) is **off by default** and
opt-in per space, since bare CamelCase is noisy and policy-laden.
The link *model and resolution* are core; turning a :class:`ResolvedLink` into an ``<a>`` (or a
red anchor) is L6 presentation and lives outside this package. Link spans are byte/char offsets in
the body so a later layer can address them precisely.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from shard_wiki.union import Resolution, UnionGraph
__all__ = ["WikiLink", "ResolvedLink", "extract_links", "resolve_links"]
_WIKILINK_RE = re.compile(r"\[\[\s*([^\]|]+?)\s*(?:\|\s*([^\]]+?)\s*)?\]\]")
# A WikiWord: ≥2 capitalized alphanumeric segments run together (e.g. FrontPage, WikiWord).
_CAMELCASE_RE = re.compile(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b")
_FENCED_RE = re.compile(r"```.*?```", re.DOTALL)
_INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
@dataclass(frozen=True, slots=True)
class WikiLink:
"""One extracted reference. ``target`` is the resolve key; ``label`` is the display text (or
None to use the target); ``span`` is the ``[start, end)`` offset of the whole token in the body;
``auto`` marks a CamelCase auto-link (vs an explicit ``[[...]]``)."""
target: str
label: str | None
span: tuple[int, int]
auto: bool = False
@property
def text(self) -> str:
return self.label or self.target
@dataclass(frozen=True, slots=True)
class ResolvedLink:
"""A :class:`WikiLink` paired with its union :class:`Resolution` (the link's truth status)."""
link: WikiLink
resolution: Resolution
@property
def is_red_link(self) -> bool:
return self.resolution.is_red_link
def _mask(body: str, pattern: re.Pattern[str]) -> str:
"""Blank out ``pattern`` matches with equal-length spaces so later scans skip them while every
surviving match keeps its true offset."""
return pattern.sub(lambda m: " " * len(m.group(0)), body)
def extract_links(body: str, *, camelcase: bool = False) -> tuple[WikiLink, ...]:
"""Extract wikilinks from ``body`` in document order, skipping fenced/inline code.
With ``camelcase=True`` (per-space opt-in), bare ``WikiWord`` tokens outside code and outside
existing ``[[...]]`` also become links.
"""
scan = _mask(_mask(body, _FENCED_RE), _INLINE_CODE_RE)
links: list[WikiLink] = []
for m in _WIKILINK_RE.finditer(scan):
links.append(WikiLink(target=m.group(1).strip(), label=m.group(2), span=m.span()))
if camelcase:
# Mask explicit-link spans too, so a CamelCase target inside [[...]] isn't double-counted.
cc_scan = _mask(scan, _WIKILINK_RE)
for m in _CAMELCASE_RE.finditer(cc_scan):
links.append(WikiLink(target=m.group(1), label=None, span=m.span(), auto=True))
return tuple(sorted(links, key=lambda link: link.span[0]))
def resolve_links(
union: UnionGraph, body: str, *, camelcase: bool = False
) -> tuple[ResolvedLink, ...]:
"""Extract and resolve every link in ``body`` against ``union`` (link vs red-link, ADR-01)."""
return tuple(
ResolvedLink(link, union.resolve(link.target))
for link in extract_links(body, camelcase=camelcase)
)

View File

@@ -0,0 +1,108 @@
"""RecentChanges — a merged change feed over the union (SHARD-WP-0010 T3; UC-17).
Two streams, one ordered feed (newest-first):
* the **coordination journal** — overlay/alias/fork/merge/binding decisions from the decision log,
each carrying its actor and the decision payload; and
* **shard change signals** — a page's current revision (folder mtime / ``source_rev``), i.e. the
backend's own "this changed" evidence.
Every entry carries provenance: which shard the edit came from, or that it was a coordination
decision (and by whom). Derived/recomputable — `notify`-driven streaming is a later binding.
"""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass, field
from datetime import datetime
from shard_wiki.coordination import DecisionLog, EventType
from shard_wiki.union import UnionGraph
__all__ = ["ChangeEntry", "recent_changes"]
_COORDINATION = "coordination"
# How each journal event names the thing it touched + a human kind label.
_EVENT_KIND = {
EventType.ALIAS_SET: "alias",
EventType.OVERLAY_CREATED: "overlay",
EventType.MERGE_DECIDED: "merge",
EventType.PAGE_FORKED: "fork",
EventType.BINDING_MADE: "binding",
}
@dataclass(frozen=True, slots=True)
class ChangeEntry:
"""One change in the feed. ``source`` is the shard id (a shard edit) or ``"coordination"``."""
when: datetime
kind: str
ref: str
source: str
actor: str | None = None
detail: Mapping[str, object] = field(default_factory=dict)
def _event_ref(event_type: EventType, payload: Mapping[str, object]) -> str:
if event_type is EventType.ALIAS_SET:
return str(payload.get("alias", ""))
if event_type is EventType.OVERLAY_CREATED:
return f"{payload.get('target_shard')}:{payload.get('target_key')}"
if event_type is EventType.PAGE_FORKED:
return f"{payload.get('source')}{payload.get('fork')}"
if event_type is EventType.BINDING_MADE:
return ", ".join(str(m) for m in payload.get("members", ()))
return str(payload.get("overlay_id", "")) # MERGE_DECIDED
def recent_changes(
union: UnionGraph,
log: DecisionLog,
space: str,
*,
limit: int | None = None,
) -> tuple[ChangeEntry, ...]:
"""Merge the coordination journal and shard change signals into one newest-first feed."""
entries: list[ChangeEntry] = []
for event in log.events(space):
entries.append(
ChangeEntry(
when=event.timestamp,
kind=_EVENT_KIND.get(event.type, event.type.value),
ref=_event_ref(event.type, event.payload),
source=_COORDINATION,
actor=event.actor,
detail=dict(event.payload),
)
)
for page in union.iter_pages():
rev = page.envelope.source_rev
when = _parse_rev(rev)
if when is None:
continue # shard offers no change signal for this page — skip gracefully
entries.append(
ChangeEntry(
when=when,
kind="edit",
ref=str(page.identity),
source=page.identity.shard,
detail={"source_rev": rev},
)
)
entries.sort(key=lambda e: e.when, reverse=True)
return tuple(entries if limit is None else entries[:limit])
def _parse_rev(rev: str | None) -> datetime | None:
if rev is None:
return None
try:
return datetime.fromisoformat(rev)
except ValueError:
return None # non-temporal revision token (e.g. a content hash) — no feed timestamp

View File

@@ -0,0 +1,120 @@
"""Tests for the per-space append authority / lease (SHARD-WP-0009 T2).
A single append authority per space serializes appends into a total order; non-holders forward
intents to the holder; the lease is time-bounded and re-grantable (HA hand-off); a stale ex-holder
cannot fork the log.
"""
from datetime import datetime, timedelta, timezone
import pytest
from shard_wiki.coordination import (
AppendAuthority,
EventType,
GitEventStore,
InMemoryEventStore,
LeaseHeld,
LeaseRegistry,
)
class FakeClock:
def __init__(self):
self.now = datetime(2026, 1, 1, tzinfo=timezone.utc)
def __call__(self):
return self.now
def advance(self, seconds):
self.now += timedelta(seconds=seconds)
def test_only_one_node_holds_a_space_at_a_time():
reg = LeaseRegistry()
a = AppendAuthority("A", InMemoryEventStore(), reg)
b = AppendAuthority("B", InMemoryEventStore(), reg)
a.acquire("s")
with pytest.raises(LeaseHeld):
b.acquire("s") # B is refused while A's lease is valid
def test_concurrent_appends_serialize_into_one_total_order():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
# B is a non-holder: its append forwards to A, the holder. Interleave A and B writers.
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"}) # forwarded
a.append("s", EventType.ALIAS_SET, {"alias": "3", "target": "x:3"})
seqs = [e.seq for e in store.events("s")]
aliases = [e.payload["alias"] for e in store.events("s")]
assert seqs == [0, 1, 2] # contiguous total order despite two writers
assert aliases == ["1", "2", "3"]
def test_non_holder_forwards_rather_than_writing_directly():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
assert not b.holds("s")
b.append("s", EventType.ALIAS_SET, {"alias": "fwd", "target": "x:1"})
# The write landed on the shared store under A's authority, in one stream.
assert [e.payload["alias"] for e in store.events("s")] == ["fwd"]
def test_lease_handoff_resumes_from_head():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
clock.advance(20) # A's lease expires (A "dies")
b.acquire("s") # re-grantable: B takes over
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
assert [e.seq for e in store.events("s")] == [0, 1, 2] # contiguous across hand-off
def test_stale_ex_holder_cannot_fork_the_log():
clock = FakeClock()
reg = LeaseRegistry(clock=clock)
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg, ttl_seconds=10)
b = AppendAuthority("B", store, reg, ttl_seconds=10)
a.acquire("s")
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
clock.advance(20)
b.acquire("s") # B is now the holder; A's lease is stale
b.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
# A still thinks it can write, but it's no longer the holder: its intent forwards to B.
assert not a.holds("s")
a.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
aliases = [e.payload["alias"] for e in store.events("s")]
assert aliases == ["0", "1", "2"] # one stream, no fork
def test_authority_over_git_store_keeps_total_order(tmp_path):
reg = LeaseRegistry()
store = GitEventStore(tmp_path / "coord")
a = AppendAuthority("A", store, reg)
b = AppendAuthority("B", store, reg)
a.acquire("s")
a.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
b.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"}) # forwarded
assert [e.seq for e in store.events("s")] == [0, 1]
def test_unleased_space_self_acquires_on_append():
reg = LeaseRegistry()
store = InMemoryEventStore()
a = AppendAuthority("A", store, reg)
a.append("s", EventType.ALIAS_SET, {"alias": "x", "target": "y:1"}) # no explicit acquire
assert a.holds("s")
assert len(store.events("s")) == 1

View File

@@ -0,0 +1,74 @@
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
export_jsonl,
import_jsonl,
migrate_space,
)
from shard_wiki.space import InformationSpace
def test_information_space_git_backed_uses_git_log(tmp_path):
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
assert isinstance(space.log._store, GitEventStore)
space.alias("Home", "shardA:Index")
# Read-your-writes through the orchestrator's git-backed log.
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
def test_default_information_space_stays_in_memory():
space = InformationSpace("space-1")
assert isinstance(space.log._store, InMemoryEventStore)
def test_migrate_space_preserves_order_and_provenance(tmp_path):
source = InMemoryEventStore()
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
dest = GitEventStore(tmp_path / "coord")
n = migrate_space(source, "s", dest)
assert n == 2
migrated = dest.events("s")
assert [e.seq for e in migrated] == [0, 1]
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
assert migrated[0].actor == "ana"
assert migrated[1].actor == "ben"
assert migrated[0].timestamp == e0.timestamp
def test_migration_yields_identical_fold(tmp_path):
source = DecisionLog(InMemoryEventStore())
for typ, payload in [
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
]:
source.append("s", typ, payload)
dest = GitEventStore(tmp_path / "coord")
migrate_space(source._store, "s", dest)
after = DecisionLog(dest)
assert after.fold("s").aliases == source.fold("s").aliases
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
def test_jsonl_round_trip_into_git(tmp_path):
source = InMemoryEventStore()
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
path = tmp_path / "log.jsonl"
assert export_jsonl(source.events("s"), path) == 2
dest = GitEventStore(tmp_path / "coord")
assert import_jsonl(path, dest) == 2
state = DecisionLog(dest).fold("s")
assert state.resolve_alias("Home") == "x:1"
assert state.equivalent_to("p") == frozenset({"p", "q"})

View File

@@ -0,0 +1,83 @@
"""Cross-process read-your-writes over the git log + fold parity (SHARD-WP-0009 T3).
The git backend's value over the in-memory double is that the totally ordered log is durable and
shared: a write by one process/handle is immediately visible to another opening the same ref, and
the derived fold is identical to the in-memory fold of the same event sequence (derived = f(log)).
"""
import os
import subprocess
import sys
import textwrap
from pathlib import Path
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
)
_SRC = str(Path(__file__).resolve().parents[1] / "src")
def test_new_handle_sees_prior_writes(tmp_path):
repo = tmp_path / "coord"
writer = DecisionLog(GitEventStore(repo))
writer.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
writer.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
# A second, independent handle on the same repo — read-your-writes across handles.
reader = DecisionLog(GitEventStore(repo))
assert [e.seq for e in reader.events("s")] == [0, 1]
assert reader.fold("s").resolve_alias("Home") == "shardA:Index"
def test_append_in_separate_process_is_visible(tmp_path):
repo = tmp_path / "coord"
# Seed from this process so the repo exists.
DecisionLog(GitEventStore(repo)).append(
"s", EventType.ALIAS_SET, {"alias": "A", "target": "x:1"}
)
child = textwrap.dedent(
f"""
from shard_wiki.coordination import DecisionLog, EventType, GitEventStore
log = DecisionLog(GitEventStore({str(repo)!r}))
log.append("s", EventType.ALIAS_SET, {{"alias": "B", "target": "x:2"}})
"""
)
result = subprocess.run(
[sys.executable, "-c", child],
capture_output=True,
text=True,
env={"PYTHONPATH": _SRC, "PATH": os.environ.get("PATH", "")},
)
assert result.returncode == 0, result.stderr
# This process, with a fresh handle, sees the child's append in order.
reader = DecisionLog(GitEventStore(repo))
assert [e.payload["alias"] for e in reader.events("s")] == ["A", "B"]
assert [e.seq for e in reader.events("s")] == [0, 1]
def test_cross_process_fold_equals_in_memory_fold(tmp_path):
sequence = [
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.PAGE_FORKED, {"source": "p", "fork": "q"}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
]
mem = DecisionLog(InMemoryEventStore())
for typ, payload in sequence:
mem.append("s", typ, payload)
repo = tmp_path / "coord"
DecisionLog(GitEventStore(repo)) # init repo
for typ, payload in sequence:
# Each append from a fresh handle to simulate distinct writers over time.
DecisionLog(GitEventStore(repo)).append("s", typ, payload)
git_state = DecisionLog(GitEventStore(repo)).fold("s")
mem_state = mem.fold("s")
assert git_state.aliases == mem_state.aliases
assert git_state.equivalence_groups == mem_state.equivalence_groups
assert git_state.equivalent_to("a") == frozenset({"a", "b", "c"})

View File

@@ -0,0 +1,61 @@
"""Tests for per-shard extension activation (SHARD-WP-0014 T3, ADR-0001)."""
from shard_wiki.engine import (
ActivationContext,
ActivationResolver,
StaticProvider,
feature_control_provider,
)
CANDIDATES = ["ext.overlay", "ext.views", "ext.struct", "ext.compute"]
def test_static_provider_default_off():
r = ActivationResolver(StaticProvider()) # nothing enabled
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == set()
def test_static_provider_global_flags():
r = ActivationResolver(StaticProvider(flags={"ext.overlay": True, "ext.views": True}))
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == {"ext.overlay", "ext.views"}
def test_per_shard_scoping_overrides_global():
provider = StaticProvider(
flags={"ext.views": True},
per_shard={"engB": {"ext.struct": True, "ext.views": False}},
)
r = ActivationResolver(provider)
assert r.active_extensions(CANDIDATES, ActivationContext("engA")) == {"ext.views"}
assert r.active_extensions(CANDIDATES, ActivationContext("engB")) == {"ext.struct"}
def test_context_carries_tenant():
captured = {}
class Spy(StaticProvider):
def is_active(self, feature_key, context):
captured.update(context)
return super().is_active(feature_key, context)
ActivationResolver(Spy(flags={"ext.views": True})).active_extensions(
["ext.views"], ActivationContext("s1", tenant_id="acme")
)
assert captured["shard_id"] == "s1" and captured["tenant_id"] == "acme"
def test_activation_profile_returns_config_for_active():
provider = StaticProvider(
flags={"ext.struct": True},
configs={"ext.struct": {"max_fields": 50}},
)
profile = ActivationResolver(provider).activation_profile(CANDIDATES, ActivationContext("s"))
assert profile == {"ext.struct": {"max_fields": 50}}
def test_feature_control_provider_degrades_gracefully():
# feature_control_sdk is not a dependency of shard-wiki: when absent the factory returns
# None (standalone path stays dependency-free, ADR-0001); when present it yields a usable
# ActivationProvider. Either way it must not raise.
provider = feature_control_provider()
assert provider is None or hasattr(provider, "is_active")

View File

@@ -0,0 +1,80 @@
"""Tests for EngineShardAdapter (SHARD-WP-0014 T5): engine as a canonical-mode shard."""
from shard_wiki import InformationSpace
from shard_wiki.adapters import assert_conformant
from shard_wiki.engine import (
Extension,
ExtensionRuntime,
Hook,
ProfileContribution,
StaticProvider,
build_engine_shard,
)
from shard_wiki.engine.activation import ActivationContext
from shard_wiki.model import Verb
class StructProfileExt(Extension):
"""Profile-only extension (no body transform → write stays content-preserving)."""
id = "ext.struct"
def hooks(self):
return {
Hook.ON_PROFILE: lambda p, c: ProfileContribution(
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
)
}
def _runtime():
rt = ExtensionRuntime()
rt.register(StructProfileExt())
return rt
def test_kernel_only_engine_shard_is_conformant():
shard = build_engine_shard("eng", ExtensionRuntime(), activate=set())
shard.write("Home", "hi")
report = assert_conformant(shard) # read + positive write probe
assert report.ok
assert shard.profile().supports(Verb.WRITE)
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
def test_profile_reflects_activated_extensions():
off = build_engine_shard("a", _runtime(), activate=set())
on = build_engine_shard("b", _runtime(), activate={"ext.struct"})
assert not off.profile().supports(Verb.STRUCTURED_PAYLOAD)
assert on.profile().supports(Verb.STRUCTURED_PAYLOAD) # E-5
assert_conformant(on)
def test_activation_via_provider():
provider = StaticProvider(flags={"ext.struct": True})
shard = build_engine_shard("c", _runtime(), provider=provider, context=ActivationContext("c"))
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
def test_attach_resolve_edit_through_engine_shard(tmp_path):
space = InformationSpace("team")
space.attach(build_engine_shard("wikiE", ExtensionRuntime(), activate=set()))
# seed a page directly via the shard, then read + edit through the orchestrator
space.union.shard("wikiE").write("Home", "v1")
assert space.read("Home").body == "v1"
result = space.edit("Home", "v2") # overlay -> apply-under-drift -> write-through
assert result.status.value == "applied"
assert space.read("Home").body == "v2"
def test_on_write_transform_runs():
class Upper(Extension):
id = "ext.upper"
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
rt = ExtensionRuntime()
rt.register(Upper())
shard = build_engine_shard("u", rt, activate={"ext.upper"})
shard.write("P", "quiet")
assert shard.read("P").body == "QUIET" # extension transformed the write

View File

@@ -0,0 +1,109 @@
"""Tests for the typed-extension runtime (SHARD-WP-0014 T2)."""
import pytest
from shard_wiki.engine import Extension, ExtensionError, ExtensionRuntime, Hook
class Upper(Extension):
id = "ext.upper"
declares_types = ("upper",)
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
class Bang(Extension):
id = "ext.bang"
depends_on = ("ext.upper",)
def hooks(self):
return {Hook.ON_WRITE: lambda body, ctx: body + "!"}
class Profiler(Extension):
id = "ext.profiler"
def hooks(self):
return {Hook.ON_PROFILE: lambda payload, ctx: {"structure": "typed"}}
def _runtime(*exts):
rt = ExtensionRuntime()
for e in exts:
rt.register(e)
return rt
def test_register_rejects_bad_id_and_noncallable_hook():
rt = ExtensionRuntime()
class BadId(Extension):
id = "upper" # missing 'ext.' prefix
with pytest.raises(ExtensionError, match="ext."):
rt.register(BadId())
class Liar(Extension):
id = "ext.liar"
def hooks(self):
return {Hook.ON_WRITE: "not-callable"} # type: ignore[dict-item]
with pytest.raises(ExtensionError, match="not callable"):
rt.register(Liar())
def test_transform_hook_chains_in_dependency_order():
rt = _runtime(Upper(), Bang())
active = rt.activate({"ext.bang"}) # pulls ext.upper via depends_on
assert set(active.ids) == {"ext.upper", "ext.bang"}
# upper (dependency) runs before bang (dependent): "hi" -> "HI" -> "HI!"
assert active.handlers(Hook.ON_WRITE) == ("ext.upper", "ext.bang")
assert active.dispatch_transform(Hook.ON_WRITE, "hi") == "HI!"
def test_dependency_closure_is_automatic():
rt = _runtime(Upper(), Bang())
assert set(rt.activate({"ext.bang"}).ids) == {"ext.upper", "ext.bang"}
def test_unknown_extension_rejected():
with pytest.raises(ExtensionError, match="unknown"):
ExtensionRuntime().activate({"ext.ghost"})
def test_conflict_rejected():
class A(Extension):
id = "ext.a"
conflicts_with = ("ext.b",)
class B(Extension):
id = "ext.b"
with pytest.raises(ExtensionError, match="conflicts"):
_runtime(A(), B()).activate({"ext.a", "ext.b"})
def test_type_collision_rejected():
class S1(Extension):
id = "ext.s1"
declares_types = ("record",)
class S2(Extension):
id = "ext.s2"
declares_types = ("record",)
with pytest.raises(ExtensionError, match="type collision"):
_runtime(S1(), S2()).activate({"ext.s1", "ext.s2"})
def test_collect_hook_gathers_contributions():
active = _runtime(Profiler()).activate({"ext.profiler"})
assert active.dispatch_collect(Hook.ON_PROFILE) == [{"structure": "typed"}]
def test_wrong_dispatch_kind_errors():
active = _runtime(Profiler()).activate({"ext.profiler"})
with pytest.raises(ExtensionError, match="not a transform hook"):
active.dispatch_transform(Hook.ON_PROFILE, "x")
def test_unmet_dependency_rejected():
# Bang depends on ext.upper, but only Bang is registered.
rt = ExtensionRuntime()
rt.register(Bang())
with pytest.raises(ExtensionError, match="unmet dependency|unknown"):
rt.activate({"ext.bang"})

View File

@@ -0,0 +1,59 @@
"""Tests for the engine kernel (SHARD-WP-0014 T1)."""
import pytest
from shard_wiki.engine import EngineKernel, extract_wikilinks
from shard_wiki.model import Identity
def test_write_creates_then_edits_as_history():
k = EngineKernel("eng")
p1 = k.write("Home", "first")
assert p1.identity == Identity("eng", "Home")
assert p1.envelope.source_rev == "1"
p2 = k.write("Home", "second")
assert p2.envelope.source_rev == "2"
assert k.read("Home").body == "second" # latest
assert [v.body for v in k.history("Home")] == ["first", "second"] # recoverable history
def test_read_missing_raises():
k = EngineKernel("eng")
with pytest.raises(KeyError):
k.read("Nope")
def test_delete_is_recoverable():
k = EngineKernel("eng")
k.write("Doc", "v1")
k.delete("Doc")
assert not k.exists("Doc")
with pytest.raises(KeyError):
k.read("Doc")
assert [v.body for v in k.history("Doc")] == ["v1"] # history retained
k.write("Doc", "v2") # restore by writing
assert k.exists("Doc") and k.read("Doc").body == "v2"
def test_keys_and_current_rev():
k = EngineKernel("eng")
k.write("A", "a")
k.write("B", "b")
k.write("A", "a2")
assert set(k.keys()) == {"A", "B"}
assert k.current_rev("A") == "2"
assert k.current_rev("Missing") is None
def test_links_and_red_link_resolution():
k = EngineKernel("eng")
k.write("Home", "see [[Target]] and [[Other|labelled]] and [[Target]] again")
k.write("Target", "exists")
assert k.links("Home") == ["Target", "Other"] # ordered, de-duped, label dropped
assert k.resolve_link("Target") == Identity("eng", "Target")
assert k.resolve_link("Other") is None # red-link (not yet created)
def test_extract_wikilinks_helper():
assert extract_wikilinks("none here") == []
assert extract_wikilinks("[[A]] [[B|x]] [[A]]") == ["A", "B"]

View File

@@ -0,0 +1,98 @@
"""Tests for capability-profile-derived-from-extensions (SHARD-WP-0014 T4, E-5)."""
import pytest
from shard_wiki.engine import (
Extension,
ExtensionRuntime,
Hook,
ProfileContribution,
derive_profile,
engine_base_profile,
)
from shard_wiki.model import (
Addressing,
ContentOpacity,
NativeQuery,
ProfileError,
Verb,
)
class StructExt(Extension):
id = "ext.struct"
def hooks(self):
return {
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
)
}
class AddrExt(Extension):
id = "ext.addr"
def hooks(self):
return {
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
addressing=Addressing.SPAN, verbs_add=frozenset({Verb.TRANSCLUDE_SOURCE})
)
}
class EncryptExt(Extension):
id = "ext.encrypt"
def hooks(self):
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(content_opacity=ContentOpacity.ENCRYPTED)}
class QueryExt(Extension):
id = "ext.query"
def hooks(self):
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(native_query=NativeQuery.DB_QUERY)}
def _active(*exts, ids=None):
rt = ExtensionRuntime()
for e in exts:
rt.register(e)
return rt.activate(ids if ids is not None else {e.id for e in exts})
def test_base_profile_is_valid_kernel_minimum():
p = engine_base_profile()
assert p.supports(Verb.READ) and p.supports(Verb.WRITE)
assert not p.supports(Verb.STRUCTURED_PAYLOAD)
assert p.addressing is Addressing.PATH
def test_no_extensions_yields_base():
rt = ExtensionRuntime()
active = rt.activate(set())
assert derive_profile(active) == engine_base_profile()
def test_activating_extension_raises_the_profile():
active = _active(StructExt(), AddrExt())
p = derive_profile(active)
assert p.supports(Verb.STRUCTURED_PAYLOAD) # from ext.struct
assert p.supports(Verb.TRANSCLUDE_SOURCE) # from ext.addr
assert p.addressing is Addressing.SPAN # raised by ext.addr
assert p.supports(Verb.READ) # base preserved
def test_profile_changes_with_active_set():
only_struct = derive_profile(_active(StructExt(), AddrExt(), ids={"ext.struct"}))
both = derive_profile(_active(StructExt(), AddrExt()))
assert not only_struct.supports(Verb.TRANSCLUDE_SOURCE)
assert both.supports(Verb.TRANSCLUDE_SOURCE) # E-5: profile reflects what's active
def test_composition_cannot_yield_an_impossible_profile():
# encrypted opacity + native query violates §6.5 implication rules -> derive must reject.
active = _active(EncryptExt(), QueryExt())
with pytest.raises(ProfileError):
derive_profile(active)

76
tests/test_error_paths.py Normal file
View File

@@ -0,0 +1,76 @@
"""Error-path / contract tests across modules (keeps the suite honest about failure behaviour)."""
from collections.abc import Iterable
import pytest
from shard_wiki import InformationSpace
from shard_wiki.adapters import FolderAdapter, ShardAdapter, run_conformance
from shard_wiki.engine import EngineKernel
from shard_wiki.model import CapabilityProfile, Identity, Page, Placement
from shard_wiki.provenance import ProvenanceEnvelope
from shard_wiki.union import ResolutionKind, UnionGraph
def _folder(tmp_path, name, files, writable=False):
root = tmp_path / name
root.mkdir(parents=True, exist_ok=True)
for rel, text in files.items():
(root / rel).write_text(text, encoding="utf-8")
return FolderAdapter(name, root, writable=writable)
def test_resolution_single_on_red_link_raises():
u = UnionGraph("s")
res = u.resolve("ghost")
assert res.kind is ResolutionKind.RED_LINK
with pytest.raises(KeyError):
res.single()
def test_apply_unknown_overlay_raises(tmp_path):
space = InformationSpace("t")
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
with pytest.raises(KeyError):
space.apply_overlay("does-not-exist")
def test_apply_overlay_for_unattached_shard_raises(tmp_path):
space = InformationSpace("t")
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
# draft an overlay whose target shard is not attached -> apply can't find an adapter
ov = space.overlays.draft(Identity("ghost", "X"), "body", base_rev=None)
with pytest.raises(KeyError):
space.apply_overlay(ov.overlay_id)
def test_kernel_delete_missing_raises():
with pytest.raises(KeyError):
EngineKernel("eng").delete("nope")
def test_placement_str():
assert str(Placement("shardA", "sub/Page")) == "shardA/sub/Page"
class _BrokenProfileAdapter(ShardAdapter):
"""profile() raises — the conformance battery must report failure, not crash."""
@property
def shard_id(self) -> str:
return "broken"
def profile(self) -> CapabilityProfile:
raise RuntimeError("profile blew up")
def keys(self) -> Iterable[str]:
return []
def read(self, key: str) -> Page:
return Page(Identity("broken", key), "x", ProvenanceEnvelope(source_shard="broken"))
def test_conformance_survives_a_broken_profile():
report = run_conformance(_BrokenProfileAdapter())
assert not report.ok
assert any(c.name == "profile-validates" and not c.ok for c in report.checks)

64
tests/test_ext_struct.py Normal file
View File

@@ -0,0 +1,64 @@
"""Tests for the ext.struct built-in extension (SHARD-WP-0014 T6)."""
import pytest
from shard_wiki import InformationSpace
from shard_wiki.adapters import assert_conformant
from shard_wiki.engine import ExtensionRuntime, build_engine_shard
from shard_wiki.engine.extensions import StructExt, parse_frontmatter
from shard_wiki.model import PageShape, Verb
_STRUCT_PAGE = "---\ntitle: Spec\nstatus: draft\n---\nbody text"
def test_parse_frontmatter():
fields, has = parse_frontmatter(_STRUCT_PAGE)
assert has and fields == {"title": "Spec", "status": "draft"}
assert parse_frontmatter("just prose") == ({}, False)
assert parse_frontmatter("---\nunterminated") == ({}, False)
def _runtime(allowed=None):
rt = ExtensionRuntime()
rt.register(StructExt(allowed_fields=allowed))
return rt
def test_feature_absent_when_extension_off():
shard = build_engine_shard("off", ExtensionRuntime(), activate=set())
shard.write("Spec", _STRUCT_PAGE)
assert shard.read("Spec").shape is PageShape.PROSE # kernel: opaque prose
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # honest absence
def test_feature_present_when_extension_on():
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
shard.write("Spec", _STRUCT_PAGE)
assert shard.read("Spec").shape is PageShape.TYPED_RECORD # tagged by ext.struct
assert shard.read("Spec").body.endswith("body text") # content preserved (in-text)
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # profile reflects activation (E-5)
assert_conformant(shard) # still conformant
def test_plain_page_is_not_tagged_even_when_on():
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
shard.write("Plain", "no frontmatter here")
assert shard.read("Plain").shape is PageShape.PROSE
def test_allowed_fields_validation_rejects_disallowed():
shard = build_engine_shard("v", _runtime(allowed={"title"}), activate={"ext.struct"})
with pytest.raises(ValueError, match="disallowed fields"):
shard.write("Bad", "---\ntitle: ok\nsecret: no\n---\nx")
shard.write("Good", "---\ntitle: ok\n---\nx") # allowed field passes
assert shard.read("Good").shape is PageShape.TYPED_RECORD
def test_through_information_space_edit():
space = InformationSpace("team")
space.attach(build_engine_shard("wikiE", _runtime(), activate={"ext.struct"}))
space.union.shard("wikiE").write("Doc", "---\ntitle: T\n---\nv1")
res = space.edit("Doc", "---\ntitle: T2\n---\nv2") # overlay→apply→write-through
assert res.status.value == "applied"
page = space.read("Doc")
assert page.shape is PageShape.TYPED_RECORD and "v2" in page.body

131
tests/test_git_adapter.py Normal file
View File

@@ -0,0 +1,131 @@
"""Tests for the GitShardAdapter read path + profile (SHARD-WP-0012 T1)."""
import subprocess
import pytest
from shard_wiki.adapters import GitShardAdapter, run_conformance
from shard_wiki.model import (
AttachmentMode,
History,
NotSupported,
ProfileError,
Substrate,
Verb,
)
def _git(repo, *args):
subprocess.run(
["git", "-C", str(repo), *args],
check=True,
capture_output=True,
env={"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
"PATH": __import__("os").environ.get("PATH", "")},
)
def _repo(tmp_path, files, name="repo"):
repo = tmp_path / name
repo.mkdir()
_git(repo, "init", "--quiet")
for rel, text in files.items():
p = repo / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
_git(repo, "add", rel)
_git(repo, "commit", "-m", "seed")
return repo
def test_keys_are_tracked_md_paths(tmp_path):
repo = _repo(tmp_path, {"Home.md": "h", "docs/Guide.md": "g", "ignore.txt": "x"})
adapter = GitShardAdapter("git", repo)
assert set(adapter.keys()) == {"Home", "docs/Guide"} # only tracked *.md
def test_read_returns_page_with_commit_sha_rev(tmp_path):
repo = _repo(tmp_path, {"Home.md": "welcome"})
adapter = GitShardAdapter("git", repo)
page = adapter.read("Home")
assert page.identity.shard == "git"
assert page.body == "welcome"
head = subprocess.run(
["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True, check=True
).stdout.strip()
assert page.envelope.source_rev == head # source_rev is the commit sha
assert page.envelope.lineage == "git-native"
def test_read_missing_key_raises(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(KeyError):
adapter.read("Nope")
def test_profile_validates_implication_rules(tmp_path):
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
assert profile.substrate is Substrate.GIT
assert profile.attachment_mode is AttachmentMode.GIT_IS_STORE
assert profile.history is History.GIT_NATIVE # git-is-store ⟹ git-native
profile.validate() # raises if the implication rule were violated
def test_profile_is_read_only_in_t1(tmp_path):
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
assert profile.supports(Verb.READ)
assert not profile.supports(Verb.WRITE)
def test_conformance_read_path_passes(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h", "Other.md": "o"}))
report = run_conformance(adapter)
assert report.ok, report.diff()
def test_unclaimed_write_raises_not_supported(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(NotSupported):
adapter.write("Home", "new") # read-only: honest absence
def test_empty_repo_has_no_keys(tmp_path):
repo = tmp_path / "empty"
repo.mkdir()
_git(repo, "init", "--quiet")
adapter = GitShardAdapter("git", repo)
assert list(adapter.keys()) == []
def test_bad_profile_combo_is_rejected():
# Sanity: the implication rule that backs the git profile actually bites when violated.
from shard_wiki.model import (
AccessGrant,
Addressing,
CapabilityProfile,
ContentOpacity,
MergeModel,
NativeQuery,
OperationalEnvelope,
Translation,
WriteGranularity,
)
from shard_wiki.provenance import Liveness
with pytest.raises(ProfileError):
CapabilityProfile(
substrate=Substrate.FILES, # not git, but claims git-is-store
attachment_mode=AttachmentMode.GIT_IS_STORE,
write_granularity=WriteGranularity.NONE,
content_opacity=ContentOpacity.TRANSPARENT,
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
access_grant=AccessGrant.OPEN,
liveness=Liveness.STATIC,
history=History.NONE,
merge_model=MergeModel.NONE,
addressing=Addressing.PATH,
native_query=NativeQuery.NONE,
translation=Translation.NATIVE,
supported_verbs=frozenset({Verb.READ}),
).validate()

View File

@@ -0,0 +1,116 @@
"""GitShardAdapter history adopt + cross-substrate integration (SHARD-WP-0012 T3)."""
import os
import subprocess
import pytest
from shard_wiki.adapters import FolderAdapter, GitShardAdapter
from shard_wiki.coordination import ApplyStatus
from shard_wiki.space import InformationSpace
_ENV = {
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
"PATH": os.environ.get("PATH", ""),
}
def _git(repo, *args):
return subprocess.run(
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
).stdout.strip()
def _git_repo(tmp_path, files, name="git"):
repo = tmp_path / name
repo.mkdir()
_git(repo, "init", "--quiet")
for rel, text in files.items():
(repo / rel).parent.mkdir(parents=True, exist_ok=True)
(repo / rel).write_text(text, encoding="utf-8")
_git(repo, "add", rel)
_git(repo, "commit", "-m", "seed")
return repo
def _folder(tmp_path, name, files, writable=False):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root, writable=writable)
# -- history adopt -------------------------------------------------------------
def test_history_lists_commits_newest_first(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "v1"})
adapter = GitShardAdapter("git", repo, writable=True)
adapter.write("Home", "v2")
history = adapter.history("Home")
assert len(history) == 2
assert history[0].message == "write Home.md" # newest first
assert history[-1].message == "seed"
assert all(rev.sha for rev in history)
def test_history_unknown_key_raises(tmp_path):
adapter = GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "h"}))
with pytest.raises(KeyError):
adapter.history("Nope")
# -- cross-substrate integration ----------------------------------------------
def test_resolve_across_git_and_folder(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "git home"})))
space.attach(_folder(tmp_path, "notes", {"Daily.md": "folder daily"}))
assert space.read("Home").body == "git home" # resolved from the git shard
assert space.read("Daily").body == "folder daily" # resolved from the folder shard
def test_chorus_spans_substrates_with_divergence(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Shared.md": "from git"})))
space.attach(_folder(tmp_path, "notes", {"Shared.md": "from folder"}))
res = space.resolve("Shared")
assert {p.body for p in res.pages} == {"from git", "from folder"} # chorus across substrates
git_page = next(p for p in res.pages if p.identity.shard == "git")
assert git_page.envelope.divergence # divergence recorded, not erased
def test_edit_through_git_shard_commits(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "original"})
space = InformationSpace("space")
space.attach(GitShardAdapter("git", repo, writable=True))
result = space.edit("Home", "edited via overlay")
assert result.status is ApplyStatus.APPLIED # write-through fast-forward on a git shard
assert space.read("Home").body == "edited via overlay"
assert int(_git(repo, "rev-list", "--count", "HEAD")) == 2 # the edit became a commit
def test_apply_under_drift_refuses_on_external_commit(tmp_path):
repo = _git_repo(tmp_path, {"Home.md": "original"})
space = InformationSpace("space")
space.attach(GitShardAdapter("git", repo, writable=True))
overlay = space.overlay("Home", "my draft") # base_rev = current git sha
# Another writer commits to the same path → the sha moves underneath the draft.
(repo / "Home.md").write_text("someone else", encoding="utf-8")
_git(repo, "add", "Home.md")
_git(repo, "commit", "-m", "external")
result = space.apply_overlay(overlay.overlay_id)
assert result.status is ApplyStatus.REFUSED_DRIFT # never clobber (sha drift detected)
# The shard itself is untouched — the external commit stands; the draft remains a draft.
assert space.union.shard("git").read("Home").body == "someone else"
def test_overlay_on_read_only_git_shard_kept_as_draft(tmp_path):
space = InformationSpace("space")
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "ro"}), writable=False))
result = space.edit("Home", "wanted change")
assert result.status is ApplyStatus.KEPT_DRAFT # read-only target → overlay retained

View File

@@ -0,0 +1,89 @@
"""Tests for GitShardAdapter write=commit + current_rev drift (SHARD-WP-0012 T2)."""
import os
import subprocess
from shard_wiki.adapters import GitShardAdapter, run_conformance
from shard_wiki.model import Verb
_ENV = {
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
"PATH": os.environ.get("PATH", ""),
}
def _git(repo, *args, capture=False):
return subprocess.run(
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
).stdout.strip()
def _repo(tmp_path, files):
repo = tmp_path / "repo"
repo.mkdir()
_git(repo, "init", "--quiet")
for rel, text in files.items():
(repo / rel).write_text(text, encoding="utf-8")
_git(repo, "add", rel)
_git(repo, "commit", "-m", "seed")
return repo
def test_writable_profile_declares_write_and_version(tmp_path):
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}), writable=True).profile()
assert profile.supports(Verb.WRITE)
assert profile.supports(Verb.VERSION)
profile.validate() # PER_PAGE + WRITE is a consistent combination
def test_write_creates_a_commit(tmp_path):
repo = _repo(tmp_path, {"Home.md": "old"})
adapter = GitShardAdapter("git", repo, writable=True)
before = _git(repo, "rev-list", "--count", "HEAD")
page = adapter.write("Home", "new body")
after = _git(repo, "rev-list", "--count", "HEAD")
assert int(after) == int(before) + 1 # one new commit
assert page.body == "new body"
assert page.envelope.source_rev == _git(repo, "rev-parse", "HEAD") # page is at the new sha
def test_write_advances_current_rev(tmp_path):
repo = _repo(tmp_path, {"Home.md": "old"})
adapter = GitShardAdapter("git", repo, writable=True)
rev_before = adapter.current_rev("Home")
adapter.write("Home", "changed")
assert adapter.current_rev("Home") != rev_before # sha moved → drift detectable
def test_write_new_key_tracks_it(tmp_path):
repo = _repo(tmp_path, {"Home.md": "h"})
adapter = GitShardAdapter("git", repo, writable=True)
adapter.write("docs/New", "fresh page")
assert "docs/New" in set(adapter.keys())
assert adapter.read("docs/New").body == "fresh page"
def test_noop_write_creates_no_empty_commit(tmp_path):
repo = _repo(tmp_path, {"Home.md": "same"})
adapter = GitShardAdapter("git", repo, writable=True)
before = _git(repo, "rev-list", "--count", "HEAD")
adapter.write("Home", "same") # identical body → nothing to commit
assert _git(repo, "rev-list", "--count", "HEAD") == before
def test_current_rev_reflects_external_commit(tmp_path):
repo = _repo(tmp_path, {"Home.md": "h"})
adapter = GitShardAdapter("git", repo, writable=True)
rev = adapter.current_rev("Home")
# An out-of-band commit to the same path (another writer) moves the per-path sha.
(repo / "Home.md").write_text("externally edited", encoding="utf-8")
_git(repo, "add", "Home.md")
_git(repo, "commit", "-m", "external")
assert adapter.current_rev("Home") != rev
def test_conformance_positive_write_probe_passes(tmp_path):
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "body"}), writable=True)
report = run_conformance(adapter)
assert report.ok, report.diff()

View File

@@ -0,0 +1,84 @@
"""Tests for the git-backed event store (SHARD-WP-0009 T1).
The git backend must satisfy the same EventStore contract as the in-memory one (round-trip,
ordering, determinism) while making the log git-addressable.
"""
import subprocess
import pytest
from shard_wiki.coordination import (
DecisionLog,
EventType,
GitEventStore,
InMemoryEventStore,
deserialize_event,
serialize_event,
)
@pytest.fixture
def git_store(tmp_path):
return GitEventStore(tmp_path / "coord")
def test_append_git_read_round_trips(git_store):
log = DecisionLog(git_store)
ev = log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
(read,) = log.events("s")
assert read.seq == ev.seq == 0
assert read.space == "s"
assert read.type is EventType.ALIAS_SET
assert read.payload == {"alias": "Home", "target": "shardA:Index"}
def test_ordering_preserved_and_per_space_monotonic(git_store):
log = DecisionLog(git_store)
log.append("a", EventType.ALIAS_SET, {"alias": "X", "target": "s:1"})
log.append("a", EventType.ALIAS_SET, {"alias": "Y", "target": "s:2"})
log.append("b", EventType.ALIAS_SET, {"alias": "Z", "target": "s:3"})
assert [e.seq for e in log.events("a")] == [0, 1]
assert [e.payload["alias"] for e in log.events("a")] == ["X", "Y"]
assert [e.seq for e in log.events("b")] == [0] # independent ref/ordering
def test_each_append_is_a_git_commit(git_store):
log = DecisionLog(git_store)
log.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
log.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"})
ref = GitEventStore._ref("s")
count = subprocess.run(
["git", "-C", str(git_store.repo_path), "rev-list", "--count", ref],
capture_output=True, text=True, check=True,
).stdout.strip()
assert count == "2" # one immutable commit object per append
def test_deterministic_serialization_is_stable_and_sorted():
log = InMemoryEventStore()
ev = log.append("s", EventType.ALIAS_SET, {"target": "z", "alias": "a"})
blob = serialize_event(ev)
assert serialize_event(ev) == blob # stable across calls
assert blob.index(b'"alias"') < blob.index(b'"target"') # payload keys sorted, not insertion
assert deserialize_event(blob).payload == {"alias": "a", "target": "z"}
def test_git_fold_matches_in_memory_fold(git_store):
events = [
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
]
mem = DecisionLog(InMemoryEventStore())
git = DecisionLog(git_store)
for typ, payload in events:
mem.append("s", typ, payload)
git.append("s", typ, payload)
assert git.fold("s").aliases == mem.fold("s").aliases
assert git.fold("s").equivalence_groups == mem.fold("s").equivalence_groups
def test_default_decisionlog_is_in_memory():
assert isinstance(DecisionLog()._store, InMemoryEventStore)

View File

@@ -0,0 +1,89 @@
"""Tests for the indexed equivalence relation — blocking + verify (SHARD-WP-0011 T1)."""
from itertools import combinations
from shard_wiki.incremental import EquivalenceIndex, MinHasher, band_keys, jaccard, shingles
from shard_wiki.incremental.equivalence import _fingerprint
from shard_wiki.model import Identity, Page
from shard_wiki.provenance import ProvenanceEnvelope
def _page(shard, key, body):
return Page(
identity=Identity(shard, key),
body=body,
envelope=ProvenanceEnvelope(source_shard=shard),
)
def _brute_force_groups(pages, threshold):
"""Oracle: O(N²) verify of every pair, then connected components."""
parent = {p.identity: p.identity for p in pages}
def find(x):
while parent[x] != x:
parent[x] = parent[parent[x]]
x = parent[x]
return x
for p, q in combinations(pages, 2):
same_fp = _fingerprint(p.body) == _fingerprint(q.body)
sim = jaccard(shingles(p.body), shingles(q.body))
if same_fp or sim >= threshold:
parent[find(p.identity)] = find(q.identity)
comps = {}
for p in pages:
comps.setdefault(find(p.identity), set()).add(p.identity)
return {frozenset(v) for v in comps.values() if len(v) > 1}
def test_minhash_lsh_buckets_near_duplicates_together():
hasher = MinHasher(num_perm=64)
base = "the quick brown fox jumps over the lazy dog near the river bank today"
near = base + " and then some"
far = "completely unrelated content about astrophysics and distant galaxies far"
b_base = set(band_keys(hasher.signature(shingles(base)), 32))
b_near = set(band_keys(hasher.signature(shingles(near)), 32))
b_far = set(band_keys(hasher.signature(shingles(far)), 32))
assert b_base & b_near # near-duplicates share at least one band
assert not (b_base & b_far) # unrelated pages do not
def test_exact_duplicate_across_shards_is_equivalent():
idx = EquivalenceIndex()
idx.add(_page("A", "Foo", "identical body text here"))
idx.add(_page("B", "Bar", "identical body text here"))
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
{Identity("A", "Foo"), Identity("B", "Bar")}
)
def test_unrelated_pages_are_not_equivalent():
idx = EquivalenceIndex()
idx.add(_page("A", "Foo", "alpha beta gamma delta epsilon"))
idx.add(_page("B", "Bar", "nothing in common whatsoever entirely"))
assert idx.groups() == ()
def test_curator_binding_forces_equivalence_regardless_of_content():
idx = EquivalenceIndex()
idx.add(_page("A", "Foo", "one thing"))
idx.add(_page("B", "Bar", "totally different"))
idx.bind(Identity("A", "Foo"), Identity("B", "Bar"))
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
{Identity("A", "Foo"), Identity("B", "Bar")}
)
def test_index_matches_brute_force_oracle():
threshold = 0.7
shared = "shared sentence one shared sentence two shared sentence three end"
pages = [
_page("A", "Doc1", shared),
_page("B", "Doc1copy", shared + " minor tail"), # near-dup of A
_page("C", "Other", "a totally distinct page with no overlapping shingles at all here"),
_page("D", "Lonely", "yet another isolated document about unrelated subject matter alone"),
]
idx = EquivalenceIndex(threshold=threshold)
idx.build(pages)
assert set(idx.groups()) == _brute_force_groups(pages, threshold)

View File

@@ -0,0 +1,84 @@
"""Incremental maintenance == rebuild, with retraction + propagation (SHARD-WP-0011 T2)."""
from shard_wiki.incremental import EquivalenceIndex
from shard_wiki.model import Identity, Page
from shard_wiki.provenance import ProvenanceEnvelope
def _page(shard, key, body):
return Page(
identity=Identity(shard, key),
body=body,
envelope=ProvenanceEnvelope(source_shard=shard),
)
def _rebuilt(pages, curator=()):
idx = EquivalenceIndex()
idx.build(pages, curator)
return idx
def _equal(a, b):
return a.edges() == b.edges() and set(a.groups()) == set(b.groups())
def test_add_keeps_index_equal_to_rebuild():
pages = [_page("A", "Foo", "same content here"), _page("B", "Bar", "same content here")]
idx = EquivalenceIndex()
for p in pages:
idx.add(p)
assert _equal(idx, _rebuilt(pages))
assert idx.groups() # the two collapse
def test_remove_keeps_index_equal_to_rebuild():
pages = [
_page("A", "Foo", "same content here"),
_page("B", "Bar", "same content here"),
_page("C", "Baz", "unrelated isolated material entirely"),
]
idx = _rebuilt(pages)
idx.remove(Identity("B", "Bar"))
assert _equal(idx, _rebuilt([pages[0], pages[2]]))
def test_edit_into_new_bucket_retracts_stale_edge():
a = _page("A", "Foo", "shared identical body text")
b = _page("B", "Bar", "shared identical body text")
idx = _rebuilt([a, b])
assert idx.groups() # A ≡ B initially
# Edit B to something completely different: it exits A's buckets, the edge is retracted.
b2 = _page("B", "Bar", "now totally divergent unrelated prose about nothing")
idx.update(b2)
assert idx.groups() == () # stale edge gone
assert _equal(idx, _rebuilt([a, b2]))
def test_edit_into_equivalence_adds_edge():
a = _page("A", "Foo", "target body to converge on later")
b = _page("B", "Bar", "initially completely separate writing here")
idx = _rebuilt([a, b])
assert idx.groups() == ()
b2 = _page("B", "Bar", "target body to converge on later") # now identical to A
idx.update(b2)
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
{Identity("A", "Foo"), Identity("B", "Bar")}
)
assert _equal(idx, _rebuilt([a, b2]))
def test_removing_connector_splits_a_chorus():
# Curator chain A—B—C (no direct A—C): one group of three.
a, b, c = (_page("A", "X", "aaa"), _page("B", "Y", "bbb"), _page("C", "Z", "ccc"))
idx = EquivalenceIndex()
for p in (a, b, c):
idx.add(p)
idx.bind(a.identity, b.identity)
idx.bind(b.identity, c.identity)
assert idx.equivalent_to(a.identity) == {a.identity, b.identity, c.identity}
# Removing the connector B retracts/propagates: the chorus splits.
idx.remove(b.identity)
assert idx.groups() == ()
chain = [(a.identity, b.identity), (b.identity, c.identity)]
assert _equal(idx, _rebuilt([a, c], curator=chain))

View File

@@ -0,0 +1,89 @@
"""Tests for I-2 verification — digest + consistency-checker (SHARD-WP-0011 T3)."""
from shard_wiki.incremental import (
ConsistencyChecker,
EquivalenceIndex,
derived_digest,
)
from shard_wiki.model import Identity, Page
from shard_wiki.provenance import ProvenanceEnvelope
def _page(shard, key, body):
return Page(
identity=Identity(shard, key),
body=body,
envelope=ProvenanceEnvelope(source_shard=shard),
)
def test_digest_is_stable_under_equivalent_event_orders():
pages = [
_page("A", "Foo", "shared body text here"),
_page("B", "Bar", "shared body text here"),
_page("C", "Baz", "an entirely separate unrelated document"),
]
forward = EquivalenceIndex()
for p in pages:
forward.add(p)
reverse = EquivalenceIndex()
for p in reversed(pages):
reverse.add(p)
assert derived_digest(forward) == derived_digest(reverse)
def test_clean_index_reports_healthy():
pages = [_page("A", "Foo", "same body"), _page("B", "Bar", "same body")]
idx = EquivalenceIndex()
idx.build(pages)
checker = ConsistencyChecker(idx, pages_fn := (lambda: pages))
report = checker.check_and_repair()
assert report.drifted is False and report.healthy is True
assert pages_fn() # source unchanged
def test_missed_delta_drift_is_detected_and_repaired():
a = _page("A", "Foo", "converging target body")
b = _page("B", "Bar", "initially unrelated separate text")
source = {"pages": [a, b]}
idx = EquivalenceIndex()
idx.build(source["pages"])
assert idx.groups() == () # not equivalent yet
# Source changes B to match A, but the index is never told (a missed delta → drift).
b2 = _page("B", "Bar", "converging target body")
source["pages"] = [a, b2]
checker = ConsistencyChecker(idx, lambda: source["pages"])
report = checker.check_and_repair()
assert report.drifted is True and report.repaired is True and report.healthy is True
# Self-healed: the index now reflects the equivalence.
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
{Identity("A", "Foo"), Identity("B", "Bar")}
)
def test_corrupted_internal_state_is_healed():
a = _page("A", "Foo", "identical content")
b = _page("B", "Bar", "identical content")
idx = EquivalenceIndex()
idx.build([a, b])
# Corrupt the derived tier directly: delete a true edge (simulated index corruption).
idx._content_edges.clear()
assert idx.groups() == () # corrupted away
checker = ConsistencyChecker(idx, lambda: [a, b])
report = checker.check_and_repair()
assert report.drifted is True and report.healthy is True
assert idx.groups() # edge restored by scoped recompute
def test_removed_source_page_is_reconciled():
a = _page("A", "Foo", "same body")
b = _page("B", "Bar", "same body")
idx = EquivalenceIndex()
idx.build([a, b])
checker = ConsistencyChecker(idx, lambda: [a]) # B vanished from source
report = checker.check_and_repair()
assert report.healthy is True
assert Identity("B", "Bar") not in idx.identities()

View File

@@ -0,0 +1,74 @@
"""Wire the incremental tier behind InformationSpace views (SHARD-WP-0011 T4)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.coordination import EventType
from shard_wiki.model import Identity
from shard_wiki.space import InformationSpace
from shard_wiki.views import all_pages
def _shard(tmp_path, name, files):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root)
def test_all_pages_via_index_matches_direct_fold(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "wiki", {"Home.md": "welcome", "Guide.md": "the guide"}))
space.attach(_shard(tmp_path, "notes", {"Daily.md": "today"}))
# Routed-through-index result equals the direct fold-based computation (behaviour unchanged).
via_index = {(e.name, e.members) for e in space.all_pages()}
direct = {(e.name, e.members) for e in all_pages(space.union)}
assert via_index == direct
def test_curator_binding_collapses_via_maintained_index(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "a", {"Foo.md": "x"}))
space.attach(_shard(tmp_path, "b", {"Bar.md": "y"}))
space.log.append(
"space", EventType.BINDING_MADE, {"members": ["a:Foo", "b:Bar"]}
)
# The maintained index re-syncs curator edges live from the log fold.
collapsed = [e for e in space.all_pages() if len(e.members) == 2]
assert len(collapsed) == 1
assert set(collapsed[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")}
def test_content_duplicate_collapses_via_index(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "a", {"Foo.md": "the very same body content here"}))
space.attach(_shard(tmp_path, "b", {"Bar.md": "the very same body content here"}))
dup = [e for e in space.all_pages() if len(e.members) == 2]
assert len(dup) == 1 # content equivalence detected by the maintained index
assert set(dup[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")}
def test_attach_invalidates_index(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"}))
assert space.all_pages() # builds the index (one page, no groups)
space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"})) # marks index stale
dup = [e for e in space.all_pages() if len(e.members) == 2]
assert len(dup) == 1 # rebuilt fallback picks up the new equivalent page
def test_verify_index_reports_healthy_when_consistent(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"}))
space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"}))
space.all_pages() # ensure built
report = space.verify_index()
assert report.healthy is True
def test_reindex_is_an_explicit_fallback(tmp_path):
space = InformationSpace("space")
space.attach(_shard(tmp_path, "a", {"Foo.md": "content"}))
before = space.index.digest()
space.reindex()
assert space.index.digest() == before # rebuild is deterministic

View File

@@ -0,0 +1,76 @@
"""Tests for the AllPages + SiteMap enumeration views (SHARD-WP-0010 T4)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.coordination import DecisionLog, EventType
from shard_wiki.model import Identity
from shard_wiki.union import UnionGraph
from shard_wiki.views import all_pages, site_map
def _shard(tmp_path, name, files):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root)
def test_all_pages_spans_shards(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"A.md": "a"}))
u.attach(_shard(tmp_path, "shardB", {"B.md": "b"}))
names = {e.name for e in all_pages(u)}
assert names == {"A", "B"}
def test_chorus_collapses_to_one_entry_with_divergence(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A home"}))
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B home"}))
entries = all_pages(u)
home = [e for e in entries if e.name == "Home"]
assert len(home) == 1 # chorus → single entry
assert set(home[0].members) == {Identity("shardA", "Home"), Identity("shardB", "Home")}
assert home[0].diverges is True # bodies differ — collapse acknowledged, not silent
def test_chorus_same_body_does_not_diverge(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "same"}))
u.attach(_shard(tmp_path, "shardB", {"Home.md": "same"}))
(home,) = [e for e in all_pages(u) if e.name == "Home"]
assert home.diverges is False
def test_equivalence_binding_collapses_distinct_keys(tmp_path):
log = DecisionLog()
log.append(
"space", EventType.BINDING_MADE, {"members": ["shardA:Foo", "shardB:Bar"]}
)
u = UnionGraph("space", log=log)
u.attach(_shard(tmp_path, "shardA", {"Foo.md": "x"}))
u.attach(_shard(tmp_path, "shardB", {"Bar.md": "x"}))
pair = {Identity("shardA", "Foo"), Identity("shardB", "Bar")}
# The two bound identities fold into one entry (named by the min key, "Bar").
bound = [e for e in all_pages(u) if {*e.members} == pair]
assert len(bound) == 1
assert bound[0].name == "Bar"
def test_sitemap_reflects_namespace_paths(tmp_path):
u = UnionGraph("space")
u.attach(
_shard(
tmp_path,
"shardA",
{"Home.md": "h", "docs/Guide.md": "g", "docs/api/Ref.md": "r"},
)
)
root = site_map(u)
# Top level: "Home" page directly, and a "docs" namespace.
assert any(p.key == "Home" for p in root.pages)
docs = next(c for c in root.children if c.name == "docs")
assert any(p.key == "docs/Guide" for p in docs.pages)
api = next(c for c in docs.children if c.name == "api")
assert any(p.key == "docs/api/Ref" for p in api.pages)

View File

@@ -0,0 +1,51 @@
"""Tests for the BackLinks derived view (SHARD-WP-0010 T2)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.model import Identity
from shard_wiki.union import UnionGraph
from shard_wiki.views import build_backlinks
def _shard(tmp_path, name, files):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root)
def test_link_yields_backlink_with_provenance(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"A.md": "see [[B]]", "B.md": "target"}))
index = build_backlinks(u)
assert index.sources("B") == frozenset({Identity("shardA", "A")})
(bl,) = index.to("B")
assert bl.source_shard == "shardA" # entry carries source provenance
def test_red_links_create_no_backlinks(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"A.md": "see [[Ghost]]"}))
index = build_backlinks(u)
assert index.to("Ghost") == () # unresolved target → no backlink
assert "Ghost" not in index.names()
def test_chorus_target_aggregates_backlinks(tmp_path):
# "Home" exists in two shards (a chorus); links to it from anywhere aggregate under one name.
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A home", "A.md": "[[Home]]"}))
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B home", "B.md": "[[Home]]"}))
index = build_backlinks(u)
assert index.sources("Home") == frozenset(
{Identity("shardA", "A"), Identity("shardB", "B")}
)
def test_backlinks_span_shards(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Index.md": "x"}))
u.attach(_shard(tmp_path, "shardB", {"B.md": "links [[Index]]"}))
index = build_backlinks(u)
assert index.sources("Index") == frozenset({Identity("shardB", "B")})

View File

@@ -0,0 +1,52 @@
"""Integration: derived views exposed on InformationSpace over two shards (SHARD-WP-0010 T5)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.model import Identity
from shard_wiki.space import InformationSpace
def _shard(tmp_path, name, files):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root)
def _space(tmp_path):
space = InformationSpace("space")
space.attach(
_shard(tmp_path, "wiki", {"Home.md": "welcome, see [[Guide]]", "Guide.md": "the guide"})
)
space.attach(_shard(tmp_path, "notes", {"Daily.md": "today I read [[Guide]]"}))
return space
def test_backlinks_across_two_shards(tmp_path):
space = _space(tmp_path)
sources = {bl.source for bl in space.backlinks("Guide")}
assert sources == {Identity("wiki", "Home"), Identity("notes", "Daily")}
def test_all_pages_and_site_map_over_union(tmp_path):
space = _space(tmp_path)
names = {e.name for e in space.all_pages()}
assert names == {"Home", "Guide", "Daily"}
leaves = {p.key for p in space.site_map().pages}
assert {"Home", "Guide", "Daily"} <= leaves
def test_recent_changes_includes_alias_and_edits(tmp_path):
space = _space(tmp_path)
space.alias("Start", "wiki:Home", actor="ana")
feed = space.recent_changes()
kinds = {e.kind for e in feed}
assert "alias" in kinds and "edit" in kinds
alias = next(e for e in feed if e.kind == "alias")
assert alias.source == "coordination" and alias.actor == "ana"
def test_red_link_creates_no_backlink_via_space(tmp_path):
space = _space(tmp_path)
assert space.backlinks("Nonexistent") == ()

69
tests/test_views_links.py Normal file
View File

@@ -0,0 +1,69 @@
"""Tests for the wikilink + red-link model (SHARD-WP-0010 T1)."""
from shard_wiki.adapters import FolderAdapter
from shard_wiki.union import ResolutionKind, UnionGraph
from shard_wiki.views import extract_links, resolve_links
def _shard(tmp_path, name, files):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
return FolderAdapter(name, root)
def test_extracts_plain_and_labelled_links():
links = extract_links("See [[Home]] and [[Index|the index]].")
assert [(link.target, link.label, link.text) for link in links] == [
("Home", None, "Home"),
("Index", "the index", "the index"),
]
def test_links_carry_body_offsets_in_document_order():
body = "a [[One]] b [[Two]]"
links = extract_links(body)
assert [link.target for link in links] == ["One", "Two"]
s, e = links[0].span
assert body[s:e] == "[[One]]"
def test_code_regions_are_not_scanned():
body = "real [[Home]]\n```\n[[NotALink]]\n```\ninline `[[AlsoNot]]` done"
targets = [link.target for link in extract_links(body)]
assert targets == ["Home"]
def test_camelcase_off_by_default_then_opt_in():
body = "FrontPage links to [[Home]]"
assert [link.target for link in extract_links(body)] == ["Home"] # CamelCase ignored
on = extract_links(body, camelcase=True)
assert {link.target for link in on} == {"FrontPage", "Home"}
assert next(link for link in on if link.target == "FrontPage").auto is True
def test_camelcase_does_not_double_count_inside_explicit_link():
# [[FrontPage]] is one explicit link, not also a CamelCase auto-link.
links = extract_links("[[FrontPage]]", camelcase=True)
assert len(links) == 1
assert links[0].auto is False
def test_resolve_links_distinguishes_link_from_red_link(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "home"}))
resolved = resolve_links(u, "[[Home]] and [[Ghost]]")
by_target = {r.link.target: r for r in resolved}
assert by_target["Home"].resolution.kind is ResolutionKind.SINGLE
assert by_target["Home"].is_red_link is False
assert by_target["Ghost"].is_red_link is True # unresolved → createable red-link
def test_resolve_links_surfaces_chorus(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A"}))
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B"}))
(resolved,) = resolve_links(u, "[[Home]]")
assert resolved.resolution.kind is ResolutionKind.CHORUS

View File

@@ -0,0 +1,67 @@
"""Tests for the RecentChanges merged feed (SHARD-WP-0010 T3)."""
import os
from datetime import datetime, timezone
from shard_wiki.adapters import FolderAdapter
from shard_wiki.coordination import DecisionLog, EventType
from shard_wiki.union import UnionGraph
from shard_wiki.views import recent_changes
def _shard(tmp_path, name, files, mtime=None):
root = tmp_path / name
for rel, text in files.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(text, encoding="utf-8")
if mtime is not None:
os.utime(p, (mtime, mtime))
return FolderAdapter(name, root)
def test_edit_and_alias_both_appear_newest_first(tmp_path):
# Page edit signal pinned to an old mtime; the alias decision happens "now" → alias is newest.
old = datetime(2020, 1, 1, tzinfo=timezone.utc).timestamp()
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Home.md": "home"}, mtime=old))
log = DecisionLog()
log.append("space", EventType.ALIAS_SET, {"alias": "Start", "target": "shardA:Home"})
feed = recent_changes(u, log, "space")
kinds = [e.kind for e in feed]
assert "edit" in kinds and "alias" in kinds
assert feed[0].kind == "alias" # newest first
assert feed[-1].kind == "edit"
# Monotonic non-increasing by time.
assert all(feed[i].when >= feed[i + 1].when for i in range(len(feed) - 1))
def test_per_shard_attribution_present(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"A.md": "a"}))
u.attach(_shard(tmp_path, "shardB", {"B.md": "b"}))
feed = recent_changes(u, DecisionLog(), "space")
edits = {e.ref: e.source for e in feed if e.kind == "edit"}
assert edits["shardA:A"] == "shardA"
assert edits["shardB:B"] == "shardB" # each edit attributed to its shard
def test_coordination_entries_carry_actor_and_ref(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"Doc.md": "x"}))
log = DecisionLog()
log.append(
"space", EventType.PAGE_FORKED, {"source": "shardA:Doc", "fork": "shardB:Doc"}, actor="ana"
)
fork = next(e for e in recent_changes(u, log, "space") if e.kind == "fork")
assert fork.source == "coordination"
assert fork.actor == "ana"
assert fork.ref == "shardA:Doc→shardB:Doc"
def test_limit_truncates_to_newest(tmp_path):
u = UnionGraph("space")
u.attach(_shard(tmp_path, "shardA", {"A.md": "a", "B.md": "b", "C.md": "c"}))
feed = recent_changes(u, DecisionLog(), "space", limit=2)
assert len(feed) == 2

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0001
type: workplan
title: "shard-wiki requirements from yawex prior art"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0002
type: workplan
title: "federation architecture design"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0003
type: workplan
title: "wiki-engine deep-dive batch (new-insight + git-forge + classic engines)"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0004
type: workplan
title: "computational / interactive-knowledge systems research"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0005
type: workplan
title: "core architecture hardening (blueprint review fixes)"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0006
type: workplan
title: "core architecture hardening II (round-2 review fixes)"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0007
type: workplan
title: "foundation implementation — model, contract, decision log, union read"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,7 +2,7 @@
id: SHARD-WP-0008
type: workplan
title: "write path — overlay engine, writable adapter, apply-under-drift"
domain: whynot
domain: consumer
repo: shard-wiki
status: done
owner: tegwick

View File

@@ -2,9 +2,9 @@
id: SHARD-WP-0009
type: workplan
title: "git-backed DecisionLog + per-space append authority"
domain: whynot
domain: consumer
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -39,7 +39,7 @@ sharding (blueprint O-12). Single append authority per space is the target.
```task
id: SHARD-WP-0009-T1
status: todo
status: done
priority: high
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
```
@@ -54,7 +54,7 @@ ordering preserved; deterministic serialization.
```task
id: SHARD-WP-0009-T2
status: todo
status: done
priority: high
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
```
@@ -69,7 +69,7 @@ hand-off resumes from head; a partitioned non-holder cannot fork the log.
```task
id: SHARD-WP-0009-T3
status: todo
status: done
priority: high
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
```
@@ -82,7 +82,7 @@ process B (new handle) sees it; fold equals the in-memory fold for the same even
```task
id: SHARD-WP-0009-T4
status: todo
status: done
priority: medium
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
```

View File

@@ -2,9 +2,9 @@
id: SHARD-WP-0010
type: workplan
title: "derived views — wikilinks, BackLinks, RecentChanges, AllPages/SiteMap"
domain: whynot
domain: consumer
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -36,7 +36,7 @@ later by SHARD-WP-0011) and carry provenance. Presentation stays out of core (L6
```task
id: SHARD-WP-0010-T1
status: todo
status: done
priority: high
state_hub_task_id: "792660c3-9be9-4771-9f51-69d01f0c7f13"
```
@@ -51,7 +51,7 @@ red-link, CamelCase opt-in.
```task
id: SHARD-WP-0010-T2
status: todo
status: done
priority: high
state_hub_task_id: "431a54c3-82b5-4b08-b3f0-762624d4c91d"
```
@@ -65,7 +65,7 @@ chorus pages aggregate.
```task
id: SHARD-WP-0010-T3
status: todo
status: done
priority: medium
state_hub_task_id: "270c1c31-0445-42b9-9a49-92d32c298eb2"
```
@@ -79,7 +79,7 @@ alias both appear, newest-first; per-shard attribution present.
```task
id: SHARD-WP-0010-T4
status: todo
status: done
priority: low
state_hub_task_id: "898ba43e-cdef-4ce8-9fa3-4ce60ebb4fdd"
```
@@ -92,7 +92,7 @@ collapses to one entry with divergence noted; sitemap reflects paths.
```task
id: SHARD-WP-0010-T5
status: todo
status: done
priority: medium
state_hub_task_id: "7157544b-5d3b-45a2-ba5a-c32244c59323"
```

View File

@@ -2,9 +2,9 @@
id: SHARD-WP-0011
type: workplan
title: "incremental union maintenance + equivalence index + I-2 verification"
domain: whynot
domain: consumer
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -41,7 +41,7 @@ deployment is later.
```task
id: SHARD-WP-0011-T1
status: todo
status: done
priority: high
state_hub_task_id: "842f480b-7b14-47cd-818b-012dbda9c187"
```
@@ -55,7 +55,7 @@ unrelated pages don't; verified edges match a brute-force oracle on a small corp
```task
id: SHARD-WP-0011-T2
status: todo
status: done
priority: high
state_hub_task_id: "2da4e0b8-22cc-4ad1-a9aa-b5e991515d30"
```
@@ -70,7 +70,7 @@ stale edge.
```task
id: SHARD-WP-0011-T3
status: todo
status: done
priority: high
state_hub_task_id: "b602ce31-ad9a-4c7f-b596-f039722373fc"
```
@@ -85,7 +85,7 @@ equivalent event orders.
```task
id: SHARD-WP-0011-T4
status: todo
status: done
priority: medium
state_hub_task_id: "2f3d083c-0b2e-4b58-9e96-c0461c5eb089"
```

View File

@@ -2,9 +2,9 @@
id: SHARD-WP-0012
type: workplan
title: "second adapter — git-IS-store shard (contract validation on a new substrate)"
domain: whynot
domain: consumer
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -40,7 +40,7 @@ merge beyond fast-forward (apply-under-drift refuse is enough, as in SHARD-WP-00
```task
id: SHARD-WP-0012-T1
status: todo
status: done
priority: high
state_hub_task_id: "8a1c7c80-a0cc-4e02-a611-1f1fd7dec57b"
```
@@ -54,7 +54,7 @@ implication rules. Tests: read tracked files; profile validates; conformance rea
```task
id: SHARD-WP-0012-T2
status: todo
status: done
priority: high
state_hub_task_id: "b47dfb86-46c1-4e97-a62f-377719499ff2"
```
@@ -68,7 +68,7 @@ changes after an external commit.
```task
id: SHARD-WP-0012-T3
status: todo
status: done
priority: medium
state_hub_task_id: "4c895f42-671d-4948-8bdf-941fd85644bb"
```

View File

@@ -2,9 +2,9 @@
id: SHARD-WP-0013
type: workplan
title: "wiki-engine prep — reuse-surface registration, UC-catalog systematization, WikiEngineCoreArchitecture"
domain: whynot
domain: consumer
repo: shard-wiki
status: active
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
@@ -123,7 +123,7 @@ state hub").
```task
id: SHARD-WP-0013-T4
status: todo
status: done
priority: high
state_hub_task_id: "1d0ef72b-2762-4086-9848-fde3b48c8454"
```
@@ -140,7 +140,7 @@ auth-in-core amendment.
```task
id: SHARD-WP-0013-T5
status: todo
status: done
priority: high
state_hub_task_id: "4712bbfe-4ff3-4631-a9fb-e8857e1c0a2c"
```
@@ -163,7 +163,7 @@ exotic case possible (mirror the CoreArchitectureBlueprint discipline).
```task
id: SHARD-WP-0013-T6
status: todo
status: done
priority: medium
state_hub_task_id: "1c383414-2c8b-41c4-957d-8d1a9ed88143"
```

View File

@@ -0,0 +1,155 @@
---
id: SHARD-WP-0014
type: workplan
title: "wiki-engine implementation — kernel + typed-extension runtime + activation"
domain: consumer
repo: shard-wiki
status: done
owner: tegwick
topic_slug: whynot
created: "2026-06-15"
updated: "2026-06-15"
depends_on:
- SHARD-WP-0007
- SHARD-WP-0013
state_hub_workstream_id: "bfce1644-d93d-44c7-af2c-6b0cb50cedd4"
---
# SHARD-WP-0014 — Wiki-engine implementation
## Goal
Implement the native **headless wiki engine** specified in `spec/WikiEngineCoreArchitecture.md`:
a **small page-store kernel** + a **stringent typed-extension runtime**, with **per-shard
activation** (ADR-0001: via feature-control/OpenFeature, LocalProvider default), the engine's
**§A capability profile derived from active extensions** (E-5), and exposure as a
**canonical-mode shard** (`EngineShardAdapter`). Target capability: **stand up an engine shard,
activate a chosen extension set, and attach it to an `InformationSpace` — its declared
capability profile reflecting exactly what is active** — proven end-to-end with one real
built-in extension.
**Non-goal (this slice):** the headless network API protocol; git-IS-store backing (kernel uses
the existing simple store now; git backing integrates with SHARD-WP-0009/0012 later);
computational extensions; the full feature-control control plane. Build the framework, prove it.
## Guiding rules (from WikiEngineCoreArchitecture)
- E-1 engine is **one shard**, not a federation layer. E-2 small kernel. E-3 everything-else is a
typed extension. E-4 per-shard activation. E-5 capability profile derived from active
extensions. E-8 reuse (feature-control activation; model/provenance/coordination/adapters).
E-9 extensions are typed + conformance-verified.
- Honour the §11 dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`,
`adapters/`, `policy/`; it is consumed only via its `EngineShardAdapter`. No orchestrator-tier
(`union/`, `projection/`) import. Pure-stdlib core; OpenFeature is an optional engine extra.
---
## Engine kernel skeleton
```task
id: SHARD-WP-0014-T1
status: done
priority: high
state_hub_task_id: "e81ba881-7e92-4581-99ff-b12ad2bcabb3"
```
`src/shard_wiki/engine/kernel.py`: the minimal kernel — a page store + lifecycle over existing
primitives (reuse `model.Page`/`Identity`/`provenance`; a simple in-memory/folder-backed store
now, git-IS-store later) and a recoverable history hook into the decision log. Kernel covers the
c2-minimum (create/read/edit-as-history; `[[wikilink]]`+red-link resolution can be a thin kernel
helper). No extensions yet. Tests: page CRUD-as-history; kernel-only shard works.
## Typed-extension runtime
```task
id: SHARD-WP-0014-T2
status: done
priority: high
state_hub_task_id: "8ae8e58a-f081-432b-b2c5-b6435fbf3843"
```
`src/shard_wiki/engine/extension.py`: the `Extension` contract (id, provides, types, hooks,
depends_on, conflicts_with, config), a registry, a **typed hook dispatcher** (typed
inputs/outputs, declared deterministic order), a **type checker**, and **composition** that
builds the dependency closure and **rejects impossible profiles** (conflicts / unmet deps /
incompatible types) — the §6.5 discipline for extensions. Extensions ship a conformance check
(mirrors §6.6). Tests: register/compose; deterministic hook order; impossible profile rejected;
conformance catches a lying extension.
## Per-shard activation (feature-control / OpenFeature)
```task
id: SHARD-WP-0014-T3
status: done
priority: high
state_hub_task_id: "c4fe9df4-e6a8-4b7d-891b-59ceec6aebac"
```
`src/shard_wiki/engine/activation.py` (ADR-0001): resolve a shard's **activation profile**
(`{extension id → config}`) through an OpenFeature-shaped client with a **static LocalProvider
default** (standalone, zero external dep); context = `{tenant_id: root, shard_id, …}`; OpenFeature/
feature-control is an optional provider plugged in when present (degrade gracefully, mirror the
identity ladder). **Availability only — never authorization.** Tests: LocalProvider activates a
subset; absent-provider falls back to defaults; context scoping works.
## Capability profile derived from active extensions (E-5)
```task
id: SHARD-WP-0014-T4
status: done
priority: high
state_hub_task_id: "15fc8db7-cd80-4675-b387-81aa9bc7d308"
```
`src/shard_wiki/engine/profile.py`: fold the active extensions' `on_profile` contributions into a
§A `CapabilityProfile` (e.g. `ext.struct` active ⟹ structure spectrum rises + `structured-payload`
verb), then `validate()` + conformance. Tests: activating an extension changes the derived
profile; the derived profile is valid and conformance-passes.
## EngineShardAdapter (engine as a canonical-mode shard)
```task
id: SHARD-WP-0014-T5
status: done
priority: high
state_hub_task_id: "2fbf498c-efe9-400a-8a13-7f1b521b3534"
```
`src/shard_wiki/engine/adapter.py`: `EngineShardAdapter` implements `adapters.ShardAdapter`,
backed by the kernel + active extensions, declaring the derived profile (T4). Attach an engine
shard to an `InformationSpace` and read/resolve/edit through it like any shard. Tests +
integration: engine shard passes `assert_conformant`; attach → resolve → edit works.
## First built-in extension (prove the framework end-to-end)
```task
id: SHARD-WP-0014-T6
status: done
priority: medium
state_hub_task_id: "b88d1640-9afa-4957-aec3-a7264b09494c"
```
Implement one real extension end-to-end — **`ext.views` (BackLinks)** or **`ext.struct`
(typed records)** — binding kernel hooks, declaring types, contributing to the derived profile,
activatable per shard. Integration test: with the extension OFF the capability is absent (honest
profile); ON it works and the profile reflects it. Update SCOPE + spec/README; `pytest` +
pyflakes green.
---
## Acceptance criteria
- `pytest` green, pyflakes clean; engine core pure-stdlib (OpenFeature optional, behind the
LocalProvider default).
- The vertical slice works: stand up an engine shard, activate a chosen extension set, attach to
an `InformationSpace`; the engine's declared §A profile **matches the active extensions** and
passes conformance.
- Module boundaries honour §11 (engine consumed only via `EngineShardAdapter`; no union/projection
import); E-1…E-9 respected.
- Activation is availability-only (no authz); standalone path has no external dependency.
- Each task committed; state-hub synced.
## Suggested task order
T1 kernel → T2 extension runtime → T3 activation → T4 derived profile → T5 EngineShardAdapter →
T6 first extension + integration.