generated from coulomb/repo-seed
Compare commits
105 Commits
2c978d71f0
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 776f5af5a7 | |||
| fd961c83b4 | |||
| cca5bf83c3 | |||
| def699c1eb | |||
| a4e0f52ec1 | |||
| 4231daf94f | |||
| 37681d89b6 | |||
| a8e65235a8 | |||
| d7d046cac0 | |||
| 0b3ab2086f | |||
| d85d019543 | |||
| 3a5acdcb28 | |||
| 34b0c539f3 | |||
| da540d4eea | |||
| 951b24300d | |||
| c731c96634 | |||
| f0fee65cc0 | |||
| 34432c2e15 | |||
| 45a858ead0 | |||
| b31e9bc337 | |||
| e50dcc6b5c | |||
| a165cced33 | |||
| 8393a9c55d | |||
| ff96ee0c48 | |||
| 8b353f1077 | |||
| b9bb1f7d10 | |||
| c40fa3c934 | |||
| 54c2bf2ae5 | |||
| 6d8bd837a4 | |||
| b48a99d3c2 | |||
| 9b7f86ba69 | |||
| 74142096d0 | |||
| 2100e956aa | |||
| e62560eb5a | |||
| b147d3e831 | |||
| cdcf4b09aa | |||
| b21efe307b | |||
| e18397272a | |||
| 0ee972f2e2 | |||
| bb1b54e0af | |||
| b70f1c9acc | |||
| 8de044bbde | |||
| 67d851be0b | |||
| 46ec6f2a5f | |||
| ad4a2dbf5a | |||
| 23bc597343 | |||
| ec7a1ec946 | |||
| d6d7cc555f | |||
| 1dd1def40f | |||
| 6d341cd4e6 | |||
| 1b3d4aaa39 | |||
| 6e49bd3a4b | |||
| d2c73b02d9 | |||
| 042e198286 | |||
| 3ea0cc1226 | |||
| 4be2f190a0 | |||
| 7d00ae758e | |||
| 715ab1ca00 | |||
| d797bc5ee4 | |||
| 92d5774baf | |||
| e24f0034a0 | |||
| b2ea276c00 | |||
| 9fedb31d8b | |||
| 517cf1d282 | |||
| b44b2a74a4 | |||
| 24108b65aa | |||
| 6d48a1d3e6 | |||
| 9a4e00a05a | |||
| 5a77ea879c | |||
| aca9bf30f9 | |||
| 3b4f10a349 | |||
| f2dd2e124a | |||
| 1f3aba7aad | |||
| d65f9e21f3 | |||
| 802a80231a | |||
| 40575045ca | |||
| a2f15806f5 | |||
| 9b5fceb451 | |||
| d39e7f7765 | |||
| 1ad70a9c8a | |||
| 3a753a6f3b | |||
| 08a2148079 | |||
| cbd29e0a32 | |||
| 079652b61f | |||
| 84ab25eeb9 | |||
| 229d24d1cf | |||
| f21b7b5259 | |||
| c895d33091 | |||
| 59c36ac9d1 | |||
| 012d151fe8 | |||
| af840576e9 | |||
| 0383d78440 | |||
| dc451b0f4e | |||
| 04be66161e | |||
| c8dbe9b573 | |||
| b5d2cbc330 | |||
| ee2449d987 | |||
| f1dc7aff61 | |||
| dd812abb81 | |||
| 9b5b393519 | |||
| 6a88eb7710 | |||
| d898307b7e | |||
| 90a854f841 | |||
| 3d137c96b6 | |||
| f2f9f31df8 |
@@ -1,21 +0,0 @@
|
|||||||
---
|
|
||||||
active: true
|
|
||||||
iteration: 1
|
|
||||||
session_id: b24b53ca-a25b-4a18-8331-925c4100d39f
|
|
||||||
max_iterations: 20
|
|
||||||
completion_promise: "HEUREKA"
|
|
||||||
workplan_id: SHARD-WP-0004
|
|
||||||
workplan_file: workplans/SHARD-WP-0004-computational-knowledge-systems.md
|
|
||||||
started_at: "2026-06-14T20:35:07Z"
|
|
||||||
---
|
|
||||||
|
|
||||||
Read the workplan at `workplans/SHARD-WP-0004-computational-knowledge-systems.md`.
|
|
||||||
|
|
||||||
If every task has `status: done` AND frontmatter `status: done`:
|
|
||||||
run `rm -f .claude/ralph-loop.local.md` first (deactivates the loop so the stop hook exits cleanly),
|
|
||||||
then output <promise>HEUREKA</promise>.
|
|
||||||
|
|
||||||
Otherwise implement the next `todo` task as described in the workplan.
|
|
||||||
Set task `in_progress` when starting, `done` when complete.
|
|
||||||
When all tasks are done set frontmatter `status: done`.
|
|
||||||
|
|
||||||
20
.claude/rules/agents.md
Normal file
20
.claude/rules/agents.md
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
## Kaizen Agents
|
||||||
|
|
||||||
|
Specialized agent personas available on demand via the state-hub MCP.
|
||||||
|
|
||||||
|
**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category
|
||||||
|
**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them
|
||||||
|
|
||||||
|
Common agents:
|
||||||
|
|
||||||
|
| Agent | Category | When to use |
|
||||||
|
|-------|----------|-------------|
|
||||||
|
| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature |
|
||||||
|
| `code-refactoring` | quality | Code quality analysis and safe refactoring |
|
||||||
|
| `test-maintenance` | testing | Diagnose and fix failing tests |
|
||||||
|
| `requirements-engineering` | process | Prevent interface/mock mismatches upfront |
|
||||||
|
| `keepaTodofile` | process | Maintain TODO.md during work |
|
||||||
|
| `project-management` | process | Track status, determine next steps |
|
||||||
|
| `datamodel-optimization` | quality | Optimize dataclasses and data structures |
|
||||||
|
|
||||||
|
All 17 agents: call `list_kaizen_agents()` for the full list.
|
||||||
8
.claude/rules/architecture.md
Normal file
8
.claude/rules/architecture.md
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
## Architecture
|
||||||
|
|
||||||
|
<!-- TODO: Describe the key design decisions and component structure.
|
||||||
|
Key modules, data flows, external integrations, state machines, etc. -->
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference
|
||||||
50
.claude/rules/credential-routing.md
Normal file
50
.claude/rules/credential-routing.md
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
# Credential and access routing
|
||||||
|
|
||||||
|
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||||
|
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||||
|
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||||
|
|
||||||
|
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||||
|
other credential need belongs to another subsystem. **Do not** message
|
||||||
|
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||||
|
|
||||||
|
### Lookup (do this first)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
warden route find "<describe your need>" --json
|
||||||
|
warden route show <catalog-id> --json
|
||||||
|
```
|
||||||
|
|
||||||
|
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||||
|
|
||||||
|
| Agent runtime | How to orient |
|
||||||
|
| --- | --- |
|
||||||
|
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
|
||||||
|
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||||
|
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||||
|
|
||||||
|
### Quick routing table
|
||||||
|
|
||||||
|
| I need… | Owner | ops-warden executes? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||||
|
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||||
|
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||||
|
| Authorization decision | flex-auth | No — route only |
|
||||||
|
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||||
|
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||||
|
|
||||||
|
### Anti-patterns (do not do these)
|
||||||
|
|
||||||
|
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||||
|
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||||
|
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||||
|
|
||||||
|
### Other capabilities (reuse-surface)
|
||||||
|
|
||||||
|
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||||
|
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||||
|
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||||
|
get wrong.
|
||||||
|
|
||||||
|
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||||
38
.claude/rules/first-session.md
Normal file
38
.claude/rules/first-session.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
## First Session Protocol
|
||||||
|
|
||||||
|
Triggered when `get_domain_summary("consumer")` shows **no workstreams**.
|
||||||
|
The project is registered but work has not yet been structured.
|
||||||
|
|
||||||
|
**Step 1 — Read, don't write**
|
||||||
|
- `~/the-custodian/canon/projects/consumer/project_charter_v0.1.md` — purpose, scope
|
||||||
|
- `~/the-custodian/canon/projects/consumer/roadmap_v0.1.md` — planned phases
|
||||||
|
- Scan repo root: README, directory structure, existing code or docs
|
||||||
|
|
||||||
|
**Step 2 — Survey in-progress work**
|
||||||
|
Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete.
|
||||||
|
|
||||||
|
**Step 3 — Propose workstreams to Bernd**
|
||||||
|
Propose 1–3 workstreams — each a coherent strand, weeks to months, anchored to a
|
||||||
|
roadmap phase. **Wait for approval before creating.**
|
||||||
|
|
||||||
|
**Step 4 — Create workplan file first, then DB record (ADR-001)**
|
||||||
|
```
|
||||||
|
workplans/SHARD-WP-NNNN-<slug>.md ← write this first
|
||||||
|
```
|
||||||
|
Then register in the hub:
|
||||||
|
```
|
||||||
|
create_workstream(topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", title="...", owner="...", description="...")
|
||||||
|
create_task(workstream_id="<id>", title="...", priority="high|medium|low")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 5 — Record the setup**
|
||||||
|
```
|
||||||
|
add_progress_event(
|
||||||
|
summary="First session: structured consumer into N workstreams, M tasks",
|
||||||
|
event_type="milestone",
|
||||||
|
topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0",
|
||||||
|
detail={"workstreams": [...], "tasks_created": M}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
<!-- Delete or archive this file once past first session -->
|
||||||
8
.claude/rules/repo-boundary.md
Normal file
8
.claude/rules/repo-boundary.md
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
## Repo boundary
|
||||||
|
|
||||||
|
This repo owns **shard-wiki** only. It does not own:
|
||||||
|
|
||||||
|
<!-- TODO: List what belongs in adjacent repos, e.g.:
|
||||||
|
- SSH key management → railiance-infra/
|
||||||
|
- State hub code → state-hub/
|
||||||
|
-->
|
||||||
5
.claude/rules/repo-identity.md
Normal file
5
.claude/rules/repo-identity.md
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
|
||||||
|
|
||||||
|
**Domain:** consumer
|
||||||
|
**Repo slug:** shard-wiki
|
||||||
|
**Topic ID:** 4c2e5315-2cb9-447c-9d16-a39bdb0aabd0
|
||||||
85
.claude/rules/session-protocol.md
Normal file
85
.claude/rules/session-protocol.md
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
## Session Protocol
|
||||||
|
|
||||||
|
Dev Hub (State Hub API): http://127.0.0.1:8000
|
||||||
|
MCP server name in `~/.claude.json`: `dev-hub`
|
||||||
|
|
||||||
|
**Step 1 — Orient**
|
||||||
|
|
||||||
|
Read the offline-safe brief first — it works without a live hub connection:
|
||||||
|
```bash
|
||||||
|
cat .custodian-brief.md
|
||||||
|
```
|
||||||
|
Then call the MCP tool for richer cross-domain context when MCP tools are exposed:
|
||||||
|
```
|
||||||
|
get_domain_summary("consumer")
|
||||||
|
```
|
||||||
|
If MCP tools are unavailable in the current agent session, use the REST API:
|
||||||
|
```bash
|
||||||
|
curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool
|
||||||
|
```
|
||||||
|
If the hub is offline: `cd ~/state-hub && make api`
|
||||||
|
|
||||||
|
**Step 2 — Check inbox**
|
||||||
|
With MCP tools:
|
||||||
|
```
|
||||||
|
get_messages(to_agent="shard-wiki", unread_only=True)
|
||||||
|
```
|
||||||
|
Mark read with `mark_message_read(message_id)`. Reply or act on coordination
|
||||||
|
requests before proceeding.
|
||||||
|
|
||||||
|
Without MCP tools:
|
||||||
|
```bash
|
||||||
|
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
|
||||||
|
| python3 -m json.tool
|
||||||
|
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||||
|
-H "Content-Type: application/json" -d '{}'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Step 3 — Scan workplans**
|
||||||
|
```bash
|
||||||
|
ls workplans/
|
||||||
|
```
|
||||||
|
For each file with `status: ready`, `active`, or `blocked`, note pending
|
||||||
|
`wait`/`todo`/`progress` tasks.
|
||||||
|
|
||||||
|
**Step 4 — Present brief**
|
||||||
|
|
||||||
|
1. **Active workstreams** for `consumer` — title, task counts, blocking decisions
|
||||||
|
2. **Pending tasks** from `workplans/` + any `[repo:shard-wiki]` hub tasks
|
||||||
|
3. **Goal guidance** — if `goal_guidance` in summary:
|
||||||
|
- `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"*
|
||||||
|
- `alignment_warnings`: flag if active work is not aligned with current goal
|
||||||
|
4. **Suggested next action** — highest-priority open item
|
||||||
|
5. **SBOM status** — flag if `last_sbom_at` is unset for this repo
|
||||||
|
|
||||||
|
If no workstreams: follow First Session Protocol (`first-session.md`).
|
||||||
|
|
||||||
|
**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()`
|
||||||
|
|
||||||
|
> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`)
|
||||||
|
> are First Session Protocol only. Work structure belongs in repo files (ADR-001).
|
||||||
|
|
||||||
|
**Session close:**
|
||||||
|
With MCP tools:
|
||||||
|
```
|
||||||
|
add_progress_event(summary="...", topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", workstream_id="<uuid>")
|
||||||
|
```
|
||||||
|
Without MCP tools:
|
||||||
|
```bash
|
||||||
|
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"topic_id":"4c2e5315-2cb9-447c-9d16-a39bdb0aabd0","workstream_id":"<uuid>","event_type":"note","summary":"what changed","author":"codex"}'
|
||||||
|
```
|
||||||
|
If workplan files were modified, ensure the local copy is up to date first:
|
||||||
|
```bash
|
||||||
|
git -C <repo_path> pull --ff-only
|
||||||
|
cd ~/state-hub && make fix-consistency REPO=shard-wiki
|
||||||
|
```
|
||||||
|
For repos where implementation runs on a remote machine (e.g. CoulombCore),
|
||||||
|
use the combined target which pulls before fixing:
|
||||||
|
```bash
|
||||||
|
cd ~/state-hub && make fix-consistency-remote REPO=shard-wiki
|
||||||
|
```
|
||||||
|
**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback
|
||||||
|
will sync the file to match DB. **C-16** (repo behind remote) blocks all writes
|
||||||
|
until you pull — intentional to prevent clobbering remote progress.
|
||||||
19
.claude/rules/stack-and-commands.md
Normal file
19
.claude/rules/stack-and-commands.md
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
## Stack
|
||||||
|
|
||||||
|
<!-- TODO: Fill in language, frameworks, and key dependencies -->
|
||||||
|
- **Language:**
|
||||||
|
- **Key deps:**
|
||||||
|
|
||||||
|
## Dev Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# TODO: Fill in the standard commands for this repo
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
|
||||||
|
# Lint / type check
|
||||||
|
|
||||||
|
# Build / package (if applicable)
|
||||||
|
```
|
||||||
40
.claude/rules/workplan-convention.md
Normal file
40
.claude/rules/workplan-convention.md
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
## Workplan Convention (ADR-001)
|
||||||
|
|
||||||
|
File location: `workplans/SHARD-WP-NNNN-<slug>.md`
|
||||||
|
ID prefix: `SHARD-WP-`
|
||||||
|
|
||||||
|
Work items originate as files in this repo **before** being registered in the hub.
|
||||||
|
|
||||||
|
Canonical workplan/workstream frontmatter statuses are:
|
||||||
|
`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`.
|
||||||
|
Use `proposed` for a newly drafted plan, `ready` after review against current
|
||||||
|
repo state, and `finished` when implementation is complete. `stalled` and
|
||||||
|
`needs_review` are derived health labels, not stored statuses.
|
||||||
|
|
||||||
|
Closed workplans may be moved to `workplans/archived/` with a completion-date
|
||||||
|
prefix: `YYMMDD-SHARD-WP-NNNN-<slug>.md`. The frontmatter id remains
|
||||||
|
unchanged; the prefix is only for quick visual reference.
|
||||||
|
|
||||||
|
Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**:
|
||||||
|
`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids
|
||||||
|
`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed
|
||||||
|
directly. Promote anything requiring analysis, design, approval, dependencies, or
|
||||||
|
multiple planned phases into a normal workplan.
|
||||||
|
|
||||||
|
Ecosystem todos from other agents arrive as `[repo:shard-wiki]` hub tasks —
|
||||||
|
visible at session start. Pick one up by creating the workplan file, then registering
|
||||||
|
the workstream.
|
||||||
|
|
||||||
|
Task blocks use this shape:
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: SHARD-WP-NNNN-T01
|
||||||
|
status: wait | todo | progress | done | cancel
|
||||||
|
priority: high | medium | low
|
||||||
|
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||||
|
```
|
||||||
|
|
||||||
|
Status progression is `todo` → `progress` → `done`; use `wait` for waiting or
|
||||||
|
blocked work and `cancel` for stopped work.
|
||||||
|
|
||||||
|
<!-- Ralph Loop rules and HEUREKA sequence: ~/.claude/CLAUDE.md — do not duplicate here -->
|
||||||
@@ -2,41 +2,46 @@
|
|||||||
# Custodian Brief — shard-wiki
|
# Custodian Brief — shard-wiki
|
||||||
|
|
||||||
**Domain:** whynot
|
**Domain:** whynot
|
||||||
**Last synced:** 2026-06-14 21:55 UTC
|
**Last synced:** 2026-06-15 22:57 UTC
|
||||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||||
|
|
||||||
## Active Workstreams
|
## Active Workstreams
|
||||||
|
|
||||||
### computational / interactive-knowledge systems research
|
### second adapter — git-IS-store shard (contract validation on a new substrate)
|
||||||
Progress: 6/8 done | workstream_id: `5fc3911a-1c68-4826-bbd2-b892dec8f981`
|
Progress: 0/3 done | workstream_id: `9e24eeb0-c0f0-41e6-a1ca-88d71e4139ea`
|
||||||
|
|
||||||
**Open tasks:**
|
**Open tasks:**
|
||||||
- · Processing.org and Processing.js `0f54cb0e`
|
- · GitShardAdapter — read over a git working tree/repo `8a1c7c80`
|
||||||
- · Strudel.cc — live-coding REPL `2b099639`
|
- · Write = commit; current_rev = sha (drift) `b47dfb86`
|
||||||
|
- · History adopt + integration with union/overlay `4c895f42`
|
||||||
|
|
||||||
### federation architecture design
|
### incremental union maintenance + equivalence index + I-2 verification
|
||||||
Progress: 0/16 done | workstream_id: `2af4c46d-cbfd-40ea-a94b-d9e60b0f9945`
|
Progress: 0/4 done | workstream_id: `78d48bcf-6482-4266-bc81-084b7ec1cd80`
|
||||||
|
|
||||||
**Open tasks:**
|
**Open tasks:**
|
||||||
- · Architecture positioning and boundaries `ea8fdb22`
|
- · Equivalence index: blocking + verify `842f480b`
|
||||||
- · Remix primitives: fork, overlay, import, reference `fb7d4bce`
|
- · Incremental maintenance (delta, not additive) `2da4e0b8`
|
||||||
- · Equivalent page identity and multi-version presentation `8f2a333d`
|
- · I-2 verification: digest + consistency-checker `b602ce31`
|
||||||
- · History, attribution, and coordination journal `5f39f48d`
|
- · Wire incremental tier behind resolution + views `2f3d083c`
|
||||||
- · Union composition layer `3ff71e11`
|
|
||||||
- · Change notification and subscription transports `9596e5e8`
|
|
||||||
- · Information space lifecycle `38134064`
|
|
||||||
- … and 9 more open tasks
|
|
||||||
|
|
||||||
### shard-wiki requirements from yawex prior art
|
### derived views — wikilinks, BackLinks, RecentChanges, AllPages/SiteMap
|
||||||
Progress: 0/6 done | workstream_id: `0ed023a2-760b-4990-b931-8ee1f41ea08f`
|
Progress: 0/5 done | workstream_id: `2fe15330-ddf6-4b0f-8e55-ada341375d35`
|
||||||
|
|
||||||
**Open tasks:**
|
**Open tasks:**
|
||||||
- · Design federation page-resolution model (yawex state space as inspiration) `ebc036e4`
|
- · Wikilink + red-link model `792660c3`
|
||||||
- · Define namespace/path model and page+shard roles `431b4d28`
|
- · BackLinks (core) `431a54c3`
|
||||||
- · Specify union-level derived views (BackLinks, RecentChanges, AllPages, SiteMap, Search) `564545ec`
|
- · RecentChanges (core) `270c1c31`
|
||||||
- · Provenance & freshness model for pages/revisions/projections `738326f5`
|
- · AllPages / SiteMap (core) `898ba43e`
|
||||||
- · Overlay / lightweight-patch model (from yawex append/comment) `a268de6a`
|
- · Wiring + integration `7157544b`
|
||||||
- · Markdown link semantics: wikilink + red-link extension `a7499f3e`
|
|
||||||
|
### git-backed DecisionLog + per-space append authority
|
||||||
|
Progress: 0/4 done | workstream_id: `4fb5b29b-955c-4f37-85cf-58b4643ab1ca`
|
||||||
|
|
||||||
|
**Open tasks:**
|
||||||
|
- · Git event-store backend (append = commit/object) `a8fcbb3e`
|
||||||
|
- · Per-space append authority (lease) `62abd162`
|
||||||
|
- · Fold over the git log + read-your-writes across processes `8cc3691e`
|
||||||
|
- · Migration + wiring `281e1db4`
|
||||||
|
|
||||||
---
|
---
|
||||||
## MCP Orientation (when available)
|
## MCP Orientation (when available)
|
||||||
|
|||||||
17
.repo-classification.yaml
Normal file
17
.repo-classification.yaml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
repo_classification:
|
||||||
|
standard: Repo Classification Standard
|
||||||
|
version: '1.0'
|
||||||
|
classified_at: '2026-06-22'
|
||||||
|
classified_by: agent
|
||||||
|
category: project
|
||||||
|
domain: consumer
|
||||||
|
secondary_domains: []
|
||||||
|
capability_tags:
|
||||||
|
- knowledge
|
||||||
|
- documentation
|
||||||
|
business_stake:
|
||||||
|
- product
|
||||||
|
- experience
|
||||||
|
business_mechanics:
|
||||||
|
- coordination
|
||||||
|
- operation
|
||||||
243
AGENTS.md
243
AGENTS.md
@@ -1,62 +1,219 @@
|
|||||||
# AGENTS.md
|
# shard-wiki — Agent Instructions
|
||||||
|
|
||||||
Guidance for agents working in `shard-wiki`.
|
## Repo Identity
|
||||||
|
|
||||||
## Read First
|
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
|
||||||
|
|
||||||
1. `INTENT.md` — aspiration and boundaries (stable; architectural changes are rare).
|
**Domain:** consumer
|
||||||
2. `SCOPE.md` — what we are achieving now and current maturity.
|
**Repo slug:** shard-wiki
|
||||||
3. `.custodian-brief.md` — State Hub snapshot (generated; do not edit manually).
|
**Topic ID:** `4c2e5315-2cb9-447c-9d16-a39bdb0aabd0`
|
||||||
|
**Workplan prefix:** `SHARD-WP-`
|
||||||
|
|
||||||
## Documentation Layout
|
---
|
||||||
|
|
||||||
This repo follows the CoulombSocial / HelixForge / MarkiTect documentation
|
## State Hub Integration
|
||||||
layout (recommendation, not strict law). Efficient retrieval by purpose:
|
|
||||||
|
|
||||||
| Path | Purpose |
|
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
|
||||||
|------|---------|
|
there is no MCP server for Codex agents.
|
||||||
| `INTENT.md` | Aspiration and boundaries |
|
|
||||||
| `SCOPE.md` | Top-level view of current achievement; closes gap to INTENT |
|
|
||||||
| `research/` | Exploration results (`yymmdd-` prefix on files or subdirs) |
|
|
||||||
| `demand/` | Inbound requests not yet reviewed into spec or workplans |
|
|
||||||
| `spec/` | Implementation guardrails (PRD, TSD, use cases, architecture) |
|
|
||||||
| `workplans/` | State Hub–registered implementation tasks |
|
|
||||||
| `docs/` | Stakeholder documentation (users, developers, humans, agents) |
|
|
||||||
| `wiki/` | Perspective-free interconnected knowledge (wiki UI when connected) |
|
|
||||||
| `issues/` | Mirror of relevant open tickets when ticket systems are in use |
|
|
||||||
| `history/` | Archived material (`yymmdd-` prefix); out of scope for daily work |
|
|
||||||
|
|
||||||
**Mode of operation:** close SCOPE → INTENT while learning; refine both as needed.
|
| Context | URL |
|
||||||
|
|---------|-----|
|
||||||
|
| Local workstation | `http://127.0.0.1:8000` |
|
||||||
|
| Remote via tunnel | `http://127.0.0.1:18000` |
|
||||||
|
|
||||||
## Domain Vocabulary
|
### Orient at session start
|
||||||
|
|
||||||
Honor terms from `INTENT.md`: shard, root entity, adapter contract, projection,
|
|
||||||
overlay, coordination journal, shard modes. Do not invent parallel vocabulary.
|
|
||||||
|
|
||||||
## Build And Test
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -e ".[dev]"
|
# Offline brief — works without hub connection
|
||||||
pytest
|
cat .custodian-brief.md
|
||||||
ruff check
|
|
||||||
ruff format
|
# Active workstreams for this domain
|
||||||
|
curl -s "http://127.0.0.1:8000/workstreams/?topic_id=4c2e5315-2cb9-447c-9d16-a39bdb0aabd0&status=active" \
|
||||||
|
| python3 -m json.tool
|
||||||
|
|
||||||
|
# Check inbox
|
||||||
|
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
|
||||||
|
| python3 -m json.tool
|
||||||
```
|
```
|
||||||
|
|
||||||
## State Hub
|
Mark a message read:
|
||||||
|
```bash
|
||||||
|
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||||
|
-H "Content-Type: application/json" -d '{}'
|
||||||
|
```
|
||||||
|
|
||||||
Workplans register with State Hub. After workplan changes:
|
### Log progress (required at session close)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd ~/state-hub && make fix-consistency REPO=shard-wiki
|
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"summary": "what was done",
|
||||||
|
"event_type": "note",
|
||||||
|
"author": "codex",
|
||||||
|
"workstream_id": "<uuid>",
|
||||||
|
"task_id": "<uuid>"
|
||||||
|
}'
|
||||||
```
|
```
|
||||||
|
|
||||||
Finished or canceled workplans move to `history/` with a `yymmdd-` archive prefix.
|
Omit `workstream_id` / `task_id` when not applicable.
|
||||||
|
|
||||||
## Where To Put New Material
|
### Update task status
|
||||||
|
|
||||||
- Exploratory analysis → `research/yymmdd-<topic>/`
|
```bash
|
||||||
- Raw feature ask or external requirement → `demand/`
|
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||||
- Reviewed design ready to guide code → `spec/`
|
-H "Content-Type: application/json" \
|
||||||
- Implementation tasks → `workplans/`
|
-d '{"status": "progress"}'
|
||||||
- User/dev/agent how-to → `docs/`
|
# values: wait | todo | progress | done | cancel
|
||||||
- Collaborative unstructured notes → `wiki/`
|
```
|
||||||
|
|
||||||
|
### Flag a task for human review
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"needs_human": true, "intervention_note": "reason"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session Protocol
|
||||||
|
|
||||||
|
**Start:**
|
||||||
|
1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe)
|
||||||
|
2. Check inbox: `GET /messages/?to_agent=shard-wiki&unread_only=true`; mark read
|
||||||
|
3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks
|
||||||
|
4. Check human-needed tasks: `GET /tasks/?needs_human=true`
|
||||||
|
|
||||||
|
**During work:**
|
||||||
|
- Update task statuses in workplan files as tasks progress
|
||||||
|
- Record significant decisions via `POST /decisions/`
|
||||||
|
|
||||||
|
**Close:**
|
||||||
|
1. Update workplan file task statuses to reflect progress
|
||||||
|
2. Log: `POST /progress/` with a summary of what changed
|
||||||
|
3. Note for the custodian operator: after workplan file changes, run from
|
||||||
|
`~/state-hub`:
|
||||||
|
```bash
|
||||||
|
make fix-consistency REPO=shard-wiki
|
||||||
|
```
|
||||||
|
This syncs task status from files into the hub DB.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credential and access routing
|
||||||
|
|
||||||
|
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||||
|
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||||
|
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||||
|
|
||||||
|
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||||
|
other credential need belongs to another subsystem. **Do not** message
|
||||||
|
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||||
|
|
||||||
|
### Lookup (do this first)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
warden route find "<describe your need>" --json
|
||||||
|
warden route show <catalog-id> --json
|
||||||
|
```
|
||||||
|
|
||||||
|
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||||
|
|
||||||
|
| Agent runtime | How to orient |
|
||||||
|
| --- | --- |
|
||||||
|
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
|
||||||
|
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||||
|
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||||
|
|
||||||
|
### Quick routing table
|
||||||
|
|
||||||
|
| I need… | Owner | ops-warden executes? |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||||
|
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||||
|
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||||
|
| Authorization decision | flex-auth | No — route only |
|
||||||
|
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||||
|
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||||
|
|
||||||
|
### Anti-patterns (do not do these)
|
||||||
|
|
||||||
|
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||||
|
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||||
|
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||||
|
|
||||||
|
### Other capabilities (reuse-surface)
|
||||||
|
|
||||||
|
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||||
|
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||||
|
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||||
|
get wrong.
|
||||||
|
|
||||||
|
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||||
|
|
||||||
|
<!-- REPO-AGENTS-EXTENSIONS -->
|
||||||
|
<!-- Append repo-specific agent instructions below this marker.
|
||||||
|
The state-hub template sync preserves content after this line. -->
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Workplan Convention (ADR-001)
|
||||||
|
|
||||||
|
Work items originate as files in this repo — not in the hub. The hub is a
|
||||||
|
read/cache/index layer that rebuilds from files.
|
||||||
|
|
||||||
|
**File location:** `workplans/SHARD-WP-NNNN-<slug>.md`
|
||||||
|
|
||||||
|
**Archived location:** finished workplans may move to
|
||||||
|
`workplans/archived/YYMMDD-SHARD-WP-NNNN-<slug>.md`. The `YYMMDD` prefix is
|
||||||
|
the completion/archive date; the frontmatter `id` does not change.
|
||||||
|
|
||||||
|
**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use
|
||||||
|
`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use
|
||||||
|
this only for low-risk work completed directly; create a normal workplan for
|
||||||
|
anything needing analysis, design, approval, dependencies, or multiple phases.
|
||||||
|
|
||||||
|
**Frontmatter:**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
id: SHARD-WP-NNNN
|
||||||
|
type: workplan
|
||||||
|
title: "..."
|
||||||
|
domain: consumer
|
||||||
|
repo: shard-wiki
|
||||||
|
status: proposed | ready | active | blocked | backlog | finished | archived
|
||||||
|
owner: codex
|
||||||
|
topic_slug: ...
|
||||||
|
created: "YYYY-MM-DD"
|
||||||
|
updated: "YYYY-MM-DD"
|
||||||
|
state_hub_workstream_id: "<uuid>" # written by fix-consistency — do not edit
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `proposed` for a new draft, `ready` after review against current repo
|
||||||
|
state, and `finished` after implementation. `stalled` and `needs_review` are
|
||||||
|
derived health labels, not frontmatter statuses.
|
||||||
|
|
||||||
|
**Task block format** (one per `##` section):
|
||||||
|
|
||||||
|
```
|
||||||
|
## Task Title
|
||||||
|
|
||||||
|
` ` `task
|
||||||
|
id: SHARD-WP-NNNN-T01
|
||||||
|
status: wait | todo | progress | done | cancel
|
||||||
|
priority: high | medium | low
|
||||||
|
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||||
|
` ` `
|
||||||
|
|
||||||
|
Task description text.
|
||||||
|
```
|
||||||
|
|
||||||
|
Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work.
|
||||||
|
|
||||||
|
To create a new workplan:
|
||||||
|
1. Write the file following the format above
|
||||||
|
2. Notify the custodian operator to run `make fix-consistency REPO=shard-wiki`
|
||||||
|
(or send a message to the hub agent via `POST /messages/`)
|
||||||
|
|||||||
63
CLAUDE.md
63
CLAUDE.md
@@ -1,53 +1,12 @@
|
|||||||
# CLAUDE.md
|
# shard-wiki — Claude Code Instructions
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
@SCOPE.md
|
||||||
|
@.claude/rules/repo-identity.md
|
||||||
## Repository status
|
@.claude/rules/session-protocol.md
|
||||||
|
@.claude/rules/first-session.md
|
||||||
This is an **early-stage Python repository**. The package scaffold (`src/shard_wiki/`, `tests/`, `pyproject.toml`) exists with only smoke tests — the domain model is not yet implemented. Read `INTENT.md` (aspiration), `SCOPE.md` (current achievement), and `AGENTS.md` (layout and conventions) before designing anything. Close the gap from SCOPE to INTENT via `research/`, `spec/`, and `workplans/`.
|
@.claude/rules/workplan-convention.md
|
||||||
|
@.claude/rules/stack-and-commands.md
|
||||||
## What this project is
|
@.claude/rules/architecture.md
|
||||||
|
@.claude/rules/repo-boundary.md
|
||||||
`shard-wiki` is a **Git-based Markdown wiki orchestrator and federation layer**, not a wiki engine. It lets multiple heterogeneous wiki-shaped page stores (**shards**) attach to a shared root entity and be presented as a **union of pages**, while preserving each shard's separate storage, provenance, capabilities, and history.
|
@.claude/rules/credential-routing.md
|
||||||
|
@.claude/rules/agents.md
|
||||||
The core job is orchestration across backends — Git repos, repo subdirectories (`wiki/`), Gitea wikis, local folders, Obsidian vaults, WebDAV/Nextcloud directories, Coulomb spaces — never replacing or homogenizing them.
|
|
||||||
|
|
||||||
## Core domain model (the concepts code must honor)
|
|
||||||
|
|
||||||
These abstractions come from `INTENT.md` and define the architecture. New code should map onto them rather than inventing parallel vocabulary:
|
|
||||||
|
|
||||||
- **Shard** — an independently meaningful page store attached to a root entity. Shards have *sovereignty*: their own backend, capabilities, limits, history, and identity model. Not all shards are Git-native.
|
|
||||||
- **Root entity / information space** — the joined space that shards attach to. Each information space should have a **Git-addressable coordination layer** (history, patches, review, backup, reconciliation) even when individual shards are not Git-native.
|
|
||||||
- **Shard adapter contract** — the versioned interface a backend implements to participate. Adapters are **capability-aware**: the core must model explicitly which operations a shard supports (read, write, diff, merge, lock, version, publish, accept patches) rather than assuming uniformity.
|
|
||||||
- **Wiki page model** — a stable, versioned, Markdown-first but backend-neutral representation of pages, paths, links, metadata, revisions.
|
|
||||||
- **Projection** — a lazy, cache-like local view of remote/external shard content. Prefer lazy projection over eager copying.
|
|
||||||
- **Overlay** — a non-destructive local edit against a remote, read-only, or capability-limited shard, representable as drafts/patches/commits/merge requests *before* destructive application ("overlay before mutation").
|
|
||||||
- **Coordination journal** — the Git-backed record of change flows for an information space.
|
|
||||||
- **Shard modes** — read-only, write-through, mirrored, projected, cached, canonical.
|
|
||||||
|
|
||||||
## Design constraints to enforce in code
|
|
||||||
|
|
||||||
These are hard boundaries from `INTENT.md`; treat violations as design bugs:
|
|
||||||
|
|
||||||
- **Mechanism over policy.** Provide primitives for federation, sync, overlays, patching, conflict detection, projection, reconciliation. Do *not* hard-code one editorial/sync/conflict/canonical-source policy — keep those configurable.
|
|
||||||
- **Union without erasure.** Always preserve provenance: which shard a page came from, its freshness, whether it is cached, whether it has overlays, whether it diverges from an equivalent page elsewhere. Never hide authorship, conflicts, freshness, or backend limitations.
|
|
||||||
- **No silent remote mutation.** Do not mutate remote systems without explicit adapter support and user intent.
|
|
||||||
- **Graceful degradation.** Limited backends must still be usable as read-only/cache/projection/backup/patch targets.
|
|
||||||
- **Not a file-sync daemon.** Synchronization is wiki-page-semantic, not generic file mirroring.
|
|
||||||
|
|
||||||
`INTENT.md` has a "Stability Note": changes that redefine what a shard is, Git's role, how root entities are modeled, or whether this is an orchestrator vs. an engine are **architectural changes** and should be rare and deliberate.
|
|
||||||
|
|
||||||
## Build, test, run
|
|
||||||
|
|
||||||
Python with a `src/` layout, built via hatchling, tested with pytest. Tests run against the source tree directly (`pythonpath = ["src"]` in `pyproject.toml`), so no install/editable step is required to run them.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -e ".[dev]" # one-time: install dev tooling (pytest, pytest-cov, ruff)
|
|
||||||
pytest # run the full test suite
|
|
||||||
pytest tests/test_package.py::test_version_is_exposed # run a single test
|
|
||||||
pytest --cov # run with coverage
|
|
||||||
ruff check # lint
|
|
||||||
ruff format # format
|
|
||||||
```
|
|
||||||
|
|
||||||
Note: the system `pytest` is 7.4.x; `minversion` in `pyproject.toml` is pinned to `7.0` to match. Bump it if a newer pytest is installed into the dev environment.
|
|
||||||
|
|||||||
10
INTENT.md
10
INTENT.md
@@ -16,6 +16,8 @@ The goal is to allow independently stored and differently implemented wikis, pag
|
|||||||
|
|
||||||
The repository provides a **shard orchestration layer** for interconnected Markdown and markup-based wiki content.
|
The repository provides a **shard orchestration layer** for interconnected Markdown and markup-based wiki content.
|
||||||
|
|
||||||
|
Equivalently, shard-wiki can be used as a **headless, API-first wiki engine** — optimized for **integrating heterogeneous data sources** and for **efficient access by agents and automation** — that ships its own native engine as one (canonical-mode) shard among many. There is no bundled UI: presentation and rendering are consumer concerns.
|
||||||
|
|
||||||
It allows wiki-like systems to:
|
It allows wiki-like systems to:
|
||||||
|
|
||||||
* Attach heterogeneous page stores as shards of a shared information space
|
* Attach heterogeneous page stores as shards of a shared information space
|
||||||
@@ -30,6 +32,7 @@ It allows wiki-like systems to:
|
|||||||
* Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
|
* Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
|
||||||
* Allow existing wiki engines to become federation-capable through a shared API
|
* Allow existing wiki engines to become federation-capable through a shared API
|
||||||
* Allow non-federation-aware systems to participate through adapters and projections
|
* Allow non-federation-aware systems to participate through adapters and projections
|
||||||
|
* Serve as a **headless, API-first wiki engine** (a small typed-extension core) that integrates heterogeneous data sources and is consumed efficiently by agents and automation
|
||||||
|
|
||||||
It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a **composable federated wiki space**.
|
It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a **composable federated wiki space**.
|
||||||
|
|
||||||
@@ -85,7 +88,7 @@ A mature `shard-wiki` should allow each participating shard to see the others as
|
|||||||
|
|
||||||
This repository is **not** intended to:
|
This repository is **not** intended to:
|
||||||
|
|
||||||
* Replace all wiki engines with a single canonical wiki implementation
|
* Replace all wiki engines with a single canonical wiki implementation *(shard-wiki MAY still provide its own native, headless, API-first engine as one optional shard backend — see Design Principles — but never as a mandated or universal replacement)*
|
||||||
* Force every shard to use the same backend, database, directory layout, or storage format
|
* Force every shard to use the same backend, database, directory layout, or storage format
|
||||||
* Require every participating system to become federation-aware
|
* Require every participating system to become federation-aware
|
||||||
* Require every participating shard to be Git-native
|
* Require every participating shard to be Git-native
|
||||||
@@ -148,6 +151,9 @@ Policy decisions such as conflict preference, canonical source selection, public
|
|||||||
* **Composable integration**
|
* **Composable integration**
|
||||||
Wiki engines should be able to use the `shard-wiki` API to become federation-enabled without reimplementing federation internally.
|
Wiki engines should be able to use the `shard-wiki` API to become federation-enabled without reimplementing federation internally.
|
||||||
|
|
||||||
|
* **Native reference engine (additive, headless & API-first)**
|
||||||
|
shard-wiki MAY provide its own native wiki-engine as a **canonical-mode shard backend** — a **small core** with a **typed-extension framework**, activated **per shard** (only what you need). It is **headless and API-first** (no bundled UI; presentation/rendering are consumer concerns) and tuned for **integrating heterogeneous data sources** and **efficient agent/automation access**. It is *one shard type among many*, implemented against shard-wiki's own adapter contract; it does **not** replace other engines, mandate a single implementation, or change shard-wiki's role as an orchestrator. Shard sovereignty and union-without-erasure are preserved.
|
||||||
|
|
||||||
* **Open by default, progressively governed**
|
* **Open by default, progressively governed**
|
||||||
A standalone `shard-wiki` must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an *additive capability*, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).
|
A standalone `shard-wiki` must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an *additive capability*, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).
|
||||||
|
|
||||||
@@ -201,3 +207,5 @@ Such changes should be rare, because they affect all downstream systems relying
|
|||||||
|
|
||||||
In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether `shard-wiki` is an orchestrator rather than a wiki engine should be treated as architectural changes.
|
In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether `shard-wiki` is an orchestrator rather than a wiki engine should be treated as architectural changes.
|
||||||
|
|
||||||
|
**Amendment — 2026-06-15 (SHARD-WP-0013 T4, decision `84ffdb48`):** admits an **additive** native reference wiki-engine — **headless, API-first**, a small typed-extension core — as a **canonical-mode shard backend** optimized for data-source integration and agent access. Deliberate, narrow scope change; shard-wiki remains an orchestrator and neither mandates nor replaces other engines. (Mirrors the earlier auth-in-core amendment precedent.)
|
||||||
|
|
||||||
|
|||||||
12
SCOPE.md
12
SCOPE.md
@@ -17,12 +17,12 @@ Learnings update both SCOPE and INTENT where necessary.
|
|||||||
|
|
||||||
| Layer | State |
|
| Layer | State |
|
||||||
|-------|-------|
|
|-------|-------|
|
||||||
| Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
|
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. Incremental-first derived tier (SHARD-WP-0011): `incremental` (indexed equivalence via MinHash/LSH blocking + verify, change-driven delta maintenance with retraction/propagation, Merkle-style digest + self-healing I-2 consistency-checker, `UnionIndex` routed behind `InformationSpace.all_pages` with rebuild as explicit fallback). Second adapter (SHARD-WP-0012): `GitShardAdapter` — git-IS-store substrate (read=tracked *.md, write=commit, current_rev=per-path sha for drift, adopted git-native history), passes conformance, works across folder+git shards in union/overlay/edit with no core change (capability-as-data proven on a second substrate). 196 tests green, ~97% coverage |
|
||||||
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
||||||
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
||||||
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
||||||
| Spec | Architecture blueprint drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds |
|
| Spec | CoreArchitectureBlueprint (whole-system, hardened via SHARD-WP-0005/0006) + FederationArchitecture + FederationRequirements + TSD §A adapter contract + ArchitectureBlueprint (auth/history) + WikiEngineCoreArchitecture (headless API-first engine, SHARD-WP-0013) drafted; UseCaseCatalog 84 UCs (+ engine capability-structure layer); PRD scaffold |
|
||||||
| Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` active (8 computational-knowledge dives: T1 literate programming, T2 Mathematica, T3 Jupyter, T4 Processing, T5 Strudel, T6 Squeak + T8 Pharo (merged), T7 Glamorous Toolkit done — all 8 dives complete) |
|
| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` **done** (18 tasks → `FederationArchitecture.md` [T1–T10, T17] + `TechnicalSpecificationDocument.md` §A adapter contract [T11–T16, T18]); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) |
|
||||||
|
|
||||||
## In Scope (today)
|
## In Scope (today)
|
||||||
|
|
||||||
@@ -32,11 +32,15 @@ Learnings update both SCOPE and INTENT where necessary.
|
|||||||
- Authorization model design (delegated authentication, core authorization).
|
- Authorization model design (delegated authentication, core authorization).
|
||||||
- Shard adapter contract and wiki page model (to be specified, then implemented).
|
- Shard adapter contract and wiki page model (to be specified, then implemented).
|
||||||
- Git-backed coordination journal for information spaces.
|
- Git-backed coordination journal for information spaces.
|
||||||
|
- A **native, headless, API-first wiki-engine core** (small typed-extension core, as a
|
||||||
|
canonical-mode shard backend) — design via SHARD-WP-0013; optimized for data-source
|
||||||
|
integration and agent access.
|
||||||
- State Hub workplan registration and consistency sync.
|
- State Hub workplan registration and consistency sync.
|
||||||
|
|
||||||
## Out Of Scope (today)
|
## Out Of Scope (today)
|
||||||
|
|
||||||
- A standalone wiki engine UI or rendering pipeline.
|
- A wiki-engine **UI or rendering pipeline** (the engine is headless/API-first; presentation
|
||||||
|
is a consumer concern). A bundled standalone UI is not provided.
|
||||||
- Authentication, credential storage, or user directory implementation.
|
- Authentication, credential storage, or user directory implementation.
|
||||||
- Hard-coded editorial, sync, or conflict-resolution policy.
|
- Hard-coded editorial, sync, or conflict-resolution policy.
|
||||||
- Generic file mirroring independent of wiki-page semantics.
|
- Generic file mirroring independent of wiki-page semantics.
|
||||||
|
|||||||
91
history/260615-core-architecture-blueprint-review-2.md
Normal file
91
history/260615-core-architecture-blueprint-review-2.md
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
# Critical review (round 2) — CoreArchitectureBlueprint.md (hardened)
|
||||||
|
|
||||||
|
Date: 2026-06-15 · Reviewer: tegwick (with Claude) · Subject:
|
||||||
|
`spec/CoreArchitectureBlueprint.md` after **SHARD-WP-0005** (commit f21b7b5) · Feeds:
|
||||||
|
**SHARD-WP-0006**
|
||||||
|
|
||||||
|
A second hostile pass over the *hardened* blueprint. Round 1 found design bugs; this round
|
||||||
|
finds (a) self-consistency regressions the surgical hardening introduced, and (b) deeper
|
||||||
|
second-order gaps — some of which the hardening *sharpened*. Verdict: the architecture is now
|
||||||
|
substantially sound; what remains in §B/§C are hard distributed-systems/operational questions,
|
||||||
|
not design smells — except §A (a real regression) and three foundational gaps in §B.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## A. Self-consistency regressions introduced by surgical hardening
|
||||||
|
|
||||||
|
The 9 edits deepened §6–§9 but did not propagate to the **overview surfaces**, so the document
|
||||||
|
now contradicts itself between its summary and its body (and readers trust the summary).
|
||||||
|
|
||||||
|
- **A-1 (real contradiction).** §4 still says "Addressing, **equivalence**, and transclusion
|
||||||
|
key on identity" — the exact conflation T2 fixed in §7.2 (equivalence keys on *content
|
||||||
|
fingerprint across distinct identities*). → **WP-0006 T1**
|
||||||
|
- **A-2.** §4 "Projection — typed on two axes" and §4 "Provenance envelope … every artifact
|
||||||
|
carries [full wrapper]" are stale vs T7's §8.4 (two-axis = extension point; trivial default)
|
||||||
|
and §7.3 (layered effective-vs-own). → **T1**
|
||||||
|
- **A-3.** §10 policy surface omits knobs the hardening added (freshness/staleness §8.8,
|
||||||
|
squash-compaction §8.1, conflict-resolution preset §8.6, tenant-partition) — yet §11 defines
|
||||||
|
`policy/` as "owns the §10 surface." The module contract points at a stale list. → **T1**
|
||||||
|
- **A-4 (cosmetic).** §3 diagram + §11 header still say "L4 rebuildable cache" / "15 spectra,"
|
||||||
|
advertising the pre-hardening model (§8.7 incremental-first; §6.5 orthogonal-core). → **T1**
|
||||||
|
|
||||||
|
Meta-point: surgical editing hardened the body but regressed whole-document coherence; v2
|
||||||
|
needs an **overview-reconciliation pass**.
|
||||||
|
|
||||||
|
## B. Foundational gaps (serious; some sharpened by the hardening)
|
||||||
|
|
||||||
|
- **B-1 — The journal is now a concurrent-write DB, but it's single-writer Git.** §8.6's
|
||||||
|
consistency model assumes "the journal is local Git, read-your-writes." L4 multi-tenant + the
|
||||||
|
L6 Orchestrator API imply a server; HA/scale implies *multiple* instances. Concurrent commits
|
||||||
|
of coordination-canonical state to one git journal = lock contention / merge races; Git is not
|
||||||
|
a concurrent-write store. Either single-writer-per-space (an unstated HA ceiling) or a real
|
||||||
|
concurrent coordination store with Git as an *export*. **T1 of WP-0005 worsened this** by
|
||||||
|
loading more canonical state into the journal. The keystone unanswered question. → **T2**
|
||||||
|
- **B-2 — Capability-as-data trusts self-reported profiles with no conformance check.** I-3 +
|
||||||
|
§6.5's degradation contract assume the profile tells the truth. A buggy adapter (claims
|
||||||
|
`merge=git/text`, corrupts; claims `notify`, never emits) silently poisons every degradation
|
||||||
|
decision. No **adapter conformance suite** (declared profile == observed behavior) exists.
|
||||||
|
Foundational for an architecture whose correctness rests on profile accuracy. → **T3**
|
||||||
|
- **B-3 — "Coordination-canonical state in the journal" has no representation design.** T1
|
||||||
|
relocated overlays/bindings/aliases/equivalence-sets/merges into "the journal" without saying
|
||||||
|
*how* Git stores structured mutable state. "All equivalences touching X" over a git-of-files
|
||||||
|
is O(scan) unless indexed — and an index is L4/derived. The new central concept is a black
|
||||||
|
box; resolve *with* B-1. → **T2**
|
||||||
|
- **B-4 — Incremental equivalence is under-specified/likely incorrect; I-2 only eventually
|
||||||
|
true.** §8.7 re-verifies a changed page's *new* candidate set but not the pairs it *leaves*
|
||||||
|
(a page exiting an LSH bucket can break an existing equivalence edge); the delta is not
|
||||||
|
additive. Deeper: incremental maintenance drifts from `f(canonical)`, so I-2 holds only
|
||||||
|
eventually, guaranteed solely by an expensive reconcile-against-rebuild. Needs a stated
|
||||||
|
verification mechanism (background checker / digest-vs-sampled-rebuild). → **T4**
|
||||||
|
|
||||||
|
## C. Real but second-tier (track as open problems O-8…O-11)
|
||||||
|
|
||||||
|
- **C-1 — Mechanism-over-policy → operator burden; no preset bundles.** ~7 knob families with
|
||||||
|
sub-modes and interactions; only authz (L0–L4) bundles into personas. Need named bundles
|
||||||
|
("personal vault" / "team wiki" / "enterprise federation"). → **O-8 / T5**
|
||||||
|
- **C-2 — Tenant partitioning (I-13) vs shard sharing + lazy projection.** A shard in two roots
|
||||||
|
is cached twice → duplicate storage + double refresh on rate-limited backends. Shard
|
||||||
|
exclusive-to-one-root or shareable? Unresolved. → **O-9 / T5**
|
||||||
|
- **C-3 — Span-level authz + transclusion is an unmodeled leak path.** Authz is per
|
||||||
|
page/shard/tenant; transclusion crosses shards at span granularity → a page can leak a span
|
||||||
|
past its ACL (aggregation/inference). §7.3's ⊕ also stops being simple two-level inheritance
|
||||||
|
across a transclusion boundary. → **O-10 / T5**
|
||||||
|
- **C-4 — Union-under-unavailability undefined.** Freshness covers *stale*, nothing covers
|
||||||
|
*down*. The dead-shard read path (partial union? error? last-known?) is unspecified though
|
||||||
|
it's the commonest real failure. → **O-11 / T5**
|
||||||
|
|
||||||
|
## D. Recommended resolution (→ SHARD-WP-0006)
|
||||||
|
|
||||||
|
1. **§A reconciliation** (T1) — make the overview match the hardened body.
|
||||||
|
2. **Journal & coordination-state model** (T2) — settle single-vs-multi-writer and separate the
|
||||||
|
**coordination-state store** from the **content-history journal**. Likely resolution:
|
||||||
|
**event-sourced coordination** — an append-only *decision log* is the coordination-canonical
|
||||||
|
tier (git-addressable, I-6 preserved); the queryable current state (alias table, equivalence
|
||||||
|
set) is a *derived fold* of the log (disposable). Append-logs tolerate concurrency far better
|
||||||
|
than mutable-file Git; state a concurrency model. Resolves **B-1 + B-3** together.
|
||||||
|
3. **Adapter conformance suite** (T3) — make a passing conformance run part of the contract
|
||||||
|
(B-2): every adapter proves declared profile == observed behavior.
|
||||||
|
4. **Incremental correctness + I-2 verification** (T4) — fix the leaving-bucket re-verification
|
||||||
|
and propagation; add a background consistency-checker / derived-tier digest so I-2 is
|
||||||
|
verifiable, not merely asserted (B-4).
|
||||||
|
5. **Track §C** (T5) — O-8…O-11 with chosen direction + revisit trigger; close-out.
|
||||||
122
history/260615-core-architecture-blueprint-review.md
Normal file
122
history/260615-core-architecture-blueprint-review.md
Normal file
@@ -0,0 +1,122 @@
|
|||||||
|
# Critical review — CoreArchitectureBlueprint.md
|
||||||
|
|
||||||
|
Date: 2026-06-15 · Reviewer: tegwick (with Claude) · Subject:
|
||||||
|
`spec/CoreArchitectureBlueprint.md` @ commit **9b5b393** · Feeds: **SHARD-WP-0005**
|
||||||
|
|
||||||
|
A deliberately hostile review of the first whole-system architecture, to find where it
|
||||||
|
**breaks (correctness)**, **fails to scale**, and **could be more elegant/efficient** before
|
||||||
|
any implementation. Findings are prioritised; each is the input to a SHARD-WP-0005 task.
|
||||||
|
|
||||||
|
## Verdict in one line
|
||||||
|
|
||||||
|
The **layering and the dual narrow waist are sound and stay**. The **thesis is ~90% right**;
|
||||||
|
the missing 10% (curatorial / coordination-canonical state) breaks its clean story. There are
|
||||||
|
**two genuine bugs**, **two large unaddressed scaling risks**, and several **elegance/efficiency
|
||||||
|
debts** — all fixable without touching INTENT.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## A. The framing crack (fix resolves three issues)
|
||||||
|
|
||||||
|
**A-1 — Two buckets hide a third.** The thesis "canonical at the edges, derived in the middle"
|
||||||
|
omits **born-in-the-middle-but-canonical** state: overlays that are the local truth against a
|
||||||
|
read-only shard (Flow C), manual **curator equivalence bindings**, alias tables, merge
|
||||||
|
decisions. These encode human judgment or local-only content and **cannot be rebuilt** from
|
||||||
|
shards+journal.
|
||||||
|
|
||||||
|
**Contradiction:** I-2 declares L4 rebuildable, yet §8.4 puts "alias table, curator binding"
|
||||||
|
in L4. You cannot rebuild a curator's manual binding.
|
||||||
|
|
||||||
|
**Fix:** three states — **sharded-canonical**, **coordination-canonical** (journal: overlays,
|
||||||
|
bindings, aliases, merges — durable, born in the middle), **derived-disposable** (union graph,
|
||||||
|
indexes, projections). Re-frame §1 as **canonical (sharded + coordination) vs derived
|
||||||
|
(disposable)**; `derived = f(canonical)` then becomes actually true. → **T1**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## B. Where it breaks (correctness)
|
||||||
|
|
||||||
|
**B-1 — Identity conflated with content-fingerprint (BUG).** §7.2 derives page identity from
|
||||||
|
content fingerprint. That makes **editing a page change its identity**, breaking every
|
||||||
|
reference. Fingerprints identify *versions/equivalence*, not *identity*. Page identity must be
|
||||||
|
a **stable handle (uid)** surviving edits; fingerprints belong to the **equivalence** mechanism
|
||||||
|
(§8.4). One word, two concepts, wrong implementation for the stable one. → **T2**
|
||||||
|
|
||||||
|
**B-2 — No concurrency/consistency model.** Concurrent overlays on one page, overlay applied
|
||||||
|
after source drift, journal-commit vs shard-native-write ordering — all undefined. Conflict
|
||||||
|
handling is deferred to "policy presets," but **conflict *detection + representation* is core
|
||||||
|
mechanism**; only *resolution* is policy. The union's consistency guarantee is unstated
|
||||||
|
(eventually-consistent? read-your-writes? causal-via-journal?). → **T3**
|
||||||
|
|
||||||
|
**B-3 — Persisted union cache + multi-tenant = leak surface.** §13 recommends a persisted L4
|
||||||
|
cache; §9 protects content by *read-time* filtering on the provenance envelope. A persisted
|
||||||
|
cross-tenant union cache guarded only by read-time filtering is an L4 attack surface. Tension
|
||||||
|
between I-2 (persisted rebuildable cache), scale, and L5 isolation is unacknowledged. → **T8**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## C. Where it fails to scale
|
||||||
|
|
||||||
|
**C-1 — Equivalence detection is O(N²), no indexing/incremental story.** Fingerprint /
|
||||||
|
span-set-overlap across all pages of all shards is combinatorial (10 shards × 100k pages ≈
|
||||||
|
10¹² comparisons). No blocking/LSH/indexing, no incremental maintenance. Biggest scaling
|
||||||
|
hazard in the document. → **T4**
|
||||||
|
|
||||||
|
**C-2 — "Rebuildable cache" collides with the operational-envelope axis.** A byte-exact
|
||||||
|
rebuild requires reading *every page of every shard*, including rate-limited/paginated
|
||||||
|
external APIs (Notion) and irreducibly-live sources — hours-to-days. I-2 contradicts axis-10.
|
||||||
|
**Incremental, change-driven maintenance must be primary** (notify→delta), rebuild a rare
|
||||||
|
fallback. Cache invalidation — the actual hard problem — is named once and never designed. →
|
||||||
|
**T4, T5**
|
||||||
|
|
||||||
|
**C-3 — Unbounded history at open L0 = DoS/perf.** "Every write a commit" + "open for all" ⇒
|
||||||
|
the git journal grows without bound under bots/vandalism and git degrades on huge histories.
|
||||||
|
"History is the floor" has an unacknowledged cost: packing, compaction, per-shard offload. →
|
||||||
|
**T8**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## D. Elegance / efficiency debts
|
||||||
|
|
||||||
|
**D-1 — The 15 spectra assert a clean degradation function never demonstrated.** Either most
|
||||||
|
axes are irrelevant to most ops (then the 15-D profile is ceremony), or behavior depends on
|
||||||
|
several axes *jointly* (then "no per-backend code" becomes a sprawling axis-interaction matrix
|
||||||
|
— the flat-checklist problem in higher dimensions). And the axes **aren't orthogonal**
|
||||||
|
(git-native history ⟺ git-IS-store ⟺ git/text merge; encrypted opacity ⟹ query/translation
|
||||||
|
collapse). Model a **smaller orthogonal core** + **derived/implied** positions, and state the
|
||||||
|
**axis-interaction subset** the degradation logic truly uses. → **T6**
|
||||||
|
|
||||||
|
**D-2 — Provenance envelope isn't inherited; it'll dwarf the content.** Per-span envelopes at
|
||||||
|
block granularity = 10k near-identical envelopes for a 10k-block graph. The doc already
|
||||||
|
invented the right pattern for Trilium ("effective-vs-own with per-attribute provenance") and
|
||||||
|
failed to apply it to its own envelope. Make provenance **layered (page envelope + span
|
||||||
|
deltas)**. → **T7**
|
||||||
|
|
||||||
|
**D-3 — Projection machinery over-fit to the exotic tail.** Two-axis model + three facets +
|
||||||
|
view registry exist mostly for UC-83/84 (2 of 84 UCs); the 95% case (markdown in git) pays the
|
||||||
|
weight. Make the **common case trivial** (default = plain lazy replication) and
|
||||||
|
derivation/liveness an **extension point**, not a taxonomy every projection instantiates. →
|
||||||
|
**T7**
|
||||||
|
|
||||||
|
**D-4 — Cross-cutting rails are the highest-coupling components, presented as clean.**
|
||||||
|
`provenance/` and capability types are imported by every layer (god-modules); an envelope
|
||||||
|
change ripples everywhere. And **policy has no module** (§10 enumerates it; §11 omits it)
|
||||||
|
despite being consulted by L3/L4/L5. Give policy a home; pin the rails behind stable narrow
|
||||||
|
interfaces. → **T7**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## E. What explicitly stays
|
||||||
|
|
||||||
|
- The 6-layer model + the dual narrow waist (adapter contract / page model).
|
||||||
|
- Capability-as-data (I-3), union-without-erasure (I-4), overlay-before-mutation (I-5),
|
||||||
|
Git-addressable coordination (I-6), mechanism-over-policy (I-7), graceful degradation (I-8).
|
||||||
|
- The federation-model taxonomy and the auth ladder (ArchitectureBlueprint.md).
|
||||||
|
|
||||||
|
## F. Disposition
|
||||||
|
|
||||||
|
Some findings are **solvable now** (A-1, B-1, D-2, D-3, D-4, C-3); some are **partially open**
|
||||||
|
and should be tracked honestly rather than pretend-solved (B-2 consistency model: pick a
|
||||||
|
guarantee; C-1 equivalence-at-scale: pick a blocking strategy; D-1 axis interactions: enumerate
|
||||||
|
the real subset). SHARD-WP-0005 closes the solvable ones and records the open ones in a new
|
||||||
|
"Known scaling risks & open problems" section of the blueprint. → **T9**
|
||||||
45
history/260615-reuse-surface-contributions.md
Normal file
45
history/260615-reuse-surface-contributions.md
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
# reuse-surface contributions — shard-wiki (SHARD-WP-0013 T3)
|
||||||
|
|
||||||
|
Date: 2026-06-15 · From: shard-wiki (whynot) · To: reuse-surface (helix_forge) · Tracked list
|
||||||
|
of (A) capabilities shard-wiki registered, (B) **gaps** shard-wiki proposes the reuse surface
|
||||||
|
add, and (C) capabilities shard-wiki will **consume** rather than rebuild. Communicated to the
|
||||||
|
reuse-surface agent via state-hub `send_message`.
|
||||||
|
|
||||||
|
## A. Registered (T1 + T3) — 8 entries
|
||||||
|
|
||||||
|
`capability.wiki.{shard-orchestration, adapter-contract, page-model, coordination-journal,
|
||||||
|
overlay, federation-models, engine-typed-extensions, derived-views}` — committed in
|
||||||
|
reuse-surface, `validate` ok (20 entries total).
|
||||||
|
|
||||||
|
## B. Proposed gaps (cross-cutting; not shard-wiki-internal) → reuse-surface should own/define
|
||||||
|
|
||||||
|
- **G1 — `capability.platform.typed-extension-framework`** *(suggested)*
|
||||||
|
A reusable pattern: a **small core + a stringent typed-extension framework** where extensions
|
||||||
|
declare typed contracts, compose, and are **activated per context**. shard-wiki's wiki engine
|
||||||
|
(`capability.wiki.engine-typed-extensions`) is one instance, but the *pattern* is
|
||||||
|
cross-domain (any HelixForge capability platform). Evidence: shard-wiki UseCaseCatalog
|
||||||
|
"Capability structure" layer (core + 10 typed extensions + conflict-mediation map).
|
||||||
|
Suggested owner: helix_forge / reuse-surface. Relation: would `generalize`
|
||||||
|
`capability.wiki.engine-typed-extensions`.
|
||||||
|
|
||||||
|
- **G2 — `capability.content.translation-fidelity`** *(suggested)*
|
||||||
|
Lossless/lossy **content translation with an explicit fidelity report** (what round-trips
|
||||||
|
cleanly vs degrades, non-mappable elements preserved as provenance). Reusable well beyond
|
||||||
|
wiki (any format-bridging consumer). Evidence: shard-wiki TSD §A.6, UC-42/UC-59.
|
||||||
|
|
||||||
|
## C. Consumptions (reuse, do not rebuild)
|
||||||
|
|
||||||
|
- **`capability.feature-control.evaluate`** (helix_forge/feature-control) → shard-wiki's
|
||||||
|
**per-shard extension/feature activation** (the "activate only what you need" mechanism).
|
||||||
|
Already recorded as a relation on `capability.wiki.engine-typed-extensions`.
|
||||||
|
- **`capability.authorization.policy-evaluate`** (flex-auth) → shard-wiki's **X-AUTHZ** policy
|
||||||
|
decisions. shard-wiki owns the authz *model* (authz-in-core) but can reuse this evaluation
|
||||||
|
engine rather than building one.
|
||||||
|
- **`capability.statehub.{progress-log, workstream-coordinate}`** → already in use for
|
||||||
|
coordination across this work.
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Gaps G1/G2 are **suggestions** to the reuse-surface owner (not unilateral registrations, since
|
||||||
|
they are cross-cutting, not shard-wiki-internal). Consumptions are recorded for the engine
|
||||||
|
architecture (T5) so it reuses rather than reinvents.
|
||||||
@@ -1,8 +1,19 @@
|
|||||||
# history/
|
# history/
|
||||||
|
|
||||||
Archived material that is no longer needed for daily work but should be kept.
|
Archived material and the project's **meta-history**: finished/canceled workplans kept for
|
||||||
|
the record, plus durable **reviews, critical assessments, and decision records** — the
|
||||||
|
reasoning behind the specs, captured at a point in time.
|
||||||
|
|
||||||
Use a `yymmdd-` prefix when archiving files or directories. Content here is
|
Use a `yymmdd-` prefix. Archived material is **out of scope** for regular tasks (consult only
|
||||||
**out of scope** for regular tasks — consult only for research or diagnostics.
|
for research or diagnostics); assessment/review records are point-in-time and may seed active
|
||||||
|
workplans, but are not edited after the fact — supersede with a new dated record and link back.
|
||||||
|
|
||||||
Finished or canceled workplans from `workplans/` are archived here.
|
Distinct from the **coordination journal** (a runtime Git-backed record of *content* change
|
||||||
|
flows inside an information space, an INTENT domain concept); `history/` is the *project's own*
|
||||||
|
design evolution.
|
||||||
|
|
||||||
|
| Date | Record | Subject |
|
||||||
|
|------|--------|---------|
|
||||||
|
| 2026-06-15 | `260615-core-architecture-blueprint-review.md` | Critical review of `spec/CoreArchitectureBlueprint.md` (commit 9b5b393); inputs to `SHARD-WP-0005` |
|
||||||
|
| 2026-06-15 | `260615-core-architecture-blueprint-review-2.md` | Round-2 review of the hardened blueprint (post-`SHARD-WP-0005`, f21b7b5); inputs to `SHARD-WP-0006` |
|
||||||
|
| 2026-06-15 | `260615-reuse-surface-contributions.md` | shard-wiki's reuse-surface registrations, proposed gaps (G1/G2), and consumptions (`SHARD-WP-0013` T3) |
|
||||||
@@ -36,6 +36,11 @@ pythonpath = ["src"]
|
|||||||
branch = true
|
branch = true
|
||||||
source = ["shard_wiki"]
|
source = ["shard_wiki"]
|
||||||
|
|
||||||
|
[tool.coverage.report]
|
||||||
|
show_missing = true
|
||||||
|
# Quality floor for `pytest --cov` / `coverage report` (not forced on a bare `pytest` run).
|
||||||
|
fail_under = 90
|
||||||
|
|
||||||
[tool.ruff]
|
[tool.ruff]
|
||||||
src = ["src", "tests"]
|
src = ["src", "tests"]
|
||||||
target-version = "py311"
|
target-version = "py311"
|
||||||
|
|||||||
12
registry/README.md
Normal file
12
registry/README.md
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
# Capability Registry
|
||||||
|
|
||||||
|
Markdown-first capability index for federation and reuse planning.
|
||||||
|
|
||||||
|
## Authoring
|
||||||
|
|
||||||
|
1. Copy a capability entry template (see reuse-surface `templates/capability-entry.template.md`).
|
||||||
|
2. Add the row to `indexes/capabilities.yaml`.
|
||||||
|
3. Run `reuse-surface validate` from a checkout with the CLI installed.
|
||||||
|
4. Merge to `main` and verify publish with `reuse-surface establish --publish-check`.
|
||||||
|
|
||||||
|
Federation contract: reuse-surface `docs/RegistryFederation.md`.
|
||||||
103
registry/capabilities/capability.wiki.adapter-contract.md
Normal file
103
registry/capabilities/capability.wiki.adapter-contract.md
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.adapter-contract
|
||||||
|
name: Capability-Aware Shard Adapter Contract
|
||||||
|
summary: A versioned backend interface where each binding declares a verified capability profile (positions on capability spectra), so federation ops degrade by capability.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, adapter, capability, contract, conformance, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D5
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
Fifteen capability spectra with an orthogonal core + implication rules, plus
|
||||||
|
a normative contract spec (TSD Section A); derived from a ~23-system synthesis.
|
||||||
|
availability:
|
||||||
|
current: A2
|
||||||
|
target: A5
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
AdapterContract + a read/write FolderAdapter + a conformance suite that
|
||||||
|
verifies declared profile == observed behaviour exist as a source module.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C2
|
||||||
|
name: Partial
|
||||||
|
confidence: medium
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- versioned interface with declared, conformance-verified capability profiles
|
||||||
|
- one concrete adapter (file-store) passes the conformance suite
|
||||||
|
broken_expectations:
|
||||||
|
- only one substrate implemented (git-IS-store, REST, CRDT adapters planned)
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- hosting backends
|
||||||
|
reliability:
|
||||||
|
level: R1
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- single adapter implemented so far
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Mediate heterogeneity at one narrow waist: a backend participates by implementing a
|
||||||
|
versioned interface and declaring a verified position on each capability spectrum.
|
||||||
|
includes:
|
||||||
|
- capability profile as data (orthogonal-core spectra + implied positions)
|
||||||
|
- operation verbs (read/write/diff/merge/notify/.../derive-projection/execute)
|
||||||
|
- a conformance suite (profiles verified, not self-asserted)
|
||||||
|
excludes:
|
||||||
|
- assuming uniform backend capabilities
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-34..UC-43, UC-50, UC-57, UC-60..UC-69 (shard attachment & adapter binding)"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A2
|
||||||
|
target_level: A5
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/src/shard_wiki/adapters/"
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.page-model
|
||||||
|
supports:
|
||||||
|
- capability.wiki.shard-orchestration
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/TechnicalSpecificationDocument.md (Section A)"
|
||||||
|
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 6)"
|
||||||
|
tests:
|
||||||
|
- "shard-wiki/tests/test_folder_adapter.py"
|
||||||
|
- "shard-wiki/tests/test_conformance.py"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- exposing any page store as a capability-described, conformance-checked shard
|
||||||
|
not_recommended_for:
|
||||||
|
- backends that cannot honestly describe their capabilities
|
||||||
|
known_limitations:
|
||||||
|
- reference implementation covers the file-store substrate only so far
|
||||||
|
---
|
||||||
|
|
||||||
|
# Capability-Aware Shard Adapter Contract
|
||||||
|
|
||||||
|
The bottom narrow waist of shard-wiki: a versioned interface plus a **verified** capability
|
||||||
|
profile per binding. Core logic is written once against capabilities (not per-backend), and
|
||||||
|
the conformance suite rejects profiles whose declared abilities don't match observed behaviour.
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Fifteen spectra reduced to an orthogonal core with implication rules (CoreArchitectureBlueprint
|
||||||
|
Section 6.5); normative in TSD Section A.
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
`adapters/` ships the contract, a folder adapter, and `assert_conformant`.
|
||||||
103
registry/capabilities/capability.wiki.coordination-journal.md
Normal file
103
registry/capabilities/capability.wiki.coordination-journal.md
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.coordination-journal
|
||||||
|
name: Event-Sourced Coordination Journal
|
||||||
|
summary: An append-only, totally-ordered-per-space decision log (overlays, bindings, aliases, merges, forks) whose current state is a derived fold; git-addressable history.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, event-sourcing, coordination, git, journal, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D5
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
Keystone resolved across two architecture reviews: coordination-canonical state
|
||||||
|
as an append-only decision log with a per-space append authority; current state
|
||||||
|
is a derived fold (derived = f(log)).
|
||||||
|
availability:
|
||||||
|
current: A2
|
||||||
|
target: A4
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
In-memory DecisionLog + fold work as a source module; the git-backed store with a
|
||||||
|
per-space lease (the production backing) is planned.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C2
|
||||||
|
name: Partial
|
||||||
|
confidence: medium
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- append-only, totally-ordered-per-space log with read-your-writes
|
||||||
|
- derived fold to aliases + transitively-merged equivalence groups
|
||||||
|
broken_expectations:
|
||||||
|
- git-backed storage and per-space lease/append-authority not yet implemented
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- general-purpose event bus
|
||||||
|
reliability:
|
||||||
|
level: R1
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- in-memory backing only; cross-process durability pending
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Make coordination-canonical decisions durable and git-addressable as events, with the
|
||||||
|
queryable current state always recomputable by replay.
|
||||||
|
includes:
|
||||||
|
- append-only decision log, totally ordered per information space
|
||||||
|
- derived fold to current coordination state (aliases, equivalence groups, overlays)
|
||||||
|
- per-space append authority (concurrency model)
|
||||||
|
excludes:
|
||||||
|
- storing derived/disposable union state
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-29, UC-33 (history, attribution, coordination journal)"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A2
|
||||||
|
target_level: A4
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/src/shard_wiki/coordination/decision_log.py"
|
||||||
|
target_artifacts:
|
||||||
|
- git-backed log store with per-space lease
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
|
||||||
|
relations:
|
||||||
|
supports:
|
||||||
|
- capability.wiki.shard-orchestration
|
||||||
|
- capability.wiki.overlay
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.1)"
|
||||||
|
tests:
|
||||||
|
- "shard-wiki/tests/test_decision_log.py"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- durable, replayable, git-addressable coordination state for a federated space
|
||||||
|
not_recommended_for:
|
||||||
|
- high-frequency general event streaming
|
||||||
|
known_limitations:
|
||||||
|
- production git backing + lease are still on the roadmap (SHARD-WP-0009)
|
||||||
|
---
|
||||||
|
|
||||||
|
# Event-Sourced Coordination Journal
|
||||||
|
|
||||||
|
The keystone: coordination-canonical state (overlays, equivalence bindings, aliases, merges,
|
||||||
|
forks) is an append-only **decision log**, totally ordered per information space; the queryable
|
||||||
|
current state is a derived **fold** of the log (`derived = f(log)`). The log is git-addressable,
|
||||||
|
giving history/patch/review/backup for coordination decisions for free.
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Resolved across the round-1/round-2 architecture reviews (CoreArchitectureBlueprint Section 8.1).
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
`decision_log.py` ships an in-memory, totally-ordered log + fold; git+lease backing is planned.
|
||||||
87
registry/capabilities/capability.wiki.derived-views.md
Normal file
87
registry/capabilities/capability.wiki.derived-views.md
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.derived-views
|
||||||
|
name: Wiki Derived Views
|
||||||
|
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages, SiteMap, and (delegate-or-derive) Search — carrying provenance.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, derived-views, backlinks, recentchanges, search, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D3
|
||||||
|
target: D5
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
Core-vs-adapter classification and behaviours are decided (FederationRequirements ADR-03);
|
||||||
|
implementation is planned (SHARD-WP-0010), not built.
|
||||||
|
availability:
|
||||||
|
current: A0
|
||||||
|
target: A4
|
||||||
|
confidence: low
|
||||||
|
rationale: >
|
||||||
|
Designed; no implementation yet. Informational/planning reuse only today.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C0
|
||||||
|
name: Absent
|
||||||
|
confidence: low
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations: []
|
||||||
|
broken_expectations:
|
||||||
|
- no derived view is implemented yet
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- presentation / rendering of views
|
||||||
|
reliability:
|
||||||
|
level: R0
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- planning-stage
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Provide recomputable, provenance-carrying views over the union (link graph, change feed,
|
||||||
|
enumeration, search) without introducing canonical state.
|
||||||
|
includes:
|
||||||
|
- BackLinks (link graph), RecentChanges (journal + shard signals), AllPages, SiteMap
|
||||||
|
- Search as delegate-to-native-or-derive-index
|
||||||
|
excludes:
|
||||||
|
- view presentation / UI
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-17..UC-21, UC-63"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A0
|
||||||
|
target_level: A4
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/workplans/SHARD-WP-0010-derived-views.md"
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.shard-orchestration
|
||||||
|
- capability.wiki.page-model
|
||||||
|
related_to:
|
||||||
|
- capability.wiki.engine-typed-extensions
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/FederationRequirements.md (ADR-03)"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- planning derived navigation/discovery over a federated wiki union
|
||||||
|
not_recommended_for:
|
||||||
|
- implementation reuse today (planning-stage)
|
||||||
|
known_limitations:
|
||||||
|
- not implemented; Search ranking policy undecided
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wiki Derived Views
|
||||||
|
|
||||||
|
Recomputable views over the union (BackLinks, RecentChanges, AllPages, SiteMap, Search). All
|
||||||
|
are derived/disposable (no canonical state) and carry provenance; Search is delegate-to-native
|
||||||
|
where a shard's query capability allows, else a derived index. Planned in SHARD-WP-0010.
|
||||||
115
registry/capabilities/capability.wiki.engine-typed-extensions.md
Normal file
115
registry/capabilities/capability.wiki.engine-typed-extensions.md
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.engine-typed-extensions
|
||||||
|
name: Wiki Engine with Typed Extensions
|
||||||
|
summary: A small-core wiki engine realizing a stringent typed-extension framework that addresses all wiki use cases and lets each shard activate only the features it needs.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, engine, typed-extensions, feature-activation, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D3
|
||||||
|
target: D5
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
Architecture authored (shard-wiki/spec/WikiEngineCoreArchitecture.md): small page-store
|
||||||
|
kernel + typed-extension framework, per-shard activation, engine-as-canonical-mode-shard,
|
||||||
|
and a conflict-mediation realization are explored. Detailed extension SDK/ABI and the API
|
||||||
|
protocol remain (so D3 Explored, not yet D4/D5).
|
||||||
|
availability:
|
||||||
|
current: A0
|
||||||
|
target: A4
|
||||||
|
confidence: low
|
||||||
|
rationale: >
|
||||||
|
Planned. No engine kernel or extensions exist yet; informational/planning reuse only.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C0
|
||||||
|
name: Absent
|
||||||
|
confidence: low
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations: []
|
||||||
|
broken_expectations:
|
||||||
|
- engine core and typed-extension mechanism not yet designed in detail
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- replacing other wiki engines or mandating one implementation
|
||||||
|
reliability:
|
||||||
|
level: R0
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- planning-stage capability
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Provide shard-wiki's reference first-party shard backend: a small core + a stringent
|
||||||
|
typed-extension framework covering all collected use cases, mediating conflicting
|
||||||
|
requirements into an integrated whole, with per-shard activation (only what you need).
|
||||||
|
includes:
|
||||||
|
- a minimal engine kernel (page lifecycle, storage via the adapter contract, the typing mechanism)
|
||||||
|
- typed extensions that declare contracts and compose
|
||||||
|
- per-shard feature activation
|
||||||
|
excludes:
|
||||||
|
- replacing or mandating other wiki engines (it is one shard type among many)
|
||||||
|
- a single canonical implementation for all wikis
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-08..UC-25 and the full catalog (the engine must cover all)"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A0
|
||||||
|
target_level: A4
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
|
||||||
|
- "shard-wiki/spec/WikiEngineCoreArchitecture.md"
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.adapter-contract
|
||||||
|
- capability.wiki.page-model
|
||||||
|
related_to:
|
||||||
|
- capability.feature-control.evaluate
|
||||||
|
- capability.authorization.policy-evaluate
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- planning a composable, feature-activatable native wiki engine
|
||||||
|
not_recommended_for:
|
||||||
|
- implementation reuse today (planning-stage)
|
||||||
|
known_limitations:
|
||||||
|
- architecture authored; extension SDK/ABI + API protocol still to design; not yet built
|
||||||
|
|
||||||
|
promotion_history:
|
||||||
|
- date: "2026-06-15"
|
||||||
|
dimension: discovery
|
||||||
|
from: D2
|
||||||
|
to: D3
|
||||||
|
rationale: WikiEngineCoreArchitecture.md authored (kernel + typed-extension framework explored); INTENT amendment ratified.
|
||||||
|
author: shard-wiki
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wiki Engine with Typed Extensions
|
||||||
|
|
||||||
|
shard-wiki's planned reference first-party shard backend — a *canonical-mode shard* it
|
||||||
|
implements natively: a small core plus a stringent typed-extension framework addressing all
|
||||||
|
collected use cases, mediating conflicting requirements into a consistent whole, with per-shard
|
||||||
|
activation (activate only what you need). It is one shard type among many — not a replacement
|
||||||
|
for other engines. Per-shard activation is a candidate consumer of
|
||||||
|
`capability.feature-control.evaluate`.
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Architecture authored: `shard-wiki/spec/WikiEngineCoreArchitecture.md` (small kernel +
|
||||||
|
typed-extension framework; engine = canonical-mode shard). INTENT amendment ratified
|
||||||
|
(2026-06-15, decision 84ffdb48). Extension SDK/ABI + API protocol are the next deliverables.
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
Planning-stage; informational reuse only.
|
||||||
97
registry/capabilities/capability.wiki.federation-models.md
Normal file
97
registry/capabilities/capability.wiki.federation-models.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.federation-models
|
||||||
|
name: Selectable Federation-Model Taxonomy
|
||||||
|
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication+ping, query-time graph-join, feed, activity-streams, engine-mirror) selected per space.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, federation, taxonomy, composable, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D4
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
A six-model taxonomy distilled from a ~23-system synthesis, each model anchored in a
|
||||||
|
real system, with capability prerequisites and per-space/per-shard composition rules.
|
||||||
|
availability:
|
||||||
|
current: A0
|
||||||
|
target: A4
|
||||||
|
confidence: low
|
||||||
|
rationale: >
|
||||||
|
Designed and specified (FederationArchitecture T17) but not implemented; informational
|
||||||
|
reuse only today.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C1
|
||||||
|
name: Sparse
|
||||||
|
confidence: low
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- the model taxonomy and selection/composition rules are documented
|
||||||
|
broken_expectations:
|
||||||
|
- no federation transport is implemented yet
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- mandating a single federation mechanism
|
||||||
|
reliability:
|
||||||
|
level: R0
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- design-stage; no runtime evidence
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Treat federation as selectable and composable rather than one mechanism, so each space
|
||||||
|
picks fork+journal, VCS-replication, query-join, feed, activity-streams, or engine-mirror.
|
||||||
|
includes:
|
||||||
|
- the six federation models + their capability floors
|
||||||
|
- per-space selection and per-shard composition
|
||||||
|
excludes:
|
||||||
|
- imposing one homogeneous federation network
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-26, UC-31, UC-33, UC-71, UC-72, UC-74, UC-79"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A0
|
||||||
|
target_level: A4
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/spec/FederationArchitecture.md (T17)"
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.shard-orchestration
|
||||||
|
- capability.wiki.coordination-journal
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/FederationArchitecture.md"
|
||||||
|
- "shard-wiki/research/260614-shard-spectrum-synthesis/findings.md"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- planning a federation strategy that mixes models per source
|
||||||
|
not_recommended_for:
|
||||||
|
- implementation reuse today (design-stage)
|
||||||
|
known_limitations:
|
||||||
|
- no transport implemented; informational planning reuse only
|
||||||
|
---
|
||||||
|
|
||||||
|
# Selectable Federation-Model Taxonomy
|
||||||
|
|
||||||
|
Federation is plural and composable: fork+journal (Federated Wiki), VCS-replication+ping
|
||||||
|
(ikiwiki), query-time graph-join (Wikibase SERVICE), feed aggregation, activity streams
|
||||||
|
(ActivityPub), and engine-mirror (Wiki.js). A space selects a model and composes per shard;
|
||||||
|
the default is fork+journal over git. Design-stage capability — strong for planning reuse.
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
FederationArchitecture T17, distilled from the shard-spectrum synthesis (v3).
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
Specified, not implemented — informational reuse only.
|
||||||
102
registry/capabilities/capability.wiki.overlay.md
Normal file
102
registry/capabilities/capability.wiki.overlay.md
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.overlay
|
||||||
|
name: Overlay-Before-Mutation Write Path
|
||||||
|
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only, rate-limited, or lossy backends be edited safely without silent remote mutation.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, overlay, patch, write-path, conflict, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D5
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
Overlay lifecycle and apply-under-drift semantics are specified (ADR-05, blueprint
|
||||||
|
Section 8.6) and implemented as a single principled write path.
|
||||||
|
availability:
|
||||||
|
current: A2
|
||||||
|
target: A4
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
OverlayEngine (draft/patch/apply), writable adapter, and InformationSpace.edit
|
||||||
|
exist as a source module; three-way merge is not (refuse-on-drift only).
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C2
|
||||||
|
name: Partial
|
||||||
|
confidence: medium
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- draft -> patch -> apply with fast-forward / refuse-on-drift / keep-draft outcomes
|
||||||
|
- no silent remote mutation; overlay_state surfaced in provenance
|
||||||
|
broken_expectations:
|
||||||
|
- three-way / auto merge not implemented (refuse-on-conflict only)
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- federation propagation of applied overlays
|
||||||
|
reliability:
|
||||||
|
level: R1
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- early implementation; conflict handling is detect-and-refuse only
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Make any sub-write-through backend editable safely: an edit is an overlay first, applied
|
||||||
|
only on explicit intent and only when the source has not drifted.
|
||||||
|
includes:
|
||||||
|
- overlay drafts recorded as coordination-canonical events
|
||||||
|
- patch rendering (unified diff)
|
||||||
|
- apply-under-drift (fast-forward / refuse / keep-draft)
|
||||||
|
excludes:
|
||||||
|
- destructive write without drift check
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-04, UC-26, UC-29 (remix primitives, overlay)"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A2
|
||||||
|
target_level: A4
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/src/shard_wiki/coordination/overlay.py"
|
||||||
|
- "shard-wiki/src/shard_wiki/coordination/patch.py"
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.coordination-journal
|
||||||
|
- capability.wiki.adapter-contract
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/FederationRequirements.md (ADR-05)"
|
||||||
|
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.2, 8.6)"
|
||||||
|
tests:
|
||||||
|
- "shard-wiki/tests/test_apply.py"
|
||||||
|
- "shard-wiki/tests/test_write_path_integration.py"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- safe editing over read-only / rate-limited / lossy backends
|
||||||
|
not_recommended_for:
|
||||||
|
- workflows needing automatic conflict resolution today
|
||||||
|
known_limitations:
|
||||||
|
- merge is detect-and-refuse; three-way merge is future work
|
||||||
|
---
|
||||||
|
|
||||||
|
# Overlay-Before-Mutation Write Path
|
||||||
|
|
||||||
|
One principled write path: every edit drafts an overlay (a coordination-canonical event),
|
||||||
|
renders as a patch, and applies under drift checks — fast-forwarding a writable target,
|
||||||
|
keeping a local draft on a read-only target, and refusing (never clobbering) on external drift.
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Specified in FederationRequirements ADR-05 and CoreArchitectureBlueprint Section 8.2/8.6.
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
`overlay.py` + `patch.py` + `InformationSpace.edit` ship the path; built in SHARD-WP-0008.
|
||||||
104
registry/capabilities/capability.wiki.page-model.md
Normal file
104
registry/capabilities/capability.wiki.page-model.md
Normal file
@@ -0,0 +1,104 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.page-model
|
||||||
|
name: Backend-Neutral Wiki Page Model
|
||||||
|
summary: A Markdown-first but stretchable page model with stable identity separate from placement and layered provenance, spanning prose to typed-graph and computational shapes.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, page-model, identity, provenance, markdown, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D5
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
Page shapes (prose, typed records, typed-graph, inline-embedded, non-Markdown,
|
||||||
|
and four computational shapes) plus identity != placement and layered provenance
|
||||||
|
are specified and grounded in the dive research.
|
||||||
|
availability:
|
||||||
|
current: A2
|
||||||
|
target: A5
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
Identity/Placement/Span/Page and layered ProvenanceEnvelope exist as a source
|
||||||
|
module; richer shapes (typed-graph, notebook) are modeled but not all built.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C2
|
||||||
|
name: Partial
|
||||||
|
confidence: medium
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- stable identity distinct from placement and from content fingerprint
|
||||||
|
- layered (effective-vs-own) provenance with near-zero per-span cost
|
||||||
|
broken_expectations:
|
||||||
|
- non-prose shapes (typed-graph, notebook, inline-embedded) not fully realized
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- rendering / presentation
|
||||||
|
reliability:
|
||||||
|
level: R1
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- prose shape is the only exercised path so far
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
One backend-neutral lingua franca every consumer sees; every shape reduces to
|
||||||
|
(content|source, structure, provenance envelope, optional derivation rule).
|
||||||
|
includes:
|
||||||
|
- page identity (stable handle) vs placement (N paths/shards) vs equivalence (fingerprint)
|
||||||
|
- layered provenance envelope (page + span deltas)
|
||||||
|
- page-shape taxonomy incl. computational shapes
|
||||||
|
excludes:
|
||||||
|
- deriving identity from content (a fingerprint identifies a version, not a page)
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-34, UC-39, UC-44..UC-49, UC-55, UC-73, UC-83, UC-84"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A2
|
||||||
|
target_level: A5
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/src/shard_wiki/model/"
|
||||||
|
- "shard-wiki/src/shard_wiki/provenance/"
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
|
||||||
|
relations:
|
||||||
|
supports:
|
||||||
|
- capability.wiki.adapter-contract
|
||||||
|
- capability.wiki.shard-orchestration
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 7)"
|
||||||
|
- "shard-wiki/spec/FederationRequirements.md (ADR-02, ADR-04)"
|
||||||
|
tests:
|
||||||
|
- "shard-wiki/tests/test_model.py"
|
||||||
|
- "shard-wiki/tests/test_provenance.py"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- a portable, provenance-carrying representation of wiki pages across backends
|
||||||
|
not_recommended_for:
|
||||||
|
- cases needing a single canonical path per page (use identity, not path)
|
||||||
|
known_limitations:
|
||||||
|
- non-prose shapes specified ahead of implementation
|
||||||
|
---
|
||||||
|
|
||||||
|
# Backend-Neutral Wiki Page Model
|
||||||
|
|
||||||
|
The top narrow waist: a Markdown-first model that stretches to typed records, typed-graph
|
||||||
|
statements, inline-embedded objects, non-Markdown assets, and computational shapes. Identity
|
||||||
|
is a stable handle; placement and equivalence are separate mechanisms; provenance is layered
|
||||||
|
(effective = page envelope + span delta).
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Specified in CoreArchitectureBlueprint Section 7 and FederationRequirements ADR-02/04.
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
`model/` + `provenance/` ship the prose path and the layered envelope today.
|
||||||
114
registry/capabilities/capability.wiki.shard-orchestration.md
Normal file
114
registry/capabilities/capability.wiki.shard-orchestration.md
Normal file
@@ -0,0 +1,114 @@
|
|||||||
|
---
|
||||||
|
id: capability.wiki.shard-orchestration
|
||||||
|
name: Wiki Shard Orchestration
|
||||||
|
summary: Present a union of pages across heterogeneous wiki-shaped shards while preserving each shard's provenance, capabilities, and history.
|
||||||
|
owner: shard-wiki
|
||||||
|
status: draft
|
||||||
|
domain: helix_forge
|
||||||
|
tags: [wiki, federation, orchestration, union, shard-wiki]
|
||||||
|
|
||||||
|
maturity:
|
||||||
|
discovery:
|
||||||
|
current: D5
|
||||||
|
target: D6
|
||||||
|
confidence: high
|
||||||
|
rationale: >
|
||||||
|
Grounded in 84 documented use cases and a twice-reviewed whole-system
|
||||||
|
architecture (CoreArchitectureBlueprint) derived from ~23 prior-art systems.
|
||||||
|
availability:
|
||||||
|
current: A2
|
||||||
|
target: A5
|
||||||
|
confidence: medium
|
||||||
|
rationale: >
|
||||||
|
InformationSpace orchestrator (attach -> resolve -> read, chorus on
|
||||||
|
ambiguity) works as a Python source module; network API and incremental
|
||||||
|
union are planned.
|
||||||
|
|
||||||
|
external_evidence:
|
||||||
|
completeness:
|
||||||
|
level: C2
|
||||||
|
name: Partial
|
||||||
|
confidence: medium
|
||||||
|
basis: scope_vs_intent_and_consumer_expectations
|
||||||
|
satisfied_expectations:
|
||||||
|
- attach folder shards and read a union page with layered provenance
|
||||||
|
- chorus presentation of equivalent-but-divergent pages (union without erasure)
|
||||||
|
broken_expectations:
|
||||||
|
- incremental union maintenance and equivalence index not yet built
|
||||||
|
- write-through federation transports not yet built
|
||||||
|
out_of_scope_expectations:
|
||||||
|
- hosting or replacing the underlying wiki engines
|
||||||
|
reliability:
|
||||||
|
level: R1
|
||||||
|
confidence: low
|
||||||
|
basis: consumer_quality_signals
|
||||||
|
known_reliability_risks:
|
||||||
|
- early implementation; 64 tests but no production exposure
|
||||||
|
|
||||||
|
discovery:
|
||||||
|
intent: >
|
||||||
|
Let independently stored, differently implemented wikis behave as one
|
||||||
|
coherent, versionable, inspectable information space without homogenizing them.
|
||||||
|
includes:
|
||||||
|
- union resolution across shards (identity-keyed)
|
||||||
|
- chorus / designated-canonical presentation of equivalent pages
|
||||||
|
- lazy replication projection of remote content with freshness
|
||||||
|
excludes:
|
||||||
|
- implementing a backend wiki engine (see capability.wiki.engine-typed-extensions)
|
||||||
|
- silent remote mutation
|
||||||
|
assumptions:
|
||||||
|
- canonical truth lives in shards + a git coordination journal; the union is derived
|
||||||
|
use_cases:
|
||||||
|
- "shard-wiki UseCaseCatalog UC-01..UC-07, UC-26..UC-33 (information space, federation, coordination)"
|
||||||
|
|
||||||
|
availability:
|
||||||
|
current_level: A2
|
||||||
|
target_level: A5
|
||||||
|
current_artifacts:
|
||||||
|
- "shard-wiki/src/shard_wiki/union/"
|
||||||
|
- "shard-wiki/src/shard_wiki/space.py"
|
||||||
|
target_artifacts:
|
||||||
|
- orchestrator network API
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
|
||||||
|
relations:
|
||||||
|
depends_on:
|
||||||
|
- capability.wiki.adapter-contract
|
||||||
|
- capability.wiki.page-model
|
||||||
|
- capability.wiki.coordination-journal
|
||||||
|
supports:
|
||||||
|
- capability.wiki.federation-models
|
||||||
|
|
||||||
|
evidence:
|
||||||
|
documentation:
|
||||||
|
- "shard-wiki/spec/CoreArchitectureBlueprint.md"
|
||||||
|
- "shard-wiki/spec/FederationArchitecture.md"
|
||||||
|
tests:
|
||||||
|
- "shard-wiki/tests/test_union.py"
|
||||||
|
- "shard-wiki/tests/test_integration.py"
|
||||||
|
|
||||||
|
consumer_guidance:
|
||||||
|
recommended_for:
|
||||||
|
- composing multiple Markdown/wiki stores into one provenance-preserving view
|
||||||
|
not_recommended_for:
|
||||||
|
- replacing a single wiki engine
|
||||||
|
known_limitations:
|
||||||
|
- resolution is recompute-on-read until the incremental tier lands
|
||||||
|
---
|
||||||
|
|
||||||
|
# Wiki Shard Orchestration
|
||||||
|
|
||||||
|
shard-wiki's core capability: orchestrate wiki-shaped content across heterogeneous *shards*
|
||||||
|
as a union of pages, preserving provenance, capabilities, and history per shard. Canonical
|
||||||
|
truth stays at the edges (shards + the git coordination journal); the union is a derived,
|
||||||
|
recomputable view (orchestrator, not engine).
|
||||||
|
|
||||||
|
## Assessment notes
|
||||||
|
|
||||||
|
### Discovery
|
||||||
|
Grounded by `UseCaseCatalog.md` (84 UCs) and the hardened `CoreArchitectureBlueprint.md`.
|
||||||
|
|
||||||
|
### Availability
|
||||||
|
`InformationSpace` provides attach/resolve/read today (source module); a network API is the
|
||||||
|
target availability step.
|
||||||
145
registry/indexes/capabilities.yaml
Normal file
145
registry/indexes/capabilities.yaml
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
version: 1
|
||||||
|
updated: '2026-06-16'
|
||||||
|
domain: helix_forge
|
||||||
|
capabilities:
|
||||||
|
- id: capability.wiki.shard-orchestration
|
||||||
|
name: Wiki Shard Orchestration
|
||||||
|
summary: Present a union of pages across heterogeneous wiki-shaped shards while
|
||||||
|
preserving each shard's provenance, capabilities, and history.
|
||||||
|
vector: D5 / A2 / C2 / R1
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.shard-orchestration.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- federation
|
||||||
|
- orchestration
|
||||||
|
- union
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
- id: capability.wiki.adapter-contract
|
||||||
|
name: Capability-Aware Shard Adapter Contract
|
||||||
|
summary: A versioned backend interface where each binding declares a verified capability
|
||||||
|
profile, so federation ops degrade by capability.
|
||||||
|
vector: D5 / A2 / C2 / R1
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.adapter-contract.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- adapter
|
||||||
|
- capability
|
||||||
|
- contract
|
||||||
|
- conformance
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
- id: capability.wiki.page-model
|
||||||
|
name: Backend-Neutral Wiki Page Model
|
||||||
|
summary: A Markdown-first but stretchable page model with stable identity separate
|
||||||
|
from placement and layered provenance.
|
||||||
|
vector: D5 / A2 / C2 / R1
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.page-model.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- page-model
|
||||||
|
- identity
|
||||||
|
- provenance
|
||||||
|
- markdown
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
- id: capability.wiki.coordination-journal
|
||||||
|
name: Event-Sourced Coordination Journal
|
||||||
|
summary: An append-only, totally-ordered-per-space decision log whose current state
|
||||||
|
is a derived fold; git-addressable history.
|
||||||
|
vector: D5 / A2 / C2 / R1
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.coordination-journal.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- event-sourcing
|
||||||
|
- coordination
|
||||||
|
- git
|
||||||
|
- journal
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
- id: capability.wiki.overlay
|
||||||
|
name: Overlay-Before-Mutation Write Path
|
||||||
|
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only
|
||||||
|
or limited backends be edited safely without silent remote mutation.
|
||||||
|
vector: D5 / A2 / C2 / R1
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.overlay.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- overlay
|
||||||
|
- patch
|
||||||
|
- write-path
|
||||||
|
- conflict
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- source module
|
||||||
|
- id: capability.wiki.federation-models
|
||||||
|
name: Selectable Federation-Model Taxonomy
|
||||||
|
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication,
|
||||||
|
query-join, feed, activity-streams, engine-mirror) selected per space.
|
||||||
|
vector: D4 / A0 / C1 / R0
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.federation-models.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- federation
|
||||||
|
- taxonomy
|
||||||
|
- composable
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
|
- id: capability.wiki.engine-typed-extensions
|
||||||
|
name: Wiki Engine with Typed Extensions
|
||||||
|
summary: A small-core wiki engine realizing a typed-extension framework that addresses
|
||||||
|
all wiki use cases and lets each shard activate only the features it needs.
|
||||||
|
vector: D3 / A0 / C0 / R0
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.engine-typed-extensions.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- engine
|
||||||
|
- typed-extensions
|
||||||
|
- feature-activation
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
|
- id: capability.wiki.derived-views
|
||||||
|
name: Wiki Derived Views
|
||||||
|
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages,
|
||||||
|
SiteMap, and (delegate-or-derive) Search — carrying provenance.
|
||||||
|
vector: D3 / A0 / C0 / R0
|
||||||
|
domain: helix_forge
|
||||||
|
status: draft
|
||||||
|
owner: shard-wiki
|
||||||
|
path: registry/capabilities/capability.wiki.derived-views.md
|
||||||
|
tags:
|
||||||
|
- wiki
|
||||||
|
- derived-views
|
||||||
|
- backlinks
|
||||||
|
- recentchanges
|
||||||
|
- search
|
||||||
|
- shard-wiki
|
||||||
|
consumption_modes:
|
||||||
|
- informational
|
||||||
38
research/260614-computational-page-model-synthesis/README.md
Normal file
38
research/260614-computational-page-model-synthesis/README.md
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
# 260614 — The computational page model (SHARD-WP-0004 synthesis)
|
||||||
|
|
||||||
|
Date: 2026-06-15 · Source: **SHARD-WP-0004** post-batch synthesis (T1–T8)
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
The acceptance-criteria synthesis for SHARD-WP-0004 — *"the computational page model"* —
|
||||||
|
reading the eight computational/interactive-knowledge dives across each other and distilling
|
||||||
|
them into **one model**: **source is canonical; everything rendered/computed is a
|
||||||
|
projection**, placed on **two axes** (projection-kind: replication vs derivation; liveness:
|
||||||
|
live↔snapshot), with a recommendation on an executable-content capability.
|
||||||
|
|
||||||
|
## The answer to the carried question
|
||||||
|
|
||||||
|
*Can a shard-wiki page be a live computational artifact?* **Yes — as a page-model +
|
||||||
|
projection concern, not as an execution platform.** Every system externalizes to a canonical
|
||||||
|
source and treats the live/computed form as derived; shard-wiki **recognizes** computational
|
||||||
|
content, **attaches the source**, and **presents derivations as provenance- and
|
||||||
|
liveness-marked projections**, with **execution as a gated capability (off by default,
|
||||||
|
degrade to snapshot)**. No INTENT amendment required.
|
||||||
|
|
||||||
|
## Key contributions
|
||||||
|
|
||||||
|
- **One model:** `(source, derivation rule, projection with provenance + liveness)` covers
|
||||||
|
all four computational page shapes (one-source-many-projections UC-83; notebook UC-84;
|
||||||
|
program-as-page; live/temporal content).
|
||||||
|
- **Two axes for T16:** replication vs **derivation-projection** (timing / multiplicity /
|
||||||
|
continuity facets) × the **live↔snapshot** axis (bounded at the irreducibly-live far end by
|
||||||
|
Strudel).
|
||||||
|
- **One snapshot-provenance record** reused for notebook outputs, renders, recordings.
|
||||||
|
- **Hard boundaries:** never host a kernel/runtime as store; **image-is-not-a-store**; never
|
||||||
|
present a derivation without output→source provenance.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
| Path | Role |
|
||||||
|
|------|------|
|
||||||
|
| `findings.md` | The source/derivation/projection model, the two axes, the four page shapes, provenance/reproducibility, the recommendation, the SHARD-WP-0002 fold-in, escalated open questions |
|
||||||
178
research/260614-computational-page-model-synthesis/findings.md
Normal file
178
research/260614-computational-page-model-synthesis/findings.md
Normal file
@@ -0,0 +1,178 @@
|
|||||||
|
# The computational page model — synthesis (SHARD-WP-0004)
|
||||||
|
|
||||||
|
**Date:** 2026-06-15 · **Source:** SHARD-WP-0004 (post-batch synthesis, T1–T8) · **Kind:**
|
||||||
|
synthesis (no new external research) reading the eight computational/interactive-knowledge
|
||||||
|
dives *across* each other.
|
||||||
|
|
||||||
|
## What this is
|
||||||
|
|
||||||
|
The acceptance-criteria synthesis for SHARD-WP-0004. The batch asked one carried question:
|
||||||
|
*can a shard-wiki page be a live computational artifact — a source woven/evaluated into
|
||||||
|
rendered forms — and if so, how do projection, transclusion, provenance, and the adapter
|
||||||
|
contract treat the source, the environment, and the computed output?* This memo answers it
|
||||||
|
by distilling the eight dives into **one model**: the **source / derivation / projection**
|
||||||
|
view of computational content, anchored on two axes, plus a recommendation on whether
|
||||||
|
shard-wiki needs an executable-content capability.
|
||||||
|
|
||||||
|
Dives consolidated: **T1** literate programming (WEB/weave/tangle), **T2** Mathematica,
|
||||||
|
**T3** Jupyter, **T4** Processing/p5.js, **T5** Strudel, **T6** Squeak, **T7** Glamorous
|
||||||
|
Toolkit, **T8** Pharo. Catalog yield: **UC-83** (literate one-source-many-projections),
|
||||||
|
**UC-84** (notebook with computed-output provenance); the other six are **enrichment/boundary**
|
||||||
|
dives (UC-54/55/47/48 + the projection model).
|
||||||
|
|
||||||
|
## 1. The one finding: source is canonical, everything rendered is a projection
|
||||||
|
|
||||||
|
Every system in the batch, however "live," **externalizes to a canonical artifact** and
|
||||||
|
treats the rendered/computed form as **derived**:
|
||||||
|
|
||||||
|
| System | Canonical (attach this) | Derived (a projection) |
|
||||||
|
|--------|-------------------------|------------------------|
|
||||||
|
| Literate (WEB) | the WEB/`.nw`/`.org` **source** | woven docs **and** tangled code |
|
||||||
|
| Mathematica | the `.nb` (a Wolfram expression) | cached `Output` cells; CDF; `Dynamic` widgets |
|
||||||
|
| Jupyter | `.ipynb` source (ideally paired text) | embedded outputs; nbconvert/nbviewer renders |
|
||||||
|
| Processing | the **sketch source** | the view-time canvas render |
|
||||||
|
| Strudel | the **pattern source** | the live audio performance / a recording |
|
||||||
|
| GT / Lepiter | git-versionable **page/Tonel files** | moldable `gtView`s; live snippet results |
|
||||||
|
| Squeak/Pharo | exported **files** (Tonel/git) | the live image / running objects |
|
||||||
|
|
||||||
|
**This is the same principle shard-wiki already holds** (files-canonical, index/render
|
||||||
|
derived — ikiwiki UC-79, Logseq UC-62, nbstripout). The computational batch did not break it;
|
||||||
|
it **stress-tested it to the live extreme and it held**. *The image (Squeak/Pharo) is the
|
||||||
|
only would-be exception, and it is a boundary, not a counterexample: an image is not a store.*
|
||||||
|
|
||||||
|
## 2. Two axes the contract must add
|
||||||
|
|
||||||
|
The batch refines the **projection model** (SHARD-WP-0002 T16) with two orthogonal axes:
|
||||||
|
|
||||||
|
### 2.1 Projection kind: replication vs derivation
|
||||||
|
- **Replication-projection** (the existing default): a lazy **cache/copy** of remote content
|
||||||
|
(Obsidian/Notion mirrors, UC-53/57).
|
||||||
|
- **Derivation-projection** (new, from T1): a **transform/compile/weave/evaluate** of a
|
||||||
|
source into rendered forms — regenerable, may delegate to the source's tool, **degrades to
|
||||||
|
a captured snapshot** when the tool is absent. Covers weave/tangle, nbconvert, CDF, sketch
|
||||||
|
render, audio recording, `gtView`.
|
||||||
|
|
||||||
|
Sub-facets of derivation (from T4): **materialization timing** = *ahead-of-time* (CDF,
|
||||||
|
nbconvert, static HTML) vs *view-time* (Processing/Strudel); **multiplicity** = one output
|
||||||
|
(UC-79) vs **N co-equal projections** (UC-83 weave+tangle; UC-47/48/54 moldable views).
|
||||||
|
|
||||||
|
### 2.2 Liveness: the live↔snapshot axis (from T6, bounded by T5)
|
||||||
|
Every derived view sits on a spectrum, and the more live the source, the more its static form
|
||||||
|
is a clearly-marked degrading snapshot:
|
||||||
|
|
||||||
|
```
|
||||||
|
static source ── captured output ── live-over-files ── view-time one-shot ── continuous/
|
||||||
|
(literate) (notebook UC-84) (GT/Lepiter) (Processing) interactive
|
||||||
|
── irreducibly
|
||||||
|
live/temporal
|
||||||
|
(Strudel: source
|
||||||
|
+ recording only)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Honesty rule (union-without-erasure):** a computed/live view must always declare *what it
|
||||||
|
is* — "captured at run N, environment unguaranteed" (UC-84), "one performance, time T, source
|
||||||
|
rev R" (UC-83/Strudel), "live render needs the runtime." Never present a snapshot as live or a
|
||||||
|
static page as capturing a live artifact.
|
||||||
|
|
||||||
|
## 3. The computational page-model shapes (T12)
|
||||||
|
|
||||||
|
The batch adds these page shapes (beyond prose / typed records / query-defined / inline-
|
||||||
|
embedded objects / typed-graph already catalogued):
|
||||||
|
|
||||||
|
1. **One-source-many-projections** (UC-83) — a source whose presented forms are co-equal
|
||||||
|
derivations (docs + code), each with output→source provenance; **named-chunk transclusion**
|
||||||
|
assembles fragments by name at derivation time (UC-32/44).
|
||||||
|
2. **Notebook** (UC-84) — **ordered/nestable cells** (Mathematica adds the outline tree)
|
||||||
|
where code cells own **embedded computed outputs** (the derived output is stored *inside*
|
||||||
|
the source) with **weak execution provenance**; outputs may be MIME blobs or **structured
|
||||||
|
re-evaluable values** (Mathematica) — a new point on the content-opacity spectrum.
|
||||||
|
3. **Program-as-page** (Processing) — canonical content = **source text**, presentation = an
|
||||||
|
**executable render** with **no cached output**; non-Markdown executable content.
|
||||||
|
4. **Live/temporal/generative content** (Strudel) — source canonical, render irreducibly
|
||||||
|
live; static = source + a marked recording.
|
||||||
|
|
||||||
|
All four reduce to **(source, derivation rule, projection with provenance + liveness)** — one
|
||||||
|
model, four positions.
|
||||||
|
|
||||||
|
## 4. Provenance & reproducibility (the honest weakness)
|
||||||
|
|
||||||
|
Computed output provenance is **real but fragile** everywhere: Jupyter/Mathematica
|
||||||
|
`execution_count`/`In`-`Out` can be **out-of-order**; **environment/versions/data are not
|
||||||
|
captured**; Strudel/Processing may be **non-deterministic**. Implication for the contract:
|
||||||
|
treat a computed output as a **snapshot with declared, incomplete provenance** (run id, source
|
||||||
|
rev, timestamp; environment "unguaranteed"), reusing **one snapshot-provenance machinery**
|
||||||
|
across notebooks, recordings, and renders (UC-84 is the template). This is consistent with
|
||||||
|
shard-wiki's existing "surface freshness/completeness, never imply more than you have"
|
||||||
|
(Oddmuse partial-history UC-82).
|
||||||
|
|
||||||
|
## 5. The recommendation
|
||||||
|
|
||||||
|
**Does shard-wiki need an executable/computational content capability? — Yes, but only as
|
||||||
|
recognition + projection + capability-gating, never as an execution engine.**
|
||||||
|
|
||||||
|
1. **Adopt the source/derivation/projection model** (no execution required). shard-wiki
|
||||||
|
**recognizes** computational content types (literate source, notebook, sketch, pattern),
|
||||||
|
**attaches the canonical source**, and **presents derived forms as projections with
|
||||||
|
provenance + liveness markers**. This alone delivers UC-83/UC-84 and the enrichment of
|
||||||
|
UC-54/55 — and needs **no kernel, no sandbox, no runtime**.
|
||||||
|
2. **Make execution a capability, off by default.** "Drive a derivation" (run tangle/weave,
|
||||||
|
re-execute a notebook, render a sketch, evaluate a pattern in the viewer) is a **gated
|
||||||
|
capability** with a **trust/sandbox** sub-concern (T11). Absent it, **degrade to the
|
||||||
|
captured snapshot / static render / recording** — the graceful-degradation rule, which the
|
||||||
|
batch shows always has an honest fallback (source is tiny and diffable everywhere).
|
||||||
|
3. **One projection model, two axes** (T16): projection-kind (replication vs derivation; with
|
||||||
|
timing + multiplicity facets) × liveness (live↔snapshot). The **moldable view registry**
|
||||||
|
(T7) is the unifying structure — an open, type-keyed set of co-equal projections, none
|
||||||
|
canonical-by-fact (display-canonical is policy).
|
||||||
|
4. **One snapshot-provenance record** reused for notebook outputs, renders, and recordings
|
||||||
|
(run id, source rev, timestamp, environment "unguaranteed").
|
||||||
|
5. **Hard boundaries** (design-bugs if violated): never host a kernel/runtime as the store;
|
||||||
|
**image-is-not-a-store** (attach exported files); never present a derivation without
|
||||||
|
output→source provenance; never imply a static view captures a live artifact.
|
||||||
|
|
||||||
|
**Net:** computational content is **in scope as a page-model + projection concern**, **out of
|
||||||
|
scope as an execution platform** — exactly the mechanism-over-policy, capability-aware,
|
||||||
|
degradable posture INTENT already mandates. No INTENT amendment is required; this extends the
|
||||||
|
page model and projection model within existing constraints.
|
||||||
|
|
||||||
|
## 6. Fold into SHARD-WP-0002
|
||||||
|
|
||||||
|
- **T12 (page model):** add the four computational shapes (§3); allow nestable cells and
|
||||||
|
structured re-evaluable outputs; "derived output may live inside the source" (notebook).
|
||||||
|
- **T16 (projection):** the **two-axis model** (§2) + the **moldable view registry** (§3/T7);
|
||||||
|
materialization-timing, multiplicity, continuity, and the live↔snapshot far end as explicit
|
||||||
|
projection metadata.
|
||||||
|
- **T11 (capabilities):** "derive/execute/render/evaluate" as gated capabilities with trust/
|
||||||
|
sandbox; default off → snapshot.
|
||||||
|
- **T15 (fidelity):** non-Markdown executable/computed content; lossy renders; the
|
||||||
|
structured-re-evaluable-value point on the content-opacity spectrum.
|
||||||
|
- **T13 (history):** paired-text (Jupytext) / cell-aware (nbdime) strategies for embedded-
|
||||||
|
output documents; outputs-as-derived (nbstripout ethos).
|
||||||
|
- **T14 (binding):** **image-is-not-a-store** boundary (export→files only).
|
||||||
|
|
||||||
|
## 7. Open questions (escalated)
|
||||||
|
|
||||||
|
1. Is **liveness** (and "irreducibly live / no faithful static form") an explicit first-class
|
||||||
|
metadata flag on every projection, so the union renders the honest fallback automatically?
|
||||||
|
(T5/T6 far-end question.)
|
||||||
|
2. Does shard-wiki **ever drive a derivation** (sandboxed), or strictly attach + present
|
||||||
|
snapshots? (Recurs UC-56/UC-83/UC-84/T4/T5 — a single capability/trust policy decision.)
|
||||||
|
3. Is a computed output's **structured re-evaluable value** (Mathematica/Wolfram) modeled as a
|
||||||
|
typed value or stored opaquely with provenance? (UC-55 open-Q #10; UC-84 Q3.)
|
||||||
|
4. Should **UC-83** and **UC-84** eventually merge as two positions of one "source +
|
||||||
|
derivations" shape, or stay distinct? (Kept distinct: UC-84's defining trait is *output
|
||||||
|
embedded in source with weak provenance*; UC-83's is *N co-equal external derivations*.)
|
||||||
|
|
||||||
|
## 8. Sources
|
||||||
|
|
||||||
|
The eight SHARD-WP-0004 dives: `research/260614-{literate-programming,mathematica,jupyter,
|
||||||
|
processing,strudel,squeak-pharo,glamorous-toolkit}-deep-dive/`. Prior projection/structure
|
||||||
|
anchors: `research/260614-{ikiwiki,logseq,zigzag}-deep-dive/`,
|
||||||
|
`research/260614-shard-spectrum-synthesis/`.
|
||||||
|
|
||||||
|
## 9. Traceability
|
||||||
|
|
||||||
|
No new UC (consolidation). Consolidates **UC-83, UC-84** and enrichments to **UC-32, UC-44,
|
||||||
|
UC-47, UC-48, UC-54, UC-55, UC-79, UC-37, UC-35**. Feeds **SHARD-WP-0002 T11/T12/T13/T14/T15/
|
||||||
|
T16** (see §6). Recommendation: computational content is **in scope as page-model + projection
|
||||||
|
mechanism, out of scope as an execution platform**; no INTENT amendment required.
|
||||||
@@ -43,4 +43,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
|
|||||||
| 2026-06-14 | `260614-mathematica-deep-dive/` | Mathematica Notebooks — the original computational notebook (`.nb` = a Wolfram expression); nestable cell groups, structured re-evaluable outputs, `Manipulate` live widgets, CDF; confirms UC-84 notebook shape is a genus; SHARD-WP-0004 T2; enrichment-only (reinforces UC-84; UC-54/55) |
|
| 2026-06-14 | `260614-mathematica-deep-dive/` | Mathematica Notebooks — the original computational notebook (`.nb` = a Wolfram expression); nestable cell groups, structured re-evaluable outputs, `Manipulate` live widgets, CDF; confirms UC-84 notebook shape is a genus; SHARD-WP-0004 T2; enrichment-only (reinforces UC-84; UC-54/55) |
|
||||||
| 2026-06-14 | `260614-squeak-pharo-deep-dive/` | Squeak & Pharo (image-based Smalltalk) — the live-object image (purest "live" end); image-is-not-a-store boundary (export→files only); Pharo Tonel/Iceberg externalizes code to git text; names the live↔snapshot projection axis; SHARD-WP-0004 **T6 + T8** (merged); boundary/enrichment-only, no new UC |
|
| 2026-06-14 | `260614-squeak-pharo-deep-dive/` | Squeak & Pharo (image-based Smalltalk) — the live-object image (purest "live" end); image-is-not-a-store boundary (export→files only); Pharo Tonel/Iceberg externalizes code to git text; names the live↔snapshot projection axis; SHARD-WP-0004 **T6 + T8** (merged); boundary/enrichment-only, no new UC |
|
||||||
| 2026-06-14 | `260614-processing-deep-dive/` | Processing / p5.js — program-as-page rendered live at view time (no cached output); adds materialization-timing + continuity facets to projection; execute-in-viewer = capability+trust; SHARD-WP-0004 T4; enrichment-only (UC-54/55) |
|
| 2026-06-14 | `260614-processing-deep-dive/` | Processing / p5.js — program-as-page rendered live at view time (no cached output); adds materialization-timing + continuity facets to projection; execute-in-viewer = capability+trust; SHARD-WP-0004 T4; enrichment-only (UC-54/55) |
|
||||||
| 2026-06-14 | `260614-strudel-deep-dive/` | Strudel.cc (TidalCycles JS) live-coding REPL — code as live time-based audio performance; the far live end (no faithful static form; static = source + marked recording); honesty test for graceful degradation; SHARD-WP-0004 T5; enrichment-only (UC-54/55) |
|
| 2026-06-14 | `260614-strudel-deep-dive/` | Strudel.cc (TidalCycles JS) live-coding REPL — code as live time-based audio performance; the far live end (no faithful static form; static = source + marked recording); honesty test for graceful degradation; SHARD-WP-0004 T5; enrichment-only (UC-54/55) |
|
||||||
|
| 2026-06-15 | `260614-computational-page-model-synthesis/` | **SHARD-WP-0004 synthesis** — *the computational page model*: source canonical / everything rendered is a projection; two axes (replication↔derivation, live↔snapshot); four computational page shapes; recommendation — executable content in scope as page-model+projection, out of scope as an execution platform (no INTENT amendment); feeds SHARD-WP-0002 T11–T16 |
|
||||||
976
spec/CoreArchitectureBlueprint.md
Normal file
976
spec/CoreArchitectureBlueprint.md
Normal file
@@ -0,0 +1,976 @@
|
|||||||
|
# CoreArchitectureBlueprint — shard-wiki
|
||||||
|
|
||||||
|
Status: **draft for review** · Date: 2026-06-15 · Owner: tegwick
|
||||||
|
|
||||||
|
The whole-system architecture for shard-wiki, synthesised from `INTENT.md`, the 84-entry
|
||||||
|
`UseCaseCatalog.md`, and the full research arc (`research/260608-*`, `research/260613-*`,
|
||||||
|
`research/260614-*` — ~23 wiki/knowledge systems plus two cross-dive syntheses). This is the
|
||||||
|
**core** blueprint: it defines the layers, the abstractions, and the load-bearing decisions
|
||||||
|
that everything else implements.
|
||||||
|
|
||||||
|
Scope relationship to the other specs:
|
||||||
|
|
||||||
|
- **`ArchitectureBlueprint.md`** (existing) is the **authorization & history sub-blueprint**
|
||||||
|
(the L0–L4 ladder). This document references it as the design of the cross-cutting
|
||||||
|
authorization layer (§9) and does not restate it.
|
||||||
|
- **`SHARD-WP-0002`** is the workplan that turns §6–§8 into
|
||||||
|
`spec/FederationArchitecture.md` + the adapter-contract section of
|
||||||
|
`spec/TechnicalSpecificationDocument.md`.
|
||||||
|
- **`UseCaseCatalog.md`** is the demand this architecture must satisfy; UC references below
|
||||||
|
are load tests, not decoration.
|
||||||
|
- **`WikiEngineCoreArchitecture.md`** designs shard-wiki's native, headless, API-first wiki
|
||||||
|
engine as a **canonical-mode shard backend** (one shard behind §6/§A — federation, union, and
|
||||||
|
projection stay here in the orchestrator, not in the engine). Added per the 2026-06-15 INTENT
|
||||||
|
amendment (decision `84ffdb48`, SHARD-WP-0013).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The thesis: *canonical vs derived* (three states)
|
||||||
|
|
||||||
|
Everything in shard-wiki follows from one organising decision — that state comes in exactly
|
||||||
|
**three kinds**, and only one of them is disposable:
|
||||||
|
|
||||||
|
> **1. Sharded-canonical** — content owned by each shard (shard sovereignty).
|
||||||
|
> **2. Coordination-canonical** — durable state *born inside shard-wiki* that encodes human
|
||||||
|
> or cross-shard decisions and exists nowhere else: overlays (the local truth against a
|
||||||
|
> read-only shard), curator equivalence bindings, alias tables, merge/reconciliation
|
||||||
|
> decisions. It is recorded as an **append-only decision log in the Git coordination
|
||||||
|
> journal** (event-sourced, §8.1); the *queryable current form* of that state (the effective
|
||||||
|
> alias table, the equivalence set) is a **derived fold** of the log — i.e. tier 3, not tier 2.
|
||||||
|
> What is canonical is the **log of decisions**, not any mutable snapshot of them.
|
||||||
|
> **3. Derived-disposable** — everything shard-wiki *computes* from (1)+(2): the union graph,
|
||||||
|
> equivalence index, query indexes, projections, views. It can be deleted and recomputed.
|
||||||
|
>
|
||||||
|
> **Canonical = sharded ∪ coordination. Derived = a pure function of canonical:**
|
||||||
|
> `derived = f(sharded, coordination)`.
|
||||||
|
|
||||||
|
This is the architectural form of "orchestrator, not engine." shard-wiki never *becomes* the
|
||||||
|
source of truth; it composes sources and records the decisions it makes about them. The
|
||||||
|
research earned the *files-canonical* half empirically — every serious system externalises its
|
||||||
|
durable truth to files+VCS and treats the rest as derived: Logseq (DataScript index over plain
|
||||||
|
files), ikiwiki (static HTML compiled from a git repo), Glamorous Toolkit / Lepiter (live
|
||||||
|
views over git-versioned JSON), Pharo (Tonel/Iceberg code as git text), Jupyter teams
|
||||||
|
(nbstripout — outputs are derived noise). The one tradition that refuses this — the Smalltalk
|
||||||
|
**image** — is exactly the one we record as a *boundary, not a backend*
|
||||||
|
(`research/260614-squeak-pharo-deep-dive`).
|
||||||
|
|
||||||
|
The earlier draft of this blueprint used a two-bucket framing ("canonical at the edges,
|
||||||
|
derived in the middle"). That was wrong by omission: it had no home for **coordination-
|
||||||
|
canonical** state, and so contradicted itself by listing curator bindings and alias tables as
|
||||||
|
"derived/rebuildable" when a human binding manifestly cannot be rebuilt. The three-state model
|
||||||
|
fixes that crack (review finding A-1) and makes `derived = f(canonical)` *literally* true.
|
||||||
|
|
||||||
|
Three consequences fall straight out, and they are the spine of the rest of this document:
|
||||||
|
|
||||||
|
1. **Graceful degradation is free.** If the derived tier is always recomputable, a backend
|
||||||
|
that can only be read is still a first-class participant — you just derive less from it.
|
||||||
|
2. **Provenance is tractable.** Because shard-wiki never claims to *be* the source, every
|
||||||
|
derived artifact can always point back to the canonical input it came from (union without
|
||||||
|
erasure is a structural property, not a feature bolted on).
|
||||||
|
3. **The derived tier is a pure function of canonical state.** `derived = f(sharded,
|
||||||
|
coordination)`. Bugs in the derived tier are recoverable by recompute; only the two
|
||||||
|
canonical tiers must be durably protected — sharded by each shard, coordination by the
|
||||||
|
Git journal (history). *Recomputability is a correctness property of the derived tier, not
|
||||||
|
a promise that a from-scratch rebuild is operationally cheap — see §8.4 and the
|
||||||
|
operational-envelope axis.*
|
||||||
|
|
||||||
|
### The dual narrow waist
|
||||||
|
|
||||||
|
Heterogeneity is mediated at exactly two interfaces, and nowhere else:
|
||||||
|
|
||||||
|
- **Bottom waist — the Shard Adapter Contract (§6).** Every backend, however weird, enters
|
||||||
|
through one versioned, capability-described interface.
|
||||||
|
- **Top waist — the Wiki Page Model (§7).** Every consumer, however demanding, sees one
|
||||||
|
backend-neutral, Markdown-first-but-stretchable page model.
|
||||||
|
|
||||||
|
Between the waists, core logic is written **once** against capabilities and the page model —
|
||||||
|
never against a specific backend. Adding TiddlyWiki or Notion or a git forge is writing an
|
||||||
|
adapter and declaring a capability profile, not editing core algorithms.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Architectural invariants
|
||||||
|
|
||||||
|
These are non-negotiable. Violating one is a design bug, not a tradeoff. They are INTENT's
|
||||||
|
principles fused with the research through-lines.
|
||||||
|
|
||||||
|
| # | Invariant | Source |
|
||||||
|
|---|-----------|--------|
|
||||||
|
| I-1 | **Orchestrator, not engine.** Core composes shards; it never replaces or homogenises them. | INTENT Stability Note |
|
||||||
|
| I-2 | **Three states; derived = f(canonical).** State is sharded-canonical, coordination-canonical (journal), or derived-disposable. The derived tier (union/index/projection) is a pure, recomputable function of the two canonical tiers; only canonical state is durably protected. | §1; Logseq/ikiwiki/GT through-line |
|
||||||
|
| I-3 | **Capability-awareness is data.** A binding's abilities are a *profile* (positions on spectra), read by generic core logic — not per-backend branches. | synthesis v3 §2; INTENT capability-aware adapters |
|
||||||
|
| I-4 | **Union without erasure.** Every page/revision/projection/overlay/view carries its provenance, freshness, liveness, and divergence. | INTENT; provenance-granularity spectrum (Wikibase) |
|
||||||
|
| I-5 | **Overlay before mutation.** Writes to anything below write-through land as drafts/patches/MRs first; no silent remote mutation. | INTENT |
|
||||||
|
| I-6 | **Git-addressable coordination.** Every information space has a Git-backed journal even when its shards are not git-native. | INTENT |
|
||||||
|
| I-7 | **Mechanism over policy.** Canonical-source, conflict, editorial, sync cadence are configurable presets, never hard-coded. | INTENT |
|
||||||
|
| I-8 | **Graceful degradation.** A limited backend is still usable as read-only / cache / projection / backup / patch target. | INTENT |
|
||||||
|
| I-9 | **Identity ≠ placement.** A page is an entity that may occupy N locations; address by identity, not by path. | Trilium note/branch; ZigZag |
|
||||||
|
| I-10 | **History is the floor.** Every write is a recoverable commit; recoverability, not gatekeeping, is the baseline protection. | ArchitectureBlueprint §2 |
|
||||||
|
| I-11 | **Authorization in core, authentication delegated.** Core decides who-may; an external provider says who-is. | INTENT; ArchitectureBlueprint |
|
||||||
|
| I-12 | **Not a file-sync daemon; not an execution platform.** Sync is wiki-page-semantic; computation is recognised+projected, not hosted. | INTENT; computational-page-model synthesis |
|
||||||
|
| I-13 | **Tenant-partitioned derived state.** Derived state is partitioned by tenant/root entity; no derived artifact spans tenants except via explicit, authorised cross-root federation. | §9.1; review B-3 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The layered architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌───────────────────────────────────────────────────────────────┐
|
||||||
|
│ L6 Consumers — Orchestrator API · CLI/agents · Web/Obsidian │
|
||||||
|
├───────────────────────────────────────────────────────────────┤
|
||||||
|
X-cut │ L5 Authorization (PEP/PDP, identity-provider iface) → │ X-cut
|
||||||
|
Prove- │ see ArchitectureBlueprint.md (L0–L4 ladder) │ Capa-
|
||||||
|
nance ├───────────────────────────────────────────────────────────────┤ bility
|
||||||
|
▲ │ L4 Union & Projection (DERIVED · rebuild=fallback) │ ▲
|
||||||
|
│ │ identity resolution · equivalence/chorus · union graph · │ │
|
||||||
|
│ │ replication+derivation projections · moldable view registry│ │
|
||||||
|
│ │ · derived query index │ │
|
||||||
|
│ ├───────────────────────────────────────────────────────────────┤ │
|
||||||
|
│ │ L3 Coordination (Git journal · overlay/patch engine · │ │
|
||||||
|
│ │ federation-model strategies · reconciliation) │ │
|
||||||
|
│ ├───────────────────────────────────────────────────────────────┤ │
|
||||||
|
│ │ L2 Wiki Page Model ── TOP WAIST ── │ │
|
||||||
|
│ │ backend-neutral pages · identity≠placement · span address ·│ │
|
||||||
|
│ │ provenance envelope · the page shapes │ │
|
||||||
|
│ ├───────────────────────────────────────────────────────────────┤ │
|
||||||
|
│ │ L1 Shard Adapter Contract ── BOTTOM WAIST ── │ │
|
||||||
|
│ │ versioned iface · capability profile (orthogonal) · │ │
|
||||||
|
│ │ attachment-mode binding · operation verbs │ │
|
||||||
|
└──── ├───────────────────────────────────────────────────────────────┤ ──┘
|
||||||
|
│ L0 Backends (not ours): git repos, wiki/ subdirs, Gitea/ │
|
||||||
|
│ GitLab/GitHub wikis, folders, Obsidian, WebDAV, Notion, │
|
||||||
|
│ Coulomb spaces, notebooks, … │
|
||||||
|
└───────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Provenance** and **Capability** are drawn as vertical rails because they are not layers —
|
||||||
|
they are present at every layer. A page object at L2 carries provenance; a projection at L4
|
||||||
|
carries provenance; an authz decision at L5 records the context under which content was read.
|
||||||
|
Likewise a capability profile declared at L1 is consulted at L3 (can we write-through?), L4
|
||||||
|
(can we delegate a query?), and L5 (can this principal even reach the op?).
|
||||||
|
|
||||||
|
The dependency rule is strict and downward, and it tracks the **three states (§1)**, not the
|
||||||
|
layer numbers: **the derived-disposable tier (the whole of L4) may be deleted and recomputed
|
||||||
|
from canonical state (sharded content at L1 + coordination-canonical state in the L3 journal).**
|
||||||
|
Nothing canonical may depend on derived state. Note the journal at L3 is *canonical* (it holds
|
||||||
|
overlays, bindings, aliases, merges); only L4 is disposable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Core abstractions (the vocabulary code must use)
|
||||||
|
|
||||||
|
Straight from INTENT, sharpened by research. New code maps onto these; it does not invent
|
||||||
|
parallel terms.
|
||||||
|
|
||||||
|
- **Shard** — an independently meaningful page store attached to a root entity, with
|
||||||
|
*sovereignty*: its own backend, capability profile, history, identity model, limits.
|
||||||
|
- **Root entity / information space** — the joined space shards attach to; the unit of
|
||||||
|
Git coordination and of multi-tenancy (a tenant maps to a root entity, ArchitectureBlueprint).
|
||||||
|
- **Shard adapter contract** — the versioned L1 interface; the bottom waist.
|
||||||
|
- **Capability profile** — a shard binding's position on each of the 15 spectra (§6) plus its
|
||||||
|
supported verbs. *The* data structure that drives degradation.
|
||||||
|
- **Wiki page model** — the L2 backend-neutral page; the top waist.
|
||||||
|
- **Page identity vs placement vs equivalence** — a page is an entity with a *stable handle*
|
||||||
|
(identity); it may have N placements (paths/shards); **addressing and transclusion key on
|
||||||
|
identity, but equivalence keys on content fingerprint *across* identities** (§7.2, I-9). The
|
||||||
|
three are distinct mechanisms — never conflate identity with a fingerprint.
|
||||||
|
- **Provenance envelope** — the metadata each artifact carries (source shard, freshness,
|
||||||
|
liveness, authz context, overlay status, divergence, lineage), stored **layered**: a
|
||||||
|
page-level envelope + span-level *deltas*, so per-span cost is near-zero when uniform (§7.3).
|
||||||
|
- **Coordination journal** — the L3 Git-backed, **append-only decision log** for a space: the
|
||||||
|
durable home of all **coordination-canonical** state (§1, §8.1) as *events* (overlay-created,
|
||||||
|
binding-made, alias-set, merge-decided), plus the content change-flow record. It is event-
|
||||||
|
sourced — committed, never overwritten; the queryable current coordination state is a derived
|
||||||
|
fold of it (§8.1).
|
||||||
|
- **Overlay** — a non-destructive local edit against a remote/read-only/limited shard,
|
||||||
|
representable as draft/patch/commit/MR before destructive apply. Coordination-canonical: an
|
||||||
|
unapplied overlay is the local truth and lives in the journal.
|
||||||
|
- **Projection** — a derived view of shard content. The default is a **plain lazy
|
||||||
|
replication-projection** (a freshness-stamped cache); only *source* content needing
|
||||||
|
transform/evaluate uses the **derivation-projection** extension point with its two-axis
|
||||||
|
typing (kind × liveness) and the moldable view registry (§8.4–§8.5).
|
||||||
|
- **Federation model** — the selected coordination strategy for a space (§ taxonomy, T17).
|
||||||
|
- **Shard mode** — read-only · write-through · mirrored · projected · cached · canonical
|
||||||
|
(a *policy* selection constrained by the capability profile).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Why "layered" and not "pipeline" or "plugin-bus"
|
||||||
|
|
||||||
|
Two rejected alternatives, recorded so the choice is legible:
|
||||||
|
|
||||||
|
- **A sync pipeline** (source → transform → sink) was rejected: it implies a privileged
|
||||||
|
direction and a canonical sink, which violates shard sovereignty (I-1) and union-without-
|
||||||
|
erasure (I-4). shard-wiki is a *star* (many shards ↔ one space), not a pipe.
|
||||||
|
- **A flat plugin bus** (every backend a peer plugin emitting events) was rejected as the
|
||||||
|
*top-level* shape: it has no narrow waist, so heterogeneity leaks into every consumer.
|
||||||
|
We keep the plugin idea but confine it to L1 (adapters) and L3 (federation strategies),
|
||||||
|
behind the waists.
|
||||||
|
|
||||||
|
The layered-with-rails shape is what makes I-2/I-3/I-4 hold simultaneously.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Bottom waist — the Shard Adapter Contract (L1)
|
||||||
|
|
||||||
|
The single most important design decision in the project: **the adapter contract models
|
||||||
|
positions on capability spectra, not a flat checklist of boolean verbs.** A backend is not
|
||||||
|
"can/can't merge"; it sits *somewhere* on the merge spectrum, and federation operations
|
||||||
|
degrade by position. This is the lesson of putting ~23 systems in one matrix
|
||||||
|
(`research/260614-shard-spectrum-synthesis`, v3).
|
||||||
|
|
||||||
|
### 6.1 The fifteen capability spectra
|
||||||
|
|
||||||
|
Each binding declares a position on each axis. Core algorithms read these positions; there is
|
||||||
|
no per-backend code in core (I-3).
|
||||||
|
|
||||||
|
1. **Addressing granularity** — none → path → page-level store-id → in-file span → in-file
|
||||||
|
block id (Logseq `id::`) → store-UUID → portable tumbler (Xanadu, the unreached ideal)
|
||||||
|
2. **Content identity** — none → path/title → fingerprint → span-set
|
||||||
|
3. **Identity vs placement** — path=identity → identity separated from placement (Trilium
|
||||||
|
note/branch = a DAG)
|
||||||
|
4. **Structure** — flat MD → frontmatter/`key::` → `%META%` → typed objects → DB schema+
|
||||||
|
relations → object-graph/ontology → computed (inherited+templated) → typed-graph statements
|
||||||
|
5. **History** — none → internal-only / CRDT-log → open-file → git-native
|
||||||
|
6. **Merge model** — none → git/text → conflict-notes/keep-both → native-CRDT → coexist-with-rank
|
||||||
|
7. **Native query** — none → text → build-your-own derived index → datalog/graph → DB query → SPARQL
|
||||||
|
8. **Translation** — native → lossless → lossy-with-fidelity-report (incl. HTML)
|
||||||
|
9. **Attachment mode** — file-store (native | interchange-mirror) → git-IS-store → in-engine-host
|
||||||
|
→ local-REST → external-API → direct-DB → CRDT-replica → P2P/no-central-endpoint
|
||||||
|
10. **Operational envelope** — local/unbounded → realtime CRDT/WebSocket → rate-limited/
|
||||||
|
eventually-consistent/paginated
|
||||||
|
11. **Access grant** — open → token → OAuth scoped+revocable → P2P key/invite → enterprise ACL
|
||||||
|
12. **Content opacity** — plaintext → structured re-evaluable value → encrypted whole-shard →
|
||||||
|
per-item → proprietary-lossy-exportable
|
||||||
|
13. **Write granularity** — whole-file (TiddlyWiki) → per-page → section/anchor → per-block → story-item
|
||||||
|
14. **Provenance granularity** — per-shard → per-page → per-edit → per-statement/value (Wikibase rank+refs)
|
||||||
|
15. **Computational / liveness** — static source → captured-output snapshot → live-over-files →
|
||||||
|
view-time render → irreducibly-live/temporal
|
||||||
|
|
||||||
|
### 6.2 Operation verbs
|
||||||
|
|
||||||
|
`read, write, diff, merge, lock, version, publish, notify, transclude-source,
|
||||||
|
translate-syntax, structured-payload, derive-projection, execute/evaluate`. The last two are
|
||||||
|
**gated, off by default** (§8, computational content). Verb support is part of the profile and
|
||||||
|
must reconcile with the federation-ops capability matrix (SHARD-WP-0002 T10).
|
||||||
|
|
||||||
|
### 6.3 Attachment-mode taxonomy (axis 9, expanded)
|
||||||
|
|
||||||
|
A backend may offer **several** modes; attach mode is a **per-binding, capability-gated
|
||||||
|
choice**, with one declared authoritative. Modes: file-store (native vault/folder *or* an
|
||||||
|
interchange/sync mirror), **git-IS-store** (the home case — forge wikis & ikiwiki: git is the
|
||||||
|
store *and* the journal at once, resolving the engine-mirror write-race), in-engine hosted
|
||||||
|
adapter (XWiki component, Obsidian/Logseq/Roam plugin, Trilium script), local-REST (Joplin
|
||||||
|
Data API, Trilium ETAPI), external-API-only (Notion), direct-DB (MojoMojo schema→model),
|
||||||
|
CRDT-replica (Anytype/AFFiNE/AppFlowy), P2P/no-central-endpoint. **Boundary:** a monolithic
|
||||||
|
live-memory blob (Smalltalk image, a kernel) is **never** an attach target — it participates
|
||||||
|
only via export→files (I-12).
|
||||||
|
|
||||||
|
### 6.4 Contract rules
|
||||||
|
|
||||||
|
- **Versioned interface** (Foswiki::Store + Foswiki::Meta is the proof that a stable
|
||||||
|
store-interface-with-swappable-backends works). Capability discovery is a static profile
|
||||||
|
with optional runtime negotiation.
|
||||||
|
- **Backend-swap tolerance** — shard identity/provenance survives a substrate change
|
||||||
|
(RCS↔PlainFile, folder→Git, Logseq file→SQLite): bind to *capabilities*, not to "it's files."
|
||||||
|
- **Absence is first-class** — the profile must express *can't* cleanly (Oddmuse floor), so
|
||||||
|
degradation paths are explicit, never guessed.
|
||||||
|
|
||||||
|
### 6.5 Orthogonal core, implied positions, and the interaction subset
|
||||||
|
|
||||||
|
Fifteen independent ordinal axes is *descriptively* right but would be *operationally* a mess
|
||||||
|
if treated as fifteen free dimensions: the axes are **not orthogonal**, and a degradation
|
||||||
|
function over all 15 jointly is the flat-checklist problem returning in higher dimensions
|
||||||
|
(review D-1). Three rules tame it.
|
||||||
|
|
||||||
|
**(a) A small orthogonal core; the rest are implied.** Most axes are *correlated* and collapse
|
||||||
|
to a few independent choices. The **core axes** an adapter must independently declare:
|
||||||
|
|
||||||
|
1. **Substrate** → drives attachment-mode, history, merge, and native-query positions together
|
||||||
|
(git-IS-store ⟹ history=git-native ⟹ merge=git/text ⟹ query=build-your-own-index;
|
||||||
|
relational-DB ⟹ direct-DB attach ⟹ DB-version-row history ⟹ DB query).
|
||||||
|
2. **Write granularity** → drives addressing granularity and the overlay/patch shape.
|
||||||
|
3. **Content opacity** → drives translation and (where encrypted) collapses native-query.
|
||||||
|
4. **Operational envelope** → drives freshness mode (§8.8) and rebuild expectations (§8.7).
|
||||||
|
5. **Access grant** → independent (authz, L5).
|
||||||
|
6. **Computational/liveness** → independent (projection kind, §8.5).
|
||||||
|
|
||||||
|
The remaining axes are **implied/derived** from these via published implication rules; an
|
||||||
|
adapter *may* override an implied position, but the default is computed, not hand-set. This
|
||||||
|
turns ~15 free dimensions into ~6 independent ones plus derivations — fewer things to get
|
||||||
|
wrong, and impossible combinations become unrepresentable.
|
||||||
|
|
||||||
|
**(b) Implication rules forbid impossible profiles.** E.g. `attachment=git-IS-store ⟹
|
||||||
|
history≥git-native`; `opacity=encrypted-whole-shard ⟹ native-query=none ∧ translation≤opaque`;
|
||||||
|
`merge=native-CRDT ⟹ history=CRDT-log ∧ envelope=realtime`. A profile that violates an
|
||||||
|
implication is rejected at registration — capability-as-data (I-3) with integrity constraints.
|
||||||
|
|
||||||
|
**(c) The degradation function reads a *named, small* interaction subset — not all pairs.**
|
||||||
|
"No per-backend code" is only credible if we say *which* axis interactions the generic logic
|
||||||
|
actually consults. They are:
|
||||||
|
|
||||||
|
| Operation | Axes consulted (jointly) |
|
||||||
|
|-----------|--------------------------|
|
||||||
|
| **write / overlay-apply** | write-granularity × merge-model × history × access-grant |
|
||||||
|
| **transclude / address a span** | addressing-granularity × write-granularity × identity-vs-placement |
|
||||||
|
| **project / cache** | operational-envelope × computational-liveness × content-opacity |
|
||||||
|
| **query** | native-query × content-opacity (encrypted ⇒ derive-index-or-none) |
|
||||||
|
| **translate** | translation × content-opacity × structure |
|
||||||
|
| **federate** | substrate × history × merge (per the §8.3 model) |
|
||||||
|
|
||||||
|
Everything else is a single-axis check. This table *is* the degradation contract: it is small,
|
||||||
|
enumerated, and testable — the proof obligation behind "core logic written once."
|
||||||
|
|
||||||
|
### 6.6 Conformance — profiles are verified, never self-asserted
|
||||||
|
|
||||||
|
Capability-as-data (I-3) and the entire degradation contract (§6.5) rest on one assumption:
|
||||||
|
**the profile tells the truth.** If an adapter declares `merge=git/text` but corrupts merges,
|
||||||
|
or claims `notify` and never emits, it silently poisons every degradation decision in core —
|
||||||
|
the failure is invisible because core *believed the data* (review B-2). So the profile is not
|
||||||
|
taken on trust:
|
||||||
|
|
||||||
|
- **The contract ships a versioned conformance suite.** A published battery that, given a live
|
||||||
|
binding, **exercises each declared verb and each declared spectrum position and checks that
|
||||||
|
observed behaviour matches the claim** (a `write` round-trips; a `diff` is real; `notify`
|
||||||
|
actually fires; an "encrypted/opaque" shard genuinely refuses plaintext query; an
|
||||||
|
implication-rule position, §6.5(b), holds). The suite is versioned *with* the contract, so an
|
||||||
|
adapter proves conformance against a known contract version.
|
||||||
|
- **Passing conformance is an admissibility precondition.** A binding that fails (declares a
|
||||||
|
capability it does not honour) is **rejected at registration**, not run in production with a
|
||||||
|
lying profile. Capability discovery (§6.4) therefore yields a *verified* profile.
|
||||||
|
- **Self-reported, then verified.** Adapters still *declare* their profile (discovery stays
|
||||||
|
cheap); conformance *verifies* the declaration. The two together are what make I-3 and §6.5
|
||||||
|
sound rather than aspirational — degradation logic acts on verified data.
|
||||||
|
- **Mismatch is data, not a crash.** A conformance gap is reported as a precise
|
||||||
|
capability-by-capability diff (what was claimed vs observed), so an adapter author fixes the
|
||||||
|
profile or the code; degraded-but-honest registration (drop the unsupported claim) is allowed.
|
||||||
|
|
||||||
|
This is the same discipline a versioned store interface needs in general (the `Foswiki::Store`
|
||||||
|
lineage that inspired the contract): a backend may only participate behind the interface if it
|
||||||
|
*demonstrably* behaves as the interface says.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Top waist — the Wiki Page Model (L2)
|
||||||
|
|
||||||
|
Backend-neutral, **Markdown-first but stretchable many ways at once**. The page model is the
|
||||||
|
lingua franca every consumer sees; an adapter's job is to project its backend into this model
|
||||||
|
(read) and accept overlays back (write), within its capabilities.
|
||||||
|
|
||||||
|
### 7.1 Page shapes the model must carry
|
||||||
|
|
||||||
|
- **Prose Markdown** — the baseline.
|
||||||
|
- **Typed / computed records** — frontmatter/`%META%`/XObjects/Notion DB rows; **computed
|
||||||
|
metadata** (Trilium inherited+templated) represented as *effective-vs-own with per-attribute
|
||||||
|
provenance*.
|
||||||
|
- **Typed-graph statements** — Wikibase claim + qualifiers + references + rank (structure
|
||||||
|
far-end).
|
||||||
|
- **Inline-embedded objects** — Quip/Notion spreadsheets & live apps inside prose.
|
||||||
|
- **Non-Markdown assets** — drawings, canvases, images: typed asset / opaque blob / pluggable
|
||||||
|
content-type registry, never silent-flattened.
|
||||||
|
- **The four computational shapes** (§8): one-source-many-projections, notebook (embedded
|
||||||
|
computed output), program-as-page, live/temporal.
|
||||||
|
|
||||||
|
All shapes reduce to a common skeleton: **`(content | source, structure, provenance envelope,
|
||||||
|
[derivation rule])`**. The page model stores the richest faithful form as canonical and treats
|
||||||
|
any Markdown rendering of a non-Markdown shape as a *lossy projection* (I-4 + fidelity report).
|
||||||
|
|
||||||
|
### 7.2 Identity, placement, addressing — three distinct concepts
|
||||||
|
|
||||||
|
The earlier draft used "identity" for two different things and (worse) suggested deriving page
|
||||||
|
identity from a content fingerprint — which would make *editing a page change its identity* and
|
||||||
|
break every reference to it (review bug B-1). They are pulled apart here:
|
||||||
|
|
||||||
|
- **Page identity — a *stable handle*.** A shard-scoped, durable key that **survives edits**:
|
||||||
|
the backend's native page/note id where one exists (Roam/Notion/Trilium uid, a git path
|
||||||
|
treated as a name, a wiki page name), wrapped in a shard scope so it survives projection and
|
||||||
|
never collides across shards. Identity is *assigned/minted, not computed from content*.
|
||||||
|
References, placement, transclusion targets, and overlays all key on identity.
|
||||||
|
- **Placement — *where* an identity sits.** One identity → N placements (paths/shards) = a DAG;
|
||||||
|
no single canonical path (I-9). Placement can change without changing identity.
|
||||||
|
- **Content equivalence — *detecting sameness*, never identity.** A **content fingerprint** (or
|
||||||
|
span-set overlap) identifies a *version / a piece of content*, used to detect that two
|
||||||
|
*distinct identities* hold the same or derived content (the equivalence/chorus mechanism,
|
||||||
|
§8.4). A fingerprint is never a page's identity: same page, edited → new fingerprint, **same
|
||||||
|
identity**; two pages, identical content → same fingerprint, **different identities**.
|
||||||
|
- **Span addressing** — a sub-page address within an identity: adopt native span IDs where
|
||||||
|
minted (Roam `:block/uid`, Logseq `id::`, Notion/CRDT UUID); else a *position* address
|
||||||
|
(path+range) or a *content-fingerprint* address for equivalence/transclusion. The Xanadu
|
||||||
|
tumbler is the portable ideal the scheme aims at without requiring.
|
||||||
|
- **Provenance envelope** rides on pages and spans (see §7.3 for its layered, low-cost form).
|
||||||
|
|
||||||
|
So the chain is: **identity (stable) → placements (N, mutable) → equivalence (cross-identity
|
||||||
|
sameness, fingerprint-based)** — three concepts, three mechanisms, never conflated.
|
||||||
|
|
||||||
|
### 7.3 Provenance is layered, not per-span-duplicated
|
||||||
|
|
||||||
|
A provenance envelope on *every span* (source shard, freshness, liveness, overlay status,
|
||||||
|
authz context, divergence, lineage) would, at block granularity, mean ~10k near-identical
|
||||||
|
envelopes for a 10k-block page — provenance dwarfing content (review D-2). The fix is the exact
|
||||||
|
pattern the page model already uses for Trilium's computed metadata: **effective-vs-own**.
|
||||||
|
|
||||||
|
- **Page-level envelope** holds the values that are uniform across the page (almost always:
|
||||||
|
source shard, observed-at, liveness, authz context).
|
||||||
|
- **Span-level deltas** record *only where a span differs* from its page envelope — a
|
||||||
|
transcluded span from another shard, an overlaid span, a span that diverges. A span with no
|
||||||
|
delta inherits the page envelope at zero storage cost.
|
||||||
|
- **Effective provenance** for any span = page envelope ⊕ span delta, computed on read.
|
||||||
|
|
||||||
|
Per-span cost is therefore **near-zero in the common (uniform) case** and pays only for genuine
|
||||||
|
heterogeneity — the same "carry only the difference" principle, applied to shard-wiki's own
|
||||||
|
metadata. Provenance remains complete (I-4); it is just not redundantly materialised.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Coordination, federation & projection
|
||||||
|
|
||||||
|
### 8.1 Coordination journal (L3) — Git as the spine
|
||||||
|
|
||||||
|
Every information space has a Git-backed coordination journal (I-6). It records cross-shard
|
||||||
|
operations (fork, import, reconcile, overlay-apply, space-branch) and **is** the history floor
|
||||||
|
(I-10). For git-IS-store shards the shard's own git log *is* this journal; for non-git shards
|
||||||
|
the journal supplements (begins-now / mirrors-forward / snapshots-replica) or imports
|
||||||
|
(backfill open file history). History portability is a spectrum, handled per profile (axis 5).
|
||||||
|
|
||||||
|
**The journal is an append-only decision log; current coordination state is a derived fold
|
||||||
|
(review B-3).** The first draft said coordination-canonical state "lives in the journal"
|
||||||
|
without saying how Git — excellent for history, poor for mutable structured state — represents
|
||||||
|
an alias table or an equivalence graph. Resolution: **event sourcing.** The journal stores
|
||||||
|
*decisions as events* (`overlay-created`, `binding-made`, `alias-set`, `merge-decided`,
|
||||||
|
`page-forked`), append-only and git-addressable (so history/patch/review/backup over
|
||||||
|
coordination state come for free — I-6 is *strengthened*, not bypassed). The **queryable
|
||||||
|
current state** (the effective alias table, the live equivalence set) is a **derived fold** of
|
||||||
|
the log — tier-3 disposable, indexed like any other derived structure (§8.7), rebuilt by
|
||||||
|
replaying the log. So "all equivalences touching X" is an index lookup, not an O(scan) of Git.
|
||||||
|
This is the clean form of the §1 three-state model: **the log is canonical; its folded current
|
||||||
|
state is derived.**
|
||||||
|
|
||||||
|
**Concurrency: who may append (review B-1).** A multi-tenant L4 deployment runs several
|
||||||
|
orchestrator instances, so "the journal is local Git, single writer" is not given. The model:
|
||||||
|
|
||||||
|
- **One *append authority* per information space.** Appends to a space's log are serialized
|
||||||
|
through a single logical writer (a per-space lease/leader; instances without the lease forward
|
||||||
|
their append intents to it). This makes the log a **totally-ordered event sequence** per space
|
||||||
|
— the ordering authority §8.6 relies on — without a distributed transaction. Spaces are
|
||||||
|
independent, so this scales horizontally *across* spaces (the unit of partition is the space /
|
||||||
|
root entity, matching the tenant partition, I-13); it is a per-space serialization point, not
|
||||||
|
a global one.
|
||||||
|
- **Git is the durable, addressable form; appends are commits** (or fast objects batched into
|
||||||
|
commits) under the lease — no concurrent-writer merge races because there is one writer at a
|
||||||
|
time per space.
|
||||||
|
- **Read-your-writes** holds within a space because every reader resolves current state from
|
||||||
|
the same ordered log (or its fold); across spaces there is no shared state to be inconsistent.
|
||||||
|
- **HA / failover:** the lease is time-bounded and re-grantable; a failed append-authority is
|
||||||
|
replaced and resumes from the log's head (the log is the recovery point). A partition that
|
||||||
|
splits the authority degrades that *space* to read-only until a single writer is re-elected —
|
||||||
|
it never forks the log (availability yields to log integrity; an explicit, stated trade).
|
||||||
|
- **Open residual (→ §12, O-3-adjacent):** whether very high append rates need per-space log
|
||||||
|
*sharding* (sub-logs merged by a deterministic order) is an implementation spike, not an
|
||||||
|
architectural change.
|
||||||
|
|
||||||
|
**History must stay recoverable *and* bounded (review C-3).** "Every write is a commit" + open
|
||||||
|
L0 means an unbounded, bot-/vandalism-amplified journal that eventually degrades Git itself.
|
||||||
|
Recoverability (I-10) is non-negotiable, so the answer is *compaction, not deletion*:
|
||||||
|
|
||||||
|
- **Routine git maintenance** — background `gc`/repack, commit-graph, and (for very large
|
||||||
|
spaces) partial-clone / sparse strategies; operational, no semantic change.
|
||||||
|
- **Squash-compaction of low-value churn (policy, §10)** — long runs of rapid same-author
|
||||||
|
edits or revert-pairs can be folded into checkpoint commits *while preserving the recoverable
|
||||||
|
endpoints*; what is squashed is configurable and always leaves the content recoverable (it
|
||||||
|
compacts the *path*, not the *reachable states*).
|
||||||
|
- **Per-shard history offload** — a git-IS-store shard keeps its own history in its own repo;
|
||||||
|
the coordination journal references it rather than duplicating it (the journal records
|
||||||
|
*coordination* events, not a second copy of every shard commit).
|
||||||
|
- **Anti-abuse hooks (policy)** — rate-limiting / quarantine for anonymous L0 writers feed the
|
||||||
|
authz/policy layer; they throttle *abuse*, never legitimate history. Recoverability is the
|
||||||
|
floor; bounding is how it survives at scale.
|
||||||
|
|
||||||
|
### 8.2 Overlay / patch engine (L3)
|
||||||
|
|
||||||
|
The default write path for anything below write-through capability (I-5): an edit becomes a
|
||||||
|
draft → patch/commit → MR, applied destructively only on explicit intent and only where the
|
||||||
|
profile + policy both permit. This is what lets a read-only or rate-limited or lossy backend
|
||||||
|
still be *edited* safely.
|
||||||
|
|
||||||
|
### 8.3 Federation is plural & composable (L3) — the model taxonomy
|
||||||
|
|
||||||
|
Federation is not one mechanism. shard-wiki selects a **federation model per space and
|
||||||
|
composes per shard** (mechanism over policy, I-7):
|
||||||
|
|
||||||
|
| Model | Anchor | Coordination shape |
|
||||||
|
|-------|--------|--------------------|
|
||||||
|
| **Fork + journal** (default home case) | Federated Wiki | copy-with-provenance + per-page action journal (story = replay) |
|
||||||
|
| **VCS-replication + ping** | ikiwiki | git clone/pull/push + change-ping |
|
||||||
|
| **Query-time graph-join** | Wikibase SPARQL `SERVICE` | join remote graphs at query time, no copy |
|
||||||
|
| **Feed aggregation** | RSS/Atom | inbound feed → pages |
|
||||||
|
| **Activity streams** | ActivityPub | Create/Update events, notify or content-bearing |
|
||||||
|
| **Engine-mirror** | Wiki.js DB↔Git | engine syncs its own store to a git mirror |
|
||||||
|
|
||||||
|
### 8.4 Union & projection (L4) — the derived cache
|
||||||
|
|
||||||
|
This whole layer is **derived-disposable**: recomputable from canonical state — sharded
|
||||||
|
content + the **coordination-canonical** inputs in the journal (I-2). Crucially, the *automatic*
|
||||||
|
equivalence results are derived, but the **human/curatorial inputs they consume — alias tables
|
||||||
|
and curator equivalence bindings — are coordination-canonical (they live in the journal), not
|
||||||
|
derived**; recompute reads them, never regenerates them. It comprises:
|
||||||
|
|
||||||
|
- **Identity resolution & equivalence** — detect "same topic / derived content" path-
|
||||||
|
independently from *derived* signals (content fingerprint, span-set overlap) **plus** the
|
||||||
|
*coordination-canonical* inputs (alias table, curator binding); present as
|
||||||
|
**chorus-of-voices** or designated-canonical (a *policy* preset). (Scaling: §8.7.)
|
||||||
|
- **Union graph** — the navigable join of pages, links, and dimensions (namespace, genealogy,
|
||||||
|
version, shard, equivalence). A *derived lens over canonical files+journal, never a new
|
||||||
|
store* (the ZigZag boundary).
|
||||||
|
- **Transclusion** — one **reference-not-copy** primitive unifying Xanadu transclusion, ZigZag
|
||||||
|
clone, Roam/Obsidian/Logseq embed, Notion synced block, Trilium note-cloning, and literate
|
||||||
|
named-chunk assembly, over the addressable union.
|
||||||
|
- **Projection — trivial by default, extensible for the tail.** The 95% case (Markdown in a
|
||||||
|
shard) must cost nothing conceptually, so:
|
||||||
|
- **Default = plain lazy replication-projection** — a freshness-stamped cache of remote
|
||||||
|
content (§8.8). This is *the* projection for ordinary pages; it needs no taxonomy, no
|
||||||
|
liveness reasoning, no registry. Most shards never touch anything below.
|
||||||
|
- **Extension point — derivation-projection** — invoked *only* for content that is a
|
||||||
|
*source* needing transform/compile/weave/evaluate (computational/typed content, §8.5). It
|
||||||
|
adds the liveness axis (static → captured → live-over-files → view-time → irreducibly-live)
|
||||||
|
and facets (materialization timing, multiplicity, continuity); the irreducibly-live far end
|
||||||
|
has no faithful static form (source + a marked recording). A binding that never serves such
|
||||||
|
content never instantiates any of this.
|
||||||
|
- Both kinds stamp freshness + provenance; only derivation carries the liveness machinery.
|
||||||
|
- **Moldable view registry — also an extension point, not a tax on every page.** Where a content
|
||||||
|
type offers multiple co-equal views (typed/computed/dimensional content), they are registered
|
||||||
|
as an **open, type-keyed set, none canonical-by-fact** (display-canonical is policy; GT prior
|
||||||
|
art, answers the "pluggable content-type registry" question). An ordinary Markdown page has
|
||||||
|
exactly one view and never consults the registry — the registry is queried only when a type
|
||||||
|
declares >1 view.
|
||||||
|
- **Derived query index** — delegate to a shard's native query engine where present
|
||||||
|
(Roam/Logseq Datalog, Notion DB query, XWiki XWQL, Wikibase SPARQL); else build a derived
|
||||||
|
index over the projection (the Logseq DataScript-over-files pattern). The index is
|
||||||
|
disposable (I-2).
|
||||||
|
|
||||||
|
### 8.5 Computational / executable content — the scope decision
|
||||||
|
|
||||||
|
**In scope as a page-model + projection concern; out of scope as an execution platform.**
|
||||||
|
shard-wiki *recognises* computational types, attaches the **canonical source**, and presents
|
||||||
|
derived forms as **provenance- and liveness-marked projections**. Driving a derivation
|
||||||
|
(tangle/weave, re-execute a notebook, render a sketch, evaluate a pattern) is a **gated
|
||||||
|
capability, off by default, with a trust/sandbox concern, degrading to a captured snapshot**.
|
||||||
|
One snapshot-provenance record (run id, source rev, timestamp, environment "unguaranteed")
|
||||||
|
serves notebooks, renders, and recordings alike. **No INTENT amendment is required** — this
|
||||||
|
lives inside the existing page model (L2) and projection model (L4).
|
||||||
|
|
||||||
|
### 8.6 Consistency, concurrency & conflict model
|
||||||
|
|
||||||
|
INTENT makes real-time cross-shard consistency a non-goal — but "no strong consistency" is not
|
||||||
|
the same as "no defined consistency." This is the guarantee shard-wiki *does* offer, and the
|
||||||
|
mechanism (not policy) that makes concurrent editing safe (review bug B-2).
|
||||||
|
|
||||||
|
**The consistency guarantee — causal, anchored on the journal:**
|
||||||
|
|
||||||
|
- **Read-your-writes for coordination-canonical state.** Once an overlay/binding/merge is
|
||||||
|
appended to the space's decision log, every reader of that space sees it — because the log is
|
||||||
|
a **single totally-ordered sequence per space** (one append authority, §8.1), and all readers
|
||||||
|
resolve current state from that one order. The guarantee holds across orchestrator instances,
|
||||||
|
not just within one process; it is cheap because ordering is per-space, never global.
|
||||||
|
- **Causal consistency across the derived tier.** The union/index/projections reflect a causal
|
||||||
|
cut of `(sharded inputs seen so far, journal)`. Effects never appear before their causes; a
|
||||||
|
projection that has seen journal commit *C* has seen everything *C* depends on.
|
||||||
|
- **Eventual convergence for sharded-canonical inputs.** Remote shard content is pulled
|
||||||
|
asynchronously (lazily or by notify/poll, §8.7); the union converges to each shard's latest
|
||||||
|
*as observed*, bounded by the shard's operational envelope. Freshness is always *shown*
|
||||||
|
(provenance envelope), never faked — a stale projection is labelled stale, not wrong.
|
||||||
|
|
||||||
|
So: **strong + read-your-writes for what shard-wiki owns (the journal); causal for what it
|
||||||
|
derives; eventual + freshness-labelled for what shards own.** No global clock, no distributed
|
||||||
|
transaction, no two-phase commit across shards — none is needed, because shard-wiki coordinates
|
||||||
|
rather than controls.
|
||||||
|
|
||||||
|
**Conflict detection & representation is core mechanism; only resolution is policy (I-7).**
|
||||||
|
The split the earlier draft elided:
|
||||||
|
|
||||||
|
- **Detection (core).** Divergence is detected structurally: two identities resolve as
|
||||||
|
equivalent (§8.4) but their content fingerprints differ, or an overlay's base revision no
|
||||||
|
longer matches the shard's current revision. Detection is always on; it is never optional.
|
||||||
|
- **Representation (core).** A detected conflict is **first-class data in the union**, not an
|
||||||
|
error: equivalent-but-divergent pages are presented as a **coexisting set** (the
|
||||||
|
chorus/keep-both representation), each fully attributed, with the divergence recorded in the
|
||||||
|
provenance envelope (union without erasure — a conflict is information, not a failure).
|
||||||
|
- **Resolution (policy).** *Which* version wins, or whether they stay coexisting, is a
|
||||||
|
configurable preset (§10): chorus / designated-canonical / git-merge / vote-to-merge /
|
||||||
|
overlay-only. Core never hard-codes one.
|
||||||
|
|
||||||
|
**Overlay-apply under source drift (the concurrent-write case).** An overlay carries the
|
||||||
|
**base revision** of the shard content it was authored against. On apply, core compares base to
|
||||||
|
the shard's current revision:
|
||||||
|
|
||||||
|
- *unchanged* → apply (fast-forward), commit to journal, propagate if the profile permits;
|
||||||
|
- *changed, non-overlapping* → three-way merge where the merge capability allows (axis 6),
|
||||||
|
else keep-both;
|
||||||
|
- *changed, overlapping* → **refuse + re-present** as a conflict (above); never silently
|
||||||
|
clobber (I-5, no silent remote mutation). The unapplied overlay remains coordination-
|
||||||
|
canonical and valid against its base.
|
||||||
|
|
||||||
|
**Ordering.** The journal commit is the ordering authority for coordination-canonical effects;
|
||||||
|
a shard-native write is only *acknowledged* in the journal after the adapter confirms it, so a
|
||||||
|
crash between journal-intent and shard-write is recoverable (the intent is replayable, the
|
||||||
|
write is idempotent-keyed on identity+base-rev). Cross-shard operations are ordered by their
|
||||||
|
journal commits, giving the causal cut above.
|
||||||
|
|
||||||
|
**Residual open items** (tracked in *Known scaling risks & open problems*, §12, not pretended
|
||||||
|
solved): the exact convergence bound for
|
||||||
|
high-write CRDT shards under partition, and whether per-equivalence-set divergence needs a
|
||||||
|
vector clock vs. a simple base-rev comparison, are deferred to implementation spikes.
|
||||||
|
|
||||||
|
### 8.7 Scaling the union — incremental-first, rebuild as fallback
|
||||||
|
|
||||||
|
The derived tier is *recomputable* (I-2) but recompute must never be the **operational**
|
||||||
|
mechanism. A from-scratch rebuild reads every page of every shard — including rate-limited,
|
||||||
|
paginated external APIs (Notion) and irreducibly-live sources — which can take hours to days
|
||||||
|
and directly fights the operational-envelope axis (review C-2). So:
|
||||||
|
|
||||||
|
**Incremental, change-driven maintenance is the primary mechanism.** Each shard's `notify`
|
||||||
|
capability (or a poll/ETag fallback where it has none, §8.8) emits **change events**; an event
|
||||||
|
drives a **delta update** to exactly the affected union nodes, equivalence candidates, indexes,
|
||||||
|
and projections. The derived tier is a continuously-maintained materialised view, not a
|
||||||
|
periodically-recomputed one. Steady-state cost is O(changes), not O(corpus).
|
||||||
|
|
||||||
|
**Full rebuild is a rare, bounded fallback** — for cold start, schema/algorithm change, or
|
||||||
|
suspected corruption — and it is **explicitly not required to be cheap**. It respects each
|
||||||
|
shard's envelope (it may be slow, throttled, or resumable for a rate-limited shard) and runs
|
||||||
|
*concurrently with serving the existing derived tier*; it swaps in atomically on completion.
|
||||||
|
I-2 guarantees rebuild is *possible and correct*, not instant.
|
||||||
|
|
||||||
|
**Equivalence detection is indexed, not pairwise (review C-1).** Naive fingerprint/span-set
|
||||||
|
comparison across all pages of all shards is O(N²) and is forbidden. Instead:
|
||||||
|
|
||||||
|
1. **Blocking / candidate generation** — cheap keys bucket pages that *could* be equivalent:
|
||||||
|
normalised title, normalised path tail, explicit alias-table entries (coordination-
|
||||||
|
canonical), and **MinHash/LSH bands over content shingles** for near-duplicate and
|
||||||
|
derived-content detection. Only within-bucket pairs are considered — turning O(N²) into
|
||||||
|
≈O(N) candidates.
|
||||||
|
2. **Verification** — candidate pairs are confirmed by full fingerprint / span-set overlap and
|
||||||
|
any curator binding. Confirmed equivalences become union edges.
|
||||||
|
3. **Incremental maintenance — the delta is *not* additive (review B-4).** A changed page may
|
||||||
|
*leave* buckets as well as *enter* them, and leaving a bucket can **break an existing
|
||||||
|
equivalence edge** another page relied on. So a change is processed as: (i) recompute the
|
||||||
|
page's bucket membership; (ii) for buckets it **left**, re-verify the pairs that depended on
|
||||||
|
the shared bucket and **retract** edges no longer supported; (iii) for buckets it **entered**,
|
||||||
|
verify the new candidate pairs and **add** edges; (iv) **propagate** to the equivalence
|
||||||
|
neighbours of any retracted/added edge (equivalence is transitive-ish via chorus sets, so a
|
||||||
|
retraction can split a set). Maintenance is per-change and bounded by the page's
|
||||||
|
neighbourhood, but it covers retraction and propagation — not just additions.
|
||||||
|
|
||||||
|
**The index is itself derived** (disposable, recomputable) and per-tenant-partitioned (§9).
|
||||||
|
Its parameters (LSH band/row counts, shingle size, precision/recall) are tunable; the accepted
|
||||||
|
**false-negative rate of blocking** is a known, tracked limitation (§12) — blocking trades a
|
||||||
|
small miss rate for tractability, and curator bindings are the escape hatch for misses.
|
||||||
|
|
||||||
|
**Verifying I-2 (`derived = f(canonical)`) — eventually, not on faith (review B-4).**
|
||||||
|
Incremental maintenance can drift from a from-scratch fold over time (a missed retraction, a
|
||||||
|
dropped event, a bug). I-2 is therefore an **eventually-verified** property, not a free one,
|
||||||
|
and the architecture names the mechanism that verifies it:
|
||||||
|
|
||||||
|
- **A digest of the derived tier.** Each partition's derived tier carries a rolling content
|
||||||
|
digest (a Merkle-style hash over union nodes/edges/index entries) maintained alongside the
|
||||||
|
incremental updates.
|
||||||
|
- **A background consistency-checker** periodically recomputes the digest over a *sampled* (or,
|
||||||
|
on a slow cadence, full) fold of canonical state and compares. A mismatch localises the drift
|
||||||
|
to a partition/region and triggers a **scoped recompute** of just that region — cheap relative
|
||||||
|
to a global rebuild, and self-healing.
|
||||||
|
- **So I-2 holds *eventually and verifiably*:** the incremental engine is the fast path, the
|
||||||
|
checker is the guarantee, and divergence is detected and repaired rather than silently
|
||||||
|
accumulating. The exact sampling rate / digest granularity is an implementation spike (§12).
|
||||||
|
|
||||||
|
### 8.8 Cache freshness & invalidation
|
||||||
|
|
||||||
|
Replication-projection caches remote shard content; cache invalidation is the actual hard part
|
||||||
|
and was missing from the first draft (review C-2). The protocol is **per-binding, driven by the
|
||||||
|
capability profile**, with one rule: **freshness is always represented, never assumed** — every
|
||||||
|
cached page's provenance envelope carries `(observed-at, source-rev-if-known, staleness-state)`,
|
||||||
|
so a consumer can always tell live from stale.
|
||||||
|
|
||||||
|
**Three invalidation modes, chosen by capability, not hard-coded:**
|
||||||
|
|
||||||
|
| Mode | When | Mechanism |
|
||||||
|
|------|------|-----------|
|
||||||
|
| **Event-driven (push)** | shard has `notify` | a change event invalidates exactly the affected entries and enqueues a delta refresh (§8.7); the preferred mode |
|
||||||
|
| **Validator poll** | shard exposes ETag / Last-Modified / rev | conditional fetch (`If-None-Match`); cheap "still fresh?" checks without transferring bodies |
|
||||||
|
| **TTL** | shard offers neither | time-bounded staleness; the floor mode (Oddmuse-class shards) |
|
||||||
|
|
||||||
|
Most real bindings are **hybrid**: event-driven for invalidation + a long TTL as a safety net
|
||||||
|
for missed events + validator polls on read when an entry is past a soft age.
|
||||||
|
|
||||||
|
**Operational-envelope coupling.** The mode is constrained by axis-10: a **rate-limited** shard
|
||||||
|
(Notion) *must* favour event-driven + long TTL and *must not* poll per-read — the freshness
|
||||||
|
policy is capability-gated like everything else. A local file shard can watch the filesystem
|
||||||
|
(near-instant invalidation, effectively event-driven for free).
|
||||||
|
|
||||||
|
**Thundering-herd / coalescing.** Concurrent reads of the same stale entry trigger a **single
|
||||||
|
in-flight refresh** (single-flight); other readers await it or are served the stale-but-labelled
|
||||||
|
value per policy. Bulk invalidations (a shard-wide event) are **batched and rate-shaped** to the
|
||||||
|
shard's envelope rather than fired as N concurrent fetches.
|
||||||
|
|
||||||
|
**Staleness is a policy knob, not a correctness bug.** Whether a reader gets *stale-but-fast* or
|
||||||
|
*blocks-for-fresh* is a §10 preset (per space or per request); either way the envelope tells the
|
||||||
|
truth about what was served. This is union-without-erasure applied to time.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Cross-cut — Authorization (L5)
|
||||||
|
|
||||||
|
Fully specified in **`ArchitectureBlueprint.md`** (the access & history sub-blueprint);
|
||||||
|
summarised here for completeness:
|
||||||
|
|
||||||
|
- **One core, a ladder of modes** L0 (open/c2, zero deps) → L1 (attributed) → L2
|
||||||
|
(authenticated) → L3 (role/group) → L4 (multi-tenant enterprise). Climbing is configuration,
|
||||||
|
not re-architecture.
|
||||||
|
- **PEP** wraps every adapter op; **PDP** decides `(principal, action, target)` over actions
|
||||||
|
`read/write/patch/merge/administer`, layered on the adapter's capability profile (a shard
|
||||||
|
that can't write can't be written regardless of policy — L5 consults the L1 rail).
|
||||||
|
- **Authentication delegated** to a pluggable IdentityProvider (null provider = L0 default);
|
||||||
|
real identity from `user-engine` over `net-kingdom` IAM.
|
||||||
|
- **Fail open only at L0, fail closed at L2+.** Authorization is pure/offline once a Principal
|
||||||
|
is resolved. Provenance carries authz context so the union never leaks unreadable content
|
||||||
|
(the L5↔provenance-rail interaction).
|
||||||
|
|
||||||
|
### 9.1 Tenant isolation of the derived tier (review B-3)
|
||||||
|
|
||||||
|
Read-time authz filtering is necessary but **not sufficient** when the derived tier is
|
||||||
|
*persisted*: a single cross-tenant union/index cache guarded only by a filter on read is a
|
||||||
|
standing leak surface (one filtering bug exposes another tenant's content). So isolation is
|
||||||
|
**structural, not just procedural**:
|
||||||
|
|
||||||
|
- **The derived tier is partitioned per tenant / root entity.** A tenant maps to a root entity
|
||||||
|
(§4); its union graph, equivalence index, projections, and caches live in a **separate
|
||||||
|
partition** keyed by that tenant. There is no shared cross-tenant derived store to leak from.
|
||||||
|
- **No cross-tenant equivalence by default.** Blocking/LSH (§8.7) operates *within* a partition;
|
||||||
|
cross-tenant equivalence is an explicit, authorised, opt-in federation between roots, never an
|
||||||
|
accident of a shared index.
|
||||||
|
- **Read-time filtering remains, as defence-in-depth** — the provenance envelope's authz context
|
||||||
|
is still checked, so even within a partition a principal sees only what it may; partitioning
|
||||||
|
removes the *blast radius*, filtering removes the *fine-grained* leak.
|
||||||
|
- **This reconciles I-2 with L5:** recomputability (a persisted-but-disposable derived tier) is
|
||||||
|
preserved *per partition* — each tenant's derived tier is independently rebuildable from that
|
||||||
|
tenant's canonical state — so isolation costs nothing in the rebuild model. At L0/L1 (single
|
||||||
|
tenant) there is one partition and the machinery is invisible.
|
||||||
|
|
||||||
|
**Isolation invariant (add to §2 as I-13):** *derived state is partitioned by tenant; no
|
||||||
|
derived artifact spans tenants except through an explicit, authorised cross-root federation.*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. The policy surface (mechanism over policy, made concrete)
|
||||||
|
|
||||||
|
I-7 only means something if the policy knobs are enumerated and kept *out* of core algorithms.
|
||||||
|
The configurable presets are:
|
||||||
|
|
||||||
|
- **Canonical-source policy** — chorus / designated-canonical / git-merge / overlay-only /
|
||||||
|
vote-to-merge (per space or per equivalence set).
|
||||||
|
- **Federation model** — the §8.3 taxonomy, per space, composable per shard.
|
||||||
|
- **Shard mode** — read-only / write-through / mirrored / projected / cached / canonical
|
||||||
|
(constrained by the capability profile).
|
||||||
|
- **Reconciliation cadence & conflict exposure** — push/poll/manual; show-conflicts vs
|
||||||
|
auto-merge-when-supported.
|
||||||
|
- **Conflict-resolution preset** — chorus / designated-canonical / git-merge / vote-to-merge /
|
||||||
|
overlay-only (the *resolution* policy over §8.6's core detection; per space or equivalence set).
|
||||||
|
- **Freshness / invalidation mode** — event-driven / validator-poll / TTL / hybrid, and
|
||||||
|
stale-but-fast vs block-for-fresh on read (§8.8; constrained by the operational envelope).
|
||||||
|
- **History compaction** — squash policy for low-value churn, gc/repack cadence, per-shard
|
||||||
|
offload (§8.1), always preserving recoverable endpoints.
|
||||||
|
- **Tenant partition mapping** — tenant ↔ root-entity, and any explicit cross-root federation
|
||||||
|
(§9.1, I-13).
|
||||||
|
- **Execution policy** — derive/execute off (default) / sandboxed / per-shard-allowed.
|
||||||
|
- **Authorization mode** — the L0–L4 ladder.
|
||||||
|
- **Projection materialization** — lazy/eager; snapshot vs view-time; recording retention.
|
||||||
|
|
||||||
|
Core ships sane defaults (L0 open; fork+journal; lazy replication-projection; event-driven+TTL
|
||||||
|
freshness; overlay-before-mutation; execution off; one tenant = one root) and never hard-codes
|
||||||
|
any of the above. (**Preset bundles** that package coherent knob-sets per persona are tracked
|
||||||
|
as O-8, §12 — flexibility without bundles is operator burden.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Concrete module structure (bridge to implementation)
|
||||||
|
|
||||||
|
A proposed package layout for `src/shard_wiki/`, mapping 1:1 to the layers so the dependency
|
||||||
|
rule (downward only; the derived tier is incrementally maintained, rebuild = fallback) is
|
||||||
|
enforceable by import lint:
|
||||||
|
|
||||||
|
```
|
||||||
|
src/shard_wiki/
|
||||||
|
model/ # L2 top waist: Page, Identity, Placement, ProvenanceEnvelope,
|
||||||
|
# Span, the page-shape types; capability-spectrum value types
|
||||||
|
adapters/ # L1 bottom waist: AdapterContract (versioned iface), CapabilityProfile,
|
||||||
|
# attachment-mode binding; concrete adapters:
|
||||||
|
git/ folder/ gitea/ obsidian/ webdav/ notion/ … # each: profile + verbs
|
||||||
|
coordination/ # L3: DecisionLog (append-only, git-backed, per-space append authority/
|
||||||
|
# lease), OverlayEngine (draft→patch→MR), reconcile
|
||||||
|
# (current coordination state = a derived fold → lives in union/)
|
||||||
|
federation/ # L3: FederationModel strategies (fork_journal, vcs_ping,
|
||||||
|
# graph_join, feed, activitypub, engine_mirror)
|
||||||
|
union/ # L4 (derived): IdentityResolver, EquivalenceGraph, UnionGraph,
|
||||||
|
# Transclusion (reference-not-copy)
|
||||||
|
projection/ # L4 (derived): ReplicationProjection, DerivationProjection,
|
||||||
|
# ViewRegistry (moldable), QueryIndex (delegate|derive)
|
||||||
|
authz/ # L5 cross-cut: PDP, PEP, IdentityProvider iface, NullProvider
|
||||||
|
provenance/ # cross-cut LEAF: ProvenanceEnvelope type + ⊕ (effective) only — pure data
|
||||||
|
policy/ # cross-cut LEAF: the §10 policy surface (presets + a resolve() read by
|
||||||
|
# coordination/federation/projection/authz); owns NO mechanism
|
||||||
|
api/ # L6: orchestrator API (server-side union for agents/CLI)
|
||||||
|
```
|
||||||
|
|
||||||
|
**The cross-cutting rails are leaves, not god-modules (review D-4).** `provenance/` and
|
||||||
|
`policy/` are imported widely, so they are the highest coupling risk; the discipline that caps
|
||||||
|
it is: **they may import *nothing* in the tree and contain *only* stable data types + pure
|
||||||
|
functions** (the envelope and its `⊕`; the policy presets and a `resolve(question) → choice`).
|
||||||
|
Mechanism never lives in a rail — `policy/` says *what* the preset is, `coordination/`/
|
||||||
|
`projection/` decide *how* to honour it. A change to a rail is then a change to a small, stable,
|
||||||
|
dependency-free leaf, not a ripple through every layer. Capability-spectrum value types live in
|
||||||
|
`model/` (also leaf-like) for the same reason.
|
||||||
|
|
||||||
|
Hard import rules (enforced by import lint):
|
||||||
|
- `union/` and `projection/` may import `model/`, `adapters/`, `coordination/`, `policy/`,
|
||||||
|
`provenance/` — but **nothing may import them** (they are the disposable derived tier).
|
||||||
|
- `model/`, `adapters/`, `provenance/`, `policy/` import nothing else in the tree (the waists
|
||||||
|
and rails stay thin); `provenance/` and `policy/` import nothing at all.
|
||||||
|
- `coordination/` and `federation/` may import the waists + rails, never the derived tier.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Known scaling risks & open problems
|
||||||
|
|
||||||
|
Tracked honestly rather than pretend-solved (review disposition F). Each has a **chosen
|
||||||
|
direction** and a **revisit trigger** — the thing that, if observed, forces a redesign.
|
||||||
|
|
||||||
|
| # | Risk / open problem | Chosen direction | Revisit trigger |
|
||||||
|
|---|---------------------|------------------|-----------------|
|
||||||
|
| O-1 | **Equivalence blocking misses true matches** (LSH false negatives, §8.7) | accept a small miss rate; curator bindings are the escape hatch | measured recall below an agreed threshold on real corpora |
|
||||||
|
| O-2 | **Convergence bound for high-write CRDT shards under partition** (§8.6) | causal via journal + CRDT-native merge at the shard; no global bound promised | user-visible divergence that outlives a partition |
|
||||||
|
| O-3 | **Per-equivalence-set divergence tracking** (§8.6) | start with base-rev comparison; add vector clocks only if needed | 3-way concurrent divergence that base-rev mis-orders |
|
||||||
|
| O-4 | **Persisted derived-tier cost ceiling** (§8.7/§9.1) | per-tenant partition, incremental-maintained, rebuild is fallback | a tenant whose incremental cost still exceeds budget |
|
||||||
|
| O-5 | **Axis-interaction completeness** (§6.5) | the named interaction table is the contract; extend deliberately | a real adapter needing an interaction not in the table |
|
||||||
|
| O-6 | **Span-address portability across projection** (§7.2) | shard-scoped native-id wrapping now; tumbler later | cross-shard transclusion that native ids can't satisfy |
|
||||||
|
| O-7 | **Squash-compaction vs. perfect auditability** (§8.1) | compact the *path*, preserve reachable states; configurable | a compliance need for every intermediate keystroke |
|
||||||
|
| O-8 | **Policy-knob proliferation → operator burden** (§10) | ship named **preset bundles** ("personal vault" / "team wiki" / "enterprise federation") over the policy surface | operators mis-configuring interacting knobs |
|
||||||
|
| O-9 | **Shard sharing across roots vs tenant partition** (§9.1, I-13) | shard exclusive to one root by default; explicit shared-read binding otherwise (avoids double-caching a rate-limited shard) | a shard legitimately needed live in two tenants |
|
||||||
|
| O-10 | **Span-level authz under transclusion** (aggregation/inference leak; ⊕ across boundaries, §7.3/§9) | a transcluded span inherits the **stricter** of source & host authz; provenance ⊕ composes the source-page envelope under the host | a real cross-authz transclusion |
|
||||||
|
| O-11 | **Union under shard unavailability** (§8.8 covers stale, not down) | **partial union** + per-shard "unavailable" provenance + last-known-projection where policy allows | an SLA need on partial reads |
|
||||||
|
| O-12 | **Per-space append-log throughput ceiling** (§8.1 append authority) | single writer per space scales across spaces; per-space log sharding if needed | a single space exceeding one writer's append rate |
|
||||||
|
|
||||||
|
These are the spec-writing inputs for `SHARD-WP-0002`; none blocks the architecture, each
|
||||||
|
scopes an implementation spike.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 13. Canonical data flows (the architecture exercised)
|
||||||
|
|
||||||
|
**A. Attach a shard.** Adapter binds (chosen attachment mode) → probes/declares a capability
|
||||||
|
profile → core registers the shard under a root entity → if not git-native, the coordination
|
||||||
|
journal is seeded (begin-now/mirror/import per axis 5). No union rebuild yet (lazy).
|
||||||
|
|
||||||
|
**B. Read a page through the union.** Consumer asks the union for an identity → Identity
|
||||||
|
resolver maps it to placements across shards → equivalence yields chorus or canonical →
|
||||||
|
replication-projection lazily fetches from each shard (cache + freshness) → page returned
|
||||||
|
wrapped in its provenance envelope → L5 filters anything the principal can't see at source.
|
||||||
|
|
||||||
|
**C. Edit a read-only / limited shard.** Write request → L5 PDP allows → capability profile
|
||||||
|
says < write-through → OverlayEngine records a draft → renders a patch/MR in the shard's native
|
||||||
|
syntax (lossless) or Markdown (lossy-with-report) → on explicit apply, commit to the journal
|
||||||
|
and (if the profile permits) propagate; otherwise the overlay stands as the local truth, fully
|
||||||
|
attributed.
|
||||||
|
|
||||||
|
**D. Attach a computational notebook.** Adapter declares profile (attachment=file-store,
|
||||||
|
opacity=mixed, computational=captured-output). Core attaches the `.ipynb` **source** as
|
||||||
|
canonical; presents cells + embedded outputs as **derivation-projection snapshots** marked
|
||||||
|
"run N, env unguaranteed"; offers a static render via the view registry; re-execution stays
|
||||||
|
gated off. History uses paired-text/nbdime per axis 5.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 14. Key tradeoffs & decisions
|
||||||
|
|
||||||
|
Decided:
|
||||||
|
|
||||||
|
- **Capability spectra over a verb checklist** — richer contract for precise, uniform
|
||||||
|
degradation; tamed by an orthogonal core + implied positions + a named interaction table
|
||||||
|
(§6.5). (Decided.)
|
||||||
|
- **Three states; derived = f(canonical)** — sharded + coordination canonical, derived
|
||||||
|
disposable (§1). (Decided; supersedes the earlier "edges vs middle" framing.)
|
||||||
|
- **Event-sourced coordination, one append authority per space** — coordination-canonical state
|
||||||
|
is an append-only **decision log** in the git journal; current state is a derived fold; a
|
||||||
|
per-space append lease gives a totally-ordered log and read-your-writes across orchestrator
|
||||||
|
instances (§8.1). (Decided — resolves the single-vs-multi-writer keystone.)
|
||||||
|
- **Profiles are verified, not asserted** — a versioned **conformance suite** gates adapter
|
||||||
|
admission; capability-as-data acts on verified data (§6.6). (Decided.)
|
||||||
|
- **I-2 is eventually-verified** — incremental maintenance is the fast path; a digest +
|
||||||
|
background consistency-checker detects and self-heals drift (§8.7). (Decided.)
|
||||||
|
- **Incremental-first, rebuild-as-fallback** — the derived tier is continuously maintained from
|
||||||
|
change events; full rebuild is rare and need not be cheap (§8.7). (Decided — resolves the
|
||||||
|
earlier "union graph persistence" open item: **persisted, per-tenant, incrementally
|
||||||
|
maintained, rebuildable**, §9.1.)
|
||||||
|
- **Identity ≠ fingerprint** — page identity is a stable handle; fingerprints are for
|
||||||
|
equivalence (§7.2). (Decided.)
|
||||||
|
- **Consistency = read-your-writes (journal) + causal (derived) + eventual/freshness-labelled
|
||||||
|
(shards)**; conflict detection/representation is core, resolution is policy (§8.6). (Decided.)
|
||||||
|
- **Address scheme** — shard-scoped native-id wrapping now; portable tumbler later (§7.2, O-6).
|
||||||
|
(Decided.)
|
||||||
|
- **Default federation = fork+journal over Git**; other models opt-in (§8.3). (Decided.)
|
||||||
|
- **Execution off by default** — recognise+project always; execute only when gated (§8.5). (Decided.)
|
||||||
|
- **Derived tier is tenant-partitioned** (I-13, §9.1). (Decided.)
|
||||||
|
|
||||||
|
Still open (carried to §12 / policy):
|
||||||
|
|
||||||
|
1. **L1 "attributed-but-open" mode** — ship it or jump L0→L2? (Carried from ArchitectureBlueprint.)
|
||||||
|
2. **Per-page ACL default** — off (per-shard/namespace) confirmed; revisit only if demand appears.
|
||||||
|
3. The implementation spikes in **§12** (O-1…O-7).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 15. What this architecture is *not*
|
||||||
|
|
||||||
|
- Not a wiki engine, UI, or rendering pipeline (those are consumers at L6).
|
||||||
|
- Not a canonical-source-of-truth — shards keep sovereignty; the middle is derived.
|
||||||
|
- Not a generic file-sync daemon — synchronisation is wiki-page-semantic.
|
||||||
|
- Not an execution platform — computation is recognised and projected, not hosted.
|
||||||
|
- Not a universal ontology — no single schema is imposed on all shards.
|
||||||
|
- Not an authentication/identity store — that is delegated (authorization is owned).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 16. Traceability
|
||||||
|
|
||||||
|
- **INTENT** — every invariant in §2 (I-1…I-13) cites an INTENT principle or boundary; no
|
||||||
|
invariant contradicts the Stability Note.
|
||||||
|
- **Review & hardening** — this revision folds in
|
||||||
|
`history/260615-core-architecture-blueprint-review.md` via **`SHARD-WP-0005`**: A-1→§1/§3/§4
|
||||||
|
(three-state re-frame), B-1→§7.2 (identity vs equivalence), B-2→§8.6 (consistency/conflict),
|
||||||
|
B-3→§9.1+I-13 (tenant isolation), C-1/C-2→§8.7/§8.8 (incremental + indexed + invalidation),
|
||||||
|
C-3→§8.1 (history scaling), D-1→§6.5 (orthogonal core), D-2→§7.3 (layered provenance),
|
||||||
|
D-3→§8.4 (common-case projection), D-4→§11 (policy module + rail discipline); open items→§12.
|
||||||
|
- **Round-2 review & hardening II** — folds in
|
||||||
|
`history/260615-core-architecture-blueprint-review-2.md` via **`SHARD-WP-0006`**:
|
||||||
|
A-1…A-4→§3/§4/§10/§11 (overview reconciled to the body), B-1+B-3→§8.1 (event-sourced
|
||||||
|
coordination + per-space append authority), B-2→§6.6 (adapter conformance suite),
|
||||||
|
B-4→§8.7 (incremental retraction/propagation + I-2 digest/checker); C-1…C-4→§12 (O-8…O-11).
|
||||||
|
- **Research** — §6 (spectra) ← `260614-shard-spectrum-synthesis` v3; §8.3 (federation
|
||||||
|
taxonomy) ← v3 §2.5; §8.4–§8.5 (two-axis projection, view registry, computational scope) ←
|
||||||
|
`260614-computational-page-model-synthesis`; §7 page shapes ← the engine + modern-tool +
|
||||||
|
computational dives; §1 thesis ← the files-canonical/index-derived through-line across
|
||||||
|
Logseq/ikiwiki/GT/Pharo/Jupyter.
|
||||||
|
- **Use cases** — the architecture is sized to UC-01–UC-84: federation/coordination (UC-01–07,
|
||||||
|
26–33, 56, 71–72, 79) → §8; attachment/adapter (UC-34–43, 50, 53, 57, 60–62, 64–66, 68–70,
|
||||||
|
76–82) → §6; page model & fidelity (UC-34, 39, 42, 55, 58–59, 67, 73, 80, 83–84) → §7/§8.5;
|
||||||
|
addressing/identity/query (UC-32, 44–49, 51–52, 54, 63, 74) → §7.2/§8.4; provenance &
|
||||||
|
metadata (UC-24–25, 75) → the provenance rail; collaboration & discovery (UC-08–23) → L6
|
||||||
|
consumers over the union.
|
||||||
|
- **Workplans** — §6–§8 are the design target of `SHARD-WP-0002` (T11–T18); §9 is owned by
|
||||||
|
`ArchitectureBlueprint.md`; §1 (yawex-derived resolution/overlay) aligns with
|
||||||
|
`SHARD-WP-0001`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 17. Stability note
|
||||||
|
|
||||||
|
This document defines shard-wiki's **internal** architecture; it may evolve as the spec
|
||||||
|
workplans land. But the **thesis (§1)**, the **invariants (§2)**, and the **dual narrow waist
|
||||||
|
(§1, §6, §7)** are load-bearing — changing any of them is an architectural change in the sense
|
||||||
|
of INTENT's Stability Note and should be rare and deliberate.
|
||||||
220
spec/FederationArchitecture.md
Normal file
220
spec/FederationArchitecture.md
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
# FederationArchitecture
|
||||||
|
|
||||||
|
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0002** (T1–T10, T17)
|
||||||
|
|
||||||
|
The federation **design decisions** for shard-wiki: *what the union does*. It records, per
|
||||||
|
topic, a decision with rationale, tradeoffs, and a **Decided / Deferred / Open** footer. It
|
||||||
|
**references** `spec/CoreArchitectureBlueprint.md` (the whole-system architecture) and
|
||||||
|
`spec/FederationRequirements.md` (yawex-derived ADRs) rather than restating them; the adapter
|
||||||
|
contract (*what a backend must expose*) is the companion deliverable in
|
||||||
|
`spec/TechnicalSpecificationDocument.md` §A (T11–T16, T18). UC references → `UseCaseCatalog.md`.
|
||||||
|
|
||||||
|
Cross-cutting invariants assumed throughout (blueprint §2): orchestrator-not-engine (I-1),
|
||||||
|
three-state canonical/derived (I-2), capability-as-data (I-3), union-without-erasure (I-4),
|
||||||
|
overlay-before-mutation (I-5), git-addressable coordination (I-6), mechanism-over-policy (I-7),
|
||||||
|
graceful degradation (I-8), identity≠placement (I-9), history-as-floor (I-10), authz-in-core
|
||||||
|
(I-11), tenant-partitioned derived state (I-13).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## T1 — Orchestrator positioning & boundaries
|
||||||
|
|
||||||
|
**Decision.** shard-wiki is an **adapter-layer orchestrator** in a **star** shape (many
|
||||||
|
sovereign shards ↔ one information space), **not** a homogeneous federation network. It
|
||||||
|
**compares, does not equate** (Federated Wiki homogeneous JSON sites, ikiwiki homogeneous git
|
||||||
|
wikis, ActivityPub activity streams are *models it can speak*, §T17 — not shapes it imposes).
|
||||||
|
Core owns: the union, the coordination journal, projection, overlay, resolution, authorization.
|
||||||
|
Adapters own backend specifics; UI and editorial policy live outside core. (Blueprint §1, §5;
|
||||||
|
UC-02, UC-26.)
|
||||||
|
|
||||||
|
*Decided:* star orchestrator; compare-not-equate; core/adapter/UI/policy boundaries.
|
||||||
|
*Deferred:* the reference UI (L6) is out of scope here. *Open:* none.
|
||||||
|
|
||||||
|
## T2 — Remix primitives: reference / projection / overlay / import-fork
|
||||||
|
|
||||||
|
**Decision.** Four primitives, escalating in commitment; **overlay-before-mutation is the
|
||||||
|
default write path** (I-5), fork is *one federation model* (§T17) not the default for editing:
|
||||||
|
|
||||||
|
| Primitive | Trigger | Writes remote? | Coordination-canonical? |
|
||||||
|
|-----------|---------|----------------|--------------------------|
|
||||||
|
| **Reference** | link only | no | no (a link in content) |
|
||||||
|
| **Projection** | read remote page | no (cache) | no (derived) |
|
||||||
|
| **Overlay** | edit a sub-write-through shard | not until explicit apply | **yes** (decision log) |
|
||||||
|
| **Import / fork** | copy into a writable shard / fork-with-provenance | source unchanged | **yes** (fork event) |
|
||||||
|
|
||||||
|
(Blueprint §8.2; FederationRequirements ADR-05; UC-04, UC-26, UC-29.)
|
||||||
|
|
||||||
|
*Decided:* the four primitives + overlay-default. *Deferred:* fork-attribution portability
|
||||||
|
format. *Open:* whether "import" and "fork" are one primitive with a flag or two (impl spike).
|
||||||
|
|
||||||
|
## T3 — Equivalent-page identity & multi-version presentation
|
||||||
|
|
||||||
|
**Decision.** Separate **identity** (stable handle), **placement** (N paths/shards), and
|
||||||
|
**equivalence** (content-fingerprint / span-set overlap *across* identities), per
|
||||||
|
FederationRequirements ADR-01/ADR-02 (I-9). Equivalence signals: normalized title/path, alias
|
||||||
|
table (coordination-canonical), link-graph overlap, fingerprint, **curator binding**. Default
|
||||||
|
presentation = **chorus-of-voices** (all equivalent versions, attributed); **designated-
|
||||||
|
canonical** is a policy preset (§T9). Divergence is data in the provenance envelope (I-4),
|
||||||
|
detected as core mechanism (blueprint §8.6). (UC-07, UC-27; UC-46.)
|
||||||
|
|
||||||
|
*Decided:* identity/placement/equivalence split; chorus default. *Deferred:* fingerprint
|
||||||
|
algorithm + blocking params (blueprint O-1). *Open:* curator-binding UX (L6).
|
||||||
|
|
||||||
|
## T4 — History, attribution & the coordination journal
|
||||||
|
|
||||||
|
**Decision.** Each information space has a **git-addressable, event-sourced coordination
|
||||||
|
journal** (blueprint §8.1, I-6/I-10): coordination-canonical decisions (overlay/binding/alias/
|
||||||
|
merge/fork) are append-only events; current state is a derived fold. **One append authority per
|
||||||
|
space** gives a total order (read-your-writes across instances). Per-shard native history is
|
||||||
|
**adopted** (git-native), **supplemented** (DB/CRDT internal → journal begins-now/mirrors), or
|
||||||
|
**imported** (open file history backfilled) per the history-portability axis (TSD §A T13).
|
||||||
|
Attribution is portable on the event (UC-29). (UC-29, UC-33.)
|
||||||
|
|
||||||
|
*Decided:* event-sourced journal + per-space append authority + adopt/supplement/import.
|
||||||
|
*Deferred:* per-space log sharding for extreme write rates (blueprint O-12). *Open:* none.
|
||||||
|
|
||||||
|
## T5 — Union composition layer
|
||||||
|
|
||||||
|
**Decision.** The **server-side orchestrator union is primary** — the derived tier (union
|
||||||
|
graph, indexes, projections) is composed in core for agents/CLI/non-browser consumers
|
||||||
|
(blueprint §8.4), **incrementally maintained** (not recomputed, §8.7), **tenant-partitioned**
|
||||||
|
(I-13). **Client-side composition** (Federated-Wiki-style browser pull) is a supported
|
||||||
|
*consumer pattern over the same union*, not the only path. Freshness/caching per §8.8. (UC-05,
|
||||||
|
UC-27, UC-03, UC-31.)
|
||||||
|
|
||||||
|
*Decided:* server-primary, client-optional; incremental, partitioned. *Deferred:* client
|
||||||
|
SDK shape. *Open:* persisted-union digest granularity (blueprint O-4, B-4 checker).
|
||||||
|
|
||||||
|
## T6 — Change notification & subscription transports
|
||||||
|
|
||||||
|
**Decision.** Change-notification is an **optional adapter capability** (`notify`), and it is
|
||||||
|
the **primary driver of incremental maintenance** (blueprint §8.7/§8.8). Transports, per
|
||||||
|
profile: git hook / ikiwiki-style ping, ActivityPub Create/Update, webhook, WebDAV/HTTP ETag
|
||||||
|
or `Last-Modified` poll, plain polling fallback. A notification → invalidate affected entries
|
||||||
|
+ enqueue delta refresh + a RecentChanges union entry (FederationRequirements ADR-03; UC-31,
|
||||||
|
UC-17). Rate-limited shards favour event-driven + long TTL (operational-envelope axis).
|
||||||
|
|
||||||
|
*Decided:* notify = optional capability, drives incremental + RecentChanges; transport per
|
||||||
|
profile. *Deferred:* ActivityPub as content-bearing (vs notify-only) — start notify-only.
|
||||||
|
*Open:* fediverse-dependency posture for v1.
|
||||||
|
|
||||||
|
## T7 — Information-space lifecycle
|
||||||
|
|
||||||
|
**Decision.** Root entities have lifecycle states: **active → read-only-archived → retired
|
||||||
|
(detached)**, and **merged-into-successor**. **Carry-forward** is selective import from an
|
||||||
|
archived shard (UC-28, UC-30). **Space-fork / branch** (UC-33) = branch the coordination
|
||||||
|
journal + shard-config (a fork event, §T2/§T17 fork+journal model). Retirement **preserves
|
||||||
|
read-only union entries** by default (history-as-floor, I-10), never hard-deletes projections.
|
||||||
|
(UC-28, UC-30, UC-33.)
|
||||||
|
|
||||||
|
*Decided:* the lifecycle states + carry-forward + space-branch; retire-preserves. *Deferred:*
|
||||||
|
ephemeral "happenings" as a first-class mode. *Open:* GC policy for retired-space derived tier.
|
||||||
|
|
||||||
|
## T8 — Transclusion & projection depth
|
||||||
|
|
||||||
|
**Decision.** Transclusion is **one reference-not-copy primitive** over the addressable union
|
||||||
|
(blueprint §8.4), at three depths:
|
||||||
|
|
||||||
|
| Level | UC | Behaviour |
|
||||||
|
|-------|-----|-----------|
|
||||||
|
| Whole-page projection | UC-03 | lazy full page from a remote shard (replication-projection) |
|
||||||
|
| Block/span transclusion | UC-32 | inline embed by span address, with origin + freshness |
|
||||||
|
| Link reference | UC-08 | pointer only (ADR-06) |
|
||||||
|
|
||||||
|
Every transcluded artifact carries provenance + **staleness** (I-4, §8.8); live-transclusion
|
||||||
|
fragility (link rot / remote down) degrades to last-known + an `unavailable`/`stale` marker
|
||||||
|
(blueprint O-11). Span-level authz under transclusion = the stricter of source/host (O-10).
|
||||||
|
Reference-not-copy unifies Xanadu transclusion, ZigZag clone, embeds, synced blocks, Trilium
|
||||||
|
note-clone, literate named-chunk. (UC-03, UC-32.)
|
||||||
|
|
||||||
|
*Decided:* one primitive, three depths, provenance+staleness mandatory. *Deferred:* snapshot-
|
||||||
|
on-import vs live default (a policy). *Open:* span-authz composition (O-10), addressing (O-6).
|
||||||
|
|
||||||
|
## T9 — Consensus & reconciliation policy catalog
|
||||||
|
|
||||||
|
**Decision.** **Detection is core, resolution is policy** (I-7, blueprint §8.6). Core always
|
||||||
|
detects divergence and represents it as a coexisting (chorus) set; the **resolution preset** is
|
||||||
|
configurable per space / per equivalence set:
|
||||||
|
|
||||||
|
- **chorus-spread** (versions coexist; default) · **designated-canonical** (explicit write
|
||||||
|
target) · **git-merge** (where merge capability allows) · **vote-to-merge / editorial gate**
|
||||||
|
(optional, L6-assisted) · **overlay-only** (no destructive merge on read-only sources).
|
||||||
|
|
||||||
|
(FederationRequirements ADR-03/ADR-05; UC-07, UC-27.)
|
||||||
|
|
||||||
|
*Decided:* the preset catalog; detection-core/resolution-policy. *Deferred:* vote/editorial
|
||||||
|
gate mechanics (L6). *Open:* default preset per persona bundle (blueprint O-8).
|
||||||
|
|
||||||
|
## T10 — Federation-operations capability matrix
|
||||||
|
|
||||||
|
**Decision.** Each federation operation requires a minimum **verified capability profile**
|
||||||
|
(TSD §A T11/§6.6); below it, the op **degrades** along the named interaction axes (blueprint
|
||||||
|
§6.5) rather than failing. The matrix (consuming the T11 vocabulary):
|
||||||
|
|
||||||
|
| Federation op | Min capabilities | Degradation when absent |
|
||||||
|
|---------------|------------------|--------------------------|
|
||||||
|
| **read / project** | `read` | (floor — every shard) |
|
||||||
|
| **transclude span** | `read` + addressing≥span | whole-page projection only |
|
||||||
|
| **overlay** | `read` | always available (overlay is local) |
|
||||||
|
| **write-through** | `write` (+ `merge` for concurrent) | → overlay/patch target |
|
||||||
|
| **diff / 3-way merge** | `diff`,`merge` | → keep-both / chorus |
|
||||||
|
| **notify-driven freshness** | `notify` | → validator-poll → TTL (§8.8) |
|
||||||
|
| **native search delegate** | native-query≥index | → derived index over projection |
|
||||||
|
| **publish / mirror** | `publish` | → read-only / static projection |
|
||||||
|
| **fork+journal federation** | `read` + history (any) | → projection-only participant |
|
||||||
|
|
||||||
|
Every backend, down to the Oddmuse flat-file floor, is usable as **read-only / cache /
|
||||||
|
projection / backup / patch target** (I-8). This matrix and the T11 spectra are kept in sync
|
||||||
|
(it consumes that vocabulary). (UC-02–UC-07.)
|
||||||
|
|
||||||
|
*Decided:* the op→capability matrix + degradation paths. *Deferred:* exhaustive per-op test
|
||||||
|
vectors → the conformance suite (§6.6). *Open:* axis-interaction completeness (blueprint O-5).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## T17 — Federation-model taxonomy (selectable / composable)
|
||||||
|
|
||||||
|
**Decision.** Federation is **plural and composable**, not one mechanism (blueprint §8.3). A
|
||||||
|
space selects a model (composing per shard); the default is **fork+journal over Git** (the home
|
||||||
|
case):
|
||||||
|
|
||||||
|
| Model | Anchor | Coordination shape | Capability floor | UCs |
|
||||||
|
|-------|--------|--------------------|------------------|-----|
|
||||||
|
| **Fork + journal** *(default)* | Federated Wiki | copy-with-provenance + per-page action journal (story = replay) | `read` + journal | UC-26, UC-71, UC-72 |
|
||||||
|
| **VCS-replication + ping** | ikiwiki | git clone/pull/push + change-ping | git-IS-store | UC-31, UC-33, UC-79 |
|
||||||
|
| **Query-time graph-join** | Wikibase `SERVICE` | join remote graphs at query, no copy | native-query≥graph | UC-74 |
|
||||||
|
| **Feed aggregation** | RSS/Atom | inbound feed → pages | `read` | UC-03 |
|
||||||
|
| **Activity streams** | ActivityPub | Create/Update events (notify or content) | `notify` | UC-31 |
|
||||||
|
| **Engine-mirror** | Wiki.js DB↔Git | engine mirrors its store to git | `publish`/mirror | UC-68, UC-69 |
|
||||||
|
|
||||||
|
The coordination journal (T4) is the fork+journal model's home; **git IS the journal** in
|
||||||
|
VCS-replication and engine-mirror (forge wikis make git the store *and* journal, resolving the
|
||||||
|
engine-mirror write-race). Models compose: e.g. fork+journal locally, query-join for a
|
||||||
|
typed-graph shard, feed-aggregation for an external source. (Mechanism over policy, I-7.)
|
||||||
|
|
||||||
|
*Decided:* the six models, selectable per space + composable per shard, default fork+journal.
|
||||||
|
*Deferred:* activity-streams content-bearing mode. *Open:* multi-model interaction edge cases
|
||||||
|
(when two models touch one shard).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Consolidated decisions / deferred / open
|
||||||
|
|
||||||
|
- **Decided (architecture-settling):** star orchestrator (T1); overlay-default remix (T2);
|
||||||
|
identity/placement/equivalence + chorus (T3); event-sourced journal + append authority (T4);
|
||||||
|
server-primary incremental union (T5); notify-driven freshness (T6); space lifecycle (T7);
|
||||||
|
reference-not-copy transclusion (T8); detection-core/resolution-policy preset catalog (T9);
|
||||||
|
op→capability matrix (T10); selectable/composable federation models (T17).
|
||||||
|
- **Deferred (to impl / L6 / persona bundles):** reference UI; fork-attribution format; client
|
||||||
|
SDK; vote/editorial mechanics; ephemeral spaces; ActivityPub content-bearing; default-preset
|
||||||
|
bundles (O-8).
|
||||||
|
- **Open (tracked in blueprint §12):** O-1 equivalence blocking, O-4 union digest, O-5 axis
|
||||||
|
interactions, O-6 addressing, O-8 presets, O-10 span-authz, O-11 unavailability, O-12 log
|
||||||
|
throughput.
|
||||||
|
|
||||||
|
## Acceptance (SHARD-WP-0002 federation half)
|
||||||
|
|
||||||
|
T1–T10 + T17 each have a decision record with a Decided/Deferred/Open footer; all honour INTENT;
|
||||||
|
UC-26–UC-33, UC-56, UC-71–UC-72, UC-74, UC-79 trace here; the adapter-contract companion is
|
||||||
|
`spec/TechnicalSpecificationDocument.md` §A (T11–T16, T18). Conflicts with SHARD-WP-0001 are
|
||||||
|
none (FederationRequirements ADRs are consumed, not duplicated).
|
||||||
214
spec/FederationRequirements.md
Normal file
214
spec/FederationRequirements.md
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
# FederationRequirements — yawex-derived design notes
|
||||||
|
|
||||||
|
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0001**
|
||||||
|
|
||||||
|
Concrete, ADR-ready design decisions for the union/federation layer, distilled from the yawex
|
||||||
|
prior-art review (`research/260608-yawex-prior-art/findings.md`) and made **consistent with**
|
||||||
|
`spec/CoreArchitectureBlueprint.md` (the whole-system architecture) and `INTENT.md`. yawex is
|
||||||
|
*inspiration and a case checklist*, never an interface to inherit (decision 2026-06-08).
|
||||||
|
|
||||||
|
These are **requirements** (what the union must do); `spec/FederationArchitecture.md`
|
||||||
|
(SHARD-WP-0002) is the architecture that realises them. UC references are to
|
||||||
|
`spec/UseCaseCatalog.md`.
|
||||||
|
|
||||||
|
**Resolved precondition.** The yawex "minimal access model in core" thread is **settled**:
|
||||||
|
authorization-in-core / authentication-delegated, the L0–L4 ladder — INTENT amended,
|
||||||
|
`ArchitectureBlueprint.md` ratified. Auth is therefore *not* re-litigated here; these ADRs
|
||||||
|
assume it (the resolver and overlay paths run behind the L5 PEP/PDP).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-01 — Cross-shard page resolution
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §8.4 (union), §7.2 (identity)
|
||||||
|
|
||||||
|
**Context.** yawex's most-tested component was `PageLookUp`, a context-relative resolver with
|
||||||
|
ordered states `LOCAL, CLIMB, DROP, GLOBAL, REMOTE, SWITCH, JUMP, VIRTUAL, FAILED`. Decision
|
||||||
|
(2026-06-08): **inspiration only** — use the states as a *checklist of cases*, design the
|
||||||
|
federation resolver fresh.
|
||||||
|
|
||||||
|
**Decision.** Resolution is a pure function over the derived union:
|
||||||
|
`resolve(name, from_context, policy) → ResolutionResult`, evaluated in a defined order and
|
||||||
|
keyed on **page identity** (the stable handle), not path (I-9):
|
||||||
|
|
||||||
|
1. **Same-namespace, same-shard** exact identity match — *(yawex LOCAL)*.
|
||||||
|
2. **Namespace walk** — climb/descend ancestor namespaces of `from_context` — *(CLIMB/DROP)*.
|
||||||
|
3. **Union lookup** — match identity / alias-table entry across all attached shards — *(GLOBAL)*.
|
||||||
|
4. **Equivalence set** — if several shards hold an equivalent page, return the **chorus set**;
|
||||||
|
the *canonical-source* policy preset (chorus / designated-canonical) decides presentation —
|
||||||
|
never silently pick one (union without erasure, I-4). *(implicit in yawex; explicit here.)*
|
||||||
|
5. **Projection / virtual** — a page whose content lives in a remote shard (lazy replication,
|
||||||
|
*REMOTE*) or is computed/query-defined (derivation, *VIRTUAL*).
|
||||||
|
6. **Explicit address** — a shard- or space-qualified reference resolves directly — *(SWITCH/JUMP)*.
|
||||||
|
7. **Not found** → **red-link**: a resolvable *target* with no page yet, createable (UC-23) —
|
||||||
|
*(FAILED)*.
|
||||||
|
|
||||||
|
`ResolutionResult` carries the resolved identity (or red-link), the placement(s), the
|
||||||
|
provenance envelope (ADR-04), and — on ambiguity — the full chorus set. Resolution is a
|
||||||
|
**derived-tier read** (union graph + equivalence index, §8.4); it is incremental-maintained and
|
||||||
|
respects the L5 authz filter (a principal never resolves to content it can't see).
|
||||||
|
|
||||||
|
**Consequences.** Deterministic, policy-driven, no privileged shard; every yawex state has a
|
||||||
|
home without inheriting its structure. **Open:** cross-space `JUMP` addressing syntax ties to
|
||||||
|
the portable-address scheme (CoreArchitectureBlueprint O-6).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-02 — Namespace / path model & shard roles
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §7.2 (identity≠placement), §4 (shard mode)
|
||||||
|
|
||||||
|
**Context.** yawex modelled **topics as directories** (a topic's gateway page shares the dir
|
||||||
|
name) with relative (`../`) / absolute (`/`) paths and normalization, and page classes
|
||||||
|
`local / global / virtual`.
|
||||||
|
|
||||||
|
**Decision.**
|
||||||
|
- The union exposes a **namespace tree**; a **path is a placement coordinate, not an identity**
|
||||||
|
(I-9). One identity may have **N placements** (paths/shards) → a DAG, no single canonical path.
|
||||||
|
- **Path grammar:** relative (`../`, `./`) resolved against `from_context`; absolute (`/`)
|
||||||
|
against the space root; **normalization** (collapse `.`/`..`; case & separator handling) is a
|
||||||
|
per-space **policy** knob (§10) — defaulting to case-preserving, `/`-separated.
|
||||||
|
- **yawex page classes → shard roles / modes:** `local → canonical`, `global → cross-cutting`
|
||||||
|
(a shard whose pages augment/overlay across namespaces), `virtual → projected/computed`. These
|
||||||
|
are the INTENT shard modes (read-only · write-through · mirrored · projected · cached ·
|
||||||
|
canonical), selected by policy, constrained by the capability profile.
|
||||||
|
- Cross-shard name collisions are resolved by ADR-01's order + equivalence, not by a global
|
||||||
|
flat namespace.
|
||||||
|
|
||||||
|
**Consequences.** The namespace is a navigable view over placements (a dimension of the union),
|
||||||
|
not a storage layout. **Open:** per-space normalization policy defaults (case-sensitivity,
|
||||||
|
unicode) — a policy preset, bundled per persona (O-8).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-03 — Union-level derived views: core vs adapter
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §8.4 (derived query index), §8.7
|
||||||
|
(incremental) · **resolves findings open-Q #3**
|
||||||
|
|
||||||
|
**Context.** yawex shipped BackLinks, RecentChanges, AllPages, SiteMap, full-text Search.
|
||||||
|
|
||||||
|
**Decision.** Each is a **derived-tier** view (rebuildable, incrementally maintained); classify
|
||||||
|
by where it is computed:
|
||||||
|
|
||||||
|
| View | Tier | Rationale |
|
||||||
|
|------|------|-----------|
|
||||||
|
| **BackLinks** | **core** | the link-graph over the union — squarely shard-wiki's concern (UC-18); the strongest core view |
|
||||||
|
| **RecentChanges** | **core** | merge the coordination journal (§8.1) with shard change events (notify/poll, §8.8) across the union (UC-17) |
|
||||||
|
| **AllPages / SiteMap** | **core** | an enumeration/projection of the union graph; cheap |
|
||||||
|
| **Search (full-text)** | **hybrid** | **delegate** to a shard's native search where the native-query axis allows; **else** build a derived index over projections (the Logseq DataScript-over-files pattern, UC-19/UC-63) |
|
||||||
|
|
||||||
|
All are computed in the derived tier and carry provenance; **presentation is L6 (UI)**, never
|
||||||
|
hard-coded in core (mechanism over policy, I-7).
|
||||||
|
|
||||||
|
**Consequences.** BackLinks/RecentChanges/AllPages/SiteMap are core deliverables; Search is a
|
||||||
|
capability-gated delegate-or-derive. **Open:** cross-shard search ranking/relevance is a policy
|
||||||
|
concern, not core mechanism.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-04 — Provenance & freshness model (concrete fields)
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §7.3 (layered provenance), §8.8 (freshness)
|
||||||
|
|
||||||
|
**Context.** yawex `Page::info` exposed modtime (and TODO'd last-editor / hits / edits). INTENT
|
||||||
|
mandates explicit provenance & freshness (I-4).
|
||||||
|
|
||||||
|
**Decision.** Concretise the **provenance envelope** (layered: page-level + span-level deltas):
|
||||||
|
|
||||||
|
```
|
||||||
|
ProvenanceEnvelope (page-level):
|
||||||
|
source_shard # which shard this came from
|
||||||
|
source_rev? # shard-native revision id, if the shard exposes one
|
||||||
|
observed_at # when shard-wiki last read it
|
||||||
|
liveness # static | captured-snapshot | live-over-files | view-time | irreducibly-live
|
||||||
|
staleness_state # live | fresh | stale | unavailable
|
||||||
|
authz_context # the L5 context under which it was read (no-leak, §9)
|
||||||
|
overlay_state # none | draft | patch-pending | applied
|
||||||
|
divergence[] # equivalence-set peers that differ (the chorus, ADR-01 step 4)
|
||||||
|
derivation_lineage? # for derived/derivation-projection content (source → this view)
|
||||||
|
Span-level: only the fields that differ from the page envelope (effective = page ⊕ delta).
|
||||||
|
```
|
||||||
|
|
||||||
|
`freshness = (observed_at, source_rev?, staleness_state)`; `unavailable` is the dead-shard
|
||||||
|
state (feeds ADR-03 RecentChanges and the union-under-unavailability open item O-11). This
|
||||||
|
envelope is the data behind union-without-erasure and the input to conflict display.
|
||||||
|
|
||||||
|
**Consequences.** Provenance is queryable, layered (cheap per-span), and drives views/conflict
|
||||||
|
UI. **Open / DROP:** hits/edits *analytics* are an L6/analytics concern, out of core (dropped
|
||||||
|
from the yawex feature set).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-05 — Overlay / lightweight-patch model
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §8.2 (overlay engine), §8.1 (decision
|
||||||
|
log), §8.6 (apply-under-drift)
|
||||||
|
|
||||||
|
**Context.** yawex had an append + threaded-comment workflow (edit a page without a full
|
||||||
|
rewrite). INTENT mandates overlay-before-mutation (I-5).
|
||||||
|
|
||||||
|
**Decision.** The overlay lifecycle for any shard below write-through capability:
|
||||||
|
|
||||||
|
1. **Draft** — an edit becomes a draft recorded as a **coordination-canonical event** in the
|
||||||
|
space's decision log (§8.1); the unapplied overlay is the **local truth**, fully attributed
|
||||||
|
(`overlay_state`, ADR-04).
|
||||||
|
2. **Patch / MR** — rendered in the shard's **native syntax (lossless)** or **Markdown
|
||||||
|
(lossy-with-fidelity-report)** per the translation axis.
|
||||||
|
3. **Apply** — only on explicit intent, only where profile + policy permit, with
|
||||||
|
**apply-under-drift** semantics (§8.6: fast-forward / three-way / refuse-and-re-present).
|
||||||
|
4. **Append / comment** = a **constrained overlay subtype** (purely additive, low-conflict) —
|
||||||
|
the direct generalisation of yawex's workflow; safe even on the most limited shards.
|
||||||
|
|
||||||
|
This makes read-only, rate-limited, and lossy shards **editable** without silent remote
|
||||||
|
mutation (graceful degradation, I-8; no silent mutation).
|
||||||
|
|
||||||
|
**Consequences.** One overlay mechanism spans drafts, patches, MRs, and comments; storage is the
|
||||||
|
decision log (no separate store). **Open:** whether threaded-comment threading is a first-class
|
||||||
|
overlay subtype or a generic structured-append — defer to implementation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ADR-06 — Markdown link semantics (wikilink + red-link)
|
||||||
|
|
||||||
|
**Status:** accepted · **CoreArchitectureBlueprint:** §7 (page model), ADR-01 (resolution) ·
|
||||||
|
**resolves findings open-Q #2**
|
||||||
|
|
||||||
|
**Context.** yawex had CamelCase WikiLinks, `[[free links]]`, red-`?` links for nonexistent
|
||||||
|
pages, `::` labels. INTENT mandates Markdown-first; TRANSFORM the *semantics*, drop the bespoke
|
||||||
|
syntax.
|
||||||
|
|
||||||
|
**Decision.**
|
||||||
|
- Adopt a **CommonMark wikilink extension**: `[[Target]]` / `[[Target|label]]`, resolved via
|
||||||
|
ADR-01 (by identity/name across the union).
|
||||||
|
- **Red-link** = a wikilink whose target resolves to FAILED (ADR-01 step 7): a valid,
|
||||||
|
*createable* target with no page yet (the soft-create affordance, UC-23).
|
||||||
|
- **CamelCase auto-linking is OFF by default** (a legacy affordance); opt-in **per space** for
|
||||||
|
migrating CamelCase wikis (UseModWiki/c2 lineage, UC-25).
|
||||||
|
- Links are stored **as text** (git-diffable; structure/links federate iff in-text).
|
||||||
|
- **Boundary (resolves open-Q #2):** the link **model + resolution is core**; the **visual
|
||||||
|
rendering** of wikilinks/red-links is a **reference-UI (L6)** concern. Core decides *what a
|
||||||
|
link resolves to and whether it's a red-link*; the UI decides how it looks.
|
||||||
|
|
||||||
|
**Consequences.** Markdown-first, backend-neutral, resolution-unified with ADR-01. **Open:**
|
||||||
|
cross-shard wikilink target disambiguation syntax ties to the portable-address scheme (O-6).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Coverage & open questions
|
||||||
|
|
||||||
|
- **Findings open-Q #1** (per-page vs per-shard ACL) — answered by the settled access model:
|
||||||
|
per-shard / per-namespace default, per-page ACL opt-in at L4 (`ArchitectureBlueprint.md`).
|
||||||
|
- **Findings open-Q #2** (wikilink core vs UI) — ADR-06: model/resolution core, presentation UI.
|
||||||
|
- **Findings open-Q #3** (derived views core vs adapter) — ADR-03.
|
||||||
|
- Carried to CoreArchitectureBlueprint §12: cross-space addressing syntax (O-6), namespace
|
||||||
|
normalization presets (O-8), cross-shard search ranking (policy), union-under-unavailability
|
||||||
|
(O-11).
|
||||||
|
|
||||||
|
## Acceptance (SHARD-WP-0001)
|
||||||
|
|
||||||
|
Each task T1–T6 has an ADR (T1→ADR-01, T2→ADR-02, T3→ADR-03, T4→ADR-04, T5→ADR-05, T6→ADR-06);
|
||||||
|
all honour INTENT (mechanism over policy, union without erasure, overlay before mutation,
|
||||||
|
capability-aware adapters) and are consistent with `CoreArchitectureBlueprint.md`. The
|
||||||
|
access-model thread is ratified (not deferred). The next implementation workplan (domain model /
|
||||||
|
adapter contract) can proceed without unresolved yawex-derived design gaps.
|
||||||
@@ -6,10 +6,15 @@ Background on document types: InfoTechPrimers on coulomb.social.
|
|||||||
|
|
||||||
| File | Status | Role |
|
| File | Status | Role |
|
||||||
|------|--------|------|
|
|------|--------|------|
|
||||||
|
| `CoreArchitectureBlueprint.md` | draft for review | **Whole-system architecture** — layers, abstractions, load-bearing decisions (synthesised from all research) |
|
||||||
|
| `FederationArchitecture.md` | draft for review | federation design — *what the union does*: T1–T10 decision records + the federation-model taxonomy (SHARD-WP-0002) |
|
||||||
|
| `WikiEngineCoreArchitecture.md` | draft for review | the native **headless, API-first wiki engine** — small page-store kernel + typed-extension framework, as a canonical-mode shard backend (SHARD-WP-0013) |
|
||||||
|
| `adr/` | living | Architecture Decision Records (ADR-0001: engine activation via feature-control) |
|
||||||
|
| `FederationRequirements.md` | draft for review | yawex-derived union/federation design notes — resolution, namespace, derived views, provenance, overlay, links (ADR-01…06; SHARD-WP-0001) |
|
||||||
| `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver |
|
| `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver |
|
||||||
| `TechnicalSpecificationDocument.md` | draft scaffold | How the system is built |
|
| `TechnicalSpecificationDocument.md` | draft + §A | How the system is built; **§A = the normative shard adapter contract** (T11–T16, T18; SHARD-WP-0002) |
|
||||||
| `UseCaseCatalog.md` | draft | 25 use cases promoted from c2 + yawex research |
|
| `UseCaseCatalog.md` | draft | 84 use cases promoted from c2 + yawex + ~23-system research |
|
||||||
| `ArchitectureBlueprint.md` | draft | Access, history, and identity architecture |
|
| `ArchitectureBlueprint.md` | draft | Access, history, and identity sub-blueprint (the L0–L4 authorization ladder; referenced by CoreArchitectureBlueprint §9) |
|
||||||
|
|
||||||
Promote material from `research/` and reviewed items from `demand/` into spec
|
Promote material from `research/` and reviewed items from `demand/` into spec
|
||||||
before treating it as implementation authority.
|
before treating it as implementation authority.
|
||||||
@@ -33,7 +33,134 @@ per information space.
|
|||||||
|
|
||||||
## 4. Architecture References
|
## 4. Architecture References
|
||||||
|
|
||||||
- `spec/ArchitectureBlueprint.md` — access, history, identity delegation
|
- `spec/CoreArchitectureBlueprint.md` — whole-system architecture (layers, the dual narrow
|
||||||
|
waist, the 15 capability spectra, projection, consistency); the normative source for §A.
|
||||||
|
- `spec/FederationArchitecture.md` — federation design (*what the union does*); §A is its
|
||||||
|
companion (*what a backend must expose*).
|
||||||
|
- `spec/FederationRequirements.md` — yawex-derived ADRs (resolution, page model, overlay).
|
||||||
|
- `spec/ArchitectureBlueprint.md` — access, history, identity delegation (L5).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# A. Shard Adapter Contract (normative)
|
||||||
|
|
||||||
|
Deliverable of **SHARD-WP-0002** (T11–T16, T18): the versioned interface a backend implements
|
||||||
|
to participate as a shard, and the *normative* rules core relies on. It is the **bottom narrow
|
||||||
|
waist** (blueprint §6); the page model is the top waist (§7). This section is normative where it
|
||||||
|
says **MUST/SHOULD**; design rationale lives in the blueprint (cited, not restated).
|
||||||
|
|
||||||
|
## A.1 (T11) Capability model & versioned interface
|
||||||
|
|
||||||
|
- The contract is a **versioned interface** (`Foswiki::Store`/`Foswiki::Meta` is the proof a
|
||||||
|
swappable-backend-behind-a-stable-interface works). A binding declares the contract version
|
||||||
|
it implements; core checks compatibility at registration.
|
||||||
|
- **Operation verbs:** `read, write, diff, merge, lock, version, publish, notify,
|
||||||
|
transclude-source, translate-syntax, structured-payload, derive-projection, execute`. The
|
||||||
|
last two are **gated, OFF by default** (A.6/T18). Verb support is part of the profile and MUST
|
||||||
|
reconcile with the T10 federation-ops matrix.
|
||||||
|
- **Capability profile = a position on each capability spectrum**, not a boolean checklist. The
|
||||||
|
full set is the **fifteen spectra** (blueprint §6.1); operationally they reduce to a **small
|
||||||
|
orthogonal core** (substrate, write-granularity, content-opacity, operational-envelope,
|
||||||
|
access-grant, computational/liveness) with the rest **implied** via published rules that
|
||||||
|
**forbid impossible profiles** (blueprint §6.5). Degradation reads only the **named
|
||||||
|
axis-interaction subset** (blueprint §6.5 table) — that table *is* the degradation contract.
|
||||||
|
- A profile MUST express **absence** cleanly (the Oddmuse floor); core never guesses a missing
|
||||||
|
capability.
|
||||||
|
|
||||||
|
## A.2 (T11/§6.6) Conformance — profiles are verified, not asserted
|
||||||
|
|
||||||
|
- The contract ships a **versioned conformance suite**. A binding is **admissible only if it
|
||||||
|
passes**: the suite exercises each declared verb and spectrum position and checks observed
|
||||||
|
behaviour matches the claim (a `write` round-trips, `notify` actually fires, an opaque shard
|
||||||
|
refuses plaintext query, implied-position rules hold). A lying/buggy profile is **rejected at
|
||||||
|
registration**, not run in production (blueprint §6.6).
|
||||||
|
- Conformance makes capability-as-data (I-3) and the §6.5 degradation contract **sound**.
|
||||||
|
Mismatch is reported as a capability-by-capability diff; an adapter may register
|
||||||
|
*degraded-but-honest* (drop the unsupported claim).
|
||||||
|
|
||||||
|
## A.3 (T14) Adapter binding — attachment-mode taxonomy
|
||||||
|
|
||||||
|
A backend MAY offer several modes; attach mode is a **per-binding, capability-gated choice**
|
||||||
|
with one declared authoritative (blueprint §6.3):
|
||||||
|
|
||||||
|
- **file-store** (native vault/folder *or* an interchange/sync mirror) · **git-IS-store** (forge
|
||||||
|
wikis & ikiwiki — git is store *and* journal; the home case) · **in-engine-hosted** (XWiki
|
||||||
|
component, Obsidian/Logseq/Roam plugin, Trilium script) · **local-REST** (Joplin Data API,
|
||||||
|
Trilium ETAPI) · **external-API-only** (Notion) · **direct-DB** (MojoMojo schema→model) ·
|
||||||
|
**CRDT-replica** (Anytype/AFFiNE/AppFlowy) · **P2P / no-central-endpoint**.
|
||||||
|
- **Backend-swap tolerance:** identity/provenance MUST survive a substrate change (bind to
|
||||||
|
capabilities, not "it's files"). **Boundary:** an **image / live-memory blob is never an
|
||||||
|
attach target** (I-12) — participation is export→files only.
|
||||||
|
- (UC-38, UC-40, UC-43, UC-50, UC-53, UC-57, UC-60, UC-62, UC-64, UC-65, UC-76, UC-79, UC-81.)
|
||||||
|
|
||||||
|
## A.4 (T12) Page model — structured / computational payload
|
||||||
|
|
||||||
|
The backend-neutral, Markdown-first page model MUST carry, without lossy flattening, every
|
||||||
|
shape in blueprint §7.1: prose; typed/computed records (incl. effective-vs-own computed
|
||||||
|
metadata); typed-graph statements (Wikibase); inline-embedded objects (Quip/Notion);
|
||||||
|
non-Markdown assets (typed asset / opaque blob / content-type registry); and the **four
|
||||||
|
computational shapes** (one-source-many-projections UC-83; notebook-with-embedded-output UC-84;
|
||||||
|
program-as-page; live/temporal). All reduce to `(content|source, structure, provenance
|
||||||
|
envelope, [derivation rule])`. **Identity is a stable handle; placement is separate; equivalence
|
||||||
|
is fingerprint-based** (blueprint §7.2, FederationRequirements ADR-01/02, I-9). The **provenance
|
||||||
|
envelope is layered** (page + span deltas; effective = page ⊕ delta, §7.3). (UC-34, UC-39,
|
||||||
|
UC-55, UC-58, UC-54, UC-44, UC-66, UC-67, UC-73, UC-83, UC-84.)
|
||||||
|
|
||||||
|
## A.5 (T13) History portability — adopt / supplement / import
|
||||||
|
|
||||||
|
Per the history axis: **adopt** git-native history as-is; **supplement** non-portable internal
|
||||||
|
history (DB / Notion / Joplin / Trilium revisions, CRDT-log) — the journal begins-now / mirrors
|
||||||
|
/ snapshots; **import** open file history (RCS, PlainFile, MojoMojo DB-version-rows) backfilled
|
||||||
|
preserving author/timestamp. **Partial/truncated history MUST be reported honestly** ("history
|
||||||
|
begins at", UC-24), never implied complete. Embedded-output documents (notebooks) use
|
||||||
|
paired-text (Jupytext) / cell-aware merge (nbdime). The space's own coordination history is the
|
||||||
|
event-sourced decision log (blueprint §8.1). (UC-36, UC-41, UC-24.)
|
||||||
|
|
||||||
|
## A.6 (T15) Syntax translation & content fidelity
|
||||||
|
|
||||||
|
Translation is a spectrum: **native → lossless → lossy-with-fidelity-report**. Read native
|
||||||
|
markup (TML, XWiki syntax, HTML) into the page model; accept Markdown overlays back via
|
||||||
|
**lossless bidirectional translation** where possible (Foswiki WysiwygPlugin is the proof). For
|
||||||
|
non-round-tripping models (Notion blocks, Trilium HTML, CRDT, typed-graph, notebook JSON/MIME):
|
||||||
|
translate lossily but **make the fidelity loss visible** — a per-shard/per-page report of what
|
||||||
|
projects cleanly vs degrades, with non-mappable elements preserved as provenance/sidecar (I-4).
|
||||||
|
Add a **structured-re-evaluable-value** point to the opacity spectrum (Wolfram expression).
|
||||||
|
Where no acceptable translation exists, the shard is a **read-only/projection** participant
|
||||||
|
(I-8), never silently corrupted. (UC-42, UC-59, UC-03, UC-73.)
|
||||||
|
|
||||||
|
## A.7 (T16) Addressing, identity & navigation
|
||||||
|
|
||||||
|
- **Span addressing:** adopt native span IDs where minted (Roam `:block/uid`, Logseq `id::`,
|
||||||
|
Notion/CRDT UUID), shard-scoped so they survive projection and don't collide; else a
|
||||||
|
position address (path+range) or content-fingerprint address. Portable tumbler is the ideal
|
||||||
|
(blueprint O-6).
|
||||||
|
- **Transclusion** = one reference-not-copy primitive over the addressable union (FederationArch
|
||||||
|
T8). **Query/navigation:** delegate to a shard's native query where capable (Datalog, DB
|
||||||
|
query, XWQL, SPARQL), **else build a derived index over the projection** (Logseq pattern);
|
||||||
|
dimensional/query-defined views are derived-tier. (UC-44–48, UC-51, UC-52, UC-54, UC-63,
|
||||||
|
UC-74.)
|
||||||
|
|
||||||
|
## A.8 (T18) Computational / executable content capability
|
||||||
|
|
||||||
|
**In scope as a page-model + projection concern; out of scope as an execution platform**
|
||||||
|
(blueprint §8.5). Core **recognises** computational types, attaches the **canonical source**,
|
||||||
|
and presents derived forms as **provenance- and liveness-marked projections**. `derive-
|
||||||
|
projection`/`execute` are **gated capabilities, OFF by default**, carrying a **trust/sandbox**
|
||||||
|
concern, **degrading to a captured snapshot / static render / recording**. One snapshot-
|
||||||
|
provenance record (run id, source rev, timestamp, environment "unguaranteed") serves notebooks,
|
||||||
|
renders, recordings. No INTENT amendment required. (UC-54, UC-55, UC-83, UC-84.)
|
||||||
|
|
||||||
|
## A.9 Conformance & module mapping
|
||||||
|
|
||||||
|
The contract maps to `src/shard_wiki/adapters/` (the bottom waist: `AdapterContract`,
|
||||||
|
`CapabilityProfile`, attachment-mode binding, the conformance suite) consuming `model/`
|
||||||
|
(page model + capability value types) and `provenance/` (blueprint §11). Each concrete adapter
|
||||||
|
ships its declared profile + a conformance pass.
|
||||||
|
|
||||||
|
*Decided:* A.1–A.8 (versioned capability contract, verified conformance, binding taxonomy,
|
||||||
|
page model, history portability, translation, addressing, gated computational content).
|
||||||
|
*Deferred:* per-backend adapter specs (one per backend, later). *Open:* blueprint §12 O-items
|
||||||
|
(addressing O-6, axis interactions O-5, span-authz O-10).
|
||||||
|
|
||||||
## 5. Integration Boundaries
|
## 5. Integration Boundaries
|
||||||
|
|
||||||
@@ -47,10 +174,13 @@ Package scaffold only (`__version__`, smoke tests). Domain model not yet coded.
|
|||||||
|
|
||||||
## 7. Use Cases
|
## 7. Use Cases
|
||||||
|
|
||||||
`spec/UseCaseCatalog.md` — 25 use cases (UC-01–UC-25) promoted from c2 wiki
|
`spec/UseCaseCatalog.md` — 84 use cases (UC-01–UC-84) from c2/yawex origins,
|
||||||
origins and yawex prior-art research.
|
the wiki-engine + modern-tool + computational research, and the syntheses.
|
||||||
|
|
||||||
## 8. Next Specification Work
|
## 8. Next Specification Work
|
||||||
|
|
||||||
Outputs from `SHARD-WP-0001` tasks (page resolution, namespaces, derived views,
|
The design layer is complete: `SHARD-WP-0001` (→ `FederationRequirements.md`),
|
||||||
provenance, overlays, link semantics) will be incorporated here as they complete.
|
`SHARD-WP-0002` (→ `FederationArchitecture.md` + §A above), and the hardened
|
||||||
|
`CoreArchitectureBlueprint.md`. The next workplan is the **implementation** of the
|
||||||
|
domain model + adapter contract (starting from §A and blueprint §11's module layout),
|
||||||
|
likely with a first spike on the keystone (the event-sourced decision log + derived fold).
|
||||||
@@ -45,6 +45,71 @@ of the shard-spectrum work; E–G carry the c2/yawex authoring and reading patte
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Capability structure for the wiki engine (reuse-surface-aligned)
|
||||||
|
|
||||||
|
*Added by SHARD-WP-0013 T2 — a structural **layer over** the thematic catalog (UC IDs unchanged),
|
||||||
|
mapping the 84 UCs onto a small **engine core** plus **typed, per-shard-activatable extensions**.
|
||||||
|
This is the bridge from demand (UCs) to the engine's typed-extension model
|
||||||
|
(`WikiEngineCoreArchitecture.md`, T5) and is aligned with the shard-wiki capability entries on the
|
||||||
|
reuse surface (`capability.wiki.*`).*
|
||||||
|
|
||||||
|
**Reading rule:** the engine is a **small always-on core**; everything else is a **typed extension
|
||||||
|
a shard activates only if it needs it** (the activation mechanism itself reuses
|
||||||
|
`capability.feature-control.evaluate`). "Default" = on/off for a standalone single-shard wiki.
|
||||||
|
|
||||||
|
### Engine core (always on — the irreducible kernel)
|
||||||
|
|
||||||
|
| Core unit | What it is | UCs | reuse-surface |
|
||||||
|
|-----------|------------|-----|---------------|
|
||||||
|
| **EC-1 Page lifecycle & content** | author / read / edit a Markdown page; delete is a history event | UC-08, UC-09 | `capability.wiki.page-model` |
|
||||||
|
| **EC-2 Identity, naming & placement** | stable page identity ≠ placement; namespace/path | UC-22 (placement), page-model kernel | `capability.wiki.page-model` |
|
||||||
|
| **EC-3 History & recoverability** | every write recoverable; history is the floor | UC-24 | `capability.wiki.coordination-journal` |
|
||||||
|
| **EC-4 Links (wikilink + red-link)** | `[[Target]]` resolution; createable red-links | UC-23 | `capability.wiki.page-model` |
|
||||||
|
| **EC-5 Storage via the adapter contract** | the engine *is* a canonical-mode shard behind §A | — | `capability.wiki.adapter-contract` (minimal profile) |
|
||||||
|
|
||||||
|
EC-1…EC-5 are the Ward-Cunningham/c2 open-wiki minimum: write a page, link pages, never lose
|
||||||
|
an edit. A shard can run with **only** the core.
|
||||||
|
|
||||||
|
### Typed extensions (activate per shard)
|
||||||
|
|
||||||
|
| Ext | Cluster | UCs | reuse-surface capability | Activatable | Default (standalone) |
|
||||||
|
|-----|---------|-----|--------------------------|-------------|----------------------|
|
||||||
|
| **X-OVERLAY** | overlay / lightweight-patch write path | UC-04, 26, 29 | `capability.wiki.overlay` | yes | on (no-op without remote/limited targets) |
|
||||||
|
| **X-AUTHZ** | access-control ladder L0→L4 | UC-06 + the L-ladder | `capability.authorization.policy-evaluate` (reuse) | yes (tiered) | L0 (open) |
|
||||||
|
| **X-VIEW** | derived views (RecentChanges, BackLinks, AllPages, SiteMap, Search) | UC-17–UC-21, UC-63 | (planned `capability.wiki.derived-views` — gap, T3) | yes | BackLinks/RecentChanges on; Search off |
|
||||||
|
| **X-STRUCT** | structured / typed / computed page data + typed-graph | UC-34, 39, 58, 67, 73 | `capability.wiki.page-model` (typed) | yes | off |
|
||||||
|
| **X-ADDR** | span addressing, transclusion, dimensional / query navigation | UC-32, 44–49, 51, 52, 74 | `capability.wiki.page-model` + query | yes (transclusion is core-lite) | transclusion on; dimensional/query off |
|
||||||
|
| **X-COMP** | computational / executable content (literate, notebook, program-as-page, live) | UC-54, 55, 83, 84 | `capability.wiki.engine-typed-extensions` (computational) | yes (gated, trust/sandbox) | off |
|
||||||
|
| **X-PROV** | rich provenance & page metadata | UC-25, 75 | `capability.wiki.page-model` (provenance) | yes (base provenance is core) | base on; rich off |
|
||||||
|
| **X-COLLAB** | c2 collaboration / social patterns | UC-10–UC-16 | (UI/convention; partial gap, T3) | yes | varies (mostly UI-layer) |
|
||||||
|
| **X-FED** | federation & union across shards | UC-01–07, 27, 28, 30–33, 56, 71, 72, 79 | `capability.wiki.shard-orchestration` + `…federation-models` | yes | off (single-shard engine) |
|
||||||
|
| **X-ATT** | attach *foreign* backends as shards | UC-35–43, 50, 53, 57, 60–62, 64–66, 68–70, 76–82 | `capability.wiki.adapter-contract` | yes | off (orchestrator role, not the engine's own shard) |
|
||||||
|
|
||||||
|
Note: X-FED and X-ATT are the **orchestrator** capabilities — for a *standalone* engine they are
|
||||||
|
off; when shard-wiki orchestrates, the engine is simply one attached (canonical) shard among many.
|
||||||
|
That is the engine↔orchestrator boundary made concrete (engine = a canonical-mode shard).
|
||||||
|
|
||||||
|
### Conflict-mediation map (how the framework reconciles tensions into one whole)
|
||||||
|
|
||||||
|
The collected UCs embed genuinely conflicting requirements. The framework mediates each with a
|
||||||
|
**mechanism**, never a hard-coded policy — so one consistent featureset serves all:
|
||||||
|
|
||||||
|
| Tension (conflicting UCs) | Mediation mechanism |
|
||||||
|
|---------------------------|---------------------|
|
||||||
|
| **Open vs governed** (UC-01 open edit ↔ UC-06 enterprise ACL) | the **L0→L4 authz ladder** (X-AUTHZ, additive); **history is the floor** at L0 — recoverability, not gatekeeping |
|
||||||
|
| **Lossless vs lossy content** (UC-42 ↔ UC-59) | **translation spectrum + fidelity report** (TSD §A.6); native form stays canonical, loss is shown not hidden |
|
||||||
|
| **Live vs snapshot** (UC-32 live transclusion ↔ UC-84/UC-3 cached) | the **live↔snapshot axis** + projection kind; degrade to a **marked** snapshot, never imply live |
|
||||||
|
| **Canonical vs chorus** (UC-07 reconcile ↔ UC-27 coexist) | **conflict detection is core, resolution is a policy preset** (chorus / designated-canonical / …) |
|
||||||
|
| **Eager vs lazy** (UC-03 projection ↔ UC-05 union) | **projection materialization** policy (lazy default; eager opt-in) |
|
||||||
|
| **Single vs multi-writer** (concurrent edits) | **per-space append authority** on the coordination journal (total order, read-your-writes) |
|
||||||
|
| **Integrated whole vs only-what-you-need** | the **typed-extension framework** + **per-shard activation** (reuse `feature-control`) — *the headline mediation*; extensions compose against typed contracts, so the featureset is one integrated whole yet selectively enabled |
|
||||||
|
| **Minimal core vs feature-rich** | **small core (EC-1…EC-5)** + extensions; nothing beyond the c2 minimum is mandatory |
|
||||||
|
|
||||||
|
This table is the seed of the engine's design rationale (T5): every conflict has a *mechanism*
|
||||||
|
that lets a shard sit anywhere on the spectrum without forking the framework.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## A. Information space, federation & coordination
|
## A. Information space, federation & coordination
|
||||||
|
|
||||||
*Federation, union, overlay, projection, and the Git-backed coordination layer (INTENT + yawex/federation seeds).*
|
*Federation, union, overlay, projection, and the Git-backed coordination layer (INTENT + yawex/federation seeds).*
|
||||||
|
|||||||
269
spec/WikiEngineCoreArchitecture.md
Normal file
269
spec/WikiEngineCoreArchitecture.md
Normal file
@@ -0,0 +1,269 @@
|
|||||||
|
# WikiEngineCoreArchitecture
|
||||||
|
|
||||||
|
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0013 T5**
|
||||||
|
|
||||||
|
The architecture of shard-wiki's **native reference wiki-engine**: a **headless, API-first**
|
||||||
|
engine — a **small core** plus a **stringent typed-extension framework** — that addresses the
|
||||||
|
whole use-case catalogue, mediates conflicting requirements into one integrated featureset, and
|
||||||
|
lets each shard **activate only what it needs**. Authoritative as of the ratified INTENT
|
||||||
|
amendment (2026-06-15, decision `84ffdb48`): the engine is **additive** and is shard-wiki's
|
||||||
|
**reference first-party shard backend (a canonical-mode shard)** — not a replacement for other
|
||||||
|
engines, not a UI.
|
||||||
|
|
||||||
|
Relation to other specs (referenced, not restated):
|
||||||
|
- `CoreArchitectureBlueprint.md` — the orchestrator/whole-system architecture. **The engine is
|
||||||
|
one shard behind §A; federation, union, projection, and cross-shard coordination are the
|
||||||
|
orchestrator's job, not the engine's.** That is what keeps the engine small.
|
||||||
|
- `TechnicalSpecificationDocument.md §A` — the shard adapter contract the engine implements.
|
||||||
|
- `FederationRequirements.md` — page resolution, overlay, link semantics (ADRs the engine reuses).
|
||||||
|
- `UseCaseCatalog.md` "Capability structure" layer (T2) — the core-vs-extension map + the
|
||||||
|
conflict-mediation map this document realizes.
|
||||||
|
- reuse surface (`capability.wiki.*`, plus consumed `feature-control` / `authorization`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Thesis: a small page-store kernel; everything else is a typed extension
|
||||||
|
|
||||||
|
> **The engine is a page-store kernel with a typed-extension runtime. Every capability beyond
|
||||||
|
> the c2-minimum is a *typed extension* a shard activates only if it needs it — and a shard's
|
||||||
|
> externally-visible capability profile is *computed from its active extension set*.**
|
||||||
|
|
||||||
|
That single chain — **configuration (which extensions) → capability (what the shard can do) →
|
||||||
|
conformance (verified)** — is the whole design. It mirrors the orchestrator's discipline
|
||||||
|
(`CoreArchitectureBlueprint` §6.5: capability-as-data, verified, no per-backend code) and turns
|
||||||
|
"integrated whole, yet activate only what you need" from a slogan into a mechanism.
|
||||||
|
|
||||||
|
The engine stays small for a structural reason: it is **one shard**, not a federation layer.
|
||||||
|
Union, projection, equivalence, cross-shard overlay-orchestration, and the federation models all
|
||||||
|
live in shard-wiki's orchestrator (the blueprint). The engine implements `ShardAdapter` (§A) and
|
||||||
|
nothing above it. So "wiki engine" here means *a really good single canonical shard with a
|
||||||
|
typed-extension framework and a headless agent-first API* — not a re-implementation of shard-wiki.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Engine invariants
|
||||||
|
|
||||||
|
| # | Invariant | Why |
|
||||||
|
|---|-----------|-----|
|
||||||
|
| E-1 | **One shard, not a federation layer.** The engine implements `ShardAdapter` (§A); union/projection/federation are the orchestrator's. | Keeps the engine small; no duplication of the blueprint. |
|
||||||
|
| E-2 | **Small kernel.** The kernel is only: page store + history, the page model (reused), the extension runtime, the API. | Common case (a plain wiki) is trivial. |
|
||||||
|
| E-3 | **Everything else is a typed extension.** No feature beyond the c2-minimum is baked into the kernel. | Integrated-whole-yet-selective; testable boundary. |
|
||||||
|
| E-4 | **Per-shard activation.** A shard runs an *activation profile* (a set of extensions + config); unused features cost nothing. | "Activate only what you need." |
|
||||||
|
| E-5 | **Capability profile is derived from active extensions.** The §A profile the engine declares is computed from its activation profile, then conformance-verified. | One source of truth; honest, verified capabilities. |
|
||||||
|
| E-6 | **Headless & API-first.** The API is the only interface; no bundled UI/rendering (consumer concern, L6). | INTENT amendment; clean orchestrator/consumer split. |
|
||||||
|
| E-7 | **Agent-first ergonomics.** The API is typed, introspectable, batchable, low-round-trip. | INTENT: optimized for efficient agent/automation access. |
|
||||||
|
| E-8 | **Reuse over reinvent.** Page model, history/journal, activation, and authz are *consumed* (existing capabilities), not rebuilt. | Smallness; reuse-surface alignment. |
|
||||||
|
| E-9 | **Extensions are typed & verified.** An extension declares its types/hooks/deps; activation is rejected if types conflict or deps are unmet (impossible profiles forbidden). | Stringency; mirrors §6.5 + conformance. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. The kernel (four concepts)
|
||||||
|
|
||||||
|
The kernel is deliberately four things — nothing more is mandatory.
|
||||||
|
|
||||||
|
1. **Page** — the backend-neutral page model (`capability.wiki.page-model`, reused as-is):
|
||||||
|
stable identity ≠ placement, layered provenance, page shapes. The kernel does **not** redefine
|
||||||
|
it; extensions may *register additional shapes/types* (§4).
|
||||||
|
2. **Store + history** — a git-backed page store (the engine is the *git-IS-store* case from the
|
||||||
|
blueprint): a write is a commit; history is native and recoverable (E-3/I-10). Coordination
|
||||||
|
decisions reuse the event-sourced journal (`capability.wiki.coordination-journal`).
|
||||||
|
3. **Extension runtime** — the typed-extension registry, hook dispatcher, type checker, and
|
||||||
|
activation engine (§4). *This is the core innovation; it is the only “framework” in the kernel.*
|
||||||
|
4. **API** — the headless, typed, agent-first surface (§7). Kernel endpoints cover the c2-minimum
|
||||||
|
(page CRUD-as-history, links, history); extensions extend the surface through typed routes.
|
||||||
|
|
||||||
|
The **c2-minimum** a kernel-only shard delivers (no extensions): write a page, link pages
|
||||||
|
(`[[wikilink]]` + red-link), never lose an edit. That is a complete, useful headless wiki.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. The typed-extension model (the framework)
|
||||||
|
|
||||||
|
An **Extension** is a typed unit declaring a contract the runtime enforces:
|
||||||
|
|
||||||
|
```
|
||||||
|
Extension:
|
||||||
|
id : reverse-domain id (e.g. ext.struct.typed-records)
|
||||||
|
provides : capability ids it realizes (reuse-surface; e.g. capability.wiki.page-model[typed])
|
||||||
|
types : page shapes / field schemas / content-types it introduces (typed, validated)
|
||||||
|
hooks : kernel lifecycle bindings it implements (see below)
|
||||||
|
api : typed routes it adds to the headless surface
|
||||||
|
depends_on : other extensions / consumed capabilities required
|
||||||
|
conflicts_with: extensions it cannot co-activate with
|
||||||
|
config : declared, schema-checked activation parameters
|
||||||
|
```
|
||||||
|
|
||||||
|
**Hooks (the kernel lifecycle the runtime dispatches):**
|
||||||
|
`on_resolve` (name→page), `on_read`, `on_write` (validate/transform a draft), `on_link`
|
||||||
|
(link/transclusion resolution), `on_history`, `on_query`, `on_render_request` (produce a derived
|
||||||
|
representation for a consumer), `on_profile` (contribute capability-spectrum positions, E-5).
|
||||||
|
Hooks are **typed** (typed inputs/outputs) and dispatched in a **declared, deterministic order**.
|
||||||
|
|
||||||
|
**Typing & composition (stringency):**
|
||||||
|
- At activation, the runtime builds the **dependency closure**, checks **type consistency** (no
|
||||||
|
two active extensions claim incompatible types for the same page shape/field; `conflicts_with`
|
||||||
|
honoured), and rejects an **impossible profile** — exactly the §6.5 implication-rule discipline,
|
||||||
|
applied to extensions. A rejected profile fails fast at boot, never silently.
|
||||||
|
- Composition is **deterministic**: hook order is declared; conflicts are resolved by explicit
|
||||||
|
precedence or rejection, never by accident.
|
||||||
|
- Extensions ship a **conformance check** (mirrors §6.6): an activated extension is exercised
|
||||||
|
against its declared types/hooks before the shard serves traffic — *typed contracts verified,
|
||||||
|
not trusted*.
|
||||||
|
|
||||||
|
**Per-shard activation (reuse, not reinvent):**
|
||||||
|
- A shard's **activation profile** = `{extension id → config}`. Activation/evaluation **reuses
|
||||||
|
`capability.feature-control.evaluate`** (helix_forge/feature-control) — shard-wiki does not
|
||||||
|
build a bespoke flagging system (T3 consumption).
|
||||||
|
- **E-5 in action:** the engine's `on_profile` hooks fold the active extensions into the §A
|
||||||
|
**capability profile** the shard advertises to the orchestrator (e.g. activate
|
||||||
|
`ext.struct.typed-records` → the `structure` spectrum rises and `structured-payload` is
|
||||||
|
declared). The profile is then conformance-verified (§A.2). *Configuration → capability →
|
||||||
|
conformance is one chain.*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Featureset map: core vs extensions, and conflict mediation
|
||||||
|
|
||||||
|
The engine realizes the T2 "Capability structure" layer (`UseCaseCatalog.md`). Mapping (the
|
||||||
|
*page/content-level* clusters; **X-FED and X-ATT are orchestrator concerns, not engine
|
||||||
|
extensions** — E-1):
|
||||||
|
|
||||||
|
| Engine kernel (always on) | T2 | reuse-surface |
|
||||||
|
|---------------------------|----|---------------|
|
||||||
|
| Page lifecycle, identity/placement, history, links, store | EC-1…EC-5 | `capability.wiki.page-model`, `…coordination-journal`, `…adapter-contract` |
|
||||||
|
|
||||||
|
| Built-in typed extension | T2 cluster | provides / consumes | default |
|
||||||
|
|--------------------------|-----------|---------------------|---------|
|
||||||
|
| `ext.overlay` | X-OVERLAY | `capability.wiki.overlay` | on (no-op locally) |
|
||||||
|
| `ext.authz` (L0→L4 tiers) | X-AUTHZ | consumes `capability.authorization.policy-evaluate` | L0 |
|
||||||
|
| `ext.views` (BackLinks/RecentChanges/…) | X-VIEW | `capability.wiki.derived-views` | BackLinks/RecentChanges on |
|
||||||
|
| `ext.struct` (typed/computed/graph) | X-STRUCT | `capability.wiki.page-model[typed]` | off |
|
||||||
|
| `ext.addr` (span addr / transclusion / query) | X-ADDR | `capability.wiki.page-model`+query | transclusion on |
|
||||||
|
| `ext.compute` (literate/notebook/program/live) | X-COMP | `capability.wiki.engine-typed-extensions` | off (gated, sandbox) |
|
||||||
|
| `ext.prov` (rich provenance/metadata) | X-PROV | `capability.wiki.page-model[provenance]` | base on |
|
||||||
|
| `ext.collab` (c2 social patterns) | X-COLLAB | (UI/convention; mostly consumer) | off |
|
||||||
|
|
||||||
|
**Conflict mediation (T2 map) realized by the framework** — every tension is a *mechanism*, not a
|
||||||
|
baked-in choice, so one featureset serves all:
|
||||||
|
|
||||||
|
| Tension | Realized by |
|
||||||
|
|---------|-------------|
|
||||||
|
| open vs governed | `ext.authz` tiers (additive); kernel history is the floor at L0 |
|
||||||
|
| lossless vs lossy | a `translate` hook + fidelity report (consumes the proposed `capability.content.translation-fidelity`, G2) |
|
||||||
|
| live vs snapshot | `ext.compute`/`ext.addr` mark liveness; degrade to snapshot (never imply live) |
|
||||||
|
| canonical vs chorus | detection in kernel; resolution is a policy preset (orchestrator) |
|
||||||
|
| integrated-whole vs only-what-you-need | **the activation profile** (E-4) + typed composition (§4) — the headline mediation |
|
||||||
|
| minimal vs feature-rich | small kernel (§3) + extensions; nothing beyond c2 is mandatory |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. The engine as a canonical-mode shard
|
||||||
|
|
||||||
|
The engine exposes itself through an `EngineShardAdapter` implementing §A:
|
||||||
|
|
||||||
|
- **Substrate** git-IS-store; **history** git-native; **write** = commit; `current_rev` = sha
|
||||||
|
(apply-under-drift works out of the box). It is the **most capable shard** shard-wiki can
|
||||||
|
attach — it dogfoods the contract.
|
||||||
|
- Its **capability profile is computed from active extensions** (E-5) and **conformance-verified**
|
||||||
|
(§A.2) — so the orchestrator sees an honest profile, and federation ops degrade by the engine's
|
||||||
|
*actually-activated* capabilities.
|
||||||
|
- The orchestrator attaches it like any shard; **federation/union/projection are not in the
|
||||||
|
engine** (E-1). A standalone deployment is "the engine as the sole canonical shard"; a
|
||||||
|
federated deployment is "the engine as one shard among many." Same engine, no re-architecture.
|
||||||
|
|
||||||
|
This is the precise realization of the INTENT reconciliation: shard-wiki orchestrates; the engine
|
||||||
|
is the first-party shard it can attach.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Headless API surface & agent ergonomics (E-6/E-7)
|
||||||
|
|
||||||
|
API-first means the typed API is the product; there is no UI. Agent-first means it is designed
|
||||||
|
for cheap, deterministic machine consumption:
|
||||||
|
|
||||||
|
- **Typed resource API** over pages, links, history, spans — content-negotiated (raw Markdown,
|
||||||
|
the structured page model, or an extension-rendered representation via `on_render_request`).
|
||||||
|
- **Capability/extension introspection** — an endpoint returns the shard's **active extensions,
|
||||||
|
their types, and the derived §A capability profile**, so an agent can discover *what this shard
|
||||||
|
can do* before acting (no trial-and-error). This is the agent-facing twin of E-5.
|
||||||
|
- **Batch & query** — multi-page reads, link-graph and RecentChanges queries (via `ext.views`),
|
||||||
|
and `on_query` delegation — minimizing round-trips.
|
||||||
|
- **Write via overlay** — edits go through the overlay path (FederationRequirements ADR-05), so
|
||||||
|
agent writes are safe (draft → apply-under-drift) and attributable.
|
||||||
|
- **Deterministic & provenance-carrying** — every response carries the provenance envelope;
|
||||||
|
identical inputs yield identical outputs (no hidden state) — friendly to caching agents.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Implementation sketch (module layout)
|
||||||
|
|
||||||
|
The engine lives under the shard-wiki package as a backend (it sits at L0/L1 — a shard behind the
|
||||||
|
adapter; nothing in the orchestrator depends *up* on it):
|
||||||
|
|
||||||
|
```
|
||||||
|
src/shard_wiki/engine/
|
||||||
|
kernel.py # page store + history (git-IS-store), lifecycle; reuses model/, provenance/, coordination/
|
||||||
|
extension.py # Extension contract, registry, typed hook dispatcher, type checker
|
||||||
|
activation.py # activation profile; reuses capability.feature-control.evaluate
|
||||||
|
profile.py # derive the §A CapabilityProfile from active extensions (E-5) + conformance
|
||||||
|
api.py # headless, typed, agent-first surface (+ extension introspection)
|
||||||
|
adapter.py # EngineShardAdapter implements adapters/ ShardAdapter (canonical-mode shard)
|
||||||
|
extensions/ # built-ins: overlay/ authz/ views/ struct/ addr/ compute/ prov/ collab/
|
||||||
|
```
|
||||||
|
|
||||||
|
Dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`, `adapters/`
|
||||||
|
(contract), `policy/`; it is consumed *only* via its `EngineShardAdapter` (the orchestrator
|
||||||
|
attaches it as a shard). No orchestrator-tier (`union/`, `projection/`) import.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Reuse (what the engine consumes vs registers)
|
||||||
|
|
||||||
|
- **Consumes:** `capability.feature-control.evaluate` (activation), `capability.authorization.
|
||||||
|
policy-evaluate` (`ext.authz`), the proposed `capability.content.translation-fidelity` (G2,
|
||||||
|
lossy translation), and shard-wiki's own `capability.wiki.{page-model, coordination-journal,
|
||||||
|
adapter-contract, overlay, derived-views}`.
|
||||||
|
- **Registers / realizes:** `capability.wiki.engine-typed-extensions` (this document is its
|
||||||
|
Discovery evidence — D2→D3 on ratification). The cross-cutting **typed-extension framework**
|
||||||
|
pattern is proposed back to the reuse surface as **G1** (`capability.platform.typed-extension-
|
||||||
|
framework`); this engine is its first instance.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Traceability
|
||||||
|
|
||||||
|
- **INTENT** — realizes the 2026-06-15 amendment (decision `84ffdb48`): headless, API-first,
|
||||||
|
additive native engine = canonical-mode shard backend; honours all engine invariants and the
|
||||||
|
orchestrator boundary (E-1).
|
||||||
|
- **Use cases** — the kernel/extension split *is* the T2 "Capability structure" layer
|
||||||
|
(`UseCaseCatalog.md`); every UC is either kernel (EC-1…EC-5) or a named extension; conflicts
|
||||||
|
use the T2 mediation map (§5). The engine must ultimately cover UC-01–UC-84 (per-shard subsets).
|
||||||
|
- **Architecture** — consistent with `CoreArchitectureBlueprint` (engine = canonical-mode shard,
|
||||||
|
§6 contract, §7 page model, §8.1 journal) and `TechnicalSpecificationDocument §A` (the contract
|
||||||
|
it implements). `FederationRequirements` ADR-05/06 supply overlay + link semantics.
|
||||||
|
- **Reuse surface** — §9; G1/G2 proposals from SHARD-WP-0013 T3.
|
||||||
|
|
||||||
|
## 11. Decisions / deferred / open
|
||||||
|
|
||||||
|
**Decided:** small page-store kernel + typed-extension runtime (E-2/E-3); engine is one shard,
|
||||||
|
not a federation layer (E-1); capability profile derived from active extensions (E-5); headless,
|
||||||
|
API-first, agent-first (E-6/E-7); activation reuses `feature-control` (E-8); extensions are
|
||||||
|
typed + conformance-verified (E-9).
|
||||||
|
|
||||||
|
**Deferred:** the concrete extension SDK/ABI and hook signatures; the API protocol (REST/GraphQL/
|
||||||
|
MCP) — agent-first introspection is required, the wire format is an implementation spike; the
|
||||||
|
built-in extensions' internal designs (each is a later workplan).
|
||||||
|
|
||||||
|
**Open (tracked):** does `ext.compute` ever execute in-process or strictly delegate/snapshot
|
||||||
|
(ties blueprint §8.5 + trust/sandbox); is the typed-extension framework promoted to the
|
||||||
|
reuse-surface platform capability (G1) and then *consumed* here rather than engine-owned;
|
||||||
|
introspection granularity vs. leaking internal structure to agents.
|
||||||
|
|
||||||
|
## 12. Stability note
|
||||||
|
|
||||||
|
The **thesis (§1)** and **invariants (§2)** — especially *engine-is-one-shard* (E-1),
|
||||||
|
*small-kernel/everything-else-typed-extension* (E-2/E-3), and *capability-profile-derived-from-
|
||||||
|
extensions* (E-5) — are load-bearing. Changing them (e.g. moving federation into the engine, or
|
||||||
|
baking a feature into the kernel) is an architectural change in the sense of INTENT's Stability
|
||||||
|
Note and should be rare and deliberate. The headless/API-first posture is fixed by the ratified
|
||||||
|
INTENT amendment.
|
||||||
79
spec/adr/ADR-0001-engine-activation-via-feature-control.md
Normal file
79
spec/adr/ADR-0001-engine-activation-via-feature-control.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# ADR-0001 — Engine extension activation via feature-control (OpenFeature)
|
||||||
|
|
||||||
|
Status: **Accepted** · Date: 2026-06-15 · Deciders: tegwick · Source: SHARD-WP-0013 follow-up
|
||||||
|
(feature-control assessment)
|
||||||
|
|
||||||
|
> First repo-level ADR. (Note: `FederationRequirements.md` contains document-internal
|
||||||
|
> "ADR-01…06" design notes — those are scoped to that spec; this `spec/adr/` series is the
|
||||||
|
> repository's standalone architecture decision log, starting here.)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`WikiEngineCoreArchitecture.md` (SHARD-WP-0013 T5) defines the engine as a small kernel plus a
|
||||||
|
**typed-extension framework** where each shard **activates only the extensions it needs**
|
||||||
|
(invariants E-4 activation, E-8 reuse-not-reinvent). It needs a mechanism to decide, per shard
|
||||||
|
(and per tenant/context), which extensions/features are active — without baking a bespoke flag
|
||||||
|
system into the engine, and without breaking the **standalone, zero-external-dependency** L0
|
||||||
|
posture shard-wiki guarantees.
|
||||||
|
|
||||||
|
The helix_forge sibling **`feature-control`** (`capability.feature-control.evaluate`, registered
|
||||||
|
at **D5 / A4 / C3 / R3**) provides exactly this: an **OpenFeature**-based feature-availability
|
||||||
|
control plane with a working SDK (`feature_control_sdk`: `FeatureControlClient`, `Resolver`, a
|
||||||
|
static `LocalProvider`), context-scoped evaluation (`tenant_id`/scope), explainable decisions,
|
||||||
|
and graceful degradation when OpenFeature is absent. shard-wiki already proposed this as a T3
|
||||||
|
*consumption* (reuse, don't rebuild).
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
**Adopt `feature-control` (via the OpenFeature standard) as the engine's per-shard extension/
|
||||||
|
feature activation mechanism** — *availability only* — with these constraints:
|
||||||
|
|
||||||
|
1. **OpenFeature-shaped, provider-pluggable.** The engine evaluates activation through an
|
||||||
|
OpenFeature-style client. A static **`LocalProvider`** is the **standalone/L0 default**
|
||||||
|
(zero external dependency); a `feature-control`/remote provider is plugged in for governed
|
||||||
|
deployments. This mirrors shard-wiki's existing **identity-provider ladder** (null/local
|
||||||
|
default → external when present).
|
||||||
|
2. **Availability ≠ authorization.** feature-control decides *which extensions are active*,
|
||||||
|
never *who may read/write*. Authorization stays in core (X-AUTHZ / `authorization.policy-
|
||||||
|
evaluate`). The two are composed but never conflated. (feature-control's own INTENT requires
|
||||||
|
this.)
|
||||||
|
3. **Engine layer, not the orchestrator foundation.** Integration lives in
|
||||||
|
`engine/activation.py`; the current `src/shard_wiki/` core stays dependency-free. OpenFeature/
|
||||||
|
feature-control is an optional extra, kept out of the standalone path by the `LocalProvider`.
|
||||||
|
4. **Thin slice only.** Consume `feature-control.evaluate` (mature, A4). Do **not** take a
|
||||||
|
dependency on the heavier control-plane governance / `rollout` / `visibility` (A2) until a
|
||||||
|
concrete need appears.
|
||||||
|
|
||||||
|
Activation keys = extension ids; evaluation context = `{tenant_id: root-entity, shard_id, …}`;
|
||||||
|
the resulting active-extension set then **derives** the shard's §A capability profile (E-5).
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
**Positive**
|
||||||
|
- No bespoke flag system; reuses a mature (D5/A4) capability — reuse-surface aligned.
|
||||||
|
- Standalone stays zero-dep (LocalProvider); governed deployments get real runtime control,
|
||||||
|
multi-tenant scoping, and **explainable** decisions that feed the engine's agent-introspection
|
||||||
|
API (E-7: "why is extension X off for this shard?").
|
||||||
|
- "Activate only what you need" + compute control become first-class and reversible at runtime.
|
||||||
|
- Clean layering: availability (feature-control) vs authorization (core) vs identity (provider).
|
||||||
|
|
||||||
|
**Negative / risks (mitigated by the constraints)**
|
||||||
|
- An optional OpenFeature dependency at the engine layer (mitigated: out of the standalone path).
|
||||||
|
- Coupling to an external control plane in governed mode (mitigated: provider-pluggable, degrade
|
||||||
|
to LocalProvider).
|
||||||
|
- Temptation to route authz through it (mitigated: constraint 2, hard boundary).
|
||||||
|
|
||||||
|
## Alternatives considered
|
||||||
|
|
||||||
|
- **Bespoke per-shard flag/config in the engine** — rejected: reinvents feature-control, no
|
||||||
|
standard, no multi-tenant/explainability, violates reuse-not-reinvent (E-8).
|
||||||
|
- **No activation (all extensions always on)** — rejected: defeats "small core + activate only
|
||||||
|
what you need" (E-2/E-4) and the compute-control goal.
|
||||||
|
- **Build on the heavier feature-control control-plane now** — deferred: over-scoping a single
|
||||||
|
engine's activation; revisit if rollout/governance needs emerge.
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
`WikiEngineCoreArchitecture.md` (E-4/E-8, §4 activation), `UseCaseCatalog.md` capability-structure
|
||||||
|
layer (X-AUTHZ vs activation), `history/260615-reuse-surface-contributions.md` (T3 consumption),
|
||||||
|
reuse-surface `capability.feature-control.evaluate`, INTENT amendment decision `84ffdb48`.
|
||||||
11
spec/adr/README.md
Normal file
11
spec/adr/README.md
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
# spec/adr/ — Architecture Decision Records
|
||||||
|
|
||||||
|
Repository-level ADRs: one decision per file, `ADR-NNNN-<slug>.md`, status
|
||||||
|
**Proposed / Accepted / Superseded**. Each records Context · Decision · Consequences ·
|
||||||
|
Alternatives. These are the standalone, numbered decision log; design-note "ADRs" embedded
|
||||||
|
inside a spec (e.g. `FederationRequirements.md` ADR-01…06) are scoped to that document and are
|
||||||
|
not part of this series.
|
||||||
|
|
||||||
|
| ADR | Status | Subject |
|
||||||
|
|-----|--------|---------|
|
||||||
|
| [ADR-0001](ADR-0001-engine-activation-via-feature-control.md) | Accepted | Engine extension activation via feature-control (OpenFeature), availability-only, LocalProvider standalone default |
|
||||||
@@ -1,10 +1,16 @@
|
|||||||
"""shard-wiki — Git-based Markdown wiki orchestrator and federation layer.
|
"""shard-wiki — Git-based Markdown wiki orchestrator and federation layer.
|
||||||
|
|
||||||
See INTENT.md for the authoritative specification of scope and boundaries.
|
See INTENT.md for the authoritative specification of scope and boundaries, and
|
||||||
This package orchestrates wiki-shaped content across heterogeneous *shards*;
|
spec/CoreArchitectureBlueprint.md for the architecture. This package orchestrates
|
||||||
it is not itself a wiki engine.
|
wiki-shaped content across heterogeneous *shards*; it is not itself a wiki engine.
|
||||||
|
|
||||||
|
Foundation slice (SHARD-WP-0007): attach folder shard(s) to an
|
||||||
|
:class:`~shard_wiki.space.InformationSpace`, resolve a name through the union, and
|
||||||
|
read a page with layered provenance (chorus on ambiguity).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.space import InformationSpace
|
||||||
|
|
||||||
__version__ = "0.0.0"
|
__version__ = "0.0.0"
|
||||||
|
|
||||||
__all__ = ["__version__"]
|
__all__ = ["__version__", "InformationSpace"]
|
||||||
|
|||||||
25
src/shard_wiki/adapters/__init__.py
Normal file
25
src/shard_wiki/adapters/__init__.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
"""adapters/ — the shard adapter contract (bottom waist) and concrete adapters."""
|
||||||
|
|
||||||
|
from shard_wiki.adapters.conformance import (
|
||||||
|
Check,
|
||||||
|
ConformanceError,
|
||||||
|
ConformanceReport,
|
||||||
|
assert_conformant,
|
||||||
|
run_conformance,
|
||||||
|
)
|
||||||
|
from shard_wiki.adapters.contract import CONTRACT_VERSION, ShardAdapter
|
||||||
|
from shard_wiki.adapters.folder import FolderAdapter
|
||||||
|
from shard_wiki.adapters.git import GitShardAdapter, PageRevision
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ShardAdapter",
|
||||||
|
"FolderAdapter",
|
||||||
|
"GitShardAdapter",
|
||||||
|
"PageRevision",
|
||||||
|
"CONTRACT_VERSION",
|
||||||
|
"Check",
|
||||||
|
"ConformanceReport",
|
||||||
|
"ConformanceError",
|
||||||
|
"run_conformance",
|
||||||
|
"assert_conformant",
|
||||||
|
]
|
||||||
142
src/shard_wiki/adapters/conformance.py
Normal file
142
src/shard_wiki/adapters/conformance.py
Normal file
@@ -0,0 +1,142 @@
|
|||||||
|
"""Adapter conformance — profiles are verified, not self-asserted (TSD §A.2, blueprint §6.6).
|
||||||
|
|
||||||
|
Capability-as-data (I-3) is only sound if a binding's *declared* profile matches its *observed*
|
||||||
|
behaviour. This battery exercises a binding and reports, check by check, whether claim == reality;
|
||||||
|
``assert_conformant`` gates registration. A lying/buggy profile fails here instead of silently
|
||||||
|
poisoning degradation decisions downstream.
|
||||||
|
|
||||||
|
This slice verifies the read path + honest absence of unclaimed verbs. Positive probes for
|
||||||
|
claimed write/diff/merge are deferred (they mutate) to a later workplan.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.adapters.contract import ShardAdapter
|
||||||
|
from shard_wiki.model import NotSupported, Verb
|
||||||
|
|
||||||
|
__all__ = ["Check", "ConformanceReport", "ConformanceError", "run_conformance", "assert_conformant"]
|
||||||
|
|
||||||
|
# Optional verbs whose *absence* must be honest (calling an unclaimed one raises NotSupported).
|
||||||
|
_HONEST_ABSENCE_VERBS = (Verb.WRITE, Verb.DIFF, Verb.NOTIFY)
|
||||||
|
_PROBE = {
|
||||||
|
Verb.WRITE: lambda a: a.write("__probe__", ""),
|
||||||
|
Verb.DIFF: lambda a: a.diff("__probe__", "__probe2__"),
|
||||||
|
Verb.NOTIFY: lambda a: a.notify(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Check:
|
||||||
|
name: str
|
||||||
|
ok: bool
|
||||||
|
detail: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ConformanceReport:
|
||||||
|
adapter: str
|
||||||
|
checks: tuple[Check, ...]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ok(self) -> bool:
|
||||||
|
return all(c.ok for c in self.checks)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def failures(self) -> tuple[Check, ...]:
|
||||||
|
return tuple(c for c in self.checks if not c.ok)
|
||||||
|
|
||||||
|
def diff(self) -> str:
|
||||||
|
return "; ".join(f"{c.name}: {c.detail}" for c in self.failures) or "conformant"
|
||||||
|
|
||||||
|
|
||||||
|
class ConformanceError(Exception):
|
||||||
|
def __init__(self, report: ConformanceReport) -> None:
|
||||||
|
super().__init__(f"{report.adapter} not conformant — {report.diff()}")
|
||||||
|
self.report = report
|
||||||
|
|
||||||
|
|
||||||
|
def _safe(fn, name: str, ok_detail: str = "") -> Check:
|
||||||
|
try:
|
||||||
|
return fn()
|
||||||
|
except Exception as exc: # noqa: BLE001 — a check must never crash the battery
|
||||||
|
return Check(name, False, f"unexpected error: {exc!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def run_conformance(adapter: ShardAdapter) -> ConformanceReport:
|
||||||
|
checks: list[Check] = []
|
||||||
|
profile = None
|
||||||
|
|
||||||
|
def _profile_validates() -> Check:
|
||||||
|
nonlocal profile
|
||||||
|
profile = adapter.profile()
|
||||||
|
profile.validate()
|
||||||
|
return Check("profile-validates", True)
|
||||||
|
|
||||||
|
checks.append(_safe(_profile_validates, "profile-validates"))
|
||||||
|
if profile is None: # profile() or validate() failed; can't probe further meaningfully
|
||||||
|
return ConformanceReport(type(adapter).__name__, tuple(checks))
|
||||||
|
|
||||||
|
# READ is the capability floor.
|
||||||
|
checks.append(Check("supports-read", profile.supports(Verb.READ),
|
||||||
|
"" if profile.supports(Verb.READ) else "READ not declared"))
|
||||||
|
|
||||||
|
# READ round-trips: a declared-readable shard must actually read its own keys.
|
||||||
|
def _read_round_trips() -> Check:
|
||||||
|
keys = list(adapter.keys())
|
||||||
|
if not keys:
|
||||||
|
return Check("read-round-trips", True, "empty shard")
|
||||||
|
page = adapter.read(keys[0])
|
||||||
|
if page.identity.shard != adapter.shard_id:
|
||||||
|
return Check("read-round-trips", False,
|
||||||
|
f"page shard {page.identity.shard!r} != {adapter.shard_id!r}")
|
||||||
|
if not isinstance(page.body, str):
|
||||||
|
return Check("read-round-trips", False, "body is not text")
|
||||||
|
return Check("read-round-trips", True)
|
||||||
|
|
||||||
|
if profile.supports(Verb.READ):
|
||||||
|
checks.append(_safe(_read_round_trips, "read-round-trips"))
|
||||||
|
|
||||||
|
# WRITE positive probe: a claimed-writable shard must actually round-trip a write. The probe
|
||||||
|
# is content-preserving (rewrite an existing page with its own body) so it is non-destructive.
|
||||||
|
def _write_round_trips() -> Check:
|
||||||
|
keys = list(adapter.keys())
|
||||||
|
if not keys:
|
||||||
|
return Check("write-round-trips", True, "empty shard")
|
||||||
|
k = keys[0]
|
||||||
|
original = adapter.read(k).body
|
||||||
|
adapter.write(k, original)
|
||||||
|
if adapter.read(k).body != original:
|
||||||
|
return Check("write-round-trips", False, "rewrite did not preserve body")
|
||||||
|
return Check("write-round-trips", True)
|
||||||
|
|
||||||
|
if profile.supports(Verb.WRITE):
|
||||||
|
checks.append(_safe(_write_round_trips, "write-round-trips"))
|
||||||
|
|
||||||
|
# Honest absence: an *unclaimed* optional verb must raise NotSupported when invoked.
|
||||||
|
for verb in _HONEST_ABSENCE_VERBS:
|
||||||
|
if profile.supports(verb):
|
||||||
|
continue # claimed → positive probe deferred (would mutate)
|
||||||
|
name = f"honest-absence:{verb.value}"
|
||||||
|
|
||||||
|
def _probe(v=verb, n=name) -> Check:
|
||||||
|
try:
|
||||||
|
_PROBE[v](adapter)
|
||||||
|
except NotSupported:
|
||||||
|
return Check(n, True)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return Check(n, False, f"raised {type(exc).__name__}, expected NotSupported")
|
||||||
|
return Check(n, False, "did not raise NotSupported though verb is unclaimed")
|
||||||
|
|
||||||
|
checks.append(_probe())
|
||||||
|
|
||||||
|
return ConformanceReport(type(adapter).__name__, tuple(checks))
|
||||||
|
|
||||||
|
|
||||||
|
def assert_conformant(adapter: ShardAdapter) -> ConformanceReport:
|
||||||
|
"""Run the battery; raise :class:`ConformanceError` if any check fails. Returns the report."""
|
||||||
|
report = run_conformance(adapter)
|
||||||
|
if not report.ok:
|
||||||
|
raise ConformanceError(report)
|
||||||
|
return report
|
||||||
52
src/shard_wiki/adapters/contract.py
Normal file
52
src/shard_wiki/adapters/contract.py
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
"""The shard adapter contract — the bottom narrow waist (CoreArchitectureBlueprint §6, TSD §A).
|
||||||
|
|
||||||
|
A backend participates by implementing :class:`ShardAdapter`. ``shard_id``, ``profile`` and
|
||||||
|
``read`` are mandatory; everything else is an optional capability that defaults to raising
|
||||||
|
:class:`~shard_wiki.model.NotSupported` — so a limited backend is honest about what it can't do
|
||||||
|
(graceful degradation, I-8) and core never assumes a capability it wasn't given (capability-as-
|
||||||
|
data, I-3). Declared profiles are verified by the conformance suite (T4), never taken on trust.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import abc
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
from shard_wiki.model import CapabilityProfile, NotSupported, Page
|
||||||
|
|
||||||
|
__all__ = ["ShardAdapter", "CONTRACT_VERSION"]
|
||||||
|
|
||||||
|
CONTRACT_VERSION = "0.1"
|
||||||
|
|
||||||
|
|
||||||
|
class ShardAdapter(abc.ABC):
|
||||||
|
"""Versioned interface a backend implements to attach as a shard."""
|
||||||
|
|
||||||
|
contract_version: str = CONTRACT_VERSION
|
||||||
|
|
||||||
|
@property
|
||||||
|
@abc.abstractmethod
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
"""Stable id scoping every Identity this shard mints."""
|
||||||
|
|
||||||
|
@abc.abstractmethod
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
"""The (to-be-verified) capability profile of this binding."""
|
||||||
|
|
||||||
|
@abc.abstractmethod
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
"""The stable page keys this shard offers (the handle half of Identity)."""
|
||||||
|
|
||||||
|
@abc.abstractmethod
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
"""Read one page by its stable key. Raises ``KeyError`` if absent."""
|
||||||
|
|
||||||
|
# --- optional capability verbs: honest NotSupported by default ---
|
||||||
|
def write(self, key: str, body: str) -> Page: # noqa: ARG002
|
||||||
|
raise NotSupported(f"{type(self).__name__} does not support write")
|
||||||
|
|
||||||
|
def diff(self, key: str, other: str) -> str: # noqa: ARG002
|
||||||
|
raise NotSupported(f"{type(self).__name__} does not support diff")
|
||||||
|
|
||||||
|
def notify(self):
|
||||||
|
raise NotSupported(f"{type(self).__name__} does not support notify")
|
||||||
111
src/shard_wiki/adapters/folder.py
Normal file
111
src/shard_wiki/adapters/folder.py
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
"""FolderAdapter — a read-only file-store shard over a directory of Markdown.
|
||||||
|
|
||||||
|
The home-case substrate: a plain folder of ``.md`` files. The relative path (sans extension,
|
||||||
|
``/``-separated) is the stable page **key**; the file is the page **body**; mtime gives a
|
||||||
|
freshness stamp. Read-only in this slice (overlay/write-through come later); declared profile
|
||||||
|
reflects exactly that (read-only, file-store, path addressing, no native history/query).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.adapters.contract import ShardAdapter
|
||||||
|
from shard_wiki.model import (
|
||||||
|
AccessGrant,
|
||||||
|
Addressing,
|
||||||
|
AttachmentMode,
|
||||||
|
CapabilityProfile,
|
||||||
|
ContentOpacity,
|
||||||
|
History,
|
||||||
|
Identity,
|
||||||
|
MergeModel,
|
||||||
|
NativeQuery,
|
||||||
|
NotSupported,
|
||||||
|
OperationalEnvelope,
|
||||||
|
Page,
|
||||||
|
Placement,
|
||||||
|
Substrate,
|
||||||
|
Translation,
|
||||||
|
Verb,
|
||||||
|
WriteGranularity,
|
||||||
|
)
|
||||||
|
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
|
||||||
|
|
||||||
|
__all__ = ["FolderAdapter"]
|
||||||
|
|
||||||
|
|
||||||
|
class FolderAdapter(ShardAdapter):
|
||||||
|
def __init__(self, shard_id: str, root: str | Path, writable: bool = False) -> None:
|
||||||
|
self._shard_id = shard_id
|
||||||
|
self._root = Path(root)
|
||||||
|
self._writable = writable
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
return self._shard_id
|
||||||
|
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
verbs = {Verb.READ, Verb.WRITE} if self._writable else {Verb.READ}
|
||||||
|
granularity = WriteGranularity.PER_PAGE if self._writable else WriteGranularity.NONE
|
||||||
|
return CapabilityProfile(
|
||||||
|
substrate=Substrate.FILES,
|
||||||
|
attachment_mode=AttachmentMode.FILE_STORE,
|
||||||
|
write_granularity=granularity,
|
||||||
|
content_opacity=ContentOpacity.TRANSPARENT,
|
||||||
|
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||||
|
access_grant=AccessGrant.OPEN,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
history=History.NONE,
|
||||||
|
merge_model=MergeModel.NONE,
|
||||||
|
addressing=Addressing.PATH,
|
||||||
|
native_query=NativeQuery.NONE,
|
||||||
|
translation=Translation.NATIVE,
|
||||||
|
supported_verbs=frozenset(verbs),
|
||||||
|
).validate()
|
||||||
|
|
||||||
|
def _path_for(self, key: str) -> Path:
|
||||||
|
return self._root / f"{key}.md"
|
||||||
|
|
||||||
|
def current_rev(self, key: str) -> str | None:
|
||||||
|
"""The shard's current revision token for ``key`` (mtime iso), or ``None`` if absent.
|
||||||
|
Used for apply-under-drift comparison (blueprint §8.6)."""
|
||||||
|
path = self._path_for(key)
|
||||||
|
if not path.is_file():
|
||||||
|
return None
|
||||||
|
return datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc).isoformat()
|
||||||
|
|
||||||
|
def write(self, key: str, body: str) -> Page:
|
||||||
|
if not self._writable:
|
||||||
|
raise NotSupported(f"{type(self).__name__} is read-only")
|
||||||
|
path = self._path_for(key)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(body, encoding="utf-8")
|
||||||
|
return self.read(key)
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
for p in sorted(self._root.rglob("*.md")):
|
||||||
|
yield p.relative_to(self._root).with_suffix("").as_posix()
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
path = self._path_for(key)
|
||||||
|
if not path.is_file():
|
||||||
|
raise KeyError(key)
|
||||||
|
body = path.read_text(encoding="utf-8")
|
||||||
|
mtime = datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc)
|
||||||
|
envelope = ProvenanceEnvelope(
|
||||||
|
source_shard=self._shard_id,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
staleness=Staleness.FRESH,
|
||||||
|
source_rev=mtime.isoformat(),
|
||||||
|
observed_at=datetime.now(tz=timezone.utc),
|
||||||
|
)
|
||||||
|
rel = path.relative_to(self._root).as_posix()
|
||||||
|
return Page(
|
||||||
|
identity=Identity(self._shard_id, key),
|
||||||
|
body=body,
|
||||||
|
envelope=envelope,
|
||||||
|
placements=(Placement(self._shard_id, rel),),
|
||||||
|
)
|
||||||
180
src/shard_wiki/adapters/git.py
Normal file
180
src/shard_wiki/adapters/git.py
Normal file
@@ -0,0 +1,180 @@
|
|||||||
|
"""GitShardAdapter — a second substrate: git-as-store (SHARD-WP-0012; TSD §A.3 git-IS-store).
|
||||||
|
|
||||||
|
The home case where **git is the store *and* the journal**. Tracked ``*.md`` paths are the page
|
||||||
|
keys; the working-tree file is the body; a page's ``source_rev`` is the **commit sha of the last
|
||||||
|
commit touching its path** (per-path, so an edit to one page never drifts another). The declared
|
||||||
|
profile is *git-IS-store ⟹ substrate=git ∧ history=git-native* — the implication rule the
|
||||||
|
capability model enforces (§6.5), validated at registration like any other binding.
|
||||||
|
|
||||||
|
This adapter adds **no core changes**: it implements the same :class:`ShardAdapter` contract the
|
||||||
|
folder adapter does, proving "write an adapter + declare a verified profile" is the whole cost of a
|
||||||
|
new substrate (capability-as-data, I-3). Built on the ``git`` CLI via subprocess — zero new deps.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.adapters.contract import ShardAdapter
|
||||||
|
from shard_wiki.model import (
|
||||||
|
AccessGrant,
|
||||||
|
Addressing,
|
||||||
|
AttachmentMode,
|
||||||
|
CapabilityProfile,
|
||||||
|
ContentOpacity,
|
||||||
|
History,
|
||||||
|
Identity,
|
||||||
|
MergeModel,
|
||||||
|
NativeQuery,
|
||||||
|
NotSupported,
|
||||||
|
OperationalEnvelope,
|
||||||
|
Page,
|
||||||
|
Placement,
|
||||||
|
Substrate,
|
||||||
|
Translation,
|
||||||
|
Verb,
|
||||||
|
WriteGranularity,
|
||||||
|
)
|
||||||
|
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
|
||||||
|
|
||||||
|
__all__ = ["GitShardAdapter", "PageRevision"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class PageRevision:
|
||||||
|
"""One adopted git-native revision of a page: the commit sha and its subject line."""
|
||||||
|
|
||||||
|
sha: str
|
||||||
|
message: str
|
||||||
|
|
||||||
|
_GIT_IDENTITY = {
|
||||||
|
"GIT_AUTHOR_NAME": "shard-wiki",
|
||||||
|
"GIT_AUTHOR_EMAIL": "shard@shard-wiki",
|
||||||
|
"GIT_COMMITTER_NAME": "shard-wiki",
|
||||||
|
"GIT_COMMITTER_EMAIL": "shard@shard-wiki",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class GitShardAdapter(ShardAdapter):
|
||||||
|
"""A shard whose store is a git repo: keys are tracked ``*.md`` paths, revs are commit shas."""
|
||||||
|
|
||||||
|
def __init__(self, shard_id: str, repo_path: str | Path, writable: bool = False) -> None:
|
||||||
|
self._shard_id = shard_id
|
||||||
|
self._repo = Path(repo_path)
|
||||||
|
self._writable = writable
|
||||||
|
self._repo.mkdir(parents=True, exist_ok=True)
|
||||||
|
if not (self._repo / ".git").exists():
|
||||||
|
self._git("init", "--quiet")
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
return self._shard_id
|
||||||
|
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
# VERSION is always available — a git-IS-store has git-native history to adopt (§A.5),
|
||||||
|
# read-only or not. WRITE (= commit, PER_PAGE) is added only in writable mode.
|
||||||
|
verbs = {Verb.READ, Verb.VERSION}
|
||||||
|
granularity = WriteGranularity.NONE
|
||||||
|
if self._writable:
|
||||||
|
verbs |= {Verb.WRITE}
|
||||||
|
granularity = WriteGranularity.PER_PAGE
|
||||||
|
return CapabilityProfile(
|
||||||
|
substrate=Substrate.GIT,
|
||||||
|
attachment_mode=AttachmentMode.GIT_IS_STORE,
|
||||||
|
write_granularity=granularity,
|
||||||
|
content_opacity=ContentOpacity.TRANSPARENT,
|
||||||
|
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||||
|
access_grant=AccessGrant.OPEN,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
history=History.GIT_NATIVE, # git-is-store ⟹ git-native (§6.5)
|
||||||
|
merge_model=MergeModel.GIT_TEXT,
|
||||||
|
addressing=Addressing.PATH,
|
||||||
|
native_query=NativeQuery.NONE,
|
||||||
|
translation=Translation.NATIVE,
|
||||||
|
supported_verbs=frozenset(verbs),
|
||||||
|
).validate()
|
||||||
|
|
||||||
|
def write(self, key: str, body: str) -> Page:
|
||||||
|
"""Write = **commit**: stage the file and commit it (skip a no-op so no empty commit),
|
||||||
|
returning the page at the new sha. Drift detection rides on ``current_rev`` = that sha."""
|
||||||
|
if not self._writable:
|
||||||
|
raise NotSupported(f"{type(self).__name__} is read-only")
|
||||||
|
rel = f"{key}.md"
|
||||||
|
path = self._path_for(key)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(body, encoding="utf-8")
|
||||||
|
self._git("add", "--", rel)
|
||||||
|
if self._run("diff", "--cached", "--quiet").returncode != 0: # staged changes present
|
||||||
|
self._git("commit", "-m", f"write {rel}", env=_GIT_IDENTITY)
|
||||||
|
return self.read(key)
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
out = self._git("ls-files", "*.md").decode()
|
||||||
|
for line in out.splitlines():
|
||||||
|
yield line[: -len(".md")] if line.endswith(".md") else line
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
path = self._path_for(key)
|
||||||
|
if not path.is_file():
|
||||||
|
raise KeyError(key)
|
||||||
|
rev = self.current_rev(key)
|
||||||
|
return Page(
|
||||||
|
identity=Identity(self._shard_id, key),
|
||||||
|
body=path.read_text(encoding="utf-8"),
|
||||||
|
envelope=ProvenanceEnvelope(
|
||||||
|
source_shard=self._shard_id,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
staleness=Staleness.FRESH,
|
||||||
|
source_rev=rev,
|
||||||
|
lineage="git-native",
|
||||||
|
),
|
||||||
|
placements=(Placement(self._shard_id, f"{key}.md"),),
|
||||||
|
)
|
||||||
|
|
||||||
|
def current_rev(self, key: str) -> str | None:
|
||||||
|
"""The sha of the last commit touching ``key``'s path (per-path drift token), or None."""
|
||||||
|
rel = f"{key}.md"
|
||||||
|
if not self._path_for(key).is_file():
|
||||||
|
return None
|
||||||
|
sha = self._git("log", "-1", "--format=%H", "--", rel).decode().strip()
|
||||||
|
return sha or None
|
||||||
|
|
||||||
|
def history(self, key: str) -> tuple[PageRevision, ...]:
|
||||||
|
"""Adopt git-native history (§A.5): the commit list for ``key``'s path, newest-first.
|
||||||
|
|
||||||
|
VERSION-gated; raises ``KeyError`` for an unknown page. Each revision is a commit sha +
|
||||||
|
subject — the native log surfaced through the contract, not re-implemented.
|
||||||
|
"""
|
||||||
|
if not self.profile().supports(Verb.VERSION):
|
||||||
|
raise NotSupported(f"{type(self).__name__} does not support version")
|
||||||
|
if not self._path_for(key).is_file():
|
||||||
|
raise KeyError(key)
|
||||||
|
out = self._git("log", "--format=%H%x00%s", "--", f"{key}.md").decode()
|
||||||
|
revisions = []
|
||||||
|
for line in out.splitlines():
|
||||||
|
sha, _, message = line.partition("\x00")
|
||||||
|
revisions.append(PageRevision(sha=sha, message=message))
|
||||||
|
return tuple(revisions)
|
||||||
|
|
||||||
|
# -- git plumbing --------------------------------------------------------
|
||||||
|
|
||||||
|
def _path_for(self, key: str) -> Path:
|
||||||
|
return self._repo / f"{key}.md"
|
||||||
|
|
||||||
|
def _git(self, *args: str, stdin: bytes | None = None, env: dict | None = None) -> bytes:
|
||||||
|
return self._run(*args, stdin=stdin, env=env, check=True).stdout
|
||||||
|
|
||||||
|
def _run(
|
||||||
|
self, *args: str, stdin: bytes | None = None, env: dict | None = None, check: bool = False
|
||||||
|
) -> subprocess.CompletedProcess:
|
||||||
|
return subprocess.run(
|
||||||
|
["git", "-C", str(self._repo), *args],
|
||||||
|
input=stdin,
|
||||||
|
capture_output=True,
|
||||||
|
env={**os.environ, **(env or {})},
|
||||||
|
check=check,
|
||||||
|
)
|
||||||
58
src/shard_wiki/coordination/__init__.py
Normal file
58
src/shard_wiki/coordination/__init__.py
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
"""coordination/ — the event-sourced decision log (L3, coordination-canonical state)."""
|
||||||
|
|
||||||
|
from shard_wiki.coordination.decision_log import (
|
||||||
|
CoordinationState,
|
||||||
|
DecisionEvent,
|
||||||
|
DecisionLog,
|
||||||
|
EventStore,
|
||||||
|
EventType,
|
||||||
|
InMemoryEventStore,
|
||||||
|
deserialize_event,
|
||||||
|
serialize_event,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.append_authority import (
|
||||||
|
AppendAuthority,
|
||||||
|
Lease,
|
||||||
|
LeaseHeld,
|
||||||
|
LeaseRegistry,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||||
|
from shard_wiki.coordination.migration import (
|
||||||
|
export_jsonl,
|
||||||
|
import_jsonl,
|
||||||
|
import_log,
|
||||||
|
migrate_space,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.overlay import (
|
||||||
|
ApplyResult,
|
||||||
|
ApplyStatus,
|
||||||
|
Overlay,
|
||||||
|
OverlayEngine,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.patch import Patch, render_patch
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"DecisionLog",
|
||||||
|
"DecisionEvent",
|
||||||
|
"EventType",
|
||||||
|
"CoordinationState",
|
||||||
|
"EventStore",
|
||||||
|
"InMemoryEventStore",
|
||||||
|
"GitEventStore",
|
||||||
|
"Lease",
|
||||||
|
"LeaseHeld",
|
||||||
|
"LeaseRegistry",
|
||||||
|
"AppendAuthority",
|
||||||
|
"import_log",
|
||||||
|
"migrate_space",
|
||||||
|
"export_jsonl",
|
||||||
|
"import_jsonl",
|
||||||
|
"serialize_event",
|
||||||
|
"deserialize_event",
|
||||||
|
"Overlay",
|
||||||
|
"OverlayEngine",
|
||||||
|
"ApplyStatus",
|
||||||
|
"ApplyResult",
|
||||||
|
"Patch",
|
||||||
|
"render_patch",
|
||||||
|
]
|
||||||
158
src/shard_wiki/coordination/append_authority.py
Normal file
158
src/shard_wiki/coordination/append_authority.py
Normal file
@@ -0,0 +1,158 @@
|
|||||||
|
"""Per-space append authority — the single-writer lease over the log (SHARD-WP-0009 T2).
|
||||||
|
|
||||||
|
The log is a *total order per space* (§8.6). :class:`~shard_wiki.coordination.git_event_store`
|
||||||
|
makes a fork physically impossible via compare-and-swap; this layer adds the **policy** that gives
|
||||||
|
the order a single designated writer: a **per-space lease**. At most one node holds a space's lease
|
||||||
|
at a time; only the holder writes to the store. A non-holder does not write — it **forwards** its
|
||||||
|
append intent to the current holder, so intents from anywhere still land in one serialized stream.
|
||||||
|
|
||||||
|
The lease is **time-bounded and re-grantable** (HA): if a holder dies, its lease expires and a new
|
||||||
|
node may take it, resuming appends from the log head (``seq`` stays contiguous across the hand-off).
|
||||||
|
A node holding a *stale* lease (already re-granted elsewhere) cannot write either — it discovers it
|
||||||
|
is no longer the holder and forwards instead, so a partitioned ex-holder can never fork the log.
|
||||||
|
|
||||||
|
Mechanism over policy (CLAUDE.md): this provides the leasing *primitive*; who acquires when, and
|
||||||
|
the TTL, are the caller's policy. Single-coordinator only — distributed multi-node leasing and log
|
||||||
|
sharding are explicit non-goals of this workplan.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from collections.abc import Callable, Mapping
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from shard_wiki.coordination.decision_log import DecisionEvent, EventStore, EventType
|
||||||
|
|
||||||
|
__all__ = ["Lease", "LeaseHeld", "LeaseRegistry", "AppendAuthority"]
|
||||||
|
|
||||||
|
|
||||||
|
def _utcnow() -> datetime:
|
||||||
|
return datetime.now(tz=timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Lease:
|
||||||
|
"""A time-bounded grant of single-writer authority over one space."""
|
||||||
|
|
||||||
|
space: str
|
||||||
|
holder: str
|
||||||
|
token: str
|
||||||
|
expires_at: datetime
|
||||||
|
|
||||||
|
def valid_at(self, now: datetime) -> bool:
|
||||||
|
return now < self.expires_at
|
||||||
|
|
||||||
|
|
||||||
|
class LeaseHeld(Exception):
|
||||||
|
"""Raised when a space's lease is validly held by a different node."""
|
||||||
|
|
||||||
|
def __init__(self, lease: Lease) -> None:
|
||||||
|
super().__init__(
|
||||||
|
f"space {lease.space!r} leased to {lease.holder!r} until {lease.expires_at}"
|
||||||
|
)
|
||||||
|
self.lease = lease
|
||||||
|
|
||||||
|
|
||||||
|
class LeaseRegistry:
|
||||||
|
"""The single coordinator's grant table: at most one *valid* lease per space.
|
||||||
|
|
||||||
|
A lease that has expired is freely re-grantable to any node (the HA replacement path); a still
|
||||||
|
valid lease is exclusive to its holder (renewable by that holder). The registry also routes
|
||||||
|
forwarded append intents to the current holder node.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, clock: Callable[[], datetime] = _utcnow) -> None:
|
||||||
|
self._clock = clock
|
||||||
|
self._leases: dict[str, Lease] = {}
|
||||||
|
self._nodes: dict[str, AppendAuthority] = {}
|
||||||
|
|
||||||
|
def register(self, node: AppendAuthority) -> None:
|
||||||
|
self._nodes[node.node_id] = node
|
||||||
|
|
||||||
|
def grant(self, space: str, holder: str, ttl_seconds: float) -> Lease:
|
||||||
|
"""Grant/renew the lease for ``space`` to ``holder``; raise :class:`LeaseHeld` if another
|
||||||
|
node still holds it validly. An expired lease is re-grantable to anyone."""
|
||||||
|
now = self._clock()
|
||||||
|
current = self._leases.get(space)
|
||||||
|
if current is not None and current.valid_at(now) and current.holder != holder:
|
||||||
|
raise LeaseHeld(current)
|
||||||
|
lease = Lease(
|
||||||
|
space=space,
|
||||||
|
holder=holder,
|
||||||
|
token=uuid.uuid4().hex,
|
||||||
|
expires_at=now + timedelta(seconds=ttl_seconds),
|
||||||
|
)
|
||||||
|
self._leases[space] = lease
|
||||||
|
return lease
|
||||||
|
|
||||||
|
def current(self, space: str) -> Lease | None:
|
||||||
|
"""The lease for ``space`` if one is currently valid, else None (expired/absent)."""
|
||||||
|
lease = self._leases.get(space)
|
||||||
|
return lease if lease is not None and lease.valid_at(self._clock()) else None
|
||||||
|
|
||||||
|
def holder_node(self, space: str) -> AppendAuthority | None:
|
||||||
|
lease = self.current(space)
|
||||||
|
return self._nodes.get(lease.holder) if lease is not None else None
|
||||||
|
|
||||||
|
|
||||||
|
class AppendAuthority:
|
||||||
|
"""A coordinator node that appends to the shared log only when it holds the space's lease.
|
||||||
|
|
||||||
|
Nodes share one :class:`EventStore` and one :class:`LeaseRegistry`. ``append`` routes itself:
|
||||||
|
the holder writes; a non-holder forwards to whoever holds the lease (acquiring it first if the
|
||||||
|
space is currently unleased). The append API mirrors :class:`EventStore` so the authority is a
|
||||||
|
drop-in single-writer guard.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
node_id: str,
|
||||||
|
store: EventStore,
|
||||||
|
registry: LeaseRegistry,
|
||||||
|
ttl_seconds: float = 30.0,
|
||||||
|
) -> None:
|
||||||
|
self.node_id = node_id
|
||||||
|
self._store = store
|
||||||
|
self._registry = registry
|
||||||
|
self._ttl = ttl_seconds
|
||||||
|
registry.register(self)
|
||||||
|
|
||||||
|
def acquire(self, space: str) -> Lease:
|
||||||
|
"""Take (or renew) the lease for ``space``. Raises :class:`LeaseHeld` if another node holds
|
||||||
|
it validly."""
|
||||||
|
return self._registry.grant(space, self.node_id, self._ttl)
|
||||||
|
|
||||||
|
def holds(self, space: str) -> bool:
|
||||||
|
lease = self._registry.current(space)
|
||||||
|
return lease is not None and lease.holder == self.node_id
|
||||||
|
|
||||||
|
def append(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
type: EventType,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> DecisionEvent:
|
||||||
|
"""Append via the single authority. If we hold the lease, write; otherwise forward to the
|
||||||
|
holder. If the space is unleased, acquire it first. A node with a *stale* lease forwards
|
||||||
|
(it is not the current holder) rather than writing — so it cannot fork the log."""
|
||||||
|
holder_node = self._registry.holder_node(space)
|
||||||
|
if holder_node is None:
|
||||||
|
self.acquire(space) # unleased: take authority, then write below
|
||||||
|
holder_node = self
|
||||||
|
if holder_node is self:
|
||||||
|
return self._store.append(space, type, payload, actor=actor)
|
||||||
|
return holder_node._write(space, type, payload, actor=actor)
|
||||||
|
|
||||||
|
def _write(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
type: EventType,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
actor: str | None,
|
||||||
|
) -> DecisionEvent:
|
||||||
|
"""Apply a forwarded intent. Called only on the lease holder by a forwarding peer."""
|
||||||
|
return self._store.append(space, type, payload, actor=actor)
|
||||||
208
src/shard_wiki/coordination/decision_log.py
Normal file
208
src/shard_wiki/coordination/decision_log.py
Normal file
@@ -0,0 +1,208 @@
|
|||||||
|
"""The event-sourced coordination decision log — the keystone (CoreArchitectureBlueprint §8.1).
|
||||||
|
|
||||||
|
Coordination-canonical state (overlays, equivalence bindings, aliases, merges, forks) is an
|
||||||
|
**append-only decision log**, not a mutable file; the queryable *current* state is a **derived
|
||||||
|
fold** of the log (tier-3 disposable). The log is **totally ordered per space** via a single
|
||||||
|
**append authority**. That total order is what gives read-your-writes across readers (§8.6).
|
||||||
|
|
||||||
|
Storage lives behind :class:`EventStore`: :class:`InMemoryEventStore` is the default test double
|
||||||
|
(an in-process counter); :class:`~shard_wiki.coordination.git_event_store.GitEventStore` is the
|
||||||
|
git-addressable backend (SHARD-WP-0009). The :class:`DecisionLog` API and the :meth:`fold` are
|
||||||
|
identical across backends — only storage + the concurrency model differ.
|
||||||
|
|
||||||
|
`derived = f(canonical)`: :class:`CoordinationState` is always reproducible by replaying the log.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from enum import Enum
|
||||||
|
from types import MappingProxyType
|
||||||
|
from typing import Any, Protocol, runtime_checkable
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"EventType",
|
||||||
|
"DecisionEvent",
|
||||||
|
"CoordinationState",
|
||||||
|
"EventStore",
|
||||||
|
"InMemoryEventStore",
|
||||||
|
"DecisionLog",
|
||||||
|
"serialize_event",
|
||||||
|
"deserialize_event",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class EventType(Enum):
|
||||||
|
OVERLAY_CREATED = "overlay-created"
|
||||||
|
BINDING_MADE = "binding-made"
|
||||||
|
ALIAS_SET = "alias-set"
|
||||||
|
MERGE_DECIDED = "merge-decided"
|
||||||
|
PAGE_FORKED = "page-forked"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class DecisionEvent:
|
||||||
|
"""One immutable, ordered decision. ``seq`` is the per-space total order."""
|
||||||
|
|
||||||
|
seq: int
|
||||||
|
space: str
|
||||||
|
type: EventType
|
||||||
|
payload: Mapping[str, Any]
|
||||||
|
actor: str | None = None
|
||||||
|
timestamp: datetime = field(default_factory=lambda: datetime.now(tz=timezone.utc))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class CoordinationState:
|
||||||
|
"""The derived fold of a space's log: current aliases + equivalence groups + open overlays.
|
||||||
|
|
||||||
|
Disposable (tier-3): always recomputable from the log via :meth:`DecisionLog.fold`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
aliases: Mapping[str, str]
|
||||||
|
equivalence_groups: tuple[frozenset[str], ...]
|
||||||
|
open_overlays: Mapping[str, Mapping[str, Any]]
|
||||||
|
|
||||||
|
def resolve_alias(self, name: str) -> str | None:
|
||||||
|
return self.aliases.get(name)
|
||||||
|
|
||||||
|
def equivalent_to(self, identity: str) -> frozenset[str]:
|
||||||
|
"""All identities equivalent to ``identity`` (including itself if bound), else just it."""
|
||||||
|
for group in self.equivalence_groups:
|
||||||
|
if identity in group:
|
||||||
|
return group
|
||||||
|
return frozenset({identity})
|
||||||
|
|
||||||
|
|
||||||
|
def serialize_event(event: DecisionEvent) -> bytes:
|
||||||
|
"""Deterministic, stable-JSON wire form of an event (same bytes for equal events, any process).
|
||||||
|
|
||||||
|
Sorted keys + compact separators make the serialization canonical, so a git object hashed from
|
||||||
|
it is reproducible — the basis for content-addressable, comparable logs across backends.
|
||||||
|
"""
|
||||||
|
obj = {
|
||||||
|
"seq": event.seq,
|
||||||
|
"space": event.space,
|
||||||
|
"type": event.type.value,
|
||||||
|
"payload": event.payload,
|
||||||
|
"actor": event.actor,
|
||||||
|
"timestamp": event.timestamp.isoformat(),
|
||||||
|
}
|
||||||
|
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
|
||||||
|
|
||||||
|
|
||||||
|
def deserialize_event(data: bytes | str) -> DecisionEvent:
|
||||||
|
"""Inverse of :func:`serialize_event` — round-trips an event byte-for-byte by field."""
|
||||||
|
obj = json.loads(data)
|
||||||
|
return DecisionEvent(
|
||||||
|
seq=obj["seq"],
|
||||||
|
space=obj["space"],
|
||||||
|
type=EventType(obj["type"]),
|
||||||
|
payload=obj["payload"],
|
||||||
|
actor=obj["actor"],
|
||||||
|
timestamp=datetime.fromisoformat(obj["timestamp"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@runtime_checkable
|
||||||
|
class EventStore(Protocol):
|
||||||
|
"""Append-only, per-space ordered storage behind :class:`DecisionLog`.
|
||||||
|
|
||||||
|
Two bindings exist: :class:`InMemoryEventStore` (default/test double) and
|
||||||
|
:class:`~shard_wiki.coordination.git_event_store.GitEventStore` (git-addressable). Both assign
|
||||||
|
a per-space monotonic ``seq`` at the log head and guarantee read-your-writes for their reach
|
||||||
|
(in-process for memory; cross-process for git).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def append(
|
||||||
|
self, space: str, type: EventType, payload: Mapping[str, Any], actor: str | None = None
|
||||||
|
) -> DecisionEvent: ...
|
||||||
|
|
||||||
|
def events(self, space: str) -> tuple[DecisionEvent, ...]: ...
|
||||||
|
|
||||||
|
|
||||||
|
class InMemoryEventStore:
|
||||||
|
"""In-process append-only store, totally ordered per space (the append authority for a process).
|
||||||
|
|
||||||
|
The default test double; the git backend preserves this exact contract on durable storage.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._events: dict[str, list[DecisionEvent]] = {}
|
||||||
|
|
||||||
|
def append(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
type: EventType,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> DecisionEvent:
|
||||||
|
seq = len(self._events.get(space, ()))
|
||||||
|
event = DecisionEvent(seq=seq, space=space, type=type, payload=dict(payload), actor=actor)
|
||||||
|
self._events.setdefault(space, []).append(event)
|
||||||
|
return event
|
||||||
|
|
||||||
|
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||||
|
return tuple(self._events.get(space, ()))
|
||||||
|
|
||||||
|
|
||||||
|
class DecisionLog:
|
||||||
|
"""Append-only decision log, totally ordered per space, with a derived :meth:`fold`.
|
||||||
|
|
||||||
|
Storage is delegated to an :class:`EventStore` (default :class:`InMemoryEventStore`); swapping
|
||||||
|
in the git backend changes only durability + the concurrency model, not this API or the fold.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, store: EventStore | None = None) -> None:
|
||||||
|
self._store: EventStore = store if store is not None else InMemoryEventStore()
|
||||||
|
|
||||||
|
def append(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
type: EventType,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> DecisionEvent:
|
||||||
|
return self._store.append(space, type, payload, actor=actor)
|
||||||
|
|
||||||
|
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||||
|
"""The space's events in append (total) order. Read-your-writes: a just-appended event
|
||||||
|
is present immediately."""
|
||||||
|
return self._store.events(space)
|
||||||
|
|
||||||
|
def fold(self, space: str) -> CoordinationState:
|
||||||
|
"""Replay the log into current coordination state (derived = f(log))."""
|
||||||
|
aliases: dict[str, str] = {}
|
||||||
|
overlays: dict[str, dict[str, Any]] = {}
|
||||||
|
groups: list[set[str]] = []
|
||||||
|
|
||||||
|
for event in self.events(space):
|
||||||
|
if event.type is EventType.ALIAS_SET:
|
||||||
|
aliases[event.payload["alias"]] = event.payload["target"]
|
||||||
|
elif event.type is EventType.BINDING_MADE:
|
||||||
|
_merge_group(groups, {str(m) for m in event.payload["members"]})
|
||||||
|
elif event.type is EventType.OVERLAY_CREATED:
|
||||||
|
overlays[event.payload["overlay_id"]] = dict(event.payload)
|
||||||
|
elif event.type is EventType.MERGE_DECIDED:
|
||||||
|
# A merge resolution may collapse an overlay; minimal handling for the slice.
|
||||||
|
overlays.pop(event.payload.get("overlay_id", ""), None)
|
||||||
|
elif event.type is EventType.PAGE_FORKED:
|
||||||
|
_merge_group(groups, {str(event.payload["source"]), str(event.payload["fork"])})
|
||||||
|
|
||||||
|
return CoordinationState(
|
||||||
|
aliases=MappingProxyType(dict(aliases)),
|
||||||
|
equivalence_groups=tuple(frozenset(g) for g in groups),
|
||||||
|
open_overlays=MappingProxyType({k: MappingProxyType(v) for k, v in overlays.items()}),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _merge_group(groups: list[set[str]], members: set[str]) -> None:
|
||||||
|
"""Union-merge ``members`` into ``groups`` (any existing group sharing a member absorbs it)."""
|
||||||
|
touching = [g for g in groups if g & members]
|
||||||
|
for g in touching:
|
||||||
|
groups.remove(g)
|
||||||
|
members |= g
|
||||||
|
groups.append(members)
|
||||||
172
src/shard_wiki/coordination/git_event_store.py
Normal file
172
src/shard_wiki/coordination/git_event_store.py
Normal file
@@ -0,0 +1,172 @@
|
|||||||
|
"""GitEventStore — a git-addressable binding of :class:`EventStore` (SHARD-WP-0009 T1).
|
||||||
|
|
||||||
|
Each space is a ref (``refs/spaces/<sha1(space)>``); each ``append`` writes the event as an
|
||||||
|
immutable git object (a one-blob tree committed onto the ref) and advances the ref. The commit
|
||||||
|
chain *is* the totally ordered log: ``seq`` is the depth, ``events`` walks first-parent from the
|
||||||
|
head oldest→newest. Coordination-canonical state therefore inherits git's history / patch /
|
||||||
|
review / backup affordances (I-6) and is read-your-writes correct across processes.
|
||||||
|
|
||||||
|
The total order is enforced at storage by a **compare-and-swap** ref update
|
||||||
|
(``git update-ref <ref> <new> <old>``): two appenders racing off the same head — the loser's CAS
|
||||||
|
fails and it retries off the new head, so a non-holder can never fork the log. The lease layer
|
||||||
|
(T2) sits *above* this as the append-authority policy; CAS is the mechanism that makes it safe.
|
||||||
|
|
||||||
|
Implemented over the ``git`` CLI through :mod:`subprocess` — zero runtime dependencies.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from shard_wiki.coordination.decision_log import (
|
||||||
|
DecisionEvent,
|
||||||
|
EventType,
|
||||||
|
deserialize_event,
|
||||||
|
serialize_event,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = ["GitEventStore"]
|
||||||
|
|
||||||
|
# Fixed identity so commit objects are reproducible and never prompt for git config; the event's
|
||||||
|
# own timestamp/actor carry the real provenance, the commit is just the ordered container.
|
||||||
|
_GIT_IDENTITY = {
|
||||||
|
"GIT_AUTHOR_NAME": "shard-wiki",
|
||||||
|
"GIT_AUTHOR_EMAIL": "coordination@shard-wiki",
|
||||||
|
"GIT_COMMITTER_NAME": "shard-wiki",
|
||||||
|
"GIT_COMMITTER_EMAIL": "coordination@shard-wiki",
|
||||||
|
}
|
||||||
|
_EVENT_PATH = "event.json"
|
||||||
|
_MAX_CAS_RETRIES = 50
|
||||||
|
|
||||||
|
|
||||||
|
class GitEventStore:
|
||||||
|
"""Git-backed, append-only, per-space ordered event store (an :class:`EventStore`)."""
|
||||||
|
|
||||||
|
def __init__(self, repo_path: str | Path) -> None:
|
||||||
|
self.repo_path = Path(repo_path)
|
||||||
|
self.repo_path.mkdir(parents=True, exist_ok=True)
|
||||||
|
if not (self.repo_path / "HEAD").exists() and not (self.repo_path / ".git").exists():
|
||||||
|
self._git("init", "--quiet", str(self.repo_path), at_cwd=True)
|
||||||
|
|
||||||
|
# -- EventStore contract -------------------------------------------------
|
||||||
|
|
||||||
|
def append(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
type: EventType,
|
||||||
|
payload: Mapping[str, Any],
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> DecisionEvent:
|
||||||
|
"""Append one event, advancing the space ref under compare-and-swap (retry-on-race)."""
|
||||||
|
ref = self._ref(space)
|
||||||
|
for _ in range(_MAX_CAS_RETRIES):
|
||||||
|
head = self._head(ref)
|
||||||
|
seq = self._count(ref, head)
|
||||||
|
event = DecisionEvent(
|
||||||
|
seq=seq, space=space, type=type, payload=dict(payload), actor=actor
|
||||||
|
)
|
||||||
|
commit = self._commit_event(event, parent=head)
|
||||||
|
if self._cas_update(ref, new=commit, old=head):
|
||||||
|
return event
|
||||||
|
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
|
||||||
|
|
||||||
|
def import_event(self, event: DecisionEvent) -> None:
|
||||||
|
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
|
||||||
|
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
|
||||||
|
|
||||||
|
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
|
||||||
|
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
|
||||||
|
"""
|
||||||
|
ref = self._ref(event.space)
|
||||||
|
head = self._head(ref)
|
||||||
|
expected = self._count(ref, head)
|
||||||
|
if event.seq != expected:
|
||||||
|
raise ValueError(
|
||||||
|
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
|
||||||
|
)
|
||||||
|
commit = self._commit_event(event, parent=head)
|
||||||
|
if not self._cas_update(ref, new=commit, old=head):
|
||||||
|
raise RuntimeError(f"import race on {ref}")
|
||||||
|
|
||||||
|
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||||
|
"""The space's events oldest→newest (append/total order)."""
|
||||||
|
ref = self._ref(space)
|
||||||
|
head = self._head(ref)
|
||||||
|
if head is None:
|
||||||
|
return ()
|
||||||
|
shas = self._git("rev-list", "--reverse", "--first-parent", ref).decode().split()
|
||||||
|
return tuple(
|
||||||
|
deserialize_event(self._git("cat-file", "blob", f"{sha}:{_EVENT_PATH}"))
|
||||||
|
for sha in shas
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- git plumbing --------------------------------------------------------
|
||||||
|
|
||||||
|
def _commit_event(self, event: DecisionEvent, parent: str | None) -> str:
|
||||||
|
blob = self._git(
|
||||||
|
"hash-object", "-w", "--stdin", stdin=serialize_event(event)
|
||||||
|
).decode().strip()
|
||||||
|
tree = self._git(
|
||||||
|
"mktree", stdin=f"100644 blob {blob}\t{_EVENT_PATH}\n".encode()
|
||||||
|
).decode().strip()
|
||||||
|
args = ["commit-tree", tree, "-m", f"event {event.seq} {event.type.value}"]
|
||||||
|
if parent is not None:
|
||||||
|
args += ["-p", parent]
|
||||||
|
# Pin the commit date to the event's timestamp for reproducible objects.
|
||||||
|
date = event.timestamp.isoformat()
|
||||||
|
env = {**_GIT_IDENTITY, "GIT_AUTHOR_DATE": date, "GIT_COMMITTER_DATE": date}
|
||||||
|
return self._git(*args, env=env).decode().strip()
|
||||||
|
|
||||||
|
def _cas_update(self, ref: str, new: str, old: str | None) -> bool:
|
||||||
|
"""``git update-ref`` with the old value as a CAS guard (empty oldvalue == must-not-exist).
|
||||||
|
|
||||||
|
Returns False if the ref moved since we read ``old`` (lost the race) — the caller retries.
|
||||||
|
"""
|
||||||
|
result = self._run("update-ref", ref, new, old if old is not None else "")
|
||||||
|
return result.returncode == 0
|
||||||
|
|
||||||
|
def _head(self, ref: str) -> str | None:
|
||||||
|
result = self._run("rev-parse", "--verify", "--quiet", ref)
|
||||||
|
out = result.stdout.decode().strip()
|
||||||
|
return out or None
|
||||||
|
|
||||||
|
def _count(self, ref: str, head: str | None) -> int:
|
||||||
|
if head is None:
|
||||||
|
return 0
|
||||||
|
return int(self._git("rev-list", "--count", "--first-parent", ref).decode().strip())
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _ref(space: str) -> str:
|
||||||
|
return f"refs/spaces/{hashlib.sha1(space.encode()).hexdigest()}"
|
||||||
|
|
||||||
|
def _git(
|
||||||
|
self,
|
||||||
|
*args: str,
|
||||||
|
stdin: bytes | None = None,
|
||||||
|
env: dict | None = None,
|
||||||
|
at_cwd: bool = False,
|
||||||
|
) -> bytes:
|
||||||
|
result = self._run(*args, stdin=stdin, env=env, at_cwd=at_cwd, check=True)
|
||||||
|
return result.stdout
|
||||||
|
|
||||||
|
def _run(
|
||||||
|
self,
|
||||||
|
*args: str,
|
||||||
|
stdin: bytes | None = None,
|
||||||
|
env: dict | None = None,
|
||||||
|
at_cwd: bool = False,
|
||||||
|
check: bool = False,
|
||||||
|
) -> subprocess.CompletedProcess:
|
||||||
|
base = ["git"] if at_cwd else ["git", "-C", str(self.repo_path)]
|
||||||
|
return subprocess.run(
|
||||||
|
[*base, *args],
|
||||||
|
input=stdin,
|
||||||
|
capture_output=True,
|
||||||
|
env={**os.environ, **(env or {})},
|
||||||
|
check=check,
|
||||||
|
)
|
||||||
53
src/shard_wiki/coordination/migration.py
Normal file
53
src/shard_wiki/coordination/migration.py
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
|
||||||
|
|
||||||
|
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
|
||||||
|
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
|
||||||
|
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
|
||||||
|
reproduces identical coordination state; only durability changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.coordination.decision_log import (
|
||||||
|
DecisionEvent,
|
||||||
|
EventStore,
|
||||||
|
deserialize_event,
|
||||||
|
serialize_event,
|
||||||
|
)
|
||||||
|
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||||
|
|
||||||
|
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
|
||||||
|
|
||||||
|
|
||||||
|
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
|
||||||
|
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
|
||||||
|
count = 0
|
||||||
|
for event in events:
|
||||||
|
dest.import_event(event)
|
||||||
|
count += 1
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
|
||||||
|
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
|
||||||
|
return import_log(source.events(space), dest)
|
||||||
|
|
||||||
|
|
||||||
|
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
|
||||||
|
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
|
||||||
|
count = 0
|
||||||
|
with open(path, "wb") as handle:
|
||||||
|
for event in events:
|
||||||
|
handle.write(serialize_event(event) + b"\n")
|
||||||
|
count += 1
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
|
||||||
|
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
|
||||||
|
with open(path, "rb") as handle:
|
||||||
|
events = [deserialize_event(line) for line in handle if line.strip()]
|
||||||
|
return import_log(events, dest)
|
||||||
134
src/shard_wiki/coordination/overlay.py
Normal file
134
src/shard_wiki/coordination/overlay.py
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
"""Overlay engine — overlay-before-mutation (FederationRequirements ADR-05, blueprint §8.2).
|
||||||
|
|
||||||
|
An overlay is a non-destructive local edit against a page. It is **coordination-canonical**: an
|
||||||
|
``OVERLAY_CREATED`` event in the decision log (§8.1), not a mutable side file. The current set
|
||||||
|
of open overlays is the log fold. ``draft`` records one; ``apply`` (T4) resolves it under drift.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from shard_wiki.adapters import ShardAdapter
|
||||||
|
from shard_wiki.coordination.decision_log import DecisionLog, EventType
|
||||||
|
from shard_wiki.model import Identity, Page, Verb
|
||||||
|
from shard_wiki.provenance import OverlayState
|
||||||
|
|
||||||
|
__all__ = ["Overlay", "OverlayEngine", "ApplyStatus", "ApplyResult"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Overlay:
|
||||||
|
"""A non-destructive edit: the proposed ``body`` for ``target``, authored against
|
||||||
|
``base_rev`` (the shard revision seen at draft time, for drift detection)."""
|
||||||
|
|
||||||
|
overlay_id: str
|
||||||
|
target: Identity
|
||||||
|
base_rev: str | None
|
||||||
|
body: str
|
||||||
|
state: OverlayState = OverlayState.DRAFT
|
||||||
|
|
||||||
|
def to_payload(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"overlay_id": self.overlay_id,
|
||||||
|
"target_shard": self.target.shard,
|
||||||
|
"target_key": self.target.key,
|
||||||
|
"base_rev": self.base_rev,
|
||||||
|
"body": self.body,
|
||||||
|
"state": self.state.value,
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_payload(cls, payload: Mapping[str, Any]) -> Overlay:
|
||||||
|
return cls(
|
||||||
|
overlay_id=payload["overlay_id"],
|
||||||
|
target=Identity(payload["target_shard"], payload["target_key"]),
|
||||||
|
base_rev=payload["base_rev"],
|
||||||
|
body=payload["body"],
|
||||||
|
state=OverlayState(payload.get("state", OverlayState.DRAFT.value)),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class ApplyStatus(Enum):
|
||||||
|
APPLIED = "applied" # fast-forwarded and written through
|
||||||
|
REFUSED_DRIFT = "refused-drift" # source moved under the overlay; no clobber
|
||||||
|
KEPT_DRAFT = "kept-draft" # target read-only; overlay remains the local truth
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ApplyResult:
|
||||||
|
status: ApplyStatus
|
||||||
|
overlay_id: str
|
||||||
|
page: Page | None = None
|
||||||
|
detail: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
class OverlayEngine:
|
||||||
|
def __init__(self, space: str, log: DecisionLog) -> None:
|
||||||
|
self.space = space
|
||||||
|
self.log = log
|
||||||
|
|
||||||
|
def draft(
|
||||||
|
self,
|
||||||
|
target: Identity,
|
||||||
|
body: str,
|
||||||
|
base_rev: str | None,
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> Overlay:
|
||||||
|
"""Create a draft overlay and record it in the decision log (coordination-canonical)."""
|
||||||
|
overlay = Overlay(uuid.uuid4().hex, target, base_rev, body)
|
||||||
|
self.log.append(self.space, EventType.OVERLAY_CREATED, overlay.to_payload(), actor=actor)
|
||||||
|
return overlay
|
||||||
|
|
||||||
|
def get(self, overlay_id: str) -> Overlay | None:
|
||||||
|
payload = self.log.fold(self.space).open_overlays.get(overlay_id)
|
||||||
|
return Overlay.from_payload(payload) if payload is not None else None
|
||||||
|
|
||||||
|
def open_overlays(self) -> tuple[Overlay, ...]:
|
||||||
|
state = self.log.fold(self.space)
|
||||||
|
return tuple(Overlay.from_payload(p) for p in state.open_overlays.values())
|
||||||
|
|
||||||
|
def apply(self, overlay_id: str, adapter: ShardAdapter) -> ApplyResult:
|
||||||
|
"""Resolve an overlay against its target shard with apply-under-drift semantics (§8.6).
|
||||||
|
|
||||||
|
Read-only target → ``KEPT_DRAFT`` (the overlay stays the local truth). Otherwise compare
|
||||||
|
the overlay's ``base_rev`` to the shard's current rev: equal → fast-forward write-through
|
||||||
|
(``APPLIED``); changed → ``REFUSED_DRIFT`` (never clobber, I-5). Applying records a
|
||||||
|
``MERGE_DECIDED`` event that closes the overlay in the fold.
|
||||||
|
"""
|
||||||
|
overlay = self.get(overlay_id)
|
||||||
|
if overlay is None:
|
||||||
|
raise KeyError(overlay_id)
|
||||||
|
if adapter.shard_id != overlay.target.shard:
|
||||||
|
raise ValueError(f"adapter {adapter.shard_id!r} != target {overlay.target.shard!r}")
|
||||||
|
|
||||||
|
if not adapter.profile().supports(Verb.WRITE):
|
||||||
|
return ApplyResult(
|
||||||
|
ApplyStatus.KEPT_DRAFT, overlay_id, detail="target is read-only; overlay retained"
|
||||||
|
)
|
||||||
|
|
||||||
|
current = _current_rev(adapter, overlay.target.key)
|
||||||
|
if current != overlay.base_rev:
|
||||||
|
return ApplyResult(
|
||||||
|
ApplyStatus.REFUSED_DRIFT,
|
||||||
|
overlay_id,
|
||||||
|
detail=f"base_rev {overlay.base_rev!r} != current {current!r}",
|
||||||
|
)
|
||||||
|
|
||||||
|
page = adapter.write(overlay.target.key, overlay.body)
|
||||||
|
self.log.append(
|
||||||
|
self.space,
|
||||||
|
EventType.MERGE_DECIDED,
|
||||||
|
{"overlay_id": overlay_id, "outcome": "applied"},
|
||||||
|
)
|
||||||
|
return ApplyResult(ApplyStatus.APPLIED, overlay_id, page=page)
|
||||||
|
|
||||||
|
|
||||||
|
def _current_rev(adapter: ShardAdapter, key: str) -> str | None:
|
||||||
|
"""Best-effort current-revision probe; adapters without one are treated as no-rev."""
|
||||||
|
probe = getattr(adapter, "current_rev", None)
|
||||||
|
return probe(key) if callable(probe) else None
|
||||||
38
src/shard_wiki/coordination/patch.py
Normal file
38
src/shard_wiki/coordination/patch.py
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
"""Patch rendering — an overlay as a reviewable diff (FederationRequirements ADR-05).
|
||||||
|
|
||||||
|
A pure function over (base body, overlay body): the auditable change proposal that an overlay
|
||||||
|
becomes before it is applied. Markdown/native text in this slice (lossless); a lossy
|
||||||
|
native-syntax-with-fidelity-report variant is later (TSD §A.6).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import difflib
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.coordination.overlay import Overlay
|
||||||
|
from shard_wiki.model import Identity
|
||||||
|
|
||||||
|
__all__ = ["Patch", "render_patch"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Patch:
|
||||||
|
target: Identity
|
||||||
|
diff: str
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_empty(self) -> bool:
|
||||||
|
return self.diff == ""
|
||||||
|
|
||||||
|
|
||||||
|
def render_patch(overlay: Overlay, base_body: str) -> Patch:
|
||||||
|
"""Render ``base_body`` → ``overlay.body`` as a unified diff against the overlay target."""
|
||||||
|
label = str(overlay.target)
|
||||||
|
lines = difflib.unified_diff(
|
||||||
|
base_body.splitlines(keepends=True),
|
||||||
|
overlay.body.splitlines(keepends=True),
|
||||||
|
fromfile=f"a/{label}",
|
||||||
|
tofile=f"b/{label}",
|
||||||
|
)
|
||||||
|
return Patch(target=overlay.target, diff="".join(lines))
|
||||||
49
src/shard_wiki/engine/__init__.py
Normal file
49
src/shard_wiki/engine/__init__.py
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
"""engine/ — shard-wiki's native, headless wiki engine (a canonical-mode shard backend).
|
||||||
|
|
||||||
|
A small page-store kernel + a typed-extension runtime (WikiEngineCoreArchitecture). The engine
|
||||||
|
is *one shard*: it is consumed by the orchestrator only via its `EngineShardAdapter`; it never
|
||||||
|
imports the derived tier (`union`/`projection`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.engine.activation import (
|
||||||
|
ActivationContext,
|
||||||
|
ActivationProvider,
|
||||||
|
ActivationResolver,
|
||||||
|
StaticProvider,
|
||||||
|
feature_control_provider,
|
||||||
|
)
|
||||||
|
from shard_wiki.engine.extension import (
|
||||||
|
ActiveExtensions,
|
||||||
|
Extension,
|
||||||
|
ExtensionError,
|
||||||
|
ExtensionRuntime,
|
||||||
|
Hook,
|
||||||
|
)
|
||||||
|
from shard_wiki.engine.adapter import EngineShardAdapter, build_engine_shard
|
||||||
|
from shard_wiki.engine.kernel import EngineKernel
|
||||||
|
from shard_wiki.engine.links import extract_wikilinks
|
||||||
|
from shard_wiki.engine.profile import (
|
||||||
|
ProfileContribution,
|
||||||
|
derive_profile,
|
||||||
|
engine_base_profile,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"EngineKernel",
|
||||||
|
"extract_wikilinks",
|
||||||
|
"Hook",
|
||||||
|
"Extension",
|
||||||
|
"ExtensionError",
|
||||||
|
"ExtensionRuntime",
|
||||||
|
"ActiveExtensions",
|
||||||
|
"ActivationContext",
|
||||||
|
"ActivationProvider",
|
||||||
|
"StaticProvider",
|
||||||
|
"ActivationResolver",
|
||||||
|
"feature_control_provider",
|
||||||
|
"engine_base_profile",
|
||||||
|
"ProfileContribution",
|
||||||
|
"derive_profile",
|
||||||
|
"EngineShardAdapter",
|
||||||
|
"build_engine_shard",
|
||||||
|
]
|
||||||
129
src/shard_wiki/engine/activation.py
Normal file
129
src/shard_wiki/engine/activation.py
Normal file
@@ -0,0 +1,129 @@
|
|||||||
|
"""Per-shard extension activation (WikiEngineCoreArchitecture E-4/E-8, ADR-0001).
|
||||||
|
|
||||||
|
Decides *which registered extensions are active* for a given shard — **availability only, never
|
||||||
|
authorization**. The mechanism is OpenFeature-shaped and **provider-pluggable**:
|
||||||
|
|
||||||
|
- :class:`StaticProvider` is the **standalone / L0 default** — zero external dependency, in-process
|
||||||
|
flags with optional per-shard scoping. A kernel-only or offline shard uses this.
|
||||||
|
- :func:`feature_control_provider` lazily wraps the helix_forge ``feature_control_sdk`` (over
|
||||||
|
OpenFeature) when it is installed; absent, it returns ``None`` and the caller falls back to the
|
||||||
|
static default. This mirrors shard-wiki's identity-provider ladder (local default → external
|
||||||
|
when present), and keeps the engine core pure-stdlib.
|
||||||
|
|
||||||
|
An *activation profile* is ``{extension id → config}``; the active id set then feeds the
|
||||||
|
extension runtime's `activate()` (T2) and the derived capability profile (T4, E-5).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable, Mapping
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Protocol, runtime_checkable
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ActivationContext",
|
||||||
|
"ActivationProvider",
|
||||||
|
"StaticProvider",
|
||||||
|
"ActivationResolver",
|
||||||
|
"feature_control_provider",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ActivationContext:
|
||||||
|
"""Scope for an activation decision. Carries no principal/authz — availability only."""
|
||||||
|
|
||||||
|
shard_id: str
|
||||||
|
tenant_id: str | None = None
|
||||||
|
extra: Mapping[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def as_dict(self) -> dict[str, Any]:
|
||||||
|
d: dict[str, Any] = {"shard_id": self.shard_id}
|
||||||
|
if self.tenant_id is not None:
|
||||||
|
d["tenant_id"] = self.tenant_id
|
||||||
|
d.update(self.extra)
|
||||||
|
return d
|
||||||
|
|
||||||
|
|
||||||
|
@runtime_checkable
|
||||||
|
class ActivationProvider(Protocol):
|
||||||
|
"""Evaluates feature availability for an extension key in a context (OpenFeature-shaped)."""
|
||||||
|
|
||||||
|
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool: ...
|
||||||
|
|
||||||
|
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class StaticProvider:
|
||||||
|
"""The standalone default: in-process flags, optionally overridden per shard. Zero deps.
|
||||||
|
|
||||||
|
``flags`` is the base availability map; ``per_shard[shard_id]`` overrides it for one shard;
|
||||||
|
``configs[feature_key]`` supplies per-extension config. Unknown keys → ``default``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
flags: Mapping[str, bool] = field(default_factory=dict)
|
||||||
|
per_shard: Mapping[str, Mapping[str, bool]] = field(default_factory=dict)
|
||||||
|
configs: Mapping[str, Mapping[str, Any]] = field(default_factory=dict)
|
||||||
|
default: bool = False
|
||||||
|
|
||||||
|
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
|
||||||
|
shard = context.get("shard_id")
|
||||||
|
scoped = self.per_shard.get(shard, {}) if shard is not None else {}
|
||||||
|
if feature_key in scoped:
|
||||||
|
return scoped[feature_key]
|
||||||
|
return self.flags.get(feature_key, self.default)
|
||||||
|
|
||||||
|
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
|
||||||
|
return self.configs.get(feature_key, {})
|
||||||
|
|
||||||
|
|
||||||
|
class ActivationResolver:
|
||||||
|
"""Maps candidate extension ids → the active set / activation profile for a context."""
|
||||||
|
|
||||||
|
def __init__(self, provider: ActivationProvider) -> None:
|
||||||
|
self.provider = provider
|
||||||
|
|
||||||
|
def active_extensions(
|
||||||
|
self, candidate_ids: Iterable[str], context: ActivationContext
|
||||||
|
) -> set[str]:
|
||||||
|
ctx = context.as_dict()
|
||||||
|
return {eid for eid in candidate_ids if self.provider.is_active(eid, ctx)}
|
||||||
|
|
||||||
|
def activation_profile(
|
||||||
|
self, candidate_ids: Iterable[str], context: ActivationContext
|
||||||
|
) -> dict[str, Mapping[str, Any]]:
|
||||||
|
"""``{extension id → config}`` for the active subset."""
|
||||||
|
ctx = context.as_dict()
|
||||||
|
return {
|
||||||
|
eid: self.provider.config(eid, ctx)
|
||||||
|
for eid in candidate_ids
|
||||||
|
if self.provider.is_active(eid, ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def feature_control_provider(domain: str | None = None) -> ActivationProvider | None:
|
||||||
|
"""Return a feature-control-backed provider if ``feature_control_sdk`` is importable, else
|
||||||
|
``None`` (caller falls back to :class:`StaticProvider`). Lazy import keeps the engine core
|
||||||
|
dependency-free (ADR-0001)."""
|
||||||
|
try: # optional engine extra — not a core dependency
|
||||||
|
from feature_control_sdk import FeatureControlClient # type: ignore
|
||||||
|
except ImportError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
client = FeatureControlClient(domain=domain)
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class _FeatureControlProvider:
|
||||||
|
_client: Any
|
||||||
|
|
||||||
|
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
|
||||||
|
return bool(
|
||||||
|
self._client.get_boolean_value(feature_key, False, context=dict(context))
|
||||||
|
)
|
||||||
|
|
||||||
|
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
|
||||||
|
getter = getattr(self._client, "get_object_value", None)
|
||||||
|
return dict(getter(feature_key, {}, context=dict(context))) if getter else {}
|
||||||
|
|
||||||
|
return _FeatureControlProvider(client)
|
||||||
82
src/shard_wiki/engine/adapter.py
Normal file
82
src/shard_wiki/engine/adapter.py
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
"""EngineShardAdapter — the engine exposed as a canonical-mode shard (WikiEngineCoreArchitecture
|
||||||
|
§6, E-1/E-5).
|
||||||
|
|
||||||
|
The engine is *one shard*: the orchestrator consumes it only through this `ShardAdapter`. The
|
||||||
|
adapter is backed by the kernel (T1) + a composed extension set (T2/T3); its §A capability
|
||||||
|
profile is **derived from the active extensions** (T4), so the orchestrator sees an honest,
|
||||||
|
conformance-verifiable profile that reflects exactly what is activated. Read/write run the
|
||||||
|
extensions' transform hooks; everything above this stays in the orchestrator (no union/projection
|
||||||
|
import).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
from shard_wiki.adapters import ShardAdapter
|
||||||
|
from shard_wiki.engine.activation import ActivationContext, ActivationProvider, ActivationResolver
|
||||||
|
from shard_wiki.engine.extension import ActiveExtensions, ExtensionRuntime, Hook
|
||||||
|
from shard_wiki.engine.kernel import EngineKernel
|
||||||
|
from shard_wiki.engine.profile import derive_profile
|
||||||
|
from shard_wiki.model import CapabilityProfile, NotSupported, Page, Verb
|
||||||
|
|
||||||
|
__all__ = ["EngineShardAdapter", "build_engine_shard"]
|
||||||
|
|
||||||
|
|
||||||
|
class EngineShardAdapter(ShardAdapter):
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
kernel: EngineKernel,
|
||||||
|
active: ActiveExtensions,
|
||||||
|
base_profile: CapabilityProfile | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._kernel = kernel
|
||||||
|
self._active = active
|
||||||
|
self._profile = derive_profile(active, base_profile) # validated (E-5)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
return self._kernel.shard_id
|
||||||
|
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
return self._profile
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
return self._kernel.keys()
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
page = self._kernel.read(key)
|
||||||
|
return self._active.dispatch_transform(Hook.ON_READ, page, {"shard_id": self.shard_id})
|
||||||
|
|
||||||
|
def current_rev(self, key: str) -> str | None:
|
||||||
|
return self._kernel.current_rev(key)
|
||||||
|
|
||||||
|
def write(self, key: str, body: str) -> Page:
|
||||||
|
if not self._profile.supports(Verb.WRITE):
|
||||||
|
raise NotSupported(f"{type(self).__name__} ({self.shard_id}) is read-only")
|
||||||
|
body = self._active.dispatch_transform(
|
||||||
|
Hook.ON_WRITE, body, {"shard_id": self.shard_id, "key": key}
|
||||||
|
)
|
||||||
|
return self._kernel.write(key, body)
|
||||||
|
|
||||||
|
|
||||||
|
def build_engine_shard(
|
||||||
|
shard_id: str,
|
||||||
|
runtime: ExtensionRuntime,
|
||||||
|
*,
|
||||||
|
activate: Iterable[str] | None = None,
|
||||||
|
provider: ActivationProvider | None = None,
|
||||||
|
context: ActivationContext | None = None,
|
||||||
|
base_profile: CapabilityProfile | None = None,
|
||||||
|
) -> EngineShardAdapter:
|
||||||
|
"""Stand up an engine shard: resolve which extensions are active (explicit ``activate`` ids,
|
||||||
|
or via an activation ``provider`` over the runtime's available set), compose them, and wrap a
|
||||||
|
fresh kernel as a `ShardAdapter`.
|
||||||
|
"""
|
||||||
|
if provider is not None:
|
||||||
|
ctx = context or ActivationContext(shard_id)
|
||||||
|
ids = ActivationResolver(provider).active_extensions(runtime.available(), ctx)
|
||||||
|
else:
|
||||||
|
ids = set(activate or ())
|
||||||
|
active = runtime.activate(ids)
|
||||||
|
return EngineShardAdapter(EngineKernel(shard_id), active, base_profile)
|
||||||
165
src/shard_wiki/engine/extension.py
Normal file
165
src/shard_wiki/engine/extension.py
Normal file
@@ -0,0 +1,165 @@
|
|||||||
|
"""Typed-extension runtime — the engine framework (WikiEngineCoreArchitecture §4, E-3/E-9).
|
||||||
|
|
||||||
|
Everything beyond the kernel's c2-minimum is an :class:`Extension`: it declares a typed
|
||||||
|
contract (id, provided capabilities, declared types, bound hooks, dependencies, conflicts) and
|
||||||
|
the runtime **composes** an activation set deterministically, **rejecting impossible profiles**
|
||||||
|
(unmet deps / conflicts / type collisions) — the §6.5 capability-as-data discipline applied to
|
||||||
|
extensions. Extension structure is **verified at registration** (mirrors §6.6 conformance):
|
||||||
|
bad ids or non-callable hook handlers are refused, so the framework acts on verified data.
|
||||||
|
|
||||||
|
Hooks are dispatched in a declared, deterministic order (dependency-topological, ties by id):
|
||||||
|
*transform* hooks chain a payload through handlers; *collect* hooks gather contributions.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable, Iterable, Mapping
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Any, ClassVar
|
||||||
|
|
||||||
|
__all__ = ["Hook", "Extension", "ExtensionError", "ExtensionRuntime", "ActiveExtensions"]
|
||||||
|
|
||||||
|
|
||||||
|
class ExtensionError(ValueError):
|
||||||
|
"""Raised when an extension is malformed or an activation set is impossible (§6.5)."""
|
||||||
|
|
||||||
|
|
||||||
|
class Hook(Enum):
|
||||||
|
# transform hooks: each handler takes (payload, ctx) and returns the next payload
|
||||||
|
ON_WRITE = "on_write" # transform a draft before persist
|
||||||
|
ON_READ = "on_read" # transform a page on read
|
||||||
|
ON_RESOLVE = "on_resolve" # transform a name resolution
|
||||||
|
ON_RENDER = "on_render" # produce a derived representation
|
||||||
|
# collect hooks: each handler takes (payload, ctx) and returns a contribution
|
||||||
|
ON_LINK = "on_link" # contribute link/transclusion edges
|
||||||
|
ON_QUERY = "on_query" # answer a query
|
||||||
|
ON_PROFILE = "on_profile" # contribute capability-profile positions (E-5)
|
||||||
|
|
||||||
|
|
||||||
|
_TRANSFORM = frozenset({Hook.ON_WRITE, Hook.ON_READ, Hook.ON_RESOLVE, Hook.ON_RENDER})
|
||||||
|
_COLLECT = frozenset({Hook.ON_LINK, Hook.ON_QUERY, Hook.ON_PROFILE})
|
||||||
|
|
||||||
|
|
||||||
|
class Extension:
|
||||||
|
"""Base class for a typed extension. Subclasses set the class vars and override
|
||||||
|
:meth:`hooks` to bind handlers (signature ``handler(payload, ctx) -> result``)."""
|
||||||
|
|
||||||
|
id: ClassVar[str] = ""
|
||||||
|
provides: ClassVar[tuple[str, ...]] = ()
|
||||||
|
declares_types: ClassVar[tuple[str, ...]] = ()
|
||||||
|
depends_on: ClassVar[tuple[str, ...]] = ()
|
||||||
|
conflicts_with: ClassVar[tuple[str, ...]] = ()
|
||||||
|
|
||||||
|
def hooks(self) -> Mapping[Hook, Callable[[Any, Any], Any]]:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
class ActiveExtensions:
|
||||||
|
"""A composed, ordered activation set with deterministic hook dispatch."""
|
||||||
|
|
||||||
|
def __init__(self, ordered: list[Extension]) -> None:
|
||||||
|
self._ordered = ordered
|
||||||
|
self.ids: tuple[str, ...] = tuple(e.id for e in ordered)
|
||||||
|
self._tables: dict[Hook, list[tuple[str, Callable[[Any, Any], Any]]]] = {}
|
||||||
|
for ext in ordered:
|
||||||
|
for hook, fn in ext.hooks().items():
|
||||||
|
self._tables.setdefault(hook, []).append((ext.id, fn))
|
||||||
|
|
||||||
|
def handlers(self, hook: Hook) -> tuple[str, ...]:
|
||||||
|
"""The extension ids bound to ``hook``, in dispatch order (for introspection)."""
|
||||||
|
return tuple(eid for eid, _ in self._tables.get(hook, ()))
|
||||||
|
|
||||||
|
def dispatch_transform(self, hook: Hook, payload: Any, ctx: Any = None) -> Any:
|
||||||
|
if hook not in _TRANSFORM:
|
||||||
|
raise ExtensionError(f"{hook} is not a transform hook")
|
||||||
|
for _eid, fn in self._tables.get(hook, ()):
|
||||||
|
payload = fn(payload, ctx)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
def dispatch_collect(self, hook: Hook, payload: Any = None, ctx: Any = None) -> list[Any]:
|
||||||
|
if hook not in _COLLECT:
|
||||||
|
raise ExtensionError(f"{hook} is not a collect hook")
|
||||||
|
return [fn(payload, ctx) for _eid, fn in self._tables.get(hook, ())]
|
||||||
|
|
||||||
|
|
||||||
|
class ExtensionRuntime:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._registered: dict[str, Extension] = {}
|
||||||
|
|
||||||
|
def available(self) -> frozenset[str]:
|
||||||
|
"""Ids of all registered extensions (the candidate set for activation)."""
|
||||||
|
return frozenset(self._registered)
|
||||||
|
|
||||||
|
def register(self, ext: Extension) -> Extension:
|
||||||
|
"""Register an extension after structural verification (mirrors §6.6)."""
|
||||||
|
if not ext.id or not ext.id.startswith("ext."):
|
||||||
|
raise ExtensionError(f"extension id must be 'ext.<name>', got {ext.id!r}")
|
||||||
|
if ext.id in self._registered:
|
||||||
|
raise ExtensionError(f"duplicate extension id: {ext.id}")
|
||||||
|
bound = ext.hooks()
|
||||||
|
for hook, fn in bound.items():
|
||||||
|
if not isinstance(hook, Hook):
|
||||||
|
raise ExtensionError(f"{ext.id}: hook key {hook!r} is not a Hook")
|
||||||
|
if not callable(fn):
|
||||||
|
raise ExtensionError(f"{ext.id}: handler for {hook} is not callable")
|
||||||
|
self._registered[ext.id] = ext
|
||||||
|
return ext
|
||||||
|
|
||||||
|
def activate(self, ids: Iterable[str]) -> ActiveExtensions:
|
||||||
|
"""Compose an activation set: dependency closure → conflict/type checks → deterministic
|
||||||
|
order. Raises :class:`ExtensionError` on an impossible profile."""
|
||||||
|
requested = set(ids)
|
||||||
|
unknown = requested - self._registered.keys()
|
||||||
|
if unknown:
|
||||||
|
raise ExtensionError(f"unknown extensions: {sorted(unknown)}")
|
||||||
|
|
||||||
|
# dependency closure
|
||||||
|
active: set[str] = set()
|
||||||
|
frontier = list(requested)
|
||||||
|
while frontier:
|
||||||
|
eid = frontier.pop()
|
||||||
|
if eid in active:
|
||||||
|
continue
|
||||||
|
ext = self._registered.get(eid)
|
||||||
|
if ext is None:
|
||||||
|
raise ExtensionError(f"unmet dependency: {eid}")
|
||||||
|
active.add(eid)
|
||||||
|
frontier.extend(d for d in ext.depends_on if d not in active)
|
||||||
|
|
||||||
|
exts = [self._registered[e] for e in active]
|
||||||
|
|
||||||
|
# conflicts
|
||||||
|
for ext in exts:
|
||||||
|
clash = active & set(ext.conflicts_with)
|
||||||
|
if clash:
|
||||||
|
raise ExtensionError(f"{ext.id} conflicts with active {sorted(clash)}")
|
||||||
|
|
||||||
|
# type collisions (two active extensions claiming the same type id)
|
||||||
|
owner: dict[str, str] = {}
|
||||||
|
for ext in exts:
|
||||||
|
for t in ext.declares_types:
|
||||||
|
if t in owner:
|
||||||
|
raise ExtensionError(
|
||||||
|
f"type collision on {t!r}: {owner[t]} and {ext.id}"
|
||||||
|
)
|
||||||
|
owner[t] = ext.id
|
||||||
|
|
||||||
|
return ActiveExtensions(self._topo_order(exts))
|
||||||
|
|
||||||
|
def _topo_order(self, exts: list[Extension]) -> list[Extension]:
|
||||||
|
"""Dependencies before dependents; ties broken by id (deterministic)."""
|
||||||
|
by_id = {e.id: e for e in exts}
|
||||||
|
ordered: list[Extension] = []
|
||||||
|
placed: set[str] = set()
|
||||||
|
|
||||||
|
def visit(ext: Extension) -> None:
|
||||||
|
if ext.id in placed:
|
||||||
|
return
|
||||||
|
for dep in sorted(d for d in ext.depends_on if d in by_id):
|
||||||
|
visit(by_id[dep])
|
||||||
|
placed.add(ext.id)
|
||||||
|
ordered.append(ext)
|
||||||
|
|
||||||
|
for ext in sorted(exts, key=lambda e: e.id):
|
||||||
|
visit(ext)
|
||||||
|
return ordered
|
||||||
10
src/shard_wiki/engine/extensions/__init__.py
Normal file
10
src/shard_wiki/engine/extensions/__init__.py
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
"""engine/extensions/ — built-in typed extensions for the wiki engine.
|
||||||
|
|
||||||
|
Each is a typed :class:`~shard_wiki.engine.extension.Extension` a shard activates only if needed.
|
||||||
|
``ext.struct`` (typed records) is the first; more (views, addressing, computational, authz) follow
|
||||||
|
the same pattern.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.engine.extensions.struct import StructExt, parse_frontmatter
|
||||||
|
|
||||||
|
__all__ = ["StructExt", "parse_frontmatter"]
|
||||||
81
src/shard_wiki/engine/extensions/struct.py
Normal file
81
src/shard_wiki/engine/extensions/struct.py
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
"""ext.struct — typed records, a first built-in extension (WikiEngineCoreArchitecture X-STRUCT).
|
||||||
|
|
||||||
|
Demonstrates the typed-extension framework end-to-end. A page may carry a leading in-text
|
||||||
|
frontmatter block (`---` … `---`, `key: value` lines — git-diffable structure, blueprint T12).
|
||||||
|
With this extension **active**, the engine:
|
||||||
|
|
||||||
|
- **ON_WRITE** validates the structured block (optionally against an allowed-field set) — a
|
||||||
|
malformed/disallowed structured page is rejected; the body is otherwise unchanged
|
||||||
|
(content-preserving, so write conformance holds);
|
||||||
|
- **ON_READ** tags such pages as `PageShape.TYPED_RECORD`;
|
||||||
|
- **ON_PROFILE** raises the shard's profile with the `structured-payload` verb (E-5).
|
||||||
|
|
||||||
|
With the extension **inactive**, the kernel treats the same page as opaque prose — the feature
|
||||||
|
is genuinely absent (honest profile). This is "activate only what you need" in action.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
from collections.abc import Iterable, Mapping
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from shard_wiki.engine.extension import Extension, Hook
|
||||||
|
from shard_wiki.engine.profile import ProfileContribution
|
||||||
|
from shard_wiki.model import Page, PageShape, Verb
|
||||||
|
|
||||||
|
__all__ = ["StructExt", "parse_frontmatter"]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_frontmatter(body: str) -> tuple[dict[str, str], bool]:
|
||||||
|
"""Parse a leading ``---`` … ``---`` block of ``key: value`` lines.
|
||||||
|
|
||||||
|
Returns ``(fields, has_block)``. An unterminated opening ``---`` is *not* a valid block.
|
||||||
|
"""
|
||||||
|
lines = body.splitlines()
|
||||||
|
if not lines or lines[0].strip() != "---":
|
||||||
|
return {}, False
|
||||||
|
fields: dict[str, str] = {}
|
||||||
|
for line in lines[1:]:
|
||||||
|
if line.strip() == "---":
|
||||||
|
return fields, True
|
||||||
|
if ":" in line:
|
||||||
|
key, _, value = line.partition(":")
|
||||||
|
fields[key.strip()] = value.strip()
|
||||||
|
return {}, False # no closing fence → not a frontmatter block
|
||||||
|
|
||||||
|
|
||||||
|
class StructExt(Extension):
|
||||||
|
id = "ext.struct"
|
||||||
|
declares_types = ("record",)
|
||||||
|
provides = ("capability.wiki.page-model",)
|
||||||
|
|
||||||
|
def __init__(self, allowed_fields: Iterable[str] | None = None) -> None:
|
||||||
|
self._allowed: set[str] | None = set(allowed_fields) if allowed_fields is not None else None
|
||||||
|
|
||||||
|
def hooks(self) -> Mapping[Hook, Any]:
|
||||||
|
return {
|
||||||
|
Hook.ON_WRITE: self._on_write,
|
||||||
|
Hook.ON_READ: self._on_read,
|
||||||
|
Hook.ON_PROFILE: self._on_profile,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _on_write(self, body: str, ctx: Any) -> str:
|
||||||
|
fields, has_block = parse_frontmatter(body)
|
||||||
|
if has_block and self._allowed is not None:
|
||||||
|
disallowed = set(fields) - self._allowed
|
||||||
|
if disallowed:
|
||||||
|
raise ValueError(f"ext.struct: disallowed fields {sorted(disallowed)}")
|
||||||
|
return body # structure stays in-text (git-diffable); body unchanged
|
||||||
|
|
||||||
|
def _on_read(self, page: Page, ctx: Any) -> Page:
|
||||||
|
_, has_block = parse_frontmatter(page.body)
|
||||||
|
return dataclasses.replace(page, shape=PageShape.TYPED_RECORD) if has_block else page
|
||||||
|
|
||||||
|
def _on_profile(self, payload: Any, ctx: Any) -> ProfileContribution:
|
||||||
|
return ProfileContribution(verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD}))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def fields(body: str) -> dict[str, str]:
|
||||||
|
"""Parsed structured fields of a page body (empty if it has no frontmatter block)."""
|
||||||
|
return parse_frontmatter(body)[0]
|
||||||
87
src/shard_wiki/engine/kernel.py
Normal file
87
src/shard_wiki/engine/kernel.py
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
"""Engine kernel — the small page-store core (WikiEngineCoreArchitecture §3, EC-1…EC-4).
|
||||||
|
|
||||||
|
The irreducible engine: author/read/edit pages (edit = a new version; delete = a recoverable
|
||||||
|
tombstone — history is the floor, I-10), enumerate keys, and resolve `[[wikilinks]]` (red-link =
|
||||||
|
an unresolved target). No feature beyond this c2-minimum lives in the kernel; everything else is
|
||||||
|
a typed extension (E-3).
|
||||||
|
|
||||||
|
Storage is intentionally simple here (in-memory version history); the git-IS-store backing
|
||||||
|
(SHARD-WP-0009/0012) slots in behind the same API later. The kernel reuses the page model and
|
||||||
|
provenance leaf; it does not redefine them.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
from shard_wiki.engine.links import extract_wikilinks
|
||||||
|
from shard_wiki.model import Identity, Page, Placement
|
||||||
|
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
|
||||||
|
|
||||||
|
__all__ = ["EngineKernel"]
|
||||||
|
|
||||||
|
|
||||||
|
class EngineKernel:
|
||||||
|
"""An in-process page store with per-page version history for one engine shard."""
|
||||||
|
|
||||||
|
def __init__(self, shard_id: str) -> None:
|
||||||
|
self.shard_id = shard_id
|
||||||
|
self._versions: dict[str, list[Page]] = {}
|
||||||
|
self._deleted: set[str] = set()
|
||||||
|
|
||||||
|
# --- write path (create/edit are one operation; both append a version) ---
|
||||||
|
def write(self, key: str, body: str) -> Page:
|
||||||
|
versions = self._versions.setdefault(key, [])
|
||||||
|
rev = str(len(versions) + 1)
|
||||||
|
page = Page(
|
||||||
|
identity=Identity(self.shard_id, key),
|
||||||
|
body=body,
|
||||||
|
envelope=ProvenanceEnvelope(
|
||||||
|
source_shard=self.shard_id,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
staleness=Staleness.FRESH,
|
||||||
|
source_rev=rev,
|
||||||
|
observed_at=datetime.now(tz=timezone.utc),
|
||||||
|
),
|
||||||
|
placements=(Placement(self.shard_id, key),),
|
||||||
|
)
|
||||||
|
versions.append(page)
|
||||||
|
self._deleted.discard(key)
|
||||||
|
return page
|
||||||
|
|
||||||
|
# --- read path ---
|
||||||
|
def exists(self, key: str) -> bool:
|
||||||
|
return key in self._versions and key not in self._deleted
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
"""Latest version of a live page. Raises ``KeyError`` if absent or deleted."""
|
||||||
|
if not self.exists(key):
|
||||||
|
raise KeyError(key)
|
||||||
|
return self._versions[key][-1]
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
return (k for k in sorted(self._versions) if k not in self._deleted)
|
||||||
|
|
||||||
|
def current_rev(self, key: str) -> str | None:
|
||||||
|
return self._versions[key][-1].envelope.source_rev if self.exists(key) else None
|
||||||
|
|
||||||
|
# --- history & recoverability (I-10): versions are retained across delete ---
|
||||||
|
def history(self, key: str) -> tuple[Page, ...]:
|
||||||
|
"""All versions ever written for ``key`` (oldest→newest), even after delete."""
|
||||||
|
return tuple(self._versions.get(key, ()))
|
||||||
|
|
||||||
|
def delete(self, key: str) -> None:
|
||||||
|
"""Tombstone a page (history retained; restore by writing again)."""
|
||||||
|
if key not in self._versions:
|
||||||
|
raise KeyError(key)
|
||||||
|
self._deleted.add(key)
|
||||||
|
|
||||||
|
# --- links (EC-4): resolution + red-link detection within this shard ---
|
||||||
|
def links(self, key: str) -> list[str]:
|
||||||
|
"""Wikilink targets in a page's current body."""
|
||||||
|
return extract_wikilinks(self.read(key).body)
|
||||||
|
|
||||||
|
def resolve_link(self, target: str) -> Identity | None:
|
||||||
|
"""Resolve a wikilink target to a live page identity, or ``None`` (a **red-link**)."""
|
||||||
|
return self.read(target).identity if self.exists(target) else None
|
||||||
25
src/shard_wiki/engine/links.py
Normal file
25
src/shard_wiki/engine/links.py
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
"""Wikilink extraction — the kernel's link primitive (WikiEngineCoreArchitecture EC-4).
|
||||||
|
|
||||||
|
`[[Target]]` and `[[Target|label]]`. CamelCase auto-linking is intentionally NOT here (it is an
|
||||||
|
opt-in concern per FederationRequirements ADR-06); the kernel only knows explicit wikilinks.
|
||||||
|
Link *resolution* (and red-link detection) is the kernel's job (it knows which keys exist);
|
||||||
|
*rendering* is a consumer concern (headless engine, no UI).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
__all__ = ["extract_wikilinks"]
|
||||||
|
|
||||||
|
_WIKILINK = re.compile(r"\[\[([^\]|]+?)(?:\|[^\]]*)?\]\]")
|
||||||
|
|
||||||
|
|
||||||
|
def extract_wikilinks(body: str) -> list[str]:
|
||||||
|
"""Return the ordered, de-duplicated wikilink targets in ``body`` (label part dropped)."""
|
||||||
|
seen: dict[str, None] = {}
|
||||||
|
for m in _WIKILINK.finditer(body):
|
||||||
|
target = m.group(1).strip()
|
||||||
|
if target:
|
||||||
|
seen.setdefault(target, None)
|
||||||
|
return list(seen)
|
||||||
112
src/shard_wiki/engine/profile.py
Normal file
112
src/shard_wiki/engine/profile.py
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
"""Capability profile derived from active extensions (WikiEngineCoreArchitecture E-5).
|
||||||
|
|
||||||
|
The engine's §A `CapabilityProfile` is **computed**, not hand-set: start from the kernel base
|
||||||
|
profile, then fold each active extension's `on_profile` contribution (in the runtime's
|
||||||
|
deterministic order), then `validate()`. This realizes the chain *configuration (which
|
||||||
|
extensions) → capability (the profile) → conformance* — activating an extension raises the
|
||||||
|
shard's advertised capabilities, and composition can never yield an impossible profile (validate
|
||||||
|
rejects it, §6.5).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.engine.extension import ActiveExtensions, Hook
|
||||||
|
from shard_wiki.model import (
|
||||||
|
AccessGrant,
|
||||||
|
Addressing,
|
||||||
|
AttachmentMode,
|
||||||
|
CapabilityProfile,
|
||||||
|
ContentOpacity,
|
||||||
|
History,
|
||||||
|
MergeModel,
|
||||||
|
NativeQuery,
|
||||||
|
OperationalEnvelope,
|
||||||
|
Substrate,
|
||||||
|
Translation,
|
||||||
|
Verb,
|
||||||
|
WriteGranularity,
|
||||||
|
)
|
||||||
|
from shard_wiki.provenance import Liveness
|
||||||
|
|
||||||
|
__all__ = ["engine_base_profile", "ProfileContribution", "derive_profile"]
|
||||||
|
|
||||||
|
# Profile fields an extension may *raise* via on_profile (substrate/attachment are kernel-fixed).
|
||||||
|
_OVERRIDABLE = (
|
||||||
|
"write_granularity",
|
||||||
|
"content_opacity",
|
||||||
|
"liveness",
|
||||||
|
"history",
|
||||||
|
"merge_model",
|
||||||
|
"addressing",
|
||||||
|
"native_query",
|
||||||
|
"translation",
|
||||||
|
"access_grant",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def engine_base_profile() -> CapabilityProfile:
|
||||||
|
"""The kernel-only (no extensions) capability profile — the c2-minimum engine shard."""
|
||||||
|
return CapabilityProfile(
|
||||||
|
substrate=Substrate.FILES,
|
||||||
|
attachment_mode=AttachmentMode.IN_ENGINE_HOST,
|
||||||
|
write_granularity=WriteGranularity.PER_PAGE,
|
||||||
|
content_opacity=ContentOpacity.TRANSPARENT,
|
||||||
|
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||||
|
access_grant=AccessGrant.OPEN,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
history=History.INTERNAL_ONLY,
|
||||||
|
merge_model=MergeModel.NONE,
|
||||||
|
addressing=Addressing.PATH,
|
||||||
|
native_query=NativeQuery.NONE,
|
||||||
|
translation=Translation.NATIVE,
|
||||||
|
supported_verbs=frozenset({Verb.READ, Verb.WRITE}),
|
||||||
|
).validate()
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ProfileContribution:
|
||||||
|
"""An extension's contribution to the derived profile (returned from its ON_PROFILE hook).
|
||||||
|
|
||||||
|
A non-``None`` axis overrides that axis; ``verbs_add`` are unioned in. Order = the runtime's
|
||||||
|
deterministic dispatch order, so later extensions win on a contested axis."""
|
||||||
|
|
||||||
|
write_granularity: WriteGranularity | None = None
|
||||||
|
content_opacity: ContentOpacity | None = None
|
||||||
|
liveness: Liveness | None = None
|
||||||
|
history: History | None = None
|
||||||
|
merge_model: MergeModel | None = None
|
||||||
|
addressing: Addressing | None = None
|
||||||
|
native_query: NativeQuery | None = None
|
||||||
|
translation: Translation | None = None
|
||||||
|
access_grant: AccessGrant | None = None
|
||||||
|
verbs_add: frozenset[Verb] = frozenset()
|
||||||
|
|
||||||
|
|
||||||
|
def derive_profile(
|
||||||
|
active: ActiveExtensions, base: CapabilityProfile | None = None
|
||||||
|
) -> CapabilityProfile:
|
||||||
|
"""Fold active extensions' ON_PROFILE contributions onto ``base`` and validate the result.
|
||||||
|
|
||||||
|
Raises :class:`~shard_wiki.model.ProfileError` if the composed profile is impossible — so an
|
||||||
|
activation set can never advertise an invalid capability profile.
|
||||||
|
"""
|
||||||
|
profile = base or engine_base_profile()
|
||||||
|
contributions = active.dispatch_collect(Hook.ON_PROFILE)
|
||||||
|
|
||||||
|
overrides: dict[str, object] = {}
|
||||||
|
verbs: set[Verb] = set(profile.supported_verbs)
|
||||||
|
for contrib in contributions:
|
||||||
|
if not isinstance(contrib, ProfileContribution):
|
||||||
|
continue
|
||||||
|
for field_name in _OVERRIDABLE:
|
||||||
|
value = getattr(contrib, field_name)
|
||||||
|
if value is not None:
|
||||||
|
overrides[field_name] = value
|
||||||
|
verbs |= set(contrib.verbs_add)
|
||||||
|
|
||||||
|
return dataclasses.replace(
|
||||||
|
profile, supported_verbs=frozenset(verbs), **overrides
|
||||||
|
).validate()
|
||||||
46
src/shard_wiki/incremental/__init__.py
Normal file
46
src/shard_wiki/incremental/__init__.py
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
"""incremental/ — the incremental-first derived tier (CoreArchitectureBlueprint §8.7).
|
||||||
|
|
||||||
|
Equivalence is **indexed** (blocking/LSH + verify), not pairwise O(N²); maintenance is
|
||||||
|
**change-driven** (delta with retraction + propagation, review B-4), keeping the derived tier equal
|
||||||
|
to a from-scratch rebuild — which becomes a bounded fallback, not the operational path. A
|
||||||
|
Merkle-style **digest** plus a background **consistency-checker** make ``derived = f(canonical)``
|
||||||
|
verified rather than asserted (I-2), self-healing on detected drift.
|
||||||
|
|
||||||
|
In-memory only for this slice (no persisted index store); per-partition structure is honoured but
|
||||||
|
multi-tenant deployment is later. Per the dependency rule this imports down (model/provenance) and
|
||||||
|
is wired by the orchestrator.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.incremental.equivalence import (
|
||||||
|
EquivalenceEdge,
|
||||||
|
EquivalenceIndex,
|
||||||
|
normalized_title,
|
||||||
|
)
|
||||||
|
from shard_wiki.incremental.minhash import (
|
||||||
|
MinHasher,
|
||||||
|
band_keys,
|
||||||
|
jaccard,
|
||||||
|
shingles,
|
||||||
|
)
|
||||||
|
from shard_wiki.incremental.union_index import UnionIndex
|
||||||
|
from shard_wiki.incremental.verification import (
|
||||||
|
ConsistencyChecker,
|
||||||
|
ConsistencyReport,
|
||||||
|
derived_digest,
|
||||||
|
region_digest,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"shingles",
|
||||||
|
"MinHasher",
|
||||||
|
"band_keys",
|
||||||
|
"jaccard",
|
||||||
|
"EquivalenceEdge",
|
||||||
|
"EquivalenceIndex",
|
||||||
|
"normalized_title",
|
||||||
|
"derived_digest",
|
||||||
|
"region_digest",
|
||||||
|
"ConsistencyReport",
|
||||||
|
"ConsistencyChecker",
|
||||||
|
"UnionIndex",
|
||||||
|
]
|
||||||
225
src/shard_wiki/incremental/equivalence.py
Normal file
225
src/shard_wiki/incremental/equivalence.py
Normal file
@@ -0,0 +1,225 @@
|
|||||||
|
"""Indexed equivalence — blocking + verify, incrementally maintained (SHARD-WP-0011 T1/T2).
|
||||||
|
|
||||||
|
Equivalence (two *distinct* identities holding the same page) is detected without pairwise O(N²):
|
||||||
|
|
||||||
|
1. **Blocking** generates candidate pairs — pages sharing a normalized-title bucket or an LSH band
|
||||||
|
(MinHash over content shingles).
|
||||||
|
2. **Verify** confirms a candidate — exact-body fingerprint match, or shingle Jaccard ≥ threshold —
|
||||||
|
plus **curator bindings** (explicit decision-log edges) which are always equivalence edges.
|
||||||
|
|
||||||
|
The index is **incrementally maintained** (T2): ``add`` / ``update`` / ``remove`` re-bucket the
|
||||||
|
changed page, **retract** the edges it leaves and **add** the edges it enters; equivalence groups
|
||||||
|
are the connected components of the current edge set, so a retraction that disconnects a component
|
||||||
|
**splits** a chorus automatically. A full :meth:`build` is just repeated ``add`` — the bounded
|
||||||
|
rebuild fallback. The invariant (and the test oracle): incremental state == a from-scratch rebuild.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import re
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.incremental.minhash import MinHasher, band_keys, jaccard, shingles
|
||||||
|
from shard_wiki.model import Identity, Page
|
||||||
|
|
||||||
|
__all__ = ["EquivalenceEdge", "EquivalenceIndex", "normalized_title"]
|
||||||
|
|
||||||
|
_NONALNUM_RE = re.compile(r"[^a-z0-9]+")
|
||||||
|
|
||||||
|
|
||||||
|
def normalized_title(key: str) -> str:
|
||||||
|
"""A blocking bucket key: the last path segment, lowercased, stripped of non-alphanumerics."""
|
||||||
|
leaf = key.rsplit("/", 1)[-1]
|
||||||
|
return _NONALNUM_RE.sub("", leaf.lower())
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class EquivalenceEdge:
|
||||||
|
"""A verified equivalence between two identities, tagged with why it was accepted."""
|
||||||
|
|
||||||
|
a: Identity
|
||||||
|
b: Identity
|
||||||
|
reason: str # "fingerprint" | "content" | "curator"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class _Entry:
|
||||||
|
shingle_set: frozenset[str]
|
||||||
|
bands: tuple[tuple[int, tuple[int, ...]], ...]
|
||||||
|
title: str
|
||||||
|
fingerprint: str
|
||||||
|
|
||||||
|
|
||||||
|
def _fingerprint(body: str) -> str:
|
||||||
|
return hashlib.blake2b(body.strip().encode("utf-8"), digest_size=16).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _pair(a: Identity, b: Identity) -> frozenset[Identity]:
|
||||||
|
return frozenset((a, b))
|
||||||
|
|
||||||
|
|
||||||
|
class EquivalenceIndex:
|
||||||
|
"""An incrementally maintained, blocked-and-verified equivalence relation over union pages."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
num_perm: int = 64,
|
||||||
|
num_bands: int = 32,
|
||||||
|
threshold: float = 0.7,
|
||||||
|
hasher: MinHasher | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.threshold = threshold
|
||||||
|
self.num_bands = num_bands
|
||||||
|
self._hasher = hasher or MinHasher(num_perm=num_perm)
|
||||||
|
self._entries: dict[Identity, _Entry] = {}
|
||||||
|
self._band_buckets: dict[tuple[int, tuple[int, ...]], set[Identity]] = {}
|
||||||
|
self._title_buckets: dict[str, set[Identity]] = {}
|
||||||
|
self._content_edges: dict[frozenset[Identity], str] = {}
|
||||||
|
self._curator_edges: set[frozenset[Identity]] = set()
|
||||||
|
|
||||||
|
# -- build / maintain ----------------------------------------------------
|
||||||
|
|
||||||
|
def build(
|
||||||
|
self,
|
||||||
|
pages: Iterable[Page],
|
||||||
|
curator_edges: Iterable[tuple[Identity, Identity]] = (),
|
||||||
|
) -> None:
|
||||||
|
"""Rebuild from scratch (the bounded fallback): add every page, then curator edges."""
|
||||||
|
self.__init__(
|
||||||
|
num_bands=self.num_bands, threshold=self.threshold, hasher=self._hasher
|
||||||
|
)
|
||||||
|
for page in pages:
|
||||||
|
self.add(page)
|
||||||
|
for a, b in curator_edges:
|
||||||
|
self.bind(a, b)
|
||||||
|
|
||||||
|
def add(self, page: Page) -> None:
|
||||||
|
"""Index a new (or, via :meth:`update`, refreshed) page and add its equivalence edges."""
|
||||||
|
identity = page.identity
|
||||||
|
entry = self._make_entry(page)
|
||||||
|
self._entries[identity] = entry
|
||||||
|
for key in entry.bands:
|
||||||
|
self._band_buckets.setdefault(key, set()).add(identity)
|
||||||
|
self._title_buckets.setdefault(entry.title, set()).add(identity)
|
||||||
|
|
||||||
|
for candidate in self._candidates(identity, entry):
|
||||||
|
reason = self._verify(identity, candidate)
|
||||||
|
if reason is not None:
|
||||||
|
self._content_edges[_pair(identity, candidate)] = reason
|
||||||
|
|
||||||
|
def remove(self, identity: Identity) -> None:
|
||||||
|
"""Drop a page: de-bucket it and retract every content edge incident to it."""
|
||||||
|
entry = self._entries.pop(identity, None)
|
||||||
|
if entry is None:
|
||||||
|
return
|
||||||
|
for key in entry.bands:
|
||||||
|
self._discard_bucket(self._band_buckets, key, identity)
|
||||||
|
self._discard_bucket(self._title_buckets, entry.title, identity)
|
||||||
|
for edge in [e for e in self._content_edges if identity in e]:
|
||||||
|
del self._content_edges[edge]
|
||||||
|
|
||||||
|
def update(self, page: Page) -> None:
|
||||||
|
"""Apply a change as retract-then-add: stale (bucket-exit) edges go, new edges arrive."""
|
||||||
|
self.remove(page.identity)
|
||||||
|
self.add(page)
|
||||||
|
|
||||||
|
def bind(self, a: Identity, b: Identity) -> None:
|
||||||
|
"""Record a curator equivalence (an explicit decision-log binding); always an edge."""
|
||||||
|
if a != b:
|
||||||
|
self._curator_edges.add(_pair(a, b))
|
||||||
|
|
||||||
|
def unbind(self, a: Identity, b: Identity) -> None:
|
||||||
|
self._curator_edges.discard(_pair(a, b))
|
||||||
|
|
||||||
|
def set_curator_edges(self, edges: Iterable[tuple[Identity, Identity]]) -> None:
|
||||||
|
"""Replace all curator edges at once (re-syncing from the decision-log fold)."""
|
||||||
|
self._curator_edges = {_pair(a, b) for a, b in edges if a != b}
|
||||||
|
|
||||||
|
# -- queries -------------------------------------------------------------
|
||||||
|
|
||||||
|
def identities(self) -> frozenset[Identity]:
|
||||||
|
"""All identities currently present in the index."""
|
||||||
|
return frozenset(self._entries)
|
||||||
|
|
||||||
|
def fingerprint(self, identity: Identity) -> str | None:
|
||||||
|
"""The content fingerprint indexed for ``identity`` (None if absent) — a digest leaf."""
|
||||||
|
entry = self._entries.get(identity)
|
||||||
|
return entry.fingerprint if entry is not None else None
|
||||||
|
|
||||||
|
def edges(self) -> frozenset[frozenset[Identity]]:
|
||||||
|
"""All equivalence edges (content + curator) among currently present identities."""
|
||||||
|
present = self._entries.keys()
|
||||||
|
curator = {e for e in self._curator_edges if e <= present}
|
||||||
|
return frozenset(set(self._content_edges) | curator)
|
||||||
|
|
||||||
|
def groups(self) -> tuple[frozenset[Identity], ...]:
|
||||||
|
"""Equivalence groups: connected components of size ≥ 2 (union-find over the edges)."""
|
||||||
|
parent: dict[Identity, Identity] = {}
|
||||||
|
|
||||||
|
def find(x: Identity) -> Identity:
|
||||||
|
parent.setdefault(x, x)
|
||||||
|
root = x
|
||||||
|
while parent[root] != root:
|
||||||
|
root = parent[root]
|
||||||
|
while parent[x] != root:
|
||||||
|
parent[x], x = root, parent[x]
|
||||||
|
return root
|
||||||
|
|
||||||
|
for edge in self.edges():
|
||||||
|
a, b = tuple(edge)
|
||||||
|
ra, rb = find(a), find(b)
|
||||||
|
if ra != rb:
|
||||||
|
parent[ra] = rb
|
||||||
|
|
||||||
|
comps: dict[Identity, set[Identity]] = {}
|
||||||
|
for node in parent:
|
||||||
|
comps.setdefault(find(node), set()).add(node)
|
||||||
|
return tuple(
|
||||||
|
frozenset(members) for members in comps.values() if len(members) > 1
|
||||||
|
)
|
||||||
|
|
||||||
|
def equivalent_to(self, identity: Identity) -> frozenset[Identity]:
|
||||||
|
"""The equivalence group containing ``identity`` (including itself), else just itself."""
|
||||||
|
for group in self.groups():
|
||||||
|
if identity in group:
|
||||||
|
return group
|
||||||
|
return frozenset({identity})
|
||||||
|
|
||||||
|
# -- internals -----------------------------------------------------------
|
||||||
|
|
||||||
|
def _make_entry(self, page: Page) -> _Entry:
|
||||||
|
shingle_set = shingles(page.body)
|
||||||
|
signature = self._hasher.signature(shingle_set)
|
||||||
|
return _Entry(
|
||||||
|
shingle_set=shingle_set,
|
||||||
|
bands=band_keys(signature, self.num_bands),
|
||||||
|
title=normalized_title(page.identity.key),
|
||||||
|
fingerprint=_fingerprint(page.body),
|
||||||
|
)
|
||||||
|
|
||||||
|
def _candidates(self, identity: Identity, entry: _Entry) -> set[Identity]:
|
||||||
|
candidates: set[Identity] = set()
|
||||||
|
for key in entry.bands:
|
||||||
|
candidates |= self._band_buckets.get(key, set())
|
||||||
|
candidates |= self._title_buckets.get(entry.title, set())
|
||||||
|
candidates.discard(identity)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
def _verify(self, a: Identity, b: Identity) -> str | None:
|
||||||
|
ea, eb = self._entries[a], self._entries[b]
|
||||||
|
if ea.fingerprint == eb.fingerprint:
|
||||||
|
return "fingerprint"
|
||||||
|
if jaccard(ea.shingle_set, eb.shingle_set) >= self.threshold:
|
||||||
|
return "content"
|
||||||
|
return None
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _discard_bucket(buckets: dict, key, identity: Identity) -> None:
|
||||||
|
bucket = buckets.get(key)
|
||||||
|
if bucket is not None:
|
||||||
|
bucket.discard(identity)
|
||||||
|
if not bucket:
|
||||||
|
del buckets[key]
|
||||||
71
src/shard_wiki/incremental/minhash.py
Normal file
71
src/shard_wiki/incremental/minhash.py
Normal file
@@ -0,0 +1,71 @@
|
|||||||
|
"""MinHash + LSH banding primitives for content-similarity blocking (SHARD-WP-0011 T1).
|
||||||
|
|
||||||
|
Pure, deterministic functions (fixed hashing, no per-run randomness) so the derived tier and its
|
||||||
|
digest are reproducible. Shingle a body into k-grams, MinHash the shingle set into a signature,
|
||||||
|
split the signature into LSH bands; two pages sharing a band are *candidates* for equivalence —
|
||||||
|
the cheap pre-filter that replaces pairwise O(N²) comparison.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import random
|
||||||
|
import re
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
__all__ = ["shingles", "MinHasher", "band_keys", "jaccard"]
|
||||||
|
|
||||||
|
_WORD_RE = re.compile(r"\w+")
|
||||||
|
# Largest Mersenne prime below 2**61 — the modulus for the universal-hash permutations.
|
||||||
|
_PRIME = (1 << 61) - 1
|
||||||
|
|
||||||
|
|
||||||
|
def shingles(text: str, k: int = 3) -> frozenset[str]:
|
||||||
|
"""The set of word k-grams in ``text`` (lowercased). Short texts fall back to their word set."""
|
||||||
|
words = _WORD_RE.findall(text.lower())
|
||||||
|
if len(words) < k:
|
||||||
|
return frozenset(words)
|
||||||
|
return frozenset(" ".join(words[i : i + k]) for i in range(len(words) - k + 1))
|
||||||
|
|
||||||
|
|
||||||
|
def _stable_hash(token: str) -> int:
|
||||||
|
return int.from_bytes(hashlib.blake2b(token.encode("utf-8"), digest_size=8).digest(), "big")
|
||||||
|
|
||||||
|
|
||||||
|
class MinHasher:
|
||||||
|
"""A bank of ``num_perm`` universal hash permutations producing a fixed-length signature."""
|
||||||
|
|
||||||
|
def __init__(self, num_perm: int = 64, seed: int = 1) -> None:
|
||||||
|
self.num_perm = num_perm
|
||||||
|
rng = random.Random(seed)
|
||||||
|
self._coeffs = [
|
||||||
|
(rng.randrange(1, _PRIME), rng.randrange(0, _PRIME)) for _ in range(num_perm)
|
||||||
|
]
|
||||||
|
|
||||||
|
def signature(self, shingle_set: Iterable[str]) -> tuple[int, ...]:
|
||||||
|
"""The MinHash signature of ``shingle_set`` (empty set → all-``_PRIME`` sentinel)."""
|
||||||
|
hashed = [_stable_hash(s) for s in shingle_set]
|
||||||
|
if not hashed:
|
||||||
|
return tuple(_PRIME for _ in self._coeffs)
|
||||||
|
return tuple(min((a * h + b) % _PRIME for h in hashed) for a, b in self._coeffs)
|
||||||
|
|
||||||
|
|
||||||
|
def band_keys(
|
||||||
|
signature: tuple[int, ...], num_bands: int
|
||||||
|
) -> tuple[tuple[int, tuple[int, ...]], ...]:
|
||||||
|
"""Split a signature into ``num_bands`` band keys; two pages sharing one are LSH candidates."""
|
||||||
|
if num_bands <= 0 or len(signature) % num_bands != 0:
|
||||||
|
raise ValueError(f"signature length {len(signature)} not divisible into {num_bands} bands")
|
||||||
|
rows = len(signature) // num_bands
|
||||||
|
return tuple(
|
||||||
|
(b, signature[b * rows : (b + 1) * rows]) for b in range(num_bands)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def jaccard(a: frozenset[str], b: frozenset[str]) -> float:
|
||||||
|
"""Jaccard similarity of two shingle sets; two empty sets are defined as identical (1.0)."""
|
||||||
|
if not a and not b:
|
||||||
|
return 1.0
|
||||||
|
if not a or not b:
|
||||||
|
return 0.0
|
||||||
|
return len(a & b) / len(a | b)
|
||||||
91
src/shard_wiki/incremental/union_index.py
Normal file
91
src/shard_wiki/incremental/union_index.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
"""UnionIndex — the maintained derived tier wired behind resolution + views (SHARD-WP-0011 T4).
|
||||||
|
|
||||||
|
Wraps a :class:`UnionGraph` + decision log with an incrementally maintained
|
||||||
|
:class:`EquivalenceIndex`. Content equivalence is kept fresh by deltas (``note_change`` /
|
||||||
|
``note_removed``); curator bindings are re-synced live from the log fold. A full :meth:`rebuild`
|
||||||
|
is the bounded fallback. :meth:`verify` runs the I-2 consistency-checker over the live source.
|
||||||
|
|
||||||
|
Consumer-visible results are unchanged — equivalence groups are exposed in the same string form the
|
||||||
|
decision-log fold uses, a *superset* that additionally collapses genuine content duplicates — only
|
||||||
|
freshness and cost differ (recompute-on-read becomes change-driven).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from shard_wiki.coordination import DecisionLog
|
||||||
|
from shard_wiki.incremental.equivalence import EquivalenceIndex
|
||||||
|
from shard_wiki.incremental.verification import (
|
||||||
|
ConsistencyChecker,
|
||||||
|
ConsistencyReport,
|
||||||
|
derived_digest,
|
||||||
|
)
|
||||||
|
from shard_wiki.model import Identity, Page
|
||||||
|
from shard_wiki.union import UnionGraph
|
||||||
|
|
||||||
|
__all__ = ["UnionIndex"]
|
||||||
|
|
||||||
|
|
||||||
|
def _identity(token: str) -> Identity:
|
||||||
|
shard, _, key = token.partition(":")
|
||||||
|
return Identity(shard, key)
|
||||||
|
|
||||||
|
|
||||||
|
class UnionIndex:
|
||||||
|
"""An incrementally maintained equivalence index over a union, with a rebuild fallback."""
|
||||||
|
|
||||||
|
def __init__(self, union: UnionGraph, log: DecisionLog, space: str) -> None:
|
||||||
|
self._union = union
|
||||||
|
self._log = log
|
||||||
|
self._space = space
|
||||||
|
self._eq = EquivalenceIndex()
|
||||||
|
self.rebuild()
|
||||||
|
|
||||||
|
def rebuild(self) -> None:
|
||||||
|
"""The bounded fallback: re-derive the whole index from current union pages + bindings."""
|
||||||
|
self._eq.build(self._union.iter_pages())
|
||||||
|
self._sync_curator()
|
||||||
|
|
||||||
|
def note_change(self, page: Page) -> None:
|
||||||
|
"""Change-driven update for one added/edited page (the operational path)."""
|
||||||
|
self._eq.update(page)
|
||||||
|
|
||||||
|
def note_removed(self, identity: Identity) -> None:
|
||||||
|
self._eq.remove(identity)
|
||||||
|
|
||||||
|
def _sync_curator(self) -> None:
|
||||||
|
"""Re-sync curator equivalence from the live decision-log fold (cheap, always correct)."""
|
||||||
|
groups = self._log.fold(self._space).equivalence_groups
|
||||||
|
edges: list[tuple[Identity, Identity]] = []
|
||||||
|
for group in groups:
|
||||||
|
members = [_identity(m) for m in group]
|
||||||
|
edges.extend((members[0], other) for other in members[1:])
|
||||||
|
self._eq.set_curator_edges(edges)
|
||||||
|
|
||||||
|
def equivalence_groups(self) -> tuple[frozenset[str], ...]:
|
||||||
|
"""Equivalence groups in decision-log string form (curator ∪ content), for the views."""
|
||||||
|
self._sync_curator()
|
||||||
|
return tuple(
|
||||||
|
frozenset(str(identity) for identity in group) for group in self._eq.groups()
|
||||||
|
)
|
||||||
|
|
||||||
|
def digest(self) -> str:
|
||||||
|
"""The Merkle-style digest of the maintained derived tier (I-2)."""
|
||||||
|
self._sync_curator()
|
||||||
|
return derived_digest(self._eq)
|
||||||
|
|
||||||
|
def verify(self) -> ConsistencyReport:
|
||||||
|
"""Check the maintained index against a from-scratch fold of the live source; self-heal."""
|
||||||
|
self._sync_curator()
|
||||||
|
checker = ConsistencyChecker(
|
||||||
|
self._eq,
|
||||||
|
pages=lambda: list(self._union.iter_pages()),
|
||||||
|
curator_edges=self._curator_pairs,
|
||||||
|
)
|
||||||
|
return checker.check_and_repair()
|
||||||
|
|
||||||
|
def _curator_pairs(self) -> list[tuple[Identity, Identity]]:
|
||||||
|
pairs: list[tuple[Identity, Identity]] = []
|
||||||
|
for group in self._log.fold(self._space).equivalence_groups:
|
||||||
|
members = [_identity(m) for m in group]
|
||||||
|
pairs.extend((members[0], other) for other in members[1:])
|
||||||
|
return pairs
|
||||||
112
src/shard_wiki/incremental/verification.py
Normal file
112
src/shard_wiki/incremental/verification.py
Normal file
@@ -0,0 +1,112 @@
|
|||||||
|
"""I-2 verification — digest + background consistency-checker (SHARD-WP-0011 T3).
|
||||||
|
|
||||||
|
``derived = f(canonical)`` is made *verified*, not asserted. A **Merkle-style digest** summarizes
|
||||||
|
the derived tier (each identity's content fingerprint + its incident equivalence edges as a leaf,
|
||||||
|
order-independently combined into a root) so two derived states are equal iff their digests match.
|
||||||
|
A **consistency-checker** recomputes the authoritative fold from the current source, compares it to
|
||||||
|
the maintained index over a (sampled) region, and on mismatch performs a **scoped recompute** of
|
||||||
|
just the affected identities — self-healing drift from a missed delta or corrupted state.
|
||||||
|
|
||||||
|
The digest is a pure function of index state, so it is "maintained alongside deltas" for free and
|
||||||
|
is stable under equivalent event orders (leaves are sorted before combination).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
from collections.abc import Callable, Iterable
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.incremental.equivalence import EquivalenceIndex
|
||||||
|
from shard_wiki.model import Identity, Page
|
||||||
|
|
||||||
|
__all__ = ["region_digest", "derived_digest", "ConsistencyReport", "ConsistencyChecker"]
|
||||||
|
|
||||||
|
CuratorEdges = Iterable[tuple[Identity, Identity]]
|
||||||
|
|
||||||
|
|
||||||
|
def _leaf(index: EquivalenceIndex, identity: Identity) -> str:
|
||||||
|
"""A digest leaf for one identity: its fingerprint + its incident edges (as sorted peers)."""
|
||||||
|
fingerprint = index.fingerprint(identity) or "∅"
|
||||||
|
peers = sorted(
|
||||||
|
str(other)
|
||||||
|
for edge in index.edges()
|
||||||
|
if identity in edge
|
||||||
|
for other in edge
|
||||||
|
if other != identity
|
||||||
|
)
|
||||||
|
payload = f"{identity}|{fingerprint}|{','.join(peers)}"
|
||||||
|
return hashlib.blake2b(payload.encode("utf-8"), digest_size=16).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def region_digest(index: EquivalenceIndex, identities: Iterable[Identity]) -> str:
|
||||||
|
"""A Merkle-style root over the given identities' leaves (order-independent)."""
|
||||||
|
leaves = sorted(_leaf(index, identity) for identity in identities)
|
||||||
|
root = hashlib.blake2b(digest_size=16)
|
||||||
|
for leaf in leaves:
|
||||||
|
root.update(leaf.encode("utf-8"))
|
||||||
|
return root.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def derived_digest(index: EquivalenceIndex) -> str:
|
||||||
|
"""The digest of the whole maintained derived tier."""
|
||||||
|
return region_digest(index, index.identities())
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ConsistencyReport:
|
||||||
|
"""Outcome of a consistency check: what was examined, whether it drifted, and if it healed."""
|
||||||
|
|
||||||
|
checked: int
|
||||||
|
drifted: bool
|
||||||
|
repaired: bool
|
||||||
|
healthy: bool
|
||||||
|
|
||||||
|
|
||||||
|
class ConsistencyChecker:
|
||||||
|
"""Compares the maintained index against an authoritative rebuild and repairs drift in place."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
index: EquivalenceIndex,
|
||||||
|
pages: Callable[[], Iterable[Page]],
|
||||||
|
curator_edges: Callable[[], CuratorEdges] = lambda: (),
|
||||||
|
) -> None:
|
||||||
|
self._index = index
|
||||||
|
self._pages = pages
|
||||||
|
self._curator = curator_edges
|
||||||
|
|
||||||
|
def _authoritative(self) -> EquivalenceIndex:
|
||||||
|
expected = EquivalenceIndex(
|
||||||
|
num_bands=self._index.num_bands, threshold=self._index.threshold
|
||||||
|
)
|
||||||
|
expected.build(list(self._pages()), list(self._curator()))
|
||||||
|
return expected
|
||||||
|
|
||||||
|
def check_and_repair(self, sample: Iterable[Identity] | None = None) -> ConsistencyReport:
|
||||||
|
"""Verify the (sampled) region against a from-scratch fold; scoped-recompute on mismatch."""
|
||||||
|
source = {p.identity: p for p in self._pages()}
|
||||||
|
expected = self._authoritative()
|
||||||
|
region = (
|
||||||
|
set(sample)
|
||||||
|
if sample is not None
|
||||||
|
else set(source) | set(self._index.identities())
|
||||||
|
)
|
||||||
|
|
||||||
|
drifted = region_digest(self._index, region) != region_digest(expected, region)
|
||||||
|
if not drifted:
|
||||||
|
return ConsistencyReport(len(region), drifted=False, repaired=False, healthy=True)
|
||||||
|
|
||||||
|
self._repair(region, source)
|
||||||
|
healthy = region_digest(self._index, region) == region_digest(expected, region)
|
||||||
|
return ConsistencyReport(len(region), drifted=True, repaired=True, healthy=healthy)
|
||||||
|
|
||||||
|
def _repair(self, region: set[Identity], source: dict[Identity, Page]) -> None:
|
||||||
|
"""Scoped recompute: reconcile each affected identity to the current source."""
|
||||||
|
present = self._index.identities()
|
||||||
|
for identity in region:
|
||||||
|
page = source.get(identity)
|
||||||
|
if page is not None:
|
||||||
|
self._index.update(page) if identity in present else self._index.add(page)
|
||||||
|
elif identity in present:
|
||||||
|
self._index.remove(identity)
|
||||||
47
src/shard_wiki/model/__init__.py
Normal file
47
src/shard_wiki/model/__init__.py
Normal file
@@ -0,0 +1,47 @@
|
|||||||
|
"""model/ — the backend-neutral page model and capability types (the top waist).
|
||||||
|
|
||||||
|
Imports only the ``provenance`` leaf; nothing backend-specific lives here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.model.capability import (
|
||||||
|
AccessGrant,
|
||||||
|
Addressing,
|
||||||
|
AttachmentMode,
|
||||||
|
CapabilityProfile,
|
||||||
|
ContentOpacity,
|
||||||
|
History,
|
||||||
|
MergeModel,
|
||||||
|
NativeQuery,
|
||||||
|
NotSupported,
|
||||||
|
OperationalEnvelope,
|
||||||
|
ProfileError,
|
||||||
|
Substrate,
|
||||||
|
Translation,
|
||||||
|
Verb,
|
||||||
|
WriteGranularity,
|
||||||
|
)
|
||||||
|
from shard_wiki.model.identity import Identity, Placement, Span
|
||||||
|
from shard_wiki.model.page import Page, PageShape
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"Identity",
|
||||||
|
"Placement",
|
||||||
|
"Span",
|
||||||
|
"Page",
|
||||||
|
"PageShape",
|
||||||
|
"CapabilityProfile",
|
||||||
|
"Verb",
|
||||||
|
"Substrate",
|
||||||
|
"AttachmentMode",
|
||||||
|
"WriteGranularity",
|
||||||
|
"ContentOpacity",
|
||||||
|
"OperationalEnvelope",
|
||||||
|
"AccessGrant",
|
||||||
|
"History",
|
||||||
|
"MergeModel",
|
||||||
|
"Addressing",
|
||||||
|
"NativeQuery",
|
||||||
|
"Translation",
|
||||||
|
"ProfileError",
|
||||||
|
"NotSupported",
|
||||||
|
]
|
||||||
220
src/shard_wiki/model/capability.py
Normal file
220
src/shard_wiki/model/capability.py
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
"""Capability profile — capability-as-data (CoreArchitectureBlueprint §6, I-3).
|
||||||
|
|
||||||
|
A binding's abilities are a *position on each capability spectrum*, not a boolean checklist.
|
||||||
|
Descriptively there are fifteen spectra (§6.1); operationally they reduce to a small
|
||||||
|
**orthogonal core** plus **implied** axes, with **implication rules that forbid impossible
|
||||||
|
profiles** (§6.5). ``CapabilityProfile.validate()`` enforces those rules; a profile that
|
||||||
|
violates one is rejected (the structural half of "capability-as-data is sound" — the behavioural
|
||||||
|
half is the conformance suite, T4).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
from shard_wiki.provenance import Liveness
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"Verb",
|
||||||
|
"Substrate",
|
||||||
|
"AttachmentMode",
|
||||||
|
"WriteGranularity",
|
||||||
|
"ContentOpacity",
|
||||||
|
"OperationalEnvelope",
|
||||||
|
"AccessGrant",
|
||||||
|
"History",
|
||||||
|
"MergeModel",
|
||||||
|
"Addressing",
|
||||||
|
"NativeQuery",
|
||||||
|
"Translation",
|
||||||
|
"CapabilityProfile",
|
||||||
|
"ProfileError",
|
||||||
|
"NotSupported",
|
||||||
|
]
|
||||||
|
|
||||||
|
CONTRACT_VERSION = "0.1"
|
||||||
|
|
||||||
|
|
||||||
|
class ProfileError(ValueError):
|
||||||
|
"""Raised when a capability profile is internally impossible (violates §6.5 rules)."""
|
||||||
|
|
||||||
|
|
||||||
|
class NotSupported(Exception):
|
||||||
|
"""Raised by an adapter when an operation verb is not in its capability profile."""
|
||||||
|
|
||||||
|
|
||||||
|
class Verb(Enum):
|
||||||
|
READ = "read"
|
||||||
|
WRITE = "write"
|
||||||
|
DIFF = "diff"
|
||||||
|
MERGE = "merge"
|
||||||
|
LOCK = "lock"
|
||||||
|
VERSION = "version"
|
||||||
|
PUBLISH = "publish"
|
||||||
|
NOTIFY = "notify"
|
||||||
|
TRANSCLUDE_SOURCE = "transclude-source"
|
||||||
|
TRANSLATE_SYNTAX = "translate-syntax"
|
||||||
|
STRUCTURED_PAYLOAD = "structured-payload"
|
||||||
|
DERIVE_PROJECTION = "derive-projection" # gated, off by default (T18)
|
||||||
|
EXECUTE = "execute" # gated, off by default (T18)
|
||||||
|
|
||||||
|
|
||||||
|
# --- orthogonal core axes (§6.5a) ---
|
||||||
|
class Substrate(Enum):
|
||||||
|
GIT = "git"
|
||||||
|
FILES = "files"
|
||||||
|
RELATIONAL_DB = "relational-db"
|
||||||
|
EXTERNAL_API = "external-api"
|
||||||
|
CRDT = "crdt"
|
||||||
|
|
||||||
|
|
||||||
|
class WriteGranularity(Enum):
|
||||||
|
NONE = "none" # read-only
|
||||||
|
WHOLE_FILE = "whole-file"
|
||||||
|
SECTION = "section"
|
||||||
|
PER_PAGE = "per-page"
|
||||||
|
PER_BLOCK = "per-block"
|
||||||
|
|
||||||
|
|
||||||
|
class ContentOpacity(Enum):
|
||||||
|
TRANSPARENT = "transparent"
|
||||||
|
STRUCTURED_REEVALUABLE = "structured-re-evaluable"
|
||||||
|
ENCRYPTED = "encrypted"
|
||||||
|
PER_ITEM = "per-item"
|
||||||
|
PROPRIETARY_LOSSY = "proprietary-lossy"
|
||||||
|
|
||||||
|
|
||||||
|
class OperationalEnvelope(Enum):
|
||||||
|
LOCAL_UNBOUNDED = "local-unbounded"
|
||||||
|
REALTIME = "realtime"
|
||||||
|
RATE_LIMITED = "rate-limited"
|
||||||
|
|
||||||
|
|
||||||
|
class AccessGrant(Enum):
|
||||||
|
OPEN = "open"
|
||||||
|
TOKEN = "token"
|
||||||
|
OAUTH_SCOPED = "oauth-scoped"
|
||||||
|
P2P_KEY = "p2p-key"
|
||||||
|
ENTERPRISE_ACL = "enterprise-acl"
|
||||||
|
|
||||||
|
|
||||||
|
# --- implied axes (§6.5b): may be derived from the core, validated for consistency ---
|
||||||
|
class History(Enum):
|
||||||
|
NONE = "none"
|
||||||
|
INTERNAL_ONLY = "internal-only"
|
||||||
|
CRDT_LOG = "crdt-log"
|
||||||
|
OPEN_FILE = "open-file"
|
||||||
|
GIT_NATIVE = "git-native"
|
||||||
|
|
||||||
|
|
||||||
|
class MergeModel(Enum):
|
||||||
|
NONE = "none"
|
||||||
|
GIT_TEXT = "git-text"
|
||||||
|
KEEP_BOTH = "keep-both"
|
||||||
|
NATIVE_CRDT = "native-crdt"
|
||||||
|
COEXIST_RANK = "coexist-rank"
|
||||||
|
|
||||||
|
|
||||||
|
class AttachmentMode(Enum):
|
||||||
|
# NB: there is deliberately no IMAGE mode — an image/live-memory blob is never an attach
|
||||||
|
# target (I-12). It participates only via export→files.
|
||||||
|
FILE_STORE = "file-store"
|
||||||
|
GIT_IS_STORE = "git-is-store"
|
||||||
|
IN_ENGINE_HOST = "in-engine-host"
|
||||||
|
LOCAL_REST = "local-rest"
|
||||||
|
EXTERNAL_API = "external-api"
|
||||||
|
DIRECT_DB = "direct-db"
|
||||||
|
CRDT_REPLICA = "crdt-replica"
|
||||||
|
P2P = "p2p"
|
||||||
|
|
||||||
|
|
||||||
|
class Addressing(Enum):
|
||||||
|
NONE = "none"
|
||||||
|
PATH = "path"
|
||||||
|
PAGE_ID = "page-id"
|
||||||
|
SPAN = "span"
|
||||||
|
BLOCK_ID = "block-id"
|
||||||
|
STORE_UUID = "store-uuid"
|
||||||
|
TUMBLER = "tumbler"
|
||||||
|
|
||||||
|
|
||||||
|
class NativeQuery(Enum):
|
||||||
|
NONE = "none"
|
||||||
|
TEXT = "text"
|
||||||
|
DERIVED_INDEX = "derived-index"
|
||||||
|
DATALOG_GRAPH = "datalog-graph"
|
||||||
|
DB_QUERY = "db-query"
|
||||||
|
SPARQL = "sparql"
|
||||||
|
|
||||||
|
|
||||||
|
class Translation(Enum):
|
||||||
|
NATIVE = "native"
|
||||||
|
LOSSLESS = "lossless"
|
||||||
|
LOSSY_WITH_REPORT = "lossy-with-report"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class CapabilityProfile:
|
||||||
|
"""A verified-at-registration position on each capability axis, plus supported verbs."""
|
||||||
|
|
||||||
|
# orthogonal core
|
||||||
|
substrate: Substrate
|
||||||
|
attachment_mode: AttachmentMode
|
||||||
|
write_granularity: WriteGranularity
|
||||||
|
content_opacity: ContentOpacity
|
||||||
|
operational_envelope: OperationalEnvelope
|
||||||
|
access_grant: AccessGrant
|
||||||
|
liveness: Liveness # computational/liveness axis (15th)
|
||||||
|
# implied
|
||||||
|
history: History
|
||||||
|
merge_model: MergeModel
|
||||||
|
addressing: Addressing
|
||||||
|
native_query: NativeQuery
|
||||||
|
translation: Translation
|
||||||
|
supported_verbs: frozenset[Verb] = field(default_factory=frozenset)
|
||||||
|
contract_version: str = CONTRACT_VERSION
|
||||||
|
|
||||||
|
def supports(self, verb: Verb) -> bool:
|
||||||
|
return verb in self.supported_verbs
|
||||||
|
|
||||||
|
def validate(self) -> CapabilityProfile:
|
||||||
|
"""Apply the §6.5 implication rules; raise :class:`ProfileError` on an impossible
|
||||||
|
combination. Returns ``self`` so it can be used fluently at construction."""
|
||||||
|
rules: list[tuple[bool, str]] = [
|
||||||
|
(
|
||||||
|
Verb.READ in self.supported_verbs,
|
||||||
|
"every shard must support READ (the capability floor)",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
self.attachment_mode is not AttachmentMode.GIT_IS_STORE
|
||||||
|
or (self.substrate is Substrate.GIT and self.history is History.GIT_NATIVE),
|
||||||
|
"git-is-store ⟹ substrate=git ∧ history=git-native",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
self.content_opacity is not ContentOpacity.ENCRYPTED
|
||||||
|
or self.native_query is NativeQuery.NONE,
|
||||||
|
"encrypted opacity ⟹ native-query=none",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
self.merge_model is not MergeModel.NATIVE_CRDT
|
||||||
|
or (
|
||||||
|
self.history is History.CRDT_LOG
|
||||||
|
and self.operational_envelope is OperationalEnvelope.REALTIME
|
||||||
|
),
|
||||||
|
"native-crdt merge ⟹ history=crdt-log ∧ envelope=realtime",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
(self.write_granularity is WriteGranularity.NONE)
|
||||||
|
== (Verb.WRITE not in self.supported_verbs),
|
||||||
|
"write-granularity=none ⟺ WRITE not supported (read-only consistency)",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
Verb.MERGE not in self.supported_verbs or self.merge_model is not MergeModel.NONE,
|
||||||
|
"MERGE verb ⟹ merge-model ≠ none",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
broken = [msg for ok, msg in rules if not ok]
|
||||||
|
if broken:
|
||||||
|
raise ProfileError("; ".join(broken))
|
||||||
|
return self
|
||||||
60
src/shard_wiki/model/identity.py
Normal file
60
src/shard_wiki/model/identity.py
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
"""Identity, Placement, Span — three distinct concepts (CoreArchitectureBlueprint §7.2).
|
||||||
|
|
||||||
|
- **Identity** is a *stable handle* assigned/minted by a shard; it survives edits. It is NEVER
|
||||||
|
derived from content (a content fingerprint identifies a *version*, not a *page*).
|
||||||
|
- **Placement** is *where* an identity sits (path in a shard); one identity may have many
|
||||||
|
placements (a DAG). Placement can change without changing identity.
|
||||||
|
- **Span** is a sub-page address within an identity, optionally carrying a provenance delta.
|
||||||
|
|
||||||
|
Equivalence (detecting that two *distinct* identities hold the same content) is a separate
|
||||||
|
mechanism and lives in ``union/`` — not here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.provenance import SpanProvenanceDelta
|
||||||
|
|
||||||
|
__all__ = ["Identity", "Placement", "Span"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Identity:
|
||||||
|
"""A shard-scoped, stable page handle. Value-equal; stable across content edits.
|
||||||
|
|
||||||
|
``shard`` scopes the handle so native ids survive projection and never collide across
|
||||||
|
shards. ``key`` is the backend's stable handle (page name / native uid) — not a fingerprint.
|
||||||
|
"""
|
||||||
|
|
||||||
|
shard: str
|
||||||
|
key: str
|
||||||
|
|
||||||
|
def __str__(self) -> str:
|
||||||
|
return f"{self.shard}:{self.key}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Placement:
|
||||||
|
"""Where an identity sits: a path within a shard. Mutable over an identity's life."""
|
||||||
|
|
||||||
|
shard: str
|
||||||
|
path: str
|
||||||
|
|
||||||
|
def __str__(self) -> str:
|
||||||
|
return f"{self.shard}/{self.path}"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Span:
|
||||||
|
"""A sub-page address within a page identity, with an optional provenance delta.
|
||||||
|
|
||||||
|
``ref`` is a native span id where the backend mints one (Roam ``:block/uid``, Logseq
|
||||||
|
``id::``), else a positional/fingerprint address. ``delta`` overrides the page envelope
|
||||||
|
only where this span genuinely differs (effective-vs-own, blueprint §7.3).
|
||||||
|
"""
|
||||||
|
|
||||||
|
ref: str
|
||||||
|
start: int | None = None
|
||||||
|
end: int | None = None
|
||||||
|
delta: SpanProvenanceDelta | None = None
|
||||||
43
src/shard_wiki/model/page.py
Normal file
43
src/shard_wiki/model/page.py
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
"""The backend-neutral wiki page model — the top narrow waist (CoreArchitectureBlueprint §7).
|
||||||
|
|
||||||
|
Markdown-first but stretchable: every shape reduces to
|
||||||
|
``(content|source, structure, provenance envelope, [derivation rule])``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
from shard_wiki.model.identity import Identity, Placement, Span
|
||||||
|
from shard_wiki.provenance import ProvenanceEnvelope
|
||||||
|
|
||||||
|
__all__ = ["PageShape", "Page"]
|
||||||
|
|
||||||
|
|
||||||
|
class PageShape(Enum):
|
||||||
|
"""The page-model shapes (blueprint §7.1). The slice handles PROSE; the rest are declared
|
||||||
|
so adapters can tag content faithfully (union without erasure)."""
|
||||||
|
|
||||||
|
PROSE = "prose"
|
||||||
|
TYPED_RECORD = "typed-record"
|
||||||
|
TYPED_GRAPH = "typed-graph"
|
||||||
|
INLINE_EMBEDDED = "inline-embedded"
|
||||||
|
NON_MARKDOWN_ASSET = "non-markdown-asset"
|
||||||
|
ONE_SOURCE_MANY_PROJECTIONS = "one-source-many-projections"
|
||||||
|
NOTEBOOK = "notebook"
|
||||||
|
PROGRAM_AS_PAGE = "program-as-page"
|
||||||
|
LIVE_TEMPORAL = "live-temporal"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Page:
|
||||||
|
"""A page in the union: a stable identity, content, the page-level provenance envelope,
|
||||||
|
its placements, and its spans (whose deltas layer over the page envelope, §7.3)."""
|
||||||
|
|
||||||
|
identity: Identity
|
||||||
|
body: str
|
||||||
|
envelope: ProvenanceEnvelope
|
||||||
|
shape: PageShape = PageShape.PROSE
|
||||||
|
placements: tuple[Placement, ...] = ()
|
||||||
|
spans: tuple[Span, ...] = field(default_factory=tuple)
|
||||||
39
src/shard_wiki/policy/__init__.py
Normal file
39
src/shard_wiki/policy/__init__.py
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
"""policy/ — the configurable policy surface, a dependency-free leaf (blueprint §10, §11).
|
||||||
|
|
||||||
|
Mechanism over policy (I-7): core mechanisms read policy *choices* from here; they never
|
||||||
|
hard-code one. This leaf holds only the presets + a pure ``resolve``-style selector. Mechanism
|
||||||
|
(how a choice is honoured) lives in ``coordination``/``union``/``projection``, never here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
__all__ = ["CanonicalSource", "Policy", "DEFAULT_POLICY"]
|
||||||
|
|
||||||
|
|
||||||
|
class CanonicalSource(Enum):
|
||||||
|
"""Resolution policy over a divergent/equivalent set (FederationArchitecture T9). Detection
|
||||||
|
is core; this is only the *resolution* choice."""
|
||||||
|
|
||||||
|
CHORUS = "chorus" # default: present all versions, none privileged
|
||||||
|
DESIGNATED_CANONICAL = "designated-canonical"
|
||||||
|
GIT_MERGE = "git-merge"
|
||||||
|
VOTE = "vote"
|
||||||
|
OVERLAY_ONLY = "overlay-only"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Policy:
|
||||||
|
"""A space's policy choices. Extended as the policy surface grows (freshness, compaction,
|
||||||
|
execution, tenant mapping — blueprint §10); the slice needs canonical-source + designation."""
|
||||||
|
|
||||||
|
canonical_source: CanonicalSource = CanonicalSource.CHORUS
|
||||||
|
designated_shard: str | None = None
|
||||||
|
# Whether an unapplied overlay's body is projected over its canonical page on read. Either
|
||||||
|
# way the overlay is never *hidden* — overlay_state is always surfaced in provenance.
|
||||||
|
show_drafts: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
DEFAULT_POLICY = Policy()
|
||||||
116
src/shard_wiki/provenance/__init__.py
Normal file
116
src/shard_wiki/provenance/__init__.py
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
"""Provenance — the cross-cutting leaf rail (CoreArchitectureBlueprint §7.3, §11).
|
||||||
|
|
||||||
|
Every artifact in the union carries a :class:`ProvenanceEnvelope`. To keep per-span cost near
|
||||||
|
zero when provenance is uniform, envelopes are **layered**: a page-level envelope plus optional
|
||||||
|
per-span deltas. The *effective* provenance of a span is ``page_envelope ⊕ span_delta`` — a span
|
||||||
|
with no delta inherits the page envelope at zero cost ("effective-vs-own", the same pattern the
|
||||||
|
page model uses for computed metadata).
|
||||||
|
|
||||||
|
This module is a **dependency-free leaf**: it imports nothing else in the package and contains
|
||||||
|
only stable data types plus the pure ``effective`` composition. Mechanism never lives here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"Liveness",
|
||||||
|
"Staleness",
|
||||||
|
"OverlayState",
|
||||||
|
"ProvenanceEnvelope",
|
||||||
|
"SpanProvenanceDelta",
|
||||||
|
"effective",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Liveness(Enum):
|
||||||
|
"""Where a view sits on the live↔snapshot axis (blueprint §8, T18)."""
|
||||||
|
|
||||||
|
STATIC = "static"
|
||||||
|
CAPTURED_SNAPSHOT = "captured-snapshot"
|
||||||
|
LIVE_OVER_FILES = "live-over-files"
|
||||||
|
VIEW_TIME = "view-time"
|
||||||
|
IRREDUCIBLY_LIVE = "irreducibly-live"
|
||||||
|
|
||||||
|
|
||||||
|
class Staleness(Enum):
|
||||||
|
"""Freshness state of cached/projected content (blueprint §8.8). ``UNAVAILABLE`` is the
|
||||||
|
dead-shard state (FederationRequirements ADR-04)."""
|
||||||
|
|
||||||
|
LIVE = "live"
|
||||||
|
FRESH = "fresh"
|
||||||
|
STALE = "stale"
|
||||||
|
UNAVAILABLE = "unavailable"
|
||||||
|
|
||||||
|
|
||||||
|
class OverlayState(Enum):
|
||||||
|
"""Overlay lifecycle state of an artifact (FederationRequirements ADR-05)."""
|
||||||
|
|
||||||
|
NONE = "none"
|
||||||
|
DRAFT = "draft"
|
||||||
|
PATCH_PENDING = "patch-pending"
|
||||||
|
APPLIED = "applied"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ProvenanceEnvelope:
|
||||||
|
"""The metadata every page/span carries — union without erasure (INTENT I-4).
|
||||||
|
|
||||||
|
This is the *page-level* (or fully-resolved *effective*) form. Per-span overrides are
|
||||||
|
expressed as a :class:`SpanProvenanceDelta` and composed via :func:`effective`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
source_shard: str
|
||||||
|
liveness: Liveness = Liveness.STATIC
|
||||||
|
staleness: Staleness = Staleness.FRESH
|
||||||
|
overlay_state: OverlayState = OverlayState.NONE
|
||||||
|
source_rev: str | None = None
|
||||||
|
observed_at: datetime | None = None
|
||||||
|
authz_context: str | None = None
|
||||||
|
divergence: tuple[str, ...] = ()
|
||||||
|
lineage: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class SpanProvenanceDelta:
|
||||||
|
"""A per-span override of a page envelope. Every field is optional; ``None`` (or, for
|
||||||
|
``divergence``, the sentinel default) means *inherit the page-level value*. A span with an
|
||||||
|
all-default delta — or no delta at all — costs nothing and resolves to the page envelope.
|
||||||
|
"""
|
||||||
|
|
||||||
|
source_shard: str | None = None
|
||||||
|
liveness: Liveness | None = None
|
||||||
|
staleness: Staleness | None = None
|
||||||
|
overlay_state: OverlayState | None = None
|
||||||
|
source_rev: str | None = None
|
||||||
|
observed_at: datetime | None = None
|
||||||
|
authz_context: str | None = None
|
||||||
|
# Sentinel: ``None`` means inherit; an explicit (possibly empty) tuple overrides.
|
||||||
|
divergence: tuple[str, ...] | None = None
|
||||||
|
lineage: str | None = None
|
||||||
|
|
||||||
|
def is_empty(self) -> bool:
|
||||||
|
"""True when the delta overrides nothing (the common, zero-cost case)."""
|
||||||
|
return all(getattr(self, f.name) is None for f in dataclasses.fields(self))
|
||||||
|
|
||||||
|
|
||||||
|
def effective(
|
||||||
|
base: ProvenanceEnvelope, delta: SpanProvenanceDelta | None = None
|
||||||
|
) -> ProvenanceEnvelope:
|
||||||
|
"""Compose a page envelope with an optional span delta → the span's effective provenance.
|
||||||
|
|
||||||
|
``effective(base, None)`` and ``effective(base, <empty delta>)`` both return ``base``
|
||||||
|
unchanged (zero-cost inheritance). Any non-``None`` delta field overrides ``base``.
|
||||||
|
"""
|
||||||
|
if delta is None or delta.is_empty():
|
||||||
|
return base
|
||||||
|
overrides = {
|
||||||
|
f.name: value
|
||||||
|
for f in dataclasses.fields(delta)
|
||||||
|
if (value := getattr(delta, f.name)) is not None
|
||||||
|
}
|
||||||
|
return dataclasses.replace(base, **overrides)
|
||||||
148
src/shard_wiki/space.py
Normal file
148
src/shard_wiki/space.py
Normal file
@@ -0,0 +1,148 @@
|
|||||||
|
"""InformationSpace — the thin orchestrator entry tying the slice together (blueprint §3 L6).
|
||||||
|
|
||||||
|
A root entity / information space: shards attach to it (conformance-gated, TSD §A.2), a
|
||||||
|
coordination decision log records its canonical decisions, and a derived union resolves names to
|
||||||
|
pages. This is the L6 consumer surface for the foundation slice (attach → resolve → read);
|
||||||
|
a network API is a later workplan.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.adapters import ShardAdapter, assert_conformant
|
||||||
|
from shard_wiki.coordination import (
|
||||||
|
ApplyResult,
|
||||||
|
DecisionLog,
|
||||||
|
EventStore,
|
||||||
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
|
Overlay,
|
||||||
|
OverlayEngine,
|
||||||
|
)
|
||||||
|
from shard_wiki.incremental import ConsistencyReport, UnionIndex
|
||||||
|
from shard_wiki.model import Page
|
||||||
|
from shard_wiki.policy import DEFAULT_POLICY, Policy
|
||||||
|
from shard_wiki.union import Resolution, UnionGraph
|
||||||
|
from shard_wiki.views import (
|
||||||
|
AllPagesEntry,
|
||||||
|
BackLink,
|
||||||
|
ChangeEntry,
|
||||||
|
SiteMapNode,
|
||||||
|
all_pages,
|
||||||
|
build_backlinks,
|
||||||
|
recent_changes,
|
||||||
|
site_map,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = ["InformationSpace"]
|
||||||
|
|
||||||
|
|
||||||
|
class InformationSpace:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
space_id: str,
|
||||||
|
policy: Policy = DEFAULT_POLICY,
|
||||||
|
*,
|
||||||
|
store: EventStore | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
|
||||||
|
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
|
||||||
|
self.space_id = space_id
|
||||||
|
self.log = DecisionLog(store)
|
||||||
|
self.union = UnionGraph(space_id, log=self.log, policy=policy)
|
||||||
|
self.overlays = OverlayEngine(space_id, self.log)
|
||||||
|
self._index: UnionIndex | None = None # maintained derived tier, built lazily
|
||||||
|
self._index_stale = True
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def git_backed(
|
||||||
|
cls,
|
||||||
|
space_id: str,
|
||||||
|
repo_path: str | Path,
|
||||||
|
policy: Policy = DEFAULT_POLICY,
|
||||||
|
) -> InformationSpace:
|
||||||
|
"""An information space whose coordination log is git-addressable (history/patch/review/
|
||||||
|
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
|
||||||
|
return cls(space_id, policy, store=GitEventStore(repo_path))
|
||||||
|
|
||||||
|
def attach(self, adapter: ShardAdapter) -> None:
|
||||||
|
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
|
||||||
|
assert_conformant(adapter)
|
||||||
|
self.union.attach(adapter)
|
||||||
|
self._index_stale = True
|
||||||
|
|
||||||
|
def alias(self, name: str, target: str, actor: str | None = None) -> None:
|
||||||
|
"""Record a coordination-canonical alias (``name`` → ``"shard:key"``) in the log."""
|
||||||
|
self.log.append(
|
||||||
|
self.space_id, EventType.ALIAS_SET, {"alias": name, "target": target}, actor=actor
|
||||||
|
)
|
||||||
|
|
||||||
|
def resolve(self, name: str) -> Resolution:
|
||||||
|
return self.union.resolve(name)
|
||||||
|
|
||||||
|
def read(self, name: str) -> Page:
|
||||||
|
"""Resolve and return the page (or the canonical pick of a chorus). KeyError if red-link."""
|
||||||
|
return self.union.resolve(name).single()
|
||||||
|
|
||||||
|
def overlay(self, name: str, body: str, actor: str | None = None) -> Overlay:
|
||||||
|
"""Draft a non-destructive overlay against the resolved page (overlay-before-mutation)."""
|
||||||
|
page = self.read(name)
|
||||||
|
return self.overlays.draft(page.identity, body, page.envelope.source_rev, actor=actor)
|
||||||
|
|
||||||
|
def apply_overlay(self, overlay_id: str) -> ApplyResult:
|
||||||
|
"""Apply a draft overlay to its target shard (apply-under-drift, §8.6)."""
|
||||||
|
overlay = self.overlays.get(overlay_id)
|
||||||
|
if overlay is None:
|
||||||
|
raise KeyError(overlay_id)
|
||||||
|
adapter = self.union.shard(overlay.target.shard)
|
||||||
|
if adapter is None:
|
||||||
|
raise KeyError(overlay.target.shard)
|
||||||
|
return self.overlays.apply(overlay_id, adapter)
|
||||||
|
|
||||||
|
def edit(self, name: str, body: str, actor: str | None = None) -> ApplyResult:
|
||||||
|
"""Edit a page through the one principled path: draft an overlay, then apply it. A
|
||||||
|
write-through-capable target fast-forwards (write-through); a read-only target keeps the
|
||||||
|
draft as local truth (I-5: overlay before mutation, always)."""
|
||||||
|
overlay = self.overlay(name, body, actor=actor)
|
||||||
|
result = self.apply_overlay(overlay.overlay_id)
|
||||||
|
self._index_stale = True # the applied edit changes the derived tier
|
||||||
|
return result
|
||||||
|
|
||||||
|
# --- maintained derived tier (SHARD-WP-0011): incremental-first, rebuild as fallback ---
|
||||||
|
|
||||||
|
@property
|
||||||
|
def index(self) -> UnionIndex:
|
||||||
|
"""The maintained equivalence index (built lazily; rebuilt when the union has changed)."""
|
||||||
|
if self._index is None:
|
||||||
|
self._index = UnionIndex(self.union, self.log, self.space_id)
|
||||||
|
elif self._index_stale:
|
||||||
|
self._index.rebuild() # bounded fallback after a mutation
|
||||||
|
self._index_stale = False
|
||||||
|
return self._index
|
||||||
|
|
||||||
|
def reindex(self) -> None:
|
||||||
|
"""Force a full rebuild of the maintained derived tier (the explicit fallback path)."""
|
||||||
|
self.index.rebuild()
|
||||||
|
|
||||||
|
def verify_index(self) -> ConsistencyReport:
|
||||||
|
"""Run the I-2 consistency-checker over the maintained tier; self-heal any drift."""
|
||||||
|
return self.index.verify()
|
||||||
|
|
||||||
|
# --- derived views (SHARD-WP-0010): recomputable, provenance-carrying, presentation-free ---
|
||||||
|
|
||||||
|
def backlinks(self, name: str, *, camelcase: bool = False) -> tuple[BackLink, ...]:
|
||||||
|
"""Pages across the union that link to ``name`` (UC-18)."""
|
||||||
|
return build_backlinks(self.union, camelcase=camelcase).to(name)
|
||||||
|
|
||||||
|
def recent_changes(self, *, limit: int | None = None) -> tuple[ChangeEntry, ...]:
|
||||||
|
"""The merged newest-first change feed: coordination journal + shard signals (UC-17)."""
|
||||||
|
return recent_changes(self.union, self.log, self.space_id, limit=limit)
|
||||||
|
|
||||||
|
def all_pages(self) -> tuple[AllPagesEntry, ...]:
|
||||||
|
"""The union's distinct pages, collapsed via the maintained equivalence index."""
|
||||||
|
return all_pages(self.union, equivalence_groups=self.index.equivalence_groups())
|
||||||
|
|
||||||
|
def site_map(self) -> SiteMapNode:
|
||||||
|
"""The union namespace tree built from page placements."""
|
||||||
|
return site_map(self.union)
|
||||||
8
src/shard_wiki/union/__init__.py
Normal file
8
src/shard_wiki/union/__init__.py
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
"""union/ — the derived-tier union (resolution, equivalence, projection read path).
|
||||||
|
|
||||||
|
Disposable/recomputable (I-2); imports down only and is imported by nothing.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.union.resolver import Resolution, ResolutionKind, UnionGraph
|
||||||
|
|
||||||
|
__all__ = ["UnionGraph", "Resolution", "ResolutionKind"]
|
||||||
170
src/shard_wiki/union/resolver.py
Normal file
170
src/shard_wiki/union/resolver.py
Normal file
@@ -0,0 +1,170 @@
|
|||||||
|
"""Union resolution — the derived-tier read path (CoreArchitectureBlueprint §8.4, ADR-01).
|
||||||
|
|
||||||
|
A minimal :class:`UnionGraph` over ≥1 attached shard plus the decision-log fold. ``resolve``
|
||||||
|
keys on **identity** (FederationRequirements ADR-01): alias redirect → exact union lookup →
|
||||||
|
equivalence chorus → red-link. Ambiguity returns a **chorus set** (union without erasure, I-4):
|
||||||
|
divergent peers are recorded in each page's provenance envelope rather than silently dropped.
|
||||||
|
|
||||||
|
This is derived/disposable: it reads shards + the log fold; it stores nothing canonical. Per the
|
||||||
|
dependency rule it may import down (model/adapters/coordination/policy/provenance) and is
|
||||||
|
imported by nothing.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import dataclasses
|
||||||
|
from collections.abc import Iterator
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
from shard_wiki.adapters import ShardAdapter
|
||||||
|
from shard_wiki.coordination import DecisionLog, Overlay
|
||||||
|
from shard_wiki.model import Identity, Page
|
||||||
|
from shard_wiki.policy import DEFAULT_POLICY, CanonicalSource, Policy
|
||||||
|
from shard_wiki.provenance import OverlayState, ProvenanceEnvelope, Staleness
|
||||||
|
|
||||||
|
__all__ = ["ResolutionKind", "Resolution", "UnionGraph"]
|
||||||
|
|
||||||
|
|
||||||
|
class ResolutionKind(Enum):
|
||||||
|
SINGLE = "single"
|
||||||
|
CHORUS = "chorus"
|
||||||
|
RED_LINK = "red-link"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class Resolution:
|
||||||
|
name: str
|
||||||
|
kind: ResolutionKind
|
||||||
|
pages: tuple[Page, ...] = ()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_red_link(self) -> bool:
|
||||||
|
return self.kind is ResolutionKind.RED_LINK
|
||||||
|
|
||||||
|
def single(self) -> Page:
|
||||||
|
"""The one page (SINGLE), or the canonical pick of a CHORUS (first), else KeyError."""
|
||||||
|
if not self.pages:
|
||||||
|
raise KeyError(self.name)
|
||||||
|
return self.pages[0]
|
||||||
|
|
||||||
|
|
||||||
|
class UnionGraph:
|
||||||
|
"""Composes attached shards + the coordination-log fold into a resolvable union."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
space: str,
|
||||||
|
log: DecisionLog | None = None,
|
||||||
|
policy: Policy = DEFAULT_POLICY,
|
||||||
|
) -> None:
|
||||||
|
self.space = space
|
||||||
|
self.log = log or DecisionLog()
|
||||||
|
self.policy = policy
|
||||||
|
self._shards: list[ShardAdapter] = []
|
||||||
|
|
||||||
|
def attach(self, adapter: ShardAdapter) -> None:
|
||||||
|
self._shards.append(adapter)
|
||||||
|
|
||||||
|
def shard(self, shard_id: str) -> ShardAdapter | None:
|
||||||
|
return next((s for s in self._shards if s.shard_id == shard_id), None)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shards(self) -> tuple[ShardAdapter, ...]:
|
||||||
|
return tuple(self._shards)
|
||||||
|
|
||||||
|
def iter_pages(self) -> Iterator[Page]:
|
||||||
|
"""Every page across attached shards, raw (per-shard, not chorus-collapsed). The
|
||||||
|
enumeration substrate for derived views — BackLinks, AllPages, SiteMap (§8.4)."""
|
||||||
|
for shard in self._shards:
|
||||||
|
for key in shard.keys():
|
||||||
|
try:
|
||||||
|
yield shard.read(key)
|
||||||
|
except KeyError:
|
||||||
|
continue
|
||||||
|
|
||||||
|
def _read_all(self, key: str) -> list[Page]:
|
||||||
|
pages: list[Page] = []
|
||||||
|
for shard in self._shards:
|
||||||
|
try:
|
||||||
|
pages.append(shard.read(key))
|
||||||
|
except KeyError:
|
||||||
|
continue
|
||||||
|
return pages
|
||||||
|
|
||||||
|
def resolve(self, name: str) -> Resolution:
|
||||||
|
state = self.log.fold(self.space)
|
||||||
|
overlays = {
|
||||||
|
Overlay.from_payload(p).target: Overlay.from_payload(p)
|
||||||
|
for p in state.open_overlays.values()
|
||||||
|
}
|
||||||
|
|
||||||
|
# 1. alias redirect (coordination-canonical, via the log fold)
|
||||||
|
target = state.resolve_alias(name)
|
||||||
|
if target is not None and ":" in target:
|
||||||
|
shard_id, _, key = target.partition(":")
|
||||||
|
shard = self.shard(shard_id)
|
||||||
|
if shard is not None:
|
||||||
|
try:
|
||||||
|
page = self._with_overlay(shard.read(key), overlays)
|
||||||
|
return Resolution(name, ResolutionKind.SINGLE, (page,))
|
||||||
|
except KeyError:
|
||||||
|
pass # alias dangles → fall through to normal resolution
|
||||||
|
|
||||||
|
# 2/3. union lookup by key across shards, then layer open overlays
|
||||||
|
pages = [self._with_overlay(p, overlays) for p in self._read_all(name)]
|
||||||
|
pages.extend(self._draft_only_pages(name, pages, overlays))
|
||||||
|
|
||||||
|
if not pages:
|
||||||
|
return Resolution(name, ResolutionKind.RED_LINK)
|
||||||
|
if len(pages) == 1:
|
||||||
|
return Resolution(name, ResolutionKind.SINGLE, (pages[0],))
|
||||||
|
|
||||||
|
# ambiguity → chorus, with divergence recorded (never erased, I-4)
|
||||||
|
ordered = self._order_for_policy(pages)
|
||||||
|
marked = tuple(_with_divergence(p, ordered) for p in ordered)
|
||||||
|
return Resolution(name, ResolutionKind.CHORUS, marked)
|
||||||
|
|
||||||
|
def _with_overlay(self, page: Page, overlays: dict[Identity, Overlay]) -> Page:
|
||||||
|
"""Mark a canonical page that has an open overlay (overlay_state DRAFT; project the
|
||||||
|
overlaid body where policy shows drafts) — never hides the overlay (I-4)."""
|
||||||
|
overlay = overlays.get(page.identity)
|
||||||
|
if overlay is None:
|
||||||
|
return page
|
||||||
|
env = dataclasses.replace(page.envelope, overlay_state=OverlayState.DRAFT)
|
||||||
|
body = overlay.body if self.policy.show_drafts else page.body
|
||||||
|
return dataclasses.replace(page, body=body, envelope=env)
|
||||||
|
|
||||||
|
def _draft_only_pages(
|
||||||
|
self, name: str, existing: list[Page], overlays: dict[Identity, Overlay]
|
||||||
|
) -> list[Page]:
|
||||||
|
"""Drafts that create a not-yet-existing page on an attached shard become resolvable."""
|
||||||
|
have = {p.identity for p in existing}
|
||||||
|
out: list[Page] = []
|
||||||
|
for identity, overlay in overlays.items():
|
||||||
|
if identity.key != name or identity in have or self.shard(identity.shard) is None:
|
||||||
|
continue
|
||||||
|
env = ProvenanceEnvelope(
|
||||||
|
source_shard=identity.shard,
|
||||||
|
staleness=Staleness.FRESH,
|
||||||
|
overlay_state=OverlayState.DRAFT,
|
||||||
|
)
|
||||||
|
body = overlay.body if self.policy.show_drafts else ""
|
||||||
|
out.append(Page(identity=identity, body=body, envelope=env))
|
||||||
|
return out
|
||||||
|
|
||||||
|
def _order_for_policy(self, pages: list[Page]) -> list[Page]:
|
||||||
|
if (
|
||||||
|
self.policy.canonical_source is CanonicalSource.DESIGNATED_CANONICAL
|
||||||
|
and self.policy.designated_shard is not None
|
||||||
|
):
|
||||||
|
pages = sorted(
|
||||||
|
pages, key=lambda p: p.identity.shard != self.policy.designated_shard
|
||||||
|
)
|
||||||
|
return pages
|
||||||
|
|
||||||
|
|
||||||
|
def _with_divergence(page: Page, group: list[Page]) -> Page:
|
||||||
|
peers = tuple(str(p.identity) for p in group if p.identity != page.identity)
|
||||||
|
new_env = dataclasses.replace(page.envelope, divergence=peers)
|
||||||
|
return dataclasses.replace(page, envelope=new_env)
|
||||||
33
src/shard_wiki/views/__init__.py
Normal file
33
src/shard_wiki/views/__init__.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
"""views/ — derived, recomputable, provenance-carrying read views over the union (§8.4).
|
||||||
|
|
||||||
|
All views here are *derived tier*: pure functions of the attached shards plus the coordination-log
|
||||||
|
fold, storing nothing canonical (SHARD-WP-0011 makes them incrementally maintainable). Presentation
|
||||||
|
stays out of core (L6) — these produce models, never rendered output. Per the dependency rule this
|
||||||
|
package imports down (union/model/coordination/provenance) and is imported only by the orchestrator.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from shard_wiki.views.allpages import AllPagesEntry, SiteMapNode, all_pages, site_map
|
||||||
|
from shard_wiki.views.backlinks import BackLink, BackLinksIndex, build_backlinks
|
||||||
|
from shard_wiki.views.links import (
|
||||||
|
ResolvedLink,
|
||||||
|
WikiLink,
|
||||||
|
extract_links,
|
||||||
|
resolve_links,
|
||||||
|
)
|
||||||
|
from shard_wiki.views.recentchanges import ChangeEntry, recent_changes
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"WikiLink",
|
||||||
|
"ResolvedLink",
|
||||||
|
"extract_links",
|
||||||
|
"resolve_links",
|
||||||
|
"BackLink",
|
||||||
|
"BackLinksIndex",
|
||||||
|
"build_backlinks",
|
||||||
|
"ChangeEntry",
|
||||||
|
"recent_changes",
|
||||||
|
"AllPagesEntry",
|
||||||
|
"SiteMapNode",
|
||||||
|
"all_pages",
|
||||||
|
"site_map",
|
||||||
|
]
|
||||||
131
src/shard_wiki/views/allpages.py
Normal file
131
src/shard_wiki/views/allpages.py
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
"""AllPages + SiteMap — enumeration views over the union (SHARD-WP-0010 T4).
|
||||||
|
|
||||||
|
**AllPages** lists the union's distinct pages, collapsing identities that name the same page: a
|
||||||
|
*chorus* (same key across shards) and *equivalence-bound* identities (decision-log bindings) fold
|
||||||
|
into one entry, with divergence noted when the members' bodies differ (union without erasure — the
|
||||||
|
collapse is acknowledged, never silent). **SiteMap** is the namespace tree built from page
|
||||||
|
placements (paths), spanning shards.
|
||||||
|
|
||||||
|
Both are derived/recomputable and presentation-free (the tree is a model, not rendered HTML).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.model import Identity, Page
|
||||||
|
from shard_wiki.union import UnionGraph
|
||||||
|
|
||||||
|
__all__ = ["AllPagesEntry", "SiteMapNode", "all_pages", "site_map"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class AllPagesEntry:
|
||||||
|
"""One union page: its representative ``name``, the ``members`` collapsed into it, and whether
|
||||||
|
those members' bodies ``diverge`` (a chorus with differing content)."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
members: tuple[Identity, ...]
|
||||||
|
diverges: bool
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class SiteMapNode:
|
||||||
|
"""A namespace node: its path ``name``, child namespaces, and pages directly under it."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
children: tuple[SiteMapNode, ...]
|
||||||
|
pages: tuple[Identity, ...]
|
||||||
|
|
||||||
|
|
||||||
|
class _UnionFind:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._parent: dict[str, str] = {}
|
||||||
|
|
||||||
|
def add(self, x: str) -> None:
|
||||||
|
self._parent.setdefault(x, x)
|
||||||
|
|
||||||
|
def find(self, x: str) -> str:
|
||||||
|
self.add(x)
|
||||||
|
root = x
|
||||||
|
while self._parent[root] != root:
|
||||||
|
root = self._parent[root]
|
||||||
|
while self._parent[x] != root:
|
||||||
|
self._parent[x], x = root, self._parent[x]
|
||||||
|
return root
|
||||||
|
|
||||||
|
def union(self, a: str, b: str) -> None:
|
||||||
|
self.add(a)
|
||||||
|
self.add(b)
|
||||||
|
ra, rb = self.find(a), self.find(b)
|
||||||
|
if ra != rb:
|
||||||
|
self._parent[max(ra, rb)] = min(ra, rb)
|
||||||
|
|
||||||
|
|
||||||
|
def all_pages(
|
||||||
|
union: UnionGraph,
|
||||||
|
equivalence_groups: tuple[frozenset[str], ...] | None = None,
|
||||||
|
) -> tuple[AllPagesEntry, ...]:
|
||||||
|
"""Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members.
|
||||||
|
|
||||||
|
``equivalence_groups`` (string identities, decision-log form) overrides the source of
|
||||||
|
equivalence — the orchestrator passes the maintained index's groups (SHARD-WP-0011 T4); the
|
||||||
|
default falls back to the decision-log fold, so direct callers are unaffected.
|
||||||
|
"""
|
||||||
|
pages: dict[str, Page] = {}
|
||||||
|
by_key: dict[str, list[str]] = {}
|
||||||
|
for page in union.iter_pages():
|
||||||
|
ident = str(page.identity)
|
||||||
|
pages[ident] = page
|
||||||
|
by_key.setdefault(page.identity.key, []).append(ident)
|
||||||
|
|
||||||
|
uf = _UnionFind()
|
||||||
|
for ident in pages:
|
||||||
|
uf.add(ident)
|
||||||
|
for idents in by_key.values(): # same key across shards → chorus
|
||||||
|
for other in idents[1:]:
|
||||||
|
uf.union(idents[0], other)
|
||||||
|
if equivalence_groups is None:
|
||||||
|
equivalence_groups = union.log.fold(union.space).equivalence_groups
|
||||||
|
for group in equivalence_groups: # curator bindings (+ maintained content edges)
|
||||||
|
present = [m for m in group if m in pages]
|
||||||
|
for other in present[1:]:
|
||||||
|
uf.union(present[0], other)
|
||||||
|
|
||||||
|
groups: dict[str, list[str]] = {}
|
||||||
|
for ident in pages:
|
||||||
|
groups.setdefault(uf.find(ident), []).append(ident)
|
||||||
|
|
||||||
|
entries: list[AllPagesEntry] = []
|
||||||
|
for members in groups.values():
|
||||||
|
member_pages = [pages[m] for m in members]
|
||||||
|
identities = tuple(p.identity for p in member_pages)
|
||||||
|
name = min(p.identity.key for p in member_pages)
|
||||||
|
diverges = len({p.body for p in member_pages}) > 1
|
||||||
|
entries.append(AllPagesEntry(name=name, members=identities, diverges=diverges))
|
||||||
|
return tuple(sorted(entries, key=lambda e: e.name))
|
||||||
|
|
||||||
|
|
||||||
|
def _segments(page: Page) -> list[str]:
|
||||||
|
path = page.placements[0].path if page.placements else page.identity.key
|
||||||
|
if path.endswith(".md"):
|
||||||
|
path = path[:-3]
|
||||||
|
return [seg for seg in path.split("/") if seg]
|
||||||
|
|
||||||
|
|
||||||
|
def site_map(union: UnionGraph) -> SiteMapNode:
|
||||||
|
"""The union namespace tree from page placements (directories nest; pages sit at their dir)."""
|
||||||
|
root: dict = {"children": {}, "pages": []}
|
||||||
|
for page in union.iter_pages():
|
||||||
|
segments = _segments(page)
|
||||||
|
node = root
|
||||||
|
for seg in segments[:-1]: # directory segments build the nesting
|
||||||
|
node = node["children"].setdefault(seg, {"children": {}, "pages": []})
|
||||||
|
node["pages"].append(page.identity)
|
||||||
|
return _freeze("", root)
|
||||||
|
|
||||||
|
|
||||||
|
def _freeze(name: str, node: dict) -> SiteMapNode:
|
||||||
|
children = tuple(_freeze(k, v) for k, v in sorted(node["children"].items()))
|
||||||
|
pages = tuple(sorted(node["pages"], key=str))
|
||||||
|
return SiteMapNode(name=name, children=children, pages=pages)
|
||||||
65
src/shard_wiki/views/backlinks.py
Normal file
65
src/shard_wiki/views/backlinks.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
"""BackLinks — the strongest core derived view (SHARD-WP-0010 T2; UC-18).
|
||||||
|
|
||||||
|
For any page name, the set of pages that link to it. Built by extracting wikilinks (T1) from every
|
||||||
|
page across the attached shards and resolving each through the union: only **resolved** links
|
||||||
|
create a backlink (a red-link points at nothing, so it contributes none). Entries carry their
|
||||||
|
**source provenance** (the linking page's identity / shard). Keying by the resolved *name* means a
|
||||||
|
chorus target aggregates the backlinks of all its members into one bucket (union without erasure).
|
||||||
|
|
||||||
|
Derived/recomputable — stores nothing canonical; SHARD-WP-0011 maintains it incrementally.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.model import Identity
|
||||||
|
from shard_wiki.union import UnionGraph
|
||||||
|
from shard_wiki.views.links import resolve_links
|
||||||
|
|
||||||
|
__all__ = ["BackLink", "BackLinksIndex", "build_backlinks"]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class BackLink:
|
||||||
|
"""One inbound link: ``source`` (the linking page) references ``target_name``."""
|
||||||
|
|
||||||
|
source: Identity
|
||||||
|
target_name: str
|
||||||
|
|
||||||
|
@property
|
||||||
|
def source_shard(self) -> str:
|
||||||
|
return self.source.shard
|
||||||
|
|
||||||
|
|
||||||
|
class BackLinksIndex:
|
||||||
|
"""An immutable name → inbound-links index over the union link graph."""
|
||||||
|
|
||||||
|
def __init__(self, edges: Mapping[str, tuple[BackLink, ...]]) -> None:
|
||||||
|
self._edges = dict(edges)
|
||||||
|
|
||||||
|
def to(self, name: str) -> tuple[BackLink, ...]:
|
||||||
|
"""The backlinks pointing at ``name`` (empty if none)."""
|
||||||
|
return self._edges.get(name, ())
|
||||||
|
|
||||||
|
def sources(self, name: str) -> frozenset[Identity]:
|
||||||
|
"""Just the identities linking to ``name`` — convenient for set assertions."""
|
||||||
|
return frozenset(bl.source for bl in self.to(name))
|
||||||
|
|
||||||
|
def names(self) -> frozenset[str]:
|
||||||
|
return frozenset(self._edges)
|
||||||
|
|
||||||
|
|
||||||
|
def build_backlinks(union: UnionGraph, *, camelcase: bool = False) -> BackLinksIndex:
|
||||||
|
"""Scan every union page's links and index the resolved ones by target name."""
|
||||||
|
edges: dict[str, set[BackLink]] = {}
|
||||||
|
for page in union.iter_pages():
|
||||||
|
for resolved in resolve_links(union, page.body, camelcase=camelcase):
|
||||||
|
if resolved.is_red_link:
|
||||||
|
continue # red-links don't create backlinks
|
||||||
|
backlink = BackLink(source=page.identity, target_name=resolved.link.target)
|
||||||
|
edges.setdefault(resolved.link.target, set()).add(backlink)
|
||||||
|
return BackLinksIndex(
|
||||||
|
{name: tuple(sorted(links, key=lambda bl: str(bl.source))) for name, links in edges.items()}
|
||||||
|
)
|
||||||
91
src/shard_wiki/views/links.py
Normal file
91
src/shard_wiki/views/links.py
Normal file
@@ -0,0 +1,91 @@
|
|||||||
|
"""Wikilink + red-link model (SHARD-WP-0010 T1; FederationRequirements ADR-06).
|
||||||
|
|
||||||
|
A CommonMark *wikilink extension*: ``[[Target]]`` and ``[[Target|label]]`` are extracted from a
|
||||||
|
page body and each target is resolved through the union (ADR-01). A target that resolves is a
|
||||||
|
**link**; one that does not is a **red-link** — a createable hole (UC-23), never a dropped
|
||||||
|
reference (union without erasure). CamelCase auto-linking (``WikiWord``) is **off by default** and
|
||||||
|
opt-in per space, since bare CamelCase is noisy and policy-laden.
|
||||||
|
|
||||||
|
The link *model and resolution* are core; turning a :class:`ResolvedLink` into an ``<a>`` (or a
|
||||||
|
red anchor) is L6 presentation and lives outside this package. Link spans are byte/char offsets in
|
||||||
|
the body so a later layer can address them precisely.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
from shard_wiki.union import Resolution, UnionGraph
|
||||||
|
|
||||||
|
__all__ = ["WikiLink", "ResolvedLink", "extract_links", "resolve_links"]
|
||||||
|
|
||||||
|
_WIKILINK_RE = re.compile(r"\[\[\s*([^\]|]+?)\s*(?:\|\s*([^\]]+?)\s*)?\]\]")
|
||||||
|
# A WikiWord: ≥2 capitalized alphanumeric segments run together (e.g. FrontPage, WikiWord).
|
||||||
|
_CAMELCASE_RE = re.compile(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b")
|
||||||
|
_FENCED_RE = re.compile(r"```.*?```", re.DOTALL)
|
||||||
|
_INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class WikiLink:
|
||||||
|
"""One extracted reference. ``target`` is the resolve key; ``label`` is the display text (or
|
||||||
|
None to use the target); ``span`` is the ``[start, end)`` offset of the whole token in the body;
|
||||||
|
``auto`` marks a CamelCase auto-link (vs an explicit ``[[...]]``)."""
|
||||||
|
|
||||||
|
target: str
|
||||||
|
label: str | None
|
||||||
|
span: tuple[int, int]
|
||||||
|
auto: bool = False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def text(self) -> str:
|
||||||
|
return self.label or self.target
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ResolvedLink:
|
||||||
|
"""A :class:`WikiLink` paired with its union :class:`Resolution` (the link's truth status)."""
|
||||||
|
|
||||||
|
link: WikiLink
|
||||||
|
resolution: Resolution
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_red_link(self) -> bool:
|
||||||
|
return self.resolution.is_red_link
|
||||||
|
|
||||||
|
|
||||||
|
def _mask(body: str, pattern: re.Pattern[str]) -> str:
|
||||||
|
"""Blank out ``pattern`` matches with equal-length spaces so later scans skip them while every
|
||||||
|
surviving match keeps its true offset."""
|
||||||
|
return pattern.sub(lambda m: " " * len(m.group(0)), body)
|
||||||
|
|
||||||
|
|
||||||
|
def extract_links(body: str, *, camelcase: bool = False) -> tuple[WikiLink, ...]:
|
||||||
|
"""Extract wikilinks from ``body`` in document order, skipping fenced/inline code.
|
||||||
|
|
||||||
|
With ``camelcase=True`` (per-space opt-in), bare ``WikiWord`` tokens outside code and outside
|
||||||
|
existing ``[[...]]`` also become links.
|
||||||
|
"""
|
||||||
|
scan = _mask(_mask(body, _FENCED_RE), _INLINE_CODE_RE)
|
||||||
|
links: list[WikiLink] = []
|
||||||
|
for m in _WIKILINK_RE.finditer(scan):
|
||||||
|
links.append(WikiLink(target=m.group(1).strip(), label=m.group(2), span=m.span()))
|
||||||
|
|
||||||
|
if camelcase:
|
||||||
|
# Mask explicit-link spans too, so a CamelCase target inside [[...]] isn't double-counted.
|
||||||
|
cc_scan = _mask(scan, _WIKILINK_RE)
|
||||||
|
for m in _CAMELCASE_RE.finditer(cc_scan):
|
||||||
|
links.append(WikiLink(target=m.group(1), label=None, span=m.span(), auto=True))
|
||||||
|
|
||||||
|
return tuple(sorted(links, key=lambda link: link.span[0]))
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_links(
|
||||||
|
union: UnionGraph, body: str, *, camelcase: bool = False
|
||||||
|
) -> tuple[ResolvedLink, ...]:
|
||||||
|
"""Extract and resolve every link in ``body`` against ``union`` (link vs red-link, ADR-01)."""
|
||||||
|
return tuple(
|
||||||
|
ResolvedLink(link, union.resolve(link.target))
|
||||||
|
for link in extract_links(body, camelcase=camelcase)
|
||||||
|
)
|
||||||
108
src/shard_wiki/views/recentchanges.py
Normal file
108
src/shard_wiki/views/recentchanges.py
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
"""RecentChanges — a merged change feed over the union (SHARD-WP-0010 T3; UC-17).
|
||||||
|
|
||||||
|
Two streams, one ordered feed (newest-first):
|
||||||
|
|
||||||
|
* the **coordination journal** — overlay/alias/fork/merge/binding decisions from the decision log,
|
||||||
|
each carrying its actor and the decision payload; and
|
||||||
|
* **shard change signals** — a page's current revision (folder mtime / ``source_rev``), i.e. the
|
||||||
|
backend's own "this changed" evidence.
|
||||||
|
|
||||||
|
Every entry carries provenance: which shard the edit came from, or that it was a coordination
|
||||||
|
decision (and by whom). Derived/recomputable — `notify`-driven streaming is a later binding.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from shard_wiki.coordination import DecisionLog, EventType
|
||||||
|
from shard_wiki.union import UnionGraph
|
||||||
|
|
||||||
|
__all__ = ["ChangeEntry", "recent_changes"]
|
||||||
|
|
||||||
|
_COORDINATION = "coordination"
|
||||||
|
|
||||||
|
# How each journal event names the thing it touched + a human kind label.
|
||||||
|
_EVENT_KIND = {
|
||||||
|
EventType.ALIAS_SET: "alias",
|
||||||
|
EventType.OVERLAY_CREATED: "overlay",
|
||||||
|
EventType.MERGE_DECIDED: "merge",
|
||||||
|
EventType.PAGE_FORKED: "fork",
|
||||||
|
EventType.BINDING_MADE: "binding",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class ChangeEntry:
|
||||||
|
"""One change in the feed. ``source`` is the shard id (a shard edit) or ``"coordination"``."""
|
||||||
|
|
||||||
|
when: datetime
|
||||||
|
kind: str
|
||||||
|
ref: str
|
||||||
|
source: str
|
||||||
|
actor: str | None = None
|
||||||
|
detail: Mapping[str, object] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
def _event_ref(event_type: EventType, payload: Mapping[str, object]) -> str:
|
||||||
|
if event_type is EventType.ALIAS_SET:
|
||||||
|
return str(payload.get("alias", ""))
|
||||||
|
if event_type is EventType.OVERLAY_CREATED:
|
||||||
|
return f"{payload.get('target_shard')}:{payload.get('target_key')}"
|
||||||
|
if event_type is EventType.PAGE_FORKED:
|
||||||
|
return f"{payload.get('source')}→{payload.get('fork')}"
|
||||||
|
if event_type is EventType.BINDING_MADE:
|
||||||
|
return ", ".join(str(m) for m in payload.get("members", ()))
|
||||||
|
return str(payload.get("overlay_id", "")) # MERGE_DECIDED
|
||||||
|
|
||||||
|
|
||||||
|
def recent_changes(
|
||||||
|
union: UnionGraph,
|
||||||
|
log: DecisionLog,
|
||||||
|
space: str,
|
||||||
|
*,
|
||||||
|
limit: int | None = None,
|
||||||
|
) -> tuple[ChangeEntry, ...]:
|
||||||
|
"""Merge the coordination journal and shard change signals into one newest-first feed."""
|
||||||
|
entries: list[ChangeEntry] = []
|
||||||
|
|
||||||
|
for event in log.events(space):
|
||||||
|
entries.append(
|
||||||
|
ChangeEntry(
|
||||||
|
when=event.timestamp,
|
||||||
|
kind=_EVENT_KIND.get(event.type, event.type.value),
|
||||||
|
ref=_event_ref(event.type, event.payload),
|
||||||
|
source=_COORDINATION,
|
||||||
|
actor=event.actor,
|
||||||
|
detail=dict(event.payload),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
for page in union.iter_pages():
|
||||||
|
rev = page.envelope.source_rev
|
||||||
|
when = _parse_rev(rev)
|
||||||
|
if when is None:
|
||||||
|
continue # shard offers no change signal for this page — skip gracefully
|
||||||
|
entries.append(
|
||||||
|
ChangeEntry(
|
||||||
|
when=when,
|
||||||
|
kind="edit",
|
||||||
|
ref=str(page.identity),
|
||||||
|
source=page.identity.shard,
|
||||||
|
detail={"source_rev": rev},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
entries.sort(key=lambda e: e.when, reverse=True)
|
||||||
|
return tuple(entries if limit is None else entries[:limit])
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_rev(rev: str | None) -> datetime | None:
|
||||||
|
if rev is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return datetime.fromisoformat(rev)
|
||||||
|
except ValueError:
|
||||||
|
return None # non-temporal revision token (e.g. a content hash) — no feed timestamp
|
||||||
120
tests/test_append_authority.py
Normal file
120
tests/test_append_authority.py
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
"""Tests for the per-space append authority / lease (SHARD-WP-0009 T2).
|
||||||
|
|
||||||
|
A single append authority per space serializes appends into a total order; non-holders forward
|
||||||
|
intents to the holder; the lease is time-bounded and re-grantable (HA hand-off); a stale ex-holder
|
||||||
|
cannot fork the log.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.coordination import (
|
||||||
|
AppendAuthority,
|
||||||
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
|
InMemoryEventStore,
|
||||||
|
LeaseHeld,
|
||||||
|
LeaseRegistry,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeClock:
|
||||||
|
def __init__(self):
|
||||||
|
self.now = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||||
|
|
||||||
|
def __call__(self):
|
||||||
|
return self.now
|
||||||
|
|
||||||
|
def advance(self, seconds):
|
||||||
|
self.now += timedelta(seconds=seconds)
|
||||||
|
|
||||||
|
|
||||||
|
def test_only_one_node_holds_a_space_at_a_time():
|
||||||
|
reg = LeaseRegistry()
|
||||||
|
a = AppendAuthority("A", InMemoryEventStore(), reg)
|
||||||
|
b = AppendAuthority("B", InMemoryEventStore(), reg)
|
||||||
|
a.acquire("s")
|
||||||
|
with pytest.raises(LeaseHeld):
|
||||||
|
b.acquire("s") # B is refused while A's lease is valid
|
||||||
|
|
||||||
|
|
||||||
|
def test_concurrent_appends_serialize_into_one_total_order():
|
||||||
|
reg = LeaseRegistry()
|
||||||
|
store = InMemoryEventStore()
|
||||||
|
a = AppendAuthority("A", store, reg)
|
||||||
|
b = AppendAuthority("B", store, reg)
|
||||||
|
a.acquire("s")
|
||||||
|
# B is a non-holder: its append forwards to A, the holder. Interleave A and B writers.
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||||
|
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"}) # forwarded
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "3", "target": "x:3"})
|
||||||
|
seqs = [e.seq for e in store.events("s")]
|
||||||
|
aliases = [e.payload["alias"] for e in store.events("s")]
|
||||||
|
assert seqs == [0, 1, 2] # contiguous total order despite two writers
|
||||||
|
assert aliases == ["1", "2", "3"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_non_holder_forwards_rather_than_writing_directly():
|
||||||
|
reg = LeaseRegistry()
|
||||||
|
store = InMemoryEventStore()
|
||||||
|
a = AppendAuthority("A", store, reg)
|
||||||
|
b = AppendAuthority("B", store, reg)
|
||||||
|
a.acquire("s")
|
||||||
|
assert not b.holds("s")
|
||||||
|
b.append("s", EventType.ALIAS_SET, {"alias": "fwd", "target": "x:1"})
|
||||||
|
# The write landed on the shared store under A's authority, in one stream.
|
||||||
|
assert [e.payload["alias"] for e in store.events("s")] == ["fwd"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_lease_handoff_resumes_from_head():
|
||||||
|
clock = FakeClock()
|
||||||
|
reg = LeaseRegistry(clock=clock)
|
||||||
|
store = InMemoryEventStore()
|
||||||
|
a = AppendAuthority("A", store, reg, ttl_seconds=10)
|
||||||
|
b = AppendAuthority("B", store, reg, ttl_seconds=10)
|
||||||
|
a.acquire("s")
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||||
|
clock.advance(20) # A's lease expires (A "dies")
|
||||||
|
b.acquire("s") # re-grantable: B takes over
|
||||||
|
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
|
||||||
|
assert [e.seq for e in store.events("s")] == [0, 1, 2] # contiguous across hand-off
|
||||||
|
|
||||||
|
|
||||||
|
def test_stale_ex_holder_cannot_fork_the_log():
|
||||||
|
clock = FakeClock()
|
||||||
|
reg = LeaseRegistry(clock=clock)
|
||||||
|
store = InMemoryEventStore()
|
||||||
|
a = AppendAuthority("A", store, reg, ttl_seconds=10)
|
||||||
|
b = AppendAuthority("B", store, reg, ttl_seconds=10)
|
||||||
|
a.acquire("s")
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
|
||||||
|
clock.advance(20)
|
||||||
|
b.acquire("s") # B is now the holder; A's lease is stale
|
||||||
|
b.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||||
|
# A still thinks it can write, but it's no longer the holder: its intent forwards to B.
|
||||||
|
assert not a.holds("s")
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
|
||||||
|
aliases = [e.payload["alias"] for e in store.events("s")]
|
||||||
|
assert aliases == ["0", "1", "2"] # one stream, no fork
|
||||||
|
|
||||||
|
|
||||||
|
def test_authority_over_git_store_keeps_total_order(tmp_path):
|
||||||
|
reg = LeaseRegistry()
|
||||||
|
store = GitEventStore(tmp_path / "coord")
|
||||||
|
a = AppendAuthority("A", store, reg)
|
||||||
|
b = AppendAuthority("B", store, reg)
|
||||||
|
a.acquire("s")
|
||||||
|
a.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||||
|
b.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"}) # forwarded
|
||||||
|
assert [e.seq for e in store.events("s")] == [0, 1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_unleased_space_self_acquires_on_append():
|
||||||
|
reg = LeaseRegistry()
|
||||||
|
store = InMemoryEventStore()
|
||||||
|
a = AppendAuthority("A", store, reg)
|
||||||
|
a.append("s", EventType.ALIAS_SET, {"alias": "x", "target": "y:1"}) # no explicit acquire
|
||||||
|
assert a.holds("s")
|
||||||
|
assert len(store.events("s")) == 1
|
||||||
66
tests/test_apply.py
Normal file
66
tests/test_apply.py
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
"""Tests for apply-under-drift (SHARD-WP-0008 T4)."""
|
||||||
|
|
||||||
|
from shard_wiki.adapters import FolderAdapter
|
||||||
|
from shard_wiki.coordination import ApplyStatus, DecisionLog, OverlayEngine
|
||||||
|
from shard_wiki.model import Identity
|
||||||
|
|
||||||
|
|
||||||
|
def _writable(tmp_path, files):
|
||||||
|
for rel, text in files.items():
|
||||||
|
(tmp_path / rel).write_text(text, encoding="utf-8")
|
||||||
|
return FolderAdapter("w", tmp_path, writable=True)
|
||||||
|
|
||||||
|
|
||||||
|
def test_fast_forward_apply_writes_through(tmp_path):
|
||||||
|
shard = _writable(tmp_path, {"Home.md": "old"})
|
||||||
|
eng = OverlayEngine("space", DecisionLog())
|
||||||
|
base = shard.current_rev("Home")
|
||||||
|
ov = eng.draft(Identity("w", "Home"), "new", base_rev=base)
|
||||||
|
|
||||||
|
result = eng.apply(ov.overlay_id, shard)
|
||||||
|
assert result.status is ApplyStatus.APPLIED
|
||||||
|
assert shard.read("Home").body == "new" # written through
|
||||||
|
assert eng.get(ov.overlay_id) is None # overlay closed in the fold
|
||||||
|
|
||||||
|
|
||||||
|
def test_drift_refuses_without_clobber(tmp_path):
|
||||||
|
shard = _writable(tmp_path, {"Home.md": "old"})
|
||||||
|
eng = OverlayEngine("space", DecisionLog())
|
||||||
|
ov = eng.draft(Identity("w", "Home"), "mine", base_rev="STALE-REV")
|
||||||
|
|
||||||
|
result = eng.apply(ov.overlay_id, shard)
|
||||||
|
assert result.status is ApplyStatus.REFUSED_DRIFT
|
||||||
|
assert shard.read("Home").body == "old" # not clobbered (I-5)
|
||||||
|
assert eng.get(ov.overlay_id) is not None # overlay still open
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_only_target_keeps_draft(tmp_path):
|
||||||
|
(tmp_path / "Home.md").write_text("canon", encoding="utf-8")
|
||||||
|
ro = FolderAdapter("ro", tmp_path) # not writable
|
||||||
|
eng = OverlayEngine("space", DecisionLog())
|
||||||
|
ov = eng.draft(Identity("ro", "Home"), "my edit", base_rev=ro.current_rev("Home"))
|
||||||
|
|
||||||
|
result = eng.apply(ov.overlay_id, ro)
|
||||||
|
assert result.status is ApplyStatus.KEPT_DRAFT
|
||||||
|
assert ro.read("Home").body == "canon" # source untouched
|
||||||
|
assert eng.get(ov.overlay_id) is not None # local truth retained
|
||||||
|
|
||||||
|
|
||||||
|
def test_new_page_fast_forwards(tmp_path):
|
||||||
|
shard = _writable(tmp_path, {})
|
||||||
|
eng = OverlayEngine("space", DecisionLog())
|
||||||
|
ov = eng.draft(Identity("w", "Fresh"), "brand new", base_rev=None) # didn't exist
|
||||||
|
result = eng.apply(ov.overlay_id, shard)
|
||||||
|
assert result.status is ApplyStatus.APPLIED
|
||||||
|
assert shard.read("Fresh").body == "brand new"
|
||||||
|
|
||||||
|
|
||||||
|
def test_wrong_adapter_is_rejected(tmp_path):
|
||||||
|
shard = _writable(tmp_path, {"Home.md": "x"})
|
||||||
|
eng = OverlayEngine("space", DecisionLog())
|
||||||
|
ov = eng.draft(Identity("other", "Home"), "y", base_rev=None)
|
||||||
|
try:
|
||||||
|
eng.apply(ov.overlay_id, shard)
|
||||||
|
raise AssertionError("expected ValueError")
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
67
tests/test_conformance.py
Normal file
67
tests/test_conformance.py
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
"""Tests for the adapter conformance suite (SHARD-WP-0007 T4)."""
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.adapters import (
|
||||||
|
ConformanceError,
|
||||||
|
FolderAdapter,
|
||||||
|
ShardAdapter,
|
||||||
|
assert_conformant,
|
||||||
|
run_conformance,
|
||||||
|
)
|
||||||
|
from shard_wiki.model import CapabilityProfile, Identity, Page
|
||||||
|
from shard_wiki.provenance import ProvenanceEnvelope
|
||||||
|
|
||||||
|
|
||||||
|
def test_folder_adapter_is_conformant(tmp_path):
|
||||||
|
(tmp_path / "Home.md").write_text("# Home", encoding="utf-8")
|
||||||
|
report = assert_conformant(FolderAdapter("shardA", tmp_path))
|
||||||
|
assert report.ok
|
||||||
|
assert report.diff() == "conformant"
|
||||||
|
|
||||||
|
|
||||||
|
class _LyingReadAdapter(ShardAdapter):
|
||||||
|
"""Claims READ but read() is broken — must fail conformance."""
|
||||||
|
|
||||||
|
def __init__(self, profile: CapabilityProfile) -> None:
|
||||||
|
self._profile = profile
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
return "liar"
|
||||||
|
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
return self._profile
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
return ["Home"]
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
# Returns a page attributed to the WRONG shard — a lie about its own content.
|
||||||
|
return Page(Identity("someone-else", key), "x", ProvenanceEnvelope(source_shard="x"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_lying_read_adapter_fails_with_precise_diff(tmp_path):
|
||||||
|
good = FolderAdapter("shardA", tmp_path).profile() # a valid read-only profile
|
||||||
|
liar = _LyingReadAdapter(good)
|
||||||
|
report = run_conformance(liar)
|
||||||
|
assert not report.ok
|
||||||
|
assert "read-round-trips" in report.diff()
|
||||||
|
with pytest.raises(ConformanceError, match="read-round-trips"):
|
||||||
|
assert_conformant(liar)
|
||||||
|
|
||||||
|
|
||||||
|
class _DishonestWriteAdapter(FolderAdapter):
|
||||||
|
"""Declares no WRITE (inherits read-only folder profile) but write() silently succeeds."""
|
||||||
|
|
||||||
|
def write(self, key: str, body: str) -> Page: # noqa: ARG002
|
||||||
|
return self.read(next(iter(self.keys())))
|
||||||
|
|
||||||
|
|
||||||
|
def test_dishonest_absence_fails(tmp_path):
|
||||||
|
(tmp_path / "Home.md").write_text("x", encoding="utf-8")
|
||||||
|
report = run_conformance(_DishonestWriteAdapter("shardA", tmp_path))
|
||||||
|
assert not report.ok
|
||||||
|
assert "honest-absence:write" in report.diff()
|
||||||
74
tests/test_coordination_migration.py
Normal file
74
tests/test_coordination_migration.py
Normal file
@@ -0,0 +1,74 @@
|
|||||||
|
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
|
||||||
|
|
||||||
|
from shard_wiki.coordination import (
|
||||||
|
DecisionLog,
|
||||||
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
|
InMemoryEventStore,
|
||||||
|
export_jsonl,
|
||||||
|
import_jsonl,
|
||||||
|
migrate_space,
|
||||||
|
)
|
||||||
|
from shard_wiki.space import InformationSpace
|
||||||
|
|
||||||
|
|
||||||
|
def test_information_space_git_backed_uses_git_log(tmp_path):
|
||||||
|
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
|
||||||
|
assert isinstance(space.log._store, GitEventStore)
|
||||||
|
space.alias("Home", "shardA:Index")
|
||||||
|
# Read-your-writes through the orchestrator's git-backed log.
|
||||||
|
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
|
||||||
|
|
||||||
|
|
||||||
|
def test_default_information_space_stays_in_memory():
|
||||||
|
space = InformationSpace("space-1")
|
||||||
|
assert isinstance(space.log._store, InMemoryEventStore)
|
||||||
|
|
||||||
|
|
||||||
|
def test_migrate_space_preserves_order_and_provenance(tmp_path):
|
||||||
|
source = InMemoryEventStore()
|
||||||
|
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
|
||||||
|
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
n = migrate_space(source, "s", dest)
|
||||||
|
assert n == 2
|
||||||
|
|
||||||
|
migrated = dest.events("s")
|
||||||
|
assert [e.seq for e in migrated] == [0, 1]
|
||||||
|
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
|
||||||
|
assert migrated[0].actor == "ana"
|
||||||
|
assert migrated[1].actor == "ben"
|
||||||
|
assert migrated[0].timestamp == e0.timestamp
|
||||||
|
|
||||||
|
|
||||||
|
def test_migration_yields_identical_fold(tmp_path):
|
||||||
|
source = DecisionLog(InMemoryEventStore())
|
||||||
|
for typ, payload in [
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
|
||||||
|
]:
|
||||||
|
source.append("s", typ, payload)
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
migrate_space(source._store, "s", dest)
|
||||||
|
after = DecisionLog(dest)
|
||||||
|
assert after.fold("s").aliases == source.fold("s").aliases
|
||||||
|
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
|
||||||
|
|
||||||
|
|
||||||
|
def test_jsonl_round_trip_into_git(tmp_path):
|
||||||
|
source = InMemoryEventStore()
|
||||||
|
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
|
||||||
|
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
|
||||||
|
|
||||||
|
path = tmp_path / "log.jsonl"
|
||||||
|
assert export_jsonl(source.events("s"), path) == 2
|
||||||
|
|
||||||
|
dest = GitEventStore(tmp_path / "coord")
|
||||||
|
assert import_jsonl(path, dest) == 2
|
||||||
|
state = DecisionLog(dest).fold("s")
|
||||||
|
assert state.resolve_alias("Home") == "x:1"
|
||||||
|
assert state.equivalent_to("p") == frozenset({"p", "q"})
|
||||||
83
tests/test_cross_process_fold.py
Normal file
83
tests/test_cross_process_fold.py
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
"""Cross-process read-your-writes over the git log + fold parity (SHARD-WP-0009 T3).
|
||||||
|
|
||||||
|
The git backend's value over the in-memory double is that the totally ordered log is durable and
|
||||||
|
shared: a write by one process/handle is immediately visible to another opening the same ref, and
|
||||||
|
the derived fold is identical to the in-memory fold of the same event sequence (derived = f(log)).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import textwrap
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from shard_wiki.coordination import (
|
||||||
|
DecisionLog,
|
||||||
|
EventType,
|
||||||
|
GitEventStore,
|
||||||
|
InMemoryEventStore,
|
||||||
|
)
|
||||||
|
|
||||||
|
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||||
|
|
||||||
|
|
||||||
|
def test_new_handle_sees_prior_writes(tmp_path):
|
||||||
|
repo = tmp_path / "coord"
|
||||||
|
writer = DecisionLog(GitEventStore(repo))
|
||||||
|
writer.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
|
||||||
|
writer.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||||
|
# A second, independent handle on the same repo — read-your-writes across handles.
|
||||||
|
reader = DecisionLog(GitEventStore(repo))
|
||||||
|
assert [e.seq for e in reader.events("s")] == [0, 1]
|
||||||
|
assert reader.fold("s").resolve_alias("Home") == "shardA:Index"
|
||||||
|
|
||||||
|
|
||||||
|
def test_append_in_separate_process_is_visible(tmp_path):
|
||||||
|
repo = tmp_path / "coord"
|
||||||
|
# Seed from this process so the repo exists.
|
||||||
|
DecisionLog(GitEventStore(repo)).append(
|
||||||
|
"s", EventType.ALIAS_SET, {"alias": "A", "target": "x:1"}
|
||||||
|
)
|
||||||
|
child = textwrap.dedent(
|
||||||
|
f"""
|
||||||
|
from shard_wiki.coordination import DecisionLog, EventType, GitEventStore
|
||||||
|
log = DecisionLog(GitEventStore({str(repo)!r}))
|
||||||
|
log.append("s", EventType.ALIAS_SET, {{"alias": "B", "target": "x:2"}})
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
result = subprocess.run(
|
||||||
|
[sys.executable, "-c", child],
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
env={"PYTHONPATH": _SRC, "PATH": os.environ.get("PATH", "")},
|
||||||
|
)
|
||||||
|
assert result.returncode == 0, result.stderr
|
||||||
|
# This process, with a fresh handle, sees the child's append in order.
|
||||||
|
reader = DecisionLog(GitEventStore(repo))
|
||||||
|
assert [e.payload["alias"] for e in reader.events("s")] == ["A", "B"]
|
||||||
|
assert [e.seq for e in reader.events("s")] == [0, 1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_cross_process_fold_equals_in_memory_fold(tmp_path):
|
||||||
|
sequence = [
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||||
|
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||||
|
(EventType.PAGE_FORKED, {"source": "p", "fork": "q"}),
|
||||||
|
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
|
||||||
|
]
|
||||||
|
mem = DecisionLog(InMemoryEventStore())
|
||||||
|
for typ, payload in sequence:
|
||||||
|
mem.append("s", typ, payload)
|
||||||
|
|
||||||
|
repo = tmp_path / "coord"
|
||||||
|
DecisionLog(GitEventStore(repo)) # init repo
|
||||||
|
for typ, payload in sequence:
|
||||||
|
# Each append from a fresh handle to simulate distinct writers over time.
|
||||||
|
DecisionLog(GitEventStore(repo)).append("s", typ, payload)
|
||||||
|
|
||||||
|
git_state = DecisionLog(GitEventStore(repo)).fold("s")
|
||||||
|
mem_state = mem.fold("s")
|
||||||
|
assert git_state.aliases == mem_state.aliases
|
||||||
|
assert git_state.equivalence_groups == mem_state.equivalence_groups
|
||||||
|
assert git_state.equivalent_to("a") == frozenset({"a", "b", "c"})
|
||||||
51
tests/test_decision_log.py
Normal file
51
tests/test_decision_log.py
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
"""Tests for the event-sourced DecisionLog (SHARD-WP-0007 T5)."""
|
||||||
|
|
||||||
|
from shard_wiki.coordination import DecisionLog, EventType
|
||||||
|
|
||||||
|
|
||||||
|
def test_append_is_totally_ordered_per_space():
|
||||||
|
log = DecisionLog()
|
||||||
|
log.append("spaceA", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
|
||||||
|
log.append("spaceA", EventType.ALIAS_SET, {"alias": "Start", "target": "shardA:Index"})
|
||||||
|
log.append("spaceB", EventType.ALIAS_SET, {"alias": "X", "target": "shardB:Y"})
|
||||||
|
seqs = [e.seq for e in log.events("spaceA")]
|
||||||
|
assert seqs == [0, 1] # per-space monotonic
|
||||||
|
assert [e.seq for e in log.events("spaceB")] == [0] # independent ordering
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_your_writes():
|
||||||
|
log = DecisionLog()
|
||||||
|
ev = log.append("s", EventType.ALIAS_SET, {"alias": "a", "target": "shardA:b"})
|
||||||
|
assert log.events("s")[-1] is ev
|
||||||
|
|
||||||
|
|
||||||
|
def test_fold_reproduces_current_state():
|
||||||
|
log = DecisionLog()
|
||||||
|
log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
|
||||||
|
log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}) # LWW
|
||||||
|
state = log.fold("s")
|
||||||
|
assert state.resolve_alias("Home") == "shardB:Main"
|
||||||
|
assert state.resolve_alias("missing") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_fold_is_pure_function_of_log():
|
||||||
|
log = DecisionLog()
|
||||||
|
log.append("s", EventType.BINDING_MADE, {"members": ["shardA:Home", "shardB:Home"]})
|
||||||
|
first = log.fold("s")
|
||||||
|
second = log.fold("s")
|
||||||
|
assert first.equivalence_groups == second.equivalence_groups # derived = f(log)
|
||||||
|
|
||||||
|
|
||||||
|
def test_equivalence_groups_merge_transitively():
|
||||||
|
log = DecisionLog()
|
||||||
|
log.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||||
|
log.append("s", EventType.BINDING_MADE, {"members": ["b", "c"]})
|
||||||
|
state = log.fold("s")
|
||||||
|
assert state.equivalent_to("a") == frozenset({"a", "b", "c"})
|
||||||
|
assert state.equivalent_to("lonely") == frozenset({"lonely"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_page_fork_creates_equivalence():
|
||||||
|
log = DecisionLog()
|
||||||
|
log.append("s", EventType.PAGE_FORKED, {"source": "shardA:Doc", "fork": "shardB:Doc"})
|
||||||
|
assert log.fold("s").equivalent_to("shardA:Doc") == frozenset({"shardA:Doc", "shardB:Doc"})
|
||||||
61
tests/test_engine_activation.py
Normal file
61
tests/test_engine_activation.py
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
"""Tests for per-shard extension activation (SHARD-WP-0014 T3, ADR-0001)."""
|
||||||
|
|
||||||
|
from shard_wiki.engine import (
|
||||||
|
ActivationContext,
|
||||||
|
ActivationResolver,
|
||||||
|
StaticProvider,
|
||||||
|
feature_control_provider,
|
||||||
|
)
|
||||||
|
|
||||||
|
CANDIDATES = ["ext.overlay", "ext.views", "ext.struct", "ext.compute"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_static_provider_default_off():
|
||||||
|
r = ActivationResolver(StaticProvider()) # nothing enabled
|
||||||
|
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == set()
|
||||||
|
|
||||||
|
|
||||||
|
def test_static_provider_global_flags():
|
||||||
|
r = ActivationResolver(StaticProvider(flags={"ext.overlay": True, "ext.views": True}))
|
||||||
|
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == {"ext.overlay", "ext.views"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_per_shard_scoping_overrides_global():
|
||||||
|
provider = StaticProvider(
|
||||||
|
flags={"ext.views": True},
|
||||||
|
per_shard={"engB": {"ext.struct": True, "ext.views": False}},
|
||||||
|
)
|
||||||
|
r = ActivationResolver(provider)
|
||||||
|
assert r.active_extensions(CANDIDATES, ActivationContext("engA")) == {"ext.views"}
|
||||||
|
assert r.active_extensions(CANDIDATES, ActivationContext("engB")) == {"ext.struct"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_carries_tenant():
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
class Spy(StaticProvider):
|
||||||
|
def is_active(self, feature_key, context):
|
||||||
|
captured.update(context)
|
||||||
|
return super().is_active(feature_key, context)
|
||||||
|
|
||||||
|
ActivationResolver(Spy(flags={"ext.views": True})).active_extensions(
|
||||||
|
["ext.views"], ActivationContext("s1", tenant_id="acme")
|
||||||
|
)
|
||||||
|
assert captured["shard_id"] == "s1" and captured["tenant_id"] == "acme"
|
||||||
|
|
||||||
|
|
||||||
|
def test_activation_profile_returns_config_for_active():
|
||||||
|
provider = StaticProvider(
|
||||||
|
flags={"ext.struct": True},
|
||||||
|
configs={"ext.struct": {"max_fields": 50}},
|
||||||
|
)
|
||||||
|
profile = ActivationResolver(provider).activation_profile(CANDIDATES, ActivationContext("s"))
|
||||||
|
assert profile == {"ext.struct": {"max_fields": 50}}
|
||||||
|
|
||||||
|
|
||||||
|
def test_feature_control_provider_degrades_gracefully():
|
||||||
|
# feature_control_sdk is not a dependency of shard-wiki: when absent the factory returns
|
||||||
|
# None (standalone path stays dependency-free, ADR-0001); when present it yields a usable
|
||||||
|
# ActivationProvider. Either way it must not raise.
|
||||||
|
provider = feature_control_provider()
|
||||||
|
assert provider is None or hasattr(provider, "is_active")
|
||||||
80
tests/test_engine_adapter.py
Normal file
80
tests/test_engine_adapter.py
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
"""Tests for EngineShardAdapter (SHARD-WP-0014 T5): engine as a canonical-mode shard."""
|
||||||
|
|
||||||
|
from shard_wiki import InformationSpace
|
||||||
|
from shard_wiki.adapters import assert_conformant
|
||||||
|
from shard_wiki.engine import (
|
||||||
|
Extension,
|
||||||
|
ExtensionRuntime,
|
||||||
|
Hook,
|
||||||
|
ProfileContribution,
|
||||||
|
StaticProvider,
|
||||||
|
build_engine_shard,
|
||||||
|
)
|
||||||
|
from shard_wiki.engine.activation import ActivationContext
|
||||||
|
from shard_wiki.model import Verb
|
||||||
|
|
||||||
|
|
||||||
|
class StructProfileExt(Extension):
|
||||||
|
"""Profile-only extension (no body transform → write stays content-preserving)."""
|
||||||
|
|
||||||
|
id = "ext.struct"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {
|
||||||
|
Hook.ON_PROFILE: lambda p, c: ProfileContribution(
|
||||||
|
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime():
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
rt.register(StructProfileExt())
|
||||||
|
return rt
|
||||||
|
|
||||||
|
|
||||||
|
def test_kernel_only_engine_shard_is_conformant():
|
||||||
|
shard = build_engine_shard("eng", ExtensionRuntime(), activate=set())
|
||||||
|
shard.write("Home", "hi")
|
||||||
|
report = assert_conformant(shard) # read + positive write probe
|
||||||
|
assert report.ok
|
||||||
|
assert shard.profile().supports(Verb.WRITE)
|
||||||
|
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_reflects_activated_extensions():
|
||||||
|
off = build_engine_shard("a", _runtime(), activate=set())
|
||||||
|
on = build_engine_shard("b", _runtime(), activate={"ext.struct"})
|
||||||
|
assert not off.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||||
|
assert on.profile().supports(Verb.STRUCTURED_PAYLOAD) # E-5
|
||||||
|
assert_conformant(on)
|
||||||
|
|
||||||
|
|
||||||
|
def test_activation_via_provider():
|
||||||
|
provider = StaticProvider(flags={"ext.struct": True})
|
||||||
|
shard = build_engine_shard("c", _runtime(), provider=provider, context=ActivationContext("c"))
|
||||||
|
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||||
|
|
||||||
|
|
||||||
|
def test_attach_resolve_edit_through_engine_shard(tmp_path):
|
||||||
|
space = InformationSpace("team")
|
||||||
|
space.attach(build_engine_shard("wikiE", ExtensionRuntime(), activate=set()))
|
||||||
|
# seed a page directly via the shard, then read + edit through the orchestrator
|
||||||
|
space.union.shard("wikiE").write("Home", "v1")
|
||||||
|
assert space.read("Home").body == "v1"
|
||||||
|
result = space.edit("Home", "v2") # overlay -> apply-under-drift -> write-through
|
||||||
|
assert result.status.value == "applied"
|
||||||
|
assert space.read("Home").body == "v2"
|
||||||
|
|
||||||
|
|
||||||
|
def test_on_write_transform_runs():
|
||||||
|
class Upper(Extension):
|
||||||
|
id = "ext.upper"
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
|
||||||
|
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
rt.register(Upper())
|
||||||
|
shard = build_engine_shard("u", rt, activate={"ext.upper"})
|
||||||
|
shard.write("P", "quiet")
|
||||||
|
assert shard.read("P").body == "QUIET" # extension transformed the write
|
||||||
109
tests/test_engine_extension.py
Normal file
109
tests/test_engine_extension.py
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
"""Tests for the typed-extension runtime (SHARD-WP-0014 T2)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.engine import Extension, ExtensionError, ExtensionRuntime, Hook
|
||||||
|
|
||||||
|
|
||||||
|
class Upper(Extension):
|
||||||
|
id = "ext.upper"
|
||||||
|
declares_types = ("upper",)
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
|
||||||
|
|
||||||
|
|
||||||
|
class Bang(Extension):
|
||||||
|
id = "ext.bang"
|
||||||
|
depends_on = ("ext.upper",)
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_WRITE: lambda body, ctx: body + "!"}
|
||||||
|
|
||||||
|
|
||||||
|
class Profiler(Extension):
|
||||||
|
id = "ext.profiler"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_PROFILE: lambda payload, ctx: {"structure": "typed"}}
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime(*exts):
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
for e in exts:
|
||||||
|
rt.register(e)
|
||||||
|
return rt
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_rejects_bad_id_and_noncallable_hook():
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
class BadId(Extension):
|
||||||
|
id = "upper" # missing 'ext.' prefix
|
||||||
|
with pytest.raises(ExtensionError, match="ext."):
|
||||||
|
rt.register(BadId())
|
||||||
|
|
||||||
|
class Liar(Extension):
|
||||||
|
id = "ext.liar"
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_WRITE: "not-callable"} # type: ignore[dict-item]
|
||||||
|
with pytest.raises(ExtensionError, match="not callable"):
|
||||||
|
rt.register(Liar())
|
||||||
|
|
||||||
|
|
||||||
|
def test_transform_hook_chains_in_dependency_order():
|
||||||
|
rt = _runtime(Upper(), Bang())
|
||||||
|
active = rt.activate({"ext.bang"}) # pulls ext.upper via depends_on
|
||||||
|
assert set(active.ids) == {"ext.upper", "ext.bang"}
|
||||||
|
# upper (dependency) runs before bang (dependent): "hi" -> "HI" -> "HI!"
|
||||||
|
assert active.handlers(Hook.ON_WRITE) == ("ext.upper", "ext.bang")
|
||||||
|
assert active.dispatch_transform(Hook.ON_WRITE, "hi") == "HI!"
|
||||||
|
|
||||||
|
|
||||||
|
def test_dependency_closure_is_automatic():
|
||||||
|
rt = _runtime(Upper(), Bang())
|
||||||
|
assert set(rt.activate({"ext.bang"}).ids) == {"ext.upper", "ext.bang"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_unknown_extension_rejected():
|
||||||
|
with pytest.raises(ExtensionError, match="unknown"):
|
||||||
|
ExtensionRuntime().activate({"ext.ghost"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_conflict_rejected():
|
||||||
|
class A(Extension):
|
||||||
|
id = "ext.a"
|
||||||
|
conflicts_with = ("ext.b",)
|
||||||
|
class B(Extension):
|
||||||
|
id = "ext.b"
|
||||||
|
with pytest.raises(ExtensionError, match="conflicts"):
|
||||||
|
_runtime(A(), B()).activate({"ext.a", "ext.b"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_type_collision_rejected():
|
||||||
|
class S1(Extension):
|
||||||
|
id = "ext.s1"
|
||||||
|
declares_types = ("record",)
|
||||||
|
class S2(Extension):
|
||||||
|
id = "ext.s2"
|
||||||
|
declares_types = ("record",)
|
||||||
|
with pytest.raises(ExtensionError, match="type collision"):
|
||||||
|
_runtime(S1(), S2()).activate({"ext.s1", "ext.s2"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_collect_hook_gathers_contributions():
|
||||||
|
active = _runtime(Profiler()).activate({"ext.profiler"})
|
||||||
|
assert active.dispatch_collect(Hook.ON_PROFILE) == [{"structure": "typed"}]
|
||||||
|
|
||||||
|
|
||||||
|
def test_wrong_dispatch_kind_errors():
|
||||||
|
active = _runtime(Profiler()).activate({"ext.profiler"})
|
||||||
|
with pytest.raises(ExtensionError, match="not a transform hook"):
|
||||||
|
active.dispatch_transform(Hook.ON_PROFILE, "x")
|
||||||
|
|
||||||
|
|
||||||
|
def test_unmet_dependency_rejected():
|
||||||
|
# Bang depends on ext.upper, but only Bang is registered.
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
rt.register(Bang())
|
||||||
|
with pytest.raises(ExtensionError, match="unmet dependency|unknown"):
|
||||||
|
rt.activate({"ext.bang"})
|
||||||
59
tests/test_engine_kernel.py
Normal file
59
tests/test_engine_kernel.py
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
"""Tests for the engine kernel (SHARD-WP-0014 T1)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.engine import EngineKernel, extract_wikilinks
|
||||||
|
from shard_wiki.model import Identity
|
||||||
|
|
||||||
|
|
||||||
|
def test_write_creates_then_edits_as_history():
|
||||||
|
k = EngineKernel("eng")
|
||||||
|
p1 = k.write("Home", "first")
|
||||||
|
assert p1.identity == Identity("eng", "Home")
|
||||||
|
assert p1.envelope.source_rev == "1"
|
||||||
|
p2 = k.write("Home", "second")
|
||||||
|
assert p2.envelope.source_rev == "2"
|
||||||
|
assert k.read("Home").body == "second" # latest
|
||||||
|
assert [v.body for v in k.history("Home")] == ["first", "second"] # recoverable history
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_missing_raises():
|
||||||
|
k = EngineKernel("eng")
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
k.read("Nope")
|
||||||
|
|
||||||
|
|
||||||
|
def test_delete_is_recoverable():
|
||||||
|
k = EngineKernel("eng")
|
||||||
|
k.write("Doc", "v1")
|
||||||
|
k.delete("Doc")
|
||||||
|
assert not k.exists("Doc")
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
k.read("Doc")
|
||||||
|
assert [v.body for v in k.history("Doc")] == ["v1"] # history retained
|
||||||
|
k.write("Doc", "v2") # restore by writing
|
||||||
|
assert k.exists("Doc") and k.read("Doc").body == "v2"
|
||||||
|
|
||||||
|
|
||||||
|
def test_keys_and_current_rev():
|
||||||
|
k = EngineKernel("eng")
|
||||||
|
k.write("A", "a")
|
||||||
|
k.write("B", "b")
|
||||||
|
k.write("A", "a2")
|
||||||
|
assert set(k.keys()) == {"A", "B"}
|
||||||
|
assert k.current_rev("A") == "2"
|
||||||
|
assert k.current_rev("Missing") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_links_and_red_link_resolution():
|
||||||
|
k = EngineKernel("eng")
|
||||||
|
k.write("Home", "see [[Target]] and [[Other|labelled]] and [[Target]] again")
|
||||||
|
k.write("Target", "exists")
|
||||||
|
assert k.links("Home") == ["Target", "Other"] # ordered, de-duped, label dropped
|
||||||
|
assert k.resolve_link("Target") == Identity("eng", "Target")
|
||||||
|
assert k.resolve_link("Other") is None # red-link (not yet created)
|
||||||
|
|
||||||
|
|
||||||
|
def test_extract_wikilinks_helper():
|
||||||
|
assert extract_wikilinks("none here") == []
|
||||||
|
assert extract_wikilinks("[[A]] [[B|x]] [[A]]") == ["A", "B"]
|
||||||
98
tests/test_engine_profile.py
Normal file
98
tests/test_engine_profile.py
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
"""Tests for capability-profile-derived-from-extensions (SHARD-WP-0014 T4, E-5)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.engine import (
|
||||||
|
Extension,
|
||||||
|
ExtensionRuntime,
|
||||||
|
Hook,
|
||||||
|
ProfileContribution,
|
||||||
|
derive_profile,
|
||||||
|
engine_base_profile,
|
||||||
|
)
|
||||||
|
from shard_wiki.model import (
|
||||||
|
Addressing,
|
||||||
|
ContentOpacity,
|
||||||
|
NativeQuery,
|
||||||
|
ProfileError,
|
||||||
|
Verb,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class StructExt(Extension):
|
||||||
|
id = "ext.struct"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {
|
||||||
|
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
|
||||||
|
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class AddrExt(Extension):
|
||||||
|
id = "ext.addr"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {
|
||||||
|
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
|
||||||
|
addressing=Addressing.SPAN, verbs_add=frozenset({Verb.TRANSCLUDE_SOURCE})
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class EncryptExt(Extension):
|
||||||
|
id = "ext.encrypt"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(content_opacity=ContentOpacity.ENCRYPTED)}
|
||||||
|
|
||||||
|
|
||||||
|
class QueryExt(Extension):
|
||||||
|
id = "ext.query"
|
||||||
|
|
||||||
|
def hooks(self):
|
||||||
|
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(native_query=NativeQuery.DB_QUERY)}
|
||||||
|
|
||||||
|
|
||||||
|
def _active(*exts, ids=None):
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
for e in exts:
|
||||||
|
rt.register(e)
|
||||||
|
return rt.activate(ids if ids is not None else {e.id for e in exts})
|
||||||
|
|
||||||
|
|
||||||
|
def test_base_profile_is_valid_kernel_minimum():
|
||||||
|
p = engine_base_profile()
|
||||||
|
assert p.supports(Verb.READ) and p.supports(Verb.WRITE)
|
||||||
|
assert not p.supports(Verb.STRUCTURED_PAYLOAD)
|
||||||
|
assert p.addressing is Addressing.PATH
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_extensions_yields_base():
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
active = rt.activate(set())
|
||||||
|
assert derive_profile(active) == engine_base_profile()
|
||||||
|
|
||||||
|
|
||||||
|
def test_activating_extension_raises_the_profile():
|
||||||
|
active = _active(StructExt(), AddrExt())
|
||||||
|
p = derive_profile(active)
|
||||||
|
assert p.supports(Verb.STRUCTURED_PAYLOAD) # from ext.struct
|
||||||
|
assert p.supports(Verb.TRANSCLUDE_SOURCE) # from ext.addr
|
||||||
|
assert p.addressing is Addressing.SPAN # raised by ext.addr
|
||||||
|
assert p.supports(Verb.READ) # base preserved
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_changes_with_active_set():
|
||||||
|
only_struct = derive_profile(_active(StructExt(), AddrExt(), ids={"ext.struct"}))
|
||||||
|
both = derive_profile(_active(StructExt(), AddrExt()))
|
||||||
|
assert not only_struct.supports(Verb.TRANSCLUDE_SOURCE)
|
||||||
|
assert both.supports(Verb.TRANSCLUDE_SOURCE) # E-5: profile reflects what's active
|
||||||
|
|
||||||
|
|
||||||
|
def test_composition_cannot_yield_an_impossible_profile():
|
||||||
|
# encrypted opacity + native query violates §6.5 implication rules -> derive must reject.
|
||||||
|
active = _active(EncryptExt(), QueryExt())
|
||||||
|
with pytest.raises(ProfileError):
|
||||||
|
derive_profile(active)
|
||||||
76
tests/test_error_paths.py
Normal file
76
tests/test_error_paths.py
Normal file
@@ -0,0 +1,76 @@
|
|||||||
|
"""Error-path / contract tests across modules (keeps the suite honest about failure behaviour)."""
|
||||||
|
|
||||||
|
from collections.abc import Iterable
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki import InformationSpace
|
||||||
|
from shard_wiki.adapters import FolderAdapter, ShardAdapter, run_conformance
|
||||||
|
from shard_wiki.engine import EngineKernel
|
||||||
|
from shard_wiki.model import CapabilityProfile, Identity, Page, Placement
|
||||||
|
from shard_wiki.provenance import ProvenanceEnvelope
|
||||||
|
from shard_wiki.union import ResolutionKind, UnionGraph
|
||||||
|
|
||||||
|
|
||||||
|
def _folder(tmp_path, name, files, writable=False):
|
||||||
|
root = tmp_path / name
|
||||||
|
root.mkdir(parents=True, exist_ok=True)
|
||||||
|
for rel, text in files.items():
|
||||||
|
(root / rel).write_text(text, encoding="utf-8")
|
||||||
|
return FolderAdapter(name, root, writable=writable)
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolution_single_on_red_link_raises():
|
||||||
|
u = UnionGraph("s")
|
||||||
|
res = u.resolve("ghost")
|
||||||
|
assert res.kind is ResolutionKind.RED_LINK
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
res.single()
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_unknown_overlay_raises(tmp_path):
|
||||||
|
space = InformationSpace("t")
|
||||||
|
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
space.apply_overlay("does-not-exist")
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_overlay_for_unattached_shard_raises(tmp_path):
|
||||||
|
space = InformationSpace("t")
|
||||||
|
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
|
||||||
|
# draft an overlay whose target shard is not attached -> apply can't find an adapter
|
||||||
|
ov = space.overlays.draft(Identity("ghost", "X"), "body", base_rev=None)
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
space.apply_overlay(ov.overlay_id)
|
||||||
|
|
||||||
|
|
||||||
|
def test_kernel_delete_missing_raises():
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
EngineKernel("eng").delete("nope")
|
||||||
|
|
||||||
|
|
||||||
|
def test_placement_str():
|
||||||
|
assert str(Placement("shardA", "sub/Page")) == "shardA/sub/Page"
|
||||||
|
|
||||||
|
|
||||||
|
class _BrokenProfileAdapter(ShardAdapter):
|
||||||
|
"""profile() raises — the conformance battery must report failure, not crash."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def shard_id(self) -> str:
|
||||||
|
return "broken"
|
||||||
|
|
||||||
|
def profile(self) -> CapabilityProfile:
|
||||||
|
raise RuntimeError("profile blew up")
|
||||||
|
|
||||||
|
def keys(self) -> Iterable[str]:
|
||||||
|
return []
|
||||||
|
|
||||||
|
def read(self, key: str) -> Page:
|
||||||
|
return Page(Identity("broken", key), "x", ProvenanceEnvelope(source_shard="broken"))
|
||||||
|
|
||||||
|
|
||||||
|
def test_conformance_survives_a_broken_profile():
|
||||||
|
report = run_conformance(_BrokenProfileAdapter())
|
||||||
|
assert not report.ok
|
||||||
|
assert any(c.name == "profile-validates" and not c.ok for c in report.checks)
|
||||||
64
tests/test_ext_struct.py
Normal file
64
tests/test_ext_struct.py
Normal file
@@ -0,0 +1,64 @@
|
|||||||
|
"""Tests for the ext.struct built-in extension (SHARD-WP-0014 T6)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki import InformationSpace
|
||||||
|
from shard_wiki.adapters import assert_conformant
|
||||||
|
from shard_wiki.engine import ExtensionRuntime, build_engine_shard
|
||||||
|
from shard_wiki.engine.extensions import StructExt, parse_frontmatter
|
||||||
|
from shard_wiki.model import PageShape, Verb
|
||||||
|
|
||||||
|
_STRUCT_PAGE = "---\ntitle: Spec\nstatus: draft\n---\nbody text"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_frontmatter():
|
||||||
|
fields, has = parse_frontmatter(_STRUCT_PAGE)
|
||||||
|
assert has and fields == {"title": "Spec", "status": "draft"}
|
||||||
|
assert parse_frontmatter("just prose") == ({}, False)
|
||||||
|
assert parse_frontmatter("---\nunterminated") == ({}, False)
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime(allowed=None):
|
||||||
|
rt = ExtensionRuntime()
|
||||||
|
rt.register(StructExt(allowed_fields=allowed))
|
||||||
|
return rt
|
||||||
|
|
||||||
|
|
||||||
|
def test_feature_absent_when_extension_off():
|
||||||
|
shard = build_engine_shard("off", ExtensionRuntime(), activate=set())
|
||||||
|
shard.write("Spec", _STRUCT_PAGE)
|
||||||
|
assert shard.read("Spec").shape is PageShape.PROSE # kernel: opaque prose
|
||||||
|
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # honest absence
|
||||||
|
|
||||||
|
|
||||||
|
def test_feature_present_when_extension_on():
|
||||||
|
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
|
||||||
|
shard.write("Spec", _STRUCT_PAGE)
|
||||||
|
assert shard.read("Spec").shape is PageShape.TYPED_RECORD # tagged by ext.struct
|
||||||
|
assert shard.read("Spec").body.endswith("body text") # content preserved (in-text)
|
||||||
|
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # profile reflects activation (E-5)
|
||||||
|
assert_conformant(shard) # still conformant
|
||||||
|
|
||||||
|
|
||||||
|
def test_plain_page_is_not_tagged_even_when_on():
|
||||||
|
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
|
||||||
|
shard.write("Plain", "no frontmatter here")
|
||||||
|
assert shard.read("Plain").shape is PageShape.PROSE
|
||||||
|
|
||||||
|
|
||||||
|
def test_allowed_fields_validation_rejects_disallowed():
|
||||||
|
shard = build_engine_shard("v", _runtime(allowed={"title"}), activate={"ext.struct"})
|
||||||
|
with pytest.raises(ValueError, match="disallowed fields"):
|
||||||
|
shard.write("Bad", "---\ntitle: ok\nsecret: no\n---\nx")
|
||||||
|
shard.write("Good", "---\ntitle: ok\n---\nx") # allowed field passes
|
||||||
|
assert shard.read("Good").shape is PageShape.TYPED_RECORD
|
||||||
|
|
||||||
|
|
||||||
|
def test_through_information_space_edit():
|
||||||
|
space = InformationSpace("team")
|
||||||
|
space.attach(build_engine_shard("wikiE", _runtime(), activate={"ext.struct"}))
|
||||||
|
space.union.shard("wikiE").write("Doc", "---\ntitle: T\n---\nv1")
|
||||||
|
res = space.edit("Doc", "---\ntitle: T2\n---\nv2") # overlay→apply→write-through
|
||||||
|
assert res.status.value == "applied"
|
||||||
|
page = space.read("Doc")
|
||||||
|
assert page.shape is PageShape.TYPED_RECORD and "v2" in page.body
|
||||||
44
tests/test_folder_adapter.py
Normal file
44
tests/test_folder_adapter.py
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
"""Tests for the FolderAdapter (SHARD-WP-0007 T3)."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.adapters import FolderAdapter
|
||||||
|
from shard_wiki.model import Identity, NotSupported, Verb
|
||||||
|
|
||||||
|
|
||||||
|
def _make_shard(tmp_path, files: dict[str, str]) -> FolderAdapter:
|
||||||
|
for rel, text in files.items():
|
||||||
|
p = tmp_path / rel
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
p.write_text(text, encoding="utf-8")
|
||||||
|
return FolderAdapter("shardA", tmp_path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_keys_and_read(tmp_path):
|
||||||
|
shard = _make_shard(tmp_path, {"Home.md": "# Home", "sub/Topic.md": "topic body"})
|
||||||
|
assert set(shard.keys()) == {"Home", "sub/Topic"}
|
||||||
|
page = shard.read("sub/Topic")
|
||||||
|
assert page.identity == Identity("shardA", "sub/Topic")
|
||||||
|
assert page.body == "topic body"
|
||||||
|
assert page.envelope.source_shard == "shardA"
|
||||||
|
assert page.envelope.source_rev is not None # mtime stamp
|
||||||
|
assert page.placements[0].path == "sub/Topic.md"
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_missing_raises_keyerror(tmp_path):
|
||||||
|
shard = _make_shard(tmp_path, {"Home.md": "x"})
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
shard.read("Nope")
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_is_valid_and_read_only(tmp_path):
|
||||||
|
shard = _make_shard(tmp_path, {"Home.md": "x"})
|
||||||
|
prof = shard.profile() # .validate() already called inside
|
||||||
|
assert prof.supports(Verb.READ)
|
||||||
|
assert not prof.supports(Verb.WRITE)
|
||||||
|
|
||||||
|
|
||||||
|
def test_unsupported_write_is_honest(tmp_path):
|
||||||
|
shard = _make_shard(tmp_path, {"Home.md": "x"})
|
||||||
|
with pytest.raises(NotSupported):
|
||||||
|
shard.write("Home", "new")
|
||||||
131
tests/test_git_adapter.py
Normal file
131
tests/test_git_adapter.py
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
"""Tests for the GitShardAdapter read path + profile (SHARD-WP-0012 T1)."""
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.adapters import GitShardAdapter, run_conformance
|
||||||
|
from shard_wiki.model import (
|
||||||
|
AttachmentMode,
|
||||||
|
History,
|
||||||
|
NotSupported,
|
||||||
|
ProfileError,
|
||||||
|
Substrate,
|
||||||
|
Verb,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _git(repo, *args):
|
||||||
|
subprocess.run(
|
||||||
|
["git", "-C", str(repo), *args],
|
||||||
|
check=True,
|
||||||
|
capture_output=True,
|
||||||
|
env={"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
|
||||||
|
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
|
||||||
|
"PATH": __import__("os").environ.get("PATH", "")},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _repo(tmp_path, files, name="repo"):
|
||||||
|
repo = tmp_path / name
|
||||||
|
repo.mkdir()
|
||||||
|
_git(repo, "init", "--quiet")
|
||||||
|
for rel, text in files.items():
|
||||||
|
p = repo / rel
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
p.write_text(text, encoding="utf-8")
|
||||||
|
_git(repo, "add", rel)
|
||||||
|
_git(repo, "commit", "-m", "seed")
|
||||||
|
return repo
|
||||||
|
|
||||||
|
|
||||||
|
def test_keys_are_tracked_md_paths(tmp_path):
|
||||||
|
repo = _repo(tmp_path, {"Home.md": "h", "docs/Guide.md": "g", "ignore.txt": "x"})
|
||||||
|
adapter = GitShardAdapter("git", repo)
|
||||||
|
assert set(adapter.keys()) == {"Home", "docs/Guide"} # only tracked *.md
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_returns_page_with_commit_sha_rev(tmp_path):
|
||||||
|
repo = _repo(tmp_path, {"Home.md": "welcome"})
|
||||||
|
adapter = GitShardAdapter("git", repo)
|
||||||
|
page = adapter.read("Home")
|
||||||
|
assert page.identity.shard == "git"
|
||||||
|
assert page.body == "welcome"
|
||||||
|
head = subprocess.run(
|
||||||
|
["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True, check=True
|
||||||
|
).stdout.strip()
|
||||||
|
assert page.envelope.source_rev == head # source_rev is the commit sha
|
||||||
|
assert page.envelope.lineage == "git-native"
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_missing_key_raises(tmp_path):
|
||||||
|
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
adapter.read("Nope")
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_validates_implication_rules(tmp_path):
|
||||||
|
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
|
||||||
|
assert profile.substrate is Substrate.GIT
|
||||||
|
assert profile.attachment_mode is AttachmentMode.GIT_IS_STORE
|
||||||
|
assert profile.history is History.GIT_NATIVE # git-is-store ⟹ git-native
|
||||||
|
profile.validate() # raises if the implication rule were violated
|
||||||
|
|
||||||
|
|
||||||
|
def test_profile_is_read_only_in_t1(tmp_path):
|
||||||
|
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
|
||||||
|
assert profile.supports(Verb.READ)
|
||||||
|
assert not profile.supports(Verb.WRITE)
|
||||||
|
|
||||||
|
|
||||||
|
def test_conformance_read_path_passes(tmp_path):
|
||||||
|
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h", "Other.md": "o"}))
|
||||||
|
report = run_conformance(adapter)
|
||||||
|
assert report.ok, report.diff()
|
||||||
|
|
||||||
|
|
||||||
|
def test_unclaimed_write_raises_not_supported(tmp_path):
|
||||||
|
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
|
||||||
|
with pytest.raises(NotSupported):
|
||||||
|
adapter.write("Home", "new") # read-only: honest absence
|
||||||
|
|
||||||
|
|
||||||
|
def test_empty_repo_has_no_keys(tmp_path):
|
||||||
|
repo = tmp_path / "empty"
|
||||||
|
repo.mkdir()
|
||||||
|
_git(repo, "init", "--quiet")
|
||||||
|
adapter = GitShardAdapter("git", repo)
|
||||||
|
assert list(adapter.keys()) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_bad_profile_combo_is_rejected():
|
||||||
|
# Sanity: the implication rule that backs the git profile actually bites when violated.
|
||||||
|
from shard_wiki.model import (
|
||||||
|
AccessGrant,
|
||||||
|
Addressing,
|
||||||
|
CapabilityProfile,
|
||||||
|
ContentOpacity,
|
||||||
|
MergeModel,
|
||||||
|
NativeQuery,
|
||||||
|
OperationalEnvelope,
|
||||||
|
Translation,
|
||||||
|
WriteGranularity,
|
||||||
|
)
|
||||||
|
from shard_wiki.provenance import Liveness
|
||||||
|
|
||||||
|
with pytest.raises(ProfileError):
|
||||||
|
CapabilityProfile(
|
||||||
|
substrate=Substrate.FILES, # not git, but claims git-is-store
|
||||||
|
attachment_mode=AttachmentMode.GIT_IS_STORE,
|
||||||
|
write_granularity=WriteGranularity.NONE,
|
||||||
|
content_opacity=ContentOpacity.TRANSPARENT,
|
||||||
|
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||||
|
access_grant=AccessGrant.OPEN,
|
||||||
|
liveness=Liveness.STATIC,
|
||||||
|
history=History.NONE,
|
||||||
|
merge_model=MergeModel.NONE,
|
||||||
|
addressing=Addressing.PATH,
|
||||||
|
native_query=NativeQuery.NONE,
|
||||||
|
translation=Translation.NATIVE,
|
||||||
|
supported_verbs=frozenset({Verb.READ}),
|
||||||
|
).validate()
|
||||||
116
tests/test_git_adapter_integration.py
Normal file
116
tests/test_git_adapter_integration.py
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
"""GitShardAdapter history adopt + cross-substrate integration (SHARD-WP-0012 T3)."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from shard_wiki.adapters import FolderAdapter, GitShardAdapter
|
||||||
|
from shard_wiki.coordination import ApplyStatus
|
||||||
|
from shard_wiki.space import InformationSpace
|
||||||
|
|
||||||
|
_ENV = {
|
||||||
|
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
|
||||||
|
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
|
||||||
|
"PATH": os.environ.get("PATH", ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _git(repo, *args):
|
||||||
|
return subprocess.run(
|
||||||
|
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
|
||||||
|
).stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _git_repo(tmp_path, files, name="git"):
|
||||||
|
repo = tmp_path / name
|
||||||
|
repo.mkdir()
|
||||||
|
_git(repo, "init", "--quiet")
|
||||||
|
for rel, text in files.items():
|
||||||
|
(repo / rel).parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
(repo / rel).write_text(text, encoding="utf-8")
|
||||||
|
_git(repo, "add", rel)
|
||||||
|
_git(repo, "commit", "-m", "seed")
|
||||||
|
return repo
|
||||||
|
|
||||||
|
|
||||||
|
def _folder(tmp_path, name, files, writable=False):
|
||||||
|
root = tmp_path / name
|
||||||
|
for rel, text in files.items():
|
||||||
|
p = root / rel
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
p.write_text(text, encoding="utf-8")
|
||||||
|
return FolderAdapter(name, root, writable=writable)
|
||||||
|
|
||||||
|
|
||||||
|
# -- history adopt -------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_history_lists_commits_newest_first(tmp_path):
|
||||||
|
repo = _git_repo(tmp_path, {"Home.md": "v1"})
|
||||||
|
adapter = GitShardAdapter("git", repo, writable=True)
|
||||||
|
adapter.write("Home", "v2")
|
||||||
|
history = adapter.history("Home")
|
||||||
|
assert len(history) == 2
|
||||||
|
assert history[0].message == "write Home.md" # newest first
|
||||||
|
assert history[-1].message == "seed"
|
||||||
|
assert all(rev.sha for rev in history)
|
||||||
|
|
||||||
|
|
||||||
|
def test_history_unknown_key_raises(tmp_path):
|
||||||
|
adapter = GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "h"}))
|
||||||
|
with pytest.raises(KeyError):
|
||||||
|
adapter.history("Nope")
|
||||||
|
|
||||||
|
|
||||||
|
# -- cross-substrate integration ----------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_resolve_across_git_and_folder(tmp_path):
|
||||||
|
space = InformationSpace("space")
|
||||||
|
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "git home"})))
|
||||||
|
space.attach(_folder(tmp_path, "notes", {"Daily.md": "folder daily"}))
|
||||||
|
assert space.read("Home").body == "git home" # resolved from the git shard
|
||||||
|
assert space.read("Daily").body == "folder daily" # resolved from the folder shard
|
||||||
|
|
||||||
|
|
||||||
|
def test_chorus_spans_substrates_with_divergence(tmp_path):
|
||||||
|
space = InformationSpace("space")
|
||||||
|
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Shared.md": "from git"})))
|
||||||
|
space.attach(_folder(tmp_path, "notes", {"Shared.md": "from folder"}))
|
||||||
|
res = space.resolve("Shared")
|
||||||
|
assert {p.body for p in res.pages} == {"from git", "from folder"} # chorus across substrates
|
||||||
|
git_page = next(p for p in res.pages if p.identity.shard == "git")
|
||||||
|
assert git_page.envelope.divergence # divergence recorded, not erased
|
||||||
|
|
||||||
|
|
||||||
|
def test_edit_through_git_shard_commits(tmp_path):
|
||||||
|
repo = _git_repo(tmp_path, {"Home.md": "original"})
|
||||||
|
space = InformationSpace("space")
|
||||||
|
space.attach(GitShardAdapter("git", repo, writable=True))
|
||||||
|
result = space.edit("Home", "edited via overlay")
|
||||||
|
assert result.status is ApplyStatus.APPLIED # write-through fast-forward on a git shard
|
||||||
|
assert space.read("Home").body == "edited via overlay"
|
||||||
|
assert int(_git(repo, "rev-list", "--count", "HEAD")) == 2 # the edit became a commit
|
||||||
|
|
||||||
|
|
||||||
|
def test_apply_under_drift_refuses_on_external_commit(tmp_path):
|
||||||
|
repo = _git_repo(tmp_path, {"Home.md": "original"})
|
||||||
|
space = InformationSpace("space")
|
||||||
|
space.attach(GitShardAdapter("git", repo, writable=True))
|
||||||
|
overlay = space.overlay("Home", "my draft") # base_rev = current git sha
|
||||||
|
# Another writer commits to the same path → the sha moves underneath the draft.
|
||||||
|
(repo / "Home.md").write_text("someone else", encoding="utf-8")
|
||||||
|
_git(repo, "add", "Home.md")
|
||||||
|
_git(repo, "commit", "-m", "external")
|
||||||
|
result = space.apply_overlay(overlay.overlay_id)
|
||||||
|
assert result.status is ApplyStatus.REFUSED_DRIFT # never clobber (sha drift detected)
|
||||||
|
# The shard itself is untouched — the external commit stands; the draft remains a draft.
|
||||||
|
assert space.union.shard("git").read("Home").body == "someone else"
|
||||||
|
|
||||||
|
|
||||||
|
def test_overlay_on_read_only_git_shard_kept_as_draft(tmp_path):
|
||||||
|
space = InformationSpace("space")
|
||||||
|
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "ro"}), writable=False))
|
||||||
|
result = space.edit("Home", "wanted change")
|
||||||
|
assert result.status is ApplyStatus.KEPT_DRAFT # read-only target → overlay retained
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user