generated from coulomb/repo-seed
Compare commits
42 Commits
67d851be0b
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 776f5af5a7 | |||
| fd961c83b4 | |||
| cca5bf83c3 | |||
| def699c1eb | |||
| a4e0f52ec1 | |||
| 4231daf94f | |||
| 37681d89b6 | |||
| a8e65235a8 | |||
| d7d046cac0 | |||
| 0b3ab2086f | |||
| d85d019543 | |||
| 3a5acdcb28 | |||
| 34b0c539f3 | |||
| da540d4eea | |||
| 951b24300d | |||
| c731c96634 | |||
| f0fee65cc0 | |||
| 34432c2e15 | |||
| 45a858ead0 | |||
| b31e9bc337 | |||
| e50dcc6b5c | |||
| a165cced33 | |||
| 8393a9c55d | |||
| ff96ee0c48 | |||
| 8b353f1077 | |||
| b9bb1f7d10 | |||
| c40fa3c934 | |||
| 54c2bf2ae5 | |||
| 6d8bd837a4 | |||
| b48a99d3c2 | |||
| 9b7f86ba69 | |||
| 74142096d0 | |||
| 2100e956aa | |||
| e62560eb5a | |||
| b147d3e831 | |||
| cdcf4b09aa | |||
| b21efe307b | |||
| e18397272a | |||
| 0ee972f2e2 | |||
| bb1b54e0af | |||
| b70f1c9acc | |||
| 8de044bbde |
20
.claude/rules/agents.md
Normal file
20
.claude/rules/agents.md
Normal file
@@ -0,0 +1,20 @@
|
||||
## Kaizen Agents
|
||||
|
||||
Specialized agent personas available on demand via the state-hub MCP.
|
||||
|
||||
**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category
|
||||
**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them
|
||||
|
||||
Common agents:
|
||||
|
||||
| Agent | Category | When to use |
|
||||
|-------|----------|-------------|
|
||||
| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature |
|
||||
| `code-refactoring` | quality | Code quality analysis and safe refactoring |
|
||||
| `test-maintenance` | testing | Diagnose and fix failing tests |
|
||||
| `requirements-engineering` | process | Prevent interface/mock mismatches upfront |
|
||||
| `keepaTodofile` | process | Maintain TODO.md during work |
|
||||
| `project-management` | process | Track status, determine next steps |
|
||||
| `datamodel-optimization` | quality | Optimize dataclasses and data structures |
|
||||
|
||||
All 17 agents: call `list_kaizen_agents()` for the full list.
|
||||
8
.claude/rules/architecture.md
Normal file
8
.claude/rules/architecture.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## Architecture
|
||||
|
||||
<!-- TODO: Describe the key design decisions and component structure.
|
||||
Key modules, data flows, external integrations, state machines, etc. -->
|
||||
|
||||
## Quick Reference
|
||||
|
||||
`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference
|
||||
50
.claude/rules/credential-routing.md
Normal file
50
.claude/rules/credential-routing.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Credential and access routing
|
||||
|
||||
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||
|
||||
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||
other credential need belongs to another subsystem. **Do not** message
|
||||
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||
|
||||
### Lookup (do this first)
|
||||
|
||||
```bash
|
||||
warden route find "<describe your need>" --json
|
||||
warden route show <catalog-id> --json
|
||||
```
|
||||
|
||||
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||
|
||||
| Agent runtime | How to orient |
|
||||
| --- | --- |
|
||||
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
|
||||
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||
|
||||
### Quick routing table
|
||||
|
||||
| I need… | Owner | ops-warden executes? |
|
||||
| --- | --- | --- |
|
||||
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||
| Authorization decision | flex-auth | No — route only |
|
||||
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||
|
||||
### Anti-patterns (do not do these)
|
||||
|
||||
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||
|
||||
### Other capabilities (reuse-surface)
|
||||
|
||||
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||
get wrong.
|
||||
|
||||
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||
38
.claude/rules/first-session.md
Normal file
38
.claude/rules/first-session.md
Normal file
@@ -0,0 +1,38 @@
|
||||
## First Session Protocol
|
||||
|
||||
Triggered when `get_domain_summary("consumer")` shows **no workstreams**.
|
||||
The project is registered but work has not yet been structured.
|
||||
|
||||
**Step 1 — Read, don't write**
|
||||
- `~/the-custodian/canon/projects/consumer/project_charter_v0.1.md` — purpose, scope
|
||||
- `~/the-custodian/canon/projects/consumer/roadmap_v0.1.md` — planned phases
|
||||
- Scan repo root: README, directory structure, existing code or docs
|
||||
|
||||
**Step 2 — Survey in-progress work**
|
||||
Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete.
|
||||
|
||||
**Step 3 — Propose workstreams to Bernd**
|
||||
Propose 1–3 workstreams — each a coherent strand, weeks to months, anchored to a
|
||||
roadmap phase. **Wait for approval before creating.**
|
||||
|
||||
**Step 4 — Create workplan file first, then DB record (ADR-001)**
|
||||
```
|
||||
workplans/SHARD-WP-NNNN-<slug>.md ← write this first
|
||||
```
|
||||
Then register in the hub:
|
||||
```
|
||||
create_workstream(topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", title="...", owner="...", description="...")
|
||||
create_task(workstream_id="<id>", title="...", priority="high|medium|low")
|
||||
```
|
||||
|
||||
**Step 5 — Record the setup**
|
||||
```
|
||||
add_progress_event(
|
||||
summary="First session: structured consumer into N workstreams, M tasks",
|
||||
event_type="milestone",
|
||||
topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0",
|
||||
detail={"workstreams": [...], "tasks_created": M}
|
||||
)
|
||||
```
|
||||
|
||||
<!-- Delete or archive this file once past first session -->
|
||||
8
.claude/rules/repo-boundary.md
Normal file
8
.claude/rules/repo-boundary.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## Repo boundary
|
||||
|
||||
This repo owns **shard-wiki** only. It does not own:
|
||||
|
||||
<!-- TODO: List what belongs in adjacent repos, e.g.:
|
||||
- SSH key management → railiance-infra/
|
||||
- State hub code → state-hub/
|
||||
-->
|
||||
5
.claude/rules/repo-identity.md
Normal file
5
.claude/rules/repo-identity.md
Normal file
@@ -0,0 +1,5 @@
|
||||
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
|
||||
|
||||
**Domain:** consumer
|
||||
**Repo slug:** shard-wiki
|
||||
**Topic ID:** 4c2e5315-2cb9-447c-9d16-a39bdb0aabd0
|
||||
85
.claude/rules/session-protocol.md
Normal file
85
.claude/rules/session-protocol.md
Normal file
@@ -0,0 +1,85 @@
|
||||
## Session Protocol
|
||||
|
||||
Dev Hub (State Hub API): http://127.0.0.1:8000
|
||||
MCP server name in `~/.claude.json`: `dev-hub`
|
||||
|
||||
**Step 1 — Orient**
|
||||
|
||||
Read the offline-safe brief first — it works without a live hub connection:
|
||||
```bash
|
||||
cat .custodian-brief.md
|
||||
```
|
||||
Then call the MCP tool for richer cross-domain context when MCP tools are exposed:
|
||||
```
|
||||
get_domain_summary("consumer")
|
||||
```
|
||||
If MCP tools are unavailable in the current agent session, use the REST API:
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool
|
||||
```
|
||||
If the hub is offline: `cd ~/state-hub && make api`
|
||||
|
||||
**Step 2 — Check inbox**
|
||||
With MCP tools:
|
||||
```
|
||||
get_messages(to_agent="shard-wiki", unread_only=True)
|
||||
```
|
||||
Mark read with `mark_message_read(message_id)`. Reply or act on coordination
|
||||
requests before proceeding.
|
||||
|
||||
Without MCP tools:
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
|
||||
| python3 -m json.tool
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||
-H "Content-Type: application/json" -d '{}'
|
||||
```
|
||||
|
||||
**Step 3 — Scan workplans**
|
||||
```bash
|
||||
ls workplans/
|
||||
```
|
||||
For each file with `status: ready`, `active`, or `blocked`, note pending
|
||||
`wait`/`todo`/`progress` tasks.
|
||||
|
||||
**Step 4 — Present brief**
|
||||
|
||||
1. **Active workstreams** for `consumer` — title, task counts, blocking decisions
|
||||
2. **Pending tasks** from `workplans/` + any `[repo:shard-wiki]` hub tasks
|
||||
3. **Goal guidance** — if `goal_guidance` in summary:
|
||||
- `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"*
|
||||
- `alignment_warnings`: flag if active work is not aligned with current goal
|
||||
4. **Suggested next action** — highest-priority open item
|
||||
5. **SBOM status** — flag if `last_sbom_at` is unset for this repo
|
||||
|
||||
If no workstreams: follow First Session Protocol (`first-session.md`).
|
||||
|
||||
**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()`
|
||||
|
||||
> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`)
|
||||
> are First Session Protocol only. Work structure belongs in repo files (ADR-001).
|
||||
|
||||
**Session close:**
|
||||
With MCP tools:
|
||||
```
|
||||
add_progress_event(summary="...", topic_id="4c2e5315-2cb9-447c-9d16-a39bdb0aabd0", workstream_id="<uuid>")
|
||||
```
|
||||
Without MCP tools:
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"topic_id":"4c2e5315-2cb9-447c-9d16-a39bdb0aabd0","workstream_id":"<uuid>","event_type":"note","summary":"what changed","author":"codex"}'
|
||||
```
|
||||
If workplan files were modified, ensure the local copy is up to date first:
|
||||
```bash
|
||||
git -C <repo_path> pull --ff-only
|
||||
cd ~/state-hub && make fix-consistency REPO=shard-wiki
|
||||
```
|
||||
For repos where implementation runs on a remote machine (e.g. CoulombCore),
|
||||
use the combined target which pulls before fixing:
|
||||
```bash
|
||||
cd ~/state-hub && make fix-consistency-remote REPO=shard-wiki
|
||||
```
|
||||
**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback
|
||||
will sync the file to match DB. **C-16** (repo behind remote) blocks all writes
|
||||
until you pull — intentional to prevent clobbering remote progress.
|
||||
19
.claude/rules/stack-and-commands.md
Normal file
19
.claude/rules/stack-and-commands.md
Normal file
@@ -0,0 +1,19 @@
|
||||
## Stack
|
||||
|
||||
<!-- TODO: Fill in language, frameworks, and key dependencies -->
|
||||
- **Language:**
|
||||
- **Key deps:**
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
# TODO: Fill in the standard commands for this repo
|
||||
|
||||
# Install dependencies
|
||||
|
||||
# Run tests
|
||||
|
||||
# Lint / type check
|
||||
|
||||
# Build / package (if applicable)
|
||||
```
|
||||
40
.claude/rules/workplan-convention.md
Normal file
40
.claude/rules/workplan-convention.md
Normal file
@@ -0,0 +1,40 @@
|
||||
## Workplan Convention (ADR-001)
|
||||
|
||||
File location: `workplans/SHARD-WP-NNNN-<slug>.md`
|
||||
ID prefix: `SHARD-WP-`
|
||||
|
||||
Work items originate as files in this repo **before** being registered in the hub.
|
||||
|
||||
Canonical workplan/workstream frontmatter statuses are:
|
||||
`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`.
|
||||
Use `proposed` for a newly drafted plan, `ready` after review against current
|
||||
repo state, and `finished` when implementation is complete. `stalled` and
|
||||
`needs_review` are derived health labels, not stored statuses.
|
||||
|
||||
Closed workplans may be moved to `workplans/archived/` with a completion-date
|
||||
prefix: `YYMMDD-SHARD-WP-NNNN-<slug>.md`. The frontmatter id remains
|
||||
unchanged; the prefix is only for quick visual reference.
|
||||
|
||||
Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**:
|
||||
`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids
|
||||
`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed
|
||||
directly. Promote anything requiring analysis, design, approval, dependencies, or
|
||||
multiple planned phases into a normal workplan.
|
||||
|
||||
Ecosystem todos from other agents arrive as `[repo:shard-wiki]` hub tasks —
|
||||
visible at session start. Pick one up by creating the workplan file, then registering
|
||||
the workstream.
|
||||
|
||||
Task blocks use this shape:
|
||||
|
||||
```task
|
||||
id: SHARD-WP-NNNN-T01
|
||||
status: wait | todo | progress | done | cancel
|
||||
priority: high | medium | low
|
||||
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
```
|
||||
|
||||
Status progression is `todo` → `progress` → `done`; use `wait` for waiting or
|
||||
blocked work and `cancel` for stopped work.
|
||||
|
||||
<!-- Ralph Loop rules and HEUREKA sequence: ~/.claude/CLAUDE.md — do not duplicate here -->
|
||||
@@ -2,21 +2,11 @@
|
||||
# Custodian Brief — shard-wiki
|
||||
|
||||
**Domain:** whynot
|
||||
**Last synced:** 2026-06-15 20:11 UTC
|
||||
**Last synced:** 2026-06-15 22:57 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
||||
### wiki-engine prep — reuse-surface registration, UC-catalog systematization, WikiEngineCoreArchitecture
|
||||
Progress: 1/6 done | workstream_id: `04a25a53-2169-41d4-88c0-5035a06e72ef`
|
||||
|
||||
**Open tasks:**
|
||||
- · Systematize the UseCaseCatalog around a reuse-surface capability structure `d83d0f96`
|
||||
- · Surface capability gaps / suggestions to the reuse surface `06b62406`
|
||||
- · INTENT amendment + ratified decision (additive engine = reference shard backend) `1d0ef72b`
|
||||
- · Author spec/WikiEngineCoreArchitecture.md `4712bbfe`
|
||||
- · Wire-up & close-out `1c383414`
|
||||
|
||||
### second adapter — git-IS-store shard (contract validation on a new substrate)
|
||||
Progress: 0/3 done | workstream_id: `9e24eeb0-c0f0-41e6-a1ca-88d71e4139ea`
|
||||
|
||||
|
||||
17
.repo-classification.yaml
Normal file
17
.repo-classification.yaml
Normal file
@@ -0,0 +1,17 @@
|
||||
repo_classification:
|
||||
standard: Repo Classification Standard
|
||||
version: '1.0'
|
||||
classified_at: '2026-06-22'
|
||||
classified_by: agent
|
||||
category: project
|
||||
domain: consumer
|
||||
secondary_domains: []
|
||||
capability_tags:
|
||||
- knowledge
|
||||
- documentation
|
||||
business_stake:
|
||||
- product
|
||||
- experience
|
||||
business_mechanics:
|
||||
- coordination
|
||||
- operation
|
||||
243
AGENTS.md
243
AGENTS.md
@@ -1,62 +1,219 @@
|
||||
# AGENTS.md
|
||||
# shard-wiki — Agent Instructions
|
||||
|
||||
Guidance for agents working in `shard-wiki`.
|
||||
## Repo Identity
|
||||
|
||||
## Read First
|
||||
**Purpose:** Git-based Markdown wiki orchestrator and federation layer. Python (src/ layout, hatchling, pytest). Early-stage: scaffold + INTENT.md defined, domain model not yet implemented. See INTENT.md for authoritative scope.
|
||||
|
||||
1. `INTENT.md` — aspiration and boundaries (stable; architectural changes are rare).
|
||||
2. `SCOPE.md` — what we are achieving now and current maturity.
|
||||
3. `.custodian-brief.md` — State Hub snapshot (generated; do not edit manually).
|
||||
**Domain:** consumer
|
||||
**Repo slug:** shard-wiki
|
||||
**Topic ID:** `4c2e5315-2cb9-447c-9d16-a39bdb0aabd0`
|
||||
**Workplan prefix:** `SHARD-WP-`
|
||||
|
||||
## Documentation Layout
|
||||
---
|
||||
|
||||
This repo follows the CoulombSocial / HelixForge / MarkiTect documentation
|
||||
layout (recommendation, not strict law). Efficient retrieval by purpose:
|
||||
## State Hub Integration
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `INTENT.md` | Aspiration and boundaries |
|
||||
| `SCOPE.md` | Top-level view of current achievement; closes gap to INTENT |
|
||||
| `research/` | Exploration results (`yymmdd-` prefix on files or subdirs) |
|
||||
| `demand/` | Inbound requests not yet reviewed into spec or workplans |
|
||||
| `spec/` | Implementation guardrails (PRD, TSD, use cases, architecture) |
|
||||
| `workplans/` | State Hub–registered implementation tasks |
|
||||
| `docs/` | Stakeholder documentation (users, developers, humans, agents) |
|
||||
| `wiki/` | Perspective-free interconnected knowledge (wiki UI when connected) |
|
||||
| `issues/` | Mirror of relevant open tickets when ticket systems are in use |
|
||||
| `history/` | Archived material (`yymmdd-` prefix); out of scope for daily work |
|
||||
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
|
||||
there is no MCP server for Codex agents.
|
||||
|
||||
**Mode of operation:** close SCOPE → INTENT while learning; refine both as needed.
|
||||
| Context | URL |
|
||||
|---------|-----|
|
||||
| Local workstation | `http://127.0.0.1:8000` |
|
||||
| Remote via tunnel | `http://127.0.0.1:18000` |
|
||||
|
||||
## Domain Vocabulary
|
||||
|
||||
Honor terms from `INTENT.md`: shard, root entity, adapter contract, projection,
|
||||
overlay, coordination journal, shard modes. Do not invent parallel vocabulary.
|
||||
|
||||
## Build And Test
|
||||
### Orient at session start
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
pytest
|
||||
ruff check
|
||||
ruff format
|
||||
# Offline brief — works without hub connection
|
||||
cat .custodian-brief.md
|
||||
|
||||
# Active workstreams for this domain
|
||||
curl -s "http://127.0.0.1:8000/workstreams/?topic_id=4c2e5315-2cb9-447c-9d16-a39bdb0aabd0&status=active" \
|
||||
| python3 -m json.tool
|
||||
|
||||
# Check inbox
|
||||
curl -s "http://127.0.0.1:8000/messages/?to_agent=shard-wiki&unread_only=true" \
|
||||
| python3 -m json.tool
|
||||
```
|
||||
|
||||
## State Hub
|
||||
Mark a message read:
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||
-H "Content-Type: application/json" -d '{}'
|
||||
```
|
||||
|
||||
Workplans register with State Hub. After workplan changes:
|
||||
### Log progress (required at session close)
|
||||
|
||||
```bash
|
||||
cd ~/state-hub && make fix-consistency REPO=shard-wiki
|
||||
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"summary": "what was done",
|
||||
"event_type": "note",
|
||||
"author": "codex",
|
||||
"workstream_id": "<uuid>",
|
||||
"task_id": "<uuid>"
|
||||
}'
|
||||
```
|
||||
|
||||
Finished or canceled workplans move to `history/` with a `yymmdd-` archive prefix.
|
||||
Omit `workstream_id` / `task_id` when not applicable.
|
||||
|
||||
## Where To Put New Material
|
||||
### Update task status
|
||||
|
||||
- Exploratory analysis → `research/yymmdd-<topic>/`
|
||||
- Raw feature ask or external requirement → `demand/`
|
||||
- Reviewed design ready to guide code → `spec/`
|
||||
- Implementation tasks → `workplans/`
|
||||
- User/dev/agent how-to → `docs/`
|
||||
- Collaborative unstructured notes → `wiki/`
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"status": "progress"}'
|
||||
# values: wait | todo | progress | done | cancel
|
||||
```
|
||||
|
||||
### Flag a task for human review
|
||||
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"needs_human": true, "intervention_note": "reason"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Session Protocol
|
||||
|
||||
**Start:**
|
||||
1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe)
|
||||
2. Check inbox: `GET /messages/?to_agent=shard-wiki&unread_only=true`; mark read
|
||||
3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks
|
||||
4. Check human-needed tasks: `GET /tasks/?needs_human=true`
|
||||
|
||||
**During work:**
|
||||
- Update task statuses in workplan files as tasks progress
|
||||
- Record significant decisions via `POST /decisions/`
|
||||
|
||||
**Close:**
|
||||
1. Update workplan file task statuses to reflect progress
|
||||
2. Log: `POST /progress/` with a summary of what changed
|
||||
3. Note for the custodian operator: after workplan file changes, run from
|
||||
`~/state-hub`:
|
||||
```bash
|
||||
make fix-consistency REPO=shard-wiki
|
||||
```
|
||||
This syncs task status from files into the hub DB.
|
||||
|
||||
---
|
||||
|
||||
## Credential and access routing
|
||||
|
||||
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||
|
||||
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||
other credential need belongs to another subsystem. **Do not** message
|
||||
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||
|
||||
### Lookup (do this first)
|
||||
|
||||
```bash
|
||||
warden route find "<describe your need>" --json
|
||||
warden route show <catalog-id> --json
|
||||
```
|
||||
|
||||
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||
|
||||
| Agent runtime | How to orient |
|
||||
| --- | --- |
|
||||
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=shard-wiki` is for coordination, not secret vending |
|
||||
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||
|
||||
### Quick routing table
|
||||
|
||||
| I need… | Owner | ops-warden executes? |
|
||||
| --- | --- | --- |
|
||||
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||
| Authorization decision | flex-auth | No — route only |
|
||||
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||
|
||||
### Anti-patterns (do not do these)
|
||||
|
||||
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||
|
||||
### Other capabilities (reuse-surface)
|
||||
|
||||
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||
get wrong.
|
||||
|
||||
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||
|
||||
<!-- REPO-AGENTS-EXTENSIONS -->
|
||||
<!-- Append repo-specific agent instructions below this marker.
|
||||
The state-hub template sync preserves content after this line. -->
|
||||
|
||||
---
|
||||
|
||||
## Workplan Convention (ADR-001)
|
||||
|
||||
Work items originate as files in this repo — not in the hub. The hub is a
|
||||
read/cache/index layer that rebuilds from files.
|
||||
|
||||
**File location:** `workplans/SHARD-WP-NNNN-<slug>.md`
|
||||
|
||||
**Archived location:** finished workplans may move to
|
||||
`workplans/archived/YYMMDD-SHARD-WP-NNNN-<slug>.md`. The `YYMMDD` prefix is
|
||||
the completion/archive date; the frontmatter `id` does not change.
|
||||
|
||||
**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use
|
||||
`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use
|
||||
this only for low-risk work completed directly; create a normal workplan for
|
||||
anything needing analysis, design, approval, dependencies, or multiple phases.
|
||||
|
||||
**Frontmatter:**
|
||||
|
||||
```yaml
|
||||
---
|
||||
id: SHARD-WP-NNNN
|
||||
type: workplan
|
||||
title: "..."
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: proposed | ready | active | blocked | backlog | finished | archived
|
||||
owner: codex
|
||||
topic_slug: ...
|
||||
created: "YYYY-MM-DD"
|
||||
updated: "YYYY-MM-DD"
|
||||
state_hub_workstream_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
---
|
||||
```
|
||||
|
||||
Use `proposed` for a new draft, `ready` after review against current repo
|
||||
state, and `finished` after implementation. `stalled` and `needs_review` are
|
||||
derived health labels, not frontmatter statuses.
|
||||
|
||||
**Task block format** (one per `##` section):
|
||||
|
||||
```
|
||||
## Task Title
|
||||
|
||||
` ` `task
|
||||
id: SHARD-WP-NNNN-T01
|
||||
status: wait | todo | progress | done | cancel
|
||||
priority: high | medium | low
|
||||
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
` ` `
|
||||
|
||||
Task description text.
|
||||
```
|
||||
|
||||
Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work.
|
||||
|
||||
To create a new workplan:
|
||||
1. Write the file following the format above
|
||||
2. Notify the custodian operator to run `make fix-consistency REPO=shard-wiki`
|
||||
(or send a message to the hub agent via `POST /messages/`)
|
||||
|
||||
63
CLAUDE.md
63
CLAUDE.md
@@ -1,53 +1,12 @@
|
||||
# CLAUDE.md
|
||||
# shard-wiki — Claude Code Instructions
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Repository status
|
||||
|
||||
This is an **early-stage Python repository**. The package scaffold (`src/shard_wiki/`, `tests/`, `pyproject.toml`) exists with only smoke tests — the domain model is not yet implemented. Read `INTENT.md` (aspiration), `SCOPE.md` (current achievement), and `AGENTS.md` (layout and conventions) before designing anything. Close the gap from SCOPE to INTENT via `research/`, `spec/`, and `workplans/`.
|
||||
|
||||
## What this project is
|
||||
|
||||
`shard-wiki` is a **Git-based Markdown wiki orchestrator and federation layer**, not a wiki engine. It lets multiple heterogeneous wiki-shaped page stores (**shards**) attach to a shared root entity and be presented as a **union of pages**, while preserving each shard's separate storage, provenance, capabilities, and history.
|
||||
|
||||
The core job is orchestration across backends — Git repos, repo subdirectories (`wiki/`), Gitea wikis, local folders, Obsidian vaults, WebDAV/Nextcloud directories, Coulomb spaces — never replacing or homogenizing them.
|
||||
|
||||
## Core domain model (the concepts code must honor)
|
||||
|
||||
These abstractions come from `INTENT.md` and define the architecture. New code should map onto them rather than inventing parallel vocabulary:
|
||||
|
||||
- **Shard** — an independently meaningful page store attached to a root entity. Shards have *sovereignty*: their own backend, capabilities, limits, history, and identity model. Not all shards are Git-native.
|
||||
- **Root entity / information space** — the joined space that shards attach to. Each information space should have a **Git-addressable coordination layer** (history, patches, review, backup, reconciliation) even when individual shards are not Git-native.
|
||||
- **Shard adapter contract** — the versioned interface a backend implements to participate. Adapters are **capability-aware**: the core must model explicitly which operations a shard supports (read, write, diff, merge, lock, version, publish, accept patches) rather than assuming uniformity.
|
||||
- **Wiki page model** — a stable, versioned, Markdown-first but backend-neutral representation of pages, paths, links, metadata, revisions.
|
||||
- **Projection** — a lazy, cache-like local view of remote/external shard content. Prefer lazy projection over eager copying.
|
||||
- **Overlay** — a non-destructive local edit against a remote, read-only, or capability-limited shard, representable as drafts/patches/commits/merge requests *before* destructive application ("overlay before mutation").
|
||||
- **Coordination journal** — the Git-backed record of change flows for an information space.
|
||||
- **Shard modes** — read-only, write-through, mirrored, projected, cached, canonical.
|
||||
|
||||
## Design constraints to enforce in code
|
||||
|
||||
These are hard boundaries from `INTENT.md`; treat violations as design bugs:
|
||||
|
||||
- **Mechanism over policy.** Provide primitives for federation, sync, overlays, patching, conflict detection, projection, reconciliation. Do *not* hard-code one editorial/sync/conflict/canonical-source policy — keep those configurable.
|
||||
- **Union without erasure.** Always preserve provenance: which shard a page came from, its freshness, whether it is cached, whether it has overlays, whether it diverges from an equivalent page elsewhere. Never hide authorship, conflicts, freshness, or backend limitations.
|
||||
- **No silent remote mutation.** Do not mutate remote systems without explicit adapter support and user intent.
|
||||
- **Graceful degradation.** Limited backends must still be usable as read-only/cache/projection/backup/patch targets.
|
||||
- **Not a file-sync daemon.** Synchronization is wiki-page-semantic, not generic file mirroring.
|
||||
|
||||
`INTENT.md` has a "Stability Note": changes that redefine what a shard is, Git's role, how root entities are modeled, or whether this is an orchestrator vs. an engine are **architectural changes** and should be rare and deliberate.
|
||||
|
||||
## Build, test, run
|
||||
|
||||
Python with a `src/` layout, built via hatchling, tested with pytest. Tests run against the source tree directly (`pythonpath = ["src"]` in `pyproject.toml`), so no install/editable step is required to run them.
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev]" # one-time: install dev tooling (pytest, pytest-cov, ruff)
|
||||
pytest # run the full test suite
|
||||
pytest tests/test_package.py::test_version_is_exposed # run a single test
|
||||
pytest --cov # run with coverage
|
||||
ruff check # lint
|
||||
ruff format # format
|
||||
```
|
||||
|
||||
Note: the system `pytest` is 7.4.x; `minversion` in `pyproject.toml` is pinned to `7.0` to match. Bump it if a newer pytest is installed into the dev environment.
|
||||
@SCOPE.md
|
||||
@.claude/rules/repo-identity.md
|
||||
@.claude/rules/session-protocol.md
|
||||
@.claude/rules/first-session.md
|
||||
@.claude/rules/workplan-convention.md
|
||||
@.claude/rules/stack-and-commands.md
|
||||
@.claude/rules/architecture.md
|
||||
@.claude/rules/repo-boundary.md
|
||||
@.claude/rules/credential-routing.md
|
||||
@.claude/rules/agents.md
|
||||
|
||||
10
INTENT.md
10
INTENT.md
@@ -16,6 +16,8 @@ The goal is to allow independently stored and differently implemented wikis, pag
|
||||
|
||||
The repository provides a **shard orchestration layer** for interconnected Markdown and markup-based wiki content.
|
||||
|
||||
Equivalently, shard-wiki can be used as a **headless, API-first wiki engine** — optimized for **integrating heterogeneous data sources** and for **efficient access by agents and automation** — that ships its own native engine as one (canonical-mode) shard among many. There is no bundled UI: presentation and rendering are consumer concerns.
|
||||
|
||||
It allows wiki-like systems to:
|
||||
|
||||
* Attach heterogeneous page stores as shards of a shared information space
|
||||
@@ -30,6 +32,7 @@ It allows wiki-like systems to:
|
||||
* Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
|
||||
* Allow existing wiki engines to become federation-capable through a shared API
|
||||
* Allow non-federation-aware systems to participate through adapters and projections
|
||||
* Serve as a **headless, API-first wiki engine** (a small typed-extension core) that integrates heterogeneous data sources and is consumed efficiently by agents and automation
|
||||
|
||||
It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a **composable federated wiki space**.
|
||||
|
||||
@@ -85,7 +88,7 @@ A mature `shard-wiki` should allow each participating shard to see the others as
|
||||
|
||||
This repository is **not** intended to:
|
||||
|
||||
* Replace all wiki engines with a single canonical wiki implementation
|
||||
* Replace all wiki engines with a single canonical wiki implementation *(shard-wiki MAY still provide its own native, headless, API-first engine as one optional shard backend — see Design Principles — but never as a mandated or universal replacement)*
|
||||
* Force every shard to use the same backend, database, directory layout, or storage format
|
||||
* Require every participating system to become federation-aware
|
||||
* Require every participating shard to be Git-native
|
||||
@@ -148,6 +151,9 @@ Policy decisions such as conflict preference, canonical source selection, public
|
||||
* **Composable integration**
|
||||
Wiki engines should be able to use the `shard-wiki` API to become federation-enabled without reimplementing federation internally.
|
||||
|
||||
* **Native reference engine (additive, headless & API-first)**
|
||||
shard-wiki MAY provide its own native wiki-engine as a **canonical-mode shard backend** — a **small core** with a **typed-extension framework**, activated **per shard** (only what you need). It is **headless and API-first** (no bundled UI; presentation/rendering are consumer concerns) and tuned for **integrating heterogeneous data sources** and **efficient agent/automation access**. It is *one shard type among many*, implemented against shard-wiki's own adapter contract; it does **not** replace other engines, mandate a single implementation, or change shard-wiki's role as an orchestrator. Shard sovereignty and union-without-erasure are preserved.
|
||||
|
||||
* **Open by default, progressively governed**
|
||||
A standalone `shard-wiki` must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an *additive capability*, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).
|
||||
|
||||
@@ -201,3 +207,5 @@ Such changes should be rare, because they affect all downstream systems relying
|
||||
|
||||
In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether `shard-wiki` is an orchestrator rather than a wiki engine should be treated as architectural changes.
|
||||
|
||||
**Amendment — 2026-06-15 (SHARD-WP-0013 T4, decision `84ffdb48`):** admits an **additive** native reference wiki-engine — **headless, API-first**, a small typed-extension core — as a **canonical-mode shard backend** optimized for data-source integration and agent access. Deliberate, narrow scope change; shard-wiki remains an orchestrator and neither mandates nor replaces other engines. (Mirrors the earlier auth-in-core amendment precedent.)
|
||||
|
||||
|
||||
10
SCOPE.md
10
SCOPE.md
@@ -17,11 +17,11 @@ Learnings update both SCOPE and INTENT where necessary.
|
||||
|
||||
| Layer | State |
|
||||
|-------|-------|
|
||||
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. attach→resolve→read + edit/overlay/apply work; 64 tests green |
|
||||
| Code | Foundation slice implemented (SHARD-WP-0007): `provenance` + `policy` leaves, `model` (Identity/Placement/Span/Page/CapabilityProfile), `adapters` (contract + FolderAdapter + conformance suite), `coordination` (event-sourced DecisionLog), `union` (resolution + chorus, overlay-aware), `InformationSpace` orchestrator. Write path added (SHARD-WP-0008): writable adapter, overlay engine (draft→patch→apply-under-drift), edit() unifies write-through + overlay-before-mutation. Native engine implemented (SHARD-WP-0014): `engine` (kernel + typed-extension runtime + per-shard activation [ADR-0001] + capability-profile-from-extensions + EngineShardAdapter + the `ext.struct` built-in) — an engine shard attaches to an InformationSpace as a canonical-mode shard. Git-backed coordination log (SHARD-WP-0009): `DecisionLog` storage factored behind an `EventStore`; `GitEventStore` makes the log git-addressable (each space a ref, append = immutable CAS-guarded commit), a per-space `AppendAuthority` (lease) gives a single-writer total order with re-grantable HA hand-off, cross-process read-your-writes verified, and a verbatim one-time importer (`migrate_space`/JSONL) replays in-memory logs into git; `InformationSpace.git_backed(...)` wires it. Derived views (SHARD-WP-0010): `views` (wikilink + red-link model, BackLinks, RecentChanges, AllPages/SiteMap) — recomputable, provenance-carrying, presentation-free, exposed via `InformationSpace.backlinks/recent_changes/all_pages/site_map`. Incremental-first derived tier (SHARD-WP-0011): `incremental` (indexed equivalence via MinHash/LSH blocking + verify, change-driven delta maintenance with retraction/propagation, Merkle-style digest + self-healing I-2 consistency-checker, `UnionIndex` routed behind `InformationSpace.all_pages` with rebuild as explicit fallback). Second adapter (SHARD-WP-0012): `GitShardAdapter` — git-IS-store substrate (read=tracked *.md, write=commit, current_rev=per-path sha for drift, adopted git-native history), passes conformance, works across folder+git shards in union/overlay/edit with no core change (capability-as-data proven on a second substrate). 196 tests green, ~97% coverage |
|
||||
| Intent | `INTENT.md` established; authorization-in-core amendments drafted |
|
||||
| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki + Quip + MojoMojo + Oddmuse + UseModWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
|
||||
| Demand | NetKingdom integration asks captured, not yet negotiated |
|
||||
| Spec | CoreArchitectureBlueprint (whole-system architecture, hardened via SHARD-WP-0005) + ArchitectureBlueprint (auth/history) drafted; UseCaseCatalog 84 UCs from research; PRD/TSD scaffolds |
|
||||
| Spec | CoreArchitectureBlueprint (whole-system, hardened via SHARD-WP-0005/0006) + FederationArchitecture + FederationRequirements + TSD §A adapter contract + ArchitectureBlueprint (auth/history) + WikiEngineCoreArchitecture (headless API-first engine, SHARD-WP-0013) drafted; UseCaseCatalog 84 UCs (+ engine capability-structure layer); PRD scaffold |
|
||||
| Work | `SHARD-WP-0001` **done** (6 ADRs: yawex-derived federation requirements → `spec/FederationRequirements.md`); `SHARD-WP-0002` **done** (18 tasks → `FederationArchitecture.md` [T1–T10, T17] + `TechnicalSpecificationDocument.md` §A adapter contract [T11–T16, T18]); `SHARD-WP-0003` **done** (9 engine dives complete); `SHARD-WP-0004` **done** (all 8 computational-knowledge dives T1–T8 complete + "computational page model" synthesis); `SHARD-WP-0005` **done** (9 tasks: CoreArchitectureBlueprint hardened against the 260615 review); `SHARD-WP-0006` **done** (5 tasks: round-2 hardening — overview reconciled, event-sourced coordination + append authority, adapter conformance, incremental correctness + I-2 verification) |
|
||||
|
||||
## In Scope (today)
|
||||
@@ -32,11 +32,15 @@ Learnings update both SCOPE and INTENT where necessary.
|
||||
- Authorization model design (delegated authentication, core authorization).
|
||||
- Shard adapter contract and wiki page model (to be specified, then implemented).
|
||||
- Git-backed coordination journal for information spaces.
|
||||
- A **native, headless, API-first wiki-engine core** (small typed-extension core, as a
|
||||
canonical-mode shard backend) — design via SHARD-WP-0013; optimized for data-source
|
||||
integration and agent access.
|
||||
- State Hub workplan registration and consistency sync.
|
||||
|
||||
## Out Of Scope (today)
|
||||
|
||||
- A standalone wiki engine UI or rendering pipeline.
|
||||
- A wiki-engine **UI or rendering pipeline** (the engine is headless/API-first; presentation
|
||||
is a consumer concern). A bundled standalone UI is not provided.
|
||||
- Authentication, credential storage, or user directory implementation.
|
||||
- Hard-coded editorial, sync, or conflict-resolution policy.
|
||||
- Generic file mirroring independent of wiki-page semantics.
|
||||
|
||||
@@ -36,6 +36,11 @@ pythonpath = ["src"]
|
||||
branch = true
|
||||
source = ["shard_wiki"]
|
||||
|
||||
[tool.coverage.report]
|
||||
show_missing = true
|
||||
# Quality floor for `pytest --cov` / `coverage report` (not forced on a bare `pytest` run).
|
||||
fail_under = 90
|
||||
|
||||
[tool.ruff]
|
||||
src = ["src", "tests"]
|
||||
target-version = "py311"
|
||||
|
||||
12
registry/README.md
Normal file
12
registry/README.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Capability Registry
|
||||
|
||||
Markdown-first capability index for federation and reuse planning.
|
||||
|
||||
## Authoring
|
||||
|
||||
1. Copy a capability entry template (see reuse-surface `templates/capability-entry.template.md`).
|
||||
2. Add the row to `indexes/capabilities.yaml`.
|
||||
3. Run `reuse-surface validate` from a checkout with the CLI installed.
|
||||
4. Merge to `main` and verify publish with `reuse-surface establish --publish-check`.
|
||||
|
||||
Federation contract: reuse-surface `docs/RegistryFederation.md`.
|
||||
103
registry/capabilities/capability.wiki.adapter-contract.md
Normal file
103
registry/capabilities/capability.wiki.adapter-contract.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
id: capability.wiki.adapter-contract
|
||||
name: Capability-Aware Shard Adapter Contract
|
||||
summary: A versioned backend interface where each binding declares a verified capability profile (positions on capability spectra), so federation ops degrade by capability.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, adapter, capability, contract, conformance, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D5
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
Fifteen capability spectra with an orthogonal core + implication rules, plus
|
||||
a normative contract spec (TSD Section A); derived from a ~23-system synthesis.
|
||||
availability:
|
||||
current: A2
|
||||
target: A5
|
||||
confidence: medium
|
||||
rationale: >
|
||||
AdapterContract + a read/write FolderAdapter + a conformance suite that
|
||||
verifies declared profile == observed behaviour exist as a source module.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: medium
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- versioned interface with declared, conformance-verified capability profiles
|
||||
- one concrete adapter (file-store) passes the conformance suite
|
||||
broken_expectations:
|
||||
- only one substrate implemented (git-IS-store, REST, CRDT adapters planned)
|
||||
out_of_scope_expectations:
|
||||
- hosting backends
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- single adapter implemented so far
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Mediate heterogeneity at one narrow waist: a backend participates by implementing a
|
||||
versioned interface and declaring a verified position on each capability spectrum.
|
||||
includes:
|
||||
- capability profile as data (orthogonal-core spectra + implied positions)
|
||||
- operation verbs (read/write/diff/merge/notify/.../derive-projection/execute)
|
||||
- a conformance suite (profiles verified, not self-asserted)
|
||||
excludes:
|
||||
- assuming uniform backend capabilities
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-34..UC-43, UC-50, UC-57, UC-60..UC-69 (shard attachment & adapter binding)"
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
target_level: A5
|
||||
current_artifacts:
|
||||
- "shard-wiki/src/shard_wiki/adapters/"
|
||||
consumption_modes:
|
||||
- source module
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.page-model
|
||||
supports:
|
||||
- capability.wiki.shard-orchestration
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/TechnicalSpecificationDocument.md (Section A)"
|
||||
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 6)"
|
||||
tests:
|
||||
- "shard-wiki/tests/test_folder_adapter.py"
|
||||
- "shard-wiki/tests/test_conformance.py"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- exposing any page store as a capability-described, conformance-checked shard
|
||||
not_recommended_for:
|
||||
- backends that cannot honestly describe their capabilities
|
||||
known_limitations:
|
||||
- reference implementation covers the file-store substrate only so far
|
||||
---
|
||||
|
||||
# Capability-Aware Shard Adapter Contract
|
||||
|
||||
The bottom narrow waist of shard-wiki: a versioned interface plus a **verified** capability
|
||||
profile per binding. Core logic is written once against capabilities (not per-backend), and
|
||||
the conformance suite rejects profiles whose declared abilities don't match observed behaviour.
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Fifteen spectra reduced to an orthogonal core with implication rules (CoreArchitectureBlueprint
|
||||
Section 6.5); normative in TSD Section A.
|
||||
|
||||
### Availability
|
||||
`adapters/` ships the contract, a folder adapter, and `assert_conformant`.
|
||||
103
registry/capabilities/capability.wiki.coordination-journal.md
Normal file
103
registry/capabilities/capability.wiki.coordination-journal.md
Normal file
@@ -0,0 +1,103 @@
|
||||
---
|
||||
id: capability.wiki.coordination-journal
|
||||
name: Event-Sourced Coordination Journal
|
||||
summary: An append-only, totally-ordered-per-space decision log (overlays, bindings, aliases, merges, forks) whose current state is a derived fold; git-addressable history.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, event-sourcing, coordination, git, journal, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D5
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
Keystone resolved across two architecture reviews: coordination-canonical state
|
||||
as an append-only decision log with a per-space append authority; current state
|
||||
is a derived fold (derived = f(log)).
|
||||
availability:
|
||||
current: A2
|
||||
target: A4
|
||||
confidence: medium
|
||||
rationale: >
|
||||
In-memory DecisionLog + fold work as a source module; the git-backed store with a
|
||||
per-space lease (the production backing) is planned.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: medium
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- append-only, totally-ordered-per-space log with read-your-writes
|
||||
- derived fold to aliases + transitively-merged equivalence groups
|
||||
broken_expectations:
|
||||
- git-backed storage and per-space lease/append-authority not yet implemented
|
||||
out_of_scope_expectations:
|
||||
- general-purpose event bus
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- in-memory backing only; cross-process durability pending
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Make coordination-canonical decisions durable and git-addressable as events, with the
|
||||
queryable current state always recomputable by replay.
|
||||
includes:
|
||||
- append-only decision log, totally ordered per information space
|
||||
- derived fold to current coordination state (aliases, equivalence groups, overlays)
|
||||
- per-space append authority (concurrency model)
|
||||
excludes:
|
||||
- storing derived/disposable union state
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-29, UC-33 (history, attribution, coordination journal)"
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
target_level: A4
|
||||
current_artifacts:
|
||||
- "shard-wiki/src/shard_wiki/coordination/decision_log.py"
|
||||
target_artifacts:
|
||||
- git-backed log store with per-space lease
|
||||
consumption_modes:
|
||||
- source module
|
||||
|
||||
relations:
|
||||
supports:
|
||||
- capability.wiki.shard-orchestration
|
||||
- capability.wiki.overlay
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.1)"
|
||||
tests:
|
||||
- "shard-wiki/tests/test_decision_log.py"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- durable, replayable, git-addressable coordination state for a federated space
|
||||
not_recommended_for:
|
||||
- high-frequency general event streaming
|
||||
known_limitations:
|
||||
- production git backing + lease are still on the roadmap (SHARD-WP-0009)
|
||||
---
|
||||
|
||||
# Event-Sourced Coordination Journal
|
||||
|
||||
The keystone: coordination-canonical state (overlays, equivalence bindings, aliases, merges,
|
||||
forks) is an append-only **decision log**, totally ordered per information space; the queryable
|
||||
current state is a derived **fold** of the log (`derived = f(log)`). The log is git-addressable,
|
||||
giving history/patch/review/backup for coordination decisions for free.
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Resolved across the round-1/round-2 architecture reviews (CoreArchitectureBlueprint Section 8.1).
|
||||
|
||||
### Availability
|
||||
`decision_log.py` ships an in-memory, totally-ordered log + fold; git+lease backing is planned.
|
||||
87
registry/capabilities/capability.wiki.derived-views.md
Normal file
87
registry/capabilities/capability.wiki.derived-views.md
Normal file
@@ -0,0 +1,87 @@
|
||||
---
|
||||
id: capability.wiki.derived-views
|
||||
name: Wiki Derived Views
|
||||
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages, SiteMap, and (delegate-or-derive) Search — carrying provenance.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, derived-views, backlinks, recentchanges, search, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D3
|
||||
target: D5
|
||||
confidence: medium
|
||||
rationale: >
|
||||
Core-vs-adapter classification and behaviours are decided (FederationRequirements ADR-03);
|
||||
implementation is planned (SHARD-WP-0010), not built.
|
||||
availability:
|
||||
current: A0
|
||||
target: A4
|
||||
confidence: low
|
||||
rationale: >
|
||||
Designed; no implementation yet. Informational/planning reuse only today.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C0
|
||||
name: Absent
|
||||
confidence: low
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations: []
|
||||
broken_expectations:
|
||||
- no derived view is implemented yet
|
||||
out_of_scope_expectations:
|
||||
- presentation / rendering of views
|
||||
reliability:
|
||||
level: R0
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- planning-stage
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Provide recomputable, provenance-carrying views over the union (link graph, change feed,
|
||||
enumeration, search) without introducing canonical state.
|
||||
includes:
|
||||
- BackLinks (link graph), RecentChanges (journal + shard signals), AllPages, SiteMap
|
||||
- Search as delegate-to-native-or-derive-index
|
||||
excludes:
|
||||
- view presentation / UI
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-17..UC-21, UC-63"
|
||||
|
||||
availability:
|
||||
current_level: A0
|
||||
target_level: A4
|
||||
current_artifacts:
|
||||
- "shard-wiki/workplans/SHARD-WP-0010-derived-views.md"
|
||||
consumption_modes:
|
||||
- informational
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.shard-orchestration
|
||||
- capability.wiki.page-model
|
||||
related_to:
|
||||
- capability.wiki.engine-typed-extensions
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/FederationRequirements.md (ADR-03)"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- planning derived navigation/discovery over a federated wiki union
|
||||
not_recommended_for:
|
||||
- implementation reuse today (planning-stage)
|
||||
known_limitations:
|
||||
- not implemented; Search ranking policy undecided
|
||||
---
|
||||
|
||||
# Wiki Derived Views
|
||||
|
||||
Recomputable views over the union (BackLinks, RecentChanges, AllPages, SiteMap, Search). All
|
||||
are derived/disposable (no canonical state) and carry provenance; Search is delegate-to-native
|
||||
where a shard's query capability allows, else a derived index. Planned in SHARD-WP-0010.
|
||||
115
registry/capabilities/capability.wiki.engine-typed-extensions.md
Normal file
115
registry/capabilities/capability.wiki.engine-typed-extensions.md
Normal file
@@ -0,0 +1,115 @@
|
||||
---
|
||||
id: capability.wiki.engine-typed-extensions
|
||||
name: Wiki Engine with Typed Extensions
|
||||
summary: A small-core wiki engine realizing a stringent typed-extension framework that addresses all wiki use cases and lets each shard activate only the features it needs.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, engine, typed-extensions, feature-activation, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D3
|
||||
target: D5
|
||||
confidence: medium
|
||||
rationale: >
|
||||
Architecture authored (shard-wiki/spec/WikiEngineCoreArchitecture.md): small page-store
|
||||
kernel + typed-extension framework, per-shard activation, engine-as-canonical-mode-shard,
|
||||
and a conflict-mediation realization are explored. Detailed extension SDK/ABI and the API
|
||||
protocol remain (so D3 Explored, not yet D4/D5).
|
||||
availability:
|
||||
current: A0
|
||||
target: A4
|
||||
confidence: low
|
||||
rationale: >
|
||||
Planned. No engine kernel or extensions exist yet; informational/planning reuse only.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C0
|
||||
name: Absent
|
||||
confidence: low
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations: []
|
||||
broken_expectations:
|
||||
- engine core and typed-extension mechanism not yet designed in detail
|
||||
out_of_scope_expectations:
|
||||
- replacing other wiki engines or mandating one implementation
|
||||
reliability:
|
||||
level: R0
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- planning-stage capability
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Provide shard-wiki's reference first-party shard backend: a small core + a stringent
|
||||
typed-extension framework covering all collected use cases, mediating conflicting
|
||||
requirements into an integrated whole, with per-shard activation (only what you need).
|
||||
includes:
|
||||
- a minimal engine kernel (page lifecycle, storage via the adapter contract, the typing mechanism)
|
||||
- typed extensions that declare contracts and compose
|
||||
- per-shard feature activation
|
||||
excludes:
|
||||
- replacing or mandating other wiki engines (it is one shard type among many)
|
||||
- a single canonical implementation for all wikis
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-08..UC-25 and the full catalog (the engine must cover all)"
|
||||
|
||||
availability:
|
||||
current_level: A0
|
||||
target_level: A4
|
||||
current_artifacts:
|
||||
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
|
||||
- "shard-wiki/spec/WikiEngineCoreArchitecture.md"
|
||||
consumption_modes:
|
||||
- informational
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.adapter-contract
|
||||
- capability.wiki.page-model
|
||||
related_to:
|
||||
- capability.feature-control.evaluate
|
||||
- capability.authorization.policy-evaluate
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/workplans/SHARD-WP-0013-wiki-engine-prep.md"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- planning a composable, feature-activatable native wiki engine
|
||||
not_recommended_for:
|
||||
- implementation reuse today (planning-stage)
|
||||
known_limitations:
|
||||
- architecture authored; extension SDK/ABI + API protocol still to design; not yet built
|
||||
|
||||
promotion_history:
|
||||
- date: "2026-06-15"
|
||||
dimension: discovery
|
||||
from: D2
|
||||
to: D3
|
||||
rationale: WikiEngineCoreArchitecture.md authored (kernel + typed-extension framework explored); INTENT amendment ratified.
|
||||
author: shard-wiki
|
||||
---
|
||||
|
||||
# Wiki Engine with Typed Extensions
|
||||
|
||||
shard-wiki's planned reference first-party shard backend — a *canonical-mode shard* it
|
||||
implements natively: a small core plus a stringent typed-extension framework addressing all
|
||||
collected use cases, mediating conflicting requirements into a consistent whole, with per-shard
|
||||
activation (activate only what you need). It is one shard type among many — not a replacement
|
||||
for other engines. Per-shard activation is a candidate consumer of
|
||||
`capability.feature-control.evaluate`.
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Architecture authored: `shard-wiki/spec/WikiEngineCoreArchitecture.md` (small kernel +
|
||||
typed-extension framework; engine = canonical-mode shard). INTENT amendment ratified
|
||||
(2026-06-15, decision 84ffdb48). Extension SDK/ABI + API protocol are the next deliverables.
|
||||
|
||||
### Availability
|
||||
Planning-stage; informational reuse only.
|
||||
97
registry/capabilities/capability.wiki.federation-models.md
Normal file
97
registry/capabilities/capability.wiki.federation-models.md
Normal file
@@ -0,0 +1,97 @@
|
||||
---
|
||||
id: capability.wiki.federation-models
|
||||
name: Selectable Federation-Model Taxonomy
|
||||
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication+ping, query-time graph-join, feed, activity-streams, engine-mirror) selected per space.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, federation, taxonomy, composable, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D4
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
A six-model taxonomy distilled from a ~23-system synthesis, each model anchored in a
|
||||
real system, with capability prerequisites and per-space/per-shard composition rules.
|
||||
availability:
|
||||
current: A0
|
||||
target: A4
|
||||
confidence: low
|
||||
rationale: >
|
||||
Designed and specified (FederationArchitecture T17) but not implemented; informational
|
||||
reuse only today.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C1
|
||||
name: Sparse
|
||||
confidence: low
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- the model taxonomy and selection/composition rules are documented
|
||||
broken_expectations:
|
||||
- no federation transport is implemented yet
|
||||
out_of_scope_expectations:
|
||||
- mandating a single federation mechanism
|
||||
reliability:
|
||||
level: R0
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- design-stage; no runtime evidence
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Treat federation as selectable and composable rather than one mechanism, so each space
|
||||
picks fork+journal, VCS-replication, query-join, feed, activity-streams, or engine-mirror.
|
||||
includes:
|
||||
- the six federation models + their capability floors
|
||||
- per-space selection and per-shard composition
|
||||
excludes:
|
||||
- imposing one homogeneous federation network
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-26, UC-31, UC-33, UC-71, UC-72, UC-74, UC-79"
|
||||
|
||||
availability:
|
||||
current_level: A0
|
||||
target_level: A4
|
||||
current_artifacts:
|
||||
- "shard-wiki/spec/FederationArchitecture.md (T17)"
|
||||
consumption_modes:
|
||||
- informational
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.shard-orchestration
|
||||
- capability.wiki.coordination-journal
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/FederationArchitecture.md"
|
||||
- "shard-wiki/research/260614-shard-spectrum-synthesis/findings.md"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- planning a federation strategy that mixes models per source
|
||||
not_recommended_for:
|
||||
- implementation reuse today (design-stage)
|
||||
known_limitations:
|
||||
- no transport implemented; informational planning reuse only
|
||||
---
|
||||
|
||||
# Selectable Federation-Model Taxonomy
|
||||
|
||||
Federation is plural and composable: fork+journal (Federated Wiki), VCS-replication+ping
|
||||
(ikiwiki), query-time graph-join (Wikibase SERVICE), feed aggregation, activity streams
|
||||
(ActivityPub), and engine-mirror (Wiki.js). A space selects a model and composes per shard;
|
||||
the default is fork+journal over git. Design-stage capability — strong for planning reuse.
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
FederationArchitecture T17, distilled from the shard-spectrum synthesis (v3).
|
||||
|
||||
### Availability
|
||||
Specified, not implemented — informational reuse only.
|
||||
102
registry/capabilities/capability.wiki.overlay.md
Normal file
102
registry/capabilities/capability.wiki.overlay.md
Normal file
@@ -0,0 +1,102 @@
|
||||
---
|
||||
id: capability.wiki.overlay
|
||||
name: Overlay-Before-Mutation Write Path
|
||||
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only, rate-limited, or lossy backends be edited safely without silent remote mutation.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, overlay, patch, write-path, conflict, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D5
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
Overlay lifecycle and apply-under-drift semantics are specified (ADR-05, blueprint
|
||||
Section 8.6) and implemented as a single principled write path.
|
||||
availability:
|
||||
current: A2
|
||||
target: A4
|
||||
confidence: medium
|
||||
rationale: >
|
||||
OverlayEngine (draft/patch/apply), writable adapter, and InformationSpace.edit
|
||||
exist as a source module; three-way merge is not (refuse-on-drift only).
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: medium
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- draft -> patch -> apply with fast-forward / refuse-on-drift / keep-draft outcomes
|
||||
- no silent remote mutation; overlay_state surfaced in provenance
|
||||
broken_expectations:
|
||||
- three-way / auto merge not implemented (refuse-on-conflict only)
|
||||
out_of_scope_expectations:
|
||||
- federation propagation of applied overlays
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- early implementation; conflict handling is detect-and-refuse only
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Make any sub-write-through backend editable safely: an edit is an overlay first, applied
|
||||
only on explicit intent and only when the source has not drifted.
|
||||
includes:
|
||||
- overlay drafts recorded as coordination-canonical events
|
||||
- patch rendering (unified diff)
|
||||
- apply-under-drift (fast-forward / refuse / keep-draft)
|
||||
excludes:
|
||||
- destructive write without drift check
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-04, UC-26, UC-29 (remix primitives, overlay)"
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
target_level: A4
|
||||
current_artifacts:
|
||||
- "shard-wiki/src/shard_wiki/coordination/overlay.py"
|
||||
- "shard-wiki/src/shard_wiki/coordination/patch.py"
|
||||
consumption_modes:
|
||||
- source module
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.coordination-journal
|
||||
- capability.wiki.adapter-contract
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/FederationRequirements.md (ADR-05)"
|
||||
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 8.2, 8.6)"
|
||||
tests:
|
||||
- "shard-wiki/tests/test_apply.py"
|
||||
- "shard-wiki/tests/test_write_path_integration.py"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- safe editing over read-only / rate-limited / lossy backends
|
||||
not_recommended_for:
|
||||
- workflows needing automatic conflict resolution today
|
||||
known_limitations:
|
||||
- merge is detect-and-refuse; three-way merge is future work
|
||||
---
|
||||
|
||||
# Overlay-Before-Mutation Write Path
|
||||
|
||||
One principled write path: every edit drafts an overlay (a coordination-canonical event),
|
||||
renders as a patch, and applies under drift checks — fast-forwarding a writable target,
|
||||
keeping a local draft on a read-only target, and refusing (never clobbering) on external drift.
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Specified in FederationRequirements ADR-05 and CoreArchitectureBlueprint Section 8.2/8.6.
|
||||
|
||||
### Availability
|
||||
`overlay.py` + `patch.py` + `InformationSpace.edit` ship the path; built in SHARD-WP-0008.
|
||||
104
registry/capabilities/capability.wiki.page-model.md
Normal file
104
registry/capabilities/capability.wiki.page-model.md
Normal file
@@ -0,0 +1,104 @@
|
||||
---
|
||||
id: capability.wiki.page-model
|
||||
name: Backend-Neutral Wiki Page Model
|
||||
summary: A Markdown-first but stretchable page model with stable identity separate from placement and layered provenance, spanning prose to typed-graph and computational shapes.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, page-model, identity, provenance, markdown, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D5
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
Page shapes (prose, typed records, typed-graph, inline-embedded, non-Markdown,
|
||||
and four computational shapes) plus identity != placement and layered provenance
|
||||
are specified and grounded in the dive research.
|
||||
availability:
|
||||
current: A2
|
||||
target: A5
|
||||
confidence: medium
|
||||
rationale: >
|
||||
Identity/Placement/Span/Page and layered ProvenanceEnvelope exist as a source
|
||||
module; richer shapes (typed-graph, notebook) are modeled but not all built.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: medium
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- stable identity distinct from placement and from content fingerprint
|
||||
- layered (effective-vs-own) provenance with near-zero per-span cost
|
||||
broken_expectations:
|
||||
- non-prose shapes (typed-graph, notebook, inline-embedded) not fully realized
|
||||
out_of_scope_expectations:
|
||||
- rendering / presentation
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- prose shape is the only exercised path so far
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
One backend-neutral lingua franca every consumer sees; every shape reduces to
|
||||
(content|source, structure, provenance envelope, optional derivation rule).
|
||||
includes:
|
||||
- page identity (stable handle) vs placement (N paths/shards) vs equivalence (fingerprint)
|
||||
- layered provenance envelope (page + span deltas)
|
||||
- page-shape taxonomy incl. computational shapes
|
||||
excludes:
|
||||
- deriving identity from content (a fingerprint identifies a version, not a page)
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-34, UC-39, UC-44..UC-49, UC-55, UC-73, UC-83, UC-84"
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
target_level: A5
|
||||
current_artifacts:
|
||||
- "shard-wiki/src/shard_wiki/model/"
|
||||
- "shard-wiki/src/shard_wiki/provenance/"
|
||||
consumption_modes:
|
||||
- source module
|
||||
|
||||
relations:
|
||||
supports:
|
||||
- capability.wiki.adapter-contract
|
||||
- capability.wiki.shard-orchestration
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/CoreArchitectureBlueprint.md (Section 7)"
|
||||
- "shard-wiki/spec/FederationRequirements.md (ADR-02, ADR-04)"
|
||||
tests:
|
||||
- "shard-wiki/tests/test_model.py"
|
||||
- "shard-wiki/tests/test_provenance.py"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- a portable, provenance-carrying representation of wiki pages across backends
|
||||
not_recommended_for:
|
||||
- cases needing a single canonical path per page (use identity, not path)
|
||||
known_limitations:
|
||||
- non-prose shapes specified ahead of implementation
|
||||
---
|
||||
|
||||
# Backend-Neutral Wiki Page Model
|
||||
|
||||
The top narrow waist: a Markdown-first model that stretches to typed records, typed-graph
|
||||
statements, inline-embedded objects, non-Markdown assets, and computational shapes. Identity
|
||||
is a stable handle; placement and equivalence are separate mechanisms; provenance is layered
|
||||
(effective = page envelope + span delta).
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Specified in CoreArchitectureBlueprint Section 7 and FederationRequirements ADR-02/04.
|
||||
|
||||
### Availability
|
||||
`model/` + `provenance/` ship the prose path and the layered envelope today.
|
||||
114
registry/capabilities/capability.wiki.shard-orchestration.md
Normal file
114
registry/capabilities/capability.wiki.shard-orchestration.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
id: capability.wiki.shard-orchestration
|
||||
name: Wiki Shard Orchestration
|
||||
summary: Present a union of pages across heterogeneous wiki-shaped shards while preserving each shard's provenance, capabilities, and history.
|
||||
owner: shard-wiki
|
||||
status: draft
|
||||
domain: helix_forge
|
||||
tags: [wiki, federation, orchestration, union, shard-wiki]
|
||||
|
||||
maturity:
|
||||
discovery:
|
||||
current: D5
|
||||
target: D6
|
||||
confidence: high
|
||||
rationale: >
|
||||
Grounded in 84 documented use cases and a twice-reviewed whole-system
|
||||
architecture (CoreArchitectureBlueprint) derived from ~23 prior-art systems.
|
||||
availability:
|
||||
current: A2
|
||||
target: A5
|
||||
confidence: medium
|
||||
rationale: >
|
||||
InformationSpace orchestrator (attach -> resolve -> read, chorus on
|
||||
ambiguity) works as a Python source module; network API and incremental
|
||||
union are planned.
|
||||
|
||||
external_evidence:
|
||||
completeness:
|
||||
level: C2
|
||||
name: Partial
|
||||
confidence: medium
|
||||
basis: scope_vs_intent_and_consumer_expectations
|
||||
satisfied_expectations:
|
||||
- attach folder shards and read a union page with layered provenance
|
||||
- chorus presentation of equivalent-but-divergent pages (union without erasure)
|
||||
broken_expectations:
|
||||
- incremental union maintenance and equivalence index not yet built
|
||||
- write-through federation transports not yet built
|
||||
out_of_scope_expectations:
|
||||
- hosting or replacing the underlying wiki engines
|
||||
reliability:
|
||||
level: R1
|
||||
confidence: low
|
||||
basis: consumer_quality_signals
|
||||
known_reliability_risks:
|
||||
- early implementation; 64 tests but no production exposure
|
||||
|
||||
discovery:
|
||||
intent: >
|
||||
Let independently stored, differently implemented wikis behave as one
|
||||
coherent, versionable, inspectable information space without homogenizing them.
|
||||
includes:
|
||||
- union resolution across shards (identity-keyed)
|
||||
- chorus / designated-canonical presentation of equivalent pages
|
||||
- lazy replication projection of remote content with freshness
|
||||
excludes:
|
||||
- implementing a backend wiki engine (see capability.wiki.engine-typed-extensions)
|
||||
- silent remote mutation
|
||||
assumptions:
|
||||
- canonical truth lives in shards + a git coordination journal; the union is derived
|
||||
use_cases:
|
||||
- "shard-wiki UseCaseCatalog UC-01..UC-07, UC-26..UC-33 (information space, federation, coordination)"
|
||||
|
||||
availability:
|
||||
current_level: A2
|
||||
target_level: A5
|
||||
current_artifacts:
|
||||
- "shard-wiki/src/shard_wiki/union/"
|
||||
- "shard-wiki/src/shard_wiki/space.py"
|
||||
target_artifacts:
|
||||
- orchestrator network API
|
||||
consumption_modes:
|
||||
- source module
|
||||
|
||||
relations:
|
||||
depends_on:
|
||||
- capability.wiki.adapter-contract
|
||||
- capability.wiki.page-model
|
||||
- capability.wiki.coordination-journal
|
||||
supports:
|
||||
- capability.wiki.federation-models
|
||||
|
||||
evidence:
|
||||
documentation:
|
||||
- "shard-wiki/spec/CoreArchitectureBlueprint.md"
|
||||
- "shard-wiki/spec/FederationArchitecture.md"
|
||||
tests:
|
||||
- "shard-wiki/tests/test_union.py"
|
||||
- "shard-wiki/tests/test_integration.py"
|
||||
|
||||
consumer_guidance:
|
||||
recommended_for:
|
||||
- composing multiple Markdown/wiki stores into one provenance-preserving view
|
||||
not_recommended_for:
|
||||
- replacing a single wiki engine
|
||||
known_limitations:
|
||||
- resolution is recompute-on-read until the incremental tier lands
|
||||
---
|
||||
|
||||
# Wiki Shard Orchestration
|
||||
|
||||
shard-wiki's core capability: orchestrate wiki-shaped content across heterogeneous *shards*
|
||||
as a union of pages, preserving provenance, capabilities, and history per shard. Canonical
|
||||
truth stays at the edges (shards + the git coordination journal); the union is a derived,
|
||||
recomputable view (orchestrator, not engine).
|
||||
|
||||
## Assessment notes
|
||||
|
||||
### Discovery
|
||||
Grounded by `UseCaseCatalog.md` (84 UCs) and the hardened `CoreArchitectureBlueprint.md`.
|
||||
|
||||
### Availability
|
||||
`InformationSpace` provides attach/resolve/read today (source module); a network API is the
|
||||
target availability step.
|
||||
145
registry/indexes/capabilities.yaml
Normal file
145
registry/indexes/capabilities.yaml
Normal file
@@ -0,0 +1,145 @@
|
||||
version: 1
|
||||
updated: '2026-06-16'
|
||||
domain: helix_forge
|
||||
capabilities:
|
||||
- id: capability.wiki.shard-orchestration
|
||||
name: Wiki Shard Orchestration
|
||||
summary: Present a union of pages across heterogeneous wiki-shaped shards while
|
||||
preserving each shard's provenance, capabilities, and history.
|
||||
vector: D5 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.shard-orchestration.md
|
||||
tags:
|
||||
- wiki
|
||||
- federation
|
||||
- orchestration
|
||||
- union
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- source module
|
||||
- id: capability.wiki.adapter-contract
|
||||
name: Capability-Aware Shard Adapter Contract
|
||||
summary: A versioned backend interface where each binding declares a verified capability
|
||||
profile, so federation ops degrade by capability.
|
||||
vector: D5 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.adapter-contract.md
|
||||
tags:
|
||||
- wiki
|
||||
- adapter
|
||||
- capability
|
||||
- contract
|
||||
- conformance
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- source module
|
||||
- id: capability.wiki.page-model
|
||||
name: Backend-Neutral Wiki Page Model
|
||||
summary: A Markdown-first but stretchable page model with stable identity separate
|
||||
from placement and layered provenance.
|
||||
vector: D5 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.page-model.md
|
||||
tags:
|
||||
- wiki
|
||||
- page-model
|
||||
- identity
|
||||
- provenance
|
||||
- markdown
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- source module
|
||||
- id: capability.wiki.coordination-journal
|
||||
name: Event-Sourced Coordination Journal
|
||||
summary: An append-only, totally-ordered-per-space decision log whose current state
|
||||
is a derived fold; git-addressable history.
|
||||
vector: D5 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.coordination-journal.md
|
||||
tags:
|
||||
- wiki
|
||||
- event-sourcing
|
||||
- coordination
|
||||
- git
|
||||
- journal
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- source module
|
||||
- id: capability.wiki.overlay
|
||||
name: Overlay-Before-Mutation Write Path
|
||||
summary: Non-destructive edits (draft -> patch -> apply-under-drift) that let read-only
|
||||
or limited backends be edited safely without silent remote mutation.
|
||||
vector: D5 / A2 / C2 / R1
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.overlay.md
|
||||
tags:
|
||||
- wiki
|
||||
- overlay
|
||||
- patch
|
||||
- write-path
|
||||
- conflict
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- source module
|
||||
- id: capability.wiki.federation-models
|
||||
name: Selectable Federation-Model Taxonomy
|
||||
summary: Federation as a plural, composable coordination axis (fork+journal, VCS-replication,
|
||||
query-join, feed, activity-streams, engine-mirror) selected per space.
|
||||
vector: D4 / A0 / C1 / R0
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.federation-models.md
|
||||
tags:
|
||||
- wiki
|
||||
- federation
|
||||
- taxonomy
|
||||
- composable
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- informational
|
||||
- id: capability.wiki.engine-typed-extensions
|
||||
name: Wiki Engine with Typed Extensions
|
||||
summary: A small-core wiki engine realizing a typed-extension framework that addresses
|
||||
all wiki use cases and lets each shard activate only the features it needs.
|
||||
vector: D3 / A0 / C0 / R0
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.engine-typed-extensions.md
|
||||
tags:
|
||||
- wiki
|
||||
- engine
|
||||
- typed-extensions
|
||||
- feature-activation
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- informational
|
||||
- id: capability.wiki.derived-views
|
||||
name: Wiki Derived Views
|
||||
summary: Recomputable views over a wiki union — BackLinks, RecentChanges, AllPages,
|
||||
SiteMap, and (delegate-or-derive) Search — carrying provenance.
|
||||
vector: D3 / A0 / C0 / R0
|
||||
domain: helix_forge
|
||||
status: draft
|
||||
owner: shard-wiki
|
||||
path: registry/capabilities/capability.wiki.derived-views.md
|
||||
tags:
|
||||
- wiki
|
||||
- derived-views
|
||||
- backlinks
|
||||
- recentchanges
|
||||
- search
|
||||
- shard-wiki
|
||||
consumption_modes:
|
||||
- informational
|
||||
@@ -18,6 +18,10 @@ Scope relationship to the other specs:
|
||||
`spec/TechnicalSpecificationDocument.md`.
|
||||
- **`UseCaseCatalog.md`** is the demand this architecture must satisfy; UC references below
|
||||
are load tests, not decoration.
|
||||
- **`WikiEngineCoreArchitecture.md`** designs shard-wiki's native, headless, API-first wiki
|
||||
engine as a **canonical-mode shard backend** (one shard behind §6/§A — federation, union, and
|
||||
projection stay here in the orchestrator, not in the engine). Added per the 2026-06-15 INTENT
|
||||
amendment (decision `84ffdb48`, SHARD-WP-0013).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -8,6 +8,8 @@ Background on document types: InfoTechPrimers on coulomb.social.
|
||||
|------|--------|------|
|
||||
| `CoreArchitectureBlueprint.md` | draft for review | **Whole-system architecture** — layers, abstractions, load-bearing decisions (synthesised from all research) |
|
||||
| `FederationArchitecture.md` | draft for review | federation design — *what the union does*: T1–T10 decision records + the federation-model taxonomy (SHARD-WP-0002) |
|
||||
| `WikiEngineCoreArchitecture.md` | draft for review | the native **headless, API-first wiki engine** — small page-store kernel + typed-extension framework, as a canonical-mode shard backend (SHARD-WP-0013) |
|
||||
| `adr/` | living | Architecture Decision Records (ADR-0001: engine activation via feature-control) |
|
||||
| `FederationRequirements.md` | draft for review | yawex-derived union/federation design notes — resolution, namespace, derived views, provenance, overlay, links (ADR-01…06; SHARD-WP-0001) |
|
||||
| `ProductRequirementsDocument.md` | draft scaffold | What the product must deliver |
|
||||
| `TechnicalSpecificationDocument.md` | draft + §A | How the system is built; **§A = the normative shard adapter contract** (T11–T16, T18; SHARD-WP-0002) |
|
||||
|
||||
269
spec/WikiEngineCoreArchitecture.md
Normal file
269
spec/WikiEngineCoreArchitecture.md
Normal file
@@ -0,0 +1,269 @@
|
||||
# WikiEngineCoreArchitecture
|
||||
|
||||
Status: **draft for review** · Date: 2026-06-15 · Deliverable of **SHARD-WP-0013 T5**
|
||||
|
||||
The architecture of shard-wiki's **native reference wiki-engine**: a **headless, API-first**
|
||||
engine — a **small core** plus a **stringent typed-extension framework** — that addresses the
|
||||
whole use-case catalogue, mediates conflicting requirements into one integrated featureset, and
|
||||
lets each shard **activate only what it needs**. Authoritative as of the ratified INTENT
|
||||
amendment (2026-06-15, decision `84ffdb48`): the engine is **additive** and is shard-wiki's
|
||||
**reference first-party shard backend (a canonical-mode shard)** — not a replacement for other
|
||||
engines, not a UI.
|
||||
|
||||
Relation to other specs (referenced, not restated):
|
||||
- `CoreArchitectureBlueprint.md` — the orchestrator/whole-system architecture. **The engine is
|
||||
one shard behind §A; federation, union, projection, and cross-shard coordination are the
|
||||
orchestrator's job, not the engine's.** That is what keeps the engine small.
|
||||
- `TechnicalSpecificationDocument.md §A` — the shard adapter contract the engine implements.
|
||||
- `FederationRequirements.md` — page resolution, overlay, link semantics (ADRs the engine reuses).
|
||||
- `UseCaseCatalog.md` "Capability structure" layer (T2) — the core-vs-extension map + the
|
||||
conflict-mediation map this document realizes.
|
||||
- reuse surface (`capability.wiki.*`, plus consumed `feature-control` / `authorization`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Thesis: a small page-store kernel; everything else is a typed extension
|
||||
|
||||
> **The engine is a page-store kernel with a typed-extension runtime. Every capability beyond
|
||||
> the c2-minimum is a *typed extension* a shard activates only if it needs it — and a shard's
|
||||
> externally-visible capability profile is *computed from its active extension set*.**
|
||||
|
||||
That single chain — **configuration (which extensions) → capability (what the shard can do) →
|
||||
conformance (verified)** — is the whole design. It mirrors the orchestrator's discipline
|
||||
(`CoreArchitectureBlueprint` §6.5: capability-as-data, verified, no per-backend code) and turns
|
||||
"integrated whole, yet activate only what you need" from a slogan into a mechanism.
|
||||
|
||||
The engine stays small for a structural reason: it is **one shard**, not a federation layer.
|
||||
Union, projection, equivalence, cross-shard overlay-orchestration, and the federation models all
|
||||
live in shard-wiki's orchestrator (the blueprint). The engine implements `ShardAdapter` (§A) and
|
||||
nothing above it. So "wiki engine" here means *a really good single canonical shard with a
|
||||
typed-extension framework and a headless agent-first API* — not a re-implementation of shard-wiki.
|
||||
|
||||
---
|
||||
|
||||
## 2. Engine invariants
|
||||
|
||||
| # | Invariant | Why |
|
||||
|---|-----------|-----|
|
||||
| E-1 | **One shard, not a federation layer.** The engine implements `ShardAdapter` (§A); union/projection/federation are the orchestrator's. | Keeps the engine small; no duplication of the blueprint. |
|
||||
| E-2 | **Small kernel.** The kernel is only: page store + history, the page model (reused), the extension runtime, the API. | Common case (a plain wiki) is trivial. |
|
||||
| E-3 | **Everything else is a typed extension.** No feature beyond the c2-minimum is baked into the kernel. | Integrated-whole-yet-selective; testable boundary. |
|
||||
| E-4 | **Per-shard activation.** A shard runs an *activation profile* (a set of extensions + config); unused features cost nothing. | "Activate only what you need." |
|
||||
| E-5 | **Capability profile is derived from active extensions.** The §A profile the engine declares is computed from its activation profile, then conformance-verified. | One source of truth; honest, verified capabilities. |
|
||||
| E-6 | **Headless & API-first.** The API is the only interface; no bundled UI/rendering (consumer concern, L6). | INTENT amendment; clean orchestrator/consumer split. |
|
||||
| E-7 | **Agent-first ergonomics.** The API is typed, introspectable, batchable, low-round-trip. | INTENT: optimized for efficient agent/automation access. |
|
||||
| E-8 | **Reuse over reinvent.** Page model, history/journal, activation, and authz are *consumed* (existing capabilities), not rebuilt. | Smallness; reuse-surface alignment. |
|
||||
| E-9 | **Extensions are typed & verified.** An extension declares its types/hooks/deps; activation is rejected if types conflict or deps are unmet (impossible profiles forbidden). | Stringency; mirrors §6.5 + conformance. |
|
||||
|
||||
---
|
||||
|
||||
## 3. The kernel (four concepts)
|
||||
|
||||
The kernel is deliberately four things — nothing more is mandatory.
|
||||
|
||||
1. **Page** — the backend-neutral page model (`capability.wiki.page-model`, reused as-is):
|
||||
stable identity ≠ placement, layered provenance, page shapes. The kernel does **not** redefine
|
||||
it; extensions may *register additional shapes/types* (§4).
|
||||
2. **Store + history** — a git-backed page store (the engine is the *git-IS-store* case from the
|
||||
blueprint): a write is a commit; history is native and recoverable (E-3/I-10). Coordination
|
||||
decisions reuse the event-sourced journal (`capability.wiki.coordination-journal`).
|
||||
3. **Extension runtime** — the typed-extension registry, hook dispatcher, type checker, and
|
||||
activation engine (§4). *This is the core innovation; it is the only “framework” in the kernel.*
|
||||
4. **API** — the headless, typed, agent-first surface (§7). Kernel endpoints cover the c2-minimum
|
||||
(page CRUD-as-history, links, history); extensions extend the surface through typed routes.
|
||||
|
||||
The **c2-minimum** a kernel-only shard delivers (no extensions): write a page, link pages
|
||||
(`[[wikilink]]` + red-link), never lose an edit. That is a complete, useful headless wiki.
|
||||
|
||||
---
|
||||
|
||||
## 4. The typed-extension model (the framework)
|
||||
|
||||
An **Extension** is a typed unit declaring a contract the runtime enforces:
|
||||
|
||||
```
|
||||
Extension:
|
||||
id : reverse-domain id (e.g. ext.struct.typed-records)
|
||||
provides : capability ids it realizes (reuse-surface; e.g. capability.wiki.page-model[typed])
|
||||
types : page shapes / field schemas / content-types it introduces (typed, validated)
|
||||
hooks : kernel lifecycle bindings it implements (see below)
|
||||
api : typed routes it adds to the headless surface
|
||||
depends_on : other extensions / consumed capabilities required
|
||||
conflicts_with: extensions it cannot co-activate with
|
||||
config : declared, schema-checked activation parameters
|
||||
```
|
||||
|
||||
**Hooks (the kernel lifecycle the runtime dispatches):**
|
||||
`on_resolve` (name→page), `on_read`, `on_write` (validate/transform a draft), `on_link`
|
||||
(link/transclusion resolution), `on_history`, `on_query`, `on_render_request` (produce a derived
|
||||
representation for a consumer), `on_profile` (contribute capability-spectrum positions, E-5).
|
||||
Hooks are **typed** (typed inputs/outputs) and dispatched in a **declared, deterministic order**.
|
||||
|
||||
**Typing & composition (stringency):**
|
||||
- At activation, the runtime builds the **dependency closure**, checks **type consistency** (no
|
||||
two active extensions claim incompatible types for the same page shape/field; `conflicts_with`
|
||||
honoured), and rejects an **impossible profile** — exactly the §6.5 implication-rule discipline,
|
||||
applied to extensions. A rejected profile fails fast at boot, never silently.
|
||||
- Composition is **deterministic**: hook order is declared; conflicts are resolved by explicit
|
||||
precedence or rejection, never by accident.
|
||||
- Extensions ship a **conformance check** (mirrors §6.6): an activated extension is exercised
|
||||
against its declared types/hooks before the shard serves traffic — *typed contracts verified,
|
||||
not trusted*.
|
||||
|
||||
**Per-shard activation (reuse, not reinvent):**
|
||||
- A shard's **activation profile** = `{extension id → config}`. Activation/evaluation **reuses
|
||||
`capability.feature-control.evaluate`** (helix_forge/feature-control) — shard-wiki does not
|
||||
build a bespoke flagging system (T3 consumption).
|
||||
- **E-5 in action:** the engine's `on_profile` hooks fold the active extensions into the §A
|
||||
**capability profile** the shard advertises to the orchestrator (e.g. activate
|
||||
`ext.struct.typed-records` → the `structure` spectrum rises and `structured-payload` is
|
||||
declared). The profile is then conformance-verified (§A.2). *Configuration → capability →
|
||||
conformance is one chain.*
|
||||
|
||||
---
|
||||
|
||||
## 5. Featureset map: core vs extensions, and conflict mediation
|
||||
|
||||
The engine realizes the T2 "Capability structure" layer (`UseCaseCatalog.md`). Mapping (the
|
||||
*page/content-level* clusters; **X-FED and X-ATT are orchestrator concerns, not engine
|
||||
extensions** — E-1):
|
||||
|
||||
| Engine kernel (always on) | T2 | reuse-surface |
|
||||
|---------------------------|----|---------------|
|
||||
| Page lifecycle, identity/placement, history, links, store | EC-1…EC-5 | `capability.wiki.page-model`, `…coordination-journal`, `…adapter-contract` |
|
||||
|
||||
| Built-in typed extension | T2 cluster | provides / consumes | default |
|
||||
|--------------------------|-----------|---------------------|---------|
|
||||
| `ext.overlay` | X-OVERLAY | `capability.wiki.overlay` | on (no-op locally) |
|
||||
| `ext.authz` (L0→L4 tiers) | X-AUTHZ | consumes `capability.authorization.policy-evaluate` | L0 |
|
||||
| `ext.views` (BackLinks/RecentChanges/…) | X-VIEW | `capability.wiki.derived-views` | BackLinks/RecentChanges on |
|
||||
| `ext.struct` (typed/computed/graph) | X-STRUCT | `capability.wiki.page-model[typed]` | off |
|
||||
| `ext.addr` (span addr / transclusion / query) | X-ADDR | `capability.wiki.page-model`+query | transclusion on |
|
||||
| `ext.compute` (literate/notebook/program/live) | X-COMP | `capability.wiki.engine-typed-extensions` | off (gated, sandbox) |
|
||||
| `ext.prov` (rich provenance/metadata) | X-PROV | `capability.wiki.page-model[provenance]` | base on |
|
||||
| `ext.collab` (c2 social patterns) | X-COLLAB | (UI/convention; mostly consumer) | off |
|
||||
|
||||
**Conflict mediation (T2 map) realized by the framework** — every tension is a *mechanism*, not a
|
||||
baked-in choice, so one featureset serves all:
|
||||
|
||||
| Tension | Realized by |
|
||||
|---------|-------------|
|
||||
| open vs governed | `ext.authz` tiers (additive); kernel history is the floor at L0 |
|
||||
| lossless vs lossy | a `translate` hook + fidelity report (consumes the proposed `capability.content.translation-fidelity`, G2) |
|
||||
| live vs snapshot | `ext.compute`/`ext.addr` mark liveness; degrade to snapshot (never imply live) |
|
||||
| canonical vs chorus | detection in kernel; resolution is a policy preset (orchestrator) |
|
||||
| integrated-whole vs only-what-you-need | **the activation profile** (E-4) + typed composition (§4) — the headline mediation |
|
||||
| minimal vs feature-rich | small kernel (§3) + extensions; nothing beyond c2 is mandatory |
|
||||
|
||||
---
|
||||
|
||||
## 6. The engine as a canonical-mode shard
|
||||
|
||||
The engine exposes itself through an `EngineShardAdapter` implementing §A:
|
||||
|
||||
- **Substrate** git-IS-store; **history** git-native; **write** = commit; `current_rev` = sha
|
||||
(apply-under-drift works out of the box). It is the **most capable shard** shard-wiki can
|
||||
attach — it dogfoods the contract.
|
||||
- Its **capability profile is computed from active extensions** (E-5) and **conformance-verified**
|
||||
(§A.2) — so the orchestrator sees an honest profile, and federation ops degrade by the engine's
|
||||
*actually-activated* capabilities.
|
||||
- The orchestrator attaches it like any shard; **federation/union/projection are not in the
|
||||
engine** (E-1). A standalone deployment is "the engine as the sole canonical shard"; a
|
||||
federated deployment is "the engine as one shard among many." Same engine, no re-architecture.
|
||||
|
||||
This is the precise realization of the INTENT reconciliation: shard-wiki orchestrates; the engine
|
||||
is the first-party shard it can attach.
|
||||
|
||||
---
|
||||
|
||||
## 7. Headless API surface & agent ergonomics (E-6/E-7)
|
||||
|
||||
API-first means the typed API is the product; there is no UI. Agent-first means it is designed
|
||||
for cheap, deterministic machine consumption:
|
||||
|
||||
- **Typed resource API** over pages, links, history, spans — content-negotiated (raw Markdown,
|
||||
the structured page model, or an extension-rendered representation via `on_render_request`).
|
||||
- **Capability/extension introspection** — an endpoint returns the shard's **active extensions,
|
||||
their types, and the derived §A capability profile**, so an agent can discover *what this shard
|
||||
can do* before acting (no trial-and-error). This is the agent-facing twin of E-5.
|
||||
- **Batch & query** — multi-page reads, link-graph and RecentChanges queries (via `ext.views`),
|
||||
and `on_query` delegation — minimizing round-trips.
|
||||
- **Write via overlay** — edits go through the overlay path (FederationRequirements ADR-05), so
|
||||
agent writes are safe (draft → apply-under-drift) and attributable.
|
||||
- **Deterministic & provenance-carrying** — every response carries the provenance envelope;
|
||||
identical inputs yield identical outputs (no hidden state) — friendly to caching agents.
|
||||
|
||||
---
|
||||
|
||||
## 8. Implementation sketch (module layout)
|
||||
|
||||
The engine lives under the shard-wiki package as a backend (it sits at L0/L1 — a shard behind the
|
||||
adapter; nothing in the orchestrator depends *up* on it):
|
||||
|
||||
```
|
||||
src/shard_wiki/engine/
|
||||
kernel.py # page store + history (git-IS-store), lifecycle; reuses model/, provenance/, coordination/
|
||||
extension.py # Extension contract, registry, typed hook dispatcher, type checker
|
||||
activation.py # activation profile; reuses capability.feature-control.evaluate
|
||||
profile.py # derive the §A CapabilityProfile from active extensions (E-5) + conformance
|
||||
api.py # headless, typed, agent-first surface (+ extension introspection)
|
||||
adapter.py # EngineShardAdapter implements adapters/ ShardAdapter (canonical-mode shard)
|
||||
extensions/ # built-ins: overlay/ authz/ views/ struct/ addr/ compute/ prov/ collab/
|
||||
```
|
||||
|
||||
Dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`, `adapters/`
|
||||
(contract), `policy/`; it is consumed *only* via its `EngineShardAdapter` (the orchestrator
|
||||
attaches it as a shard). No orchestrator-tier (`union/`, `projection/`) import.
|
||||
|
||||
---
|
||||
|
||||
## 9. Reuse (what the engine consumes vs registers)
|
||||
|
||||
- **Consumes:** `capability.feature-control.evaluate` (activation), `capability.authorization.
|
||||
policy-evaluate` (`ext.authz`), the proposed `capability.content.translation-fidelity` (G2,
|
||||
lossy translation), and shard-wiki's own `capability.wiki.{page-model, coordination-journal,
|
||||
adapter-contract, overlay, derived-views}`.
|
||||
- **Registers / realizes:** `capability.wiki.engine-typed-extensions` (this document is its
|
||||
Discovery evidence — D2→D3 on ratification). The cross-cutting **typed-extension framework**
|
||||
pattern is proposed back to the reuse surface as **G1** (`capability.platform.typed-extension-
|
||||
framework`); this engine is its first instance.
|
||||
|
||||
---
|
||||
|
||||
## 10. Traceability
|
||||
|
||||
- **INTENT** — realizes the 2026-06-15 amendment (decision `84ffdb48`): headless, API-first,
|
||||
additive native engine = canonical-mode shard backend; honours all engine invariants and the
|
||||
orchestrator boundary (E-1).
|
||||
- **Use cases** — the kernel/extension split *is* the T2 "Capability structure" layer
|
||||
(`UseCaseCatalog.md`); every UC is either kernel (EC-1…EC-5) or a named extension; conflicts
|
||||
use the T2 mediation map (§5). The engine must ultimately cover UC-01–UC-84 (per-shard subsets).
|
||||
- **Architecture** — consistent with `CoreArchitectureBlueprint` (engine = canonical-mode shard,
|
||||
§6 contract, §7 page model, §8.1 journal) and `TechnicalSpecificationDocument §A` (the contract
|
||||
it implements). `FederationRequirements` ADR-05/06 supply overlay + link semantics.
|
||||
- **Reuse surface** — §9; G1/G2 proposals from SHARD-WP-0013 T3.
|
||||
|
||||
## 11. Decisions / deferred / open
|
||||
|
||||
**Decided:** small page-store kernel + typed-extension runtime (E-2/E-3); engine is one shard,
|
||||
not a federation layer (E-1); capability profile derived from active extensions (E-5); headless,
|
||||
API-first, agent-first (E-6/E-7); activation reuses `feature-control` (E-8); extensions are
|
||||
typed + conformance-verified (E-9).
|
||||
|
||||
**Deferred:** the concrete extension SDK/ABI and hook signatures; the API protocol (REST/GraphQL/
|
||||
MCP) — agent-first introspection is required, the wire format is an implementation spike; the
|
||||
built-in extensions' internal designs (each is a later workplan).
|
||||
|
||||
**Open (tracked):** does `ext.compute` ever execute in-process or strictly delegate/snapshot
|
||||
(ties blueprint §8.5 + trust/sandbox); is the typed-extension framework promoted to the
|
||||
reuse-surface platform capability (G1) and then *consumed* here rather than engine-owned;
|
||||
introspection granularity vs. leaking internal structure to agents.
|
||||
|
||||
## 12. Stability note
|
||||
|
||||
The **thesis (§1)** and **invariants (§2)** — especially *engine-is-one-shard* (E-1),
|
||||
*small-kernel/everything-else-typed-extension* (E-2/E-3), and *capability-profile-derived-from-
|
||||
extensions* (E-5) — are load-bearing. Changing them (e.g. moving federation into the engine, or
|
||||
baking a feature into the kernel) is an architectural change in the sense of INTENT's Stability
|
||||
Note and should be rare and deliberate. The headless/API-first posture is fixed by the ratified
|
||||
INTENT amendment.
|
||||
79
spec/adr/ADR-0001-engine-activation-via-feature-control.md
Normal file
79
spec/adr/ADR-0001-engine-activation-via-feature-control.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# ADR-0001 — Engine extension activation via feature-control (OpenFeature)
|
||||
|
||||
Status: **Accepted** · Date: 2026-06-15 · Deciders: tegwick · Source: SHARD-WP-0013 follow-up
|
||||
(feature-control assessment)
|
||||
|
||||
> First repo-level ADR. (Note: `FederationRequirements.md` contains document-internal
|
||||
> "ADR-01…06" design notes — those are scoped to that spec; this `spec/adr/` series is the
|
||||
> repository's standalone architecture decision log, starting here.)
|
||||
|
||||
## Context
|
||||
|
||||
`WikiEngineCoreArchitecture.md` (SHARD-WP-0013 T5) defines the engine as a small kernel plus a
|
||||
**typed-extension framework** where each shard **activates only the extensions it needs**
|
||||
(invariants E-4 activation, E-8 reuse-not-reinvent). It needs a mechanism to decide, per shard
|
||||
(and per tenant/context), which extensions/features are active — without baking a bespoke flag
|
||||
system into the engine, and without breaking the **standalone, zero-external-dependency** L0
|
||||
posture shard-wiki guarantees.
|
||||
|
||||
The helix_forge sibling **`feature-control`** (`capability.feature-control.evaluate`, registered
|
||||
at **D5 / A4 / C3 / R3**) provides exactly this: an **OpenFeature**-based feature-availability
|
||||
control plane with a working SDK (`feature_control_sdk`: `FeatureControlClient`, `Resolver`, a
|
||||
static `LocalProvider`), context-scoped evaluation (`tenant_id`/scope), explainable decisions,
|
||||
and graceful degradation when OpenFeature is absent. shard-wiki already proposed this as a T3
|
||||
*consumption* (reuse, don't rebuild).
|
||||
|
||||
## Decision
|
||||
|
||||
**Adopt `feature-control` (via the OpenFeature standard) as the engine's per-shard extension/
|
||||
feature activation mechanism** — *availability only* — with these constraints:
|
||||
|
||||
1. **OpenFeature-shaped, provider-pluggable.** The engine evaluates activation through an
|
||||
OpenFeature-style client. A static **`LocalProvider`** is the **standalone/L0 default**
|
||||
(zero external dependency); a `feature-control`/remote provider is plugged in for governed
|
||||
deployments. This mirrors shard-wiki's existing **identity-provider ladder** (null/local
|
||||
default → external when present).
|
||||
2. **Availability ≠ authorization.** feature-control decides *which extensions are active*,
|
||||
never *who may read/write*. Authorization stays in core (X-AUTHZ / `authorization.policy-
|
||||
evaluate`). The two are composed but never conflated. (feature-control's own INTENT requires
|
||||
this.)
|
||||
3. **Engine layer, not the orchestrator foundation.** Integration lives in
|
||||
`engine/activation.py`; the current `src/shard_wiki/` core stays dependency-free. OpenFeature/
|
||||
feature-control is an optional extra, kept out of the standalone path by the `LocalProvider`.
|
||||
4. **Thin slice only.** Consume `feature-control.evaluate` (mature, A4). Do **not** take a
|
||||
dependency on the heavier control-plane governance / `rollout` / `visibility` (A2) until a
|
||||
concrete need appears.
|
||||
|
||||
Activation keys = extension ids; evaluation context = `{tenant_id: root-entity, shard_id, …}`;
|
||||
the resulting active-extension set then **derives** the shard's §A capability profile (E-5).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
- No bespoke flag system; reuses a mature (D5/A4) capability — reuse-surface aligned.
|
||||
- Standalone stays zero-dep (LocalProvider); governed deployments get real runtime control,
|
||||
multi-tenant scoping, and **explainable** decisions that feed the engine's agent-introspection
|
||||
API (E-7: "why is extension X off for this shard?").
|
||||
- "Activate only what you need" + compute control become first-class and reversible at runtime.
|
||||
- Clean layering: availability (feature-control) vs authorization (core) vs identity (provider).
|
||||
|
||||
**Negative / risks (mitigated by the constraints)**
|
||||
- An optional OpenFeature dependency at the engine layer (mitigated: out of the standalone path).
|
||||
- Coupling to an external control plane in governed mode (mitigated: provider-pluggable, degrade
|
||||
to LocalProvider).
|
||||
- Temptation to route authz through it (mitigated: constraint 2, hard boundary).
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Bespoke per-shard flag/config in the engine** — rejected: reinvents feature-control, no
|
||||
standard, no multi-tenant/explainability, violates reuse-not-reinvent (E-8).
|
||||
- **No activation (all extensions always on)** — rejected: defeats "small core + activate only
|
||||
what you need" (E-2/E-4) and the compute-control goal.
|
||||
- **Build on the heavier feature-control control-plane now** — deferred: over-scoping a single
|
||||
engine's activation; revisit if rollout/governance needs emerge.
|
||||
|
||||
## Related
|
||||
|
||||
`WikiEngineCoreArchitecture.md` (E-4/E-8, §4 activation), `UseCaseCatalog.md` capability-structure
|
||||
layer (X-AUTHZ vs activation), `history/260615-reuse-surface-contributions.md` (T3 consumption),
|
||||
reuse-surface `capability.feature-control.evaluate`, INTENT amendment decision `84ffdb48`.
|
||||
11
spec/adr/README.md
Normal file
11
spec/adr/README.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# spec/adr/ — Architecture Decision Records
|
||||
|
||||
Repository-level ADRs: one decision per file, `ADR-NNNN-<slug>.md`, status
|
||||
**Proposed / Accepted / Superseded**. Each records Context · Decision · Consequences ·
|
||||
Alternatives. These are the standalone, numbered decision log; design-note "ADRs" embedded
|
||||
inside a spec (e.g. `FederationRequirements.md` ADR-01…06) are scoped to that document and are
|
||||
not part of this series.
|
||||
|
||||
| ADR | Status | Subject |
|
||||
|-----|--------|---------|
|
||||
| [ADR-0001](ADR-0001-engine-activation-via-feature-control.md) | Accepted | Engine extension activation via feature-control (OpenFeature), availability-only, LocalProvider standalone default |
|
||||
@@ -9,10 +9,13 @@ from shard_wiki.adapters.conformance import (
|
||||
)
|
||||
from shard_wiki.adapters.contract import CONTRACT_VERSION, ShardAdapter
|
||||
from shard_wiki.adapters.folder import FolderAdapter
|
||||
from shard_wiki.adapters.git import GitShardAdapter, PageRevision
|
||||
|
||||
__all__ = [
|
||||
"ShardAdapter",
|
||||
"FolderAdapter",
|
||||
"GitShardAdapter",
|
||||
"PageRevision",
|
||||
"CONTRACT_VERSION",
|
||||
"Check",
|
||||
"ConformanceReport",
|
||||
|
||||
180
src/shard_wiki/adapters/git.py
Normal file
180
src/shard_wiki/adapters/git.py
Normal file
@@ -0,0 +1,180 @@
|
||||
"""GitShardAdapter — a second substrate: git-as-store (SHARD-WP-0012; TSD §A.3 git-IS-store).
|
||||
|
||||
The home case where **git is the store *and* the journal**. Tracked ``*.md`` paths are the page
|
||||
keys; the working-tree file is the body; a page's ``source_rev`` is the **commit sha of the last
|
||||
commit touching its path** (per-path, so an edit to one page never drifts another). The declared
|
||||
profile is *git-IS-store ⟹ substrate=git ∧ history=git-native* — the implication rule the
|
||||
capability model enforces (§6.5), validated at registration like any other binding.
|
||||
|
||||
This adapter adds **no core changes**: it implements the same :class:`ShardAdapter` contract the
|
||||
folder adapter does, proving "write an adapter + declare a verified profile" is the whole cost of a
|
||||
new substrate (capability-as-data, I-3). Built on the ``git`` CLI via subprocess — zero new deps.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
from collections.abc import Iterable
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.adapters.contract import ShardAdapter
|
||||
from shard_wiki.model import (
|
||||
AccessGrant,
|
||||
Addressing,
|
||||
AttachmentMode,
|
||||
CapabilityProfile,
|
||||
ContentOpacity,
|
||||
History,
|
||||
Identity,
|
||||
MergeModel,
|
||||
NativeQuery,
|
||||
NotSupported,
|
||||
OperationalEnvelope,
|
||||
Page,
|
||||
Placement,
|
||||
Substrate,
|
||||
Translation,
|
||||
Verb,
|
||||
WriteGranularity,
|
||||
)
|
||||
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
|
||||
|
||||
__all__ = ["GitShardAdapter", "PageRevision"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PageRevision:
|
||||
"""One adopted git-native revision of a page: the commit sha and its subject line."""
|
||||
|
||||
sha: str
|
||||
message: str
|
||||
|
||||
_GIT_IDENTITY = {
|
||||
"GIT_AUTHOR_NAME": "shard-wiki",
|
||||
"GIT_AUTHOR_EMAIL": "shard@shard-wiki",
|
||||
"GIT_COMMITTER_NAME": "shard-wiki",
|
||||
"GIT_COMMITTER_EMAIL": "shard@shard-wiki",
|
||||
}
|
||||
|
||||
|
||||
class GitShardAdapter(ShardAdapter):
|
||||
"""A shard whose store is a git repo: keys are tracked ``*.md`` paths, revs are commit shas."""
|
||||
|
||||
def __init__(self, shard_id: str, repo_path: str | Path, writable: bool = False) -> None:
|
||||
self._shard_id = shard_id
|
||||
self._repo = Path(repo_path)
|
||||
self._writable = writable
|
||||
self._repo.mkdir(parents=True, exist_ok=True)
|
||||
if not (self._repo / ".git").exists():
|
||||
self._git("init", "--quiet")
|
||||
|
||||
@property
|
||||
def shard_id(self) -> str:
|
||||
return self._shard_id
|
||||
|
||||
def profile(self) -> CapabilityProfile:
|
||||
# VERSION is always available — a git-IS-store has git-native history to adopt (§A.5),
|
||||
# read-only or not. WRITE (= commit, PER_PAGE) is added only in writable mode.
|
||||
verbs = {Verb.READ, Verb.VERSION}
|
||||
granularity = WriteGranularity.NONE
|
||||
if self._writable:
|
||||
verbs |= {Verb.WRITE}
|
||||
granularity = WriteGranularity.PER_PAGE
|
||||
return CapabilityProfile(
|
||||
substrate=Substrate.GIT,
|
||||
attachment_mode=AttachmentMode.GIT_IS_STORE,
|
||||
write_granularity=granularity,
|
||||
content_opacity=ContentOpacity.TRANSPARENT,
|
||||
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||
access_grant=AccessGrant.OPEN,
|
||||
liveness=Liveness.STATIC,
|
||||
history=History.GIT_NATIVE, # git-is-store ⟹ git-native (§6.5)
|
||||
merge_model=MergeModel.GIT_TEXT,
|
||||
addressing=Addressing.PATH,
|
||||
native_query=NativeQuery.NONE,
|
||||
translation=Translation.NATIVE,
|
||||
supported_verbs=frozenset(verbs),
|
||||
).validate()
|
||||
|
||||
def write(self, key: str, body: str) -> Page:
|
||||
"""Write = **commit**: stage the file and commit it (skip a no-op so no empty commit),
|
||||
returning the page at the new sha. Drift detection rides on ``current_rev`` = that sha."""
|
||||
if not self._writable:
|
||||
raise NotSupported(f"{type(self).__name__} is read-only")
|
||||
rel = f"{key}.md"
|
||||
path = self._path_for(key)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(body, encoding="utf-8")
|
||||
self._git("add", "--", rel)
|
||||
if self._run("diff", "--cached", "--quiet").returncode != 0: # staged changes present
|
||||
self._git("commit", "-m", f"write {rel}", env=_GIT_IDENTITY)
|
||||
return self.read(key)
|
||||
|
||||
def keys(self) -> Iterable[str]:
|
||||
out = self._git("ls-files", "*.md").decode()
|
||||
for line in out.splitlines():
|
||||
yield line[: -len(".md")] if line.endswith(".md") else line
|
||||
|
||||
def read(self, key: str) -> Page:
|
||||
path = self._path_for(key)
|
||||
if not path.is_file():
|
||||
raise KeyError(key)
|
||||
rev = self.current_rev(key)
|
||||
return Page(
|
||||
identity=Identity(self._shard_id, key),
|
||||
body=path.read_text(encoding="utf-8"),
|
||||
envelope=ProvenanceEnvelope(
|
||||
source_shard=self._shard_id,
|
||||
liveness=Liveness.STATIC,
|
||||
staleness=Staleness.FRESH,
|
||||
source_rev=rev,
|
||||
lineage="git-native",
|
||||
),
|
||||
placements=(Placement(self._shard_id, f"{key}.md"),),
|
||||
)
|
||||
|
||||
def current_rev(self, key: str) -> str | None:
|
||||
"""The sha of the last commit touching ``key``'s path (per-path drift token), or None."""
|
||||
rel = f"{key}.md"
|
||||
if not self._path_for(key).is_file():
|
||||
return None
|
||||
sha = self._git("log", "-1", "--format=%H", "--", rel).decode().strip()
|
||||
return sha or None
|
||||
|
||||
def history(self, key: str) -> tuple[PageRevision, ...]:
|
||||
"""Adopt git-native history (§A.5): the commit list for ``key``'s path, newest-first.
|
||||
|
||||
VERSION-gated; raises ``KeyError`` for an unknown page. Each revision is a commit sha +
|
||||
subject — the native log surfaced through the contract, not re-implemented.
|
||||
"""
|
||||
if not self.profile().supports(Verb.VERSION):
|
||||
raise NotSupported(f"{type(self).__name__} does not support version")
|
||||
if not self._path_for(key).is_file():
|
||||
raise KeyError(key)
|
||||
out = self._git("log", "--format=%H%x00%s", "--", f"{key}.md").decode()
|
||||
revisions = []
|
||||
for line in out.splitlines():
|
||||
sha, _, message = line.partition("\x00")
|
||||
revisions.append(PageRevision(sha=sha, message=message))
|
||||
return tuple(revisions)
|
||||
|
||||
# -- git plumbing --------------------------------------------------------
|
||||
|
||||
def _path_for(self, key: str) -> Path:
|
||||
return self._repo / f"{key}.md"
|
||||
|
||||
def _git(self, *args: str, stdin: bytes | None = None, env: dict | None = None) -> bytes:
|
||||
return self._run(*args, stdin=stdin, env=env, check=True).stdout
|
||||
|
||||
def _run(
|
||||
self, *args: str, stdin: bytes | None = None, env: dict | None = None, check: bool = False
|
||||
) -> subprocess.CompletedProcess:
|
||||
return subprocess.run(
|
||||
["git", "-C", str(self._repo), *args],
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
env={**os.environ, **(env or {})},
|
||||
check=check,
|
||||
)
|
||||
@@ -4,7 +4,24 @@ from shard_wiki.coordination.decision_log import (
|
||||
CoordinationState,
|
||||
DecisionEvent,
|
||||
DecisionLog,
|
||||
EventStore,
|
||||
EventType,
|
||||
InMemoryEventStore,
|
||||
deserialize_event,
|
||||
serialize_event,
|
||||
)
|
||||
from shard_wiki.coordination.append_authority import (
|
||||
AppendAuthority,
|
||||
Lease,
|
||||
LeaseHeld,
|
||||
LeaseRegistry,
|
||||
)
|
||||
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||
from shard_wiki.coordination.migration import (
|
||||
export_jsonl,
|
||||
import_jsonl,
|
||||
import_log,
|
||||
migrate_space,
|
||||
)
|
||||
from shard_wiki.coordination.overlay import (
|
||||
ApplyResult,
|
||||
@@ -19,6 +36,19 @@ __all__ = [
|
||||
"DecisionEvent",
|
||||
"EventType",
|
||||
"CoordinationState",
|
||||
"EventStore",
|
||||
"InMemoryEventStore",
|
||||
"GitEventStore",
|
||||
"Lease",
|
||||
"LeaseHeld",
|
||||
"LeaseRegistry",
|
||||
"AppendAuthority",
|
||||
"import_log",
|
||||
"migrate_space",
|
||||
"export_jsonl",
|
||||
"import_jsonl",
|
||||
"serialize_event",
|
||||
"deserialize_event",
|
||||
"Overlay",
|
||||
"OverlayEngine",
|
||||
"ApplyStatus",
|
||||
|
||||
158
src/shard_wiki/coordination/append_authority.py
Normal file
158
src/shard_wiki/coordination/append_authority.py
Normal file
@@ -0,0 +1,158 @@
|
||||
"""Per-space append authority — the single-writer lease over the log (SHARD-WP-0009 T2).
|
||||
|
||||
The log is a *total order per space* (§8.6). :class:`~shard_wiki.coordination.git_event_store`
|
||||
makes a fork physically impossible via compare-and-swap; this layer adds the **policy** that gives
|
||||
the order a single designated writer: a **per-space lease**. At most one node holds a space's lease
|
||||
at a time; only the holder writes to the store. A non-holder does not write — it **forwards** its
|
||||
append intent to the current holder, so intents from anywhere still land in one serialized stream.
|
||||
|
||||
The lease is **time-bounded and re-grantable** (HA): if a holder dies, its lease expires and a new
|
||||
node may take it, resuming appends from the log head (``seq`` stays contiguous across the hand-off).
|
||||
A node holding a *stale* lease (already re-granted elsewhere) cannot write either — it discovers it
|
||||
is no longer the holder and forwards instead, so a partitioned ex-holder can never fork the log.
|
||||
|
||||
Mechanism over policy (CLAUDE.md): this provides the leasing *primitive*; who acquires when, and
|
||||
the TTL, are the caller's policy. Single-coordinator only — distributed multi-node leasing and log
|
||||
sharding are explicit non-goals of this workplan.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from shard_wiki.coordination.decision_log import DecisionEvent, EventStore, EventType
|
||||
|
||||
__all__ = ["Lease", "LeaseHeld", "LeaseRegistry", "AppendAuthority"]
|
||||
|
||||
|
||||
def _utcnow() -> datetime:
|
||||
return datetime.now(tz=timezone.utc)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Lease:
|
||||
"""A time-bounded grant of single-writer authority over one space."""
|
||||
|
||||
space: str
|
||||
holder: str
|
||||
token: str
|
||||
expires_at: datetime
|
||||
|
||||
def valid_at(self, now: datetime) -> bool:
|
||||
return now < self.expires_at
|
||||
|
||||
|
||||
class LeaseHeld(Exception):
|
||||
"""Raised when a space's lease is validly held by a different node."""
|
||||
|
||||
def __init__(self, lease: Lease) -> None:
|
||||
super().__init__(
|
||||
f"space {lease.space!r} leased to {lease.holder!r} until {lease.expires_at}"
|
||||
)
|
||||
self.lease = lease
|
||||
|
||||
|
||||
class LeaseRegistry:
|
||||
"""The single coordinator's grant table: at most one *valid* lease per space.
|
||||
|
||||
A lease that has expired is freely re-grantable to any node (the HA replacement path); a still
|
||||
valid lease is exclusive to its holder (renewable by that holder). The registry also routes
|
||||
forwarded append intents to the current holder node.
|
||||
"""
|
||||
|
||||
def __init__(self, clock: Callable[[], datetime] = _utcnow) -> None:
|
||||
self._clock = clock
|
||||
self._leases: dict[str, Lease] = {}
|
||||
self._nodes: dict[str, AppendAuthority] = {}
|
||||
|
||||
def register(self, node: AppendAuthority) -> None:
|
||||
self._nodes[node.node_id] = node
|
||||
|
||||
def grant(self, space: str, holder: str, ttl_seconds: float) -> Lease:
|
||||
"""Grant/renew the lease for ``space`` to ``holder``; raise :class:`LeaseHeld` if another
|
||||
node still holds it validly. An expired lease is re-grantable to anyone."""
|
||||
now = self._clock()
|
||||
current = self._leases.get(space)
|
||||
if current is not None and current.valid_at(now) and current.holder != holder:
|
||||
raise LeaseHeld(current)
|
||||
lease = Lease(
|
||||
space=space,
|
||||
holder=holder,
|
||||
token=uuid.uuid4().hex,
|
||||
expires_at=now + timedelta(seconds=ttl_seconds),
|
||||
)
|
||||
self._leases[space] = lease
|
||||
return lease
|
||||
|
||||
def current(self, space: str) -> Lease | None:
|
||||
"""The lease for ``space`` if one is currently valid, else None (expired/absent)."""
|
||||
lease = self._leases.get(space)
|
||||
return lease if lease is not None and lease.valid_at(self._clock()) else None
|
||||
|
||||
def holder_node(self, space: str) -> AppendAuthority | None:
|
||||
lease = self.current(space)
|
||||
return self._nodes.get(lease.holder) if lease is not None else None
|
||||
|
||||
|
||||
class AppendAuthority:
|
||||
"""A coordinator node that appends to the shared log only when it holds the space's lease.
|
||||
|
||||
Nodes share one :class:`EventStore` and one :class:`LeaseRegistry`. ``append`` routes itself:
|
||||
the holder writes; a non-holder forwards to whoever holds the lease (acquiring it first if the
|
||||
space is currently unleased). The append API mirrors :class:`EventStore` so the authority is a
|
||||
drop-in single-writer guard.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
node_id: str,
|
||||
store: EventStore,
|
||||
registry: LeaseRegistry,
|
||||
ttl_seconds: float = 30.0,
|
||||
) -> None:
|
||||
self.node_id = node_id
|
||||
self._store = store
|
||||
self._registry = registry
|
||||
self._ttl = ttl_seconds
|
||||
registry.register(self)
|
||||
|
||||
def acquire(self, space: str) -> Lease:
|
||||
"""Take (or renew) the lease for ``space``. Raises :class:`LeaseHeld` if another node holds
|
||||
it validly."""
|
||||
return self._registry.grant(space, self.node_id, self._ttl)
|
||||
|
||||
def holds(self, space: str) -> bool:
|
||||
lease = self._registry.current(space)
|
||||
return lease is not None and lease.holder == self.node_id
|
||||
|
||||
def append(
|
||||
self,
|
||||
space: str,
|
||||
type: EventType,
|
||||
payload: Mapping[str, Any],
|
||||
actor: str | None = None,
|
||||
) -> DecisionEvent:
|
||||
"""Append via the single authority. If we hold the lease, write; otherwise forward to the
|
||||
holder. If the space is unleased, acquire it first. A node with a *stale* lease forwards
|
||||
(it is not the current holder) rather than writing — so it cannot fork the log."""
|
||||
holder_node = self._registry.holder_node(space)
|
||||
if holder_node is None:
|
||||
self.acquire(space) # unleased: take authority, then write below
|
||||
holder_node = self
|
||||
if holder_node is self:
|
||||
return self._store.append(space, type, payload, actor=actor)
|
||||
return holder_node._write(space, type, payload, actor=actor)
|
||||
|
||||
def _write(
|
||||
self,
|
||||
space: str,
|
||||
type: EventType,
|
||||
payload: Mapping[str, Any],
|
||||
actor: str | None,
|
||||
) -> DecisionEvent:
|
||||
"""Apply a forwarded intent. Called only on the lease holder by a forwarding peer."""
|
||||
return self._store.append(space, type, payload, actor=actor)
|
||||
@@ -3,22 +3,36 @@
|
||||
Coordination-canonical state (overlays, equivalence bindings, aliases, merges, forks) is an
|
||||
**append-only decision log**, not a mutable file; the queryable *current* state is a **derived
|
||||
fold** of the log (tier-3 disposable). The log is **totally ordered per space** via a single
|
||||
**append authority** — here an in-process counter; a git-backed, lease-held authority is a later
|
||||
binding. That total order is what gives read-your-writes across readers (§8.6).
|
||||
**append authority**. That total order is what gives read-your-writes across readers (§8.6).
|
||||
|
||||
Storage lives behind :class:`EventStore`: :class:`InMemoryEventStore` is the default test double
|
||||
(an in-process counter); :class:`~shard_wiki.coordination.git_event_store.GitEventStore` is the
|
||||
git-addressable backend (SHARD-WP-0009). The :class:`DecisionLog` API and the :meth:`fold` are
|
||||
identical across backends — only storage + the concurrency model differ.
|
||||
|
||||
`derived = f(canonical)`: :class:`CoordinationState` is always reproducible by replaying the log.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from types import MappingProxyType
|
||||
from typing import Any
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
__all__ = ["EventType", "DecisionEvent", "CoordinationState", "DecisionLog"]
|
||||
__all__ = [
|
||||
"EventType",
|
||||
"DecisionEvent",
|
||||
"CoordinationState",
|
||||
"EventStore",
|
||||
"InMemoryEventStore",
|
||||
"DecisionLog",
|
||||
"serialize_event",
|
||||
"deserialize_event",
|
||||
]
|
||||
|
||||
|
||||
class EventType(Enum):
|
||||
@@ -63,10 +77,57 @@ class CoordinationState:
|
||||
return frozenset({identity})
|
||||
|
||||
|
||||
class DecisionLog:
|
||||
"""In-memory append-only log, totally ordered per space (the append authority for a process).
|
||||
def serialize_event(event: DecisionEvent) -> bytes:
|
||||
"""Deterministic, stable-JSON wire form of an event (same bytes for equal events, any process).
|
||||
|
||||
A later binding swaps the storage for git + a per-space lease without changing this API.
|
||||
Sorted keys + compact separators make the serialization canonical, so a git object hashed from
|
||||
it is reproducible — the basis for content-addressable, comparable logs across backends.
|
||||
"""
|
||||
obj = {
|
||||
"seq": event.seq,
|
||||
"space": event.space,
|
||||
"type": event.type.value,
|
||||
"payload": event.payload,
|
||||
"actor": event.actor,
|
||||
"timestamp": event.timestamp.isoformat(),
|
||||
}
|
||||
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()
|
||||
|
||||
|
||||
def deserialize_event(data: bytes | str) -> DecisionEvent:
|
||||
"""Inverse of :func:`serialize_event` — round-trips an event byte-for-byte by field."""
|
||||
obj = json.loads(data)
|
||||
return DecisionEvent(
|
||||
seq=obj["seq"],
|
||||
space=obj["space"],
|
||||
type=EventType(obj["type"]),
|
||||
payload=obj["payload"],
|
||||
actor=obj["actor"],
|
||||
timestamp=datetime.fromisoformat(obj["timestamp"]),
|
||||
)
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class EventStore(Protocol):
|
||||
"""Append-only, per-space ordered storage behind :class:`DecisionLog`.
|
||||
|
||||
Two bindings exist: :class:`InMemoryEventStore` (default/test double) and
|
||||
:class:`~shard_wiki.coordination.git_event_store.GitEventStore` (git-addressable). Both assign
|
||||
a per-space monotonic ``seq`` at the log head and guarantee read-your-writes for their reach
|
||||
(in-process for memory; cross-process for git).
|
||||
"""
|
||||
|
||||
def append(
|
||||
self, space: str, type: EventType, payload: Mapping[str, Any], actor: str | None = None
|
||||
) -> DecisionEvent: ...
|
||||
|
||||
def events(self, space: str) -> tuple[DecisionEvent, ...]: ...
|
||||
|
||||
|
||||
class InMemoryEventStore:
|
||||
"""In-process append-only store, totally ordered per space (the append authority for a process).
|
||||
|
||||
The default test double; the git backend preserves this exact contract on durable storage.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
@@ -84,10 +145,33 @@ class DecisionLog:
|
||||
self._events.setdefault(space, []).append(event)
|
||||
return event
|
||||
|
||||
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||
return tuple(self._events.get(space, ()))
|
||||
|
||||
|
||||
class DecisionLog:
|
||||
"""Append-only decision log, totally ordered per space, with a derived :meth:`fold`.
|
||||
|
||||
Storage is delegated to an :class:`EventStore` (default :class:`InMemoryEventStore`); swapping
|
||||
in the git backend changes only durability + the concurrency model, not this API or the fold.
|
||||
"""
|
||||
|
||||
def __init__(self, store: EventStore | None = None) -> None:
|
||||
self._store: EventStore = store if store is not None else InMemoryEventStore()
|
||||
|
||||
def append(
|
||||
self,
|
||||
space: str,
|
||||
type: EventType,
|
||||
payload: Mapping[str, Any],
|
||||
actor: str | None = None,
|
||||
) -> DecisionEvent:
|
||||
return self._store.append(space, type, payload, actor=actor)
|
||||
|
||||
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||
"""The space's events in append (total) order. Read-your-writes: a just-appended event
|
||||
is present immediately."""
|
||||
return tuple(self._events.get(space, ()))
|
||||
return self._store.events(space)
|
||||
|
||||
def fold(self, space: str) -> CoordinationState:
|
||||
"""Replay the log into current coordination state (derived = f(log))."""
|
||||
|
||||
172
src/shard_wiki/coordination/git_event_store.py
Normal file
172
src/shard_wiki/coordination/git_event_store.py
Normal file
@@ -0,0 +1,172 @@
|
||||
"""GitEventStore — a git-addressable binding of :class:`EventStore` (SHARD-WP-0009 T1).
|
||||
|
||||
Each space is a ref (``refs/spaces/<sha1(space)>``); each ``append`` writes the event as an
|
||||
immutable git object (a one-blob tree committed onto the ref) and advances the ref. The commit
|
||||
chain *is* the totally ordered log: ``seq`` is the depth, ``events`` walks first-parent from the
|
||||
head oldest→newest. Coordination-canonical state therefore inherits git's history / patch /
|
||||
review / backup affordances (I-6) and is read-your-writes correct across processes.
|
||||
|
||||
The total order is enforced at storage by a **compare-and-swap** ref update
|
||||
(``git update-ref <ref> <new> <old>``): two appenders racing off the same head — the loser's CAS
|
||||
fails and it retries off the new head, so a non-holder can never fork the log. The lease layer
|
||||
(T2) sits *above* this as the append-authority policy; CAS is the mechanism that makes it safe.
|
||||
|
||||
Implemented over the ``git`` CLI through :mod:`subprocess` — zero runtime dependencies.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import subprocess
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from shard_wiki.coordination.decision_log import (
|
||||
DecisionEvent,
|
||||
EventType,
|
||||
deserialize_event,
|
||||
serialize_event,
|
||||
)
|
||||
|
||||
__all__ = ["GitEventStore"]
|
||||
|
||||
# Fixed identity so commit objects are reproducible and never prompt for git config; the event's
|
||||
# own timestamp/actor carry the real provenance, the commit is just the ordered container.
|
||||
_GIT_IDENTITY = {
|
||||
"GIT_AUTHOR_NAME": "shard-wiki",
|
||||
"GIT_AUTHOR_EMAIL": "coordination@shard-wiki",
|
||||
"GIT_COMMITTER_NAME": "shard-wiki",
|
||||
"GIT_COMMITTER_EMAIL": "coordination@shard-wiki",
|
||||
}
|
||||
_EVENT_PATH = "event.json"
|
||||
_MAX_CAS_RETRIES = 50
|
||||
|
||||
|
||||
class GitEventStore:
|
||||
"""Git-backed, append-only, per-space ordered event store (an :class:`EventStore`)."""
|
||||
|
||||
def __init__(self, repo_path: str | Path) -> None:
|
||||
self.repo_path = Path(repo_path)
|
||||
self.repo_path.mkdir(parents=True, exist_ok=True)
|
||||
if not (self.repo_path / "HEAD").exists() and not (self.repo_path / ".git").exists():
|
||||
self._git("init", "--quiet", str(self.repo_path), at_cwd=True)
|
||||
|
||||
# -- EventStore contract -------------------------------------------------
|
||||
|
||||
def append(
|
||||
self,
|
||||
space: str,
|
||||
type: EventType,
|
||||
payload: Mapping[str, Any],
|
||||
actor: str | None = None,
|
||||
) -> DecisionEvent:
|
||||
"""Append one event, advancing the space ref under compare-and-swap (retry-on-race)."""
|
||||
ref = self._ref(space)
|
||||
for _ in range(_MAX_CAS_RETRIES):
|
||||
head = self._head(ref)
|
||||
seq = self._count(ref, head)
|
||||
event = DecisionEvent(
|
||||
seq=seq, space=space, type=type, payload=dict(payload), actor=actor
|
||||
)
|
||||
commit = self._commit_event(event, parent=head)
|
||||
if self._cas_update(ref, new=commit, old=head):
|
||||
return event
|
||||
raise RuntimeError(f"append contention on {space!r}: exhausted {_MAX_CAS_RETRIES} retries")
|
||||
|
||||
def import_event(self, event: DecisionEvent) -> None:
|
||||
"""Replay one pre-existing event *verbatim* (preserving seq / timestamp / actor) onto its
|
||||
space ref — the one-time migration path (SHARD-WP-0009 T4), not a live append.
|
||||
|
||||
Refuses out-of-order import so the imported chain stays a contiguous total order; preserving
|
||||
the original fields keeps provenance intact (union-without-erasure) rather than restamping.
|
||||
"""
|
||||
ref = self._ref(event.space)
|
||||
head = self._head(ref)
|
||||
expected = self._count(ref, head)
|
||||
if event.seq != expected:
|
||||
raise ValueError(
|
||||
f"out-of-order import on {event.space!r}: expected seq {expected}, got {event.seq}"
|
||||
)
|
||||
commit = self._commit_event(event, parent=head)
|
||||
if not self._cas_update(ref, new=commit, old=head):
|
||||
raise RuntimeError(f"import race on {ref}")
|
||||
|
||||
def events(self, space: str) -> tuple[DecisionEvent, ...]:
|
||||
"""The space's events oldest→newest (append/total order)."""
|
||||
ref = self._ref(space)
|
||||
head = self._head(ref)
|
||||
if head is None:
|
||||
return ()
|
||||
shas = self._git("rev-list", "--reverse", "--first-parent", ref).decode().split()
|
||||
return tuple(
|
||||
deserialize_event(self._git("cat-file", "blob", f"{sha}:{_EVENT_PATH}"))
|
||||
for sha in shas
|
||||
)
|
||||
|
||||
# -- git plumbing --------------------------------------------------------
|
||||
|
||||
def _commit_event(self, event: DecisionEvent, parent: str | None) -> str:
|
||||
blob = self._git(
|
||||
"hash-object", "-w", "--stdin", stdin=serialize_event(event)
|
||||
).decode().strip()
|
||||
tree = self._git(
|
||||
"mktree", stdin=f"100644 blob {blob}\t{_EVENT_PATH}\n".encode()
|
||||
).decode().strip()
|
||||
args = ["commit-tree", tree, "-m", f"event {event.seq} {event.type.value}"]
|
||||
if parent is not None:
|
||||
args += ["-p", parent]
|
||||
# Pin the commit date to the event's timestamp for reproducible objects.
|
||||
date = event.timestamp.isoformat()
|
||||
env = {**_GIT_IDENTITY, "GIT_AUTHOR_DATE": date, "GIT_COMMITTER_DATE": date}
|
||||
return self._git(*args, env=env).decode().strip()
|
||||
|
||||
def _cas_update(self, ref: str, new: str, old: str | None) -> bool:
|
||||
"""``git update-ref`` with the old value as a CAS guard (empty oldvalue == must-not-exist).
|
||||
|
||||
Returns False if the ref moved since we read ``old`` (lost the race) — the caller retries.
|
||||
"""
|
||||
result = self._run("update-ref", ref, new, old if old is not None else "")
|
||||
return result.returncode == 0
|
||||
|
||||
def _head(self, ref: str) -> str | None:
|
||||
result = self._run("rev-parse", "--verify", "--quiet", ref)
|
||||
out = result.stdout.decode().strip()
|
||||
return out or None
|
||||
|
||||
def _count(self, ref: str, head: str | None) -> int:
|
||||
if head is None:
|
||||
return 0
|
||||
return int(self._git("rev-list", "--count", "--first-parent", ref).decode().strip())
|
||||
|
||||
@staticmethod
|
||||
def _ref(space: str) -> str:
|
||||
return f"refs/spaces/{hashlib.sha1(space.encode()).hexdigest()}"
|
||||
|
||||
def _git(
|
||||
self,
|
||||
*args: str,
|
||||
stdin: bytes | None = None,
|
||||
env: dict | None = None,
|
||||
at_cwd: bool = False,
|
||||
) -> bytes:
|
||||
result = self._run(*args, stdin=stdin, env=env, at_cwd=at_cwd, check=True)
|
||||
return result.stdout
|
||||
|
||||
def _run(
|
||||
self,
|
||||
*args: str,
|
||||
stdin: bytes | None = None,
|
||||
env: dict | None = None,
|
||||
at_cwd: bool = False,
|
||||
check: bool = False,
|
||||
) -> subprocess.CompletedProcess:
|
||||
base = ["git"] if at_cwd else ["git", "-C", str(self.repo_path)]
|
||||
return subprocess.run(
|
||||
[*base, *args],
|
||||
input=stdin,
|
||||
capture_output=True,
|
||||
env={**os.environ, **(env or {})},
|
||||
check=check,
|
||||
)
|
||||
53
src/shard_wiki/coordination/migration.py
Normal file
53
src/shard_wiki/coordination/migration.py
Normal file
@@ -0,0 +1,53 @@
|
||||
"""One-time migration of a coordination log into git (SHARD-WP-0009 T4).
|
||||
|
||||
Replays an existing decision log — an in-memory store, or a JSON-lines export — into a
|
||||
:class:`GitEventStore`, preserving each event verbatim (seq / timestamp / actor) so provenance
|
||||
survives the move (union-without-erasure). After migration the same :meth:`DecisionLog.fold`
|
||||
reproduces identical coordination state; only durability changes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.coordination.decision_log import (
|
||||
DecisionEvent,
|
||||
EventStore,
|
||||
deserialize_event,
|
||||
serialize_event,
|
||||
)
|
||||
from shard_wiki.coordination.git_event_store import GitEventStore
|
||||
|
||||
__all__ = ["import_log", "migrate_space", "export_jsonl", "import_jsonl"]
|
||||
|
||||
|
||||
def import_log(events: Iterable[DecisionEvent], dest: GitEventStore) -> int:
|
||||
"""Replay ``events`` (in space/seq order) into ``dest``. Returns the count imported."""
|
||||
count = 0
|
||||
for event in events:
|
||||
dest.import_event(event)
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def migrate_space(source: EventStore, space: str, dest: GitEventStore) -> int:
|
||||
"""Migrate one space's log from any :class:`EventStore` into the git backend verbatim."""
|
||||
return import_log(source.events(space), dest)
|
||||
|
||||
|
||||
def export_jsonl(events: Iterable[DecisionEvent], path: str | Path) -> int:
|
||||
"""Write events as newline-delimited canonical JSON (a portable, diffable log export)."""
|
||||
count = 0
|
||||
with open(path, "wb") as handle:
|
||||
for event in events:
|
||||
handle.write(serialize_event(event) + b"\n")
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def import_jsonl(path: str | Path, dest: GitEventStore) -> int:
|
||||
"""Replay a JSON-lines export (see :func:`export_jsonl`) into the git backend."""
|
||||
with open(path, "rb") as handle:
|
||||
events = [deserialize_event(line) for line in handle if line.strip()]
|
||||
return import_log(events, dest)
|
||||
49
src/shard_wiki/engine/__init__.py
Normal file
49
src/shard_wiki/engine/__init__.py
Normal file
@@ -0,0 +1,49 @@
|
||||
"""engine/ — shard-wiki's native, headless wiki engine (a canonical-mode shard backend).
|
||||
|
||||
A small page-store kernel + a typed-extension runtime (WikiEngineCoreArchitecture). The engine
|
||||
is *one shard*: it is consumed by the orchestrator only via its `EngineShardAdapter`; it never
|
||||
imports the derived tier (`union`/`projection`).
|
||||
"""
|
||||
|
||||
from shard_wiki.engine.activation import (
|
||||
ActivationContext,
|
||||
ActivationProvider,
|
||||
ActivationResolver,
|
||||
StaticProvider,
|
||||
feature_control_provider,
|
||||
)
|
||||
from shard_wiki.engine.extension import (
|
||||
ActiveExtensions,
|
||||
Extension,
|
||||
ExtensionError,
|
||||
ExtensionRuntime,
|
||||
Hook,
|
||||
)
|
||||
from shard_wiki.engine.adapter import EngineShardAdapter, build_engine_shard
|
||||
from shard_wiki.engine.kernel import EngineKernel
|
||||
from shard_wiki.engine.links import extract_wikilinks
|
||||
from shard_wiki.engine.profile import (
|
||||
ProfileContribution,
|
||||
derive_profile,
|
||||
engine_base_profile,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"EngineKernel",
|
||||
"extract_wikilinks",
|
||||
"Hook",
|
||||
"Extension",
|
||||
"ExtensionError",
|
||||
"ExtensionRuntime",
|
||||
"ActiveExtensions",
|
||||
"ActivationContext",
|
||||
"ActivationProvider",
|
||||
"StaticProvider",
|
||||
"ActivationResolver",
|
||||
"feature_control_provider",
|
||||
"engine_base_profile",
|
||||
"ProfileContribution",
|
||||
"derive_profile",
|
||||
"EngineShardAdapter",
|
||||
"build_engine_shard",
|
||||
]
|
||||
129
src/shard_wiki/engine/activation.py
Normal file
129
src/shard_wiki/engine/activation.py
Normal file
@@ -0,0 +1,129 @@
|
||||
"""Per-shard extension activation (WikiEngineCoreArchitecture E-4/E-8, ADR-0001).
|
||||
|
||||
Decides *which registered extensions are active* for a given shard — **availability only, never
|
||||
authorization**. The mechanism is OpenFeature-shaped and **provider-pluggable**:
|
||||
|
||||
- :class:`StaticProvider` is the **standalone / L0 default** — zero external dependency, in-process
|
||||
flags with optional per-shard scoping. A kernel-only or offline shard uses this.
|
||||
- :func:`feature_control_provider` lazily wraps the helix_forge ``feature_control_sdk`` (over
|
||||
OpenFeature) when it is installed; absent, it returns ``None`` and the caller falls back to the
|
||||
static default. This mirrors shard-wiki's identity-provider ladder (local default → external
|
||||
when present), and keeps the engine core pure-stdlib.
|
||||
|
||||
An *activation profile* is ``{extension id → config}``; the active id set then feeds the
|
||||
extension runtime's `activate()` (T2) and the derived capability profile (T4, E-5).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
__all__ = [
|
||||
"ActivationContext",
|
||||
"ActivationProvider",
|
||||
"StaticProvider",
|
||||
"ActivationResolver",
|
||||
"feature_control_provider",
|
||||
]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ActivationContext:
|
||||
"""Scope for an activation decision. Carries no principal/authz — availability only."""
|
||||
|
||||
shard_id: str
|
||||
tenant_id: str | None = None
|
||||
extra: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
d: dict[str, Any] = {"shard_id": self.shard_id}
|
||||
if self.tenant_id is not None:
|
||||
d["tenant_id"] = self.tenant_id
|
||||
d.update(self.extra)
|
||||
return d
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class ActivationProvider(Protocol):
|
||||
"""Evaluates feature availability for an extension key in a context (OpenFeature-shaped)."""
|
||||
|
||||
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool: ...
|
||||
|
||||
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class StaticProvider:
|
||||
"""The standalone default: in-process flags, optionally overridden per shard. Zero deps.
|
||||
|
||||
``flags`` is the base availability map; ``per_shard[shard_id]`` overrides it for one shard;
|
||||
``configs[feature_key]`` supplies per-extension config. Unknown keys → ``default``.
|
||||
"""
|
||||
|
||||
flags: Mapping[str, bool] = field(default_factory=dict)
|
||||
per_shard: Mapping[str, Mapping[str, bool]] = field(default_factory=dict)
|
||||
configs: Mapping[str, Mapping[str, Any]] = field(default_factory=dict)
|
||||
default: bool = False
|
||||
|
||||
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
|
||||
shard = context.get("shard_id")
|
||||
scoped = self.per_shard.get(shard, {}) if shard is not None else {}
|
||||
if feature_key in scoped:
|
||||
return scoped[feature_key]
|
||||
return self.flags.get(feature_key, self.default)
|
||||
|
||||
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
|
||||
return self.configs.get(feature_key, {})
|
||||
|
||||
|
||||
class ActivationResolver:
|
||||
"""Maps candidate extension ids → the active set / activation profile for a context."""
|
||||
|
||||
def __init__(self, provider: ActivationProvider) -> None:
|
||||
self.provider = provider
|
||||
|
||||
def active_extensions(
|
||||
self, candidate_ids: Iterable[str], context: ActivationContext
|
||||
) -> set[str]:
|
||||
ctx = context.as_dict()
|
||||
return {eid for eid in candidate_ids if self.provider.is_active(eid, ctx)}
|
||||
|
||||
def activation_profile(
|
||||
self, candidate_ids: Iterable[str], context: ActivationContext
|
||||
) -> dict[str, Mapping[str, Any]]:
|
||||
"""``{extension id → config}`` for the active subset."""
|
||||
ctx = context.as_dict()
|
||||
return {
|
||||
eid: self.provider.config(eid, ctx)
|
||||
for eid in candidate_ids
|
||||
if self.provider.is_active(eid, ctx)
|
||||
}
|
||||
|
||||
|
||||
def feature_control_provider(domain: str | None = None) -> ActivationProvider | None:
|
||||
"""Return a feature-control-backed provider if ``feature_control_sdk`` is importable, else
|
||||
``None`` (caller falls back to :class:`StaticProvider`). Lazy import keeps the engine core
|
||||
dependency-free (ADR-0001)."""
|
||||
try: # optional engine extra — not a core dependency
|
||||
from feature_control_sdk import FeatureControlClient # type: ignore
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
client = FeatureControlClient(domain=domain)
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _FeatureControlProvider:
|
||||
_client: Any
|
||||
|
||||
def is_active(self, feature_key: str, context: Mapping[str, Any]) -> bool:
|
||||
return bool(
|
||||
self._client.get_boolean_value(feature_key, False, context=dict(context))
|
||||
)
|
||||
|
||||
def config(self, feature_key: str, context: Mapping[str, Any]) -> Mapping[str, Any]:
|
||||
getter = getattr(self._client, "get_object_value", None)
|
||||
return dict(getter(feature_key, {}, context=dict(context))) if getter else {}
|
||||
|
||||
return _FeatureControlProvider(client)
|
||||
82
src/shard_wiki/engine/adapter.py
Normal file
82
src/shard_wiki/engine/adapter.py
Normal file
@@ -0,0 +1,82 @@
|
||||
"""EngineShardAdapter — the engine exposed as a canonical-mode shard (WikiEngineCoreArchitecture
|
||||
§6, E-1/E-5).
|
||||
|
||||
The engine is *one shard*: the orchestrator consumes it only through this `ShardAdapter`. The
|
||||
adapter is backed by the kernel (T1) + a composed extension set (T2/T3); its §A capability
|
||||
profile is **derived from the active extensions** (T4), so the orchestrator sees an honest,
|
||||
conformance-verifiable profile that reflects exactly what is activated. Read/write run the
|
||||
extensions' transform hooks; everything above this stays in the orchestrator (no union/projection
|
||||
import).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
|
||||
from shard_wiki.adapters import ShardAdapter
|
||||
from shard_wiki.engine.activation import ActivationContext, ActivationProvider, ActivationResolver
|
||||
from shard_wiki.engine.extension import ActiveExtensions, ExtensionRuntime, Hook
|
||||
from shard_wiki.engine.kernel import EngineKernel
|
||||
from shard_wiki.engine.profile import derive_profile
|
||||
from shard_wiki.model import CapabilityProfile, NotSupported, Page, Verb
|
||||
|
||||
__all__ = ["EngineShardAdapter", "build_engine_shard"]
|
||||
|
||||
|
||||
class EngineShardAdapter(ShardAdapter):
|
||||
def __init__(
|
||||
self,
|
||||
kernel: EngineKernel,
|
||||
active: ActiveExtensions,
|
||||
base_profile: CapabilityProfile | None = None,
|
||||
) -> None:
|
||||
self._kernel = kernel
|
||||
self._active = active
|
||||
self._profile = derive_profile(active, base_profile) # validated (E-5)
|
||||
|
||||
@property
|
||||
def shard_id(self) -> str:
|
||||
return self._kernel.shard_id
|
||||
|
||||
def profile(self) -> CapabilityProfile:
|
||||
return self._profile
|
||||
|
||||
def keys(self) -> Iterable[str]:
|
||||
return self._kernel.keys()
|
||||
|
||||
def read(self, key: str) -> Page:
|
||||
page = self._kernel.read(key)
|
||||
return self._active.dispatch_transform(Hook.ON_READ, page, {"shard_id": self.shard_id})
|
||||
|
||||
def current_rev(self, key: str) -> str | None:
|
||||
return self._kernel.current_rev(key)
|
||||
|
||||
def write(self, key: str, body: str) -> Page:
|
||||
if not self._profile.supports(Verb.WRITE):
|
||||
raise NotSupported(f"{type(self).__name__} ({self.shard_id}) is read-only")
|
||||
body = self._active.dispatch_transform(
|
||||
Hook.ON_WRITE, body, {"shard_id": self.shard_id, "key": key}
|
||||
)
|
||||
return self._kernel.write(key, body)
|
||||
|
||||
|
||||
def build_engine_shard(
|
||||
shard_id: str,
|
||||
runtime: ExtensionRuntime,
|
||||
*,
|
||||
activate: Iterable[str] | None = None,
|
||||
provider: ActivationProvider | None = None,
|
||||
context: ActivationContext | None = None,
|
||||
base_profile: CapabilityProfile | None = None,
|
||||
) -> EngineShardAdapter:
|
||||
"""Stand up an engine shard: resolve which extensions are active (explicit ``activate`` ids,
|
||||
or via an activation ``provider`` over the runtime's available set), compose them, and wrap a
|
||||
fresh kernel as a `ShardAdapter`.
|
||||
"""
|
||||
if provider is not None:
|
||||
ctx = context or ActivationContext(shard_id)
|
||||
ids = ActivationResolver(provider).active_extensions(runtime.available(), ctx)
|
||||
else:
|
||||
ids = set(activate or ())
|
||||
active = runtime.activate(ids)
|
||||
return EngineShardAdapter(EngineKernel(shard_id), active, base_profile)
|
||||
165
src/shard_wiki/engine/extension.py
Normal file
165
src/shard_wiki/engine/extension.py
Normal file
@@ -0,0 +1,165 @@
|
||||
"""Typed-extension runtime — the engine framework (WikiEngineCoreArchitecture §4, E-3/E-9).
|
||||
|
||||
Everything beyond the kernel's c2-minimum is an :class:`Extension`: it declares a typed
|
||||
contract (id, provided capabilities, declared types, bound hooks, dependencies, conflicts) and
|
||||
the runtime **composes** an activation set deterministically, **rejecting impossible profiles**
|
||||
(unmet deps / conflicts / type collisions) — the §6.5 capability-as-data discipline applied to
|
||||
extensions. Extension structure is **verified at registration** (mirrors §6.6 conformance):
|
||||
bad ids or non-callable hook handlers are refused, so the framework acts on verified data.
|
||||
|
||||
Hooks are dispatched in a declared, deterministic order (dependency-topological, ties by id):
|
||||
*transform* hooks chain a payload through handlers; *collect* hooks gather contributions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable, Mapping
|
||||
from enum import Enum
|
||||
from typing import Any, ClassVar
|
||||
|
||||
__all__ = ["Hook", "Extension", "ExtensionError", "ExtensionRuntime", "ActiveExtensions"]
|
||||
|
||||
|
||||
class ExtensionError(ValueError):
|
||||
"""Raised when an extension is malformed or an activation set is impossible (§6.5)."""
|
||||
|
||||
|
||||
class Hook(Enum):
|
||||
# transform hooks: each handler takes (payload, ctx) and returns the next payload
|
||||
ON_WRITE = "on_write" # transform a draft before persist
|
||||
ON_READ = "on_read" # transform a page on read
|
||||
ON_RESOLVE = "on_resolve" # transform a name resolution
|
||||
ON_RENDER = "on_render" # produce a derived representation
|
||||
# collect hooks: each handler takes (payload, ctx) and returns a contribution
|
||||
ON_LINK = "on_link" # contribute link/transclusion edges
|
||||
ON_QUERY = "on_query" # answer a query
|
||||
ON_PROFILE = "on_profile" # contribute capability-profile positions (E-5)
|
||||
|
||||
|
||||
_TRANSFORM = frozenset({Hook.ON_WRITE, Hook.ON_READ, Hook.ON_RESOLVE, Hook.ON_RENDER})
|
||||
_COLLECT = frozenset({Hook.ON_LINK, Hook.ON_QUERY, Hook.ON_PROFILE})
|
||||
|
||||
|
||||
class Extension:
|
||||
"""Base class for a typed extension. Subclasses set the class vars and override
|
||||
:meth:`hooks` to bind handlers (signature ``handler(payload, ctx) -> result``)."""
|
||||
|
||||
id: ClassVar[str] = ""
|
||||
provides: ClassVar[tuple[str, ...]] = ()
|
||||
declares_types: ClassVar[tuple[str, ...]] = ()
|
||||
depends_on: ClassVar[tuple[str, ...]] = ()
|
||||
conflicts_with: ClassVar[tuple[str, ...]] = ()
|
||||
|
||||
def hooks(self) -> Mapping[Hook, Callable[[Any, Any], Any]]:
|
||||
return {}
|
||||
|
||||
|
||||
class ActiveExtensions:
|
||||
"""A composed, ordered activation set with deterministic hook dispatch."""
|
||||
|
||||
def __init__(self, ordered: list[Extension]) -> None:
|
||||
self._ordered = ordered
|
||||
self.ids: tuple[str, ...] = tuple(e.id for e in ordered)
|
||||
self._tables: dict[Hook, list[tuple[str, Callable[[Any, Any], Any]]]] = {}
|
||||
for ext in ordered:
|
||||
for hook, fn in ext.hooks().items():
|
||||
self._tables.setdefault(hook, []).append((ext.id, fn))
|
||||
|
||||
def handlers(self, hook: Hook) -> tuple[str, ...]:
|
||||
"""The extension ids bound to ``hook``, in dispatch order (for introspection)."""
|
||||
return tuple(eid for eid, _ in self._tables.get(hook, ()))
|
||||
|
||||
def dispatch_transform(self, hook: Hook, payload: Any, ctx: Any = None) -> Any:
|
||||
if hook not in _TRANSFORM:
|
||||
raise ExtensionError(f"{hook} is not a transform hook")
|
||||
for _eid, fn in self._tables.get(hook, ()):
|
||||
payload = fn(payload, ctx)
|
||||
return payload
|
||||
|
||||
def dispatch_collect(self, hook: Hook, payload: Any = None, ctx: Any = None) -> list[Any]:
|
||||
if hook not in _COLLECT:
|
||||
raise ExtensionError(f"{hook} is not a collect hook")
|
||||
return [fn(payload, ctx) for _eid, fn in self._tables.get(hook, ())]
|
||||
|
||||
|
||||
class ExtensionRuntime:
|
||||
def __init__(self) -> None:
|
||||
self._registered: dict[str, Extension] = {}
|
||||
|
||||
def available(self) -> frozenset[str]:
|
||||
"""Ids of all registered extensions (the candidate set for activation)."""
|
||||
return frozenset(self._registered)
|
||||
|
||||
def register(self, ext: Extension) -> Extension:
|
||||
"""Register an extension after structural verification (mirrors §6.6)."""
|
||||
if not ext.id or not ext.id.startswith("ext."):
|
||||
raise ExtensionError(f"extension id must be 'ext.<name>', got {ext.id!r}")
|
||||
if ext.id in self._registered:
|
||||
raise ExtensionError(f"duplicate extension id: {ext.id}")
|
||||
bound = ext.hooks()
|
||||
for hook, fn in bound.items():
|
||||
if not isinstance(hook, Hook):
|
||||
raise ExtensionError(f"{ext.id}: hook key {hook!r} is not a Hook")
|
||||
if not callable(fn):
|
||||
raise ExtensionError(f"{ext.id}: handler for {hook} is not callable")
|
||||
self._registered[ext.id] = ext
|
||||
return ext
|
||||
|
||||
def activate(self, ids: Iterable[str]) -> ActiveExtensions:
|
||||
"""Compose an activation set: dependency closure → conflict/type checks → deterministic
|
||||
order. Raises :class:`ExtensionError` on an impossible profile."""
|
||||
requested = set(ids)
|
||||
unknown = requested - self._registered.keys()
|
||||
if unknown:
|
||||
raise ExtensionError(f"unknown extensions: {sorted(unknown)}")
|
||||
|
||||
# dependency closure
|
||||
active: set[str] = set()
|
||||
frontier = list(requested)
|
||||
while frontier:
|
||||
eid = frontier.pop()
|
||||
if eid in active:
|
||||
continue
|
||||
ext = self._registered.get(eid)
|
||||
if ext is None:
|
||||
raise ExtensionError(f"unmet dependency: {eid}")
|
||||
active.add(eid)
|
||||
frontier.extend(d for d in ext.depends_on if d not in active)
|
||||
|
||||
exts = [self._registered[e] for e in active]
|
||||
|
||||
# conflicts
|
||||
for ext in exts:
|
||||
clash = active & set(ext.conflicts_with)
|
||||
if clash:
|
||||
raise ExtensionError(f"{ext.id} conflicts with active {sorted(clash)}")
|
||||
|
||||
# type collisions (two active extensions claiming the same type id)
|
||||
owner: dict[str, str] = {}
|
||||
for ext in exts:
|
||||
for t in ext.declares_types:
|
||||
if t in owner:
|
||||
raise ExtensionError(
|
||||
f"type collision on {t!r}: {owner[t]} and {ext.id}"
|
||||
)
|
||||
owner[t] = ext.id
|
||||
|
||||
return ActiveExtensions(self._topo_order(exts))
|
||||
|
||||
def _topo_order(self, exts: list[Extension]) -> list[Extension]:
|
||||
"""Dependencies before dependents; ties broken by id (deterministic)."""
|
||||
by_id = {e.id: e for e in exts}
|
||||
ordered: list[Extension] = []
|
||||
placed: set[str] = set()
|
||||
|
||||
def visit(ext: Extension) -> None:
|
||||
if ext.id in placed:
|
||||
return
|
||||
for dep in sorted(d for d in ext.depends_on if d in by_id):
|
||||
visit(by_id[dep])
|
||||
placed.add(ext.id)
|
||||
ordered.append(ext)
|
||||
|
||||
for ext in sorted(exts, key=lambda e: e.id):
|
||||
visit(ext)
|
||||
return ordered
|
||||
10
src/shard_wiki/engine/extensions/__init__.py
Normal file
10
src/shard_wiki/engine/extensions/__init__.py
Normal file
@@ -0,0 +1,10 @@
|
||||
"""engine/extensions/ — built-in typed extensions for the wiki engine.
|
||||
|
||||
Each is a typed :class:`~shard_wiki.engine.extension.Extension` a shard activates only if needed.
|
||||
``ext.struct`` (typed records) is the first; more (views, addressing, computational, authz) follow
|
||||
the same pattern.
|
||||
"""
|
||||
|
||||
from shard_wiki.engine.extensions.struct import StructExt, parse_frontmatter
|
||||
|
||||
__all__ = ["StructExt", "parse_frontmatter"]
|
||||
81
src/shard_wiki/engine/extensions/struct.py
Normal file
81
src/shard_wiki/engine/extensions/struct.py
Normal file
@@ -0,0 +1,81 @@
|
||||
"""ext.struct — typed records, a first built-in extension (WikiEngineCoreArchitecture X-STRUCT).
|
||||
|
||||
Demonstrates the typed-extension framework end-to-end. A page may carry a leading in-text
|
||||
frontmatter block (`---` … `---`, `key: value` lines — git-diffable structure, blueprint T12).
|
||||
With this extension **active**, the engine:
|
||||
|
||||
- **ON_WRITE** validates the structured block (optionally against an allowed-field set) — a
|
||||
malformed/disallowed structured page is rejected; the body is otherwise unchanged
|
||||
(content-preserving, so write conformance holds);
|
||||
- **ON_READ** tags such pages as `PageShape.TYPED_RECORD`;
|
||||
- **ON_PROFILE** raises the shard's profile with the `structured-payload` verb (E-5).
|
||||
|
||||
With the extension **inactive**, the kernel treats the same page as opaque prose — the feature
|
||||
is genuinely absent (honest profile). This is "activate only what you need" in action.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from collections.abc import Iterable, Mapping
|
||||
from typing import Any
|
||||
|
||||
from shard_wiki.engine.extension import Extension, Hook
|
||||
from shard_wiki.engine.profile import ProfileContribution
|
||||
from shard_wiki.model import Page, PageShape, Verb
|
||||
|
||||
__all__ = ["StructExt", "parse_frontmatter"]
|
||||
|
||||
|
||||
def parse_frontmatter(body: str) -> tuple[dict[str, str], bool]:
|
||||
"""Parse a leading ``---`` … ``---`` block of ``key: value`` lines.
|
||||
|
||||
Returns ``(fields, has_block)``. An unterminated opening ``---`` is *not* a valid block.
|
||||
"""
|
||||
lines = body.splitlines()
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return {}, False
|
||||
fields: dict[str, str] = {}
|
||||
for line in lines[1:]:
|
||||
if line.strip() == "---":
|
||||
return fields, True
|
||||
if ":" in line:
|
||||
key, _, value = line.partition(":")
|
||||
fields[key.strip()] = value.strip()
|
||||
return {}, False # no closing fence → not a frontmatter block
|
||||
|
||||
|
||||
class StructExt(Extension):
|
||||
id = "ext.struct"
|
||||
declares_types = ("record",)
|
||||
provides = ("capability.wiki.page-model",)
|
||||
|
||||
def __init__(self, allowed_fields: Iterable[str] | None = None) -> None:
|
||||
self._allowed: set[str] | None = set(allowed_fields) if allowed_fields is not None else None
|
||||
|
||||
def hooks(self) -> Mapping[Hook, Any]:
|
||||
return {
|
||||
Hook.ON_WRITE: self._on_write,
|
||||
Hook.ON_READ: self._on_read,
|
||||
Hook.ON_PROFILE: self._on_profile,
|
||||
}
|
||||
|
||||
def _on_write(self, body: str, ctx: Any) -> str:
|
||||
fields, has_block = parse_frontmatter(body)
|
||||
if has_block and self._allowed is not None:
|
||||
disallowed = set(fields) - self._allowed
|
||||
if disallowed:
|
||||
raise ValueError(f"ext.struct: disallowed fields {sorted(disallowed)}")
|
||||
return body # structure stays in-text (git-diffable); body unchanged
|
||||
|
||||
def _on_read(self, page: Page, ctx: Any) -> Page:
|
||||
_, has_block = parse_frontmatter(page.body)
|
||||
return dataclasses.replace(page, shape=PageShape.TYPED_RECORD) if has_block else page
|
||||
|
||||
def _on_profile(self, payload: Any, ctx: Any) -> ProfileContribution:
|
||||
return ProfileContribution(verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD}))
|
||||
|
||||
@staticmethod
|
||||
def fields(body: str) -> dict[str, str]:
|
||||
"""Parsed structured fields of a page body (empty if it has no frontmatter block)."""
|
||||
return parse_frontmatter(body)[0]
|
||||
87
src/shard_wiki/engine/kernel.py
Normal file
87
src/shard_wiki/engine/kernel.py
Normal file
@@ -0,0 +1,87 @@
|
||||
"""Engine kernel — the small page-store core (WikiEngineCoreArchitecture §3, EC-1…EC-4).
|
||||
|
||||
The irreducible engine: author/read/edit pages (edit = a new version; delete = a recoverable
|
||||
tombstone — history is the floor, I-10), enumerate keys, and resolve `[[wikilinks]]` (red-link =
|
||||
an unresolved target). No feature beyond this c2-minimum lives in the kernel; everything else is
|
||||
a typed extension (E-3).
|
||||
|
||||
Storage is intentionally simple here (in-memory version history); the git-IS-store backing
|
||||
(SHARD-WP-0009/0012) slots in behind the same API later. The kernel reuses the page model and
|
||||
provenance leaf; it does not redefine them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from shard_wiki.engine.links import extract_wikilinks
|
||||
from shard_wiki.model import Identity, Page, Placement
|
||||
from shard_wiki.provenance import Liveness, ProvenanceEnvelope, Staleness
|
||||
|
||||
__all__ = ["EngineKernel"]
|
||||
|
||||
|
||||
class EngineKernel:
|
||||
"""An in-process page store with per-page version history for one engine shard."""
|
||||
|
||||
def __init__(self, shard_id: str) -> None:
|
||||
self.shard_id = shard_id
|
||||
self._versions: dict[str, list[Page]] = {}
|
||||
self._deleted: set[str] = set()
|
||||
|
||||
# --- write path (create/edit are one operation; both append a version) ---
|
||||
def write(self, key: str, body: str) -> Page:
|
||||
versions = self._versions.setdefault(key, [])
|
||||
rev = str(len(versions) + 1)
|
||||
page = Page(
|
||||
identity=Identity(self.shard_id, key),
|
||||
body=body,
|
||||
envelope=ProvenanceEnvelope(
|
||||
source_shard=self.shard_id,
|
||||
liveness=Liveness.STATIC,
|
||||
staleness=Staleness.FRESH,
|
||||
source_rev=rev,
|
||||
observed_at=datetime.now(tz=timezone.utc),
|
||||
),
|
||||
placements=(Placement(self.shard_id, key),),
|
||||
)
|
||||
versions.append(page)
|
||||
self._deleted.discard(key)
|
||||
return page
|
||||
|
||||
# --- read path ---
|
||||
def exists(self, key: str) -> bool:
|
||||
return key in self._versions and key not in self._deleted
|
||||
|
||||
def read(self, key: str) -> Page:
|
||||
"""Latest version of a live page. Raises ``KeyError`` if absent or deleted."""
|
||||
if not self.exists(key):
|
||||
raise KeyError(key)
|
||||
return self._versions[key][-1]
|
||||
|
||||
def keys(self) -> Iterable[str]:
|
||||
return (k for k in sorted(self._versions) if k not in self._deleted)
|
||||
|
||||
def current_rev(self, key: str) -> str | None:
|
||||
return self._versions[key][-1].envelope.source_rev if self.exists(key) else None
|
||||
|
||||
# --- history & recoverability (I-10): versions are retained across delete ---
|
||||
def history(self, key: str) -> tuple[Page, ...]:
|
||||
"""All versions ever written for ``key`` (oldest→newest), even after delete."""
|
||||
return tuple(self._versions.get(key, ()))
|
||||
|
||||
def delete(self, key: str) -> None:
|
||||
"""Tombstone a page (history retained; restore by writing again)."""
|
||||
if key not in self._versions:
|
||||
raise KeyError(key)
|
||||
self._deleted.add(key)
|
||||
|
||||
# --- links (EC-4): resolution + red-link detection within this shard ---
|
||||
def links(self, key: str) -> list[str]:
|
||||
"""Wikilink targets in a page's current body."""
|
||||
return extract_wikilinks(self.read(key).body)
|
||||
|
||||
def resolve_link(self, target: str) -> Identity | None:
|
||||
"""Resolve a wikilink target to a live page identity, or ``None`` (a **red-link**)."""
|
||||
return self.read(target).identity if self.exists(target) else None
|
||||
25
src/shard_wiki/engine/links.py
Normal file
25
src/shard_wiki/engine/links.py
Normal file
@@ -0,0 +1,25 @@
|
||||
"""Wikilink extraction — the kernel's link primitive (WikiEngineCoreArchitecture EC-4).
|
||||
|
||||
`[[Target]]` and `[[Target|label]]`. CamelCase auto-linking is intentionally NOT here (it is an
|
||||
opt-in concern per FederationRequirements ADR-06); the kernel only knows explicit wikilinks.
|
||||
Link *resolution* (and red-link detection) is the kernel's job (it knows which keys exist);
|
||||
*rendering* is a consumer concern (headless engine, no UI).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
__all__ = ["extract_wikilinks"]
|
||||
|
||||
_WIKILINK = re.compile(r"\[\[([^\]|]+?)(?:\|[^\]]*)?\]\]")
|
||||
|
||||
|
||||
def extract_wikilinks(body: str) -> list[str]:
|
||||
"""Return the ordered, de-duplicated wikilink targets in ``body`` (label part dropped)."""
|
||||
seen: dict[str, None] = {}
|
||||
for m in _WIKILINK.finditer(body):
|
||||
target = m.group(1).strip()
|
||||
if target:
|
||||
seen.setdefault(target, None)
|
||||
return list(seen)
|
||||
112
src/shard_wiki/engine/profile.py
Normal file
112
src/shard_wiki/engine/profile.py
Normal file
@@ -0,0 +1,112 @@
|
||||
"""Capability profile derived from active extensions (WikiEngineCoreArchitecture E-5).
|
||||
|
||||
The engine's §A `CapabilityProfile` is **computed**, not hand-set: start from the kernel base
|
||||
profile, then fold each active extension's `on_profile` contribution (in the runtime's
|
||||
deterministic order), then `validate()`. This realizes the chain *configuration (which
|
||||
extensions) → capability (the profile) → conformance* — activating an extension raises the
|
||||
shard's advertised capabilities, and composition can never yield an impossible profile (validate
|
||||
rejects it, §6.5).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.engine.extension import ActiveExtensions, Hook
|
||||
from shard_wiki.model import (
|
||||
AccessGrant,
|
||||
Addressing,
|
||||
AttachmentMode,
|
||||
CapabilityProfile,
|
||||
ContentOpacity,
|
||||
History,
|
||||
MergeModel,
|
||||
NativeQuery,
|
||||
OperationalEnvelope,
|
||||
Substrate,
|
||||
Translation,
|
||||
Verb,
|
||||
WriteGranularity,
|
||||
)
|
||||
from shard_wiki.provenance import Liveness
|
||||
|
||||
__all__ = ["engine_base_profile", "ProfileContribution", "derive_profile"]
|
||||
|
||||
# Profile fields an extension may *raise* via on_profile (substrate/attachment are kernel-fixed).
|
||||
_OVERRIDABLE = (
|
||||
"write_granularity",
|
||||
"content_opacity",
|
||||
"liveness",
|
||||
"history",
|
||||
"merge_model",
|
||||
"addressing",
|
||||
"native_query",
|
||||
"translation",
|
||||
"access_grant",
|
||||
)
|
||||
|
||||
|
||||
def engine_base_profile() -> CapabilityProfile:
|
||||
"""The kernel-only (no extensions) capability profile — the c2-minimum engine shard."""
|
||||
return CapabilityProfile(
|
||||
substrate=Substrate.FILES,
|
||||
attachment_mode=AttachmentMode.IN_ENGINE_HOST,
|
||||
write_granularity=WriteGranularity.PER_PAGE,
|
||||
content_opacity=ContentOpacity.TRANSPARENT,
|
||||
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||
access_grant=AccessGrant.OPEN,
|
||||
liveness=Liveness.STATIC,
|
||||
history=History.INTERNAL_ONLY,
|
||||
merge_model=MergeModel.NONE,
|
||||
addressing=Addressing.PATH,
|
||||
native_query=NativeQuery.NONE,
|
||||
translation=Translation.NATIVE,
|
||||
supported_verbs=frozenset({Verb.READ, Verb.WRITE}),
|
||||
).validate()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProfileContribution:
|
||||
"""An extension's contribution to the derived profile (returned from its ON_PROFILE hook).
|
||||
|
||||
A non-``None`` axis overrides that axis; ``verbs_add`` are unioned in. Order = the runtime's
|
||||
deterministic dispatch order, so later extensions win on a contested axis."""
|
||||
|
||||
write_granularity: WriteGranularity | None = None
|
||||
content_opacity: ContentOpacity | None = None
|
||||
liveness: Liveness | None = None
|
||||
history: History | None = None
|
||||
merge_model: MergeModel | None = None
|
||||
addressing: Addressing | None = None
|
||||
native_query: NativeQuery | None = None
|
||||
translation: Translation | None = None
|
||||
access_grant: AccessGrant | None = None
|
||||
verbs_add: frozenset[Verb] = frozenset()
|
||||
|
||||
|
||||
def derive_profile(
|
||||
active: ActiveExtensions, base: CapabilityProfile | None = None
|
||||
) -> CapabilityProfile:
|
||||
"""Fold active extensions' ON_PROFILE contributions onto ``base`` and validate the result.
|
||||
|
||||
Raises :class:`~shard_wiki.model.ProfileError` if the composed profile is impossible — so an
|
||||
activation set can never advertise an invalid capability profile.
|
||||
"""
|
||||
profile = base or engine_base_profile()
|
||||
contributions = active.dispatch_collect(Hook.ON_PROFILE)
|
||||
|
||||
overrides: dict[str, object] = {}
|
||||
verbs: set[Verb] = set(profile.supported_verbs)
|
||||
for contrib in contributions:
|
||||
if not isinstance(contrib, ProfileContribution):
|
||||
continue
|
||||
for field_name in _OVERRIDABLE:
|
||||
value = getattr(contrib, field_name)
|
||||
if value is not None:
|
||||
overrides[field_name] = value
|
||||
verbs |= set(contrib.verbs_add)
|
||||
|
||||
return dataclasses.replace(
|
||||
profile, supported_verbs=frozenset(verbs), **overrides
|
||||
).validate()
|
||||
46
src/shard_wiki/incremental/__init__.py
Normal file
46
src/shard_wiki/incremental/__init__.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""incremental/ — the incremental-first derived tier (CoreArchitectureBlueprint §8.7).
|
||||
|
||||
Equivalence is **indexed** (blocking/LSH + verify), not pairwise O(N²); maintenance is
|
||||
**change-driven** (delta with retraction + propagation, review B-4), keeping the derived tier equal
|
||||
to a from-scratch rebuild — which becomes a bounded fallback, not the operational path. A
|
||||
Merkle-style **digest** plus a background **consistency-checker** make ``derived = f(canonical)``
|
||||
verified rather than asserted (I-2), self-healing on detected drift.
|
||||
|
||||
In-memory only for this slice (no persisted index store); per-partition structure is honoured but
|
||||
multi-tenant deployment is later. Per the dependency rule this imports down (model/provenance) and
|
||||
is wired by the orchestrator.
|
||||
"""
|
||||
|
||||
from shard_wiki.incremental.equivalence import (
|
||||
EquivalenceEdge,
|
||||
EquivalenceIndex,
|
||||
normalized_title,
|
||||
)
|
||||
from shard_wiki.incremental.minhash import (
|
||||
MinHasher,
|
||||
band_keys,
|
||||
jaccard,
|
||||
shingles,
|
||||
)
|
||||
from shard_wiki.incremental.union_index import UnionIndex
|
||||
from shard_wiki.incremental.verification import (
|
||||
ConsistencyChecker,
|
||||
ConsistencyReport,
|
||||
derived_digest,
|
||||
region_digest,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"shingles",
|
||||
"MinHasher",
|
||||
"band_keys",
|
||||
"jaccard",
|
||||
"EquivalenceEdge",
|
||||
"EquivalenceIndex",
|
||||
"normalized_title",
|
||||
"derived_digest",
|
||||
"region_digest",
|
||||
"ConsistencyReport",
|
||||
"ConsistencyChecker",
|
||||
"UnionIndex",
|
||||
]
|
||||
225
src/shard_wiki/incremental/equivalence.py
Normal file
225
src/shard_wiki/incremental/equivalence.py
Normal file
@@ -0,0 +1,225 @@
|
||||
"""Indexed equivalence — blocking + verify, incrementally maintained (SHARD-WP-0011 T1/T2).
|
||||
|
||||
Equivalence (two *distinct* identities holding the same page) is detected without pairwise O(N²):
|
||||
|
||||
1. **Blocking** generates candidate pairs — pages sharing a normalized-title bucket or an LSH band
|
||||
(MinHash over content shingles).
|
||||
2. **Verify** confirms a candidate — exact-body fingerprint match, or shingle Jaccard ≥ threshold —
|
||||
plus **curator bindings** (explicit decision-log edges) which are always equivalence edges.
|
||||
|
||||
The index is **incrementally maintained** (T2): ``add`` / ``update`` / ``remove`` re-bucket the
|
||||
changed page, **retract** the edges it leaves and **add** the edges it enters; equivalence groups
|
||||
are the connected components of the current edge set, so a retraction that disconnects a component
|
||||
**splits** a chorus automatically. A full :meth:`build` is just repeated ``add`` — the bounded
|
||||
rebuild fallback. The invariant (and the test oracle): incremental state == a from-scratch rebuild.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.incremental.minhash import MinHasher, band_keys, jaccard, shingles
|
||||
from shard_wiki.model import Identity, Page
|
||||
|
||||
__all__ = ["EquivalenceEdge", "EquivalenceIndex", "normalized_title"]
|
||||
|
||||
_NONALNUM_RE = re.compile(r"[^a-z0-9]+")
|
||||
|
||||
|
||||
def normalized_title(key: str) -> str:
|
||||
"""A blocking bucket key: the last path segment, lowercased, stripped of non-alphanumerics."""
|
||||
leaf = key.rsplit("/", 1)[-1]
|
||||
return _NONALNUM_RE.sub("", leaf.lower())
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class EquivalenceEdge:
|
||||
"""A verified equivalence between two identities, tagged with why it was accepted."""
|
||||
|
||||
a: Identity
|
||||
b: Identity
|
||||
reason: str # "fingerprint" | "content" | "curator"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _Entry:
|
||||
shingle_set: frozenset[str]
|
||||
bands: tuple[tuple[int, tuple[int, ...]], ...]
|
||||
title: str
|
||||
fingerprint: str
|
||||
|
||||
|
||||
def _fingerprint(body: str) -> str:
|
||||
return hashlib.blake2b(body.strip().encode("utf-8"), digest_size=16).hexdigest()
|
||||
|
||||
|
||||
def _pair(a: Identity, b: Identity) -> frozenset[Identity]:
|
||||
return frozenset((a, b))
|
||||
|
||||
|
||||
class EquivalenceIndex:
|
||||
"""An incrementally maintained, blocked-and-verified equivalence relation over union pages."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
num_perm: int = 64,
|
||||
num_bands: int = 32,
|
||||
threshold: float = 0.7,
|
||||
hasher: MinHasher | None = None,
|
||||
) -> None:
|
||||
self.threshold = threshold
|
||||
self.num_bands = num_bands
|
||||
self._hasher = hasher or MinHasher(num_perm=num_perm)
|
||||
self._entries: dict[Identity, _Entry] = {}
|
||||
self._band_buckets: dict[tuple[int, tuple[int, ...]], set[Identity]] = {}
|
||||
self._title_buckets: dict[str, set[Identity]] = {}
|
||||
self._content_edges: dict[frozenset[Identity], str] = {}
|
||||
self._curator_edges: set[frozenset[Identity]] = set()
|
||||
|
||||
# -- build / maintain ----------------------------------------------------
|
||||
|
||||
def build(
|
||||
self,
|
||||
pages: Iterable[Page],
|
||||
curator_edges: Iterable[tuple[Identity, Identity]] = (),
|
||||
) -> None:
|
||||
"""Rebuild from scratch (the bounded fallback): add every page, then curator edges."""
|
||||
self.__init__(
|
||||
num_bands=self.num_bands, threshold=self.threshold, hasher=self._hasher
|
||||
)
|
||||
for page in pages:
|
||||
self.add(page)
|
||||
for a, b in curator_edges:
|
||||
self.bind(a, b)
|
||||
|
||||
def add(self, page: Page) -> None:
|
||||
"""Index a new (or, via :meth:`update`, refreshed) page and add its equivalence edges."""
|
||||
identity = page.identity
|
||||
entry = self._make_entry(page)
|
||||
self._entries[identity] = entry
|
||||
for key in entry.bands:
|
||||
self._band_buckets.setdefault(key, set()).add(identity)
|
||||
self._title_buckets.setdefault(entry.title, set()).add(identity)
|
||||
|
||||
for candidate in self._candidates(identity, entry):
|
||||
reason = self._verify(identity, candidate)
|
||||
if reason is not None:
|
||||
self._content_edges[_pair(identity, candidate)] = reason
|
||||
|
||||
def remove(self, identity: Identity) -> None:
|
||||
"""Drop a page: de-bucket it and retract every content edge incident to it."""
|
||||
entry = self._entries.pop(identity, None)
|
||||
if entry is None:
|
||||
return
|
||||
for key in entry.bands:
|
||||
self._discard_bucket(self._band_buckets, key, identity)
|
||||
self._discard_bucket(self._title_buckets, entry.title, identity)
|
||||
for edge in [e for e in self._content_edges if identity in e]:
|
||||
del self._content_edges[edge]
|
||||
|
||||
def update(self, page: Page) -> None:
|
||||
"""Apply a change as retract-then-add: stale (bucket-exit) edges go, new edges arrive."""
|
||||
self.remove(page.identity)
|
||||
self.add(page)
|
||||
|
||||
def bind(self, a: Identity, b: Identity) -> None:
|
||||
"""Record a curator equivalence (an explicit decision-log binding); always an edge."""
|
||||
if a != b:
|
||||
self._curator_edges.add(_pair(a, b))
|
||||
|
||||
def unbind(self, a: Identity, b: Identity) -> None:
|
||||
self._curator_edges.discard(_pair(a, b))
|
||||
|
||||
def set_curator_edges(self, edges: Iterable[tuple[Identity, Identity]]) -> None:
|
||||
"""Replace all curator edges at once (re-syncing from the decision-log fold)."""
|
||||
self._curator_edges = {_pair(a, b) for a, b in edges if a != b}
|
||||
|
||||
# -- queries -------------------------------------------------------------
|
||||
|
||||
def identities(self) -> frozenset[Identity]:
|
||||
"""All identities currently present in the index."""
|
||||
return frozenset(self._entries)
|
||||
|
||||
def fingerprint(self, identity: Identity) -> str | None:
|
||||
"""The content fingerprint indexed for ``identity`` (None if absent) — a digest leaf."""
|
||||
entry = self._entries.get(identity)
|
||||
return entry.fingerprint if entry is not None else None
|
||||
|
||||
def edges(self) -> frozenset[frozenset[Identity]]:
|
||||
"""All equivalence edges (content + curator) among currently present identities."""
|
||||
present = self._entries.keys()
|
||||
curator = {e for e in self._curator_edges if e <= present}
|
||||
return frozenset(set(self._content_edges) | curator)
|
||||
|
||||
def groups(self) -> tuple[frozenset[Identity], ...]:
|
||||
"""Equivalence groups: connected components of size ≥ 2 (union-find over the edges)."""
|
||||
parent: dict[Identity, Identity] = {}
|
||||
|
||||
def find(x: Identity) -> Identity:
|
||||
parent.setdefault(x, x)
|
||||
root = x
|
||||
while parent[root] != root:
|
||||
root = parent[root]
|
||||
while parent[x] != root:
|
||||
parent[x], x = root, parent[x]
|
||||
return root
|
||||
|
||||
for edge in self.edges():
|
||||
a, b = tuple(edge)
|
||||
ra, rb = find(a), find(b)
|
||||
if ra != rb:
|
||||
parent[ra] = rb
|
||||
|
||||
comps: dict[Identity, set[Identity]] = {}
|
||||
for node in parent:
|
||||
comps.setdefault(find(node), set()).add(node)
|
||||
return tuple(
|
||||
frozenset(members) for members in comps.values() if len(members) > 1
|
||||
)
|
||||
|
||||
def equivalent_to(self, identity: Identity) -> frozenset[Identity]:
|
||||
"""The equivalence group containing ``identity`` (including itself), else just itself."""
|
||||
for group in self.groups():
|
||||
if identity in group:
|
||||
return group
|
||||
return frozenset({identity})
|
||||
|
||||
# -- internals -----------------------------------------------------------
|
||||
|
||||
def _make_entry(self, page: Page) -> _Entry:
|
||||
shingle_set = shingles(page.body)
|
||||
signature = self._hasher.signature(shingle_set)
|
||||
return _Entry(
|
||||
shingle_set=shingle_set,
|
||||
bands=band_keys(signature, self.num_bands),
|
||||
title=normalized_title(page.identity.key),
|
||||
fingerprint=_fingerprint(page.body),
|
||||
)
|
||||
|
||||
def _candidates(self, identity: Identity, entry: _Entry) -> set[Identity]:
|
||||
candidates: set[Identity] = set()
|
||||
for key in entry.bands:
|
||||
candidates |= self._band_buckets.get(key, set())
|
||||
candidates |= self._title_buckets.get(entry.title, set())
|
||||
candidates.discard(identity)
|
||||
return candidates
|
||||
|
||||
def _verify(self, a: Identity, b: Identity) -> str | None:
|
||||
ea, eb = self._entries[a], self._entries[b]
|
||||
if ea.fingerprint == eb.fingerprint:
|
||||
return "fingerprint"
|
||||
if jaccard(ea.shingle_set, eb.shingle_set) >= self.threshold:
|
||||
return "content"
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _discard_bucket(buckets: dict, key, identity: Identity) -> None:
|
||||
bucket = buckets.get(key)
|
||||
if bucket is not None:
|
||||
bucket.discard(identity)
|
||||
if not bucket:
|
||||
del buckets[key]
|
||||
71
src/shard_wiki/incremental/minhash.py
Normal file
71
src/shard_wiki/incremental/minhash.py
Normal file
@@ -0,0 +1,71 @@
|
||||
"""MinHash + LSH banding primitives for content-similarity blocking (SHARD-WP-0011 T1).
|
||||
|
||||
Pure, deterministic functions (fixed hashing, no per-run randomness) so the derived tier and its
|
||||
digest are reproducible. Shingle a body into k-grams, MinHash the shingle set into a signature,
|
||||
split the signature into LSH bands; two pages sharing a band are *candidates* for equivalence —
|
||||
the cheap pre-filter that replaces pairwise O(N²) comparison.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import random
|
||||
import re
|
||||
from collections.abc import Iterable
|
||||
|
||||
__all__ = ["shingles", "MinHasher", "band_keys", "jaccard"]
|
||||
|
||||
_WORD_RE = re.compile(r"\w+")
|
||||
# Largest Mersenne prime below 2**61 — the modulus for the universal-hash permutations.
|
||||
_PRIME = (1 << 61) - 1
|
||||
|
||||
|
||||
def shingles(text: str, k: int = 3) -> frozenset[str]:
|
||||
"""The set of word k-grams in ``text`` (lowercased). Short texts fall back to their word set."""
|
||||
words = _WORD_RE.findall(text.lower())
|
||||
if len(words) < k:
|
||||
return frozenset(words)
|
||||
return frozenset(" ".join(words[i : i + k]) for i in range(len(words) - k + 1))
|
||||
|
||||
|
||||
def _stable_hash(token: str) -> int:
|
||||
return int.from_bytes(hashlib.blake2b(token.encode("utf-8"), digest_size=8).digest(), "big")
|
||||
|
||||
|
||||
class MinHasher:
|
||||
"""A bank of ``num_perm`` universal hash permutations producing a fixed-length signature."""
|
||||
|
||||
def __init__(self, num_perm: int = 64, seed: int = 1) -> None:
|
||||
self.num_perm = num_perm
|
||||
rng = random.Random(seed)
|
||||
self._coeffs = [
|
||||
(rng.randrange(1, _PRIME), rng.randrange(0, _PRIME)) for _ in range(num_perm)
|
||||
]
|
||||
|
||||
def signature(self, shingle_set: Iterable[str]) -> tuple[int, ...]:
|
||||
"""The MinHash signature of ``shingle_set`` (empty set → all-``_PRIME`` sentinel)."""
|
||||
hashed = [_stable_hash(s) for s in shingle_set]
|
||||
if not hashed:
|
||||
return tuple(_PRIME for _ in self._coeffs)
|
||||
return tuple(min((a * h + b) % _PRIME for h in hashed) for a, b in self._coeffs)
|
||||
|
||||
|
||||
def band_keys(
|
||||
signature: tuple[int, ...], num_bands: int
|
||||
) -> tuple[tuple[int, tuple[int, ...]], ...]:
|
||||
"""Split a signature into ``num_bands`` band keys; two pages sharing one are LSH candidates."""
|
||||
if num_bands <= 0 or len(signature) % num_bands != 0:
|
||||
raise ValueError(f"signature length {len(signature)} not divisible into {num_bands} bands")
|
||||
rows = len(signature) // num_bands
|
||||
return tuple(
|
||||
(b, signature[b * rows : (b + 1) * rows]) for b in range(num_bands)
|
||||
)
|
||||
|
||||
|
||||
def jaccard(a: frozenset[str], b: frozenset[str]) -> float:
|
||||
"""Jaccard similarity of two shingle sets; two empty sets are defined as identical (1.0)."""
|
||||
if not a and not b:
|
||||
return 1.0
|
||||
if not a or not b:
|
||||
return 0.0
|
||||
return len(a & b) / len(a | b)
|
||||
91
src/shard_wiki/incremental/union_index.py
Normal file
91
src/shard_wiki/incremental/union_index.py
Normal file
@@ -0,0 +1,91 @@
|
||||
"""UnionIndex — the maintained derived tier wired behind resolution + views (SHARD-WP-0011 T4).
|
||||
|
||||
Wraps a :class:`UnionGraph` + decision log with an incrementally maintained
|
||||
:class:`EquivalenceIndex`. Content equivalence is kept fresh by deltas (``note_change`` /
|
||||
``note_removed``); curator bindings are re-synced live from the log fold. A full :meth:`rebuild`
|
||||
is the bounded fallback. :meth:`verify` runs the I-2 consistency-checker over the live source.
|
||||
|
||||
Consumer-visible results are unchanged — equivalence groups are exposed in the same string form the
|
||||
decision-log fold uses, a *superset* that additionally collapses genuine content duplicates — only
|
||||
freshness and cost differ (recompute-on-read becomes change-driven).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from shard_wiki.coordination import DecisionLog
|
||||
from shard_wiki.incremental.equivalence import EquivalenceIndex
|
||||
from shard_wiki.incremental.verification import (
|
||||
ConsistencyChecker,
|
||||
ConsistencyReport,
|
||||
derived_digest,
|
||||
)
|
||||
from shard_wiki.model import Identity, Page
|
||||
from shard_wiki.union import UnionGraph
|
||||
|
||||
__all__ = ["UnionIndex"]
|
||||
|
||||
|
||||
def _identity(token: str) -> Identity:
|
||||
shard, _, key = token.partition(":")
|
||||
return Identity(shard, key)
|
||||
|
||||
|
||||
class UnionIndex:
|
||||
"""An incrementally maintained equivalence index over a union, with a rebuild fallback."""
|
||||
|
||||
def __init__(self, union: UnionGraph, log: DecisionLog, space: str) -> None:
|
||||
self._union = union
|
||||
self._log = log
|
||||
self._space = space
|
||||
self._eq = EquivalenceIndex()
|
||||
self.rebuild()
|
||||
|
||||
def rebuild(self) -> None:
|
||||
"""The bounded fallback: re-derive the whole index from current union pages + bindings."""
|
||||
self._eq.build(self._union.iter_pages())
|
||||
self._sync_curator()
|
||||
|
||||
def note_change(self, page: Page) -> None:
|
||||
"""Change-driven update for one added/edited page (the operational path)."""
|
||||
self._eq.update(page)
|
||||
|
||||
def note_removed(self, identity: Identity) -> None:
|
||||
self._eq.remove(identity)
|
||||
|
||||
def _sync_curator(self) -> None:
|
||||
"""Re-sync curator equivalence from the live decision-log fold (cheap, always correct)."""
|
||||
groups = self._log.fold(self._space).equivalence_groups
|
||||
edges: list[tuple[Identity, Identity]] = []
|
||||
for group in groups:
|
||||
members = [_identity(m) for m in group]
|
||||
edges.extend((members[0], other) for other in members[1:])
|
||||
self._eq.set_curator_edges(edges)
|
||||
|
||||
def equivalence_groups(self) -> tuple[frozenset[str], ...]:
|
||||
"""Equivalence groups in decision-log string form (curator ∪ content), for the views."""
|
||||
self._sync_curator()
|
||||
return tuple(
|
||||
frozenset(str(identity) for identity in group) for group in self._eq.groups()
|
||||
)
|
||||
|
||||
def digest(self) -> str:
|
||||
"""The Merkle-style digest of the maintained derived tier (I-2)."""
|
||||
self._sync_curator()
|
||||
return derived_digest(self._eq)
|
||||
|
||||
def verify(self) -> ConsistencyReport:
|
||||
"""Check the maintained index against a from-scratch fold of the live source; self-heal."""
|
||||
self._sync_curator()
|
||||
checker = ConsistencyChecker(
|
||||
self._eq,
|
||||
pages=lambda: list(self._union.iter_pages()),
|
||||
curator_edges=self._curator_pairs,
|
||||
)
|
||||
return checker.check_and_repair()
|
||||
|
||||
def _curator_pairs(self) -> list[tuple[Identity, Identity]]:
|
||||
pairs: list[tuple[Identity, Identity]] = []
|
||||
for group in self._log.fold(self._space).equivalence_groups:
|
||||
members = [_identity(m) for m in group]
|
||||
pairs.extend((members[0], other) for other in members[1:])
|
||||
return pairs
|
||||
112
src/shard_wiki/incremental/verification.py
Normal file
112
src/shard_wiki/incremental/verification.py
Normal file
@@ -0,0 +1,112 @@
|
||||
"""I-2 verification — digest + background consistency-checker (SHARD-WP-0011 T3).
|
||||
|
||||
``derived = f(canonical)`` is made *verified*, not asserted. A **Merkle-style digest** summarizes
|
||||
the derived tier (each identity's content fingerprint + its incident equivalence edges as a leaf,
|
||||
order-independently combined into a root) so two derived states are equal iff their digests match.
|
||||
A **consistency-checker** recomputes the authoritative fold from the current source, compares it to
|
||||
the maintained index over a (sampled) region, and on mismatch performs a **scoped recompute** of
|
||||
just the affected identities — self-healing drift from a missed delta or corrupted state.
|
||||
|
||||
The digest is a pure function of index state, so it is "maintained alongside deltas" for free and
|
||||
is stable under equivalent event orders (leaves are sorted before combination).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from collections.abc import Callable, Iterable
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.incremental.equivalence import EquivalenceIndex
|
||||
from shard_wiki.model import Identity, Page
|
||||
|
||||
__all__ = ["region_digest", "derived_digest", "ConsistencyReport", "ConsistencyChecker"]
|
||||
|
||||
CuratorEdges = Iterable[tuple[Identity, Identity]]
|
||||
|
||||
|
||||
def _leaf(index: EquivalenceIndex, identity: Identity) -> str:
|
||||
"""A digest leaf for one identity: its fingerprint + its incident edges (as sorted peers)."""
|
||||
fingerprint = index.fingerprint(identity) or "∅"
|
||||
peers = sorted(
|
||||
str(other)
|
||||
for edge in index.edges()
|
||||
if identity in edge
|
||||
for other in edge
|
||||
if other != identity
|
||||
)
|
||||
payload = f"{identity}|{fingerprint}|{','.join(peers)}"
|
||||
return hashlib.blake2b(payload.encode("utf-8"), digest_size=16).hexdigest()
|
||||
|
||||
|
||||
def region_digest(index: EquivalenceIndex, identities: Iterable[Identity]) -> str:
|
||||
"""A Merkle-style root over the given identities' leaves (order-independent)."""
|
||||
leaves = sorted(_leaf(index, identity) for identity in identities)
|
||||
root = hashlib.blake2b(digest_size=16)
|
||||
for leaf in leaves:
|
||||
root.update(leaf.encode("utf-8"))
|
||||
return root.hexdigest()
|
||||
|
||||
|
||||
def derived_digest(index: EquivalenceIndex) -> str:
|
||||
"""The digest of the whole maintained derived tier."""
|
||||
return region_digest(index, index.identities())
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ConsistencyReport:
|
||||
"""Outcome of a consistency check: what was examined, whether it drifted, and if it healed."""
|
||||
|
||||
checked: int
|
||||
drifted: bool
|
||||
repaired: bool
|
||||
healthy: bool
|
||||
|
||||
|
||||
class ConsistencyChecker:
|
||||
"""Compares the maintained index against an authoritative rebuild and repairs drift in place."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
index: EquivalenceIndex,
|
||||
pages: Callable[[], Iterable[Page]],
|
||||
curator_edges: Callable[[], CuratorEdges] = lambda: (),
|
||||
) -> None:
|
||||
self._index = index
|
||||
self._pages = pages
|
||||
self._curator = curator_edges
|
||||
|
||||
def _authoritative(self) -> EquivalenceIndex:
|
||||
expected = EquivalenceIndex(
|
||||
num_bands=self._index.num_bands, threshold=self._index.threshold
|
||||
)
|
||||
expected.build(list(self._pages()), list(self._curator()))
|
||||
return expected
|
||||
|
||||
def check_and_repair(self, sample: Iterable[Identity] | None = None) -> ConsistencyReport:
|
||||
"""Verify the (sampled) region against a from-scratch fold; scoped-recompute on mismatch."""
|
||||
source = {p.identity: p for p in self._pages()}
|
||||
expected = self._authoritative()
|
||||
region = (
|
||||
set(sample)
|
||||
if sample is not None
|
||||
else set(source) | set(self._index.identities())
|
||||
)
|
||||
|
||||
drifted = region_digest(self._index, region) != region_digest(expected, region)
|
||||
if not drifted:
|
||||
return ConsistencyReport(len(region), drifted=False, repaired=False, healthy=True)
|
||||
|
||||
self._repair(region, source)
|
||||
healthy = region_digest(self._index, region) == region_digest(expected, region)
|
||||
return ConsistencyReport(len(region), drifted=True, repaired=True, healthy=healthy)
|
||||
|
||||
def _repair(self, region: set[Identity], source: dict[Identity, Page]) -> None:
|
||||
"""Scoped recompute: reconcile each affected identity to the current source."""
|
||||
present = self._index.identities()
|
||||
for identity in region:
|
||||
page = source.get(identity)
|
||||
if page is not None:
|
||||
self._index.update(page) if identity in present else self._index.add(page)
|
||||
elif identity in present:
|
||||
self._index.remove(identity)
|
||||
@@ -8,32 +8,69 @@ a network API is a later workplan.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.adapters import ShardAdapter, assert_conformant
|
||||
from shard_wiki.coordination import (
|
||||
ApplyResult,
|
||||
DecisionLog,
|
||||
EventStore,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
Overlay,
|
||||
OverlayEngine,
|
||||
)
|
||||
from shard_wiki.incremental import ConsistencyReport, UnionIndex
|
||||
from shard_wiki.model import Page
|
||||
from shard_wiki.policy import DEFAULT_POLICY, Policy
|
||||
from shard_wiki.union import Resolution, UnionGraph
|
||||
from shard_wiki.views import (
|
||||
AllPagesEntry,
|
||||
BackLink,
|
||||
ChangeEntry,
|
||||
SiteMapNode,
|
||||
all_pages,
|
||||
build_backlinks,
|
||||
recent_changes,
|
||||
site_map,
|
||||
)
|
||||
|
||||
__all__ = ["InformationSpace"]
|
||||
|
||||
|
||||
class InformationSpace:
|
||||
def __init__(self, space_id: str, policy: Policy = DEFAULT_POLICY) -> None:
|
||||
def __init__(
|
||||
self,
|
||||
space_id: str,
|
||||
policy: Policy = DEFAULT_POLICY,
|
||||
*,
|
||||
store: EventStore | None = None,
|
||||
) -> None:
|
||||
"""Tie the slice together. ``store`` selects the coordination-log backend: the default
|
||||
in-memory store (tests) or a git-addressable one. Use :meth:`git_backed` for the latter."""
|
||||
self.space_id = space_id
|
||||
self.log = DecisionLog()
|
||||
self.log = DecisionLog(store)
|
||||
self.union = UnionGraph(space_id, log=self.log, policy=policy)
|
||||
self.overlays = OverlayEngine(space_id, self.log)
|
||||
self._index: UnionIndex | None = None # maintained derived tier, built lazily
|
||||
self._index_stale = True
|
||||
|
||||
@classmethod
|
||||
def git_backed(
|
||||
cls,
|
||||
space_id: str,
|
||||
repo_path: str | Path,
|
||||
policy: Policy = DEFAULT_POLICY,
|
||||
) -> InformationSpace:
|
||||
"""An information space whose coordination log is git-addressable (history/patch/review/
|
||||
backup — I-6). The decision log lives in the git repo at ``repo_path``."""
|
||||
return cls(space_id, policy, store=GitEventStore(repo_path))
|
||||
|
||||
def attach(self, adapter: ShardAdapter) -> None:
|
||||
"""Attach a shard — only if it passes conformance (verified profile, I-3/§6.6)."""
|
||||
assert_conformant(adapter)
|
||||
self.union.attach(adapter)
|
||||
self._index_stale = True
|
||||
|
||||
def alias(self, name: str, target: str, actor: str | None = None) -> None:
|
||||
"""Record a coordination-canonical alias (``name`` → ``"shard:key"``) in the log."""
|
||||
@@ -68,4 +105,44 @@ class InformationSpace:
|
||||
write-through-capable target fast-forwards (write-through); a read-only target keeps the
|
||||
draft as local truth (I-5: overlay before mutation, always)."""
|
||||
overlay = self.overlay(name, body, actor=actor)
|
||||
return self.apply_overlay(overlay.overlay_id)
|
||||
result = self.apply_overlay(overlay.overlay_id)
|
||||
self._index_stale = True # the applied edit changes the derived tier
|
||||
return result
|
||||
|
||||
# --- maintained derived tier (SHARD-WP-0011): incremental-first, rebuild as fallback ---
|
||||
|
||||
@property
|
||||
def index(self) -> UnionIndex:
|
||||
"""The maintained equivalence index (built lazily; rebuilt when the union has changed)."""
|
||||
if self._index is None:
|
||||
self._index = UnionIndex(self.union, self.log, self.space_id)
|
||||
elif self._index_stale:
|
||||
self._index.rebuild() # bounded fallback after a mutation
|
||||
self._index_stale = False
|
||||
return self._index
|
||||
|
||||
def reindex(self) -> None:
|
||||
"""Force a full rebuild of the maintained derived tier (the explicit fallback path)."""
|
||||
self.index.rebuild()
|
||||
|
||||
def verify_index(self) -> ConsistencyReport:
|
||||
"""Run the I-2 consistency-checker over the maintained tier; self-heal any drift."""
|
||||
return self.index.verify()
|
||||
|
||||
# --- derived views (SHARD-WP-0010): recomputable, provenance-carrying, presentation-free ---
|
||||
|
||||
def backlinks(self, name: str, *, camelcase: bool = False) -> tuple[BackLink, ...]:
|
||||
"""Pages across the union that link to ``name`` (UC-18)."""
|
||||
return build_backlinks(self.union, camelcase=camelcase).to(name)
|
||||
|
||||
def recent_changes(self, *, limit: int | None = None) -> tuple[ChangeEntry, ...]:
|
||||
"""The merged newest-first change feed: coordination journal + shard signals (UC-17)."""
|
||||
return recent_changes(self.union, self.log, self.space_id, limit=limit)
|
||||
|
||||
def all_pages(self) -> tuple[AllPagesEntry, ...]:
|
||||
"""The union's distinct pages, collapsed via the maintained equivalence index."""
|
||||
return all_pages(self.union, equivalence_groups=self.index.equivalence_groups())
|
||||
|
||||
def site_map(self) -> SiteMapNode:
|
||||
"""The union namespace tree built from page placements."""
|
||||
return site_map(self.union)
|
||||
|
||||
@@ -13,6 +13,7 @@ imported by nothing.
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
|
||||
@@ -68,6 +69,20 @@ class UnionGraph:
|
||||
def shard(self, shard_id: str) -> ShardAdapter | None:
|
||||
return next((s for s in self._shards if s.shard_id == shard_id), None)
|
||||
|
||||
@property
|
||||
def shards(self) -> tuple[ShardAdapter, ...]:
|
||||
return tuple(self._shards)
|
||||
|
||||
def iter_pages(self) -> Iterator[Page]:
|
||||
"""Every page across attached shards, raw (per-shard, not chorus-collapsed). The
|
||||
enumeration substrate for derived views — BackLinks, AllPages, SiteMap (§8.4)."""
|
||||
for shard in self._shards:
|
||||
for key in shard.keys():
|
||||
try:
|
||||
yield shard.read(key)
|
||||
except KeyError:
|
||||
continue
|
||||
|
||||
def _read_all(self, key: str) -> list[Page]:
|
||||
pages: list[Page] = []
|
||||
for shard in self._shards:
|
||||
|
||||
33
src/shard_wiki/views/__init__.py
Normal file
33
src/shard_wiki/views/__init__.py
Normal file
@@ -0,0 +1,33 @@
|
||||
"""views/ — derived, recomputable, provenance-carrying read views over the union (§8.4).
|
||||
|
||||
All views here are *derived tier*: pure functions of the attached shards plus the coordination-log
|
||||
fold, storing nothing canonical (SHARD-WP-0011 makes them incrementally maintainable). Presentation
|
||||
stays out of core (L6) — these produce models, never rendered output. Per the dependency rule this
|
||||
package imports down (union/model/coordination/provenance) and is imported only by the orchestrator.
|
||||
"""
|
||||
|
||||
from shard_wiki.views.allpages import AllPagesEntry, SiteMapNode, all_pages, site_map
|
||||
from shard_wiki.views.backlinks import BackLink, BackLinksIndex, build_backlinks
|
||||
from shard_wiki.views.links import (
|
||||
ResolvedLink,
|
||||
WikiLink,
|
||||
extract_links,
|
||||
resolve_links,
|
||||
)
|
||||
from shard_wiki.views.recentchanges import ChangeEntry, recent_changes
|
||||
|
||||
__all__ = [
|
||||
"WikiLink",
|
||||
"ResolvedLink",
|
||||
"extract_links",
|
||||
"resolve_links",
|
||||
"BackLink",
|
||||
"BackLinksIndex",
|
||||
"build_backlinks",
|
||||
"ChangeEntry",
|
||||
"recent_changes",
|
||||
"AllPagesEntry",
|
||||
"SiteMapNode",
|
||||
"all_pages",
|
||||
"site_map",
|
||||
]
|
||||
131
src/shard_wiki/views/allpages.py
Normal file
131
src/shard_wiki/views/allpages.py
Normal file
@@ -0,0 +1,131 @@
|
||||
"""AllPages + SiteMap — enumeration views over the union (SHARD-WP-0010 T4).
|
||||
|
||||
**AllPages** lists the union's distinct pages, collapsing identities that name the same page: a
|
||||
*chorus* (same key across shards) and *equivalence-bound* identities (decision-log bindings) fold
|
||||
into one entry, with divergence noted when the members' bodies differ (union without erasure — the
|
||||
collapse is acknowledged, never silent). **SiteMap** is the namespace tree built from page
|
||||
placements (paths), spanning shards.
|
||||
|
||||
Both are derived/recomputable and presentation-free (the tree is a model, not rendered HTML).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.model import Identity, Page
|
||||
from shard_wiki.union import UnionGraph
|
||||
|
||||
__all__ = ["AllPagesEntry", "SiteMapNode", "all_pages", "site_map"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AllPagesEntry:
|
||||
"""One union page: its representative ``name``, the ``members`` collapsed into it, and whether
|
||||
those members' bodies ``diverge`` (a chorus with differing content)."""
|
||||
|
||||
name: str
|
||||
members: tuple[Identity, ...]
|
||||
diverges: bool
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SiteMapNode:
|
||||
"""A namespace node: its path ``name``, child namespaces, and pages directly under it."""
|
||||
|
||||
name: str
|
||||
children: tuple[SiteMapNode, ...]
|
||||
pages: tuple[Identity, ...]
|
||||
|
||||
|
||||
class _UnionFind:
|
||||
def __init__(self) -> None:
|
||||
self._parent: dict[str, str] = {}
|
||||
|
||||
def add(self, x: str) -> None:
|
||||
self._parent.setdefault(x, x)
|
||||
|
||||
def find(self, x: str) -> str:
|
||||
self.add(x)
|
||||
root = x
|
||||
while self._parent[root] != root:
|
||||
root = self._parent[root]
|
||||
while self._parent[x] != root:
|
||||
self._parent[x], x = root, self._parent[x]
|
||||
return root
|
||||
|
||||
def union(self, a: str, b: str) -> None:
|
||||
self.add(a)
|
||||
self.add(b)
|
||||
ra, rb = self.find(a), self.find(b)
|
||||
if ra != rb:
|
||||
self._parent[max(ra, rb)] = min(ra, rb)
|
||||
|
||||
|
||||
def all_pages(
|
||||
union: UnionGraph,
|
||||
equivalence_groups: tuple[frozenset[str], ...] | None = None,
|
||||
) -> tuple[AllPagesEntry, ...]:
|
||||
"""Enumerate the union's distinct pages, collapsing chorus + equivalence-bound members.
|
||||
|
||||
``equivalence_groups`` (string identities, decision-log form) overrides the source of
|
||||
equivalence — the orchestrator passes the maintained index's groups (SHARD-WP-0011 T4); the
|
||||
default falls back to the decision-log fold, so direct callers are unaffected.
|
||||
"""
|
||||
pages: dict[str, Page] = {}
|
||||
by_key: dict[str, list[str]] = {}
|
||||
for page in union.iter_pages():
|
||||
ident = str(page.identity)
|
||||
pages[ident] = page
|
||||
by_key.setdefault(page.identity.key, []).append(ident)
|
||||
|
||||
uf = _UnionFind()
|
||||
for ident in pages:
|
||||
uf.add(ident)
|
||||
for idents in by_key.values(): # same key across shards → chorus
|
||||
for other in idents[1:]:
|
||||
uf.union(idents[0], other)
|
||||
if equivalence_groups is None:
|
||||
equivalence_groups = union.log.fold(union.space).equivalence_groups
|
||||
for group in equivalence_groups: # curator bindings (+ maintained content edges)
|
||||
present = [m for m in group if m in pages]
|
||||
for other in present[1:]:
|
||||
uf.union(present[0], other)
|
||||
|
||||
groups: dict[str, list[str]] = {}
|
||||
for ident in pages:
|
||||
groups.setdefault(uf.find(ident), []).append(ident)
|
||||
|
||||
entries: list[AllPagesEntry] = []
|
||||
for members in groups.values():
|
||||
member_pages = [pages[m] for m in members]
|
||||
identities = tuple(p.identity for p in member_pages)
|
||||
name = min(p.identity.key for p in member_pages)
|
||||
diverges = len({p.body for p in member_pages}) > 1
|
||||
entries.append(AllPagesEntry(name=name, members=identities, diverges=diverges))
|
||||
return tuple(sorted(entries, key=lambda e: e.name))
|
||||
|
||||
|
||||
def _segments(page: Page) -> list[str]:
|
||||
path = page.placements[0].path if page.placements else page.identity.key
|
||||
if path.endswith(".md"):
|
||||
path = path[:-3]
|
||||
return [seg for seg in path.split("/") if seg]
|
||||
|
||||
|
||||
def site_map(union: UnionGraph) -> SiteMapNode:
|
||||
"""The union namespace tree from page placements (directories nest; pages sit at their dir)."""
|
||||
root: dict = {"children": {}, "pages": []}
|
||||
for page in union.iter_pages():
|
||||
segments = _segments(page)
|
||||
node = root
|
||||
for seg in segments[:-1]: # directory segments build the nesting
|
||||
node = node["children"].setdefault(seg, {"children": {}, "pages": []})
|
||||
node["pages"].append(page.identity)
|
||||
return _freeze("", root)
|
||||
|
||||
|
||||
def _freeze(name: str, node: dict) -> SiteMapNode:
|
||||
children = tuple(_freeze(k, v) for k, v in sorted(node["children"].items()))
|
||||
pages = tuple(sorted(node["pages"], key=str))
|
||||
return SiteMapNode(name=name, children=children, pages=pages)
|
||||
65
src/shard_wiki/views/backlinks.py
Normal file
65
src/shard_wiki/views/backlinks.py
Normal file
@@ -0,0 +1,65 @@
|
||||
"""BackLinks — the strongest core derived view (SHARD-WP-0010 T2; UC-18).
|
||||
|
||||
For any page name, the set of pages that link to it. Built by extracting wikilinks (T1) from every
|
||||
page across the attached shards and resolving each through the union: only **resolved** links
|
||||
create a backlink (a red-link points at nothing, so it contributes none). Entries carry their
|
||||
**source provenance** (the linking page's identity / shard). Keying by the resolved *name* means a
|
||||
chorus target aggregates the backlinks of all its members into one bucket (union without erasure).
|
||||
|
||||
Derived/recomputable — stores nothing canonical; SHARD-WP-0011 maintains it incrementally.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.model import Identity
|
||||
from shard_wiki.union import UnionGraph
|
||||
from shard_wiki.views.links import resolve_links
|
||||
|
||||
__all__ = ["BackLink", "BackLinksIndex", "build_backlinks"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class BackLink:
|
||||
"""One inbound link: ``source`` (the linking page) references ``target_name``."""
|
||||
|
||||
source: Identity
|
||||
target_name: str
|
||||
|
||||
@property
|
||||
def source_shard(self) -> str:
|
||||
return self.source.shard
|
||||
|
||||
|
||||
class BackLinksIndex:
|
||||
"""An immutable name → inbound-links index over the union link graph."""
|
||||
|
||||
def __init__(self, edges: Mapping[str, tuple[BackLink, ...]]) -> None:
|
||||
self._edges = dict(edges)
|
||||
|
||||
def to(self, name: str) -> tuple[BackLink, ...]:
|
||||
"""The backlinks pointing at ``name`` (empty if none)."""
|
||||
return self._edges.get(name, ())
|
||||
|
||||
def sources(self, name: str) -> frozenset[Identity]:
|
||||
"""Just the identities linking to ``name`` — convenient for set assertions."""
|
||||
return frozenset(bl.source for bl in self.to(name))
|
||||
|
||||
def names(self) -> frozenset[str]:
|
||||
return frozenset(self._edges)
|
||||
|
||||
|
||||
def build_backlinks(union: UnionGraph, *, camelcase: bool = False) -> BackLinksIndex:
|
||||
"""Scan every union page's links and index the resolved ones by target name."""
|
||||
edges: dict[str, set[BackLink]] = {}
|
||||
for page in union.iter_pages():
|
||||
for resolved in resolve_links(union, page.body, camelcase=camelcase):
|
||||
if resolved.is_red_link:
|
||||
continue # red-links don't create backlinks
|
||||
backlink = BackLink(source=page.identity, target_name=resolved.link.target)
|
||||
edges.setdefault(resolved.link.target, set()).add(backlink)
|
||||
return BackLinksIndex(
|
||||
{name: tuple(sorted(links, key=lambda bl: str(bl.source))) for name, links in edges.items()}
|
||||
)
|
||||
91
src/shard_wiki/views/links.py
Normal file
91
src/shard_wiki/views/links.py
Normal file
@@ -0,0 +1,91 @@
|
||||
"""Wikilink + red-link model (SHARD-WP-0010 T1; FederationRequirements ADR-06).
|
||||
|
||||
A CommonMark *wikilink extension*: ``[[Target]]`` and ``[[Target|label]]`` are extracted from a
|
||||
page body and each target is resolved through the union (ADR-01). A target that resolves is a
|
||||
**link**; one that does not is a **red-link** — a createable hole (UC-23), never a dropped
|
||||
reference (union without erasure). CamelCase auto-linking (``WikiWord``) is **off by default** and
|
||||
opt-in per space, since bare CamelCase is noisy and policy-laden.
|
||||
|
||||
The link *model and resolution* are core; turning a :class:`ResolvedLink` into an ``<a>`` (or a
|
||||
red anchor) is L6 presentation and lives outside this package. Link spans are byte/char offsets in
|
||||
the body so a later layer can address them precisely.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from shard_wiki.union import Resolution, UnionGraph
|
||||
|
||||
__all__ = ["WikiLink", "ResolvedLink", "extract_links", "resolve_links"]
|
||||
|
||||
_WIKILINK_RE = re.compile(r"\[\[\s*([^\]|]+?)\s*(?:\|\s*([^\]]+?)\s*)?\]\]")
|
||||
# A WikiWord: ≥2 capitalized alphanumeric segments run together (e.g. FrontPage, WikiWord).
|
||||
_CAMELCASE_RE = re.compile(r"\b([A-Z][a-z0-9]+(?:[A-Z][a-z0-9]+)+)\b")
|
||||
_FENCED_RE = re.compile(r"```.*?```", re.DOTALL)
|
||||
_INLINE_CODE_RE = re.compile(r"`[^`\n]*`")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class WikiLink:
|
||||
"""One extracted reference. ``target`` is the resolve key; ``label`` is the display text (or
|
||||
None to use the target); ``span`` is the ``[start, end)`` offset of the whole token in the body;
|
||||
``auto`` marks a CamelCase auto-link (vs an explicit ``[[...]]``)."""
|
||||
|
||||
target: str
|
||||
label: str | None
|
||||
span: tuple[int, int]
|
||||
auto: bool = False
|
||||
|
||||
@property
|
||||
def text(self) -> str:
|
||||
return self.label or self.target
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ResolvedLink:
|
||||
"""A :class:`WikiLink` paired with its union :class:`Resolution` (the link's truth status)."""
|
||||
|
||||
link: WikiLink
|
||||
resolution: Resolution
|
||||
|
||||
@property
|
||||
def is_red_link(self) -> bool:
|
||||
return self.resolution.is_red_link
|
||||
|
||||
|
||||
def _mask(body: str, pattern: re.Pattern[str]) -> str:
|
||||
"""Blank out ``pattern`` matches with equal-length spaces so later scans skip them while every
|
||||
surviving match keeps its true offset."""
|
||||
return pattern.sub(lambda m: " " * len(m.group(0)), body)
|
||||
|
||||
|
||||
def extract_links(body: str, *, camelcase: bool = False) -> tuple[WikiLink, ...]:
|
||||
"""Extract wikilinks from ``body`` in document order, skipping fenced/inline code.
|
||||
|
||||
With ``camelcase=True`` (per-space opt-in), bare ``WikiWord`` tokens outside code and outside
|
||||
existing ``[[...]]`` also become links.
|
||||
"""
|
||||
scan = _mask(_mask(body, _FENCED_RE), _INLINE_CODE_RE)
|
||||
links: list[WikiLink] = []
|
||||
for m in _WIKILINK_RE.finditer(scan):
|
||||
links.append(WikiLink(target=m.group(1).strip(), label=m.group(2), span=m.span()))
|
||||
|
||||
if camelcase:
|
||||
# Mask explicit-link spans too, so a CamelCase target inside [[...]] isn't double-counted.
|
||||
cc_scan = _mask(scan, _WIKILINK_RE)
|
||||
for m in _CAMELCASE_RE.finditer(cc_scan):
|
||||
links.append(WikiLink(target=m.group(1), label=None, span=m.span(), auto=True))
|
||||
|
||||
return tuple(sorted(links, key=lambda link: link.span[0]))
|
||||
|
||||
|
||||
def resolve_links(
|
||||
union: UnionGraph, body: str, *, camelcase: bool = False
|
||||
) -> tuple[ResolvedLink, ...]:
|
||||
"""Extract and resolve every link in ``body`` against ``union`` (link vs red-link, ADR-01)."""
|
||||
return tuple(
|
||||
ResolvedLink(link, union.resolve(link.target))
|
||||
for link in extract_links(body, camelcase=camelcase)
|
||||
)
|
||||
108
src/shard_wiki/views/recentchanges.py
Normal file
108
src/shard_wiki/views/recentchanges.py
Normal file
@@ -0,0 +1,108 @@
|
||||
"""RecentChanges — a merged change feed over the union (SHARD-WP-0010 T3; UC-17).
|
||||
|
||||
Two streams, one ordered feed (newest-first):
|
||||
|
||||
* the **coordination journal** — overlay/alias/fork/merge/binding decisions from the decision log,
|
||||
each carrying its actor and the decision payload; and
|
||||
* **shard change signals** — a page's current revision (folder mtime / ``source_rev``), i.e. the
|
||||
backend's own "this changed" evidence.
|
||||
|
||||
Every entry carries provenance: which shard the edit came from, or that it was a coordination
|
||||
decision (and by whom). Derived/recomputable — `notify`-driven streaming is a later binding.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
|
||||
from shard_wiki.coordination import DecisionLog, EventType
|
||||
from shard_wiki.union import UnionGraph
|
||||
|
||||
__all__ = ["ChangeEntry", "recent_changes"]
|
||||
|
||||
_COORDINATION = "coordination"
|
||||
|
||||
# How each journal event names the thing it touched + a human kind label.
|
||||
_EVENT_KIND = {
|
||||
EventType.ALIAS_SET: "alias",
|
||||
EventType.OVERLAY_CREATED: "overlay",
|
||||
EventType.MERGE_DECIDED: "merge",
|
||||
EventType.PAGE_FORKED: "fork",
|
||||
EventType.BINDING_MADE: "binding",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChangeEntry:
|
||||
"""One change in the feed. ``source`` is the shard id (a shard edit) or ``"coordination"``."""
|
||||
|
||||
when: datetime
|
||||
kind: str
|
||||
ref: str
|
||||
source: str
|
||||
actor: str | None = None
|
||||
detail: Mapping[str, object] = field(default_factory=dict)
|
||||
|
||||
|
||||
def _event_ref(event_type: EventType, payload: Mapping[str, object]) -> str:
|
||||
if event_type is EventType.ALIAS_SET:
|
||||
return str(payload.get("alias", ""))
|
||||
if event_type is EventType.OVERLAY_CREATED:
|
||||
return f"{payload.get('target_shard')}:{payload.get('target_key')}"
|
||||
if event_type is EventType.PAGE_FORKED:
|
||||
return f"{payload.get('source')}→{payload.get('fork')}"
|
||||
if event_type is EventType.BINDING_MADE:
|
||||
return ", ".join(str(m) for m in payload.get("members", ()))
|
||||
return str(payload.get("overlay_id", "")) # MERGE_DECIDED
|
||||
|
||||
|
||||
def recent_changes(
|
||||
union: UnionGraph,
|
||||
log: DecisionLog,
|
||||
space: str,
|
||||
*,
|
||||
limit: int | None = None,
|
||||
) -> tuple[ChangeEntry, ...]:
|
||||
"""Merge the coordination journal and shard change signals into one newest-first feed."""
|
||||
entries: list[ChangeEntry] = []
|
||||
|
||||
for event in log.events(space):
|
||||
entries.append(
|
||||
ChangeEntry(
|
||||
when=event.timestamp,
|
||||
kind=_EVENT_KIND.get(event.type, event.type.value),
|
||||
ref=_event_ref(event.type, event.payload),
|
||||
source=_COORDINATION,
|
||||
actor=event.actor,
|
||||
detail=dict(event.payload),
|
||||
)
|
||||
)
|
||||
|
||||
for page in union.iter_pages():
|
||||
rev = page.envelope.source_rev
|
||||
when = _parse_rev(rev)
|
||||
if when is None:
|
||||
continue # shard offers no change signal for this page — skip gracefully
|
||||
entries.append(
|
||||
ChangeEntry(
|
||||
when=when,
|
||||
kind="edit",
|
||||
ref=str(page.identity),
|
||||
source=page.identity.shard,
|
||||
detail={"source_rev": rev},
|
||||
)
|
||||
)
|
||||
|
||||
entries.sort(key=lambda e: e.when, reverse=True)
|
||||
return tuple(entries if limit is None else entries[:limit])
|
||||
|
||||
|
||||
def _parse_rev(rev: str | None) -> datetime | None:
|
||||
if rev is None:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromisoformat(rev)
|
||||
except ValueError:
|
||||
return None # non-temporal revision token (e.g. a content hash) — no feed timestamp
|
||||
120
tests/test_append_authority.py
Normal file
120
tests/test_append_authority.py
Normal file
@@ -0,0 +1,120 @@
|
||||
"""Tests for the per-space append authority / lease (SHARD-WP-0009 T2).
|
||||
|
||||
A single append authority per space serializes appends into a total order; non-holders forward
|
||||
intents to the holder; the lease is time-bounded and re-grantable (HA hand-off); a stale ex-holder
|
||||
cannot fork the log.
|
||||
"""
|
||||
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.coordination import (
|
||||
AppendAuthority,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
InMemoryEventStore,
|
||||
LeaseHeld,
|
||||
LeaseRegistry,
|
||||
)
|
||||
|
||||
|
||||
class FakeClock:
|
||||
def __init__(self):
|
||||
self.now = datetime(2026, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
def __call__(self):
|
||||
return self.now
|
||||
|
||||
def advance(self, seconds):
|
||||
self.now += timedelta(seconds=seconds)
|
||||
|
||||
|
||||
def test_only_one_node_holds_a_space_at_a_time():
|
||||
reg = LeaseRegistry()
|
||||
a = AppendAuthority("A", InMemoryEventStore(), reg)
|
||||
b = AppendAuthority("B", InMemoryEventStore(), reg)
|
||||
a.acquire("s")
|
||||
with pytest.raises(LeaseHeld):
|
||||
b.acquire("s") # B is refused while A's lease is valid
|
||||
|
||||
|
||||
def test_concurrent_appends_serialize_into_one_total_order():
|
||||
reg = LeaseRegistry()
|
||||
store = InMemoryEventStore()
|
||||
a = AppendAuthority("A", store, reg)
|
||||
b = AppendAuthority("B", store, reg)
|
||||
a.acquire("s")
|
||||
# B is a non-holder: its append forwards to A, the holder. Interleave A and B writers.
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"}) # forwarded
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "3", "target": "x:3"})
|
||||
seqs = [e.seq for e in store.events("s")]
|
||||
aliases = [e.payload["alias"] for e in store.events("s")]
|
||||
assert seqs == [0, 1, 2] # contiguous total order despite two writers
|
||||
assert aliases == ["1", "2", "3"]
|
||||
|
||||
|
||||
def test_non_holder_forwards_rather_than_writing_directly():
|
||||
reg = LeaseRegistry()
|
||||
store = InMemoryEventStore()
|
||||
a = AppendAuthority("A", store, reg)
|
||||
b = AppendAuthority("B", store, reg)
|
||||
a.acquire("s")
|
||||
assert not b.holds("s")
|
||||
b.append("s", EventType.ALIAS_SET, {"alias": "fwd", "target": "x:1"})
|
||||
# The write landed on the shared store under A's authority, in one stream.
|
||||
assert [e.payload["alias"] for e in store.events("s")] == ["fwd"]
|
||||
|
||||
|
||||
def test_lease_handoff_resumes_from_head():
|
||||
clock = FakeClock()
|
||||
reg = LeaseRegistry(clock=clock)
|
||||
store = InMemoryEventStore()
|
||||
a = AppendAuthority("A", store, reg, ttl_seconds=10)
|
||||
b = AppendAuthority("B", store, reg, ttl_seconds=10)
|
||||
a.acquire("s")
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||
clock.advance(20) # A's lease expires (A "dies")
|
||||
b.acquire("s") # re-grantable: B takes over
|
||||
b.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
|
||||
assert [e.seq for e in store.events("s")] == [0, 1, 2] # contiguous across hand-off
|
||||
|
||||
|
||||
def test_stale_ex_holder_cannot_fork_the_log():
|
||||
clock = FakeClock()
|
||||
reg = LeaseRegistry(clock=clock)
|
||||
store = InMemoryEventStore()
|
||||
a = AppendAuthority("A", store, reg, ttl_seconds=10)
|
||||
b = AppendAuthority("B", store, reg, ttl_seconds=10)
|
||||
a.acquire("s")
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "0", "target": "x:0"})
|
||||
clock.advance(20)
|
||||
b.acquire("s") # B is now the holder; A's lease is stale
|
||||
b.append("s", EventType.ALIAS_SET, {"alias": "1", "target": "x:1"})
|
||||
# A still thinks it can write, but it's no longer the holder: its intent forwards to B.
|
||||
assert not a.holds("s")
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "2", "target": "x:2"})
|
||||
aliases = [e.payload["alias"] for e in store.events("s")]
|
||||
assert aliases == ["0", "1", "2"] # one stream, no fork
|
||||
|
||||
|
||||
def test_authority_over_git_store_keeps_total_order(tmp_path):
|
||||
reg = LeaseRegistry()
|
||||
store = GitEventStore(tmp_path / "coord")
|
||||
a = AppendAuthority("A", store, reg)
|
||||
b = AppendAuthority("B", store, reg)
|
||||
a.acquire("s")
|
||||
a.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||
b.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"}) # forwarded
|
||||
assert [e.seq for e in store.events("s")] == [0, 1]
|
||||
|
||||
|
||||
def test_unleased_space_self_acquires_on_append():
|
||||
reg = LeaseRegistry()
|
||||
store = InMemoryEventStore()
|
||||
a = AppendAuthority("A", store, reg)
|
||||
a.append("s", EventType.ALIAS_SET, {"alias": "x", "target": "y:1"}) # no explicit acquire
|
||||
assert a.holds("s")
|
||||
assert len(store.events("s")) == 1
|
||||
74
tests/test_coordination_migration.py
Normal file
74
tests/test_coordination_migration.py
Normal file
@@ -0,0 +1,74 @@
|
||||
"""Migration + wiring of the git coordination backend (SHARD-WP-0009 T4)."""
|
||||
|
||||
from shard_wiki.coordination import (
|
||||
DecisionLog,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
InMemoryEventStore,
|
||||
export_jsonl,
|
||||
import_jsonl,
|
||||
migrate_space,
|
||||
)
|
||||
from shard_wiki.space import InformationSpace
|
||||
|
||||
|
||||
def test_information_space_git_backed_uses_git_log(tmp_path):
|
||||
space = InformationSpace.git_backed("space-1", tmp_path / "coord")
|
||||
assert isinstance(space.log._store, GitEventStore)
|
||||
space.alias("Home", "shardA:Index")
|
||||
# Read-your-writes through the orchestrator's git-backed log.
|
||||
assert space.log.fold("space-1").resolve_alias("Home") == "shardA:Index"
|
||||
|
||||
|
||||
def test_default_information_space_stays_in_memory():
|
||||
space = InformationSpace("space-1")
|
||||
assert isinstance(space.log._store, InMemoryEventStore)
|
||||
|
||||
|
||||
def test_migrate_space_preserves_order_and_provenance(tmp_path):
|
||||
source = InMemoryEventStore()
|
||||
e0 = source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}, actor="ana")
|
||||
source.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]}, actor="ben")
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
n = migrate_space(source, "s", dest)
|
||||
assert n == 2
|
||||
|
||||
migrated = dest.events("s")
|
||||
assert [e.seq for e in migrated] == [0, 1]
|
||||
# Provenance preserved verbatim — actor and timestamp survive the move (no restamping).
|
||||
assert migrated[0].actor == "ana"
|
||||
assert migrated[1].actor == "ben"
|
||||
assert migrated[0].timestamp == e0.timestamp
|
||||
|
||||
|
||||
def test_migration_yields_identical_fold(tmp_path):
|
||||
source = DecisionLog(InMemoryEventStore())
|
||||
for typ, payload in [
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"}),
|
||||
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "x:2"}),
|
||||
]:
|
||||
source.append("s", typ, payload)
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
migrate_space(source._store, "s", dest)
|
||||
after = DecisionLog(dest)
|
||||
assert after.fold("s").aliases == source.fold("s").aliases
|
||||
assert after.fold("s").equivalence_groups == source.fold("s").equivalence_groups
|
||||
|
||||
|
||||
def test_jsonl_round_trip_into_git(tmp_path):
|
||||
source = InMemoryEventStore()
|
||||
source.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "x:1"})
|
||||
source.append("s", EventType.PAGE_FORKED, {"source": "p", "fork": "q"})
|
||||
|
||||
path = tmp_path / "log.jsonl"
|
||||
assert export_jsonl(source.events("s"), path) == 2
|
||||
|
||||
dest = GitEventStore(tmp_path / "coord")
|
||||
assert import_jsonl(path, dest) == 2
|
||||
state = DecisionLog(dest).fold("s")
|
||||
assert state.resolve_alias("Home") == "x:1"
|
||||
assert state.equivalent_to("p") == frozenset({"p", "q"})
|
||||
83
tests/test_cross_process_fold.py
Normal file
83
tests/test_cross_process_fold.py
Normal file
@@ -0,0 +1,83 @@
|
||||
"""Cross-process read-your-writes over the git log + fold parity (SHARD-WP-0009 T3).
|
||||
|
||||
The git backend's value over the in-memory double is that the totally ordered log is durable and
|
||||
shared: a write by one process/handle is immediately visible to another opening the same ref, and
|
||||
the derived fold is identical to the in-memory fold of the same event sequence (derived = f(log)).
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
from shard_wiki.coordination import (
|
||||
DecisionLog,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
InMemoryEventStore,
|
||||
)
|
||||
|
||||
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
||||
|
||||
|
||||
def test_new_handle_sees_prior_writes(tmp_path):
|
||||
repo = tmp_path / "coord"
|
||||
writer = DecisionLog(GitEventStore(repo))
|
||||
writer.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
|
||||
writer.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||
# A second, independent handle on the same repo — read-your-writes across handles.
|
||||
reader = DecisionLog(GitEventStore(repo))
|
||||
assert [e.seq for e in reader.events("s")] == [0, 1]
|
||||
assert reader.fold("s").resolve_alias("Home") == "shardA:Index"
|
||||
|
||||
|
||||
def test_append_in_separate_process_is_visible(tmp_path):
|
||||
repo = tmp_path / "coord"
|
||||
# Seed from this process so the repo exists.
|
||||
DecisionLog(GitEventStore(repo)).append(
|
||||
"s", EventType.ALIAS_SET, {"alias": "A", "target": "x:1"}
|
||||
)
|
||||
child = textwrap.dedent(
|
||||
f"""
|
||||
from shard_wiki.coordination import DecisionLog, EventType, GitEventStore
|
||||
log = DecisionLog(GitEventStore({str(repo)!r}))
|
||||
log.append("s", EventType.ALIAS_SET, {{"alias": "B", "target": "x:2"}})
|
||||
"""
|
||||
)
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", child],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={"PYTHONPATH": _SRC, "PATH": os.environ.get("PATH", "")},
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
# This process, with a fresh handle, sees the child's append in order.
|
||||
reader = DecisionLog(GitEventStore(repo))
|
||||
assert [e.payload["alias"] for e in reader.events("s")] == ["A", "B"]
|
||||
assert [e.seq for e in reader.events("s")] == [0, 1]
|
||||
|
||||
|
||||
def test_cross_process_fold_equals_in_memory_fold(tmp_path):
|
||||
sequence = [
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
|
||||
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||
(EventType.PAGE_FORKED, {"source": "p", "fork": "q"}),
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
|
||||
]
|
||||
mem = DecisionLog(InMemoryEventStore())
|
||||
for typ, payload in sequence:
|
||||
mem.append("s", typ, payload)
|
||||
|
||||
repo = tmp_path / "coord"
|
||||
DecisionLog(GitEventStore(repo)) # init repo
|
||||
for typ, payload in sequence:
|
||||
# Each append from a fresh handle to simulate distinct writers over time.
|
||||
DecisionLog(GitEventStore(repo)).append("s", typ, payload)
|
||||
|
||||
git_state = DecisionLog(GitEventStore(repo)).fold("s")
|
||||
mem_state = mem.fold("s")
|
||||
assert git_state.aliases == mem_state.aliases
|
||||
assert git_state.equivalence_groups == mem_state.equivalence_groups
|
||||
assert git_state.equivalent_to("a") == frozenset({"a", "b", "c"})
|
||||
61
tests/test_engine_activation.py
Normal file
61
tests/test_engine_activation.py
Normal file
@@ -0,0 +1,61 @@
|
||||
"""Tests for per-shard extension activation (SHARD-WP-0014 T3, ADR-0001)."""
|
||||
|
||||
from shard_wiki.engine import (
|
||||
ActivationContext,
|
||||
ActivationResolver,
|
||||
StaticProvider,
|
||||
feature_control_provider,
|
||||
)
|
||||
|
||||
CANDIDATES = ["ext.overlay", "ext.views", "ext.struct", "ext.compute"]
|
||||
|
||||
|
||||
def test_static_provider_default_off():
|
||||
r = ActivationResolver(StaticProvider()) # nothing enabled
|
||||
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == set()
|
||||
|
||||
|
||||
def test_static_provider_global_flags():
|
||||
r = ActivationResolver(StaticProvider(flags={"ext.overlay": True, "ext.views": True}))
|
||||
assert r.active_extensions(CANDIDATES, ActivationContext("s")) == {"ext.overlay", "ext.views"}
|
||||
|
||||
|
||||
def test_per_shard_scoping_overrides_global():
|
||||
provider = StaticProvider(
|
||||
flags={"ext.views": True},
|
||||
per_shard={"engB": {"ext.struct": True, "ext.views": False}},
|
||||
)
|
||||
r = ActivationResolver(provider)
|
||||
assert r.active_extensions(CANDIDATES, ActivationContext("engA")) == {"ext.views"}
|
||||
assert r.active_extensions(CANDIDATES, ActivationContext("engB")) == {"ext.struct"}
|
||||
|
||||
|
||||
def test_context_carries_tenant():
|
||||
captured = {}
|
||||
|
||||
class Spy(StaticProvider):
|
||||
def is_active(self, feature_key, context):
|
||||
captured.update(context)
|
||||
return super().is_active(feature_key, context)
|
||||
|
||||
ActivationResolver(Spy(flags={"ext.views": True})).active_extensions(
|
||||
["ext.views"], ActivationContext("s1", tenant_id="acme")
|
||||
)
|
||||
assert captured["shard_id"] == "s1" and captured["tenant_id"] == "acme"
|
||||
|
||||
|
||||
def test_activation_profile_returns_config_for_active():
|
||||
provider = StaticProvider(
|
||||
flags={"ext.struct": True},
|
||||
configs={"ext.struct": {"max_fields": 50}},
|
||||
)
|
||||
profile = ActivationResolver(provider).activation_profile(CANDIDATES, ActivationContext("s"))
|
||||
assert profile == {"ext.struct": {"max_fields": 50}}
|
||||
|
||||
|
||||
def test_feature_control_provider_degrades_gracefully():
|
||||
# feature_control_sdk is not a dependency of shard-wiki: when absent the factory returns
|
||||
# None (standalone path stays dependency-free, ADR-0001); when present it yields a usable
|
||||
# ActivationProvider. Either way it must not raise.
|
||||
provider = feature_control_provider()
|
||||
assert provider is None or hasattr(provider, "is_active")
|
||||
80
tests/test_engine_adapter.py
Normal file
80
tests/test_engine_adapter.py
Normal file
@@ -0,0 +1,80 @@
|
||||
"""Tests for EngineShardAdapter (SHARD-WP-0014 T5): engine as a canonical-mode shard."""
|
||||
|
||||
from shard_wiki import InformationSpace
|
||||
from shard_wiki.adapters import assert_conformant
|
||||
from shard_wiki.engine import (
|
||||
Extension,
|
||||
ExtensionRuntime,
|
||||
Hook,
|
||||
ProfileContribution,
|
||||
StaticProvider,
|
||||
build_engine_shard,
|
||||
)
|
||||
from shard_wiki.engine.activation import ActivationContext
|
||||
from shard_wiki.model import Verb
|
||||
|
||||
|
||||
class StructProfileExt(Extension):
|
||||
"""Profile-only extension (no body transform → write stays content-preserving)."""
|
||||
|
||||
id = "ext.struct"
|
||||
|
||||
def hooks(self):
|
||||
return {
|
||||
Hook.ON_PROFILE: lambda p, c: ProfileContribution(
|
||||
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
def _runtime():
|
||||
rt = ExtensionRuntime()
|
||||
rt.register(StructProfileExt())
|
||||
return rt
|
||||
|
||||
|
||||
def test_kernel_only_engine_shard_is_conformant():
|
||||
shard = build_engine_shard("eng", ExtensionRuntime(), activate=set())
|
||||
shard.write("Home", "hi")
|
||||
report = assert_conformant(shard) # read + positive write probe
|
||||
assert report.ok
|
||||
assert shard.profile().supports(Verb.WRITE)
|
||||
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||
|
||||
|
||||
def test_profile_reflects_activated_extensions():
|
||||
off = build_engine_shard("a", _runtime(), activate=set())
|
||||
on = build_engine_shard("b", _runtime(), activate={"ext.struct"})
|
||||
assert not off.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||
assert on.profile().supports(Verb.STRUCTURED_PAYLOAD) # E-5
|
||||
assert_conformant(on)
|
||||
|
||||
|
||||
def test_activation_via_provider():
|
||||
provider = StaticProvider(flags={"ext.struct": True})
|
||||
shard = build_engine_shard("c", _runtime(), provider=provider, context=ActivationContext("c"))
|
||||
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD)
|
||||
|
||||
|
||||
def test_attach_resolve_edit_through_engine_shard(tmp_path):
|
||||
space = InformationSpace("team")
|
||||
space.attach(build_engine_shard("wikiE", ExtensionRuntime(), activate=set()))
|
||||
# seed a page directly via the shard, then read + edit through the orchestrator
|
||||
space.union.shard("wikiE").write("Home", "v1")
|
||||
assert space.read("Home").body == "v1"
|
||||
result = space.edit("Home", "v2") # overlay -> apply-under-drift -> write-through
|
||||
assert result.status.value == "applied"
|
||||
assert space.read("Home").body == "v2"
|
||||
|
||||
|
||||
def test_on_write_transform_runs():
|
||||
class Upper(Extension):
|
||||
id = "ext.upper"
|
||||
def hooks(self):
|
||||
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
|
||||
|
||||
rt = ExtensionRuntime()
|
||||
rt.register(Upper())
|
||||
shard = build_engine_shard("u", rt, activate={"ext.upper"})
|
||||
shard.write("P", "quiet")
|
||||
assert shard.read("P").body == "QUIET" # extension transformed the write
|
||||
109
tests/test_engine_extension.py
Normal file
109
tests/test_engine_extension.py
Normal file
@@ -0,0 +1,109 @@
|
||||
"""Tests for the typed-extension runtime (SHARD-WP-0014 T2)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.engine import Extension, ExtensionError, ExtensionRuntime, Hook
|
||||
|
||||
|
||||
class Upper(Extension):
|
||||
id = "ext.upper"
|
||||
declares_types = ("upper",)
|
||||
|
||||
def hooks(self):
|
||||
return {Hook.ON_WRITE: lambda body, ctx: body.upper()}
|
||||
|
||||
|
||||
class Bang(Extension):
|
||||
id = "ext.bang"
|
||||
depends_on = ("ext.upper",)
|
||||
|
||||
def hooks(self):
|
||||
return {Hook.ON_WRITE: lambda body, ctx: body + "!"}
|
||||
|
||||
|
||||
class Profiler(Extension):
|
||||
id = "ext.profiler"
|
||||
|
||||
def hooks(self):
|
||||
return {Hook.ON_PROFILE: lambda payload, ctx: {"structure": "typed"}}
|
||||
|
||||
|
||||
def _runtime(*exts):
|
||||
rt = ExtensionRuntime()
|
||||
for e in exts:
|
||||
rt.register(e)
|
||||
return rt
|
||||
|
||||
|
||||
def test_register_rejects_bad_id_and_noncallable_hook():
|
||||
rt = ExtensionRuntime()
|
||||
class BadId(Extension):
|
||||
id = "upper" # missing 'ext.' prefix
|
||||
with pytest.raises(ExtensionError, match="ext."):
|
||||
rt.register(BadId())
|
||||
|
||||
class Liar(Extension):
|
||||
id = "ext.liar"
|
||||
def hooks(self):
|
||||
return {Hook.ON_WRITE: "not-callable"} # type: ignore[dict-item]
|
||||
with pytest.raises(ExtensionError, match="not callable"):
|
||||
rt.register(Liar())
|
||||
|
||||
|
||||
def test_transform_hook_chains_in_dependency_order():
|
||||
rt = _runtime(Upper(), Bang())
|
||||
active = rt.activate({"ext.bang"}) # pulls ext.upper via depends_on
|
||||
assert set(active.ids) == {"ext.upper", "ext.bang"}
|
||||
# upper (dependency) runs before bang (dependent): "hi" -> "HI" -> "HI!"
|
||||
assert active.handlers(Hook.ON_WRITE) == ("ext.upper", "ext.bang")
|
||||
assert active.dispatch_transform(Hook.ON_WRITE, "hi") == "HI!"
|
||||
|
||||
|
||||
def test_dependency_closure_is_automatic():
|
||||
rt = _runtime(Upper(), Bang())
|
||||
assert set(rt.activate({"ext.bang"}).ids) == {"ext.upper", "ext.bang"}
|
||||
|
||||
|
||||
def test_unknown_extension_rejected():
|
||||
with pytest.raises(ExtensionError, match="unknown"):
|
||||
ExtensionRuntime().activate({"ext.ghost"})
|
||||
|
||||
|
||||
def test_conflict_rejected():
|
||||
class A(Extension):
|
||||
id = "ext.a"
|
||||
conflicts_with = ("ext.b",)
|
||||
class B(Extension):
|
||||
id = "ext.b"
|
||||
with pytest.raises(ExtensionError, match="conflicts"):
|
||||
_runtime(A(), B()).activate({"ext.a", "ext.b"})
|
||||
|
||||
|
||||
def test_type_collision_rejected():
|
||||
class S1(Extension):
|
||||
id = "ext.s1"
|
||||
declares_types = ("record",)
|
||||
class S2(Extension):
|
||||
id = "ext.s2"
|
||||
declares_types = ("record",)
|
||||
with pytest.raises(ExtensionError, match="type collision"):
|
||||
_runtime(S1(), S2()).activate({"ext.s1", "ext.s2"})
|
||||
|
||||
|
||||
def test_collect_hook_gathers_contributions():
|
||||
active = _runtime(Profiler()).activate({"ext.profiler"})
|
||||
assert active.dispatch_collect(Hook.ON_PROFILE) == [{"structure": "typed"}]
|
||||
|
||||
|
||||
def test_wrong_dispatch_kind_errors():
|
||||
active = _runtime(Profiler()).activate({"ext.profiler"})
|
||||
with pytest.raises(ExtensionError, match="not a transform hook"):
|
||||
active.dispatch_transform(Hook.ON_PROFILE, "x")
|
||||
|
||||
|
||||
def test_unmet_dependency_rejected():
|
||||
# Bang depends on ext.upper, but only Bang is registered.
|
||||
rt = ExtensionRuntime()
|
||||
rt.register(Bang())
|
||||
with pytest.raises(ExtensionError, match="unmet dependency|unknown"):
|
||||
rt.activate({"ext.bang"})
|
||||
59
tests/test_engine_kernel.py
Normal file
59
tests/test_engine_kernel.py
Normal file
@@ -0,0 +1,59 @@
|
||||
"""Tests for the engine kernel (SHARD-WP-0014 T1)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.engine import EngineKernel, extract_wikilinks
|
||||
from shard_wiki.model import Identity
|
||||
|
||||
|
||||
def test_write_creates_then_edits_as_history():
|
||||
k = EngineKernel("eng")
|
||||
p1 = k.write("Home", "first")
|
||||
assert p1.identity == Identity("eng", "Home")
|
||||
assert p1.envelope.source_rev == "1"
|
||||
p2 = k.write("Home", "second")
|
||||
assert p2.envelope.source_rev == "2"
|
||||
assert k.read("Home").body == "second" # latest
|
||||
assert [v.body for v in k.history("Home")] == ["first", "second"] # recoverable history
|
||||
|
||||
|
||||
def test_read_missing_raises():
|
||||
k = EngineKernel("eng")
|
||||
with pytest.raises(KeyError):
|
||||
k.read("Nope")
|
||||
|
||||
|
||||
def test_delete_is_recoverable():
|
||||
k = EngineKernel("eng")
|
||||
k.write("Doc", "v1")
|
||||
k.delete("Doc")
|
||||
assert not k.exists("Doc")
|
||||
with pytest.raises(KeyError):
|
||||
k.read("Doc")
|
||||
assert [v.body for v in k.history("Doc")] == ["v1"] # history retained
|
||||
k.write("Doc", "v2") # restore by writing
|
||||
assert k.exists("Doc") and k.read("Doc").body == "v2"
|
||||
|
||||
|
||||
def test_keys_and_current_rev():
|
||||
k = EngineKernel("eng")
|
||||
k.write("A", "a")
|
||||
k.write("B", "b")
|
||||
k.write("A", "a2")
|
||||
assert set(k.keys()) == {"A", "B"}
|
||||
assert k.current_rev("A") == "2"
|
||||
assert k.current_rev("Missing") is None
|
||||
|
||||
|
||||
def test_links_and_red_link_resolution():
|
||||
k = EngineKernel("eng")
|
||||
k.write("Home", "see [[Target]] and [[Other|labelled]] and [[Target]] again")
|
||||
k.write("Target", "exists")
|
||||
assert k.links("Home") == ["Target", "Other"] # ordered, de-duped, label dropped
|
||||
assert k.resolve_link("Target") == Identity("eng", "Target")
|
||||
assert k.resolve_link("Other") is None # red-link (not yet created)
|
||||
|
||||
|
||||
def test_extract_wikilinks_helper():
|
||||
assert extract_wikilinks("none here") == []
|
||||
assert extract_wikilinks("[[A]] [[B|x]] [[A]]") == ["A", "B"]
|
||||
98
tests/test_engine_profile.py
Normal file
98
tests/test_engine_profile.py
Normal file
@@ -0,0 +1,98 @@
|
||||
"""Tests for capability-profile-derived-from-extensions (SHARD-WP-0014 T4, E-5)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.engine import (
|
||||
Extension,
|
||||
ExtensionRuntime,
|
||||
Hook,
|
||||
ProfileContribution,
|
||||
derive_profile,
|
||||
engine_base_profile,
|
||||
)
|
||||
from shard_wiki.model import (
|
||||
Addressing,
|
||||
ContentOpacity,
|
||||
NativeQuery,
|
||||
ProfileError,
|
||||
Verb,
|
||||
)
|
||||
|
||||
|
||||
class StructExt(Extension):
|
||||
id = "ext.struct"
|
||||
|
||||
def hooks(self):
|
||||
return {
|
||||
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
|
||||
verbs_add=frozenset({Verb.STRUCTURED_PAYLOAD})
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
class AddrExt(Extension):
|
||||
id = "ext.addr"
|
||||
|
||||
def hooks(self):
|
||||
return {
|
||||
Hook.ON_PROFILE: lambda payload, ctx: ProfileContribution(
|
||||
addressing=Addressing.SPAN, verbs_add=frozenset({Verb.TRANSCLUDE_SOURCE})
|
||||
)
|
||||
}
|
||||
|
||||
|
||||
class EncryptExt(Extension):
|
||||
id = "ext.encrypt"
|
||||
|
||||
def hooks(self):
|
||||
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(content_opacity=ContentOpacity.ENCRYPTED)}
|
||||
|
||||
|
||||
class QueryExt(Extension):
|
||||
id = "ext.query"
|
||||
|
||||
def hooks(self):
|
||||
return {Hook.ON_PROFILE: lambda p, c: ProfileContribution(native_query=NativeQuery.DB_QUERY)}
|
||||
|
||||
|
||||
def _active(*exts, ids=None):
|
||||
rt = ExtensionRuntime()
|
||||
for e in exts:
|
||||
rt.register(e)
|
||||
return rt.activate(ids if ids is not None else {e.id for e in exts})
|
||||
|
||||
|
||||
def test_base_profile_is_valid_kernel_minimum():
|
||||
p = engine_base_profile()
|
||||
assert p.supports(Verb.READ) and p.supports(Verb.WRITE)
|
||||
assert not p.supports(Verb.STRUCTURED_PAYLOAD)
|
||||
assert p.addressing is Addressing.PATH
|
||||
|
||||
|
||||
def test_no_extensions_yields_base():
|
||||
rt = ExtensionRuntime()
|
||||
active = rt.activate(set())
|
||||
assert derive_profile(active) == engine_base_profile()
|
||||
|
||||
|
||||
def test_activating_extension_raises_the_profile():
|
||||
active = _active(StructExt(), AddrExt())
|
||||
p = derive_profile(active)
|
||||
assert p.supports(Verb.STRUCTURED_PAYLOAD) # from ext.struct
|
||||
assert p.supports(Verb.TRANSCLUDE_SOURCE) # from ext.addr
|
||||
assert p.addressing is Addressing.SPAN # raised by ext.addr
|
||||
assert p.supports(Verb.READ) # base preserved
|
||||
|
||||
|
||||
def test_profile_changes_with_active_set():
|
||||
only_struct = derive_profile(_active(StructExt(), AddrExt(), ids={"ext.struct"}))
|
||||
both = derive_profile(_active(StructExt(), AddrExt()))
|
||||
assert not only_struct.supports(Verb.TRANSCLUDE_SOURCE)
|
||||
assert both.supports(Verb.TRANSCLUDE_SOURCE) # E-5: profile reflects what's active
|
||||
|
||||
|
||||
def test_composition_cannot_yield_an_impossible_profile():
|
||||
# encrypted opacity + native query violates §6.5 implication rules -> derive must reject.
|
||||
active = _active(EncryptExt(), QueryExt())
|
||||
with pytest.raises(ProfileError):
|
||||
derive_profile(active)
|
||||
76
tests/test_error_paths.py
Normal file
76
tests/test_error_paths.py
Normal file
@@ -0,0 +1,76 @@
|
||||
"""Error-path / contract tests across modules (keeps the suite honest about failure behaviour)."""
|
||||
|
||||
from collections.abc import Iterable
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki import InformationSpace
|
||||
from shard_wiki.adapters import FolderAdapter, ShardAdapter, run_conformance
|
||||
from shard_wiki.engine import EngineKernel
|
||||
from shard_wiki.model import CapabilityProfile, Identity, Page, Placement
|
||||
from shard_wiki.provenance import ProvenanceEnvelope
|
||||
from shard_wiki.union import ResolutionKind, UnionGraph
|
||||
|
||||
|
||||
def _folder(tmp_path, name, files, writable=False):
|
||||
root = tmp_path / name
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
for rel, text in files.items():
|
||||
(root / rel).write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root, writable=writable)
|
||||
|
||||
|
||||
def test_resolution_single_on_red_link_raises():
|
||||
u = UnionGraph("s")
|
||||
res = u.resolve("ghost")
|
||||
assert res.kind is ResolutionKind.RED_LINK
|
||||
with pytest.raises(KeyError):
|
||||
res.single()
|
||||
|
||||
|
||||
def test_apply_unknown_overlay_raises(tmp_path):
|
||||
space = InformationSpace("t")
|
||||
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
|
||||
with pytest.raises(KeyError):
|
||||
space.apply_overlay("does-not-exist")
|
||||
|
||||
|
||||
def test_apply_overlay_for_unattached_shard_raises(tmp_path):
|
||||
space = InformationSpace("t")
|
||||
space.attach(_folder(tmp_path, "w", {"Home.md": "x"}, writable=True))
|
||||
# draft an overlay whose target shard is not attached -> apply can't find an adapter
|
||||
ov = space.overlays.draft(Identity("ghost", "X"), "body", base_rev=None)
|
||||
with pytest.raises(KeyError):
|
||||
space.apply_overlay(ov.overlay_id)
|
||||
|
||||
|
||||
def test_kernel_delete_missing_raises():
|
||||
with pytest.raises(KeyError):
|
||||
EngineKernel("eng").delete("nope")
|
||||
|
||||
|
||||
def test_placement_str():
|
||||
assert str(Placement("shardA", "sub/Page")) == "shardA/sub/Page"
|
||||
|
||||
|
||||
class _BrokenProfileAdapter(ShardAdapter):
|
||||
"""profile() raises — the conformance battery must report failure, not crash."""
|
||||
|
||||
@property
|
||||
def shard_id(self) -> str:
|
||||
return "broken"
|
||||
|
||||
def profile(self) -> CapabilityProfile:
|
||||
raise RuntimeError("profile blew up")
|
||||
|
||||
def keys(self) -> Iterable[str]:
|
||||
return []
|
||||
|
||||
def read(self, key: str) -> Page:
|
||||
return Page(Identity("broken", key), "x", ProvenanceEnvelope(source_shard="broken"))
|
||||
|
||||
|
||||
def test_conformance_survives_a_broken_profile():
|
||||
report = run_conformance(_BrokenProfileAdapter())
|
||||
assert not report.ok
|
||||
assert any(c.name == "profile-validates" and not c.ok for c in report.checks)
|
||||
64
tests/test_ext_struct.py
Normal file
64
tests/test_ext_struct.py
Normal file
@@ -0,0 +1,64 @@
|
||||
"""Tests for the ext.struct built-in extension (SHARD-WP-0014 T6)."""
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki import InformationSpace
|
||||
from shard_wiki.adapters import assert_conformant
|
||||
from shard_wiki.engine import ExtensionRuntime, build_engine_shard
|
||||
from shard_wiki.engine.extensions import StructExt, parse_frontmatter
|
||||
from shard_wiki.model import PageShape, Verb
|
||||
|
||||
_STRUCT_PAGE = "---\ntitle: Spec\nstatus: draft\n---\nbody text"
|
||||
|
||||
|
||||
def test_parse_frontmatter():
|
||||
fields, has = parse_frontmatter(_STRUCT_PAGE)
|
||||
assert has and fields == {"title": "Spec", "status": "draft"}
|
||||
assert parse_frontmatter("just prose") == ({}, False)
|
||||
assert parse_frontmatter("---\nunterminated") == ({}, False)
|
||||
|
||||
|
||||
def _runtime(allowed=None):
|
||||
rt = ExtensionRuntime()
|
||||
rt.register(StructExt(allowed_fields=allowed))
|
||||
return rt
|
||||
|
||||
|
||||
def test_feature_absent_when_extension_off():
|
||||
shard = build_engine_shard("off", ExtensionRuntime(), activate=set())
|
||||
shard.write("Spec", _STRUCT_PAGE)
|
||||
assert shard.read("Spec").shape is PageShape.PROSE # kernel: opaque prose
|
||||
assert not shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # honest absence
|
||||
|
||||
|
||||
def test_feature_present_when_extension_on():
|
||||
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
|
||||
shard.write("Spec", _STRUCT_PAGE)
|
||||
assert shard.read("Spec").shape is PageShape.TYPED_RECORD # tagged by ext.struct
|
||||
assert shard.read("Spec").body.endswith("body text") # content preserved (in-text)
|
||||
assert shard.profile().supports(Verb.STRUCTURED_PAYLOAD) # profile reflects activation (E-5)
|
||||
assert_conformant(shard) # still conformant
|
||||
|
||||
|
||||
def test_plain_page_is_not_tagged_even_when_on():
|
||||
shard = build_engine_shard("on", _runtime(), activate={"ext.struct"})
|
||||
shard.write("Plain", "no frontmatter here")
|
||||
assert shard.read("Plain").shape is PageShape.PROSE
|
||||
|
||||
|
||||
def test_allowed_fields_validation_rejects_disallowed():
|
||||
shard = build_engine_shard("v", _runtime(allowed={"title"}), activate={"ext.struct"})
|
||||
with pytest.raises(ValueError, match="disallowed fields"):
|
||||
shard.write("Bad", "---\ntitle: ok\nsecret: no\n---\nx")
|
||||
shard.write("Good", "---\ntitle: ok\n---\nx") # allowed field passes
|
||||
assert shard.read("Good").shape is PageShape.TYPED_RECORD
|
||||
|
||||
|
||||
def test_through_information_space_edit():
|
||||
space = InformationSpace("team")
|
||||
space.attach(build_engine_shard("wikiE", _runtime(), activate={"ext.struct"}))
|
||||
space.union.shard("wikiE").write("Doc", "---\ntitle: T\n---\nv1")
|
||||
res = space.edit("Doc", "---\ntitle: T2\n---\nv2") # overlay→apply→write-through
|
||||
assert res.status.value == "applied"
|
||||
page = space.read("Doc")
|
||||
assert page.shape is PageShape.TYPED_RECORD and "v2" in page.body
|
||||
131
tests/test_git_adapter.py
Normal file
131
tests/test_git_adapter.py
Normal file
@@ -0,0 +1,131 @@
|
||||
"""Tests for the GitShardAdapter read path + profile (SHARD-WP-0012 T1)."""
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.adapters import GitShardAdapter, run_conformance
|
||||
from shard_wiki.model import (
|
||||
AttachmentMode,
|
||||
History,
|
||||
NotSupported,
|
||||
ProfileError,
|
||||
Substrate,
|
||||
Verb,
|
||||
)
|
||||
|
||||
|
||||
def _git(repo, *args):
|
||||
subprocess.run(
|
||||
["git", "-C", str(repo), *args],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
env={"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
|
||||
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
|
||||
"PATH": __import__("os").environ.get("PATH", "")},
|
||||
)
|
||||
|
||||
|
||||
def _repo(tmp_path, files, name="repo"):
|
||||
repo = tmp_path / name
|
||||
repo.mkdir()
|
||||
_git(repo, "init", "--quiet")
|
||||
for rel, text in files.items():
|
||||
p = repo / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
_git(repo, "add", rel)
|
||||
_git(repo, "commit", "-m", "seed")
|
||||
return repo
|
||||
|
||||
|
||||
def test_keys_are_tracked_md_paths(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "h", "docs/Guide.md": "g", "ignore.txt": "x"})
|
||||
adapter = GitShardAdapter("git", repo)
|
||||
assert set(adapter.keys()) == {"Home", "docs/Guide"} # only tracked *.md
|
||||
|
||||
|
||||
def test_read_returns_page_with_commit_sha_rev(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "welcome"})
|
||||
adapter = GitShardAdapter("git", repo)
|
||||
page = adapter.read("Home")
|
||||
assert page.identity.shard == "git"
|
||||
assert page.body == "welcome"
|
||||
head = subprocess.run(
|
||||
["git", "-C", str(repo), "rev-parse", "HEAD"], capture_output=True, text=True, check=True
|
||||
).stdout.strip()
|
||||
assert page.envelope.source_rev == head # source_rev is the commit sha
|
||||
assert page.envelope.lineage == "git-native"
|
||||
|
||||
|
||||
def test_read_missing_key_raises(tmp_path):
|
||||
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
|
||||
with pytest.raises(KeyError):
|
||||
adapter.read("Nope")
|
||||
|
||||
|
||||
def test_profile_validates_implication_rules(tmp_path):
|
||||
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
|
||||
assert profile.substrate is Substrate.GIT
|
||||
assert profile.attachment_mode is AttachmentMode.GIT_IS_STORE
|
||||
assert profile.history is History.GIT_NATIVE # git-is-store ⟹ git-native
|
||||
profile.validate() # raises if the implication rule were violated
|
||||
|
||||
|
||||
def test_profile_is_read_only_in_t1(tmp_path):
|
||||
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"})).profile()
|
||||
assert profile.supports(Verb.READ)
|
||||
assert not profile.supports(Verb.WRITE)
|
||||
|
||||
|
||||
def test_conformance_read_path_passes(tmp_path):
|
||||
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h", "Other.md": "o"}))
|
||||
report = run_conformance(adapter)
|
||||
assert report.ok, report.diff()
|
||||
|
||||
|
||||
def test_unclaimed_write_raises_not_supported(tmp_path):
|
||||
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}))
|
||||
with pytest.raises(NotSupported):
|
||||
adapter.write("Home", "new") # read-only: honest absence
|
||||
|
||||
|
||||
def test_empty_repo_has_no_keys(tmp_path):
|
||||
repo = tmp_path / "empty"
|
||||
repo.mkdir()
|
||||
_git(repo, "init", "--quiet")
|
||||
adapter = GitShardAdapter("git", repo)
|
||||
assert list(adapter.keys()) == []
|
||||
|
||||
|
||||
def test_bad_profile_combo_is_rejected():
|
||||
# Sanity: the implication rule that backs the git profile actually bites when violated.
|
||||
from shard_wiki.model import (
|
||||
AccessGrant,
|
||||
Addressing,
|
||||
CapabilityProfile,
|
||||
ContentOpacity,
|
||||
MergeModel,
|
||||
NativeQuery,
|
||||
OperationalEnvelope,
|
||||
Translation,
|
||||
WriteGranularity,
|
||||
)
|
||||
from shard_wiki.provenance import Liveness
|
||||
|
||||
with pytest.raises(ProfileError):
|
||||
CapabilityProfile(
|
||||
substrate=Substrate.FILES, # not git, but claims git-is-store
|
||||
attachment_mode=AttachmentMode.GIT_IS_STORE,
|
||||
write_granularity=WriteGranularity.NONE,
|
||||
content_opacity=ContentOpacity.TRANSPARENT,
|
||||
operational_envelope=OperationalEnvelope.LOCAL_UNBOUNDED,
|
||||
access_grant=AccessGrant.OPEN,
|
||||
liveness=Liveness.STATIC,
|
||||
history=History.NONE,
|
||||
merge_model=MergeModel.NONE,
|
||||
addressing=Addressing.PATH,
|
||||
native_query=NativeQuery.NONE,
|
||||
translation=Translation.NATIVE,
|
||||
supported_verbs=frozenset({Verb.READ}),
|
||||
).validate()
|
||||
116
tests/test_git_adapter_integration.py
Normal file
116
tests/test_git_adapter_integration.py
Normal file
@@ -0,0 +1,116 @@
|
||||
"""GitShardAdapter history adopt + cross-substrate integration (SHARD-WP-0012 T3)."""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter, GitShardAdapter
|
||||
from shard_wiki.coordination import ApplyStatus
|
||||
from shard_wiki.space import InformationSpace
|
||||
|
||||
_ENV = {
|
||||
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
|
||||
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
}
|
||||
|
||||
|
||||
def _git(repo, *args):
|
||||
return subprocess.run(
|
||||
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def _git_repo(tmp_path, files, name="git"):
|
||||
repo = tmp_path / name
|
||||
repo.mkdir()
|
||||
_git(repo, "init", "--quiet")
|
||||
for rel, text in files.items():
|
||||
(repo / rel).parent.mkdir(parents=True, exist_ok=True)
|
||||
(repo / rel).write_text(text, encoding="utf-8")
|
||||
_git(repo, "add", rel)
|
||||
_git(repo, "commit", "-m", "seed")
|
||||
return repo
|
||||
|
||||
|
||||
def _folder(tmp_path, name, files, writable=False):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root, writable=writable)
|
||||
|
||||
|
||||
# -- history adopt -------------------------------------------------------------
|
||||
|
||||
|
||||
def test_history_lists_commits_newest_first(tmp_path):
|
||||
repo = _git_repo(tmp_path, {"Home.md": "v1"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
adapter.write("Home", "v2")
|
||||
history = adapter.history("Home")
|
||||
assert len(history) == 2
|
||||
assert history[0].message == "write Home.md" # newest first
|
||||
assert history[-1].message == "seed"
|
||||
assert all(rev.sha for rev in history)
|
||||
|
||||
|
||||
def test_history_unknown_key_raises(tmp_path):
|
||||
adapter = GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "h"}))
|
||||
with pytest.raises(KeyError):
|
||||
adapter.history("Nope")
|
||||
|
||||
|
||||
# -- cross-substrate integration ----------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_across_git_and_folder(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "git home"})))
|
||||
space.attach(_folder(tmp_path, "notes", {"Daily.md": "folder daily"}))
|
||||
assert space.read("Home").body == "git home" # resolved from the git shard
|
||||
assert space.read("Daily").body == "folder daily" # resolved from the folder shard
|
||||
|
||||
|
||||
def test_chorus_spans_substrates_with_divergence(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Shared.md": "from git"})))
|
||||
space.attach(_folder(tmp_path, "notes", {"Shared.md": "from folder"}))
|
||||
res = space.resolve("Shared")
|
||||
assert {p.body for p in res.pages} == {"from git", "from folder"} # chorus across substrates
|
||||
git_page = next(p for p in res.pages if p.identity.shard == "git")
|
||||
assert git_page.envelope.divergence # divergence recorded, not erased
|
||||
|
||||
|
||||
def test_edit_through_git_shard_commits(tmp_path):
|
||||
repo = _git_repo(tmp_path, {"Home.md": "original"})
|
||||
space = InformationSpace("space")
|
||||
space.attach(GitShardAdapter("git", repo, writable=True))
|
||||
result = space.edit("Home", "edited via overlay")
|
||||
assert result.status is ApplyStatus.APPLIED # write-through fast-forward on a git shard
|
||||
assert space.read("Home").body == "edited via overlay"
|
||||
assert int(_git(repo, "rev-list", "--count", "HEAD")) == 2 # the edit became a commit
|
||||
|
||||
|
||||
def test_apply_under_drift_refuses_on_external_commit(tmp_path):
|
||||
repo = _git_repo(tmp_path, {"Home.md": "original"})
|
||||
space = InformationSpace("space")
|
||||
space.attach(GitShardAdapter("git", repo, writable=True))
|
||||
overlay = space.overlay("Home", "my draft") # base_rev = current git sha
|
||||
# Another writer commits to the same path → the sha moves underneath the draft.
|
||||
(repo / "Home.md").write_text("someone else", encoding="utf-8")
|
||||
_git(repo, "add", "Home.md")
|
||||
_git(repo, "commit", "-m", "external")
|
||||
result = space.apply_overlay(overlay.overlay_id)
|
||||
assert result.status is ApplyStatus.REFUSED_DRIFT # never clobber (sha drift detected)
|
||||
# The shard itself is untouched — the external commit stands; the draft remains a draft.
|
||||
assert space.union.shard("git").read("Home").body == "someone else"
|
||||
|
||||
|
||||
def test_overlay_on_read_only_git_shard_kept_as_draft(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(GitShardAdapter("git", _git_repo(tmp_path, {"Home.md": "ro"}), writable=False))
|
||||
result = space.edit("Home", "wanted change")
|
||||
assert result.status is ApplyStatus.KEPT_DRAFT # read-only target → overlay retained
|
||||
89
tests/test_git_adapter_write.py
Normal file
89
tests/test_git_adapter_write.py
Normal file
@@ -0,0 +1,89 @@
|
||||
"""Tests for GitShardAdapter write=commit + current_rev drift (SHARD-WP-0012 T2)."""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from shard_wiki.adapters import GitShardAdapter, run_conformance
|
||||
from shard_wiki.model import Verb
|
||||
|
||||
_ENV = {
|
||||
"GIT_AUTHOR_NAME": "t", "GIT_AUTHOR_EMAIL": "t@t",
|
||||
"GIT_COMMITTER_NAME": "t", "GIT_COMMITTER_EMAIL": "t@t",
|
||||
"PATH": os.environ.get("PATH", ""),
|
||||
}
|
||||
|
||||
|
||||
def _git(repo, *args, capture=False):
|
||||
return subprocess.run(
|
||||
["git", "-C", str(repo), *args], check=True, capture_output=True, text=True, env=_ENV
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def _repo(tmp_path, files):
|
||||
repo = tmp_path / "repo"
|
||||
repo.mkdir()
|
||||
_git(repo, "init", "--quiet")
|
||||
for rel, text in files.items():
|
||||
(repo / rel).write_text(text, encoding="utf-8")
|
||||
_git(repo, "add", rel)
|
||||
_git(repo, "commit", "-m", "seed")
|
||||
return repo
|
||||
|
||||
|
||||
def test_writable_profile_declares_write_and_version(tmp_path):
|
||||
profile = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "h"}), writable=True).profile()
|
||||
assert profile.supports(Verb.WRITE)
|
||||
assert profile.supports(Verb.VERSION)
|
||||
profile.validate() # PER_PAGE + WRITE is a consistent combination
|
||||
|
||||
|
||||
def test_write_creates_a_commit(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "old"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
before = _git(repo, "rev-list", "--count", "HEAD")
|
||||
page = adapter.write("Home", "new body")
|
||||
after = _git(repo, "rev-list", "--count", "HEAD")
|
||||
assert int(after) == int(before) + 1 # one new commit
|
||||
assert page.body == "new body"
|
||||
assert page.envelope.source_rev == _git(repo, "rev-parse", "HEAD") # page is at the new sha
|
||||
|
||||
|
||||
def test_write_advances_current_rev(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "old"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
rev_before = adapter.current_rev("Home")
|
||||
adapter.write("Home", "changed")
|
||||
assert adapter.current_rev("Home") != rev_before # sha moved → drift detectable
|
||||
|
||||
|
||||
def test_write_new_key_tracks_it(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "h"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
adapter.write("docs/New", "fresh page")
|
||||
assert "docs/New" in set(adapter.keys())
|
||||
assert adapter.read("docs/New").body == "fresh page"
|
||||
|
||||
|
||||
def test_noop_write_creates_no_empty_commit(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "same"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
before = _git(repo, "rev-list", "--count", "HEAD")
|
||||
adapter.write("Home", "same") # identical body → nothing to commit
|
||||
assert _git(repo, "rev-list", "--count", "HEAD") == before
|
||||
|
||||
|
||||
def test_current_rev_reflects_external_commit(tmp_path):
|
||||
repo = _repo(tmp_path, {"Home.md": "h"})
|
||||
adapter = GitShardAdapter("git", repo, writable=True)
|
||||
rev = adapter.current_rev("Home")
|
||||
# An out-of-band commit to the same path (another writer) moves the per-path sha.
|
||||
(repo / "Home.md").write_text("externally edited", encoding="utf-8")
|
||||
_git(repo, "add", "Home.md")
|
||||
_git(repo, "commit", "-m", "external")
|
||||
assert adapter.current_rev("Home") != rev
|
||||
|
||||
|
||||
def test_conformance_positive_write_probe_passes(tmp_path):
|
||||
adapter = GitShardAdapter("git", _repo(tmp_path, {"Home.md": "body"}), writable=True)
|
||||
report = run_conformance(adapter)
|
||||
assert report.ok, report.diff()
|
||||
84
tests/test_git_event_store.py
Normal file
84
tests/test_git_event_store.py
Normal file
@@ -0,0 +1,84 @@
|
||||
"""Tests for the git-backed event store (SHARD-WP-0009 T1).
|
||||
|
||||
The git backend must satisfy the same EventStore contract as the in-memory one (round-trip,
|
||||
ordering, determinism) while making the log git-addressable.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
|
||||
import pytest
|
||||
|
||||
from shard_wiki.coordination import (
|
||||
DecisionLog,
|
||||
EventType,
|
||||
GitEventStore,
|
||||
InMemoryEventStore,
|
||||
deserialize_event,
|
||||
serialize_event,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def git_store(tmp_path):
|
||||
return GitEventStore(tmp_path / "coord")
|
||||
|
||||
|
||||
def test_append_git_read_round_trips(git_store):
|
||||
log = DecisionLog(git_store)
|
||||
ev = log.append("s", EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"})
|
||||
(read,) = log.events("s")
|
||||
assert read.seq == ev.seq == 0
|
||||
assert read.space == "s"
|
||||
assert read.type is EventType.ALIAS_SET
|
||||
assert read.payload == {"alias": "Home", "target": "shardA:Index"}
|
||||
|
||||
|
||||
def test_ordering_preserved_and_per_space_monotonic(git_store):
|
||||
log = DecisionLog(git_store)
|
||||
log.append("a", EventType.ALIAS_SET, {"alias": "X", "target": "s:1"})
|
||||
log.append("a", EventType.ALIAS_SET, {"alias": "Y", "target": "s:2"})
|
||||
log.append("b", EventType.ALIAS_SET, {"alias": "Z", "target": "s:3"})
|
||||
assert [e.seq for e in log.events("a")] == [0, 1]
|
||||
assert [e.payload["alias"] for e in log.events("a")] == ["X", "Y"]
|
||||
assert [e.seq for e in log.events("b")] == [0] # independent ref/ordering
|
||||
|
||||
|
||||
def test_each_append_is_a_git_commit(git_store):
|
||||
log = DecisionLog(git_store)
|
||||
log.append("s", EventType.BINDING_MADE, {"members": ["a", "b"]})
|
||||
log.append("s", EventType.PAGE_FORKED, {"source": "a", "fork": "c"})
|
||||
ref = GitEventStore._ref("s")
|
||||
count = subprocess.run(
|
||||
["git", "-C", str(git_store.repo_path), "rev-list", "--count", ref],
|
||||
capture_output=True, text=True, check=True,
|
||||
).stdout.strip()
|
||||
assert count == "2" # one immutable commit object per append
|
||||
|
||||
|
||||
def test_deterministic_serialization_is_stable_and_sorted():
|
||||
log = InMemoryEventStore()
|
||||
ev = log.append("s", EventType.ALIAS_SET, {"target": "z", "alias": "a"})
|
||||
blob = serialize_event(ev)
|
||||
assert serialize_event(ev) == blob # stable across calls
|
||||
assert blob.index(b'"alias"') < blob.index(b'"target"') # payload keys sorted, not insertion
|
||||
assert deserialize_event(blob).payload == {"alias": "a", "target": "z"}
|
||||
|
||||
|
||||
def test_git_fold_matches_in_memory_fold(git_store):
|
||||
events = [
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardA:Index"}),
|
||||
(EventType.BINDING_MADE, {"members": ["a", "b"]}),
|
||||
(EventType.BINDING_MADE, {"members": ["b", "c"]}),
|
||||
(EventType.ALIAS_SET, {"alias": "Home", "target": "shardB:Main"}),
|
||||
]
|
||||
mem = DecisionLog(InMemoryEventStore())
|
||||
git = DecisionLog(git_store)
|
||||
for typ, payload in events:
|
||||
mem.append("s", typ, payload)
|
||||
git.append("s", typ, payload)
|
||||
assert git.fold("s").aliases == mem.fold("s").aliases
|
||||
assert git.fold("s").equivalence_groups == mem.fold("s").equivalence_groups
|
||||
|
||||
|
||||
def test_default_decisionlog_is_in_memory():
|
||||
assert isinstance(DecisionLog()._store, InMemoryEventStore)
|
||||
89
tests/test_incremental_equivalence.py
Normal file
89
tests/test_incremental_equivalence.py
Normal file
@@ -0,0 +1,89 @@
|
||||
"""Tests for the indexed equivalence relation — blocking + verify (SHARD-WP-0011 T1)."""
|
||||
|
||||
from itertools import combinations
|
||||
|
||||
from shard_wiki.incremental import EquivalenceIndex, MinHasher, band_keys, jaccard, shingles
|
||||
from shard_wiki.incremental.equivalence import _fingerprint
|
||||
from shard_wiki.model import Identity, Page
|
||||
from shard_wiki.provenance import ProvenanceEnvelope
|
||||
|
||||
|
||||
def _page(shard, key, body):
|
||||
return Page(
|
||||
identity=Identity(shard, key),
|
||||
body=body,
|
||||
envelope=ProvenanceEnvelope(source_shard=shard),
|
||||
)
|
||||
|
||||
|
||||
def _brute_force_groups(pages, threshold):
|
||||
"""Oracle: O(N²) verify of every pair, then connected components."""
|
||||
parent = {p.identity: p.identity for p in pages}
|
||||
|
||||
def find(x):
|
||||
while parent[x] != x:
|
||||
parent[x] = parent[parent[x]]
|
||||
x = parent[x]
|
||||
return x
|
||||
|
||||
for p, q in combinations(pages, 2):
|
||||
same_fp = _fingerprint(p.body) == _fingerprint(q.body)
|
||||
sim = jaccard(shingles(p.body), shingles(q.body))
|
||||
if same_fp or sim >= threshold:
|
||||
parent[find(p.identity)] = find(q.identity)
|
||||
comps = {}
|
||||
for p in pages:
|
||||
comps.setdefault(find(p.identity), set()).add(p.identity)
|
||||
return {frozenset(v) for v in comps.values() if len(v) > 1}
|
||||
|
||||
|
||||
def test_minhash_lsh_buckets_near_duplicates_together():
|
||||
hasher = MinHasher(num_perm=64)
|
||||
base = "the quick brown fox jumps over the lazy dog near the river bank today"
|
||||
near = base + " and then some"
|
||||
far = "completely unrelated content about astrophysics and distant galaxies far"
|
||||
b_base = set(band_keys(hasher.signature(shingles(base)), 32))
|
||||
b_near = set(band_keys(hasher.signature(shingles(near)), 32))
|
||||
b_far = set(band_keys(hasher.signature(shingles(far)), 32))
|
||||
assert b_base & b_near # near-duplicates share at least one band
|
||||
assert not (b_base & b_far) # unrelated pages do not
|
||||
|
||||
|
||||
def test_exact_duplicate_across_shards_is_equivalent():
|
||||
idx = EquivalenceIndex()
|
||||
idx.add(_page("A", "Foo", "identical body text here"))
|
||||
idx.add(_page("B", "Bar", "identical body text here"))
|
||||
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
|
||||
{Identity("A", "Foo"), Identity("B", "Bar")}
|
||||
)
|
||||
|
||||
|
||||
def test_unrelated_pages_are_not_equivalent():
|
||||
idx = EquivalenceIndex()
|
||||
idx.add(_page("A", "Foo", "alpha beta gamma delta epsilon"))
|
||||
idx.add(_page("B", "Bar", "nothing in common whatsoever entirely"))
|
||||
assert idx.groups() == ()
|
||||
|
||||
|
||||
def test_curator_binding_forces_equivalence_regardless_of_content():
|
||||
idx = EquivalenceIndex()
|
||||
idx.add(_page("A", "Foo", "one thing"))
|
||||
idx.add(_page("B", "Bar", "totally different"))
|
||||
idx.bind(Identity("A", "Foo"), Identity("B", "Bar"))
|
||||
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
|
||||
{Identity("A", "Foo"), Identity("B", "Bar")}
|
||||
)
|
||||
|
||||
|
||||
def test_index_matches_brute_force_oracle():
|
||||
threshold = 0.7
|
||||
shared = "shared sentence one shared sentence two shared sentence three end"
|
||||
pages = [
|
||||
_page("A", "Doc1", shared),
|
||||
_page("B", "Doc1copy", shared + " minor tail"), # near-dup of A
|
||||
_page("C", "Other", "a totally distinct page with no overlapping shingles at all here"),
|
||||
_page("D", "Lonely", "yet another isolated document about unrelated subject matter alone"),
|
||||
]
|
||||
idx = EquivalenceIndex(threshold=threshold)
|
||||
idx.build(pages)
|
||||
assert set(idx.groups()) == _brute_force_groups(pages, threshold)
|
||||
84
tests/test_incremental_maintenance.py
Normal file
84
tests/test_incremental_maintenance.py
Normal file
@@ -0,0 +1,84 @@
|
||||
"""Incremental maintenance == rebuild, with retraction + propagation (SHARD-WP-0011 T2)."""
|
||||
|
||||
from shard_wiki.incremental import EquivalenceIndex
|
||||
from shard_wiki.model import Identity, Page
|
||||
from shard_wiki.provenance import ProvenanceEnvelope
|
||||
|
||||
|
||||
def _page(shard, key, body):
|
||||
return Page(
|
||||
identity=Identity(shard, key),
|
||||
body=body,
|
||||
envelope=ProvenanceEnvelope(source_shard=shard),
|
||||
)
|
||||
|
||||
|
||||
def _rebuilt(pages, curator=()):
|
||||
idx = EquivalenceIndex()
|
||||
idx.build(pages, curator)
|
||||
return idx
|
||||
|
||||
|
||||
def _equal(a, b):
|
||||
return a.edges() == b.edges() and set(a.groups()) == set(b.groups())
|
||||
|
||||
|
||||
def test_add_keeps_index_equal_to_rebuild():
|
||||
pages = [_page("A", "Foo", "same content here"), _page("B", "Bar", "same content here")]
|
||||
idx = EquivalenceIndex()
|
||||
for p in pages:
|
||||
idx.add(p)
|
||||
assert _equal(idx, _rebuilt(pages))
|
||||
assert idx.groups() # the two collapse
|
||||
|
||||
|
||||
def test_remove_keeps_index_equal_to_rebuild():
|
||||
pages = [
|
||||
_page("A", "Foo", "same content here"),
|
||||
_page("B", "Bar", "same content here"),
|
||||
_page("C", "Baz", "unrelated isolated material entirely"),
|
||||
]
|
||||
idx = _rebuilt(pages)
|
||||
idx.remove(Identity("B", "Bar"))
|
||||
assert _equal(idx, _rebuilt([pages[0], pages[2]]))
|
||||
|
||||
|
||||
def test_edit_into_new_bucket_retracts_stale_edge():
|
||||
a = _page("A", "Foo", "shared identical body text")
|
||||
b = _page("B", "Bar", "shared identical body text")
|
||||
idx = _rebuilt([a, b])
|
||||
assert idx.groups() # A ≡ B initially
|
||||
# Edit B to something completely different: it exits A's buckets, the edge is retracted.
|
||||
b2 = _page("B", "Bar", "now totally divergent unrelated prose about nothing")
|
||||
idx.update(b2)
|
||||
assert idx.groups() == () # stale edge gone
|
||||
assert _equal(idx, _rebuilt([a, b2]))
|
||||
|
||||
|
||||
def test_edit_into_equivalence_adds_edge():
|
||||
a = _page("A", "Foo", "target body to converge on later")
|
||||
b = _page("B", "Bar", "initially completely separate writing here")
|
||||
idx = _rebuilt([a, b])
|
||||
assert idx.groups() == ()
|
||||
b2 = _page("B", "Bar", "target body to converge on later") # now identical to A
|
||||
idx.update(b2)
|
||||
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
|
||||
{Identity("A", "Foo"), Identity("B", "Bar")}
|
||||
)
|
||||
assert _equal(idx, _rebuilt([a, b2]))
|
||||
|
||||
|
||||
def test_removing_connector_splits_a_chorus():
|
||||
# Curator chain A—B—C (no direct A—C): one group of three.
|
||||
a, b, c = (_page("A", "X", "aaa"), _page("B", "Y", "bbb"), _page("C", "Z", "ccc"))
|
||||
idx = EquivalenceIndex()
|
||||
for p in (a, b, c):
|
||||
idx.add(p)
|
||||
idx.bind(a.identity, b.identity)
|
||||
idx.bind(b.identity, c.identity)
|
||||
assert idx.equivalent_to(a.identity) == {a.identity, b.identity, c.identity}
|
||||
# Removing the connector B retracts/propagates: the chorus splits.
|
||||
idx.remove(b.identity)
|
||||
assert idx.groups() == ()
|
||||
chain = [(a.identity, b.identity), (b.identity, c.identity)]
|
||||
assert _equal(idx, _rebuilt([a, c], curator=chain))
|
||||
89
tests/test_incremental_verification.py
Normal file
89
tests/test_incremental_verification.py
Normal file
@@ -0,0 +1,89 @@
|
||||
"""Tests for I-2 verification — digest + consistency-checker (SHARD-WP-0011 T3)."""
|
||||
|
||||
from shard_wiki.incremental import (
|
||||
ConsistencyChecker,
|
||||
EquivalenceIndex,
|
||||
derived_digest,
|
||||
)
|
||||
from shard_wiki.model import Identity, Page
|
||||
from shard_wiki.provenance import ProvenanceEnvelope
|
||||
|
||||
|
||||
def _page(shard, key, body):
|
||||
return Page(
|
||||
identity=Identity(shard, key),
|
||||
body=body,
|
||||
envelope=ProvenanceEnvelope(source_shard=shard),
|
||||
)
|
||||
|
||||
|
||||
def test_digest_is_stable_under_equivalent_event_orders():
|
||||
pages = [
|
||||
_page("A", "Foo", "shared body text here"),
|
||||
_page("B", "Bar", "shared body text here"),
|
||||
_page("C", "Baz", "an entirely separate unrelated document"),
|
||||
]
|
||||
forward = EquivalenceIndex()
|
||||
for p in pages:
|
||||
forward.add(p)
|
||||
reverse = EquivalenceIndex()
|
||||
for p in reversed(pages):
|
||||
reverse.add(p)
|
||||
assert derived_digest(forward) == derived_digest(reverse)
|
||||
|
||||
|
||||
def test_clean_index_reports_healthy():
|
||||
pages = [_page("A", "Foo", "same body"), _page("B", "Bar", "same body")]
|
||||
idx = EquivalenceIndex()
|
||||
idx.build(pages)
|
||||
checker = ConsistencyChecker(idx, pages_fn := (lambda: pages))
|
||||
report = checker.check_and_repair()
|
||||
assert report.drifted is False and report.healthy is True
|
||||
assert pages_fn() # source unchanged
|
||||
|
||||
|
||||
def test_missed_delta_drift_is_detected_and_repaired():
|
||||
a = _page("A", "Foo", "converging target body")
|
||||
b = _page("B", "Bar", "initially unrelated separate text")
|
||||
source = {"pages": [a, b]}
|
||||
idx = EquivalenceIndex()
|
||||
idx.build(source["pages"])
|
||||
assert idx.groups() == () # not equivalent yet
|
||||
|
||||
# Source changes B to match A, but the index is never told (a missed delta → drift).
|
||||
b2 = _page("B", "Bar", "converging target body")
|
||||
source["pages"] = [a, b2]
|
||||
|
||||
checker = ConsistencyChecker(idx, lambda: source["pages"])
|
||||
report = checker.check_and_repair()
|
||||
assert report.drifted is True and report.repaired is True and report.healthy is True
|
||||
# Self-healed: the index now reflects the equivalence.
|
||||
assert idx.equivalent_to(Identity("A", "Foo")) == frozenset(
|
||||
{Identity("A", "Foo"), Identity("B", "Bar")}
|
||||
)
|
||||
|
||||
|
||||
def test_corrupted_internal_state_is_healed():
|
||||
a = _page("A", "Foo", "identical content")
|
||||
b = _page("B", "Bar", "identical content")
|
||||
idx = EquivalenceIndex()
|
||||
idx.build([a, b])
|
||||
# Corrupt the derived tier directly: delete a true edge (simulated index corruption).
|
||||
idx._content_edges.clear()
|
||||
assert idx.groups() == () # corrupted away
|
||||
|
||||
checker = ConsistencyChecker(idx, lambda: [a, b])
|
||||
report = checker.check_and_repair()
|
||||
assert report.drifted is True and report.healthy is True
|
||||
assert idx.groups() # edge restored by scoped recompute
|
||||
|
||||
|
||||
def test_removed_source_page_is_reconciled():
|
||||
a = _page("A", "Foo", "same body")
|
||||
b = _page("B", "Bar", "same body")
|
||||
idx = EquivalenceIndex()
|
||||
idx.build([a, b])
|
||||
checker = ConsistencyChecker(idx, lambda: [a]) # B vanished from source
|
||||
report = checker.check_and_repair()
|
||||
assert report.healthy is True
|
||||
assert Identity("B", "Bar") not in idx.identities()
|
||||
74
tests/test_incremental_wiring.py
Normal file
74
tests/test_incremental_wiring.py
Normal file
@@ -0,0 +1,74 @@
|
||||
"""Wire the incremental tier behind InformationSpace views (SHARD-WP-0011 T4)."""
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.coordination import EventType
|
||||
from shard_wiki.model import Identity
|
||||
from shard_wiki.space import InformationSpace
|
||||
from shard_wiki.views import all_pages
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def test_all_pages_via_index_matches_direct_fold(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "wiki", {"Home.md": "welcome", "Guide.md": "the guide"}))
|
||||
space.attach(_shard(tmp_path, "notes", {"Daily.md": "today"}))
|
||||
# Routed-through-index result equals the direct fold-based computation (behaviour unchanged).
|
||||
via_index = {(e.name, e.members) for e in space.all_pages()}
|
||||
direct = {(e.name, e.members) for e in all_pages(space.union)}
|
||||
assert via_index == direct
|
||||
|
||||
|
||||
def test_curator_binding_collapses_via_maintained_index(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "a", {"Foo.md": "x"}))
|
||||
space.attach(_shard(tmp_path, "b", {"Bar.md": "y"}))
|
||||
space.log.append(
|
||||
"space", EventType.BINDING_MADE, {"members": ["a:Foo", "b:Bar"]}
|
||||
)
|
||||
# The maintained index re-syncs curator edges live from the log fold.
|
||||
collapsed = [e for e in space.all_pages() if len(e.members) == 2]
|
||||
assert len(collapsed) == 1
|
||||
assert set(collapsed[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")}
|
||||
|
||||
|
||||
def test_content_duplicate_collapses_via_index(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "a", {"Foo.md": "the very same body content here"}))
|
||||
space.attach(_shard(tmp_path, "b", {"Bar.md": "the very same body content here"}))
|
||||
dup = [e for e in space.all_pages() if len(e.members) == 2]
|
||||
assert len(dup) == 1 # content equivalence detected by the maintained index
|
||||
assert set(dup[0].members) == {Identity("a", "Foo"), Identity("b", "Bar")}
|
||||
|
||||
|
||||
def test_attach_invalidates_index(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"}))
|
||||
assert space.all_pages() # builds the index (one page, no groups)
|
||||
space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"})) # marks index stale
|
||||
dup = [e for e in space.all_pages() if len(e.members) == 2]
|
||||
assert len(dup) == 1 # rebuilt fallback picks up the new equivalent page
|
||||
|
||||
|
||||
def test_verify_index_reports_healthy_when_consistent(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "a", {"Foo.md": "same body"}))
|
||||
space.attach(_shard(tmp_path, "b", {"Bar.md": "same body"}))
|
||||
space.all_pages() # ensure built
|
||||
report = space.verify_index()
|
||||
assert report.healthy is True
|
||||
|
||||
|
||||
def test_reindex_is_an_explicit_fallback(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(_shard(tmp_path, "a", {"Foo.md": "content"}))
|
||||
before = space.index.digest()
|
||||
space.reindex()
|
||||
assert space.index.digest() == before # rebuild is deterministic
|
||||
76
tests/test_views_allpages.py
Normal file
76
tests/test_views_allpages.py
Normal file
@@ -0,0 +1,76 @@
|
||||
"""Tests for the AllPages + SiteMap enumeration views (SHARD-WP-0010 T4)."""
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.coordination import DecisionLog, EventType
|
||||
from shard_wiki.model import Identity
|
||||
from shard_wiki.union import UnionGraph
|
||||
from shard_wiki.views import all_pages, site_map
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def test_all_pages_spans_shards(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"A.md": "a"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"B.md": "b"}))
|
||||
names = {e.name for e in all_pages(u)}
|
||||
assert names == {"A", "B"}
|
||||
|
||||
|
||||
def test_chorus_collapses_to_one_entry_with_divergence(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A home"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B home"}))
|
||||
entries = all_pages(u)
|
||||
home = [e for e in entries if e.name == "Home"]
|
||||
assert len(home) == 1 # chorus → single entry
|
||||
assert set(home[0].members) == {Identity("shardA", "Home"), Identity("shardB", "Home")}
|
||||
assert home[0].diverges is True # bodies differ — collapse acknowledged, not silent
|
||||
|
||||
|
||||
def test_chorus_same_body_does_not_diverge(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "same"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"Home.md": "same"}))
|
||||
(home,) = [e for e in all_pages(u) if e.name == "Home"]
|
||||
assert home.diverges is False
|
||||
|
||||
|
||||
def test_equivalence_binding_collapses_distinct_keys(tmp_path):
|
||||
log = DecisionLog()
|
||||
log.append(
|
||||
"space", EventType.BINDING_MADE, {"members": ["shardA:Foo", "shardB:Bar"]}
|
||||
)
|
||||
u = UnionGraph("space", log=log)
|
||||
u.attach(_shard(tmp_path, "shardA", {"Foo.md": "x"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"Bar.md": "x"}))
|
||||
pair = {Identity("shardA", "Foo"), Identity("shardB", "Bar")}
|
||||
# The two bound identities fold into one entry (named by the min key, "Bar").
|
||||
bound = [e for e in all_pages(u) if {*e.members} == pair]
|
||||
assert len(bound) == 1
|
||||
assert bound[0].name == "Bar"
|
||||
|
||||
|
||||
def test_sitemap_reflects_namespace_paths(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(
|
||||
_shard(
|
||||
tmp_path,
|
||||
"shardA",
|
||||
{"Home.md": "h", "docs/Guide.md": "g", "docs/api/Ref.md": "r"},
|
||||
)
|
||||
)
|
||||
root = site_map(u)
|
||||
# Top level: "Home" page directly, and a "docs" namespace.
|
||||
assert any(p.key == "Home" for p in root.pages)
|
||||
docs = next(c for c in root.children if c.name == "docs")
|
||||
assert any(p.key == "docs/Guide" for p in docs.pages)
|
||||
api = next(c for c in docs.children if c.name == "api")
|
||||
assert any(p.key == "docs/api/Ref" for p in api.pages)
|
||||
51
tests/test_views_backlinks.py
Normal file
51
tests/test_views_backlinks.py
Normal file
@@ -0,0 +1,51 @@
|
||||
"""Tests for the BackLinks derived view (SHARD-WP-0010 T2)."""
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.model import Identity
|
||||
from shard_wiki.union import UnionGraph
|
||||
from shard_wiki.views import build_backlinks
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def test_link_yields_backlink_with_provenance(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"A.md": "see [[B]]", "B.md": "target"}))
|
||||
index = build_backlinks(u)
|
||||
assert index.sources("B") == frozenset({Identity("shardA", "A")})
|
||||
(bl,) = index.to("B")
|
||||
assert bl.source_shard == "shardA" # entry carries source provenance
|
||||
|
||||
|
||||
def test_red_links_create_no_backlinks(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"A.md": "see [[Ghost]]"}))
|
||||
index = build_backlinks(u)
|
||||
assert index.to("Ghost") == () # unresolved target → no backlink
|
||||
assert "Ghost" not in index.names()
|
||||
|
||||
|
||||
def test_chorus_target_aggregates_backlinks(tmp_path):
|
||||
# "Home" exists in two shards (a chorus); links to it from anywhere aggregate under one name.
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A home", "A.md": "[[Home]]"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B home", "B.md": "[[Home]]"}))
|
||||
index = build_backlinks(u)
|
||||
assert index.sources("Home") == frozenset(
|
||||
{Identity("shardA", "A"), Identity("shardB", "B")}
|
||||
)
|
||||
|
||||
|
||||
def test_backlinks_span_shards(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Index.md": "x"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"B.md": "links [[Index]]"}))
|
||||
index = build_backlinks(u)
|
||||
assert index.sources("Index") == frozenset({Identity("shardB", "B")})
|
||||
52
tests/test_views_integration.py
Normal file
52
tests/test_views_integration.py
Normal file
@@ -0,0 +1,52 @@
|
||||
"""Integration: derived views exposed on InformationSpace over two shards (SHARD-WP-0010 T5)."""
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.model import Identity
|
||||
from shard_wiki.space import InformationSpace
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def _space(tmp_path):
|
||||
space = InformationSpace("space")
|
||||
space.attach(
|
||||
_shard(tmp_path, "wiki", {"Home.md": "welcome, see [[Guide]]", "Guide.md": "the guide"})
|
||||
)
|
||||
space.attach(_shard(tmp_path, "notes", {"Daily.md": "today I read [[Guide]]"}))
|
||||
return space
|
||||
|
||||
|
||||
def test_backlinks_across_two_shards(tmp_path):
|
||||
space = _space(tmp_path)
|
||||
sources = {bl.source for bl in space.backlinks("Guide")}
|
||||
assert sources == {Identity("wiki", "Home"), Identity("notes", "Daily")}
|
||||
|
||||
|
||||
def test_all_pages_and_site_map_over_union(tmp_path):
|
||||
space = _space(tmp_path)
|
||||
names = {e.name for e in space.all_pages()}
|
||||
assert names == {"Home", "Guide", "Daily"}
|
||||
leaves = {p.key for p in space.site_map().pages}
|
||||
assert {"Home", "Guide", "Daily"} <= leaves
|
||||
|
||||
|
||||
def test_recent_changes_includes_alias_and_edits(tmp_path):
|
||||
space = _space(tmp_path)
|
||||
space.alias("Start", "wiki:Home", actor="ana")
|
||||
feed = space.recent_changes()
|
||||
kinds = {e.kind for e in feed}
|
||||
assert "alias" in kinds and "edit" in kinds
|
||||
alias = next(e for e in feed if e.kind == "alias")
|
||||
assert alias.source == "coordination" and alias.actor == "ana"
|
||||
|
||||
|
||||
def test_red_link_creates_no_backlink_via_space(tmp_path):
|
||||
space = _space(tmp_path)
|
||||
assert space.backlinks("Nonexistent") == ()
|
||||
69
tests/test_views_links.py
Normal file
69
tests/test_views_links.py
Normal file
@@ -0,0 +1,69 @@
|
||||
"""Tests for the wikilink + red-link model (SHARD-WP-0010 T1)."""
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.union import ResolutionKind, UnionGraph
|
||||
from shard_wiki.views import extract_links, resolve_links
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def test_extracts_plain_and_labelled_links():
|
||||
links = extract_links("See [[Home]] and [[Index|the index]].")
|
||||
assert [(link.target, link.label, link.text) for link in links] == [
|
||||
("Home", None, "Home"),
|
||||
("Index", "the index", "the index"),
|
||||
]
|
||||
|
||||
|
||||
def test_links_carry_body_offsets_in_document_order():
|
||||
body = "a [[One]] b [[Two]]"
|
||||
links = extract_links(body)
|
||||
assert [link.target for link in links] == ["One", "Two"]
|
||||
s, e = links[0].span
|
||||
assert body[s:e] == "[[One]]"
|
||||
|
||||
|
||||
def test_code_regions_are_not_scanned():
|
||||
body = "real [[Home]]\n```\n[[NotALink]]\n```\ninline `[[AlsoNot]]` done"
|
||||
targets = [link.target for link in extract_links(body)]
|
||||
assert targets == ["Home"]
|
||||
|
||||
|
||||
def test_camelcase_off_by_default_then_opt_in():
|
||||
body = "FrontPage links to [[Home]]"
|
||||
assert [link.target for link in extract_links(body)] == ["Home"] # CamelCase ignored
|
||||
on = extract_links(body, camelcase=True)
|
||||
assert {link.target for link in on} == {"FrontPage", "Home"}
|
||||
assert next(link for link in on if link.target == "FrontPage").auto is True
|
||||
|
||||
|
||||
def test_camelcase_does_not_double_count_inside_explicit_link():
|
||||
# [[FrontPage]] is one explicit link, not also a CamelCase auto-link.
|
||||
links = extract_links("[[FrontPage]]", camelcase=True)
|
||||
assert len(links) == 1
|
||||
assert links[0].auto is False
|
||||
|
||||
|
||||
def test_resolve_links_distinguishes_link_from_red_link(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "home"}))
|
||||
resolved = resolve_links(u, "[[Home]] and [[Ghost]]")
|
||||
by_target = {r.link.target: r for r in resolved}
|
||||
assert by_target["Home"].resolution.kind is ResolutionKind.SINGLE
|
||||
assert by_target["Home"].is_red_link is False
|
||||
assert by_target["Ghost"].is_red_link is True # unresolved → createable red-link
|
||||
|
||||
|
||||
def test_resolve_links_surfaces_chorus(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "A"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"Home.md": "B"}))
|
||||
(resolved,) = resolve_links(u, "[[Home]]")
|
||||
assert resolved.resolution.kind is ResolutionKind.CHORUS
|
||||
67
tests/test_views_recentchanges.py
Normal file
67
tests/test_views_recentchanges.py
Normal file
@@ -0,0 +1,67 @@
|
||||
"""Tests for the RecentChanges merged feed (SHARD-WP-0010 T3)."""
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from shard_wiki.adapters import FolderAdapter
|
||||
from shard_wiki.coordination import DecisionLog, EventType
|
||||
from shard_wiki.union import UnionGraph
|
||||
from shard_wiki.views import recent_changes
|
||||
|
||||
|
||||
def _shard(tmp_path, name, files, mtime=None):
|
||||
root = tmp_path / name
|
||||
for rel, text in files.items():
|
||||
p = root / rel
|
||||
p.parent.mkdir(parents=True, exist_ok=True)
|
||||
p.write_text(text, encoding="utf-8")
|
||||
if mtime is not None:
|
||||
os.utime(p, (mtime, mtime))
|
||||
return FolderAdapter(name, root)
|
||||
|
||||
|
||||
def test_edit_and_alias_both_appear_newest_first(tmp_path):
|
||||
# Page edit signal pinned to an old mtime; the alias decision happens "now" → alias is newest.
|
||||
old = datetime(2020, 1, 1, tzinfo=timezone.utc).timestamp()
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Home.md": "home"}, mtime=old))
|
||||
log = DecisionLog()
|
||||
log.append("space", EventType.ALIAS_SET, {"alias": "Start", "target": "shardA:Home"})
|
||||
|
||||
feed = recent_changes(u, log, "space")
|
||||
kinds = [e.kind for e in feed]
|
||||
assert "edit" in kinds and "alias" in kinds
|
||||
assert feed[0].kind == "alias" # newest first
|
||||
assert feed[-1].kind == "edit"
|
||||
# Monotonic non-increasing by time.
|
||||
assert all(feed[i].when >= feed[i + 1].when for i in range(len(feed) - 1))
|
||||
|
||||
|
||||
def test_per_shard_attribution_present(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"A.md": "a"}))
|
||||
u.attach(_shard(tmp_path, "shardB", {"B.md": "b"}))
|
||||
feed = recent_changes(u, DecisionLog(), "space")
|
||||
edits = {e.ref: e.source for e in feed if e.kind == "edit"}
|
||||
assert edits["shardA:A"] == "shardA"
|
||||
assert edits["shardB:B"] == "shardB" # each edit attributed to its shard
|
||||
|
||||
|
||||
def test_coordination_entries_carry_actor_and_ref(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"Doc.md": "x"}))
|
||||
log = DecisionLog()
|
||||
log.append(
|
||||
"space", EventType.PAGE_FORKED, {"source": "shardA:Doc", "fork": "shardB:Doc"}, actor="ana"
|
||||
)
|
||||
fork = next(e for e in recent_changes(u, log, "space") if e.kind == "fork")
|
||||
assert fork.source == "coordination"
|
||||
assert fork.actor == "ana"
|
||||
assert fork.ref == "shardA:Doc→shardB:Doc"
|
||||
|
||||
|
||||
def test_limit_truncates_to_newest(tmp_path):
|
||||
u = UnionGraph("space")
|
||||
u.attach(_shard(tmp_path, "shardA", {"A.md": "a", "B.md": "b", "C.md": "c"}))
|
||||
feed = recent_changes(u, DecisionLog(), "space", limit=2)
|
||||
assert len(feed) == 2
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0001
|
||||
type: workplan
|
||||
title: "shard-wiki requirements from yawex prior art"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0002
|
||||
type: workplan
|
||||
title: "federation architecture design"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0003
|
||||
type: workplan
|
||||
title: "wiki-engine deep-dive batch (new-insight + git-forge + classic engines)"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0004
|
||||
type: workplan
|
||||
title: "computational / interactive-knowledge systems research"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0005
|
||||
type: workplan
|
||||
title: "core architecture hardening (blueprint review fixes)"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0006
|
||||
type: workplan
|
||||
title: "core architecture hardening II (round-2 review fixes)"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0007
|
||||
type: workplan
|
||||
title: "foundation implementation — model, contract, decision log, union read"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
id: SHARD-WP-0008
|
||||
type: workplan
|
||||
title: "write path — overlay engine, writable adapter, apply-under-drift"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
id: SHARD-WP-0009
|
||||
type: workplan
|
||||
title: "git-backed DecisionLog + per-space append authority"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -39,7 +39,7 @@ sharding (blueprint O-12). Single append authority per space is the target.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T1
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "a8fcbb3e-fbc4-4f68-9cf0-d8a6ee057191"
|
||||
```
|
||||
@@ -54,7 +54,7 @@ ordering preserved; deterministic serialization.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T2
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "62abd162-4243-4659-8d27-9fc967ab11a0"
|
||||
```
|
||||
@@ -69,7 +69,7 @@ hand-off resumes from head; a partitioned non-holder cannot fork the log.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T3
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "8cc3691e-05a7-443f-9292-a3fdf3fd59a4"
|
||||
```
|
||||
@@ -82,7 +82,7 @@ process B (new handle) sees it; fold equals the in-memory fold for the same even
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0009-T4
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "281e1db4-6a75-456b-a2bc-b761feb10609"
|
||||
```
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
id: SHARD-WP-0010
|
||||
type: workplan
|
||||
title: "derived views — wikilinks, BackLinks, RecentChanges, AllPages/SiteMap"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -36,7 +36,7 @@ later by SHARD-WP-0011) and carry provenance. Presentation stays out of core (L6
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0010-T1
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "792660c3-9be9-4771-9f51-69d01f0c7f13"
|
||||
```
|
||||
@@ -51,7 +51,7 @@ red-link, CamelCase opt-in.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0010-T2
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "431a54c3-82b5-4b08-b3f0-762624d4c91d"
|
||||
```
|
||||
@@ -65,7 +65,7 @@ chorus pages aggregate.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0010-T3
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "270c1c31-0445-42b9-9a49-92d32c298eb2"
|
||||
```
|
||||
@@ -79,7 +79,7 @@ alias both appear, newest-first; per-shard attribution present.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0010-T4
|
||||
status: todo
|
||||
status: done
|
||||
priority: low
|
||||
state_hub_task_id: "898ba43e-cdef-4ce8-9fa3-4ce60ebb4fdd"
|
||||
```
|
||||
@@ -92,7 +92,7 @@ collapses to one entry with divergence noted; sitemap reflects paths.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0010-T5
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "7157544b-5d3b-45a2-ba5a-c32244c59323"
|
||||
```
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
id: SHARD-WP-0011
|
||||
type: workplan
|
||||
title: "incremental union maintenance + equivalence index + I-2 verification"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -41,7 +41,7 @@ deployment is later.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0011-T1
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "842f480b-7b14-47cd-818b-012dbda9c187"
|
||||
```
|
||||
@@ -55,7 +55,7 @@ unrelated pages don't; verified edges match a brute-force oracle on a small corp
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0011-T2
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "2da4e0b8-22cc-4ad1-a9aa-b5e991515d30"
|
||||
```
|
||||
@@ -70,7 +70,7 @@ stale edge.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0011-T3
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "b602ce31-ad9a-4c7f-b596-f039722373fc"
|
||||
```
|
||||
@@ -85,7 +85,7 @@ equivalent event orders.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0011-T4
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "2f3d083c-0b2e-4b58-9e96-c0461c5eb089"
|
||||
```
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
id: SHARD-WP-0012
|
||||
type: workplan
|
||||
title: "second adapter — git-IS-store shard (contract validation on a new substrate)"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -40,7 +40,7 @@ merge beyond fast-forward (apply-under-drift refuse is enough, as in SHARD-WP-00
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0012-T1
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "8a1c7c80-a0cc-4e02-a611-1f1fd7dec57b"
|
||||
```
|
||||
@@ -54,7 +54,7 @@ implication rules. Tests: read tracked files; profile validates; conformance rea
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0012-T2
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "b47dfb86-46c1-4e97-a62f-377719499ff2"
|
||||
```
|
||||
@@ -68,7 +68,7 @@ changes after an external commit.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0012-T3
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "4c895f42-671d-4948-8bdf-941fd85644bb"
|
||||
```
|
||||
|
||||
@@ -2,9 +2,9 @@
|
||||
id: SHARD-WP-0013
|
||||
type: workplan
|
||||
title: "wiki-engine prep — reuse-surface registration, UC-catalog systematization, WikiEngineCoreArchitecture"
|
||||
domain: whynot
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: active
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
@@ -123,7 +123,7 @@ state hub").
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0013-T4
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "1d0ef72b-2762-4086-9848-fde3b48c8454"
|
||||
```
|
||||
@@ -140,7 +140,7 @@ auth-in-core amendment.
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0013-T5
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "4712bbfe-4ff3-4631-a9fb-e8857e1c0a2c"
|
||||
```
|
||||
@@ -163,7 +163,7 @@ exotic case possible (mirror the CoreArchitectureBlueprint discipline).
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0013-T6
|
||||
status: todo
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "1c383414-2c8b-41c4-957d-8d1a9ed88143"
|
||||
```
|
||||
|
||||
155
workplans/SHARD-WP-0014-engine-implementation.md
Normal file
155
workplans/SHARD-WP-0014-engine-implementation.md
Normal file
@@ -0,0 +1,155 @@
|
||||
---
|
||||
id: SHARD-WP-0014
|
||||
type: workplan
|
||||
title: "wiki-engine implementation — kernel + typed-extension runtime + activation"
|
||||
domain: consumer
|
||||
repo: shard-wiki
|
||||
status: done
|
||||
owner: tegwick
|
||||
topic_slug: whynot
|
||||
created: "2026-06-15"
|
||||
updated: "2026-06-15"
|
||||
depends_on:
|
||||
- SHARD-WP-0007
|
||||
- SHARD-WP-0013
|
||||
state_hub_workstream_id: "bfce1644-d93d-44c7-af2c-6b0cb50cedd4"
|
||||
---
|
||||
|
||||
# SHARD-WP-0014 — Wiki-engine implementation
|
||||
|
||||
## Goal
|
||||
|
||||
Implement the native **headless wiki engine** specified in `spec/WikiEngineCoreArchitecture.md`:
|
||||
a **small page-store kernel** + a **stringent typed-extension runtime**, with **per-shard
|
||||
activation** (ADR-0001: via feature-control/OpenFeature, LocalProvider default), the engine's
|
||||
**§A capability profile derived from active extensions** (E-5), and exposure as a
|
||||
**canonical-mode shard** (`EngineShardAdapter`). Target capability: **stand up an engine shard,
|
||||
activate a chosen extension set, and attach it to an `InformationSpace` — its declared
|
||||
capability profile reflecting exactly what is active** — proven end-to-end with one real
|
||||
built-in extension.
|
||||
|
||||
**Non-goal (this slice):** the headless network API protocol; git-IS-store backing (kernel uses
|
||||
the existing simple store now; git backing integrates with SHARD-WP-0009/0012 later);
|
||||
computational extensions; the full feature-control control plane. Build the framework, prove it.
|
||||
|
||||
## Guiding rules (from WikiEngineCoreArchitecture)
|
||||
|
||||
- E-1 engine is **one shard**, not a federation layer. E-2 small kernel. E-3 everything-else is a
|
||||
typed extension. E-4 per-shard activation. E-5 capability profile derived from active
|
||||
extensions. E-8 reuse (feature-control activation; model/provenance/coordination/adapters).
|
||||
E-9 extensions are typed + conformance-verified.
|
||||
- Honour the §11 dependency rule: `engine/` consumes `model/`, `provenance/`, `coordination/`,
|
||||
`adapters/`, `policy/`; it is consumed only via its `EngineShardAdapter`. No orchestrator-tier
|
||||
(`union/`, `projection/`) import. Pure-stdlib core; OpenFeature is an optional engine extra.
|
||||
|
||||
---
|
||||
|
||||
## Engine kernel skeleton
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T1
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "e81ba881-7e92-4581-99ff-b12ad2bcabb3"
|
||||
```
|
||||
|
||||
`src/shard_wiki/engine/kernel.py`: the minimal kernel — a page store + lifecycle over existing
|
||||
primitives (reuse `model.Page`/`Identity`/`provenance`; a simple in-memory/folder-backed store
|
||||
now, git-IS-store later) and a recoverable history hook into the decision log. Kernel covers the
|
||||
c2-minimum (create/read/edit-as-history; `[[wikilink]]`+red-link resolution can be a thin kernel
|
||||
helper). No extensions yet. Tests: page CRUD-as-history; kernel-only shard works.
|
||||
|
||||
## Typed-extension runtime
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T2
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "8ae8e58a-f081-432b-b2c5-b6435fbf3843"
|
||||
```
|
||||
|
||||
`src/shard_wiki/engine/extension.py`: the `Extension` contract (id, provides, types, hooks,
|
||||
depends_on, conflicts_with, config), a registry, a **typed hook dispatcher** (typed
|
||||
inputs/outputs, declared deterministic order), a **type checker**, and **composition** that
|
||||
builds the dependency closure and **rejects impossible profiles** (conflicts / unmet deps /
|
||||
incompatible types) — the §6.5 discipline for extensions. Extensions ship a conformance check
|
||||
(mirrors §6.6). Tests: register/compose; deterministic hook order; impossible profile rejected;
|
||||
conformance catches a lying extension.
|
||||
|
||||
## Per-shard activation (feature-control / OpenFeature)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T3
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "c4fe9df4-e6a8-4b7d-891b-59ceec6aebac"
|
||||
```
|
||||
|
||||
`src/shard_wiki/engine/activation.py` (ADR-0001): resolve a shard's **activation profile**
|
||||
(`{extension id → config}`) through an OpenFeature-shaped client with a **static LocalProvider
|
||||
default** (standalone, zero external dep); context = `{tenant_id: root, shard_id, …}`; OpenFeature/
|
||||
feature-control is an optional provider plugged in when present (degrade gracefully, mirror the
|
||||
identity ladder). **Availability only — never authorization.** Tests: LocalProvider activates a
|
||||
subset; absent-provider falls back to defaults; context scoping works.
|
||||
|
||||
## Capability profile derived from active extensions (E-5)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T4
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "15fc8db7-cd80-4675-b387-81aa9bc7d308"
|
||||
```
|
||||
|
||||
`src/shard_wiki/engine/profile.py`: fold the active extensions' `on_profile` contributions into a
|
||||
§A `CapabilityProfile` (e.g. `ext.struct` active ⟹ structure spectrum rises + `structured-payload`
|
||||
verb), then `validate()` + conformance. Tests: activating an extension changes the derived
|
||||
profile; the derived profile is valid and conformance-passes.
|
||||
|
||||
## EngineShardAdapter (engine as a canonical-mode shard)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T5
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "2fbf498c-efe9-400a-8a13-7f1b521b3534"
|
||||
```
|
||||
|
||||
`src/shard_wiki/engine/adapter.py`: `EngineShardAdapter` implements `adapters.ShardAdapter`,
|
||||
backed by the kernel + active extensions, declaring the derived profile (T4). Attach an engine
|
||||
shard to an `InformationSpace` and read/resolve/edit through it like any shard. Tests +
|
||||
integration: engine shard passes `assert_conformant`; attach → resolve → edit works.
|
||||
|
||||
## First built-in extension (prove the framework end-to-end)
|
||||
|
||||
```task
|
||||
id: SHARD-WP-0014-T6
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "b88d1640-9afa-4957-aec3-a7264b09494c"
|
||||
```
|
||||
|
||||
Implement one real extension end-to-end — **`ext.views` (BackLinks)** or **`ext.struct`
|
||||
(typed records)** — binding kernel hooks, declaring types, contributing to the derived profile,
|
||||
activatable per shard. Integration test: with the extension OFF the capability is absent (honest
|
||||
profile); ON it works and the profile reflects it. Update SCOPE + spec/README; `pytest` +
|
||||
pyflakes green.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `pytest` green, pyflakes clean; engine core pure-stdlib (OpenFeature optional, behind the
|
||||
LocalProvider default).
|
||||
- The vertical slice works: stand up an engine shard, activate a chosen extension set, attach to
|
||||
an `InformationSpace`; the engine's declared §A profile **matches the active extensions** and
|
||||
passes conformance.
|
||||
- Module boundaries honour §11 (engine consumed only via `EngineShardAdapter`; no union/projection
|
||||
import); E-1…E-9 respected.
|
||||
- Activation is availability-only (no authz); standalone path has no external dependency.
|
||||
- Each task committed; state-hub synced.
|
||||
|
||||
## Suggested task order
|
||||
|
||||
T1 kernel → T2 extension runtime → T3 activation → T4 derived profile → T5 EngineShardAdapter →
|
||||
T6 first extension + integration.
|
||||
Reference in New Issue
Block a user