Add documentation: root README and state-hub/README

Root README covers: architecture, domain table with topic IDs, quick start, project registration, Claude Code integration, governance summary, roadmap, and design principles. state-hub/README covers: full setup guide, Makefile targets, DB schema with governance constraints, API summary (incl. /state/summary shape), MCP server config, custodian CLI reference, dashboard pages, and WSL2 known issues. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 23:00:20 +01:00
parent ae10aaf33f
commit 2c3687d1c9
2 changed files with 423 additions and 20 deletions
--- a/README.md
+++ b/README.md
@@ -2,31 +2,192 @@ Confidential and Proprietary. Authorized Use Only. Subject to NDA & Contractual

 # The Custodian

-Transgenerational Cognitive Infrastructure
+**Transgenerational Cognitive Infrastructure** — a local-first, sovereignty-preserving agent system for co-creating and stewarding knowledge across six project domains.

-# Custodian Repo Skeleton (v0.1)
+The Custodian acts as co-creator and steward, not authority. Humans approve all irreversible decisions. The system is designed to still be coherent decades from now.

-This repository is an initial, local-first canon + governance skeleton for the Custodian project and its top-level domains:
+---

- Railiance
- Markitect
- Coulomb.social
- Foerster Capabilities
- Personhood
- Custodian
+## Architecture

-## Structure
+```
+the-custodian/
+├── canon/              # Curated, reviewable knowledge substrate
+│   ├── constitution/   # Governance rules (v0.1)
+│   ├── values/         # Nine foundational principles
+│   └── projects/       # Six domain charters, concept seeds, roadmaps
+├── memory/             # Operational logs — append-only, never rewritten
+│   ├── working/        # Session notes (scoped, time-bounded)
+│   └── episodic/       # Immutable event archive
+├── state-hub/          # Live state service (the operational brain)
+│   ├── api/            # FastAPI (PostgreSQL-backed REST + /state/summary)
+│   ├── mcp_server/     # FastMCP stdio — Claude Code's native interface
+│   ├── migrations/     # Alembic schema migrations
+│   ├── dashboard/      # Observable Framework telemetry dashboard
+│   ├── infra/          # docker-compose.yml (postgres + optional pgadmin)
+│   └── scripts/        # seed.py, register_project.sh, custodian CLI
+├── runtime/            # Agent runtime scaffolding (policies, prompts, adapters)
+├── infra/              # Deployment, backups, encryption scaffolding
+└── eval/               # Policy and regression test placeholders
+```

- `canon/` — curated, reviewable knowledge substrate (identity lives here)
- `memory/` — working and episodic logs (operational + append-only history)
- `eval/` — policy and regression tests (future automation)
- `runtime/` — policies/prompts/tool adapters (agent runtime scaffolding)
- `infra/` — deployment, backups, encryption scaffolding
+The **dependency chain** across domains runs bottom-up:

-## Workflow (suggested)
+```
+Railiance → Markitect → Coulomb.social → Personhood / Foerster → Custodian
+(ops)        (canon)     (interaction)     (rights/agency)        (integration)
+```

-1. Edit canon artifacts in `canon/`
-2. Propose changes via PR/patch workflow
-3. Tag and sign periodic `Canon Releases`
+---

-> v0.1 focuses on schemas, governance, and initial charters/roadmaps/concepts.
+## Project Domains
+
+| Domain | Purpose | Topic ID |
+|--------|---------|----------|
+| **Custodian** | Master agent system; integrates all layers | `cee7bedf-2b48-46ef-8601-006474f2ad7a` |
+| **Railiance** | Sovereign DevOps and operational reliability | `ca369340-a64e-442e-98f1-a4fa7dc74a38` |
+| **Markitect** | Knowledge artifact management: authoring, versioning, retrieval | `5571d954-0d30-4950-980d-7bcaaad8e3e2` |
+| **Coulomb.social** | Co-creation marketplace and governance experiment | `36c7421b-c537-4723-bf75-42a3ebc6a1dc` |
+| **Personhood** | Rights and obligations framework for mixed-intelligence societies | `084430ab-c630-48dc-9e1d-d07d1e8fce3c` |
+| **Foerster Capabilities** | Agency capability taxonomy (Foerster's Non-Trivial Machines) | `64418556-3206-457a-ba29-6884b5b12cf3` |
+
+Each domain has three canon artifacts under `canon/projects/<domain>/`:
+- `project_charter_v0.1.md` — purpose, problem, scope, success criteria
+- `concepts_seed_v0.1.md` — ten foundational concepts
+- `roadmap_v0.1.md` — multi-phase implementation plan
+
+---
+
+## State Hub — Quick Start
+
+The State Hub is the live operational layer: a PostgreSQL database, a FastAPI REST service, an MCP server for Claude Code, and an Observable dashboard.
+
+### Prerequisites
+
+- Docker Engine (WSL2: see `CLAUDE.md` → Docker Setup)
+- Python 3.12+ with `uv` (`pip install uv`)
+- Node.js 18+ (for dashboard only)
+
+### First-time setup
+
+```bash
+cd state-hub
+
+cp .env.example .env          # set POSTGRES_PASSWORD
+make install                  # uv sync → Python deps + custodian CLI in .venv
+make install-cli               # symlink custodian to ~/.local/bin
+make db                       # start postgres on 127.0.0.1:5432
+make migrate                  # alembic upgrade head
+make seed                     # insert 6 canonical topics
+make api                      # uvicorn on 127.0.0.1:8000
+```
+
+API docs: http://127.0.0.1:8000/docs
+Health check: `make check`
+
+### Shortcut
+
+```bash
+make start    # db + migrate + api (all in one)
+```
+
+### Dashboard
+
+```bash
+cd state-hub
+make dashboard    # Observable Framework dev server on :3000
+```
+
+---
+
+## Connecting a Project
+
+Any repository can be registered as a Custodian project in one command:
+
+```bash
+cd /path/to/your-project
+custodian register-project --domain railiance
+```
+
+This will:
+1. Verify the API is running
+2. Look up the topic ID for the domain
+3. Check that the MCP server is registered in `~/.claude.json`
+4. Write a `CLAUDE.md` into the project root
+5. Post a progress event to the State Hub
+
+Domain is auto-detected from `project_charter_v*.md` frontmatter if `--domain` is omitted.
+
+```bash
+custodian status              # health + summary totals + blocking decisions
+custodian register-project    # full registration from cwd
+```
+
+---
+
+## Claude Code Integration
+
+The `state-hub` MCP server is registered at user scope in `~/.claude.json`.
+It exposes 11 tools and 5 resources directly in every Claude Code session.
+
+**Session protocol** (enforced via `~/.claude/CLAUDE.md`):
+- Start: `get_state_summary()` — orientation snapshot
+- End: `add_progress_event()` — append to the immutable log
+
+Tool reference: `state-hub/mcp_server/TOOLS.md`
+
+If the MCP server is missing from a session: check `~/.claude/CLAUDE.md` → MCP Server Registration.
+
+---
+
+## Governance
+
+The constitution (`canon/constitution/custodian_constitution_v0.1.md`) defines hard boundaries:
+
+**The Custodian may without approval:**
+- Draft documents, plans, and structured artifacts
+- Read/search canon and approved repositories
+- Propose canon updates as PRs/patches (never direct writes)
+- Create working-memory notes and session summaries
+
+**Never permitted (v0.1):**
+- Financial transactions or legal commitments
+- External publication under Bernd's identity
+- Storing secrets/credentials in plaintext
+- Writing directly to `canon/` without a review gate
+
+**Must escalate when:**
+- Actions affect money, legal status, security, or external reputation
+- Instructions conflict with values or the constitution
+- Consent is unclear, especially in family-scoped data
+
+### Canon Promotion Workflow
+
+```
+Custodian proposes → run gates (attribution, consistency, clarity, sensitivity, reversibility)
+                   → human approves → merge
+```
+
+All canon changes carry provenance metadata. Episodic memory is append-only.
+
+---
+
+## Roadmap
+
+| Phase | Focus | Status |
+|-------|-------|--------|
+| 0 | Canon + constitution + State Hub v0.1 | **done** |
+| 1 | RAG over canon, drafting pipelines, consistency checks | planned |
+| 2 | Stewardship automation (health checks, drift detection) | planned |
+| 3 | Sovereign appliance (local inference on dedicated hardware) | future |
+| 4 | Family custodianship (consent model, vault, succession) | future |
+
+---
+
+## Key Design Principles
+
+- **Local-first, degrade-gracefully** — no vendor lock-in; can operate offline
+- **Auditability and reversibility** — explicit gates; proposals precede changes
+- **Safety by design** — Custodian is co-creator, not authority; humans approve irreversible decisions
+- **Targeted information processing** — narrow, high-leverage work over general intelligence
+- **Long timescale stewardship** — designed for multi-year and eventual multi-generational continuity
--- a/state-hub/README.md
+++ b/state-hub/README.md
@@ -0,0 +1,242 @@
+# State Hub v0.1
+
+The operational brain of the Custodian: a local PostgreSQL database, FastAPI REST service, FastMCP stdio server for Claude Code, Observable Framework dashboard, and a `custodian` CLI.
+
+---
+
+## Stack
+
+| Layer | Technology | Port |
+|-------|-----------|------|
+| Database | PostgreSQL 16-alpine (Docker) | `127.0.0.1:5432` |
+| API | FastAPI + SQLAlchemy 2.0 async + asyncpg | `127.0.0.1:8000` |
+| MCP server | FastMCP stdio (Claude Code native) | stdio |
+| Dashboard | Observable Framework | `127.0.0.1:3000` |
+| CLI | `custodian` (Python, uv entry point) | — |
+
+All services bind to `127.0.0.1` only — nothing exposed to the network.
+
+---
+
+## Setup
+
+### Prerequisites
+
+- Docker Engine (WSL2: see `CLAUDE.md` in repo root → Docker Setup)
+- Python 3.12+ with `uv` (`pip install uv`)
+- Node.js 18+ (dashboard only)
+
+### First-time
+
+```bash
+cd state-hub
+
+cp .env.example .env          # edit POSTGRES_PASSWORD
+make install                  # uv sync
+make db                       # docker compose up postgres
+make migrate                  # alembic upgrade head (creates 5 tables)
+make seed                     # insert 6 canonical topics
+make api                      # uvicorn :8000 --reload
+```
+
+### Shortcut
+
+```bash
+make start    # db + sleep + migrate + api
+```
+
+### Dashboard
+
+```bash
+make dashboard    # Observable dev server on :3000
+```
+
+### CLI
+
+```bash
+make install-cli              # symlink .venv/bin/custodian → ~/.local/bin
+custodian status              # API health + summary totals
+custodian register-project    # register cwd as a Custodian project
+```
+
+---
+
+## Makefile Targets
+
+| Target | What it does |
+|--------|-------------|
+| `make install` | `uv sync` — install Python deps + entry points |
+| `make install-cli` | Symlink `custodian` to `~/.local/bin` |
+| `make db` | Start postgres container |
+| `make db-tools` | Start postgres + pgadmin (http://127.0.0.1:5050) |
+| `make migrate` | `alembic upgrade head` |
+| `make seed` | Insert 6 canonical topics |
+| `make api` | `uvicorn api.main:app --reload` |
+| `make dashboard` | Observable dev server |
+| `make check` | `curl /state/health` |
+| `make start` | `db` + wait + `migrate` + `api` |
+| `make register-project DOMAIN=x PROJECT_PATH=y` | Register a project |
+| `make clean` | `docker compose down -v` (destroys DB volume) |
+
+---
+
+## Database Schema
+
+Five tables in dependency order:
+
+```
+topics
+└── workstreams
+    └── tasks (self-FK: parent_task_id)
+        └── progress_events
+decisions (FK: topic_id, workstream_id — at least one required)
+    └── progress_events
+```
+
+### Enums
+
+| Enum | Values |
+|------|--------|
+| `topic_status` | `active` · `paused` · `archived` |
+| `workstream_status` | `active` · `blocked` · `completed` · `archived` |
+| `task_status` | `todo` · `in_progress` · `blocked` · `done` · `cancelled` |
+| `task_priority` | `low` · `medium` · `high` · `critical` |
+| `decision_type` | `made` · `pending` |
+| `decision_status` | `open` · `resolved` · `escalated` · `superseded` |
+| `domain` | `custodian` · `railiance` · `markitect` · `coulomb_social` · `personhood` · `foerster_capabilities` |
+
+### Governance constraints encoded in schema
+
+- No hard DELETE endpoints — only soft: `archived`, `cancelled`, `superseded`
+- `progress_events` has no `updated_at` and no DELETE endpoint (append-only per constitution §5)
+- `decisions` with financial/legal keywords + `pending` type → auto-set `escalation_note` (§4)
+
+---
+
+## API
+
+Interactive docs at http://127.0.0.1:8000/docs once the API is running.
+
+### Key endpoint: `/state/summary`
+
+Returns a full snapshot in one call — used by both the MCP server and dashboard:
+
+```json
+{
+  "generated_at": "...",
+  "totals": {
+    "topics":      { "active": 6, "paused": 0, "archived": 0, "total": 6 },
+    "workstreams": { "active": 1, "blocked": 0, "completed": 1, "total": 2 },
+    "tasks":       { "todo": 9, "in_progress": 0, "blocked": 0, "done": 11, "total": 20 },
+    "decisions":   { "open": 1, "resolved": 0, "escalated": 0, "total": 1 }
+  },
+  "topics": [...],             // topics with nested workstream stubs
+  "blocking_decisions": [...], // pending decisions only
+  "blocked_tasks": [...],
+  "recent_progress": [...],    // last 20 events
+  "open_workstreams": [...]
+}
+```
+
+### Router summary
+
+| Prefix | Operations |
+|--------|-----------|
+| `/topics` | CRUD (soft-delete: `archived`) |
+| `/workstreams` | CRUD (soft-delete: `archived`) |
+| `/tasks` | CRUD (soft-delete: `cancelled`); `PATCH` updates status |
+| `/decisions` | CRUD (soft-delete: `superseded`); auto-escalation |
+| `/progress` | `GET` list + `POST` append — no DELETE |
+| `/state/summary` | Full snapshot |
+| `/state/health` | DB connectivity check |
+
+---
+
+## MCP Server
+
+Registered in `~/.claude.json` at user scope. Config in `.mcp.json` (repo root).
+
+Uses absolute path + `PYTHONPATH` so `cwd` is not required:
+```json
+{
+  "command": "/home/worsch/the-custodian/state-hub/.venv/bin/python",
+  "args": ["/home/worsch/the-custodian/state-hub/mcp_server/server.py"],
+  "env": { "PYTHONPATH": "/home/worsch/the-custodian/state-hub", "API_BASE": "http://127.0.0.1:8000" }
+}
+```
+
+See `mcp_server/TOOLS.md` for the full tool reference card (30 lines, faster than reading `server.py`).
+
+### Tools at a glance
+
+**Query** (read-only): `get_state_summary` · `get_topic` · `list_blocked_tasks` · `list_pending_decisions` · `get_recent_progress`
+
+**Mutate** (each auto-emits a progress event): `create_task` · `update_task_status` · `record_decision` · `resolve_decision` · `add_progress_event` · `update_workstream_status`
+
+**Resources**: `state://summary` · `state://topics` · `state://workstreams/{topic_slug}` · `state://decisions/blocking` · `state://tasks/blocked`
+
+---
+
+## `custodian` CLI
+
+Installed into `.venv/bin/custodian` by `uv sync`; symlinked to `~/.local/bin` by `make install-cli`.
+
+```
+custodian register-project [--domain DOMAIN] [--path PATH]
+```
+
+- `--path` defaults to current working directory
+- `--domain` is auto-detected from `project_charter_v*.md` frontmatter if omitted
+
+```
+custodian status
+```
+
+Prints API health, totals, and any blocking decisions.
+
+### What `register-project` does
+
+1. Verifies the API is reachable (fails fast with `make api` hint)
+2. Looks up the topic ID for the domain via `/topics/?status=active`
+3. Checks that `state-hub` is in `~/.claude.json`
+4. Writes `$PROJECT_PATH/CLAUDE.md` from `scripts/project_claude_md.template`
+5. Posts a `milestone` progress event recording the registration
+
+---
+
+## Project Registration Scripts
+
+| Script | Purpose |
+|--------|---------|
+| `scripts/register_project.sh` | Shell version of `custodian register-project` |
+| `scripts/patch_mcp_cwd.py` | Patches `cwd` into `~/.claude.json` after `claude mcp add-json` drops it |
+| `scripts/project_claude_md.template` | CLAUDE.md template with `{PROJECT_NAME}`, `{DOMAIN}`, `{TOPIC_ID}` |
+| `scripts/seed.py` | Insert the 6 canonical topics into a fresh database |
+| `scripts/pull_image.py` | WSL2 workaround: pull Docker images via Python urllib with Range-request chunking |
+
+---
+
+## Dashboard
+
+Four pages at http://127.0.0.1:3000 (dev) or built with `npm run build`:
+
+| Page | Content |
+|------|---------|
+| **Overview** | Status cards, task-by-status chart, recent activity feed, decisions due within 7 days |
+| **Workstreams** | Filterable table by domain/status/owner; selected workstream task list; progress timeline |
+| **Decisions** | Pending tab (with escalation highlights) and Made tab; resolution velocity chart |
+| **Progress** | Append-only event feed with author badges; 30-day event volume chart |
+
+Data loaders (`src/data/*.json.py`) are Python scripts that call the local API. They run at dev-server start and on `npm run build`. Clear the cache if data appears stale:
+
+```bash
+rm -rf dashboard/src/.observablehq/cache/
+```
+
+---
+
+## Known Issues / WSL2 Notes
+
+- **TLS bad record MAC on large downloads**: WSL2 corrupts packets on big TCP transfers. Use `scripts/pull_image.py` instead of `docker pull` for future image pulls.
+- **`claude mcp add-json` drops `cwd`**: Known Claude Code bug. Run `python3 scripts/patch_mcp_cwd.py` after any re-registration. The current `.mcp.json` uses absolute path + `PYTHONPATH` so `cwd` is not strictly needed.
+- **AsyncSession concurrency**: SQLAlchemy 2.0 async sessions don't support concurrent operations. All queries in `/state/summary` run sequentially on a single session.