From 2c3687d1c9726cb60eba1e5fd0fdfe184ca44b4c Mon Sep 17 00:00:00 2001 From: tegwick Date: Tue, 24 Feb 2026 23:00:20 +0100 Subject: [PATCH] Add documentation: root README and state-hub/README Root README covers: architecture, domain table with topic IDs, quick start, project registration, Claude Code integration, governance summary, roadmap, and design principles. state-hub/README covers: full setup guide, Makefile targets, DB schema with governance constraints, API summary (incl. /state/summary shape), MCP server config, custodian CLI reference, dashboard pages, and WSL2 known issues. Co-Authored-By: Claude Sonnet 4.6 --- README.md | 201 ++++++++++++++++++++++++++++++++---- state-hub/README.md | 242 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 423 insertions(+), 20 deletions(-) create mode 100644 state-hub/README.md diff --git a/README.md b/README.md index 7398f8a..7a79d95 100644 --- a/README.md +++ b/README.md @@ -2,31 +2,192 @@ Confidential and Proprietary. Authorized Use Only. Subject to NDA & Contractual # The Custodian -Transgenerational Cognitive Infrastructure +**Transgenerational Cognitive Infrastructure** — a local-first, sovereignty-preserving agent system for co-creating and stewarding knowledge across six project domains. -# Custodian Repo Skeleton (v0.1) +The Custodian acts as co-creator and steward, not authority. Humans approve all irreversible decisions. The system is designed to still be coherent decades from now. -This repository is an initial, local-first canon + governance skeleton for the Custodian project and its top-level domains: +--- -- Railiance -- Markitect -- Coulomb.social -- Foerster Capabilities -- Personhood -- Custodian +## Architecture -## Structure +``` +the-custodian/ +├── canon/ # Curated, reviewable knowledge substrate +│ ├── constitution/ # Governance rules (v0.1) +│ ├── values/ # Nine foundational principles +│ └── projects/ # Six domain charters, concept seeds, roadmaps +├── memory/ # Operational logs — append-only, never rewritten +│ ├── working/ # Session notes (scoped, time-bounded) +│ └── episodic/ # Immutable event archive +├── state-hub/ # Live state service (the operational brain) +│ ├── api/ # FastAPI (PostgreSQL-backed REST + /state/summary) +│ ├── mcp_server/ # FastMCP stdio — Claude Code's native interface +│ ├── migrations/ # Alembic schema migrations +│ ├── dashboard/ # Observable Framework telemetry dashboard +│ ├── infra/ # docker-compose.yml (postgres + optional pgadmin) +│ └── scripts/ # seed.py, register_project.sh, custodian CLI +├── runtime/ # Agent runtime scaffolding (policies, prompts, adapters) +├── infra/ # Deployment, backups, encryption scaffolding +└── eval/ # Policy and regression test placeholders +``` -- `canon/` — curated, reviewable knowledge substrate (identity lives here) -- `memory/` — working and episodic logs (operational + append-only history) -- `eval/` — policy and regression tests (future automation) -- `runtime/` — policies/prompts/tool adapters (agent runtime scaffolding) -- `infra/` — deployment, backups, encryption scaffolding +The **dependency chain** across domains runs bottom-up: -## Workflow (suggested) +``` +Railiance → Markitect → Coulomb.social → Personhood / Foerster → Custodian +(ops) (canon) (interaction) (rights/agency) (integration) +``` -1. Edit canon artifacts in `canon/` -2. Propose changes via PR/patch workflow -3. Tag and sign periodic `Canon Releases` +--- -> v0.1 focuses on schemas, governance, and initial charters/roadmaps/concepts. +## Project Domains + +| Domain | Purpose | Topic ID | +|--------|---------|----------| +| **Custodian** | Master agent system; integrates all layers | `cee7bedf-2b48-46ef-8601-006474f2ad7a` | +| **Railiance** | Sovereign DevOps and operational reliability | `ca369340-a64e-442e-98f1-a4fa7dc74a38` | +| **Markitect** | Knowledge artifact management: authoring, versioning, retrieval | `5571d954-0d30-4950-980d-7bcaaad8e3e2` | +| **Coulomb.social** | Co-creation marketplace and governance experiment | `36c7421b-c537-4723-bf75-42a3ebc6a1dc` | +| **Personhood** | Rights and obligations framework for mixed-intelligence societies | `084430ab-c630-48dc-9e1d-d07d1e8fce3c` | +| **Foerster Capabilities** | Agency capability taxonomy (Foerster's Non-Trivial Machines) | `64418556-3206-457a-ba29-6884b5b12cf3` | + +Each domain has three canon artifacts under `canon/projects//`: +- `project_charter_v0.1.md` — purpose, problem, scope, success criteria +- `concepts_seed_v0.1.md` — ten foundational concepts +- `roadmap_v0.1.md` — multi-phase implementation plan + +--- + +## State Hub — Quick Start + +The State Hub is the live operational layer: a PostgreSQL database, a FastAPI REST service, an MCP server for Claude Code, and an Observable dashboard. + +### Prerequisites + +- Docker Engine (WSL2: see `CLAUDE.md` → Docker Setup) +- Python 3.12+ with `uv` (`pip install uv`) +- Node.js 18+ (for dashboard only) + +### First-time setup + +```bash +cd state-hub + +cp .env.example .env # set POSTGRES_PASSWORD +make install # uv sync → Python deps + custodian CLI in .venv +make install-cli # symlink custodian to ~/.local/bin +make db # start postgres on 127.0.0.1:5432 +make migrate # alembic upgrade head +make seed # insert 6 canonical topics +make api # uvicorn on 127.0.0.1:8000 +``` + +API docs: http://127.0.0.1:8000/docs +Health check: `make check` + +### Shortcut + +```bash +make start # db + migrate + api (all in one) +``` + +### Dashboard + +```bash +cd state-hub +make dashboard # Observable Framework dev server on :3000 +``` + +--- + +## Connecting a Project + +Any repository can be registered as a Custodian project in one command: + +```bash +cd /path/to/your-project +custodian register-project --domain railiance +``` + +This will: +1. Verify the API is running +2. Look up the topic ID for the domain +3. Check that the MCP server is registered in `~/.claude.json` +4. Write a `CLAUDE.md` into the project root +5. Post a progress event to the State Hub + +Domain is auto-detected from `project_charter_v*.md` frontmatter if `--domain` is omitted. + +```bash +custodian status # health + summary totals + blocking decisions +custodian register-project # full registration from cwd +``` + +--- + +## Claude Code Integration + +The `state-hub` MCP server is registered at user scope in `~/.claude.json`. +It exposes 11 tools and 5 resources directly in every Claude Code session. + +**Session protocol** (enforced via `~/.claude/CLAUDE.md`): +- Start: `get_state_summary()` — orientation snapshot +- End: `add_progress_event()` — append to the immutable log + +Tool reference: `state-hub/mcp_server/TOOLS.md` + +If the MCP server is missing from a session: check `~/.claude/CLAUDE.md` → MCP Server Registration. + +--- + +## Governance + +The constitution (`canon/constitution/custodian_constitution_v0.1.md`) defines hard boundaries: + +**The Custodian may without approval:** +- Draft documents, plans, and structured artifacts +- Read/search canon and approved repositories +- Propose canon updates as PRs/patches (never direct writes) +- Create working-memory notes and session summaries + +**Never permitted (v0.1):** +- Financial transactions or legal commitments +- External publication under Bernd's identity +- Storing secrets/credentials in plaintext +- Writing directly to `canon/` without a review gate + +**Must escalate when:** +- Actions affect money, legal status, security, or external reputation +- Instructions conflict with values or the constitution +- Consent is unclear, especially in family-scoped data + +### Canon Promotion Workflow + +``` +Custodian proposes → run gates (attribution, consistency, clarity, sensitivity, reversibility) + → human approves → merge +``` + +All canon changes carry provenance metadata. Episodic memory is append-only. + +--- + +## Roadmap + +| Phase | Focus | Status | +|-------|-------|--------| +| 0 | Canon + constitution + State Hub v0.1 | **done** | +| 1 | RAG over canon, drafting pipelines, consistency checks | planned | +| 2 | Stewardship automation (health checks, drift detection) | planned | +| 3 | Sovereign appliance (local inference on dedicated hardware) | future | +| 4 | Family custodianship (consent model, vault, succession) | future | + +--- + +## Key Design Principles + +- **Local-first, degrade-gracefully** — no vendor lock-in; can operate offline +- **Auditability and reversibility** — explicit gates; proposals precede changes +- **Safety by design** — Custodian is co-creator, not authority; humans approve irreversible decisions +- **Targeted information processing** — narrow, high-leverage work over general intelligence +- **Long timescale stewardship** — designed for multi-year and eventual multi-generational continuity diff --git a/state-hub/README.md b/state-hub/README.md new file mode 100644 index 0000000..1e225de --- /dev/null +++ b/state-hub/README.md @@ -0,0 +1,242 @@ +# State Hub v0.1 + +The operational brain of the Custodian: a local PostgreSQL database, FastAPI REST service, FastMCP stdio server for Claude Code, Observable Framework dashboard, and a `custodian` CLI. + +--- + +## Stack + +| Layer | Technology | Port | +|-------|-----------|------| +| Database | PostgreSQL 16-alpine (Docker) | `127.0.0.1:5432` | +| API | FastAPI + SQLAlchemy 2.0 async + asyncpg | `127.0.0.1:8000` | +| MCP server | FastMCP stdio (Claude Code native) | stdio | +| Dashboard | Observable Framework | `127.0.0.1:3000` | +| CLI | `custodian` (Python, uv entry point) | — | + +All services bind to `127.0.0.1` only — nothing exposed to the network. + +--- + +## Setup + +### Prerequisites + +- Docker Engine (WSL2: see `CLAUDE.md` in repo root → Docker Setup) +- Python 3.12+ with `uv` (`pip install uv`) +- Node.js 18+ (dashboard only) + +### First-time + +```bash +cd state-hub + +cp .env.example .env # edit POSTGRES_PASSWORD +make install # uv sync +make db # docker compose up postgres +make migrate # alembic upgrade head (creates 5 tables) +make seed # insert 6 canonical topics +make api # uvicorn :8000 --reload +``` + +### Shortcut + +```bash +make start # db + sleep + migrate + api +``` + +### Dashboard + +```bash +make dashboard # Observable dev server on :3000 +``` + +### CLI + +```bash +make install-cli # symlink .venv/bin/custodian → ~/.local/bin +custodian status # API health + summary totals +custodian register-project # register cwd as a Custodian project +``` + +--- + +## Makefile Targets + +| Target | What it does | +|--------|-------------| +| `make install` | `uv sync` — install Python deps + entry points | +| `make install-cli` | Symlink `custodian` to `~/.local/bin` | +| `make db` | Start postgres container | +| `make db-tools` | Start postgres + pgadmin (http://127.0.0.1:5050) | +| `make migrate` | `alembic upgrade head` | +| `make seed` | Insert 6 canonical topics | +| `make api` | `uvicorn api.main:app --reload` | +| `make dashboard` | Observable dev server | +| `make check` | `curl /state/health` | +| `make start` | `db` + wait + `migrate` + `api` | +| `make register-project DOMAIN=x PROJECT_PATH=y` | Register a project | +| `make clean` | `docker compose down -v` (destroys DB volume) | + +--- + +## Database Schema + +Five tables in dependency order: + +``` +topics +└── workstreams + └── tasks (self-FK: parent_task_id) + └── progress_events +decisions (FK: topic_id, workstream_id — at least one required) + └── progress_events +``` + +### Enums + +| Enum | Values | +|------|--------| +| `topic_status` | `active` · `paused` · `archived` | +| `workstream_status` | `active` · `blocked` · `completed` · `archived` | +| `task_status` | `todo` · `in_progress` · `blocked` · `done` · `cancelled` | +| `task_priority` | `low` · `medium` · `high` · `critical` | +| `decision_type` | `made` · `pending` | +| `decision_status` | `open` · `resolved` · `escalated` · `superseded` | +| `domain` | `custodian` · `railiance` · `markitect` · `coulomb_social` · `personhood` · `foerster_capabilities` | + +### Governance constraints encoded in schema + +- No hard DELETE endpoints — only soft: `archived`, `cancelled`, `superseded` +- `progress_events` has no `updated_at` and no DELETE endpoint (append-only per constitution §5) +- `decisions` with financial/legal keywords + `pending` type → auto-set `escalation_note` (§4) + +--- + +## API + +Interactive docs at http://127.0.0.1:8000/docs once the API is running. + +### Key endpoint: `/state/summary` + +Returns a full snapshot in one call — used by both the MCP server and dashboard: + +```json +{ + "generated_at": "...", + "totals": { + "topics": { "active": 6, "paused": 0, "archived": 0, "total": 6 }, + "workstreams": { "active": 1, "blocked": 0, "completed": 1, "total": 2 }, + "tasks": { "todo": 9, "in_progress": 0, "blocked": 0, "done": 11, "total": 20 }, + "decisions": { "open": 1, "resolved": 0, "escalated": 0, "total": 1 } + }, + "topics": [...], // topics with nested workstream stubs + "blocking_decisions": [...], // pending decisions only + "blocked_tasks": [...], + "recent_progress": [...], // last 20 events + "open_workstreams": [...] +} +``` + +### Router summary + +| Prefix | Operations | +|--------|-----------| +| `/topics` | CRUD (soft-delete: `archived`) | +| `/workstreams` | CRUD (soft-delete: `archived`) | +| `/tasks` | CRUD (soft-delete: `cancelled`); `PATCH` updates status | +| `/decisions` | CRUD (soft-delete: `superseded`); auto-escalation | +| `/progress` | `GET` list + `POST` append — no DELETE | +| `/state/summary` | Full snapshot | +| `/state/health` | DB connectivity check | + +--- + +## MCP Server + +Registered in `~/.claude.json` at user scope. Config in `.mcp.json` (repo root). + +Uses absolute path + `PYTHONPATH` so `cwd` is not required: +```json +{ + "command": "/home/worsch/the-custodian/state-hub/.venv/bin/python", + "args": ["/home/worsch/the-custodian/state-hub/mcp_server/server.py"], + "env": { "PYTHONPATH": "/home/worsch/the-custodian/state-hub", "API_BASE": "http://127.0.0.1:8000" } +} +``` + +See `mcp_server/TOOLS.md` for the full tool reference card (30 lines, faster than reading `server.py`). + +### Tools at a glance + +**Query** (read-only): `get_state_summary` · `get_topic` · `list_blocked_tasks` · `list_pending_decisions` · `get_recent_progress` + +**Mutate** (each auto-emits a progress event): `create_task` · `update_task_status` · `record_decision` · `resolve_decision` · `add_progress_event` · `update_workstream_status` + +**Resources**: `state://summary` · `state://topics` · `state://workstreams/{topic_slug}` · `state://decisions/blocking` · `state://tasks/blocked` + +--- + +## `custodian` CLI + +Installed into `.venv/bin/custodian` by `uv sync`; symlinked to `~/.local/bin` by `make install-cli`. + +``` +custodian register-project [--domain DOMAIN] [--path PATH] +``` + +- `--path` defaults to current working directory +- `--domain` is auto-detected from `project_charter_v*.md` frontmatter if omitted + +``` +custodian status +``` + +Prints API health, totals, and any blocking decisions. + +### What `register-project` does + +1. Verifies the API is reachable (fails fast with `make api` hint) +2. Looks up the topic ID for the domain via `/topics/?status=active` +3. Checks that `state-hub` is in `~/.claude.json` +4. Writes `$PROJECT_PATH/CLAUDE.md` from `scripts/project_claude_md.template` +5. Posts a `milestone` progress event recording the registration + +--- + +## Project Registration Scripts + +| Script | Purpose | +|--------|---------| +| `scripts/register_project.sh` | Shell version of `custodian register-project` | +| `scripts/patch_mcp_cwd.py` | Patches `cwd` into `~/.claude.json` after `claude mcp add-json` drops it | +| `scripts/project_claude_md.template` | CLAUDE.md template with `{PROJECT_NAME}`, `{DOMAIN}`, `{TOPIC_ID}` | +| `scripts/seed.py` | Insert the 6 canonical topics into a fresh database | +| `scripts/pull_image.py` | WSL2 workaround: pull Docker images via Python urllib with Range-request chunking | + +--- + +## Dashboard + +Four pages at http://127.0.0.1:3000 (dev) or built with `npm run build`: + +| Page | Content | +|------|---------| +| **Overview** | Status cards, task-by-status chart, recent activity feed, decisions due within 7 days | +| **Workstreams** | Filterable table by domain/status/owner; selected workstream task list; progress timeline | +| **Decisions** | Pending tab (with escalation highlights) and Made tab; resolution velocity chart | +| **Progress** | Append-only event feed with author badges; 30-day event volume chart | + +Data loaders (`src/data/*.json.py`) are Python scripts that call the local API. They run at dev-server start and on `npm run build`. Clear the cache if data appears stale: + +```bash +rm -rf dashboard/src/.observablehq/cache/ +``` + +--- + +## Known Issues / WSL2 Notes + +- **TLS bad record MAC on large downloads**: WSL2 corrupts packets on big TCP transfers. Use `scripts/pull_image.py` instead of `docker pull` for future image pulls. +- **`claude mcp add-json` drops `cwd`**: Known Claude Code bug. Run `python3 scripts/patch_mcp_cwd.py` after any re-registration. The current `.mcp.json` uses absolute path + `PYTHONPATH` so `cwd` is not strictly needed. +- **AsyncSession concurrency**: SQLAlchemy 2.0 async sessions don't support concurrent operations. All queries in `/state/summary` run sequentially on a single session.