Compare commits
10 Commits
ec92c8e95e
...
6eeb715b3f
| Author | SHA1 | Date | |
|---|---|---|---|
| 6eeb715b3f | |||
| e1d5ba7417 | |||
| 4e4da1fe6a | |||
| 7c81f4fba0 | |||
| a9ba6ffd42 | |||
| 2542ff283f | |||
| 4d634b5ac7 | |||
| fb90293fc4 | |||
| 27eb6b14ad | |||
| 60b72a7b1d |
14
.mcp.json
14
.mcp.json
@@ -1,13 +1,3 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"state-hub": {
|
||||
"type": "stdio",
|
||||
"command": "/home/worsch/the-custodian/state-hub/.venv/bin/python",
|
||||
"args": ["/home/worsch/the-custodian/state-hub/mcp_server/server.py"],
|
||||
"env": {
|
||||
"PYTHONPATH": "/home/worsch/the-custodian/state-hub",
|
||||
"API_BASE": "http://127.0.0.1:8000"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
"mcpServers": {}
|
||||
}
|
||||
@@ -58,16 +58,13 @@ make db # start postgres on 127.0.0.1:5432
|
||||
make migrate # alembic upgrade head
|
||||
make seed # insert 6 canonical topics
|
||||
|
||||
# Run services
|
||||
make api # uvicorn on 127.0.0.1:8000 (docs at /docs)
|
||||
# Run services (each restarts the service if already running)
|
||||
make api # db + migrate + uvicorn on 127.0.0.1:8000
|
||||
make dashboard # Observable preview on :3000
|
||||
make check # curl /state/health
|
||||
|
||||
# Shortcut: db + migrate + api
|
||||
make start
|
||||
```
|
||||
|
||||
The MCP server is registered in `.mcp.json` at the repo root. After `make install` creates `.venv`, restart Claude Code for `/mcp` to show `state-hub`.
|
||||
The MCP server runs as a persistent SSE service (`make mcp-http`, port 8001). Registered at user scope via `claude mcp add-json -s user state-hub '{"type":"sse","url":"http://127.0.0.1:8001/sse"}'`. Restart the MCP server independently — no Claude Code restart needed.
|
||||
|
||||
### Docker Setup (WSL2, one-time)
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ Health check: `make check`
|
||||
### Shortcut
|
||||
|
||||
```bash
|
||||
make start # db + migrate + api (all in one)
|
||||
make api # db + migrate + api (restarts if already running)
|
||||
```
|
||||
|
||||
### Dashboard
|
||||
|
||||
2
SCOPE.md
2
SCOPE.md
@@ -93,7 +93,7 @@ The Custodian is both an **operational system** (State Hub: PostgreSQL + FastAPI
|
||||
|
||||
- Start with: `CLAUDE.md` (session protocol) + `README.md` (architecture overview)
|
||||
- Key files / directories: `state-hub/` (live API + MCP), `canon/` (governance), `workplans/` (active work), `state-hub/mcp_server/TOOLS.md` (tool reference)
|
||||
- Entry points: `cd state-hub && make start` (API); Claude Code with state-hub MCP registered
|
||||
- Entry points: `cd state-hub && make api` (API); Claude Code with state-hub MCP registered
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
.PHONY: install install-cli db db-tools migrate seed api dashboard check test start clean register-project validate-adr add-domain rename-domain add-repo list-repos register-path cleanup-stale tunnel tunnel-daemon tunnel-loop tunnel-status tunnel-stop install-hooks install-hooks-all gitea-inventory
|
||||
.PHONY: install install-cli db db-tools migrate seed api dashboard check test clean register-project validate-adr add-domain rename-domain add-repo list-repos register-path cleanup-stale tunnel tunnel-daemon tunnel-loop tunnel-status tunnel-stop install-hooks install-hooks-all gitea-inventory
|
||||
|
||||
COMPOSE = docker compose -f infra/docker-compose.yml --env-file .env
|
||||
|
||||
@@ -25,17 +25,15 @@ migrate:
|
||||
seed:
|
||||
uv run python scripts/seed.py
|
||||
|
||||
api:
|
||||
uv run uvicorn api.main:app --reload --host 127.0.0.1 --port 8000
|
||||
|
||||
## MCP server in SSE/HTTP mode for remote Claude Code sessions (e.g. COULOMBCORE).
|
||||
## Exposes the same tools as the stdio server over HTTP at http://127.0.0.1:8001/sse
|
||||
## Remote clients connect via the ops-bridge tunnel (port 18001 on the remote host).
|
||||
## Registration on the remote: claude mcp add-json -s user state-hub '{"type":"sse","url":"http://127.0.0.1:18001/sse"}'
|
||||
## Start (or restart) the MCP SSE server on :8001 — primary transport for Claude Code.
|
||||
## Remote clients (e.g. COULOMBCORE) connect via the ops-bridge tunnel (port 18001).
|
||||
## Registration: claude mcp add-json -s user state-hub '{"type":"sse","url":"http://127.0.0.1:8001/sse"}'
|
||||
mcp-http:
|
||||
@fuser -k 8001/tcp 2>/dev/null && echo "Stopped running MCP server" || true
|
||||
MCP_TRANSPORT=sse MCP_PORT=8001 uv run python mcp_server/server.py
|
||||
|
||||
dashboard:
|
||||
@fuser -k 3000/tcp 2>/dev/null && echo "Stopped running dashboard" || true
|
||||
cd dashboard && npm run dev
|
||||
|
||||
check:
|
||||
@@ -105,10 +103,17 @@ tunnel-stop:
|
||||
@pkill -f "autossh.*$(TUNNEL_PORT)" 2>/dev/null && echo "autossh stopped" || true
|
||||
@pkill -f "ssh.*-R $(TUNNEL_PORT)" 2>/dev/null && echo "ssh loop stopped" || true
|
||||
|
||||
start: db
|
||||
sleep 3
|
||||
## Start (or restart) the full backend — db + migrate + uvicorn.
|
||||
## Stops uvicorn on :8000 if already running, then starts fresh.
|
||||
api: db
|
||||
@echo "Waiting for postgres..."; \
|
||||
for i in 1 2 3 4 5 6 7 8 9 10; do \
|
||||
nc -z 127.0.0.1 5432 2>/dev/null && break; \
|
||||
sleep 1; \
|
||||
done
|
||||
$(MAKE) migrate
|
||||
$(MAKE) api
|
||||
@fuser -k 8000/tcp 2>/dev/null && echo "Stopped running API" || true
|
||||
uv run uvicorn api.main:app --reload --host 127.0.0.1 --port 8000
|
||||
|
||||
## Register a project: make register-project DOMAIN=railiance PROJECT_PATH=/home/worsch/railiance
|
||||
register-project:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# State Hub v0.1
|
||||
|
||||
The operational brain of the Custodian: a local PostgreSQL database, FastAPI REST service, FastMCP stdio server for Claude Code, Observable Framework dashboard, and a `custodian` CLI.
|
||||
The operational brain of the Custodian: a local PostgreSQL database, FastAPI REST service, FastMCP SSE server for Claude Code, Observable Framework dashboard, and a `custodian` CLI.
|
||||
|
||||
---
|
||||
|
||||
@@ -10,7 +10,7 @@ The operational brain of the Custodian: a local PostgreSQL database, FastAPI RES
|
||||
|-------|-----------|------|
|
||||
| Database | PostgreSQL 16-alpine (Docker) | `127.0.0.1:5432` |
|
||||
| API | FastAPI + SQLAlchemy 2.0 async + asyncpg | `127.0.0.1:8000` |
|
||||
| MCP server | FastMCP stdio (Claude Code native) | stdio |
|
||||
| MCP server | FastMCP SSE | `127.0.0.1:8001` |
|
||||
| Dashboard | Observable Framework | `127.0.0.1:3000` |
|
||||
| CLI | `custodian` (Python, uv entry point) | — |
|
||||
|
||||
@@ -36,13 +36,7 @@ make install # uv sync
|
||||
make db # docker compose up postgres
|
||||
make migrate # alembic upgrade head (creates 5 tables)
|
||||
make seed # insert 6 canonical topics
|
||||
make api # uvicorn :8000 --reload
|
||||
```
|
||||
|
||||
### Shortcut
|
||||
|
||||
```bash
|
||||
make start # db + sleep + migrate + api
|
||||
make api # db + migrate + uvicorn :8000 (restarts if running)
|
||||
```
|
||||
|
||||
### Dashboard
|
||||
@@ -71,10 +65,9 @@ custodian register-project # register cwd as a Custodian project
|
||||
| `make db-tools` | Start postgres + pgadmin (http://127.0.0.1:5050) |
|
||||
| `make migrate` | `alembic upgrade head` |
|
||||
| `make seed` | Insert 6 canonical topics |
|
||||
| `make api` | `uvicorn api.main:app --reload` |
|
||||
| `make dashboard` | Observable dev server |
|
||||
| `make api` | `db` + wait + `migrate` + `uvicorn` (restarts if running) |
|
||||
| `make dashboard` | Observable dev server (restarts if running) |
|
||||
| `make check` | `curl /state/health` |
|
||||
| `make start` | `db` + wait + `migrate` + `api` |
|
||||
| `make register-project DOMAIN=x PROJECT_PATH=y` | Register a project |
|
||||
| `make clean` | `docker compose down -v` (destroys DB volume) |
|
||||
|
||||
@@ -154,15 +147,22 @@ Returns a full snapshot in one call — used by both the MCP server and dashboar
|
||||
|
||||
## MCP Server
|
||||
|
||||
Registered in `~/.claude.json` at user scope. Config in `.mcp.json` (repo root).
|
||||
Runs as a persistent SSE service on `:8001`, independent of the Claude Code session.
|
||||
Restart it anytime without restarting Claude Code.
|
||||
|
||||
Uses absolute path + `PYTHONPATH` so `cwd` is not required:
|
||||
```bash
|
||||
make mcp-http # start (or restart) the MCP SSE server on :8001
|
||||
```
|
||||
|
||||
Registered at user scope in `~/.claude.json`:
|
||||
```json
|
||||
{
|
||||
"command": "/home/worsch/the-custodian/state-hub/.venv/bin/python",
|
||||
"args": ["/home/worsch/the-custodian/state-hub/mcp_server/server.py"],
|
||||
"env": { "PYTHONPATH": "/home/worsch/the-custodian/state-hub", "API_BASE": "http://127.0.0.1:8000" }
|
||||
}
|
||||
{ "type": "sse", "url": "http://127.0.0.1:8001/sse" }
|
||||
```
|
||||
|
||||
To re-register from scratch:
|
||||
```bash
|
||||
claude mcp remove state-hub -s user 2>/dev/null || true
|
||||
claude mcp add-json -s user state-hub '{"type":"sse","url":"http://127.0.0.1:8001/sse"}'
|
||||
```
|
||||
|
||||
See `mcp_server/TOOLS.md` for the full tool reference card (30 lines, faster than reading `server.py`).
|
||||
@@ -209,7 +209,7 @@ Prints API health, totals, and any blocking decisions.
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `scripts/register_project.sh` | Shell version of `custodian register-project` |
|
||||
| `scripts/patch_mcp_cwd.py` | Patches `cwd` into `~/.claude.json` after `claude mcp add-json` drops it |
|
||||
| `scripts/patch_mcp_cwd.py` | Legacy: patched `cwd` for the old stdio registration (no longer needed) |
|
||||
| `scripts/project_claude_md.template` | CLAUDE.md template with `{PROJECT_NAME}`, `{DOMAIN}`, `{TOPIC_ID}` |
|
||||
| `scripts/seed.py` | Insert the 6 canonical topics into a fresh database |
|
||||
| `scripts/pull_image.py` | WSL2 workaround: pull Docker images via Python urllib with Range-request chunking |
|
||||
@@ -238,5 +238,5 @@ rm -rf dashboard/src/.observablehq/cache/
|
||||
## Known Issues / WSL2 Notes
|
||||
|
||||
- **TLS bad record MAC on large downloads**: WSL2 corrupts packets on big TCP transfers. Use `scripts/pull_image.py` instead of `docker pull` for future image pulls.
|
||||
- **`claude mcp add-json` drops `cwd`**: Known Claude Code bug. Run `python3 scripts/patch_mcp_cwd.py` after any re-registration. The current `.mcp.json` uses absolute path + `PYTHONPATH` so `cwd` is not strictly needed.
|
||||
- **MCP server is now SSE, not stdio**: Re-registration is `claude mcp add-json -s user state-hub '{"type":"sse","url":"http://127.0.0.1:8001/sse"}'`. The `patch_mcp_cwd.py` script and `.mcp.json` config are legacy artifacts from the old stdio setup.
|
||||
- **AsyncSession concurrency**: SQLAlchemy 2.0 async sessions don't support concurrent operations. All queries in `/state/summary` run sequentially on a single session.
|
||||
|
||||
@@ -39,9 +39,7 @@ To restart the API:
|
||||
|
||||
```bash
|
||||
cd ~/the-custodian/state-hub
|
||||
make api # starts uvicorn on 127.0.0.1:8000
|
||||
# or, if postgres is not running:
|
||||
make start # db + migrate + api
|
||||
make api # db + migrate + uvicorn (restarts if already running)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -149,7 +149,7 @@ if (_h1) { _h1.style.position = "relative"; withDocHelp(_h1, "/docs/overview");
|
||||
```
|
||||
|
||||
```js
|
||||
if (summary.error) display(html`<div class="warning">⚠️ ${summary.error}</div>`);
|
||||
display(html`<div class="warning" style="display:${summary.error ? '' : 'none'}">⚠️ ${summary.error ?? ''}</div>`);
|
||||
```
|
||||
|
||||
## Workstreams by Domain
|
||||
|
||||
@@ -27,6 +27,7 @@ Do not use them as a substitute for formal work definition inside the domain rep
|
||||
| `get_domain_summary(domain_slug)` | `domain_slug`: e.g. `"railiance"` | **Domain session start.** Scoped snapshot: active workstreams, blocking decisions, last 5 events, repo SBOM status — ~10% of get_state_summary() token cost. |
|
||||
| `get_state_summary()` | — | **Cross-domain work / custodian sessions.** Full snapshot: totals, all blocking decisions, all blocked tasks, all open workstreams, last 20 events. Large (~10k tokens). |
|
||||
| `get_topic(slug)` | `slug`: e.g. `"markitect"` | Deep-dive on one topic + its workstreams + recent events. |
|
||||
| `list_tasks(workstream_id, status?)` | `workstream_id`: UUID (required); `status?`: todo/in_progress/blocked/done/cancelled | List all tasks in a workstream. Use this to look up task UUIDs before calling `update_task_status`, or to verify which workplan tasks are already synced to the DB. |
|
||||
| `list_blocked_tasks(workstream_id?)` | optional filter | Surface all impediments, optionally scoped to one workstream. |
|
||||
| `list_pending_decisions(topic_id?)` | optional filter | Decisions holding up work, sorted by deadline. |
|
||||
| `get_recent_progress(limit, since?)` | `limit` default 20; `since` ISO datetime | Reconstruct recent session history. |
|
||||
@@ -76,7 +77,7 @@ Use `list_human_interventions()` at session start to see Bernd's action items.
|
||||
|
||||
| Tool | Key Args | When to use |
|
||||
|------|----------|-------------|
|
||||
| `validate_repo_adr(repo_path, domain_slug?)` | `repo_path`: absolute path; `domain_slug?`: for orphan detection | Check a repo against ADR-001. Detects missing workplans/ dir, invalid frontmatter, stale workstream ID references, and DB-only orphan workstreams. Run before and after any workplan changes. |
|
||||
| `validate_repo_adr(repo_slug, domain_slug?)` | `repo_slug`: registered repo slug (e.g. `"the-custodian"`); `domain_slug?`: for orphan detection | Check a repo against ADR-001. Resolves the local path from the DB (uses this host's registered path). Detects missing workplans/ dir, invalid frontmatter, stale workstream ID references, and DB-only orphan workstreams. Always runs against the MCP server's copy — see Multi-Host section below. |
|
||||
|
||||
---
|
||||
|
||||
@@ -152,6 +153,52 @@ instruction set — load it and follow the instructions it contains.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Host & Remote Agent Usage
|
||||
|
||||
Three tools access the **local filesystem** on the MCP server machine:
|
||||
|
||||
| Tool | File-sys operation |
|
||||
|------|-------------------|
|
||||
| `validate_repo_adr` | Runs `validate_repo_adr.py` against the server's repo checkout |
|
||||
| `check_repo_consistency` | Runs `consistency_check.py` against the server's repo checkout |
|
||||
| `ingest_sbom_tool` | Runs `ingest_sbom.py` against the server's lockfiles |
|
||||
|
||||
**Design boundary:** these tools always execute on the machine where the MCP server
|
||||
runs (`bnt-lap001`), against the path registered for that host. A remote agent
|
||||
calling them gets results from the server's checkout — not from its own working copy.
|
||||
|
||||
### Implications for remote agents (e.g. workers on COULOMBCORE)
|
||||
|
||||
- **Ahead of server on a branch?** Results will be based on the server's (older) copy.
|
||||
Sync first: push your branch and pull it on the server, or accept the gap.
|
||||
- **Pure-API tools** (`get_state_summary`, `create_task`, `add_progress_event`, etc.)
|
||||
work correctly from any host — they query the DB, not the filesystem.
|
||||
|
||||
### Running file-sys scripts locally from a remote host
|
||||
|
||||
```bash
|
||||
# From COULOMBCORE (tunnel maps remote :18000 → bnt-lap001 :8000):
|
||||
python scripts/consistency_check.py --repo the-custodian --api-base http://127.0.0.1:18000
|
||||
python scripts/validate_repo_adr.py /home/tegwick/the-custodian --api-base http://127.0.0.1:18000
|
||||
```
|
||||
|
||||
### Registering a new host path
|
||||
|
||||
```bash
|
||||
# Via MCP tool:
|
||||
update_repo_path("marki-docx", "/home/tegwick/marki-docx") # defaults to current hostname
|
||||
|
||||
# Via Makefile (on the machine where the path lives):
|
||||
make register-path REPO=marki-docx PATH=/home/tegwick/marki-docx
|
||||
|
||||
# Via API directly:
|
||||
curl -X POST http://127.0.0.1:8000/repos/marki-docx/paths/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"host": "your-hostname", "path": "/home/you/marki-docx"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Domain Slugs
|
||||
|
||||
Run `list_domains()` to get the live list. Default 6: `custodian` · `railiance` · `markitect` · `coulomb_social` · `personhood` · `foerster_capabilities`
|
||||
|
||||
@@ -273,6 +273,21 @@ def get_topic(slug: str) -> str:
|
||||
return json.dumps({"topic": topic_detail, "recent_progress": recent}, indent=2)
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def list_tasks(workstream_id: str, status: str | None = None) -> str:
|
||||
"""List all tasks in a workstream, optionally filtered by status.
|
||||
|
||||
Args:
|
||||
workstream_id: UUID of the workstream (required).
|
||||
status: Optional filter — todo | in_progress | blocked | done | cancelled.
|
||||
|
||||
Returns [{id, title, status, priority, assignee, due_date, needs_human}] for every
|
||||
matching task. Use this to look up task UUIDs before calling update_task_status,
|
||||
or to check which tasks from a workplan file are already synced to the DB.
|
||||
"""
|
||||
return json.dumps(_get("/tasks", {"workstream_id": workstream_id, "status": status}), indent=2)
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def list_blocked_tasks(workstream_id: str | None = None) -> str:
|
||||
"""List all tasks with status=blocked, optionally filtered by workstream_id."""
|
||||
@@ -1105,7 +1120,7 @@ def get_kaizen_agent(name: str) -> str:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@mcp.tool()
|
||||
def validate_repo_adr(repo_path: str, domain_slug: str | None = None) -> str:
|
||||
def validate_repo_adr(repo_slug: str, domain_slug: str | None = None) -> str:
|
||||
"""Check whether a repository is consistent with ADR-001.
|
||||
|
||||
Validates that workplan files exist in workplans/ with correct frontmatter,
|
||||
@@ -1113,12 +1128,41 @@ def validate_repo_adr(repo_path: str, domain_slug: str | None = None) -> str:
|
||||
no active state-hub workstreams for the domain lack a backing file (orphan
|
||||
detection — DB-only records are an ADR-001 violation).
|
||||
|
||||
The repo path is resolved from the DB using the current machine's hostname
|
||||
(host_paths[hostname] → local_path fallback). This tool always runs against
|
||||
the server's copy of the repo. Remote agents on a different branch should
|
||||
sync first, or run validate_repo_adr.py locally with
|
||||
--api-base http://127.0.0.1:18000.
|
||||
|
||||
Args:
|
||||
repo_path: Absolute path to the repository root.
|
||||
repo_slug: Registered repo slug (e.g. 'the-custodian', 'ops-bridge').
|
||||
domain_slug: Domain slug for orphan detection (e.g. 'custodian').
|
||||
If omitted, inferred from workplan frontmatter.
|
||||
"""
|
||||
import socket as _socket
|
||||
import subprocess
|
||||
|
||||
repo = _get(f"/repos/{repo_slug}")
|
||||
if isinstance(repo, dict) and repo.get("error"):
|
||||
return f"Repo '{repo_slug}' not found: {repo['error']}"
|
||||
|
||||
hostname = _socket.gethostname()
|
||||
host_paths = repo.get("host_paths") or {}
|
||||
repo_path = host_paths.get(hostname) or repo.get("local_path") or ""
|
||||
|
||||
if not repo_path:
|
||||
return (
|
||||
f"⚠ No path registered for repo '{repo_slug}' on this host ({hostname}).\n"
|
||||
f"Register with: update_repo_path('{repo_slug}', '/path/to/repo')\n"
|
||||
f"Remote agents: run validate_repo_adr.py locally with "
|
||||
f"--api-base {API_BASE}"
|
||||
)
|
||||
if not Path(repo_path).is_dir():
|
||||
return (
|
||||
f"⚠ Registered path for '{repo_slug}' on {hostname} does not exist: {repo_path}\n"
|
||||
f"Update with: update_repo_path('{repo_slug}', '/correct/path')"
|
||||
)
|
||||
|
||||
script = Path(__file__).parent.parent / "scripts" / "validate_repo_adr.py"
|
||||
cmd = [sys.executable, str(script), repo_path, "--json",
|
||||
"--api-base", API_BASE]
|
||||
@@ -1138,7 +1182,7 @@ def validate_repo_adr(repo_path: str, domain_slug: str | None = None) -> str:
|
||||
failures = [f for f in findings if f["level"] == "FAIL"]
|
||||
warnings = [f for f in findings if f["level"] == "WARN"]
|
||||
|
||||
lines = [f"ADR-001 Compliance: {repo_path}", ""]
|
||||
lines = [f"ADR-001 Compliance: {repo_slug} ({repo_path})", ""]
|
||||
|
||||
if failures:
|
||||
lines.append(f"FAILURES ({len(failures)}):")
|
||||
@@ -1187,7 +1231,29 @@ def check_repo_consistency(repo_slug: str, fix: bool = False) -> str:
|
||||
(C-05), create missing DB workstreams (C-06), repo mismatch (C-09),
|
||||
task status drift (C-10), create unlinked tasks (C-11).
|
||||
"""
|
||||
import socket as _socket
|
||||
import subprocess
|
||||
|
||||
# Pre-flight: verify this host has the repo path registered and accessible.
|
||||
repo = _get(f"/repos/{repo_slug}")
|
||||
if isinstance(repo, dict) and repo.get("error"):
|
||||
return f"Repo '{repo_slug}' not found: {repo['error']}"
|
||||
hostname = _socket.gethostname()
|
||||
host_paths = repo.get("host_paths") or {}
|
||||
repo_path = host_paths.get(hostname) or repo.get("local_path") or ""
|
||||
if not repo_path:
|
||||
return (
|
||||
f"⚠ No path registered for repo '{repo_slug}' on this host ({hostname}).\n"
|
||||
f"Register with: update_repo_path('{repo_slug}', '/path/to/repo')\n"
|
||||
f"Remote agents: run consistency_check.py locally with "
|
||||
f"--api-base {API_BASE}"
|
||||
)
|
||||
if not Path(repo_path).is_dir():
|
||||
return (
|
||||
f"⚠ Registered path for '{repo_slug}' on {hostname} does not exist: {repo_path}\n"
|
||||
f"Update with: update_repo_path('{repo_slug}', '/correct/path')"
|
||||
)
|
||||
|
||||
script = Path(__file__).parent.parent / "scripts" / "consistency_check.py"
|
||||
cmd = [sys.executable, str(script), "--repo", repo_slug, "--json",
|
||||
"--api-base", API_BASE]
|
||||
@@ -1359,23 +1425,55 @@ def get_contributions(
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def ingest_sbom_tool(repo_slug: str, lockfile_path: str) -> str:
|
||||
def ingest_sbom_tool(repo_slug: str, lockfile_path: str | None = None) -> str:
|
||||
"""Ingest a lockfile into the State Hub SBOM store for a repo.
|
||||
|
||||
Parses the lockfile and POSTs entries to /sbom/ingest/. Each call creates
|
||||
a new SBOMSnapshot; previous snapshots are retained as history.
|
||||
|
||||
The repo root is resolved from the DB using the current machine's hostname
|
||||
(host_paths[hostname] → local_path fallback). lockfile_path, when given,
|
||||
is treated as relative to the repo root. Omit it to auto-detect the lockfile.
|
||||
|
||||
Args:
|
||||
repo_slug: Managed-repo slug (must be registered via register_repo)
|
||||
lockfile_path: Absolute path to the lockfile (uv.lock, package-lock.json, Cargo.lock, etc.)
|
||||
lockfile_path: Path to the lockfile, relative to repo root
|
||||
(e.g. "uv.lock", "frontend/package-lock.json").
|
||||
Omit to auto-detect from the repo root.
|
||||
"""
|
||||
import socket as _socket
|
||||
import subprocess
|
||||
|
||||
repo = _get(f"/repos/{repo_slug}")
|
||||
if isinstance(repo, dict) and repo.get("error"):
|
||||
return f"Repo '{repo_slug}' not found: {repo['error']}"
|
||||
|
||||
hostname = _socket.gethostname()
|
||||
host_paths = repo.get("host_paths") or {}
|
||||
repo_root = host_paths.get(hostname) or repo.get("local_path") or ""
|
||||
|
||||
if not repo_root:
|
||||
return (
|
||||
f"⚠ No path registered for repo '{repo_slug}' on this host ({hostname}).\n"
|
||||
f"Register with: update_repo_path('{repo_slug}', '/path/to/repo')"
|
||||
)
|
||||
if not Path(repo_root).is_dir():
|
||||
return (
|
||||
f"⚠ Registered path for '{repo_slug}' on {hostname} does not exist: {repo_root}\n"
|
||||
f"Update with: update_repo_path('{repo_slug}', '/correct/path')"
|
||||
)
|
||||
|
||||
script = Path(__file__).parent.parent / "scripts" / "ingest_sbom.py"
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script), "--repo", repo_slug,
|
||||
"--lockfile", lockfile_path, "--api-base", API_BASE],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
cmd = [sys.executable, str(script), "--repo", repo_slug,
|
||||
"--repo-path", repo_root, "--api-base", API_BASE]
|
||||
|
||||
if lockfile_path:
|
||||
resolved = Path(repo_root) / lockfile_path
|
||||
if not resolved.exists():
|
||||
return f"⚠ Lockfile not found: {resolved}"
|
||||
cmd += ["--lockfile", str(resolved)]
|
||||
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
output = (result.stdout + result.stderr).strip()
|
||||
if result.returncode != 0:
|
||||
return f"ingest_sbom failed (exit {result.returncode}):\n{output}"
|
||||
|
||||
@@ -10,6 +10,7 @@ Auto-detects all of the following in one scan:
|
||||
package-lock.json → node
|
||||
yarn.lock → node
|
||||
Cargo.lock → rust
|
||||
go.sum → go (reads go.mod alongside for direct/indirect)
|
||||
.terraform.lock.hcl → terraform (anywhere in tree)
|
||||
ansible/requirements.yml → ansible (anywhere under ansible/ dirs)
|
||||
ansible/requirements.yaml → ansible
|
||||
@@ -275,6 +276,62 @@ def _parse_ansible_requirements(path: Path) -> list[dict]:
|
||||
return entries
|
||||
|
||||
|
||||
def _parse_go_sum(path: Path) -> list[dict]:
|
||||
"""Parse go.sum — deduplicated Go module list with direct/indirect from go.mod."""
|
||||
# Determine direct deps by reading go.mod in the same directory
|
||||
direct: set[str] = set()
|
||||
go_mod = path.parent / "go.mod"
|
||||
if go_mod.exists():
|
||||
in_require = False
|
||||
for line in go_mod.read_text().splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("require ("):
|
||||
in_require = True
|
||||
continue
|
||||
if in_require and stripped == ")":
|
||||
in_require = False
|
||||
continue
|
||||
if in_require and stripped and not stripped.startswith("//"):
|
||||
if "// indirect" not in stripped:
|
||||
parts = stripped.split()
|
||||
if parts:
|
||||
direct.add(parts[0])
|
||||
# single-line require without parens
|
||||
elif stripped.startswith("require ") and "(" not in stripped:
|
||||
rest = stripped[len("require "):].strip()
|
||||
if "// indirect" not in rest:
|
||||
parts = rest.split()
|
||||
if parts:
|
||||
direct.add(parts[0])
|
||||
|
||||
seen: set[tuple[str, str | None]] = set()
|
||||
entries = []
|
||||
for line in path.read_text().splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("//"):
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
module, version = parts[0], parts[1]
|
||||
# Skip go.mod hash lines — only ingest the module itself
|
||||
if "/go.mod" in version:
|
||||
continue
|
||||
key = (module, version)
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
entries.append({
|
||||
"package_name": module,
|
||||
"package_version": version,
|
||||
"ecosystem": "go",
|
||||
"license_spdx": None,
|
||||
"is_direct": module in direct,
|
||||
"is_dev": False,
|
||||
})
|
||||
return entries
|
||||
|
||||
|
||||
def _parse_sbom_tools_yaml(path: Path) -> list[dict]:
|
||||
"""Parse sbom-tools.yaml — agent-generated tool manifest at repo root."""
|
||||
if not _YAML_AVAILABLE:
|
||||
@@ -333,6 +390,7 @@ _LOCKFILE_PARSERS: dict[str, object] = {
|
||||
"yarn.lock": _parse_yarn_lock,
|
||||
"Cargo.lock": _parse_cargo_lock,
|
||||
".terraform.lock.hcl": _parse_terraform_lock_hcl,
|
||||
"go.sum": _parse_go_sum,
|
||||
}
|
||||
|
||||
# Directories that never contain project-level lockfiles
|
||||
|
||||
186
workplans/CUST-WP-0021-multi-host-repo-paths.md
Normal file
186
workplans/CUST-WP-0021-multi-host-repo-paths.md
Normal file
@@ -0,0 +1,186 @@
|
||||
---
|
||||
id: CUST-WP-0021
|
||||
type: workplan
|
||||
title: "State Hub — Multi-Host Repo Path Hardening"
|
||||
domain: custodian
|
||||
status: done
|
||||
owner: custodian
|
||||
topic_slug: custodian
|
||||
created: "2026-03-18"
|
||||
updated: "2026-03-18"
|
||||
state_hub_workstream_id: "516ca332-5eac-4d6e-8bf9-b2694ed34276"
|
||||
---
|
||||
|
||||
# State Hub — Multi-Host Repo Path Hardening
|
||||
|
||||
## Summary
|
||||
|
||||
When a kaizen-agentic worker on COULOMBCORE calls file-system-touching MCP
|
||||
tools (validate_repo_adr, check_repo_consistency, ingest_sbom), the MCP server
|
||||
runs those scripts on its own machine (bnt-lap001) against its own copy of the
|
||||
repo. Two problems emerge:
|
||||
|
||||
1. **Wrong path**: `validate_repo_adr` and `ingest_sbom_tool` take raw
|
||||
filesystem paths as input. A remote agent passes their local path
|
||||
(e.g. `/home/tegwick/the-custodian`) which does not exist on the server.
|
||||
|
||||
2. **Stale state**: Even when path resolution works, the server runs against
|
||||
its own checkout. A remote agent ahead on a branch gets misleading results.
|
||||
|
||||
## Design Boundary (documented, not fixed)
|
||||
|
||||
The MCP server is a subprocess on bnt-lap001. It can only read files from
|
||||
bnt-lap001's filesystem. File-system tools will always operate against the
|
||||
server's copy. The correct workflow for remote agents working on an ahead
|
||||
branch is: push/sync first, or run consistency_check.py locally with
|
||||
`--api-base http://127.0.0.1:18000` rather than via the MCP tool.
|
||||
This rule is documented in TOOLS.md.
|
||||
|
||||
## Resolves
|
||||
|
||||
- validate_repo_adr raw-path input (broken for remote callers)
|
||||
- ingest_sbom_tool raw lockfile path (same problem)
|
||||
- host_paths empty for all repos except kaizen-agentic
|
||||
- No error when server lacks the repo — silent wrong results
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### T01 — Register COULOMBCORE host_paths for all repos it has
|
||||
|
||||
```task
|
||||
id: CUST-WP-0021-T01
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "cf2c0449-9250-4425-925b-302482d75a11"
|
||||
```
|
||||
|
||||
COULOMBCORE hostname: `254.130.205.92.host.secureserver.net`
|
||||
Repos present at `/home/tegwick/<slug>`:
|
||||
|
||||
| repo slug | COULOMBCORE path |
|
||||
|-----------|-----------------|
|
||||
| the-custodian | /home/tegwick/the-custodian |
|
||||
| kaizen-agentic | /home/tegwick/kaizen-agentic |
|
||||
| ops-bridge | /home/tegwick/ops-bridge |
|
||||
| marki-docx | /home/tegwick/marki-docx |
|
||||
| railiance-cluster | /home/tegwick/railiance-cluster |
|
||||
| railiance-infra | /home/tegwick/railiance-infra |
|
||||
|
||||
Also register bnt-lap001 paths for all repos currently using only `local_path`
|
||||
(migrate them into `host_paths` so the map is the canonical source):
|
||||
|
||||
| repo slug | bnt-lap001 path |
|
||||
|-----------|----------------|
|
||||
| the-custodian | /home/worsch/the-custodian |
|
||||
| kaizen-agentic | /home/worsch/kaizen-agentic |
|
||||
| ops-bridge | /home/worsch/ops-bridge |
|
||||
| activity-core | /home/worsch/activity-core |
|
||||
| markitect-project | /home/worsch/markitect_project |
|
||||
| railiance-apps | /home/worsch/railiance-apps |
|
||||
| railiance-cluster | /home/worsch/railiance-cluster |
|
||||
| railiance-bootstrap | /home/worsch/railiance-cluster |
|
||||
| railiance-enablement | /home/worsch/railiance-enablement |
|
||||
| railiance-hosts | /home/worsch/railiance-infra |
|
||||
| railiance-infra | /home/worsch/railiance-infra |
|
||||
| railiance-platform | /home/worsch/railiance-platform |
|
||||
| key-cape | /home/worsch/key-cape |
|
||||
| net-kingdom | /home/worsch/net-kingdom |
|
||||
|
||||
Use `POST /repos/{slug}/paths/` with `{"host": "<hostname>", "path": "<path>"}`.
|
||||
|
||||
---
|
||||
|
||||
### T02 — Fix validate_repo_adr: accept repo_slug, resolve path from DB
|
||||
|
||||
```task
|
||||
id: CUST-WP-0021-T02
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "52ed094a-4216-4cd1-a634-a29b82f26ec5"
|
||||
```
|
||||
|
||||
Change `validate_repo_adr(repo_path: str, ...)` to
|
||||
`validate_repo_adr(repo_slug: str, ...)` in `mcp_server/server.py`.
|
||||
|
||||
Resolution logic (same as `_kaizen_agents_dir()`):
|
||||
1. Fetch repo record via `_get(f"/repos/{repo_slug}")`
|
||||
2. `hostname = socket.gethostname()`
|
||||
3. `path = host_paths.get(hostname) or repo.get("local_path") or ""`
|
||||
4. If path is empty or not a directory: return a clear error message with
|
||||
instructions for remote agents to run the script locally.
|
||||
5. Pass the resolved path to the validate_repo_adr.py subprocess as before.
|
||||
|
||||
Update the tool docstring to document the new parameter and the design
|
||||
boundary (tool always runs against the server's copy).
|
||||
|
||||
---
|
||||
|
||||
### T03 — Fix ingest_sbom_tool: resolve lockfile via repo_slug + relative path
|
||||
|
||||
```task
|
||||
id: CUST-WP-0021-T03
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "2a67d5e2-f581-490c-b42e-0f7d37979c0a"
|
||||
```
|
||||
|
||||
Change `ingest_sbom_tool(repo_slug, lockfile_path: str)` so `lockfile_path`
|
||||
becomes optional and is interpreted as **relative to the repo root** when
|
||||
provided (not absolute). When omitted, the script auto-detects the lockfile
|
||||
(existing behaviour with `--repo-path`).
|
||||
|
||||
Resolution logic:
|
||||
1. Fetch repo record, resolve path via `host_paths[hostname]` / `local_path`
|
||||
2. If path empty/missing: return clear error
|
||||
3. If `lockfile_path` provided and is relative: join with resolved repo root
|
||||
4. If `lockfile_path` is absolute: use as-is (backward compat), but emit a
|
||||
deprecation warning in the result string
|
||||
5. Pass `--repo-path <resolved>` and optionally `--lockfile <lockfile>` to script
|
||||
|
||||
---
|
||||
|
||||
### T04 — Add host-path guard to check_repo_consistency
|
||||
|
||||
```task
|
||||
id: CUST-WP-0021-T04
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "3885497e-c491-4ddf-811f-0f0d19a0fc42"
|
||||
```
|
||||
|
||||
`check_repo_consistency` already resolves paths correctly via the script.
|
||||
But when `host_paths` is empty and `local_path` is absent the script silently
|
||||
skips file checks. Add a pre-flight guard in the MCP tool:
|
||||
|
||||
1. Fetch the repo record before spawning the subprocess
|
||||
2. Run `resolve_repo_path` logic: `host_paths[hostname]` → `local_path`
|
||||
3. If empty: return an early error message:
|
||||
```
|
||||
⚠ No path registered for this host (bnt-lap001).
|
||||
Register with: update_repo_path(repo_slug, "/path/to/repo")
|
||||
Remote agents: run consistency_check.py locally with --api-base http://127.0.0.1:18000
|
||||
```
|
||||
4. If path is set but directory doesn't exist: same error (not just empty string)
|
||||
|
||||
---
|
||||
|
||||
### T05 — Document design boundary in TOOLS.md
|
||||
|
||||
```task
|
||||
id: CUST-WP-0021-T05
|
||||
status: done
|
||||
priority: low
|
||||
state_hub_task_id: "b4ba8cd0-1093-43ce-8a5d-03529e3c0588"
|
||||
```
|
||||
|
||||
Add a section to `state-hub/mcp_server/TOOLS.md` under a new heading
|
||||
"## Multi-Host & Remote Agent Usage" that explains:
|
||||
|
||||
- File-sys tools (validate_repo_adr, check_repo_consistency, ingest_sbom)
|
||||
always execute on the MCP server machine against its registered path
|
||||
- Remote agents on a different branch/ahead-of-server should sync first
|
||||
OR run the scripts locally with `--api-base http://127.0.0.1:18000`
|
||||
- How to register a new host's path: `update_repo_path(slug, path, host)`
|
||||
- Pure-API tools (get_state_summary, create_task, etc.) work from any host
|
||||
Reference in New Issue
Block a user