Documents the dashboard's architecture, framework choice rationale, data-fetching
strategies (static loaders + live polling), component library, page inventory,
and key features including the Workstream Health Index and entity modals.
Also registers the new page in the Reference nav and adds runbook section for
node overload / runaway agent process (INC-002) with hardening checklist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds _write_custodian_brief() to consistency_check.py. After every fix_repo()
run, a .custodian-brief.md is written to the repo root with: domain, last-synced
timestamp, current repo goal, active workstreams with progress (done/total), and
the first 7 open tasks per workstream (blocked → in_progress → todo order) with
task IDs. The file is git-committed when content changes so remote workers (e.g.
CoulombCore) can pull it and orient without a live MCP connection.
Session protocol template and CLAUDE.md updated: read .custodian-brief.md first,
then call get_domain_summary() as an enhancement (skip if MCP unreachable).
This eliminates false "State hub is offline" alarms in subagents and remote workers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds --remote CLI flag and fix_all_remote() function. When run without a
REPO argument, the target checks all registered repos and:
- Skips repos whose local path does not exist on this machine
- Skips repos that are already clean (no fixable issues, no FAILs, not
behind remote, only C-08 background noise allowed)
- For repos that need work: git pull --ff-only then fix_repo()
Prints a summary of CLEAN (skipped) and NOT ON THIS HOST (skipped) repos
before the detailed fix reports.
Simplifies the Makefile target from shell-level curl+git to a single
uv run call using --remote. Same flag handles both single-repo and all-repos.
Also adds _git_pull() helper and 13 new tests (71 total in consistency suite).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_detect_behind_remote was comparing HEAD != @{u} which incorrectly
triggered C-16 when the local repo had unpushed commits. Fixed to use
git rev-list --count HEAD..@{u} which only counts commits the remote
has that local lacks. Adds test_returns_false_when_local_ahead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
T01 — No-regress rule (C-15): fix-consistency now detects when a DB task
status is ahead of the workplan file (e.g. marked done on CoulombCore)
and emits C-15 WARN instead of regressing the DB back to the stale file
value. STATUS_ORDER ranking: todo(0) < in_progress/blocked(1) < done/cancelled(2).
T02 — Pull gate (C-16): fix_repo runs git fetch + rev-parse at the start
of every --fix run. If the local repo is behind its remote tracking branch,
all write operations are skipped and C-16 WARN is emitted. Best-effort:
offline/no-remote silently skips the check.
T03 — DB→file writeback: C-15 fix path patches the status field in the
matching task block and git-commits the change with a standard message.
--no-writeback flag disables writeback while keeping T01/T02 active.
T04 — CLAUDE.md + session-protocol.template updated with new guidance,
C-15/C-16 semantics, and fix-consistency-remote recommendation.
T05 — Makefile: fix-consistency-remote pulls then fixes in one step.
16 new tests; 155 passed total.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a structured dispute mechanism when capability request routing is wrong:
- New `routing_disputed` status with four DB columns (dispute_reason, disputed_by,
dispute_suggested_domain, disputed_at) via Alembic migration m0h1i2j3k4l5
- POST /capability-requests/{id}/dispute — any party can flag misrouting with a reason
and optional suggested domain; notifies custodian + current fulfilling domain
- POST /capability-requests/{id}/reroute — custodian re-routes to correct domain via
catalog_entry_id or direct slug; appends audit trail to routing_note; resets to requested
- Two new MCP tools: dispute_capability_routing and reroute_capability_request
- Dashboard: amber disputed-banner at top of Summary, routing_disputed Kanban column,
dispute details (reason, suggested domain, raised-by) shown on disputed cards
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New page at /tools listing all connected applications grouped by
category: Local Services (State Hub API, KeePassXC, pgAdmin, ops-bridge),
Source Control (Gitea), Identity/Auth (KeyCape, Authelia, privacyIDEA,
LLDAP), and Dev Tooling (Claude Code, uv). Local services show live
green/red/grey status dots via no-cors fetch probes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `routing_note` column (migration l9g0h1i2j3k4) to persist why a request was routed to a given domain
- Fix substring-match bug in `_route_capability`: use `\b` word-boundary regex so 'postgres' no longer matches inside 'postgresql'
- Include `title` in keyword scoring for better routing accuracy
- Return `routing_note` string from `_route_capability` and store it on the request
- Add `PATCH /capability-requests/{id}` endpoint + `CapabilityRequestPatch` schema to correct mutable metadata (catalog_entry_id, priority, blocking_task_id, fulfilling_workstream_id)
- Add `patch_capability_request` MCP tool wrapping the new endpoint
- Add 105 lines of routing tests (word-boundary, title-match, multi-entry scoring, broadcast fallback)
- Add `tunnels-up`, `tunnels-status`, `tunnels-check` Makefile targets for ops-bridge managed tunnels
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
New page (docs/state-hub.md) covers:
- Why: the invisible state problem across repos and agents
- What: Derived Data Store, Read Model, Agent Orchestration Layer,
Cross-Repo Observatory — and what it is NOT
- Derived Data Store principle (ADR-003): fingerprint cache, rebuild
guarantee, force-refresh
- Repository Orchestrator: session protocol, cross-domain coordination
via messages + capability routing, Kaizen agents
- Architecture diagram (ASCII), technology choices, data model overview
- Running the hub, design principles, related docs
reference.md: add Architecture & Design section grouping state-hub,
TPSC, GDPR maturity, SCOPE.md, capabilities, and goals docs.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Root cause: C2/C9/C10 each made a full HTTP round-trip back to the API
(asyncio.to_thread → urllib → TCP → uvicorn → SQLAlchemy → DB) for every
repo. 16 repos × 3 calls = 48 self-calls at ~80-150ms each = ~6s total.
Fix: doi_engine.evaluate() accepts a prefetch dict. The summary endpoint
runs 3 bulk GROUP BY queries (domain status, TPSC snapshot counts, active
goal counts) and passes results directly — zero HTTP self-calls in summary
mode.
Result: /repos/doi/summary 6s → <1s (6× improvement on top of prior 13×).
Total improvement from original: 108s → <1s.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Page now renders in ~200ms. DoI badges and KPI card show a spinner
while the background fetch resolves (~6s), then update reactively
via Observable Mutable pattern (doiData / doiLoading).
Fast path: repos, SBOM, domains, workstreams — immediate render.
Slow path: /repos/doi/summary — background, non-blocking.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two fixes:
1. skip_consistency=True in summary mode — omits C7/C13 subprocess calls
(consistency_check.py) which were the main bottleneck (32 spawns for 16 repos).
Full check still available per-repo via GET /repos/{slug}/doi.
2. asyncio.gather — all repos evaluated in parallel instead of sequentially.
Also: rename Repositories page title from "Repos" to "Repositories".
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Implements the 14-criterion DoI checklist as a runnable gate with API,
MCP tools, CLI script, and dashboard integration.
Core components:
- api/doi_engine.py — async engine evaluating all 14 criteria (asyncio.to_thread
for non-blocking HTTP self-calls), shared by API and CLI
- api/schemas/doi.py — DoICriterion, DoIReport, DoISummaryEntry schemas
- api/routers/repos.py — GET /repos/{slug}/doi + GET /repos/doi/summary
- scripts/check_doi.py — CLI: make check-doi REPO=<slug> / check-doi-all
- mcp_server/server.py — check_repo_doi(), get_doi_summary() tools
Dashboard (repos.md):
- DoI tier badge per repo (None/Core/Standard/Full) colour-coded red→green
- Domain block shows lowest DoI tier across its repos
- DoI KPI card in summary row
- DoI filter in All Repos Table
- Link to Repository DoI policy page
Also fixes: TPSC snapshots 500 error (missing nested selectinload for
catalog_entry relationship in list_snapshots endpoint).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Three-tier checklist defining what 'fully integrated with the state-hub'
means for a repository:
- Core (Registered): registered, domain assigned, path resolves, remote URL
- Standard (Integrated): SCOPE.md, CLAUDE.md, workplan convention, SBOM, TPSC
- Full (Fully Integrated): repo goal, capabilities declared, agents template,
clean consistency check, host paths registered
Exposed via /policy/repo-doi (editable in dashboard) and linked under Policies.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Stale host_paths entries (wrong username, old machine) were silently overriding
the correct local_path, causing FileNotFoundError on tools like list_kaizen_agents.
Extracts _resolve_repo_path(repo) helper that tries host_paths[hostname] first
but validates the path exists on disk before trusting it, then falls back to
local_path. Both candidates support ~ expansion. Applied to all 4 call sites:
_kaizen_agents_dir, validate_repo_adr, check_repo_consistency, ingest_sbom_tool.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Introduces a capability catalog (CUST-WP-0022) so domains can advertise what
they provide and agents can request capabilities from other domains with
auto-routing, lifecycle tracking, and task-unblocking on completion.
- New models: CapabilityCatalog, CapabilityRequest with full lifecycle
(requested → accepted → in_progress → ready_for_review → completed/rejected/withdrawn)
- Migration i6d7e8f9a0b1: capability_catalog + capability_requests tables
- Router /capability-catalog and /capability-requests with accept/status endpoints
- 7 new MCP tools: register_capability, list_capabilities, request_capability,
accept_capability_request, update_capability_request_status,
list_capability_requests, get_capability_request
- StateSummary gains open_capability_requests count
- Dashboard: capability-requests.md page + docs/capabilities.md + docs/scope.md
- SCOPE.md: three seed capabilities documented (MCP registration, state tracking, SBOM)
- scope.template: Provided Capabilities section with example block
- scripts/ingest_capabilities.py + make ingest-capabilities[/-all] targets
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
repos.json.py now fetches /sbom/snapshots/ alongside /repos/ and
annotates each repo with sbom_snapshot_count, sbom_entry_count, and a
last_sbom_at fallback derived from actual snapshot data. This prevents
"LastSBOM=never" when the denormalized field is out of sync.
repo-sync.md gains SBOM KPI tiles (ingested vs no-SBOM), color-coded
SBOM age column (same green/orange/red scale as state sync), and an
entry count column showing packages from the latest snapshot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parses go.sum lockfiles for Go projects. Reads go.mod alongside to
mark direct vs indirect dependencies. Deduplicates by (module, version),
skipping go.mod hash lines.
Used to ingest key-cape (netkingdom domain): 23 Go modules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Always call display() for the warning element so Observable Framework
replaces it on each poll re-run. Previously the conditional display()
call left the warning rendered indefinitely once shown.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MCP server is now a persistent SSE service on :8001 (make mcp-http),
independent of the Claude Code session. Re-registration is a single
claude mcp add-json command; no patch_mcp_cwd.py needed.
- Makefile: mcp-http is primary transport, add fuser restart + updated comment
- state-hub/README.md: stack table, MCP section, troubleshooting note updated
- CLAUDE.md (project): registration instructions rewritten for SSE
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The old bare `api` target (uvicorn only) is subsumed into the new `api`
target (db + postgres-wait + migrate + fuser-restart + uvicorn). Updated
all doc references and cleaned up duplicate entries left by the rename.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pkill -f matched the shell subprocess's own argv (which contains the
pattern as a -c argument), causing make to receive SIGTERM and abort.
fuser -k 8000/tcp / 3000/tcp targets only the process bound to the
port — no self-kill risk.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `make backend` replaces `make start`; polls postgres with nc (up to 10s)
instead of fixed sleep, kills any running uvicorn before starting fresh
- `make dashboard` kills any running observable preview before restarting
- Update all references in CLAUDE.md, README.md, SCOPE.md, state-hub/README.md,
and dashboard/src/docs/live-data.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Agents had no way to look up task UUIDs by workstream; they were stuck
unable to call update_task_status without already knowing the UUID.
list_tasks() wraps GET /tasks with workstream_id filter, returning
[{id, title, status, priority}] for all matching tasks.
FR raised by kaizen-agentic worker on COULOMBCORE while syncing
KAIZEN-WP-0002 task IDs. Marked merged in contributions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- parse_task_blocks() now injects the nearest preceding ### heading
text as `title` — tasks no longer stored with bare IDs as their title
- C-11 fix skips creating tasks when workstream is completed/archived
(prevents duplicate task creation on repeated fix-consistency runs)
- C-12 is now fixable: auto-cancels open orphan DB tasks when the
backing workstream is finished (completed/archived)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FastMCP validates dict | None strictly, rejecting a JSON string even if
parseable. Broaden to dict | str | None and coerce in the function body
so callers don't need to pre-parse the detail payload.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
T01: Restructure nav — "Repos" → collapsible "Repository" section with
Repo Sync, SBOM, Debt as sub-pages; Debt moved out of Workstreams
T02: workstream-dod.md migrated from inline const API to config.js import
T03: todo.md suggestion filter (done in previous commit)
Closes CUST-WP-0019. Resolves UI suggestion c2fc284a.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Consolidates all open technical debt into three workplans:
- CUST-WP-0018: API hardening & code quality (TD-006–019 medium/high items)
- CUST-WP-0019: Dashboard UX polish (Repos nav restructure, config.js cleanup,
todo filter fix for new suggestion workflow statuses)
- CUST-WP-0020: pytest test suite with real DB (TD-014)
Also fixes todo.md Suggestions filter: was checking status===open but new
suggestions enter with status=submitted; now excludes terminal statuses only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Shift+click now works on #observablehq-toc links, KPI boxes, and [id] elements
- _inferWidgetName detects TOC context and labels suggestions accordingly
- Click handler adds inToc branch alongside existing inSidebar
- _updateMode: 1-second setTimeout before activating highlight mode
so normal Shift+typing doesn't flicker the UI; clears immediately on
Shift release or window blur
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Click handler: sidebar <a> and <summary> are no longer excluded;
e.preventDefault() stops navigation while shift is held
- _inferWidgetName: sidebar-first branch returns nav link text,
section heading text, or "Navigation" fallback
- pageName is "Navigation" for sidebar clicks (not the current page title)
- CSS: sidebar a and summary highlighted (dashed indigo outline + tint)
when shift is held, same as main content widgets
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The card titled "SBOM" was displaying contribution type counts (FR/BR/EP/UPR)
which is unrelated to SBOM data. Added a sbomSnapState generator that fetches
/sbom/snapshots/ and shows: total tracked packages (sum of entry_count across
all snapshots), repos tracked, and copyleft risk count from the existing
licence_risk_count in the summary. Card turns orange if licenceRisk > 0.
Resolves suggestion b6775727.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- improvement-modal.js: API expects `domain` not `domain_slug` (422 fix)
- todo.md: section heading and KPI label renamed to "Suggestions"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Observable Framework proxies all src/*.js files through its own bundler —
<script type="module"> imports from <head> resolve to circular re-export
shims and never execute. The fix: read improvement-modal.js at config load
time in Node.js, strip the ES module export keyword, and interpolate the
content as a plain <script> block in the head config. It runs on every page.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_footer.md is not a supported special file in Observable Framework 1.13.3
and was silently ignored. The preview server does serve src/*.js files at
their root-relative path, so the correct approach is a <script type="module">
in the head config — runs once on page load, persists across SPA navigation.
Removed _footer.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- updateMode() now subscribes to keydown, keyup AND mousemove so the
body class stays in sync regardless of where focus is (mirrors the
pattern from the working modifier-click demo)
- cursor: copy replaces crosshair (matches copy-affordance semantics)
- figure, h2–h4 and [data-widget-name] elements get a dashed indigo
outline + subtle background tint when shift is held, so the user
can see which elements are annotatable before clicking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
improvement-modal.js:
- Replace contextmenu handler with click+shiftKey check — browser
context menu is no longer intercepted
- Add keydown/keyup/blur listeners: holding Shift applies
.impr-shift-mode to <body>, switching cursor to crosshair
across the entire page as a visual affordance
- Update hint text to "Ctrl + Enter to submit · Escape to cancel"
todo.md:
- New "Improvements" section shows open dashboard-improvement TD items
with a "review →" link to the UI Feedback page
- KPI sidebar row added for open improvement count (indigo when > 0)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- improvement-modal.js: global contextmenu handler that opens a modal
showing page/widget context; submits suggestions as TD items with
debt_type="dashboard-improvement" to POST /technical-debt/
- _footer.md: shared Observable footer that auto-initialises the modal
on every dashboard page
- ui-feedback.md: review/approval page — lists open suggestions with
resolve / won't-fix / in-progress action buttons; archived items shown
below; live-polled
- observablehq.config.js: "UI Feedback" added under Workstreams group
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
