Commit Graph

23 Commits

Author SHA1 Message Date
5e7a72e144 feat(CUST-WP-0014): repo sync automation & Gitea inventory
- Migration e2f3a4b5c6d7: add last_state_synced_at to managed_repos
- consistency_check.py: PATCH last_state_synced_at after fix run;
  fix ~ treated as non-empty state_hub_task_id (C-03 vs C-11);
  fix _inject_task_id_into_block skipping injection when field exists
  with null value
- install_hooks.sh: idempotent post-commit hook installer for all
  registered repos (make install-hooks REPO= / install-hooks-all)
- gitea_inventory.py: compare coulomb Gitea org against state-hub
  registered repos — registered / unregistered / hub-only sections
- infra/README.md: document systemd user timer + crontab fallback
- systemd user timer: custodian-sync.{service,timer} runs
  fix-consistency-all every 15 min (enabled)
- dashboard/src/repo-sync.md: Repo Sync Health page — sync age table,
  unregistered Gitea repos, hub-only repos
- api/routers/repos.py: GET /repos/{slug}/dispatch endpoint returning
  active goal, pending tasks per workstream, human interventions
- mcp_server/server.py: get_repo_dispatch() MCP tool

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 01:41:16 +01:00
7b7b725f8b fix(consistency): fix C-04 status vocabulary mismatch + surface PATCH errors
Root cause: workplan files use "done" (task vocabulary) but the DB workstream
API only accepts "completed". The PATCH was silently failing with 422.

Fixes:
- Add FILE_TO_DB_WORKSTREAM_STATUS map and normalise_workstream_status()
- Normalise file status before C-04 comparison: done↔completed is no longer
  spurious drift
- Normalise file status before PATCHing: always send DB-valid "completed"
- _api_patch now returns {"_error": ...} instead of None on failure, so the
  fix loop reports FAILED entries rather than silently dropping them
- 9 new tests in TestNormaliseWorkstreamStatus (42 total)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 21:57:11 +01:00
df083b1840 feat(sbom): CUST-WP-0013 — expand SBOM infra to terraform, ansible, and tool manifests
- Migration d6e7f8a9b0c1: add terraform, ansible, tool to Ecosystem enum
- ingest_sbom.py: new Ansible Galaxy requirements.yml parser (collections + roles)
- ingest_sbom.py: new sbom-tools.yaml manifest parser (agent-generated tool deps)
- ingest_sbom.py: promote .terraform.lock.hcl parser from ecosystem=other → terraform
- ingest_sbom.py: detect_all() runs all four parsers in one comprehensive scan
- capture_sbom_tools.py: agent-assisted tool manifest generator (claude -p)
- prompts/sbom-capture-agent.md: parameterised prompt for repo tool discovery
- Makefile: capture-tools target; ingest-sbom updated docs and DRY_RUN support
- 29 unit tests covering all new parsers and detect_all() behaviour
- canon/standards/sbom-convention_v0.1.md: updated with four-mechanism model and workflow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 04:40:26 +01:00
651df73e3a feat(goals): add domain/repo goal tracking and update_workstream MCP tool
- Migration c5d6e7f8a9b0: domain_goals and repo_goals tables, repo_goal_id FK on workstreams
- DomainGoal: one active per domain (partial unique index), status active/archived/superseded
- RepoGoal: integer priority, status active/paused/completed/archived, optional domain_goal_id link
- WorkstreamUpdate schema and router extended with repo_goal_id and repo_goal_id filter
- 6 new MCP goal tools: create_domain_goal, get_domain_goals, activate_domain_goal, create_repo_goal, get_repo_goals, update_repo_goal
- update_workstream MCP tool: patch any subset of workstream fields (title, description, owner, due_date, repo_goal_id, status)
- get_domain_summary extended with goal_guidance (needs_workplan, alignment_warnings) signals
- Dashboard goals.md page and docs/goals.md reference page
- CLAUDE.md template updated to act on goal_guidance signals at session start
- CUST-WP-0010 workplan for this feature

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-09 00:15:29 +01:00
af25634f93 fix(template): replace get_state_summary with get_domain_summary in domain CLAUDE.md template
Avoids ~12.9k token response in domain repo sessions; get_domain_summary
returns the same actionable data scoped to the domain at ~10% of the cost.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-05 09:09:01 +01:00
c792ab0bc0 feat(tasks): add needs_human intervention flag (CUST-WP-0009)
- Migration b4c5d6e7f8a9: adds needs_human (bool) + intervention_note (text) to tasks
- API: needs_human filter on GET /tasks/; 422 if flagged without note
- 3 MCP tools: flag_for_human, clear_human_flag, list_human_interventions
- Dashboard: interventions.md with amber cards and "Mark done" button
- Policy router + workstream DoD policy (workstream-dod.md)
- Workstream lifecycle docs page + workplan CUST-WP-0010
- CLAUDE.md: add step 4 (run fix-consistency after workplan writes)
- consistency_check.py: promote C-11 unlinked tasks from INFO to WARN

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 19:44:14 +01:00
5c1b7e7e1d feat(consistency): implement ADR-001 consistency checking engine (CUST-WP-0008)
Adds state-hub/scripts/consistency_check.py with C-01 through C-12 checks:
bidirectional file↔DB validation, --fix for auto-fixable issues, --all for all
repos, --json output, exit codes 0/1/2.

MCP tool: check_repo_consistency(repo_slug, fix=False)
Makefile: check-consistency, fix-consistency, check-consistency-all, fix-consistency-all

Auto-fixes applied across all repos:
- C-09: activity-core-foundation + activity-core-triggers-ops repo_id → activity-core
- C-04: railiance phase-0-operational-baseline status → completed
- C-05: railiance phase-0 title synced from file
- C-10/C-11: task status drifts resolved; state_hub_task_id injected into
  CUST-WP-0006 and CUST-WP-0007 task blocks

Remaining orphans reported for human review: repo-integration-activity-core,
infospace-s3-closeout, testdrive-jsui-publication, staged-promotion-lifecycle,
three-phoenix-ha-cluster, current-env-safety-net.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 08:16:00 +01:00
8a9314ded6 feat(registration): write CLAUDE.custodian.md instead of overwriting CLAUDE.md
Instead of overwriting the target repo's CLAUDE.md, the registration
script now writes CLAUDE.custodian.md — a suggestion file with an
integration header. The repo's Claude agent integrates both files and
deletes the suggestion when done, preserving existing project conventions.

Also fix: `read` prompt now redirects from /dev/tty so the script
doesn't exit with code 1 when run non-interactively via make.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 01:30:28 +01:00
2d11bfa0ba feat(maintenance): add stale-task cleanup scheme
- scripts/cleanup_stale_tasks.py: daily script that cancels open tasks
  in completed/archived workstreams; handles 307 redirects; emits a
  cleanup progress event summarising results
- Makefile: add cleanup-stale target (also suitable for cron)
- ADR-001: append Workstream Closure Protocol section — mandatory closure
  review before marking workstream completed, with task classification
  table (done/cancelled/carry-forward) and Closure Review file format
- WP-0002 + WP-0005: append Closure Review sections documenting the
  2026-03-02 cleanup run (26 stale DB rows cancelled — all were legacy
  pre-ADR-001 DB-first records; file status was already done)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 00:32:35 +01:00
70c8e3cd51 feat(mcp): add get_domain_summary() for low-token domain session orientation
get_state_summary() returns ~10k tokens — too expensive for routine domain
repo sessions that only need their own workstreams and decisions.

New get_domain_summary(domain_slug):
- 5 targeted API calls: topics (filter), workstreams (topic+status), decisions
  (topic+pending), progress (topic, limit 5), repos (domain, slug+SBOM only)
- Returns: topic, active workstreams, blocking decisions, 5 recent events,
  repo SBOM status — all scoped to one domain
- Estimated ~80-90% token reduction vs get_state_summary()

get_state_summary() preserved unchanged for cross-domain / custodian sessions.
Updated its docstring to note the large response and point to get_domain_summary.

Template updated: Step 1 now calls get_domain_summary("{DOMAIN}") instead of
get_state_summary() + get_next_steps(). TOOLS.md updated with usage guidance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 22:05:31 +01:00
ba89ebfa67 feat(canon): add inter-repo communication standard with todo taxonomy
Establishes the repo boundary rule and a formal vocabulary for classifying
work items by scope:

- Task: neutral state hub data entity
- Todo: a task scoped to the current session's repo/domain
  - Internal todo: addressed within this repo by this agent
  - Ecosystem todo: work for another registered repo → state hub task [repo:<slug>]
  - Third-party todo: work for an upstream repo → contribution artifact (BR/FR/EP/UPR)

New dashboard doc: /docs/inter-repo-communication — defines the boundary rule,
the full terminology, ecosystem and third-party todo workflows, and a decision
table for classifying any piece of work found during a session.

Also:
- sbom.md: replace verbose inter-repo section with a 3-line summary + link
- observablehq.config.js: add "Inter-Repo Communication" to Reference nav
- project_claude_md.template: add "### Repo Boundary Rule" section; fix
  Workplan Convention section (removing incorrect claim that the custodian
  writes workplan files in other repos — that is the target repo's job)

Cross-repo: created state hub task [repo:railiance-bootstrap] for that repo's
agent to apply the boundary rule and workplan convention fix to its own CLAUDE.md
(task 78d43cb0, workstream 59155efb).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 20:52:07 +01:00
98e991b49f fix(template): use reliable workplan discovery in step 2
Glob with pattern 'workplans/*.md' from repo root fails silently.
Changed instruction to Glob(pattern="**/*.md", path="workplans/")
with Bash ls as fallback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 20:13:31 +01:00
00272842ca fix(template): rewrite session protocol to produce concrete orientation output
The previous template only defined a First Session Protocol (triggered when no
workstreams existed). When workstreams did exist, get_state_summary() was called
but no output was defined, causing registered-repo Claude sessions to produce
nothing useful.

New 3-step normal session protocol:
- Step 1: get_state_summary() + get_next_steps()
- Step 2: scan workplans/*.md for active tasks (todo/in_progress)
- Step 3: output orientation brief covering active workstreams, pending tasks
  for this repo (from workplans/ + [repo:<slug>] state hub tasks), suggested
  next action, and SBOM status

Also strengthens First Session Protocol, ADR-001 workplan convention section,
and SBOM ingest section (adds SCAN=1 REPO_PATH= flags).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 20:05:16 +01:00
fae9151144 feat(sbom): add Terraform .terraform.lock.hcl parser; ingest railiance repos
- ingest_sbom.py: parse .terraform.lock.hcl provider blocks (name, version);
  ecosystem stored as 'other' until terraform added to DB ENUM
- Registered railiance-bootstrap + railiance-hosts under railiance domain
- railiance-hosts ingested: 2 Terraform providers (hashicorp/template 2.2.0,
  hetznercloud/hcloud 1.52.0)
- railiance-bootstrap: no lockfile (pure Ansible/shell — noted in convention)
- sbom-convention_v0.1.md: add Terraform + Ansible rows to lockfile table;
  update registered repos status table

Total SBOM: 422 packages across 2 repos (custodian + railiance-hosts)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 18:07:56 +01:00
4c157d43a8 feat(sbom): scan mode, domain grouping dashboard, SBOM convention doc
- ingest_sbom.py: add --scan flag (recursive lockfile discovery) +
  --lockfile repeatable for explicit multi-file ingestion; skip
  .venv/node_modules/.git/dist/etc; Makefile gains SCAN= and REPO_PATH= vars
- sbom.md: add /domains/ fetch; domain-level summary table; per-repo
  accordion with details/summary; domain filter on package table; dual-
  licence false-positive note; +1 KPI card (Domains Covered)
- canon/standards/sbom-convention_v0.1.md: authoritative lockfile table,
  ingest workflow (single/scan/explicit), snapshot semantics, direct-vs-
  transitive caveats, licence governance + copyleft escalation, update
  cadence, multi-repo domain pattern, planned enhancements

First ingest: the-custodian — 420 pkgs (88 python + 332 node), 13 licence
groups, 1 copyleft flag (jszip dual-licensed MIT OR GPL-3.0-or-later)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 16:15:40 +01:00
7d3487d4fe feat(state-hub): v0.3 registration workflow + ingest-sbom + CLAUDE.md template update
- scripts/ingest_sbom.py: lockfile parser + API poster for uv.lock, requirements.txt,
  package-lock.json, yarn.lock, Cargo.lock; auto-detects from repo root
- Makefile: make ingest-sbom REPO=<slug> [LOCKFILE=<path>] target
- scripts/register_project.sh: adds {REPO_SLUG} template substitution + optional
  SBOM ingest prompt at end of registration (non-fatal if venv not ready)
- scripts/project_claude_md.template: adds Contribution Tracking + SBOM sections
  documenting register_contribution(), update_contribution_status(), ingest-sbom,
  and the contrib/ directory layout
- workplans/CUST-WP-0002: all 15 tasks → done, status → completed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 17:28:49 +01:00
fcd0f06536 feat(state-hub): implement v0.5 — dynamic domains & multi-repo
Replaces the hardcoded 6-domain PostgreSQL ENUM with a first-class
`domains` DB table, and adds a `managed_repos` table for multi-repo
support per domain.

P1 — Domain as a DB entity:
- Migration b1c2d3e4f5a6: creates `domains` table, migrates topics.domain
  ENUM column to domain_id FK, drops the domain ENUM type
- Domain ORM model (api/models/domain.py) + Pydantic schemas
- Domain API router: GET/POST /domains/, GET/PATCH /domains/{slug}/,
  rename and archive endpoints with EP/TD cascade on rename
- Topic model updated: domain_id FK + @property domain_slug for
  backwards-compatible JSON serialization (field renamed domain → domain_slug)
- TopicCreate/TopicRead updated; seed.py rewritten to use FK lookup

P2 — Multi-repo support:
- ManagedRepo ORM model (api/models/managed_repo.py) + schemas
- Repo API router: GET/POST /repos/, GET/PATCH /repos/{slug}/, archive
- Makefile: add-domain, rename-domain, add-repo, list-repos targets
- register_project.sh: verify domain via /domains/ API + POST /repos/

P3 — MCP tools & live validation:
- 6 new MCP tools: list_domains, create_domain, rename_domain,
  archive_domain, list_domain_repos, register_repo
- EP/TD routers: replace hardcoded VALID_DOMAINS set with per-request
  DB lookup — returns 422 with list of valid slugs on unknown domain
- State summary: adds domains: list[DomainSummary] (slug, name,
  repo_count, active_workstream_count, ep_count, td_count)
- TOOLS.md updated with domain management section

P4 — Dashboard:
- New domains.md page with KPI row + domain cards + repo lists
- domains.json.py + repos.json.py data loaders
- Domains page added to observablehq.config.js nav
- workstreams.md, extensions.md, techdept.md: domain_slug fix +
  dynamic domain list loaded from /domains/ API (no longer hardcoded)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 15:20:15 +01:00
c3efb099f1 feat(custodian): add ADR-001 compliance validator
Scripts, Makefile target, and MCP tool for checking a repository
against ADR-001 (workplans as repo artefacts, state-hub as cache).

Checks performed:
  File-side: workplans/ dir exists, valid YAML frontmatter (required
  fields, type, status, id format), filename matches id, embedded
  task blocks have id/status/priority.

  State-hub cross-reference: state_hub_workstream_id references
  resolve to real DB records; orphan detection flags active DB
  workstreams with no backing workplan file.

Usage:
  make validate-adr REPO=<path> [DOMAIN=<slug>]
  validate_repo_adr(repo_path, domain_slug?)  # MCP tool

Running against the-custodian itself correctly surfaces the 4
pre-ADR-001 workstreams that still need workplan files written.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 12:00:09 +01:00
f34b49ebde Implement State Hub v0.2: dependency graph, next-steps suggestions, design boundary
S0 — Design boundary formalised across all integration surfaces:
- TOOLS.md restructured with Design Boundary section, Sanctioned Write Tools,
  and Bootstrap-Only Tools (create_workstream, create_task) with explicit note
- project_claude_md.template and railiance CLAUDE.md updated with boundary note
  and get_next_steps() in session start protocol
- Global ~/.claude/CLAUDE.md updated accordingly

S1 — Workstream dependency graph:
- WorkstreamDependency model (directed edge, CASCADE on delete, unique pair constraint)
- Alembic migration 0b547c153153; script.py.mako added (was missing)
- REST API: POST/GET /workstreams/{id}/dependencies/, DELETE …/{dep_id} (hard delete)
- StateSummary open_workstreams enriched with depends_on/blocks lists
- MCP tools: create_dependency(), list_dependencies()
- Dashboard workstreams page: Dependencies section with relationship cards
- Seeded: custodian-agent-runtime → llm-shared-library + phase-0-operational-baseline

S2 — Suggesting Next Steps (sanctioned write use case #2):
- GET /state/next_steps derives suggestions from recently resolved decisions
  (→ first open task in same workstream) and cleared dependencies
  (→ first todo task in now-unblocked workstream)
- StateSummary.next_steps included on every summary call
- MCP tool: get_next_steps()
- Dashboard: "What's next?" card grid above Registered Projects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-25 23:33:14 +01:00
80e0c85281 Make first-message behaviour explicit in CLAUDE.md template
Add one-line imperative at the top of the Session Protocol:
  'On receiving your first message — before writing any response text —
  call get_state_summary() immediately.'

Previously Claude would wait for a substantive prompt before acting.
Now any first message (including 'start', 'go', or just Enter) triggers
the tool call immediately, after which the First Session Protocol takes over.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 23:48:28 +01:00
fda64c8eba Add First Session Protocol to project CLAUDE.md template
When get_state_summary() shows no workstreams for the domain, Claude
now has explicit instructions: read the canon charter + roadmap, survey
the repo for in-progress work, propose 1-3 workstreams to Bernd, wait
for approval, then create workstreams + tasks and record a milestone.

The "wait for approval before creating anything" gate keeps the human
in control while making the expected first-session behaviour unambiguous.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 23:43:39 +01:00
ad87153f2f Implement registration UX wishlist W1–W6 (260224)
W1: Document user-scope MCP config location in ~/.claude/CLAUDE.md —
    adds verification and re-registration commands, warns against
    settings.json (saves ~12K tokens per registration session).

W2: scripts/register_project.sh + make register-project —
    5-step automation: API health → topic lookup → MCP check →
    CLAUDE.md from template → progress event.

W3: state-hub/scripts/project_claude_md.template —
    parameterised CLAUDE.md with {PROJECT_NAME}/{DOMAIN}/{TOPIC_ID}
    placeholders; used by register_project.sh.

W4: Add custodian_topic_id + domain to all 6 canon project charters —
    lets agents grep for topic IDs without touching the API.

W5: state-hub/mcp_server/TOOLS.md — compact 30-line tool reference
    card; replaces reading the full server.py (~350 lines).

W6: Switch .mcp.json to absolute path + PYTHONPATH env so cwd is not
    required; add scripts/patch_mcp_cwd.py for post-registration fix.
    Update ~/.claude.json to match (cwd kept for belt-and-suspenders).

W7 (SessionStart hook) deferred: no SessionStart hook type in Claude
    Code; PreToolUse with empty matcher fires before every tool call.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 22:22:53 +01:00
0ea2788943 Add state-hub v0.1 — local-first state service for the Custodian
Implements the first live layer of the Custodian cognitive infrastructure:
PostgreSQL schema, FastAPI REST API, FastMCP stdio server, and Observable
Framework telemetry dashboard.

- state-hub/: full stack (docker-compose, FastAPI, Alembic, MCP server, dashboard)
- 5 DB tables: topics, workstreams, tasks, decisions, progress_events
- 11 MCP tools + 5 resources registered in .mcp.json
- Observable dashboard: Overview, Workstreams, Decisions, Progress pages
- CLAUDE.md: session protocol (get_state_summary / add_progress_event ritual)
- ~/.claude/CLAUDE.md: global cross-project reference to the hub
- scripts/pull_image.py: WSL2 TLS-resilient Docker image downloader

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 17:47:49 +01:00