Commit Graph

17 Commits

Author SHA1 Message Date
ca8a09ed04 feat(state-hub): CUST-WP-0040 — NATS lifecycle event publishing for activity-core
Makes the state hub an event publisher so activity-core can drive
maintenance automation declaratively via ActivityDefinitions, rather
than the hub creating tasks itself.

- api/events/: lazy JetStream publisher + EventEnvelope mirroring
  activity-core's contract; no-op when NATS_URL unset, fire-and-forget
  with logged failures so publishing never breaks an API request.
- Wired publishers on the five v1.0 lifecycle events:
    org.statehub.repo.registered        (POST /repos/)
    org.statehub.workstream.completed   (PATCH /workstreams/* on transition)
    org.statehub.decision.resolved      (POST /decisions/*/resolve)
    org.statehub.domain.goal.activated  (POST /domain-goals/*/activate)
    org.statehub.task.stale             (scripts/cleanup_stale_tasks.py)
- docs/nats-event-subjects.md: subject naming convention + catalog.
- docs/cron-migration.md: design stub for replacing custodian-sync
  systemd timer and cleanup-stale cron with ActivityDefinitions
  (depends on activity-core WP-0003).
- docs/activity-core-delegation.md: protocol, invariants, cutover plan.
- SCOPE.md: declares activity-core as downstream event consumer and
  restates that the state hub stays a read model, not a task factory.

Workplan: workplans/CUST-WP-0040-state-hub-nats-activity-core-integration.md
242 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 05:49:29 +02:00
0eb2ef0650 perf(api): CUST-WP-0041 — DB indexes, TTL caches, noload on list endpoints
- Migration t7o8p9q0r1s2: indexes on tasks.status, tasks(workstream_id,status),
  workstreams.status, sbom_snapshots(repo_id,snapshot_at)
- workplan-index: 30 s TTL cache + ?refresh param (4171 ms → 16 ms on hit)
- /state/summary: 15 s TTL cache, bypassed on Cache-Control: no-cache
- /topics/: noload(workstreams, decisions, progress_events) (2382 ms → 115 ms)
- /domains/: noload(topics, repos, goals) (2252 ms → 39 ms)
- /repos/: noload(goals) (2222 ms → 599 ms first / fast on repeat)
- conftest: reset TTL caches between tests to prevent bleed-through

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 11:12:17 +02:00
512c0a73ed feat(api): dashboard poll optimisation — T1, T2, T3
T1: Cache-Control max-age=60 on /topics/, /repos/, /domains/ list endpoints
    so repeated dashboard polls within a minute are served from browser cache.

T2: ETag middleware (md5 hash) on all JSON GET responses with conditional-GET
    (304 Not Modified) support; If-None-Match and ETag added to CORS headers.
    ETag registered inside CORS so 304s automatically carry CORS headers.

T3: GET /state/deps — lightweight dep-graph endpoint returning open workstreams
    with depends_on/blocks edges only, skipping the 10-table full-summary query.
    Prerequisite for T4 (switching workstreams.md and dependencies.md off /state/summary).

Workplan: CUST-WP-0039-dashboard-poll-optimization.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 17:26:30 +02:00
30f7ac8fea Updated repo onboarding 2026-05-04 19:18:10 +02:00
01af0f8810 scope refactoring 2026-05-01 01:47:14 +02:00
273bcb5b78 state-hub scope functionality work 2026-05-01 01:33:15 +02:00
17303d2519 feat(state-hub): Interface Change Registry (CUST-WP-0033 T01-T06)
Adds first-class tracking for API and interface mutations across the
agent ecosystem. Breaking changes are documented, affected repos are
notified via inbox, and agents discover pending changes at session
start via the dispatch endpoint.

- Migration q4l5m6n7o8p9: interface_changes table
- Model/schema: InterfaceChange with draft→published→resolved lifecycle
- Router: POST/GET/PATCH /interface-changes/, /publish, /resolve actions
  (auto-notify affected repo agents on publish; progress event on origin)
- Dispatch: GET /repos/{slug}/dispatch now returns pending_interface_changes
- MCP tools: register_interface_change, list_interface_changes,
  publish_interface_change, resolve_interface_change
- Dashboard: /interface-changes page with type badges, planned calendar,
  published cards, and draft table
- EP-CUST-ICR-001 registered: webhook subscriptions (deliberately deferred)

First record: trailing-slash normalisation (2026-04-26), published,
affecting repo-registry — visible in repo-registry dispatch immediately.

223 tests passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 15:29:08 +02:00
8dd15efde1 fix(api): normalize trailing slashes — no slash on param routes
Rule: trailing slash only on collection roots (/). Any route containing
a path parameter {…} uses no trailing slash. Applies across all routers,
scripts, Makefile, and tests. Fixes 307-redirect fragility on POST/PATCH
from naive clients (curl, Codex HTTP calls).

Also adds POST /repos/{slug}/sync — runs ADR-001 consistency check with
--fix via HTTP, so non-MCP agents (Codex) can self-service DB sync without
operator intervention.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 15:13:01 +02:00
20ce332d55 feat(dashboard): entity list UX — REF column, name cells, detail pages (CUST-WP-0030)
- ref-cell.js: REF column component — click=copy deeplink, dblclick=open
- field-help.js: field registry + fieldRow helper with help-tip decoration;
  FK fields (task_id, workstream_id, repo_id) render as async-linked cells
  with entity-title bubble-help on hover
- GET /token-events/{id} endpoint + get-by-id tests
- GET /repos/by-id/{repo_id} UUID lookup endpoint
- Landing pages: /token-events/[id], /workstreams/[id], /repos/[slug], /tasks/[id]
- token-cost.md: REF + Name columns on all three tables; parallel fetch of
  workstreams/tasks for title resolution
- reference.md: entity detail page URL scheme documented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:35:35 +02:00
091695766b feat(repos): git-fingerprint-based machine-independent repo identity
Add git_fingerprint (root commit SHA-1) to managed_repos as a stable,
machine-independent identifier — identical across every clone regardless
of checkout path, remote URL, or SSH alias.

- Migration n1i2j3k4l5m6: adds git_fingerprint column + non-unique index
  (non-unique to support repos that share ancestry via forks/splits)
- GET /repos/by-fingerprint?hash=<sha>[&remote_url=<url>]: lookup by
  fingerprint; optional remote_url disambiguates shared-ancestry repos
- GET /repos/by-remote?url=<url>: fallback lookup by remote URL
- consistency_check.py --here [PATH]: auto-detects repo slug from any
  local checkout via fingerprint (falls back to remote URL), then auto-
  registers host_paths[hostname] so subsequent runs need no override
- --all now includes repos with host_paths[current_hostname], not just
  those with local_path
- fix-consistency-here / check-consistency-here Makefile targets
- Fixed _api_get bug: httpx strips query strings when params={} is passed
- Backfilled fingerprints for 14 repos on this host

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 23:55:06 +01:00
d63e7310d5 perf(doi): fingerprint-based DB cache for DoI results
Adds doi_cache table (migration k8f9a0b1c2d3). Results are stored after
each evaluation and reused on subsequent requests when the fingerprint
matches. Fingerprint covers repo.updated_at, latest TPSC snapshot_at,
latest goal updated_at, and mtime of SCOPE.md / CLAUDE.md / tpsc.yaml.

Behaviour:
- Summary (warm cache, nothing changed): ~0.4s (was 0.9s)
- Summary (one repo stale): ~0.9s (only stale repos recomputed)
- Single repo (cache hit): ~0.2s (was 40s for full check)
- Single repo ?force_refresh=true: ~2s (full C7/C13 subprocess check)

Total journey: 108s (original) → 6s → <1s → 0.2s (cached single repo)

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:47:19 +01:00
6ae5cb6bf7 perf(doi): eliminate HTTP self-calls in summary — 48 calls → 3 bulk DB queries
Root cause: C2/C9/C10 each made a full HTTP round-trip back to the API
(asyncio.to_thread → urllib → TCP → uvicorn → SQLAlchemy → DB) for every
repo. 16 repos × 3 calls = 48 self-calls at ~80-150ms each = ~6s total.

Fix: doi_engine.evaluate() accepts a prefetch dict. The summary endpoint
runs 3 bulk GROUP BY queries (domain status, TPSC snapshot counts, active
goal counts) and passes results directly — zero HTTP self-calls in summary
mode.

Result: /repos/doi/summary 6s → <1s (6× improvement on top of prior 13×).
Total improvement from original: 108s → <1s.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:37:40 +01:00
66e3a9afe4 perf(doi): 13x speedup for /repos/doi/summary (108s → ~6s)
Two fixes:
1. skip_consistency=True in summary mode — omits C7/C13 subprocess calls
   (consistency_check.py) which were the main bottleneck (32 spawns for 16 repos).
   Full check still available per-repo via GET /repos/{slug}/doi.
2. asyncio.gather — all repos evaluated in parallel instead of sequentially.

Also: rename Repositories page title from "Repos" to "Repositories".

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:29:27 +01:00
f94ee008b5 feat(doi): Repository DoI automated gate and dashboard integration (CUST-WP-0024)
Implements the 14-criterion DoI checklist as a runnable gate with API,
MCP tools, CLI script, and dashboard integration.

Core components:
- api/doi_engine.py — async engine evaluating all 14 criteria (asyncio.to_thread
  for non-blocking HTTP self-calls), shared by API and CLI
- api/schemas/doi.py — DoICriterion, DoIReport, DoISummaryEntry schemas
- api/routers/repos.py — GET /repos/{slug}/doi + GET /repos/doi/summary
- scripts/check_doi.py — CLI: make check-doi REPO=<slug> / check-doi-all
- mcp_server/server.py — check_repo_doi(), get_doi_summary() tools

Dashboard (repos.md):
- DoI tier badge per repo (None/Core/Standard/Full) colour-coded red→green
- Domain block shows lowest DoI tier across its repos
- DoI KPI card in summary row
- DoI filter in All Repos Table
- Link to Repository DoI policy page

Also fixes: TPSC snapshots 500 error (missing nested selectinload for
catalog_entry relationship in list_snapshots endpoint).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:08:18 +01:00
2c8561be70 feat(repos): multi-machine path support via host_paths
Adds a JSONB column `host_paths` to managed_repos mapping
hostname → absolute local path. Fixes the consistency-checker
failure when the same repo lives at different paths on different
machines (e.g. /home/worsch/marki-docx on the workstation vs
/home/tegwick/marki-docx on custodiancore).

Changes:
- Migration g4b5c6d7e8f9: adds host_paths JSONB (default {})
- Model: host_paths Mapped[dict] column
- Schemas: host_paths in RepoRead; new RepoPathRegister schema
- Router: POST /repos/{slug}/paths/ — merges one host entry
- consistency_check.py: resolve_repo_path() prefers host_paths
  [hostname] over local_path; --repo-path CLI override added
- MCP: update_repo_path(slug, path, host?) tool
- Makefile: register-path target; REPO_PATH passthrough on
  check-consistency and fix-consistency targets
- TOOLS.md: documents update_repo_path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 16:30:55 +01:00
07031fa63e feat(CUST-WP-0014): repo sync automation & Gitea inventory
- Migration e2f3a4b5c6d7: add last_state_synced_at to managed_repos
- consistency_check.py: PATCH last_state_synced_at after fix run;
  fix ~ treated as non-empty state_hub_task_id (C-03 vs C-11);
  fix _inject_task_id_into_block skipping injection when field exists
  with null value
- install_hooks.sh: idempotent post-commit hook installer for all
  registered repos (make install-hooks REPO= / install-hooks-all)
- gitea_inventory.py: compare coulomb Gitea org against state-hub
  registered repos — registered / unregistered / hub-only sections
- infra/README.md: document systemd user timer + crontab fallback
- systemd user timer: custodian-sync.{service,timer} runs
  fix-consistency-all every 15 min (enabled)
- dashboard/src/repo-sync.md: Repo Sync Health page — sync age table,
  unregistered Gitea repos, hub-only repos
- api/routers/repos.py: GET /repos/{slug}/dispatch endpoint returning
  active goal, pending tasks per workstream, human interventions
- mcp_server/server.py: get_repo_dispatch() MCP tool

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 01:41:16 +01:00
07a082b7b0 feat(state-hub): implement v0.5 — dynamic domains & multi-repo
Replaces the hardcoded 6-domain PostgreSQL ENUM with a first-class
`domains` DB table, and adds a `managed_repos` table for multi-repo
support per domain.

P1 — Domain as a DB entity:
- Migration b1c2d3e4f5a6: creates `domains` table, migrates topics.domain
  ENUM column to domain_id FK, drops the domain ENUM type
- Domain ORM model (api/models/domain.py) + Pydantic schemas
- Domain API router: GET/POST /domains/, GET/PATCH /domains/{slug}/,
  rename and archive endpoints with EP/TD cascade on rename
- Topic model updated: domain_id FK + @property domain_slug for
  backwards-compatible JSON serialization (field renamed domain → domain_slug)
- TopicCreate/TopicRead updated; seed.py rewritten to use FK lookup

P2 — Multi-repo support:
- ManagedRepo ORM model (api/models/managed_repo.py) + schemas
- Repo API router: GET/POST /repos/, GET/PATCH /repos/{slug}/, archive
- Makefile: add-domain, rename-domain, add-repo, list-repos targets
- register_project.sh: verify domain via /domains/ API + POST /repos/

P3 — MCP tools & live validation:
- 6 new MCP tools: list_domains, create_domain, rename_domain,
  archive_domain, list_domain_repos, register_repo
- EP/TD routers: replace hardcoded VALID_DOMAINS set with per-request
  DB lookup — returns 422 with list of valid slugs on unknown domain
- State summary: adds domains: list[DomainSummary] (slug, name,
  repo_count, active_workstream_count, ep_count, td_count)
- TOOLS.md updated with domain management section

P4 — Dashboard:
- New domains.md page with KPI row + domain cards + repo lists
- domains.json.py + repos.json.py data loaders
- Domains page added to observablehq.config.js nav
- workstreams.md, extensions.md, techdept.md: domain_slug fix +
  dynamic domain list loaded from /domains/ API (no longer hardcoded)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-28 15:20:15 +01:00