Commit Graph

121 Commits

Author SHA1 Message Date
99a66765f3 chore: close overview counts and review reliability workplans 2026-06-07 17:36:59 +02:00
0e9a4ea93b fix: normalize overview mode after view input 2026-06-07 15:38:29 +02:00
04366c64bc fix: restore overview workstream mode reactivity 2026-06-07 15:20:40 +02:00
43742560df feat: show overview workstream mode counts 2026-06-07 13:55:35 +02:00
b340489d96 Optimize dashboard overview loading 2026-06-06 00:42:00 +02:00
166aedfa8d feat: add workplan aliases and legacy meter
Adds preferred workplan REST/event surfaces, legacy-meter telemetry and weekly review summaries, documentation/dashboard terminology updates, dashboard API loading fixes, and close-out sync for STATE-WP-0052 and STATE-WP-0054.
2026-06-04 08:25:31 +02:00
8137c98a1f Add WSJF triage dashboard review page 2026-06-03 09:54:24 +02:00
38835e9e79 feat(tasks): adopt canonical task statuses 2026-05-26 01:32:50 +02:00
d4dea7864d Add workplan execution queue 2026-05-23 19:11:30 +02:00
706b360736 Harden reconciliation conflict handling 2026-05-23 18:18:44 +02:00
430923c857 Route dashboard status changes through reconciliation 2026-05-23 18:00:57 +02:00
c12091c2eb Fixed and improved token tracking 2026-05-23 13:59:05 +02:00
bb985812e5 Implement RecentlyOnScope domain digest 2026-05-22 13:45:53 +02:00
c7150bae5f Improve dashboard loading and task sync 2026-05-19 02:32:22 +02:00
cc21c5869e Implement dashboard UI improvements 2026-05-19 02:16:24 +02:00
d6522a9a40 Complete workplan state model cleanup 2026-05-18 01:31:36 +02:00
fdc395a600 chore: update standalone state hub wiring 2026-05-17 20:01:21 +02:00
90c5ea50f7 feat(dashboard): poll optimisation — T4, T5, T6
T4: workstreams.md and dependencies.md now call /state/deps instead of the
    full /state/summary — removes 2 heavy 10-table queries per 60 s cycle.

T5: index.md's 4 independent polling loops (summaryState, sbomSnapState,
    regsState, wsChartState) consolidated into a single pageState generator
    with one Promise.all batch and a shared backoff counter.

T6: config.js gains waitForVisible(ms) — pauses polling entirely while the
    tab is hidden and fires immediately on visibilitychange.  pollDelay()
    simplified (hidden-tab POLL_HIDDEN logic removed).  All 16 polling pages
    migrated from await sleep(pollDelay(...)) to await waitForVisible(pollDelay(...)).

CUST-WP-0039 complete — all 6 tasks done.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 17:58:18 +02:00
2484ed2815 Load limiting safeguards 2026-05-06 04:04:53 +02:00
47f6971c56 Updated repo onboarding 2026-05-04 19:18:10 +02:00
a8228fc780 railiance-bootstrap to railiance-cluster rename cleanup 2026-05-03 16:30:07 +02:00
e0f6a3b7a9 Overview shows workstreams by repo now and allows drilldown 2026-05-02 12:01:45 +02:00
e521f267ca Cleanup of documentation 2026-05-02 00:46:07 +02:00
548c3efe4a feat(state-hub): Interface Change Registry (CUST-WP-0033 T01-T06)
Adds first-class tracking for API and interface mutations across the
agent ecosystem. Breaking changes are documented, affected repos are
notified via inbox, and agents discover pending changes at session
start via the dispatch endpoint.

- Migration q4l5m6n7o8p9: interface_changes table
- Model/schema: InterfaceChange with draft→published→resolved lifecycle
- Router: POST/GET/PATCH /interface-changes/, /publish, /resolve actions
  (auto-notify affected repo agents on publish; progress event on origin)
- Dispatch: GET /repos/{slug}/dispatch now returns pending_interface_changes
- MCP tools: register_interface_change, list_interface_changes,
  publish_interface_change, resolve_interface_change
- Dashboard: /interface-changes page with type badges, planned calendar,
  published cards, and draft table
- EP-CUST-ICR-001 registered: webhook subscriptions (deliberately deferred)

First record: trailing-slash normalisation (2026-04-26), published,
affecting repo-registry — visible in repo-registry dispatch immediately.

223 tests passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 15:29:08 +02:00
907e99e057 fix(dashboard): merge token + event charts into single dual-axis chart on Progress page
Replaces the two separate charts with one combined area+line chart.
Events use the left y-axis (steelblue); tokens use a normalized scale
with a right y-axis (amber) that formats values as k/M. When no token
data exists yet the right axis is omitted and a legend note explains.
Hover tooltips show actual values for both series.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 00:48:22 +02:00
063f58dfdd feat(dashboard): add tokens consumed per day chart to Progress page
Fetches /token-events/?limit=1000 in parallel with progress events and
renders a second area+line chart (amber) below the events-per-day chart,
aggregating tokens_in + tokens_out per calendar day over the same 30-day window.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 00:42:09 +02:00
71ba017ce8 feat(dashboard): add repo filter, sort order, and max results controls to Token Cost page
Three reactive dropdowns below the Token Cost heading:
- Filter by repo: client-side filter via 3-level chain resolution
- Sort by: Tokens Total (default), Tokens In, Out, Event Count, Most Recent
- Show: 10/20/50/100/500 rows per table (default 20)

Applies uniformly to By Repo, By Workplan, and Top Tasks tables.
"Most Recent" derives last_event_at per group from the fetched events.
Truncated tables show a "Showing M of N" count below.

Completes CUST-WP-0030 T07–T09.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 00:02:17 +02:00
b28298a2ec feat(dashboard): entity list UX — REF column, name cells, detail pages (CUST-WP-0030)
- ref-cell.js: REF column component — click=copy deeplink, dblclick=open
- field-help.js: field registry + fieldRow helper with help-tip decoration;
  FK fields (task_id, workstream_id, repo_id) render as async-linked cells
  with entity-title bubble-help on hover
- GET /token-events/{id} endpoint + get-by-id tests
- GET /repos/by-id/{repo_id} UUID lookup endpoint
- Landing pages: /token-events/[id], /workstreams/[id], /repos/[slug], /tasks/[id]
- token-cost.md: REF + Name columns on all three tables; parallel fetch of
  workstreams/tasks for title resolution
- reference.md: entity detail page URL scheme documented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 22:35:35 +02:00
acb30978cd feat(token-tracking): repo aggregation via graph walk (task→workstream→repo)
By Repo now resolves via the full chain rather than requiring repo_id
directly on the token event:
  1. token_events.repo_id (direct)
  2. → workstreams.repo_id (via workstream_id)
  3. → task.workstream_id → workstreams.repo_id (via task_id)

Changes:
- Auto-populate repo_id on token events at creation time (both the
  token_events router and the tasks router)
- New GET /token-events/by-repo/ endpoint with RepoTokenSummary schema;
  returns tokens_in/out/total, event_count, by_model, by_note per repo
- Dashboard By Repo section uses /by-repo/ directly and shows repo_slug
  instead of a truncated UUID
- Backfilled the three existing events (userbased) with repo_id via SQL

185 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 19:05:23 +02:00
58e1bafce9 feat(token-tracking): record AI token consumption per task (CUST-WP-0029)
Introduces end-to-end token consumption tracking so agent work is
visible as a cost/effort metric alongside tasks and workplans.

- Migration o2j3k4l5m6n7: token_events table with FK indexes on
  task_id, workstream_id, repo_id, created_at
- ORM model, Pydantic schemas (TokenEventCreate, TokenEventRead with
  computed tokens_total, TokenSummary)
- Router: POST /token-events/, GET /token-events/ (7 filters),
  GET /token-events/summary/ (task|workstream|repo|commit|release scope)
- MCP tools: record_token_event, get_token_summary (formatted table)
- update_task_status enriched with optional tokens_in/tokens_out
  passthrough — one call creates status update + token event
- Dashboard token-cost.md page: by-repo bar, by-workplan table,
  by-model bar, top-10 tasks by tokens
- ralph-workplan skill updated with token reporting guidance and
  per-task heuristics for estimating counts
- Tests: test_token_events.py + test_token_passthrough.py (182 pass)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 17:46:46 +02:00
fb6b786336 docs(dashboard): add technical reference page for Observable Framework dashboard
Documents the dashboard's architecture, framework choice rationale, data-fetching
strategies (static loaders + live polling), component library, page inventory,
and key features including the Workstream Health Index and entity modals.
Also registers the new page in the Reference nav and adds runbook section for
node overload / runaway agent process (INC-002) with hardening checklist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 00:09:18 +01:00
dff9806bb6 ops: establish ops/ directory with Gitea runbook and INC-001 incident report
- Create ops/runbooks/gitea-coulombcore.md — recovery checklist for Gitea
  on COULOMBCORE, documents containerd StartError pattern and CPU budget issue
- Create ops/incidents/2026-03-25-gitea-pgpool-crashloop.md — INC-001 post-mortem
  for 13-day Gitea outage (PGPool CrashLoopBackOff + rolling update CPU deadlock)
- Create ops/README.md — index for runbooks and incidents
- state-hub/dashboard/src/docs/connecting.md: add railiance01 tunnel config
  (was previously unsaved)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 11:30:44 +01:00
b3a44fb4f3 feat(capability-requests): add routing dispute & reroute workflow (CUST-WP-0027)
Adds a structured dispute mechanism when capability request routing is wrong:
- New `routing_disputed` status with four DB columns (dispute_reason, disputed_by,
  dispute_suggested_domain, disputed_at) via Alembic migration m0h1i2j3k4l5
- POST /capability-requests/{id}/dispute — any party can flag misrouting with a reason
  and optional suggested domain; notifies custodian + current fulfilling domain
- POST /capability-requests/{id}/reroute — custodian re-routes to correct domain via
  catalog_entry_id or direct slug; appends audit trail to routing_note; resets to requested
- Two new MCP tools: dispute_capability_routing and reroute_capability_request
- Dashboard: amber disputed-banner at top of Summary, routing_disputed Kanban column,
  dispute details (reason, suggested domain, raised-by) shown on disputed cards

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 23:58:52 +01:00
b6103d1f9f feat(dashboard): add Tools & Apps page with liveness probes
New page at /tools listing all connected applications grouped by
category: Local Services (State Hub API, KeePassXC, pgAdmin, ops-bridge),
Source Control (Gitea), Identity/Auth (KeyCape, Authelia, privacyIDEA,
LLDAP), and Dev Tooling (Claude Code, uv). Local services show live
green/red/grey status dots via no-cors fetch probes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 01:18:11 +01:00
101c953e69 docs: add State Hub reference page and restructure reference index
New page (docs/state-hub.md) covers:
- Why: the invisible state problem across repos and agents
- What: Derived Data Store, Read Model, Agent Orchestration Layer,
  Cross-Repo Observatory — and what it is NOT
- Derived Data Store principle (ADR-003): fingerprint cache, rebuild
  guarantee, force-refresh
- Repository Orchestrator: session protocol, cross-domain coordination
  via messages + capability routing, Kaizen agents
- Architecture diagram (ASCII), technology choices, data model overview
- Running the hub, design principles, related docs

reference.md: add Architecture & Design section grouping state-hub,
TPSC, GDPR maturity, SCOPE.md, capabilities, and goals docs.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 02:01:58 +01:00
9ba1501b49 perf(dashboard): lazy-load DoI tiers on Repositories page
Page now renders in ~200ms. DoI badges and KPI card show a spinner
while the background fetch resolves (~6s), then update reactively
via Observable Mutable pattern (doiData / doiLoading).

Fast path: repos, SBOM, domains, workstreams — immediate render.
Slow path: /repos/doi/summary — background, non-blocking.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:31:48 +01:00
27e755815f perf(doi): 13x speedup for /repos/doi/summary (108s → ~6s)
Two fixes:
1. skip_consistency=True in summary mode — omits C7/C13 subprocess calls
   (consistency_check.py) which were the main bottleneck (32 spawns for 16 repos).
   Full check still available per-repo via GET /repos/{slug}/doi.
2. asyncio.gather — all repos evaluated in parallel instead of sequentially.

Also: rename Repositories page title from "Repos" to "Repositories".

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:29:27 +01:00
5eeeeeb6c4 feat(doi): Repository DoI automated gate and dashboard integration (CUST-WP-0024)
Implements the 14-criterion DoI checklist as a runnable gate with API,
MCP tools, CLI script, and dashboard integration.

Core components:
- api/doi_engine.py — async engine evaluating all 14 criteria (asyncio.to_thread
  for non-blocking HTTP self-calls), shared by API and CLI
- api/schemas/doi.py — DoICriterion, DoIReport, DoISummaryEntry schemas
- api/routers/repos.py — GET /repos/{slug}/doi + GET /repos/doi/summary
- scripts/check_doi.py — CLI: make check-doi REPO=<slug> / check-doi-all
- mcp_server/server.py — check_repo_doi(), get_doi_summary() tools

Dashboard (repos.md):
- DoI tier badge per repo (None/Core/Standard/Full) colour-coded red→green
- Domain block shows lowest DoI tier across its repos
- DoI KPI card in summary row
- DoI filter in All Repos Table
- Link to Repository DoI policy page

Also fixes: TPSC snapshots 500 error (missing nested selectinload for
catalog_entry relationship in list_snapshots endpoint).

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 01:08:18 +01:00
33c58233bc docs(policy): add Repository Definition of Integrated (DoI)
Three-tier checklist defining what 'fully integrated with the state-hub'
means for a repository:
- Core (Registered): registered, domain assigned, path resolves, remote URL
- Standard (Integrated): SCOPE.md, CLAUDE.md, workplan convention, SBOM, TPSC
- Full (Fully Integrated): repo goal, capabilities declared, agents template,
  clean consistency check, host paths registered

Exposed via /policy/repo-doi (editable in dashboard) and linked under Policies.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 00:35:54 +01:00
9155d13887 docs(tpsc): add GDPR Maturity Model reference page
Full reference for the 7-level CNIL/IAPP CMMI-aligned scale used in TPSC:
source frameworks, per-level descriptions, suitability guidance, key GDPR
concepts (DPA, SCCs, adequacy, BCRs, Art.9), assignment decision tree,
and authoritative references.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 00:19:07 +01:00
60beb1ff35 feat(tpsc): Third-Party Services Catalog (CUST-WP-0023)
Introduces TPSC for tracking external service dependencies with GDPR
compliance maturity (CNIL/IAPP CMMI scale), pricing model, ToS, and
data retention information across all repos.

Primary data:
- canon/tpsc/{openai,anthropic,gemini,openrouter}-api.yaml — service definitions
- tpsc.yaml in each repo (llm-connect seeded with 4 services)

State-hub additions:
- Migration j7e8f9a0b1c2: tpsc_catalog + tpsc_snapshots + tpsc_entries
- api/models/tpsc.py, api/schemas/tpsc.py, api/routers/tpsc.py
- /tpsc/catalog/, /tpsc/ingest/, /tpsc/snapshots/, /tpsc/report/gdpr endpoints
- 4 MCP tools: register_service, list_services, ingest_tpsc_tool, get_gdpr_report
- scripts/ingest_tpsc.py + make ingest-tpsc[/-all] targets
- Dashboard: tpsc.md page + docs/tpsc.md

GDPR maturity scale: unknown | non_compliant | initial | developing | defined | managed | certified
Warnings triggered at: unknown, non_compliant, initial

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 00:15:26 +01:00
d45234531b feat(capability-requests): add cross-domain capability catalog and request routing
Introduces a capability catalog (CUST-WP-0022) so domains can advertise what
they provide and agents can request capabilities from other domains with
auto-routing, lifecycle tracking, and task-unblocking on completion.

- New models: CapabilityCatalog, CapabilityRequest with full lifecycle
  (requested → accepted → in_progress → ready_for_review → completed/rejected/withdrawn)
- Migration i6d7e8f9a0b1: capability_catalog + capability_requests tables
- Router /capability-catalog and /capability-requests with accept/status endpoints
- 7 new MCP tools: register_capability, list_capabilities, request_capability,
  accept_capability_request, update_capability_request_status,
  list_capability_requests, get_capability_request
- StateSummary gains open_capability_requests count
- Dashboard: capability-requests.md page + docs/capabilities.md + docs/scope.md
- SCOPE.md: three seed capabilities documented (MCP registration, state tracking, SBOM)
- scope.template: Provided Capabilities section with example block
- scripts/ingest_capabilities.py + make ingest-capabilities[/-all] targets

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 21:07:50 +01:00
7bf3cf583a fix(dashboard): enrich repo-sync page with live SBOM snapshot stats
repos.json.py now fetches /sbom/snapshots/ alongside /repos/ and
annotates each repo with sbom_snapshot_count, sbom_entry_count, and a
last_sbom_at fallback derived from actual snapshot data. This prevents
"LastSBOM=never" when the denormalized field is out of sync.

repo-sync.md gains SBOM KPI tiles (ingested vs no-SBOM), color-coded
SBOM age column (same green/orange/red scale as state sync), and an
entry count column showing packages from the latest snapshot.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 01:34:02 +01:00
1bcc46ea3f fix(dashboard): clear API-unreachable warning when API recovers
Always call display() for the warning element so Observable Framework
replaces it on each poll re-run. Previously the conditional display()
call left the warning rendered indefinitely once shown.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 00:51:11 +01:00
ebf7c544f6 refactor(makefile): rename backend → api, fold raw uvicorn target in
The old bare `api` target (uvicorn only) is subsumed into the new `api`
target (db + postgres-wait + migrate + fuser-restart + uvicorn). Updated
all doc references and cleaned up duplicate entries left by the rename.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:20:45 +01:00
959449d82f refactor(makefile): rename start → backend, add restart logic for api and dashboard
- `make backend` replaces `make start`; polls postgres with nc (up to 10s)
  instead of fixed sleep, kills any running uvicorn before starting fresh
- `make dashboard` kills any running observable preview before restarting
- Update all references in CLAUDE.md, README.md, SCOPE.md, state-hub/README.md,
  and dashboard/src/docs/live-data.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 23:16:44 +01:00
2d0ce8f943 feat(api): CUST-WP-0018 — API hardening & code quality
T01: Fix datetime.utcnow() → datetime.now(tz=timezone.utc) in MCP server
T02: Wrap _get/_post/_patch/_delete with try/except; return error dicts
T03: Log warnings when write_log skips missing project path
T04: Add priority + due_date_before filters to GET /tasks/
T05: Add owner + slug filters to GET /workstreams/
T06: Add offset param to GET /progress/ for proper pagination
T07: Low-severity bundle:
  - CORS origins from CORS_ORIGINS env var (TD-017)
  - seed.py upsert domains+topics on re-run (TD-011)
  - normalise filter bar CSS → filter-text-input everywhere (TD-016)
  - add 30.5 avg-days-per-month comment in decisions.md (TD-019)
  - TD-009, TD-018 already resolved by existing code

Closes CUST-WP-0018.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 02:17:04 +01:00
f0e9bb0143 feat(dashboard): CUST-WP-0019 — Repository nav section, config.js cleanup
T01: Restructure nav — "Repos" → collapsible "Repository" section with
     Repo Sync, SBOM, Debt as sub-pages; Debt moved out of Workstreams
T02: workstream-dod.md migrated from inline const API to config.js import
T03: todo.md suggestion filter (done in previous commit)

Closes CUST-WP-0019. Resolves UI suggestion c2fc284a.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 02:01:10 +01:00
0969f3258c feat(workplans): CUST-WP-0018/0019/0020 — API hardening, dashboard UX polish, test suite
Consolidates all open technical debt into three workplans:
- CUST-WP-0018: API hardening & code quality (TD-006–019 medium/high items)
- CUST-WP-0019: Dashboard UX polish (Repos nav restructure, config.js cleanup,
  todo filter fix for new suggestion workflow statuses)
- CUST-WP-0020: pytest test suite with real DB (TD-014)

Also fixes todo.md Suggestions filter: was checking status===open but new
suggestions enter with status=submitted; now excludes terminal statuses only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 01:55:37 +01:00
d9b9a0eaec feat(dashboard): extend suggestions to TOC right margin + 1s shift delay
- Shift+click now works on #observablehq-toc links, KPI boxes, and [id] elements
- _inferWidgetName detects TOC context and labels suggestions accordingly
- Click handler adds inToc branch alongside existing inSidebar
- _updateMode: 1-second setTimeout before activating highlight mode
  so normal Shift+typing doesn't flicker the UI; clears immediately on
  Shift release or window blur

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 01:42:11 +01:00