diff --git a/state-hub/docs/workplan-state-model-proposal.md b/state-hub/docs/workplan-state-model-proposal.md new file mode 100644 index 0000000..ed5144f --- /dev/null +++ b/state-hub/docs/workplan-state-model-proposal.md @@ -0,0 +1,142 @@ +# Workplan State Model Proposal + +Date: 2026-05-17 +Status: proposed + +## Problem + +State Hub currently mixes three different vocabularies for the same lifecycle: + +- Workstream API/schema: `todo`, `active`, `blocked`, `completed`, `archived` +- Workplan consistency scripts: `active`, `completed`, `archived` +- Dashboard overview modes: `active`, `accepted`, `finished`, `blocked`, + `stalled`, `oldies` + +This makes file-backed workplans harder to reason about. A workplan can be a +good proposal but not ready to execute; a ready workplan can become stale after +the repo changes; parked backlog work should not clutter the current work view. + +Task status should remain separate. Tasks can keep using +`todo`, `in_progress`, `blocked`, `done`, and `cancelled`. + +## Proposed Canonical Workplan States + +These states should be valid in workplan frontmatter, the workstream API, the +database, consistency checks, and the dashboard. + +| State | Meaning | Current-view behavior | +|-------|---------|-----------------------| +| `proposed` | A plan exists, but it must be reviewed against the current repo state before execution. | Hidden from active execution views by default; visible in planning/review views. | +| `ready` | The plan has been reviewed and is ready to execute against the current repo state. | Visible in "ready to pick up" views. | +| `active` | Work has started and should appear in current work views. | Visible in current execution views. | +| `blocked` | Work cannot proceed until a dependency, decision, or human intervention clears. | Visible in current execution views and blocked views. | +| `backlog` | Intentionally parked for later; not currently being reviewed or executed. | Hidden from current work views by default. | +| `finished` | Implementation is complete and the workplan is closed. | Hidden from current work views; visible in completed/history views. | +| `archived` | Historical record, no longer part of normal planning or current execution. | Hidden except in archive/history views. | + +Recommended normal path: + +```text +backlog -> proposed -> ready -> active -> finished -> archived + \ \ + \ -> blocked -> active + -> backlog +``` + +## Staleness And Health Labels + +`stalled`, `oldies`, and `stale` should be derived health labels, not stored +lifecycle states. + +| Label | Derivation | Recommended UI name | +|-------|------------|---------------------| +| `stale` | `ready` workplan was reviewed against an older repo state. | Needs review | +| `stalled` | `active` or `blocked` workplan has had no meaningful progress after a threshold. | Stalled | +| `oldies` | Old planning item with no progress. | Replace with `proposed`, `backlog`, or `needs review` depending on intent. | + +For `ready` workplans, add optional frontmatter: + +```yaml +reviewed_at: "YYYY-MM-DD" +reviewed_by: "human-or-agent" +reviewed_against_commit: "" +context_paths: + - "path/or/glob" +``` + +Initial consistency behavior: + +- If a workplan is `ready` and `reviewed_against_commit` differs from repo + `HEAD`, report a warning: "ready workplan may be stale". +- If `context_paths` are present, only warn when files under those paths changed + since `reviewed_against_commit`. +- In `--fix` mode, optionally move stale `ready` workplans back to `proposed` + and record a progress event. This should be guarded behind an explicit flag at + first, because automatic demotion can be surprising. + +## UI Cleanup + +Replace the current overview mode set: + +```text +active, accepted, finished, blocked, stalled, oldies +``` + +with stored status filters plus health filters: + +```text +proposed, ready, active, blocked, backlog, finished, archived +needs review, stalled +``` + +Specific cleanup: + +- Rename `accepted` to `finished` if it currently means DB `completed`. +- Stop using `finished` as a derived "no open tasks" pseudo-state. Show that as + a badge such as `all tasks done` or `ready to close`. +- Remove `oldies` as a lifecycle term. It is charming, but it blurs backlog, + stale proposal, and neglected active work into one bucket. +- Keep `stalled` as a health filter/badge, not as a status value. + +## Compatibility And Migration + +For one migration window, accept legacy aliases in parsers and API clients: + +| Legacy value | Canonical value | +|--------------|-----------------| +| `todo` | `ready` for workstreams with no started tasks; otherwise `active` | +| `completed` | `finished` | +| `accepted` | `finished` | +| `oldies` | not stored; classify as `proposed` or `backlog` manually | +| `stalled` | not stored; derive from timestamps/progress | + +The consistency engine should normalize legacy file values before comparison, +but write back canonical values when it changes files. + +## Implementation Work + +1. Add a shared `WORKPLAN_STATUSES` / `WORKSTREAM_STATUSES` constant used by API + schemas, consistency scripts, validation scripts, and dashboard code. +2. Update `api/schemas/workstream.py` to allow the canonical state set. +3. Update `scripts/consistency_check.py` and `scripts/validate_repo_adr.py`: + accept the canonical states, map legacy aliases, and add stale-ready checks. +4. Add a migration that rewrites existing DB values: + `completed -> finished`; classify `todo` as `ready` or `active`. +5. Update dashboard overview modes and docs: + stored statuses become filters; `stalled` and `needs review` become health + filters. +6. Update project registration templates and AGENTS/CLAUDE guidance so new + repos use the canonical frontmatter states. +7. Add tests for parsing, normalization, stale-ready detection, API validation, + and dashboard filter classification. + +## Acceptance Criteria + +- A file-backed workplan can use `proposed`, `ready`, `active`, `blocked`, + `backlog`, `finished`, or `archived` without consistency warnings. +- Dashboard labels match stored lifecycle states and do not invent lifecycle + terms. +- `stalled` and `needs review` appear as derived health labels. +- A `ready` workplan can be flagged as stale after relevant repo changes. +- Legacy `completed` and `todo` workstreams are migrated or normalized without + breaking existing data. diff --git a/workplans/CUST-WP-0042-workplan-state-model-cleanup.md b/workplans/CUST-WP-0042-workplan-state-model-cleanup.md new file mode 100644 index 0000000..639f4ba --- /dev/null +++ b/workplans/CUST-WP-0042-workplan-state-model-cleanup.md @@ -0,0 +1,214 @@ +--- +id: CUST-WP-0042 +type: workplan +title: "Workplan State Model Cleanup" +domain: custodian +repo: the-custodian +status: active +owner: custodian +topic_slug: custodian +planning_priority: high +planning_order: 42 +state_hub_workstream_id: "522608cb-81a2-491f-b6b3-eff00fe01269" +created: "2026-05-17" +updated: "2026-05-17" +--- + +# CUST-WP-0042 - Workplan State Model Cleanup + +## Goal + +Make the State Hub workplan lifecycle explicit and consistent across +file-backed workplans, the workstream API, consistency scripts, validation, +dashboard filters, and documentation. + +This workplan exists because `proposed`, `ready`, and `backlog` represent real +planning states that the current model cannot express cleanly: + +- `proposed`: plan exists, but must be reviewed against current repo state +- `ready`: reviewed and executable against current repo state +- `backlog`: intentionally parked so it does not clutter current work views + +Supporting design note: +`state-hub/docs/workplan-state-model-proposal.md` + +## Context + +State Hub currently has vocabulary drift: + +- Workstream API/schema: `todo`, `active`, `blocked`, `completed`, `archived` +- Workplan consistency scripts: `active`, `completed`, `archived` +- Dashboard overview modes: `active`, `accepted`, `finished`, `blocked`, + `stalled`, `oldies` + +The intended cleanup is to separate stored lifecycle states from derived health +labels. `stalled`, `oldies`, and stale-ready detection should not be stored as +primary workplan states. + +## Target State Model + +Canonical stored workplan states: + +```text +proposed, ready, active, blocked, backlog, finished, archived +``` + +Derived health labels: + +```text +needs_review, stalled +``` + +Normal progression: + +```text +backlog -> proposed -> ready -> active -> finished -> archived + \ \ + \ -> blocked -> active + -> backlog +``` + +## Tasks + +### T01 - Capture State Model Proposal + +```task +id: CUST-WP-0042-T01 +status: done +priority: high +state_hub_task_id: "fbc17b50-7b7e-41ba-bd13-d66a6a546450" +``` + +Write the design proposal that defines canonical states, derived labels, legacy +aliases, UI cleanup, and migration expectations. + +Done when `state-hub/docs/workplan-state-model-proposal.md` exists and can be +reviewed as the design source for this workplan. + +### T02 - Centralize Workplan Status Constants + +```task +id: CUST-WP-0042-T02 +status: todo +priority: high +state_hub_task_id: "9be18f7e-1db1-464b-8c65-bf64ae3462e8" +``` + +Add a shared source of truth for workplan/workstream statuses and legacy aliases +used by API schemas, consistency scripts, validation scripts, and dashboard +classification code. + +Done when supported statuses and aliases are no longer duplicated as local +string sets in multiple files. + +### T03 - Update API Schema And Database Migration + +```task +id: CUST-WP-0042-T03 +status: todo +priority: high +state_hub_task_id: "1d9964ce-7b30-49d3-a4e3-6d5b3ef8d684" +``` + +Update the workstream schema to accept the canonical state set and add a +migration for existing values: + +- `completed` -> `finished` +- `accepted` -> `finished` +- `todo` -> `ready` when no work has started, otherwise `active` + +Done when API validation accepts the canonical states and existing DB rows are +normalized without data loss. + +### T04 - Update Consistency And ADR Validation + +```task +id: CUST-WP-0042-T04 +status: todo +priority: high +state_hub_task_id: "c80df776-5d45-419d-a92c-59e3b77d9798" +``` + +Update `scripts/consistency_check.py` and `scripts/validate_repo_adr.py` so +file-backed workplans can use `proposed`, `ready`, `active`, `blocked`, +`backlog`, `finished`, or `archived` without false warnings. + +Done when consistency checks compare normalized values and write back canonical +values only when they intentionally edit files. + +### T05 - Add Ready-Staleness Detection + +```task +id: CUST-WP-0042-T05 +status: todo +priority: medium +state_hub_task_id: "97ebc285-3998-4f17-bf43-c4f803cf1e7b" +``` + +Support optional ready-review metadata: + +```yaml +reviewed_at: "YYYY-MM-DD" +reviewed_by: "human-or-agent" +reviewed_against_commit: "" +context_paths: + - "path/or/glob" +``` + +Done when `ready` workplans can be flagged as `needs_review` after relevant repo +changes, with any automatic demotion from `ready` to `proposed` guarded behind +an explicit flag. + +### T06 - Clean Up Dashboard Status Filters + +```task +id: CUST-WP-0042-T06 +status: todo +priority: high +state_hub_task_id: "210fcff3-e3e9-4f8a-9687-c5de83ace465" +``` + +Replace dashboard lifecycle filters with canonical stored states and move +`stalled`/`needs_review` into derived health filters or badges. + +Done when the overview no longer uses `accepted`, `oldies`, or derived +`finished` as primary lifecycle filters. + +### T07 - Update Docs And Registration Templates + +```task +id: CUST-WP-0042-T07 +status: todo +priority: medium +state_hub_task_id: "8ecceebb-0471-4541-96f9-c9f98df12f84" +``` + +Update dashboard docs, State Hub docs, project registration templates, and +agent guidance so new repos use the canonical workplan frontmatter states. + +Done when new generated repo guidance references the same state model as the +API and consistency engine. + +### T08 - Add Regression Tests + +```task +id: CUST-WP-0042-T08 +status: todo +priority: medium +state_hub_task_id: "7dd1f27d-4a4a-4c78-8b91-6103688559a9" +``` + +Add tests for API status validation, file status normalization, legacy alias +handling, ready-staleness detection, and dashboard classification helpers. + +Done when the new state model is covered well enough that vocabulary drift is +hard to reintroduce accidentally. + +## Acceptance Criteria + +- Workplan files and API workstreams support the same canonical state set. +- `proposed`, `ready`, and `backlog` are first-class lifecycle states. +- `stalled` and `needs_review` are derived health labels, not stored lifecycle + states. +- Legacy status values are normalized or migrated safely. +- Dashboard filters and docs use the same vocabulary as the file/API model. diff --git a/workplans/CUST-WP-0043-state-hub-repo-extraction.md b/workplans/CUST-WP-0043-state-hub-repo-extraction.md new file mode 100644 index 0000000..5bee26f --- /dev/null +++ b/workplans/CUST-WP-0043-state-hub-repo-extraction.md @@ -0,0 +1,316 @@ +--- +id: CUST-WP-0043 +type: workplan +title: "State Hub Repo Extraction" +domain: custodian +repo: the-custodian +status: active +owner: custodian +topic_slug: custodian +planning_priority: high +planning_order: 43 +created: "2026-05-17" +updated: "2026-05-17" +state_hub_workstream_id: "01fc91e6-7e4a-4eda-8322-e40a0c6c7cf7" +--- + +# CUST-WP-0043 - State Hub Repo Extraction + +## Goal + +Untangle `state-hub/` from `the-custodian` and make `/home/worsch/state-hub` +the standalone source repository for the State Hub service before continuing +State Hub implementation work such as `CUST-WP-0042`. + +The intended end state is: + +- `the-custodian` owns canon, governance, memory, agent runtime scaffolding, and + cross-domain coordination documents. +- `state-hub` owns the live service: FastAPI app, MCP server, migrations, + dashboard, scripts, policies, task-flow engine, tests, and operational docs. +- State Hub itself knows about the new `state-hub` repo path, so future + workplans can live with the service they modify. +- The recently generated workplan `CUST-WP-0042 - Workplan State Model Cleanup` + can be executed against the standalone `state-hub` repository instead of the + embedded directory. + +## Context + +`/home/worsch/state-hub` currently exists only as a repo-seed stub with a README, +license, `.gitignore`, and Git metadata. The live implementation still resides +under `the-custodian/state-hub/`. + +This creates three forms of coupling: + +- Source coupling: implementation code is versioned inside the governance repo. +- Operational coupling: Make targets, MCP registration, dashboard paths, and + consistency sync assume `state-hub/` is a subdirectory of `the-custodian`. +- Planning coupling: State Hub workplans, including `CUST-WP-0042`, are + currently file-backed from `the-custodian/workplans/` even when they primarily + modify State Hub internals. + +This workplan is deliberately narrower than the broader FOS hub extraction in +`CUST-WP-0025`: it extracts the existing State Hub into its own repo first. The +generic `hub-core` extraction and any future `dev-hub` rename remain separate +architecture work. + +## Scope + +In scope: + +- Move the existing State Hub implementation into `/home/worsch/state-hub`. +- Preserve useful Git history where practical. +- Exclude generated or environment-local directories such as `.venv/`, + `dashboard/node_modules/`, `dashboard/dist/`, `__pycache__/`, and + `.pytest_cache/`. +- Register or update the `state-hub` repo in State Hub with the local path. +- Update session guidance, runbooks, Make targets, and MCP/API path references + so agents and humans know where State Hub work now lives. +- Re-home State Hub workplans so future implementation work is file-backed from + the standalone repo. + +Out of scope: + +- Extracting the generic `hub-core` package. +- Renaming State Hub to Dev Hub. +- Changing the State Hub lifecycle/status model itself; that belongs to + `CUST-WP-0042`. +- Changing production deployment topology beyond path and repo ownership. + +## Tasks + +### T01 - Inventory Extraction Boundary + +```task +id: CUST-WP-0043-T01 +status: todo +priority: high +state_hub_task_id: "1d1e1cd6-96ac-4840-9c11-2867ada1116d" +``` + +Create an inventory of everything under `the-custodian/state-hub/` and classify +each path as: + +- move to `state-hub` +- keep in `the-custodian` +- regenerate locally +- discard as generated/cache output + +Done when the inventory names the handling for API, MCP server, migrations, +dashboard, infra, scripts, tests, docs, policies, prompts, and task-flow engine. + +### T02 - Prepare Standalone Repo Baseline + +```task +id: CUST-WP-0043-T02 +status: todo +priority: high +state_hub_task_id: "24ad0bf0-e11d-4a16-9c00-0f9cca6d208f" +``` + +Turn `/home/worsch/state-hub` from repo-seed stub into a proper service repo +baseline before moving code: + +- replace the seed README with State Hub-specific orientation +- add `AGENTS.md`/session protocol for State Hub work +- add `SCOPE.md` +- decide the repo-local workplan prefix for future State Hub plans +- ensure `.gitignore` excludes local environments, dependency installs, build + output, caches, and secrets + +Done when a fresh agent can open `/home/worsch/state-hub` and understand what +the repo owns before any implementation code is copied. + +### T03 - Move Implementation With History-Aware Strategy + +```task +id: CUST-WP-0043-T03 +status: todo +priority: high +state_hub_task_id: "fe1faa7b-fa19-4d08-b12f-b67bbe5d6774" +``` + +Move the implementation from `the-custodian/state-hub/` into the standalone +repo using a strategy that preserves useful history where possible. Candidate +approaches: + +- `git subtree split` from `the-custodian/state-hub/` into the new repo +- `git filter-repo` into a temporary extraction branch, then merge into the stub +- a clean copy only if history preservation is not worth the migration risk + +Do not migrate generated or local-only content: + +- `.venv/` +- `dashboard/node_modules/` +- `dashboard/dist/` +- `__pycache__/` +- `.pytest_cache/` +- local `.env` files or credentials + +Done when `/home/worsch/state-hub` contains the service implementation and has a +clean Git status other than intentional new extraction commits. + +### T04 - Decouple The Custodian Repository + +```task +id: CUST-WP-0043-T04 +status: todo +priority: high +state_hub_task_id: "92c6ee06-c256-4526-9bc4-6f2b1b0f5d8e" +``` + +Remove `the-custodian/state-hub/` as an owned implementation tree after the +standalone repo is verified. Leave only intentional integration surface in +`the-custodian`, such as: + +- documentation pointing to `/home/worsch/state-hub` +- optional proxy Make targets if they remain useful +- session guidance that says State Hub code work belongs in the standalone repo + +Done when there is a single authoritative implementation tree and +`the-custodian` no longer looks like it owns State Hub source code. + +### T05 - Register State Hub Repo In State Hub + +```task +id: CUST-WP-0043-T05 +status: todo +priority: high +state_hub_task_id: "5dd3547c-69b3-4760-83a7-f0783053a572" +``` + +Connect `/home/worsch/state-hub` to State Hub as a managed repo in the +`custodian` domain. + +Use the file-first rule for repo-backed workplans. If new workplan files are +created in the standalone repo, write them before running consistency sync or +registration tooling so no ghost workstreams are created. + +Done when State Hub repo metadata resolves the local host path for +`/home/worsch/state-hub`, and `fix-consistency --here` or the equivalent repo +sync can discover the repo without path overrides. + +### T06 - Re-home State Hub Workplans + +```task +id: CUST-WP-0043-T06 +status: todo +priority: high +state_hub_task_id: "1c537901-1013-43e0-81c6-9c29ed4f7dd2" +``` + +Move State Hub implementation workplans to the standalone repo once it is +registered. The first priority is the recently generated +`CUST-WP-0042 - Workplan State Model Cleanup`. + +The migration should preserve existing State Hub IDs where appropriate and avoid +creating duplicate workstreams. If the consistency engine cannot safely move an +existing file-backed workstream across repos, record the limitation and use the +least confusing transition path: + +- keep the old file as an archived pointer, or +- create a new State Hub-local continuation plan that references the old + workstream explicitly + +Done when future agents can work on the State Hub state-model cleanup from the +standalone repo without editing `the-custodian/state-hub/`. + +### T07 - Update Operational Wiring + +```task +id: CUST-WP-0043-T07 +status: todo +priority: high +state_hub_task_id: "626faa5e-314b-445b-b86b-f11014473ad5" +``` + +Update path-sensitive wiring after extraction: + +- Make targets and scripts that run API, MCP, migrations, tests, dashboard, or + consistency sync +- MCP registration examples and local config patchers +- dashboard/data path references +- docs and runbooks that mention `the-custodian/state-hub` +- deployment or container build contexts +- NATS/activity-core delegation docs if they name repo-local paths + +Done when local development and agent session startup use the standalone repo +path and no longer assume the embedded path. + +### T08 - Verification Gate + +```task +id: CUST-WP-0043-T08 +status: todo +priority: high +state_hub_task_id: "4571d9fd-0513-406e-ad4e-284e39f16ac5" +``` + +Run verification from the standalone repo: + +- Python test suite +- Alembic migration check +- API health check +- MCP server smoke test +- dashboard smoke test if frontend dependencies are available +- consistency sync for both affected repos + +Done when the standalone repo can run the same service checks that previously +ran from `the-custodian/state-hub/`, and the Custodian dashboard shows the new +repo/workplan state accurately. + +### T09 - Handoff To State Hub State-Model Work + +```task +id: CUST-WP-0043-T09 +status: todo +priority: medium +state_hub_task_id: "72b737f8-6caa-4a36-9113-f05c4fab2738" +``` + +After extraction is verified, resume `CUST-WP-0042` from the standalone +`state-hub` repo. Before implementation starts: + +- confirm the workplan paths point at the new repo layout +- confirm `state_hub_workstream_id` and task IDs still refer to the intended + workstream/tasks +- update any acceptance criteria that assumed the embedded path +- record a progress event linking this extraction workplan to the resumed + state-model cleanup work + +Done when `CUST-WP-0042` is ready to execute in the extracted repository. + +## Build Order + +```text +T01 -> T02 -> T03 -> T08 + T03 -> T04 -> T07 -> T08 +T02 -> T05 -> T06 -> T09 +T08 -> T09 +``` + +`CUST-WP-0042` should not receive implementation edits until T06 and T08 are +complete, because its changes should land in the standalone `state-hub` repo. + +## Acceptance Criteria + +- `/home/worsch/state-hub` is the authoritative State Hub implementation repo. +- `the-custodian` no longer owns a live copy of State Hub implementation code. +- State Hub repo metadata knows the standalone repo path on this host. +- New State Hub workplans can be synced from the standalone repo without ghost + workstreams. +- Existing active State Hub work, especially `CUST-WP-0042`, has a clear and + non-duplicating handoff path into the standalone repo. +- Local API, MCP, migration, dashboard, test, and consistency workflows are + verified after extraction. + +## Risks And Guardrails + +- Do not delete the embedded implementation until the standalone repo passes the + verification gate. +- Do not copy secrets, local virtualenvs, dependency installs, or generated + frontend output into the new repo. +- Do not call `create_workstream()` or `create_task()` manually for migrated + file-backed workplans before the files exist in their final repo. +- Prefer a reversible transition: keep an archived pointer or temporary bridge + doc in `the-custodian` until State Hub work has clearly moved.