- scripts/cleanup_stale_tasks.py: daily script that cancels open tasks in completed/archived workstreams; handles 307 redirects; emits a cleanup progress event summarising results - Makefile: add cleanup-stale target (also suitable for cron) - ADR-001: append Workstream Closure Protocol section — mandatory closure review before marking workstream completed, with task classification table (done/cancelled/carry-forward) and Closure Review file format - WP-0002 + WP-0005: append Closure Review sections documenting the 2026-03-02 cleanup run (26 stale DB rows cancelled — all were legacy pre-ADR-001 DB-first records; file status was already done) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
255 lines
8.8 KiB
Markdown
255 lines
8.8 KiB
Markdown
---
|
||
id: ADR-001
|
||
type: architecture-decision-record
|
||
title: "Workplans and Work Items Are Repository Artefacts"
|
||
status: accepted
|
||
decided_by: Bernd Worsch
|
||
date: "2026-02-28"
|
||
tags: ["architecture", "state-hub", "workplans", "source-of-truth", "rebuild-principle"]
|
||
---
|
||
|
||
# ADR-001: Workplans and Work Items Are Repository Artefacts
|
||
|
||
## Status
|
||
|
||
Accepted.
|
||
|
||
## Context
|
||
|
||
During early State Hub development (v0.1–v0.4), workstreams and tasks were
|
||
created directly in the PostgreSQL database via MCP bootstrap tools
|
||
(`create_workstream`, `create_task`). This made the database the **origin** of
|
||
work items — not a cache or index. The pattern was convenient for rapid
|
||
bootstrapping but is architecturally wrong for a system built on the values of
|
||
auditability, reversibility, and local-first sovereignty.
|
||
|
||
The trigger for formalising this decision was the creation of the v0.5
|
||
workplan ("Dynamic Domains & Multi-Repo") directly in the state-hub database
|
||
without a corresponding file artefact in any repository.
|
||
|
||
## Decision
|
||
|
||
**Workplans and work items MUST originate as Markdown files in the repository
|
||
that owns them.** The Custodian State Hub indexes and caches those artefacts
|
||
but is never their origin.
|
||
|
||
Formally: the state-hub must (theoretically, given sufficient compute and
|
||
time) be able to **rebuild its full representation** of repositories, their
|
||
workplans, tasks, decisions, and dependencies by reading only the files in the
|
||
registered repositories. No information that matters for coordination should
|
||
exist solely in the database.
|
||
|
||
### Corollaries
|
||
|
||
1. **Repository is authoritative.** A workplan file is the canonical record.
|
||
The state-hub database row is a materialized cache of that file.
|
||
|
||
2. **Database is disposable.** Dropping and re-creating the database from
|
||
registered repository files must produce an equivalent state. The database
|
||
is an operational convenience, not a primary store.
|
||
|
||
3. **MCP bootstrap tools become index/sync tools.** `create_workstream` and
|
||
`create_task` are acceptable as convenience wrappers only if they write the
|
||
file first and then register the row. Using them to write DB-only records
|
||
violates this ADR.
|
||
|
||
4. **The rebuild principle implies a sync mechanism.** There must be a defined
|
||
path (`make sync-workplans` or equivalent) by which the state-hub reads
|
||
workplan files from registered repositories and upserts its database state.
|
||
|
||
## Workplan File Convention
|
||
|
||
Each workplan lives in a `workplans/` directory in the repository that owns
|
||
the work. The owning repository is identified by domain.
|
||
|
||
### Location
|
||
|
||
```
|
||
<repo-root>/workplans/<id>-<slug>.md
|
||
```
|
||
|
||
Examples:
|
||
- `the-custodian/workplans/CUST-WP-0005-dynamic-domains.md`
|
||
- `railiance/workplans/RAIL-WP-0001-three-phoenix.md`
|
||
|
||
### Frontmatter Schema
|
||
|
||
```yaml
|
||
---
|
||
id: CUST-WP-0005 # human-readable workplan ID, unique per repo
|
||
type: workplan
|
||
title: "State Hub v0.5 — Dynamic Domains & Multi-Repo"
|
||
domain: custodian # must match a registered domain slug
|
||
status: active # active | completed | archived
|
||
owner: custodian
|
||
topic_slug: custodian # maps to a state-hub Topic slug
|
||
created: "2026-02-28"
|
||
updated: "2026-02-28"
|
||
---
|
||
```
|
||
|
||
### Task Items
|
||
|
||
Tasks are embedded in the workplan file as headed sections. Each task
|
||
section carries its own YAML block:
|
||
|
||
```markdown
|
||
## P1.1 — Create `domains` table + Alembic migration
|
||
|
||
```task
|
||
id: CUST-WP-0005-T001
|
||
status: todo
|
||
priority: high
|
||
```
|
||
|
||
Task description prose here.
|
||
```
|
||
|
||
The state-hub parses these embedded task blocks during ingestion and upserts
|
||
rows in the `tasks` table. The `id` field is the stable external key; the
|
||
state-hub UUID is internal and opaque.
|
||
|
||
### Decision Items
|
||
|
||
Decisions are separate files or embedded sections following the same pattern,
|
||
using `type: decision` in frontmatter.
|
||
|
||
## Rebuild Principle
|
||
|
||
The rebuild sequence for a clean state-hub:
|
||
|
||
1. `make migrate` — create schema
|
||
2. `make seed-domains` — insert domain rows (domains.yaml in canon/)
|
||
3. For each registered repository: `make sync-workplans REPO=<slug>` — parse
|
||
workplan files and upsert workstreams, tasks, decisions
|
||
4. `make sync-progress` — replay progress events from episodic memory logs
|
||
|
||
After step 4 the database must be functionally equivalent to the live state.
|
||
|
||
## Consequences
|
||
|
||
### Immediate
|
||
|
||
- The v0.5 and v0.3 workplans created DB-first in this session are **legacy
|
||
records** that violate this ADR. Remediation: write the corresponding
|
||
workplan files, then mark the DB rows as `source: db-legacy` until a sync
|
||
mechanism can reconcile them.
|
||
|
||
- The state-hub CLAUDE.md design-boundary note must be updated: the MCP
|
||
bootstrap tools are permitted only as write-through tools (file + DB), never
|
||
as DB-only tools.
|
||
|
||
### Medium Term
|
||
|
||
- A `make sync-workplans` command must be implemented as part of the
|
||
managed-repos / contribution-tracking infrastructure (see v0.3 workplan).
|
||
|
||
- The `managed_repos` table is the prerequisite: the state-hub must know which
|
||
repositories to scan.
|
||
|
||
- Workplan file format must be versioned and parsed by a dedicated loader
|
||
(`state-hub/scripts/sync_workplans.py`).
|
||
|
||
### Long Term
|
||
|
||
- When the state-hub grows to cover multiple users or teams, this principle
|
||
ensures that no coordination state can be lost by a database failure.
|
||
Every repository is its own resilient shard of the coordination graph.
|
||
|
||
- This is the foundation for the "transgenerational" property: workplans in
|
||
git survive database migrations, cloud provider changes, and system
|
||
rebuilds.
|
||
|
||
## Alternatives Considered
|
||
|
||
**Database-first with export:** Create in DB, export to files on demand.
|
||
Rejected: export is easily skipped and files become secondary/stale.
|
||
|
||
**Files-only, no database:** Parse files on every query.
|
||
Rejected: impractical at scale; the database is a necessary cache for
|
||
cross-repo aggregation and real-time dashboard queries.
|
||
|
||
**Hybrid with explicit sync flag:** Mark some records as "db-authoritative"
|
||
and others as "file-authoritative."
|
||
Rejected: introduces ambiguity about which records matter; violates the
|
||
"single source of truth" principle.
|
||
|
||
## Workstream Closure Protocol
|
||
|
||
When a workstream is about to be marked `completed`, the responsible agent
|
||
MUST perform a closure review before writing the status change. This prevents
|
||
the stale-task accumulation that this ADR was designed to make detectable.
|
||
|
||
### Steps
|
||
|
||
1. **Query all non-done tasks** in the workstream via
|
||
`GET /tasks/?workstream_id=<uuid>` (filter for `todo`, `in_progress`,
|
||
`blocked`).
|
||
|
||
2. **Classify each task** into one of three outcomes:
|
||
|
||
| Outcome | Action |
|
||
|---------|--------|
|
||
| **Done** — work was completed, DB record just wasn't updated | `PATCH /tasks/{id}/ {"status": "done"}` |
|
||
| **Cancelled** — dropped, superseded, or out of scope | `PATCH /tasks/{id}/ {"status": "cancelled", "blocking_reason": "<why>"}` |
|
||
| **Carry-forward** — genuinely unfinished, belongs in the next run | Leave open; note in closure review; trigger new workplan |
|
||
|
||
3. **Append a `## Closure Review` section** to the workplan file:
|
||
|
||
```markdown
|
||
## Closure Review — YYYY-MM-DD
|
||
|
||
**Outcome:** All tasks completed / N tasks carried forward / N tasks dropped.
|
||
|
||
### Completed (DB updated)
|
||
- TASK-ID — title
|
||
|
||
### Cancelled (dropped)
|
||
| Task | Reason |
|
||
|------|--------|
|
||
| TASK-ID — title | Superseded by X |
|
||
|
||
### Carried forward
|
||
| Task | Target workplan |
|
||
|------|----------------|
|
||
| TASK-ID — title | CUST-WP-XXXX |
|
||
```
|
||
|
||
4. **If any tasks are carried forward**: do not mark the workstream
|
||
`completed` yet. Create the new workplan file (or amend an existing active
|
||
one), then close the current workstream.
|
||
|
||
5. **Update the workplan frontmatter** `status: completed` and `updated:` date.
|
||
|
||
6. **Mark the workstream `completed`** in the state hub via MCP or API.
|
||
|
||
### Daily Stale-Task Cleanup
|
||
|
||
As a safety net for cases where the closure review was skipped or incomplete,
|
||
a cleanup script cancels any surviving open tasks in completed/archived
|
||
workstreams:
|
||
|
||
```bash
|
||
cd ~/the-custodian/state-hub
|
||
make cleanup-stale # run immediately
|
||
# or add to cron:
|
||
# 0 3 * * * cd ~/the-custodian/state-hub && make cleanup-stale
|
||
```
|
||
|
||
The script (`scripts/cleanup_stale_tasks.py`) emits a `cleanup` progress event
|
||
recording which tasks were cancelled and in which workstreams. Tasks cancelled
|
||
by the cleanup carry a `blocking_reason` noting they should be verified against
|
||
the workplan file.
|
||
|
||
The closure review is the primary mechanism; the cleanup is the fallback. If
|
||
the cleanup regularly cancels tasks, it signals that closure reviews are being
|
||
skipped — that is the process failure to address, not just the stale tasks.
|
||
|
||
## Related
|
||
|
||
- Custodian Constitution v0.1 §2 (Powers) — canon changes require review gate
|
||
- ADR-000 (forthcoming) — overall Custodian architecture principles
|
||
- State Hub v0.3 workplan — `sync_workplans.py` is a Phase 4 deliverable
|
||
- `canon/values/foundational_values_v0.1.md` — Local-first, Auditability,
|
||
Reversibility
|