chore: add workplans WP-0004, WP-0005, WP-0006

WP-0004: Stable Documentation Corpus & Architecture Records WP-0005: Diagram Renderer Integration WP-0006: Packaging & Distribution State-hub workstreams and tasks created; local_path registration deferred pending path mismatch fix. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-16 12:55:34 +00:00
parent 531af57f4d
commit 039420caee
3 changed files with 546 additions and 0 deletions
--- a/workplans/MRKD-WP-0004-stable-corpus-architecture.md
+++ b/workplans/MRKD-WP-0004-stable-corpus-architecture.md
@@ -0,0 +1,190 @@
+---
+id: MRKD-WP-0004
+type: workplan
+domain: markitect
+repo: marki-docx
+status: active
+state_hub_workstream_id: 91d06c92-caa8-42fc-b6d4-82340f1bed4f
+created: 2026-03-16
+updated: 2026-03-16
+---
+
+# MRKD-WP-0004 — Stable Documentation Corpus & Architecture Records
+
+Fulfil FR-1101 by establishing the markidocx product documentation itself as a real,
+managed markidocx project. The specs (PRD, FRS, UCC) become a live round-trip corpus
+that the `release-regression` workflow runs against on every release. This workstream
+also writes the two deferred architecture decision records and generates the first SBOM.
+
+**Scope:** FR-1101–1110 (stable corpus & self-test), ADR-002, ADR-003, SBOM
+**Out of scope:** diagram rendering, packaging, CI/CD — addressed in WP-0005/0006
+**Depends on:** MRKD-WP-0001, MRKD-WP-0002, MRKD-WP-0003 — all complete
+
+---
+
+## T01 — Set up specs as real markidocx project manifest
+
+```task
+id: MRKD-WP-0004-T01
+status: todo
+priority: high
+state_hub_task_id: f1a36613-ceaa-4786-ac39-cd3a7fd1c142
+```
+
+Create a manifest file that treats the markidocx product documentation as a live
+markidocx project. This makes the specs the stable corpus for regression testing
+as required by FR-1101.
+
+- Create `corpus/markidocx-docs/manifest.yaml`:
+  - `project.name: markidocx-docs`
+  - `project.feature_level: level1`
+  - `project.family: article`
+  - `sources`: PRD, FRS v0.2, UCC (relative paths into `specs/`)
+  - `output.dir: corpus/markidocx-docs/dist`
+- Run `markidocx validate corpus/markidocx-docs/manifest.yaml` — must exit 0
+- Run `markidocx build corpus/markidocx-docs/manifest.yaml` — must produce valid DOCX
+- Run `markidocx import` + `markidocx compare` — must report clean or expected drift only
+- Document any structural drift in `corpus/markidocx-docs/known-drift.md`
+
+Deliverable: `markidocx build corpus/markidocx-docs/manifest.yaml` succeeds; a DOCX
+of the product documentation exists in `corpus/markidocx-docs/dist/`.
+
+---
+
+## T02 — Wire release-regression workflow against specs corpus
+
+```task
+id: MRKD-WP-0004-T02
+status: todo
+priority: high
+state_hub_task_id: f17e959f-28da-4386-9004-b5e036054b06
+```
+
+Connect the `release-regression` composite workflow to the real documentation corpus
+so that `markidocx workflow release-regression corpus/markidocx-docs/manifest.yaml`
+runs a full build → import → compare cycle and records evidence (FR-1102, FR-1103,
+FR-1106, FR-1107).
+
+- Update `workflows.py` `release-regression` handler to accept a manifest path argument;
+  default to the corpus manifest when none supplied
+- Run the workflow; assert the evidence set contains build, import, and drift reports
+- Add `tests/regression/test_corpus_regression.py`:
+  - Invokes `release-regression` on the corpus manifest
+  - Asserts workflow result is `full` or `with-fallback` (not `failed`)
+  - Asserts evidence artefacts are present and have correct traceability fields (FR-1110)
+- Disclose corpus identity in regression output (FR-1109): include corpus manifest path
+  and its git HEAD SHA as `corpus_id` in the workflow result
+
+Deliverable: `pytest tests/regression/test_corpus_regression.py` passes; evidence
+written to `.markidocx/evidence/` and retrievable via CLI.
+
+---
+
+## T03 — ADR-002: python-docx as conversion engine
+
+```task
+id: MRKD-WP-0004-T03
+status: todo
+priority: medium
+state_hub_task_id: bfe2a9fa-25b2-4b4b-b21b-eae457716ce0
+```
+
+Write the architecture decision record explaining the choice of python-docx as the
+DOCX conversion engine. This was identified as a deferred deliverable during WP-0001.
+
+File: `architecture/ADR-002-python-docx-as-conversion-engine.md`
+
+Cover:
+- **Context:** need to produce and consume .docx files from Python; alternatives evaluated
+  (pandoc subprocess, docx2python, mammoth, python-docx)
+- **Decision:** python-docx for both build (write) and import (read)
+- **Consequences:** direct paragraph/run model maps cleanly to Markdown structure;
+  no subprocess dependency; limited to Open XML subset exposed by python-docx API;
+  complex Word features (track changes, SmartArt) are out of scope by design
+- **Alternatives rejected:** pandoc — heavier dependency, harder to control structure;
+  mammoth — read-only; docx2python — limited write support
+
+Deliverable: `architecture/ADR-002-*.md` present and follows ADR-001 conventions.
+
+---
+
+## T04 — ADR-003: manifest YAML schema
+
+```task
+id: MRKD-WP-0004-T04
+status: todo
+priority: medium
+state_hub_task_id: b6de6733-b332-4efc-9e23-82fce205b856
+```
+
+Write the architecture decision record documenting the manifest YAML schema design.
+
+File: `architecture/ADR-003-manifest-yaml-schema.md`
+
+Cover:
+- **Context:** need a project definition format that is human-writable, version-controlled,
+  and parseable without a schema registry
+- **Decision:** YAML with a fixed top-level structure (`project`, `sources`, `output`,
+  `metadata`); validated on load via dataclass coercion
+- **Schema snapshot:** include the current field definitions as a reference
+- **Consequences:** simple for users; no JSON Schema or Pydantic dependency; evolving
+  the schema requires coordination with manifest.py
+- **Alternatives rejected:** TOML (less familiar in doc tooling), JSON (less writable),
+  a database manifest (over-engineered for single-project use)
+
+Deliverable: `architecture/ADR-003-*.md` present.
+
+---
+
+## T05 — SBOM generation and state-hub registration
+
+```task
+id: MRKD-WP-0004-T05
+status: todo
+priority: medium
+state_hub_task_id: 36aecd50-8176-4122-9706-a8697d8f5936
+```
+
+Generate and register the first SBOM for marki-docx so the state hub has an accurate
+dependency picture.
+
+```bash
+cd ~/the-custodian/state-hub
+make ingest-sbom REPO=marki-docx SCAN=1 REPO_PATH=/home/tegwick/marki-docx
+```
+
+- Verify the SBOM ingestion completes without errors
+- Confirm `last_sbom_at` is set for `marki-docx` in the state hub
+- Document any licence issues or unexpected transitive dependencies
+- Add a note to CLAUDE.md reminding to re-run SBOM after dependency changes
+
+Deliverable: State hub shows `last_sbom_at` set for `marki-docx`; no unresolved
+licence issues.
+
+---
+
+## How to Work
+
+- Work through tasks in priority order: T01 → T02 (high), then T03 → T04 → T05 (medium)
+- T01 must complete before T02 (T02 depends on the corpus manifest)
+- T03 and T04 are independent writing tasks — can be done in any order or in parallel
+
+## Updating Task Status
+
+```
+status: todo        →  status: in_progress   (when you start it)
+status: in_progress →  status: done          (when verified complete)
+```
+
+When every task is `done`, set the frontmatter `status: done`.
+
+## Success Criteria
+
+Before marking the workplan done:
+
+1. Every task block has `status: done`
+2. Workplan frontmatter `status: done`
+3. `corpus/markidocx-docs/manifest.yaml` present and builds cleanly
+4. `pytest tests/regression/test_corpus_regression.py` passes
+5. `architecture/ADR-002-*.md` and `architecture/ADR-003-*.md` present
+6. State hub shows `last_sbom_at` set for `marki-docx`
--- a/workplans/MRKD-WP-0005-diagram-renderer-integration.md
+++ b/workplans/MRKD-WP-0005-diagram-renderer-integration.md
@@ -0,0 +1,185 @@
+---
+id: MRKD-WP-0005
+type: workplan
+domain: markitect
+repo: marki-docx
+status: active
+state_hub_workstream_id: 2ef47f11-d828-436d-8955-c58e13c50752
+created: 2026-03-16
+updated: 2026-03-16
+---
+
+# MRKD-WP-0005 — Diagram Renderer Integration
+
+Complete the deferred rendering path for FR-533/534. Currently `diagrams.py` operates
+in source-only mode: diagram fenced blocks are preserved as source through the round-trip
+but are never rendered to images. This workstream adds actual rendering support for
+Mermaid, Graphviz, and PlantUML when the corresponding CLI tools are available, while
+preserving the graceful source-only fallback when they are not.
+
+**Scope:** FR-533 (auto-diagram support), FR-534 (diagram intent preservation),
+FR-538 (processor-dependency disclosure)
+**Out of scope:** new diagram syntaxes beyond mermaid/graphviz/plantuml
+**Depends on:** MRKD-WP-0003 (diagrams.py source-only foundation) — complete
+
+---
+
+## T01 — Renderer abstraction layer and detection
+
+```task
+id: MRKD-WP-0005-T01
+status: todo
+priority: high
+state_hub_task_id: c4911ecc-1e3c-4d22-a6fb-92d1ed319274
+```
+
+Introduce a `RendererBackend` abstraction in `diagrams.py` so that each diagram type
+can be rendered by a pluggable backend. The builder calls the abstraction; concrete
+backends handle tool detection and subprocess invocation.
+
+- `DiagramRenderer` protocol: `can_render(diagram_type: str) -> bool`,
+  `render(source: str, diagram_type: str, output_path: Path) -> RendererResult`
+- `RendererResult`: `success: bool`, `output_path: Path | None`, `warning: WarningRecord | None`
+- `detect_renderers() -> dict[str, DiagramRenderer]` — probe PATH for `mmdc`, `dot`,
+  `plantuml`; return only those found
+- Builder updated to call `detect_renderers()` at build time; if a renderer is found
+  for the diagram type → render to PNG and embed; if not → source-only fallback +
+  `WarningRecord(reason="renderer-unavailable", construct=diagram_type)` (FR-538)
+- Unit tests: `test_renderer_detection` — mock PATH; `test_fallback_when_no_renderer`
+
+Deliverable: `pytest tests/test_level3_diagrams.py` still passes; renderer abstraction
+in place and tested.
+
+---
+
+## T02 — Mermaid CLI (mmdc) integration
+
+```task
+id: MRKD-WP-0005-T02
+status: todo
+priority: high
+state_hub_task_id: 87caa295-f466-4e2e-ba06-4b1a801ca976
+```
+
+Implement the `MermaidRenderer` backend that shells out to `mmdc` (Mermaid CLI).
+
+- `MermaidRenderer.can_render("mermaid") -> bool`: checks `shutil.which("mmdc")`
+- `MermaidRenderer.render(source, "mermaid", output_path)`:
+  - Write source to a temp `.mmd` file
+  - Run `mmdc -i <input> -o <output>.png`
+  - On success: return `RendererResult(success=True, output_path=...)`
+  - On failure: return `RendererResult(success=False, warning=WarningRecord(reason="render-failed"))`
+- Builder: PNG embedded into DOCX as image; alt-text carries `[mermaid-source]<source>` marker
+  for round-trip (FR-534)
+- Importer: existing alt-text marker detection already handles this — verify round-trip works
+- Add `mmdc` to `pyproject.toml` optional extras: `diagram-mermaid`
+- Integration test (skipped if `mmdc` not found): `test_mermaid_render_roundtrip`
+
+Deliverable: When `mmdc` is on PATH, mermaid diagrams render to PNG and embed in DOCX;
+round-trip restores source. When absent, source-only fallback with warning.
+
+---
+
+## T03 — Graphviz (dot) integration
+
+```task
+id: MRKD-WP-0005-T03
+status: todo
+priority: medium
+state_hub_task_id: 8ceb771d-0b16-452d-b1cc-c5c2c40fe723
+```
+
+Implement the `GraphvizRenderer` backend that shells out to `dot`.
+
+- `GraphvizRenderer.can_render("graphviz") -> bool`: checks `shutil.which("dot")`
+- `GraphvizRenderer.render(source, "graphviz", output_path)`:
+  - Write source to a temp `.dot` file
+  - Run `dot -Tpng <input> -o <output>.png`
+  - On success / failure: same pattern as MermaidRenderer
+- Alt-text marker: `[graphviz-source]<source>` for round-trip
+- Add `graphviz` to optional extras: `diagram-graphviz`
+- Integration test (skipped if `dot` not found): `test_graphviz_render_roundtrip`
+
+Deliverable: When `dot` is on PATH, graphviz diagrams render and embed; otherwise
+source-only fallback.
+
+---
+
+## T04 — PlantUML integration
+
+```task
+id: MRKD-WP-0005-T04
+status: todo
+priority: medium
+state_hub_task_id: ab0da6e4-a0f2-4a57-baac-5727b741c74f
+```
+
+Implement the `PlantUMLRenderer` backend that shells out to `plantuml`.
+
+- `PlantUMLRenderer.can_render("plantuml") -> bool`: checks `shutil.which("plantuml")`
+- `PlantUMLRenderer.render(source, "plantuml", output_path)`:
+  - Write source to a temp `.puml` file
+  - Run `plantuml -tpng <input>` (output written adjacent to input by default)
+  - Move/rename output PNG to `output_path`
+  - On success / failure: same pattern
+- Alt-text marker: `[plantuml-source]<source>`
+- Add `plantuml` to optional extras: `diagram-plantuml`
+- Integration test (skipped if `plantuml` not found): `test_plantuml_render_roundtrip`
+
+Deliverable: When `plantuml` is on PATH, PlantUML diagrams render and embed.
+
+---
+
+## T05 — Rendered diagram round-trip integration tests
+
+```task
+id: MRKD-WP-0005-T05
+status: todo
+priority: medium
+state_hub_task_id: 65b67ed9-5862-4322-acfb-5d39dab7e8d5
+```
+
+Extend the regression corpus and test harness to cover the rendered-diagram path
+end-to-end, in addition to the existing source-only tests.
+
+- Add `tests/regression/level3/rendered_diagrams_document.md` with mermaid, graphviz,
+  and plantuml blocks
+- `tests/regression/test_level3_rendered_diagrams.py`:
+  - Each test is `pytest.mark.skipif(shutil.which("mmdc") is None, ...)`
+  - Full round-trip: build → import → compare; assert diagrams preserved
+  - Assert `differ` reports zero broken diagrams when renderer was available
+- Verify existing `test_level3_diagrams.py` (source-only) still passes regardless
+  of renderer availability — the fallback path must remain stable
+
+Deliverable: Rendered-diagram regression tests pass on systems with renderers installed;
+source-only tests pass on all systems.
+
+---
+
+## How to Work
+
+- T01 must be complete before T02–T04 (abstraction must exist before concrete backends)
+- T02, T03, T04 are independent of each other — can be worked in any order or in parallel
+- T05 depends on T02–T04 (needs the renderers to test the rendered path)
+- Use `pytest.mark.skipif` to gate renderer-dependent tests — never fail on missing tools
+
+## Updating Task Status
+
+```
+status: todo        →  status: in_progress   (when you start it)
+status: in_progress →  status: done          (when verified complete)
+```
+
+When every task is `done`, set the frontmatter `status: done`.
+
+## Success Criteria
+
+Before marking the workplan done:
+
+1. Every task block has `status: done`
+2. Workplan frontmatter `status: done`
+3. All existing tests pass unchanged (source-only fallback unbroken)
+4. On a system with `mmdc`: `pytest tests/regression/test_level3_rendered_diagrams.py` passes
+5. `ruff check` and `mypy` clean
+6. Optional extras `diagram-mermaid`, `diagram-graphviz`, `diagram-plantuml` documented
+   in `pyproject.toml`
--- a/workplans/MRKD-WP-0006-packaging-distribution.md
+++ b/workplans/MRKD-WP-0006-packaging-distribution.md
@@ -0,0 +1,171 @@
+---
+id: MRKD-WP-0006
+type: workplan
+domain: markitect
+repo: marki-docx
+status: active
+state_hub_workstream_id: 7e255145-8d18-4f22-b1ca-31f02944b890
+created: 2026-03-16
+updated: 2026-03-16
+---
+
+# MRKD-WP-0006 — Packaging & Distribution
+
+Make markidocx installable from PyPI and runnable as a containerised REST service.
+Add a CI/CD pipeline that enforces quality gates on every push and automates
+release artefact production.
+
+**Scope:** CI/CD pipeline, PyPI packaging, Docker REST image, release process
+**Out of scope:** functional changes — this workstream is purely infrastructure
+**Depends on:** MRKD-WP-0001, MRKD-WP-0002, MRKD-WP-0003 — all complete
+
+---
+
+## T01 — CI pipeline — GitHub Actions
+
+```task
+id: MRKD-WP-0006-T01
+status: todo
+priority: high
+state_hub_task_id: cfca0094-b5ae-45ec-80c9-e7705f37bd12
+```
+
+Create a GitHub Actions workflow that runs on every push and pull request to `main`.
+
+File: `.github/workflows/ci.yml`
+
+Jobs:
+1. **test** — matrix: Python 3.11, 3.12
+   - `pip install -e ".[dev]"`
+   - `ruff check .`
+   - `mypy src/`
+   - `pytest --tb=short -q`
+2. **coverage** (optional, on `main` only) — `pytest --cov=markidocx --cov-report=xml`;
+   upload to Codecov or similar
+3. Cache pip downloads between runs using `actions/cache`
+
+Quality gate: the `test` job must pass before any PR can be merged.
+
+Deliverable: `.github/workflows/ci.yml` present; CI passes on a clean push to `main`.
+
+---
+
+## T02 — PyPI packaging and version management
+
+```task
+id: MRKD-WP-0006-T02
+status: todo
+priority: high
+state_hub_task_id: 44f60455-b144-48e3-93a0-74869018c2ea
+```
+
+Prepare markidocx for publication to PyPI and establish a version management convention.
+
+- Adopt **semantic versioning** (`MAJOR.MINOR.PATCH`); initial release: `0.1.0`
+- Single source of truth for version: `src/markidocx/__init__.py` (`__version__ = "0.1.0"`)
+- `pyproject.toml`:
+  - `[project] version` reads from `__init__.py` via `importlib.metadata` or static
+  - `[project.optional-dependencies]` groups: `dev`, `diagram-mermaid`,
+    `diagram-graphviz`, `diagram-plantuml`
+  - `[project.urls]`: homepage, source, tracker
+- Add `.github/workflows/publish.yml`:
+  - Triggered on `push` to tags matching `v*.*.*`
+  - Builds with `python -m build`
+  - Publishes to PyPI using `pypa/gh-action-pypi-publish`
+  - Uses a trusted-publisher setup (OIDC, no API token in repo secrets)
+- Dry-run test: `python -m build && twine check dist/*` must pass locally
+
+Deliverable: `pip install markidocx` works after first PyPI release; `markidocx --version`
+returns the correct version string.
+
+---
+
+## T03 — Docker image for REST service
+
+```task
+id: MRKD-WP-0006-T03
+status: todo
+priority: medium
+state_hub_task_id: 05ff77b0-6347-4c96-9fcd-c2b2baaf0fce
+```
+
+Provide a Docker image that runs the markidocx REST service, making it easy to deploy
+in pipeline and automation contexts.
+
+File: `Dockerfile`
+
+- Base: `python:3.12-slim`
+- Install markidocx with `pip install markidocx` (or from local wheel in CI)
+- `ENTRYPOINT ["markidocx", "serve"]`
+- Default `CMD ["--host", "0.0.0.0", "--port", "8080"]`
+- Expose port 8080
+- Non-root user for security
+- `.dockerignore`: exclude tests, docs, `.git`, `*.pyc`, `dist/`
+
+Add `.github/workflows/docker.yml`:
+- Triggered on `push` to tags matching `v*.*.*`
+- Build and push to GitHub Container Registry (`ghcr.io`)
+- Tag as `latest` and `v<version>`
+
+Usage documented in README:
+```bash
+docker run -p 8080:8080 ghcr.io/<org>/marki-docx:latest
+```
+
+Deliverable: `docker build . && docker run -p 8080:8080 <image>` starts the REST service;
+`curl http://localhost:8080/health` returns `{"status": "ok"}`.
+
+---
+
+## T04 — Release process and changelog convention
+
+```task
+id: MRKD-WP-0006-T04
+status: todo
+priority: medium
+state_hub_task_id: 177e4861-d153-4b1d-85d4-a272da14bfe5
+```
+
+Document the release process and establish a changelog convention so that releases
+are repeatable and traceable.
+
+- `CHANGELOG.md` with **Keep a Changelog** format (`Unreleased`, `Added`, `Changed`,
+  `Fixed`, `Deprecated`, `Removed`)
+- Populate with a retrospective entry for `v0.1.0` covering WP-0001 through WP-0003
+- `docs/release-process.md`:
+  - Checklist: all tests pass, ruff+mypy clean, SBOM updated, CHANGELOG populated,
+    version bumped, corpus regression passes
+  - Tag: `git tag -s v<version> -m "Release v<version>"`
+  - Push tag → triggers `publish.yml` and `docker.yml` automatically
+- Update `CLAUDE.md` with a pointer to the release process doc
+
+Deliverable: `CHANGELOG.md` present with `v0.1.0` entry; `docs/release-process.md`
+present; release checklist is executable end-to-end.
+
+---
+
+## How to Work
+
+- T01 and T02 are independent — can be worked in parallel
+- T03 depends on T02 being sufficiently complete (needs a buildable package)
+- T04 can be worked alongside any other task — it is primarily documentation
+
+## Updating Task Status
+
+```
+status: todo        →  status: in_progress   (when you start it)
+status: in_progress →  status: done          (when verified complete)
+```
+
+When every task is `done`, set the frontmatter `status: done`.
+
+## Success Criteria
+
+Before marking the workplan done:
+
+1. Every task block has `status: done`
+2. Workplan frontmatter `status: done`
+3. CI passes on `main` (GitHub Actions green)
+4. `python -m build && twine check dist/*` passes locally
+5. `docker build .` produces a working image
+6. `CHANGELOG.md` and `docs/release-process.md` present