diff --git a/workplans/MRKD-WP-0004-stable-corpus-architecture.md b/workplans/MRKD-WP-0004-stable-corpus-architecture.md new file mode 100644 index 0000000..21bb031 --- /dev/null +++ b/workplans/MRKD-WP-0004-stable-corpus-architecture.md @@ -0,0 +1,190 @@ +--- +id: MRKD-WP-0004 +type: workplan +domain: markitect +repo: marki-docx +status: active +state_hub_workstream_id: 91d06c92-caa8-42fc-b6d4-82340f1bed4f +created: 2026-03-16 +updated: 2026-03-16 +--- + +# MRKD-WP-0004 — Stable Documentation Corpus & Architecture Records + +Fulfil FR-1101 by establishing the markidocx product documentation itself as a real, +managed markidocx project. The specs (PRD, FRS, UCC) become a live round-trip corpus +that the `release-regression` workflow runs against on every release. This workstream +also writes the two deferred architecture decision records and generates the first SBOM. + +**Scope:** FR-1101–1110 (stable corpus & self-test), ADR-002, ADR-003, SBOM +**Out of scope:** diagram rendering, packaging, CI/CD — addressed in WP-0005/0006 +**Depends on:** MRKD-WP-0001, MRKD-WP-0002, MRKD-WP-0003 — all complete + +--- + +## T01 — Set up specs as real markidocx project manifest + +```task +id: MRKD-WP-0004-T01 +status: todo +priority: high +state_hub_task_id: f1a36613-ceaa-4786-ac39-cd3a7fd1c142 +``` + +Create a manifest file that treats the markidocx product documentation as a live +markidocx project. This makes the specs the stable corpus for regression testing +as required by FR-1101. + +- Create `corpus/markidocx-docs/manifest.yaml`: + - `project.name: markidocx-docs` + - `project.feature_level: level1` + - `project.family: article` + - `sources`: PRD, FRS v0.2, UCC (relative paths into `specs/`) + - `output.dir: corpus/markidocx-docs/dist` +- Run `markidocx validate corpus/markidocx-docs/manifest.yaml` — must exit 0 +- Run `markidocx build corpus/markidocx-docs/manifest.yaml` — must produce valid DOCX +- Run `markidocx import` + `markidocx compare` — must report clean or expected drift only +- Document any structural drift in `corpus/markidocx-docs/known-drift.md` + +Deliverable: `markidocx build corpus/markidocx-docs/manifest.yaml` succeeds; a DOCX +of the product documentation exists in `corpus/markidocx-docs/dist/`. + +--- + +## T02 — Wire release-regression workflow against specs corpus + +```task +id: MRKD-WP-0004-T02 +status: todo +priority: high +state_hub_task_id: f17e959f-28da-4386-9004-b5e036054b06 +``` + +Connect the `release-regression` composite workflow to the real documentation corpus +so that `markidocx workflow release-regression corpus/markidocx-docs/manifest.yaml` +runs a full build → import → compare cycle and records evidence (FR-1102, FR-1103, +FR-1106, FR-1107). + +- Update `workflows.py` `release-regression` handler to accept a manifest path argument; + default to the corpus manifest when none supplied +- Run the workflow; assert the evidence set contains build, import, and drift reports +- Add `tests/regression/test_corpus_regression.py`: + - Invokes `release-regression` on the corpus manifest + - Asserts workflow result is `full` or `with-fallback` (not `failed`) + - Asserts evidence artefacts are present and have correct traceability fields (FR-1110) +- Disclose corpus identity in regression output (FR-1109): include corpus manifest path + and its git HEAD SHA as `corpus_id` in the workflow result + +Deliverable: `pytest tests/regression/test_corpus_regression.py` passes; evidence +written to `.markidocx/evidence/` and retrievable via CLI. + +--- + +## T03 — ADR-002: python-docx as conversion engine + +```task +id: MRKD-WP-0004-T03 +status: todo +priority: medium +state_hub_task_id: bfe2a9fa-25b2-4b4b-b21b-eae457716ce0 +``` + +Write the architecture decision record explaining the choice of python-docx as the +DOCX conversion engine. This was identified as a deferred deliverable during WP-0001. + +File: `architecture/ADR-002-python-docx-as-conversion-engine.md` + +Cover: +- **Context:** need to produce and consume .docx files from Python; alternatives evaluated + (pandoc subprocess, docx2python, mammoth, python-docx) +- **Decision:** python-docx for both build (write) and import (read) +- **Consequences:** direct paragraph/run model maps cleanly to Markdown structure; + no subprocess dependency; limited to Open XML subset exposed by python-docx API; + complex Word features (track changes, SmartArt) are out of scope by design +- **Alternatives rejected:** pandoc — heavier dependency, harder to control structure; + mammoth — read-only; docx2python — limited write support + +Deliverable: `architecture/ADR-002-*.md` present and follows ADR-001 conventions. + +--- + +## T04 — ADR-003: manifest YAML schema + +```task +id: MRKD-WP-0004-T04 +status: todo +priority: medium +state_hub_task_id: b6de6733-b332-4efc-9e23-82fce205b856 +``` + +Write the architecture decision record documenting the manifest YAML schema design. + +File: `architecture/ADR-003-manifest-yaml-schema.md` + +Cover: +- **Context:** need a project definition format that is human-writable, version-controlled, + and parseable without a schema registry +- **Decision:** YAML with a fixed top-level structure (`project`, `sources`, `output`, + `metadata`); validated on load via dataclass coercion +- **Schema snapshot:** include the current field definitions as a reference +- **Consequences:** simple for users; no JSON Schema or Pydantic dependency; evolving + the schema requires coordination with manifest.py +- **Alternatives rejected:** TOML (less familiar in doc tooling), JSON (less writable), + a database manifest (over-engineered for single-project use) + +Deliverable: `architecture/ADR-003-*.md` present. + +--- + +## T05 — SBOM generation and state-hub registration + +```task +id: MRKD-WP-0004-T05 +status: todo +priority: medium +state_hub_task_id: 36aecd50-8176-4122-9706-a8697d8f5936 +``` + +Generate and register the first SBOM for marki-docx so the state hub has an accurate +dependency picture. + +```bash +cd ~/the-custodian/state-hub +make ingest-sbom REPO=marki-docx SCAN=1 REPO_PATH=/home/tegwick/marki-docx +``` + +- Verify the SBOM ingestion completes without errors +- Confirm `last_sbom_at` is set for `marki-docx` in the state hub +- Document any licence issues or unexpected transitive dependencies +- Add a note to CLAUDE.md reminding to re-run SBOM after dependency changes + +Deliverable: State hub shows `last_sbom_at` set for `marki-docx`; no unresolved +licence issues. + +--- + +## How to Work + +- Work through tasks in priority order: T01 → T02 (high), then T03 → T04 → T05 (medium) +- T01 must complete before T02 (T02 depends on the corpus manifest) +- T03 and T04 are independent writing tasks — can be done in any order or in parallel + +## Updating Task Status + +``` +status: todo → status: in_progress (when you start it) +status: in_progress → status: done (when verified complete) +``` + +When every task is `done`, set the frontmatter `status: done`. + +## Success Criteria + +Before marking the workplan done: + +1. Every task block has `status: done` +2. Workplan frontmatter `status: done` +3. `corpus/markidocx-docs/manifest.yaml` present and builds cleanly +4. `pytest tests/regression/test_corpus_regression.py` passes +5. `architecture/ADR-002-*.md` and `architecture/ADR-003-*.md` present +6. State hub shows `last_sbom_at` set for `marki-docx` diff --git a/workplans/MRKD-WP-0005-diagram-renderer-integration.md b/workplans/MRKD-WP-0005-diagram-renderer-integration.md new file mode 100644 index 0000000..ee2581a --- /dev/null +++ b/workplans/MRKD-WP-0005-diagram-renderer-integration.md @@ -0,0 +1,185 @@ +--- +id: MRKD-WP-0005 +type: workplan +domain: markitect +repo: marki-docx +status: active +state_hub_workstream_id: 2ef47f11-d828-436d-8955-c58e13c50752 +created: 2026-03-16 +updated: 2026-03-16 +--- + +# MRKD-WP-0005 — Diagram Renderer Integration + +Complete the deferred rendering path for FR-533/534. Currently `diagrams.py` operates +in source-only mode: diagram fenced blocks are preserved as source through the round-trip +but are never rendered to images. This workstream adds actual rendering support for +Mermaid, Graphviz, and PlantUML when the corresponding CLI tools are available, while +preserving the graceful source-only fallback when they are not. + +**Scope:** FR-533 (auto-diagram support), FR-534 (diagram intent preservation), +FR-538 (processor-dependency disclosure) +**Out of scope:** new diagram syntaxes beyond mermaid/graphviz/plantuml +**Depends on:** MRKD-WP-0003 (diagrams.py source-only foundation) — complete + +--- + +## T01 — Renderer abstraction layer and detection + +```task +id: MRKD-WP-0005-T01 +status: todo +priority: high +state_hub_task_id: c4911ecc-1e3c-4d22-a6fb-92d1ed319274 +``` + +Introduce a `RendererBackend` abstraction in `diagrams.py` so that each diagram type +can be rendered by a pluggable backend. The builder calls the abstraction; concrete +backends handle tool detection and subprocess invocation. + +- `DiagramRenderer` protocol: `can_render(diagram_type: str) -> bool`, + `render(source: str, diagram_type: str, output_path: Path) -> RendererResult` +- `RendererResult`: `success: bool`, `output_path: Path | None`, `warning: WarningRecord | None` +- `detect_renderers() -> dict[str, DiagramRenderer]` — probe PATH for `mmdc`, `dot`, + `plantuml`; return only those found +- Builder updated to call `detect_renderers()` at build time; if a renderer is found + for the diagram type → render to PNG and embed; if not → source-only fallback + + `WarningRecord(reason="renderer-unavailable", construct=diagram_type)` (FR-538) +- Unit tests: `test_renderer_detection` — mock PATH; `test_fallback_when_no_renderer` + +Deliverable: `pytest tests/test_level3_diagrams.py` still passes; renderer abstraction +in place and tested. + +--- + +## T02 — Mermaid CLI (mmdc) integration + +```task +id: MRKD-WP-0005-T02 +status: todo +priority: high +state_hub_task_id: 87caa295-f466-4e2e-ba06-4b1a801ca976 +``` + +Implement the `MermaidRenderer` backend that shells out to `mmdc` (Mermaid CLI). + +- `MermaidRenderer.can_render("mermaid") -> bool`: checks `shutil.which("mmdc")` +- `MermaidRenderer.render(source, "mermaid", output_path)`: + - Write source to a temp `.mmd` file + - Run `mmdc -i -o .png` + - On success: return `RendererResult(success=True, output_path=...)` + - On failure: return `RendererResult(success=False, warning=WarningRecord(reason="render-failed"))` +- Builder: PNG embedded into DOCX as image; alt-text carries `[mermaid-source]` marker + for round-trip (FR-534) +- Importer: existing alt-text marker detection already handles this — verify round-trip works +- Add `mmdc` to `pyproject.toml` optional extras: `diagram-mermaid` +- Integration test (skipped if `mmdc` not found): `test_mermaid_render_roundtrip` + +Deliverable: When `mmdc` is on PATH, mermaid diagrams render to PNG and embed in DOCX; +round-trip restores source. When absent, source-only fallback with warning. + +--- + +## T03 — Graphviz (dot) integration + +```task +id: MRKD-WP-0005-T03 +status: todo +priority: medium +state_hub_task_id: 8ceb771d-0b16-452d-b1cc-c5c2c40fe723 +``` + +Implement the `GraphvizRenderer` backend that shells out to `dot`. + +- `GraphvizRenderer.can_render("graphviz") -> bool`: checks `shutil.which("dot")` +- `GraphvizRenderer.render(source, "graphviz", output_path)`: + - Write source to a temp `.dot` file + - Run `dot -Tpng -o .png` + - On success / failure: same pattern as MermaidRenderer +- Alt-text marker: `[graphviz-source]` for round-trip +- Add `graphviz` to optional extras: `diagram-graphviz` +- Integration test (skipped if `dot` not found): `test_graphviz_render_roundtrip` + +Deliverable: When `dot` is on PATH, graphviz diagrams render and embed; otherwise +source-only fallback. + +--- + +## T04 — PlantUML integration + +```task +id: MRKD-WP-0005-T04 +status: todo +priority: medium +state_hub_task_id: ab0da6e4-a0f2-4a57-baac-5727b741c74f +``` + +Implement the `PlantUMLRenderer` backend that shells out to `plantuml`. + +- `PlantUMLRenderer.can_render("plantuml") -> bool`: checks `shutil.which("plantuml")` +- `PlantUMLRenderer.render(source, "plantuml", output_path)`: + - Write source to a temp `.puml` file + - Run `plantuml -tpng ` (output written adjacent to input by default) + - Move/rename output PNG to `output_path` + - On success / failure: same pattern +- Alt-text marker: `[plantuml-source]` +- Add `plantuml` to optional extras: `diagram-plantuml` +- Integration test (skipped if `plantuml` not found): `test_plantuml_render_roundtrip` + +Deliverable: When `plantuml` is on PATH, PlantUML diagrams render and embed. + +--- + +## T05 — Rendered diagram round-trip integration tests + +```task +id: MRKD-WP-0005-T05 +status: todo +priority: medium +state_hub_task_id: 65b67ed9-5862-4322-acfb-5d39dab7e8d5 +``` + +Extend the regression corpus and test harness to cover the rendered-diagram path +end-to-end, in addition to the existing source-only tests. + +- Add `tests/regression/level3/rendered_diagrams_document.md` with mermaid, graphviz, + and plantuml blocks +- `tests/regression/test_level3_rendered_diagrams.py`: + - Each test is `pytest.mark.skipif(shutil.which("mmdc") is None, ...)` + - Full round-trip: build → import → compare; assert diagrams preserved + - Assert `differ` reports zero broken diagrams when renderer was available +- Verify existing `test_level3_diagrams.py` (source-only) still passes regardless + of renderer availability — the fallback path must remain stable + +Deliverable: Rendered-diagram regression tests pass on systems with renderers installed; +source-only tests pass on all systems. + +--- + +## How to Work + +- T01 must be complete before T02–T04 (abstraction must exist before concrete backends) +- T02, T03, T04 are independent of each other — can be worked in any order or in parallel +- T05 depends on T02–T04 (needs the renderers to test the rendered path) +- Use `pytest.mark.skipif` to gate renderer-dependent tests — never fail on missing tools + +## Updating Task Status + +``` +status: todo → status: in_progress (when you start it) +status: in_progress → status: done (when verified complete) +``` + +When every task is `done`, set the frontmatter `status: done`. + +## Success Criteria + +Before marking the workplan done: + +1. Every task block has `status: done` +2. Workplan frontmatter `status: done` +3. All existing tests pass unchanged (source-only fallback unbroken) +4. On a system with `mmdc`: `pytest tests/regression/test_level3_rendered_diagrams.py` passes +5. `ruff check` and `mypy` clean +6. Optional extras `diagram-mermaid`, `diagram-graphviz`, `diagram-plantuml` documented + in `pyproject.toml` diff --git a/workplans/MRKD-WP-0006-packaging-distribution.md b/workplans/MRKD-WP-0006-packaging-distribution.md new file mode 100644 index 0000000..f7ea95f --- /dev/null +++ b/workplans/MRKD-WP-0006-packaging-distribution.md @@ -0,0 +1,171 @@ +--- +id: MRKD-WP-0006 +type: workplan +domain: markitect +repo: marki-docx +status: active +state_hub_workstream_id: 7e255145-8d18-4f22-b1ca-31f02944b890 +created: 2026-03-16 +updated: 2026-03-16 +--- + +# MRKD-WP-0006 — Packaging & Distribution + +Make markidocx installable from PyPI and runnable as a containerised REST service. +Add a CI/CD pipeline that enforces quality gates on every push and automates +release artefact production. + +**Scope:** CI/CD pipeline, PyPI packaging, Docker REST image, release process +**Out of scope:** functional changes — this workstream is purely infrastructure +**Depends on:** MRKD-WP-0001, MRKD-WP-0002, MRKD-WP-0003 — all complete + +--- + +## T01 — CI pipeline — GitHub Actions + +```task +id: MRKD-WP-0006-T01 +status: todo +priority: high +state_hub_task_id: cfca0094-b5ae-45ec-80c9-e7705f37bd12 +``` + +Create a GitHub Actions workflow that runs on every push and pull request to `main`. + +File: `.github/workflows/ci.yml` + +Jobs: +1. **test** — matrix: Python 3.11, 3.12 + - `pip install -e ".[dev]"` + - `ruff check .` + - `mypy src/` + - `pytest --tb=short -q` +2. **coverage** (optional, on `main` only) — `pytest --cov=markidocx --cov-report=xml`; + upload to Codecov or similar +3. Cache pip downloads between runs using `actions/cache` + +Quality gate: the `test` job must pass before any PR can be merged. + +Deliverable: `.github/workflows/ci.yml` present; CI passes on a clean push to `main`. + +--- + +## T02 — PyPI packaging and version management + +```task +id: MRKD-WP-0006-T02 +status: todo +priority: high +state_hub_task_id: 44f60455-b144-48e3-93a0-74869018c2ea +``` + +Prepare markidocx for publication to PyPI and establish a version management convention. + +- Adopt **semantic versioning** (`MAJOR.MINOR.PATCH`); initial release: `0.1.0` +- Single source of truth for version: `src/markidocx/__init__.py` (`__version__ = "0.1.0"`) +- `pyproject.toml`: + - `[project] version` reads from `__init__.py` via `importlib.metadata` or static + - `[project.optional-dependencies]` groups: `dev`, `diagram-mermaid`, + `diagram-graphviz`, `diagram-plantuml` + - `[project.urls]`: homepage, source, tracker +- Add `.github/workflows/publish.yml`: + - Triggered on `push` to tags matching `v*.*.*` + - Builds with `python -m build` + - Publishes to PyPI using `pypa/gh-action-pypi-publish` + - Uses a trusted-publisher setup (OIDC, no API token in repo secrets) +- Dry-run test: `python -m build && twine check dist/*` must pass locally + +Deliverable: `pip install markidocx` works after first PyPI release; `markidocx --version` +returns the correct version string. + +--- + +## T03 — Docker image for REST service + +```task +id: MRKD-WP-0006-T03 +status: todo +priority: medium +state_hub_task_id: 05ff77b0-6347-4c96-9fcd-c2b2baaf0fce +``` + +Provide a Docker image that runs the markidocx REST service, making it easy to deploy +in pipeline and automation contexts. + +File: `Dockerfile` + +- Base: `python:3.12-slim` +- Install markidocx with `pip install markidocx` (or from local wheel in CI) +- `ENTRYPOINT ["markidocx", "serve"]` +- Default `CMD ["--host", "0.0.0.0", "--port", "8080"]` +- Expose port 8080 +- Non-root user for security +- `.dockerignore`: exclude tests, docs, `.git`, `*.pyc`, `dist/` + +Add `.github/workflows/docker.yml`: +- Triggered on `push` to tags matching `v*.*.*` +- Build and push to GitHub Container Registry (`ghcr.io`) +- Tag as `latest` and `v` + +Usage documented in README: +```bash +docker run -p 8080:8080 ghcr.io//marki-docx:latest +``` + +Deliverable: `docker build . && docker run -p 8080:8080 ` starts the REST service; +`curl http://localhost:8080/health` returns `{"status": "ok"}`. + +--- + +## T04 — Release process and changelog convention + +```task +id: MRKD-WP-0006-T04 +status: todo +priority: medium +state_hub_task_id: 177e4861-d153-4b1d-85d4-a272da14bfe5 +``` + +Document the release process and establish a changelog convention so that releases +are repeatable and traceable. + +- `CHANGELOG.md` with **Keep a Changelog** format (`Unreleased`, `Added`, `Changed`, + `Fixed`, `Deprecated`, `Removed`) +- Populate with a retrospective entry for `v0.1.0` covering WP-0001 through WP-0003 +- `docs/release-process.md`: + - Checklist: all tests pass, ruff+mypy clean, SBOM updated, CHANGELOG populated, + version bumped, corpus regression passes + - Tag: `git tag -s v -m "Release v"` + - Push tag → triggers `publish.yml` and `docker.yml` automatically +- Update `CLAUDE.md` with a pointer to the release process doc + +Deliverable: `CHANGELOG.md` present with `v0.1.0` entry; `docs/release-process.md` +present; release checklist is executable end-to-end. + +--- + +## How to Work + +- T01 and T02 are independent — can be worked in parallel +- T03 depends on T02 being sufficiently complete (needs a buildable package) +- T04 can be worked alongside any other task — it is primarily documentation + +## Updating Task Status + +``` +status: todo → status: in_progress (when you start it) +status: in_progress → status: done (when verified complete) +``` + +When every task is `done`, set the frontmatter `status: done`. + +## Success Criteria + +Before marking the workplan done: + +1. Every task block has `status: done` +2. Workplan frontmatter `status: done` +3. CI passes on `main` (GitHub Actions green) +4. `python -m build && twine check dist/*` passes locally +5. `docker build .` produces a working image +6. `CHANGELOG.md` and `docs/release-process.md` present