E2e tests and frist use docs

This commit is contained in:
2026-05-04 22:58:59 +02:00
parent 5dfb403979
commit a7ab4904d5
9 changed files with 583 additions and 10 deletions

View File

@@ -23,6 +23,8 @@ It turns markdown from plain text into a **programmable knowledge substrate**.
The current practical usecase matrix is maintained in
`docs/practical-usecase-relevance.md`. That document translates this intent into
adoption-oriented scenarios, relevance expectations, and E2E coverage targets.
First-use documentation lives in `docs/getting-started.md`, with a command
cheatsheet and examples index beside it.
---

View File

@@ -15,6 +15,10 @@ requirements documents in `wiki/`.
- `wiki/FunctionalRequirementsSpecification.md` - observable functional behavior
- `docs/markitect-main-scope-assessment.md` - migration assessment from `markitect-main`
- `docs/practical-usecase-relevance.md` - practical adoption usecases and relevance expectations
- `docs/getting-started.md` - first-use guide from checkout to practical commands
- `docs/command-cheatsheet.md` - command-oriented workflow cheat sheet
- `docs/examples-index.md` - map from examples to usecases and commands
- `docs/performance-notes.md` - local performance posture and smoke coverage
- `docs/cli-reference.md` - generated `mkt` command reference
- `docs/api-reference.md` - generated public API reference
- `workplans/` - source-of-truth implementation and migration plans
@@ -32,6 +36,13 @@ Run the tests:
python3 -m pytest
```
Install locally from a checkout:
```bash
python3 -m pip install -e .
mkt --help
```
Try the parser CLI from a checkout:
```bash

108
docs/command-cheatsheet.md Normal file
View File

@@ -0,0 +1,108 @@
# Command Cheatsheet
Use `mkt --help` and `mkt <command> --help` for the live command surface. This
cheatsheet is organized by intent.
## Inspect Markdown
```bash
mkt parse file.md --format json
mkt parse file.md --format tree
mkt ast show file.md --format yaml
mkt ast stats file.md
mkt metrics file.md
```
## Validate Structure
```bash
mkt validate file.md --schema schema.md
mkt schema validate schema.md
mkt contract validate contract.md
mkt contract check file.md --contract contract.md
mkt contract form-state file.md --contract contract.md --context context.yaml
```
## Query And Extract
```bash
mkt query file.md 'sections[heading=Decision]'
mkt extract file.md 'sections[heading=Decision]' --format text
mkt query file.md '$.sections[*].heading.text' --engine jsonpath
```
## Transform And Reuse
```bash
mkt transform file.md --set status=ready
mkt compose one.md two.md --title Combined
mkt include file.md --base-dir .
mkt ref resolve context.md 'std:clauses.md#payment-terms' --root examples/references
mkt process file.md --root .
```
## Split And Literate Workflows
```bash
mkt explode file.md --output-dir parts
mkt implode parts --output rebuilt.md
mkt tangle examples/literate/app.md --output-dir build
mkt weave examples/literate/app.md --output woven.md
```
## Generate Markdown
```bash
mkt template inspect template.md
mkt template render template.md --data data.yaml
mkt generate stub --contract contract.md --include-optional
mkt generate rules generation.md --output-dir out
mkt function list
mkt function check file.md
mkt function render file.md
```
## Index, Search, And Refresh
```bash
mkt cache init --root .
mkt cache build docs --root .
mkt cache status docs --root .
mkt cache index docs --root .
mkt cache query 'sections[heading=Decision]' --root .
mkt search latency --root . --limit 10
mkt backend refresh-plan docs --root .
```
## Policy-Aware Access
```bash
mkt policy check public-agent read object-id --policy policy.yaml
mkt policy subject claims.yaml --policy-map policy-map.yaml
mkt policy resource-manifest manifest.yaml
mkt search internal --root . --policy policy.yaml --subject internal-agent
```
## Workflows And Context Packages
```bash
mkt workflow inspect workflow.md
mkt workflow plan workflow.md --output-dir out
mkt workflow run workflow.md --output-dir out
mkt context pack query --source file.md --max-items 5
mkt context activate package-id
mkt context explain package-id
mkt context refresh package-id
mkt context list
```
## Discovery And Docs
```bash
mkt completion bash --instructions
mkt extension list
mkt extension inspect backend.local-sqlite
mkt extension commands
mkt docs cli --output docs/cli-reference.md
mkt docs api --output docs/api-reference.md
```

50
docs/examples-index.md Normal file
View File

@@ -0,0 +1,50 @@
# Examples Index
This index maps example files to practical usecases and useful commands.
## Contracts And Diagnostics
| Files | Usecase | Try |
| --- | --- | --- |
| `examples/contracts/adr.contract.md`, `examples/documents/adr-valid.md` | ADR contract validation | `mkt contract check examples/documents/adr-valid.md --contract examples/contracts/adr.contract.md` |
| `examples/contracts/business-letter.contract.md`, `examples/documents/business-letter-valid.md` | Form-like business document validation | `mkt contract form-state examples/documents/business-letter-valid.md --contract examples/contracts/business-letter.contract.md --context examples/runtime/business-letter.context.yaml` |
| `examples/contracts/prd-frs.contract.md`, `examples/documents/prd-frs-valid.md` | Larger product/specification contract | `mkt contract check examples/documents/prd-frs-valid.md --contract examples/contracts/prd-frs.contract.md` |
| `examples/diagnostics/*.expected-diagnostics.md` | Expected failure explanations | Compare with `mkt contract check ... --format text` |
## Query, References, And Processors
| Files | Usecase | Try |
| --- | --- | --- |
| `examples/references/context.md` | Namespace, region, fence, and section references | `mkt ref resolve examples/references/context.md 'std:clauses.md#payment-terms' --root examples/references` |
| `examples/references/standard/clauses.md` | Referenced reusable content | Use with `mkt ref resolve` or `mkt process` |
| `examples/migration/legacy-path-include.md` | Migration-style include handling | `mkt include examples/migration/legacy-path-include.md --base-dir examples/migration` |
| `examples/functions/basic-functions.md` | Deterministic document functions | `mkt function render examples/functions/basic-functions.md` |
## Templates, Generation, And Workflows
| Files | Usecase | Try |
| --- | --- | --- |
| `examples/templates/adr-summary.template.md`, `examples/templates/adr-summary.data.yaml` | Data-bound Markdown template | `mkt template render examples/templates/adr-summary.template.md --data examples/templates/adr-summary.data.yaml` |
| `examples/templates/adr-summary.generation.md` | Rule-based generation plan | `mkt generate rules examples/templates/adr-summary.generation.md --output-dir /tmp/markitect-generated` |
| `examples/workflows/source-snippets.workflow.md` | Extract snippets from Markdown sources | `mkt workflow plan examples/workflows/source-snippets.workflow.md --output-dir /tmp/markitect-workflow` |
| `examples/workflows/adr-release-notes.workflow.md` | ADR release notes workflow | `mkt workflow plan examples/workflows/adr-release-notes.workflow.md --output-dir /tmp/markitect-workflow` |
| `examples/workflows/assisted-review.workflow.md` | Assisted-generation boundary shape | Inspect with `mkt workflow inspect` |
## Cache, Backend, Policy, And Context
| Files | Usecase | Try |
| --- | --- | --- |
| `examples/backends/local-sqlite-backend.md` | Backend manifest inspection | `mkt backend inspect local-sqlite --path examples/backends/local-sqlite-backend.md` |
| `examples/backend-state/snapshot-state.yaml` | Refresh planning fixture | `mkt backend refresh-plan examples/documents --state examples/backend-state/snapshot-state.yaml --root .` |
| `examples/policy/local-label-policy.yaml` | Local policy gateway | `mkt policy check public-agent read note --policy examples/policy/local-label-policy.yaml --label public` |
| `examples/policy/enterprise-policy-map.yaml` | Enterprise IAM mapping fixture | `mkt policy subject examples/policy/netkingdom-claims.yaml --policy-map examples/policy/enterprise-policy-map.yaml` |
| `examples/memory/workplan-context.manifest.yaml` | Context package manifest | `mkt context pack examples/memory/workplan-context.manifest.yaml --root . --no-save` |
## Literate And Migration
| Files | Usecase | Try |
| --- | --- | --- |
| `examples/literate/app.md` | Weave/tangle style code/document workflow | `mkt tangle examples/literate/app.md --output-dir /tmp/markitect-tangle` |
| `examples/migration/legacy-explode-source.md` | Reversible section splitting | `mkt explode examples/migration/legacy-explode-source.md --output-dir /tmp/markitect-parts` |
| `examples/migration/legacy-literate.md` | Legacy literate migration sample | `mkt weave examples/migration/legacy-literate.md` |
| `examples/classes/prd-classes.yaml` | Content class composition | `mkt class resolve examples/classes/prd-classes.yaml product-prd` |

130
docs/getting-started.md Normal file
View File

@@ -0,0 +1,130 @@
# Getting Started
This guide gets from a checkout to practical Markitect use without requiring a
service, database server, LLM provider, or project configuration.
## Install For Local Use
From the repository root:
```bash
python3 -m pip install -e .
mkt --help
```
When working from a checkout without installing:
```bash
PYTHONPATH=src python3 -m markitect_tool --help
```
## Add Shell Completion
Generate completion for your shell:
```bash
mkt completion bash --instructions
mkt completion zsh --instructions
mkt completion fish --instructions
```
For Bash, the direct form is:
```bash
mkt completion bash > ~/.mkt-complete.bash
echo '. ~/.mkt-complete.bash' >> ~/.bashrc
```
## First Useful Commands
Parse a Markdown file:
```bash
mkt parse README.md --format tree
```
Check document size and complexity:
```bash
mkt metrics examples/documents/adr-valid.md
```
Validate a document contract:
```bash
mkt contract check examples/documents/adr-valid.md \
--contract examples/contracts/adr.contract.md
```
Query a section:
```bash
mkt query examples/documents/adr-valid.md 'sections[heading=Decision]' --format text
```
Build and search a local index:
```bash
mkt cache index examples/documents --root .
mkt search contracts --root . --limit 5
```
Generate from a template:
```bash
mkt template render examples/templates/adr-summary.template.md \
--data examples/templates/adr-summary.data.yaml
```
Run a declarative workflow in plan mode:
```bash
mkt workflow plan examples/workflows/adr-release-notes.workflow.md \
--output-dir /tmp/markitect-out
```
Create an inspectable context package:
```bash
mkt context pack contracts --search --root . --max-items 5 --format yaml
```
## Discover The Surface
List built-in extension descriptors:
```bash
mkt extension list
mkt extension commands
mkt extension inspect memory.context-package
```
Regenerate reference docs from the live implementation:
```bash
mkt docs cli --output docs/cli-reference.md
mkt docs api --output docs/api-reference.md
```
## Suggested First Workflow
1. Start with `mkt parse` and `mkt metrics` on one document.
2. Use `mkt contract check` with one of the example contracts.
3. Use `mkt query` or `mkt extract` to grab a section.
4. Move to `mkt cache index` and `mkt search` once you have a directory.
5. Use `mkt workflow plan` before writing workflow outputs.
6. Use `mkt extension commands` when looking for the command attached to a
capability.
## Diagnostics
Most validation-oriented commands return structured JSON or YAML when requested:
```bash
mkt contract check examples/documents/adr-invalid.md \
--contract examples/contracts/adr.contract.md \
--format json
```
Use machine-readable output in automation, and text output for quick local
inspection.

60
docs/performance-notes.md Normal file
View File

@@ -0,0 +1,60 @@
# Performance Notes
Markitect is designed to remain useful without persistent services. The current
performance posture is therefore local-first:
- parse individual Markdown files directly for one-off use
- build a local SQLite index for repeated corpus operations
- use refresh planning to avoid unnecessary parse/index work
- keep policy filtering and context package creation deterministic
## Current Smoke Coverage
`tests/test_practical_usecases_e2e.py` includes a large-corpus smoke test that
creates 120 synthetic Markdown files, indexes them locally, and runs an FTS
search.
The thresholds are intentionally generous:
- local cache/index build: under 30 seconds
- local indexed search: under 5 seconds
These are not benchmark claims. They are regression guards to catch accidental
algorithmic or IO mistakes while keeping the test portable.
## Practical Guidance
For one file:
```bash
mkt parse file.md
mkt query file.md 'sections[heading=Decision]'
```
For a directory you will query repeatedly:
```bash
mkt cache index docs --root .
mkt search keyword --root .
mkt cache query 'sections[heading=Decision]' --root .
```
Before refreshing derived work:
```bash
mkt backend refresh-plan docs --root . --verify-hashes
```
## Future Measurement
If performance becomes a release gate, add a separate benchmark suite instead
of tightening normal E2E tests. Good benchmark dimensions would be:
- number of documents
- total bytes
- heading/section density
- frontmatter size
- number of reference/include relationships
- policy labels per document
- index rebuild versus incremental refresh
- context package item and token budgets

View File

@@ -41,7 +41,7 @@ and descriptions mirror the operational view.
| `MKTT-WP-0014` | complete | done | `MKTT-WP-0009` | Markitect-side enterprise IAM access-control integration is complete: NetKingdom/key-cape-compatible identity claims, flex-auth resource/policy contract, directory group resolution fixtures, decision-log sink, workflow declarations, CLI commands, and external PDP request examples. |
| `MKTT-WP-0012` | complete | done | `MKTT-WP-0004`, `MKTT-WP-0010`, `MKTT-WP-0011` | Document function layer is complete: deterministic Markdown-native function descriptors, registry, inline/fenced syntax, pipelines, context bindings, CLI, docs, examples, diagnostics, provenance, and extension descriptor. |
| `MKTT-WP-0008` | complete | done | `MKTT-WP-0006`, `MKTT-WP-0007`, `MKTT-WP-0009` | Agent working-memory context cache is complete: context package schema, local registry, package creation from queries/search/manifests, deterministic summaries, namespaces, activation/deactivation/refresh/explain lifecycle, policy re-checks, CLI, docs, and examples. |
| `MKTT-WP-0017` | P1 | active | `MKTT-WP-0003`, `MKTT-WP-0013` | Current polish/adoption track: CLI/API completeness, shell completion, generated command/API docs, usecase relevance matrix, E2E fixture planning, accessibility, and test/performance iteration before more advanced feature work. |
| `MKTT-WP-0017` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0013` | CLI/API polish and practical adoption track is complete: shell completion, extension discovery, generated CLI/API docs, usecase relevance matrix, E2E fixture matrix, large-corpus smoke coverage, first-use docs, examples index, and command cheat sheet. |
| `MKTT-WP-0015` | P2 | todo | `MKTT-WP-0010`, `MKTT-WP-0011`, `MKTT-WP-0012` | Future render and document-function extensions: typed values, richer syntax, document-local reusable functions, Quarkdown/export adapters, render-aware references, assets, and permission sandboxing. Defer unless publishing/export pressure becomes current. |
| `MKTT-WP-0016` | P2 | todo | `MKTT-WP-0008`, `MKTT-WP-0007`, `MKTT-WP-0009`, `MKTT-WP-0013` | Follow-on agentic memory architecture: reasoning decision graphs, conversational paths, long-term knowledge graphs, memory service blueprints/profiles, graph-to-context-package compilation, and adapter boundaries. |

View File

@@ -1,4 +1,5 @@
import json
import time
from pathlib import Path
import pytest
@@ -203,3 +204,202 @@ def test_structure_reuse_policy_and_literate_usecases(tmp_path: Path):
implode = runner.invoke(main, ["implode", str(exploded), "--format", "markdown"])
assert implode.exit_code == 0, implode.output
assert "# Use Markdown Contracts" in implode.output
@pytest.mark.parametrize(
("label", "document", "contract", "context"),
[
("small", "examples/documents/concept-note-valid.md", "examples/contracts/concept-note.contract.md", None),
(
"typical",
"examples/documents/business-letter-valid.md",
"examples/contracts/business-letter.contract.md",
"examples/runtime/business-letter.context.yaml",
),
("large", "examples/documents/prd-frs-valid.md", "examples/contracts/prd-frs.contract.md", None),
],
)
def test_contract_and_runtime_usecases_small_typical_large(
label: str,
document: str,
contract: str,
context: str | None,
):
runner = CliRunner()
command = ["contract", "check", document, "--contract", contract, "--format", "json"]
if context:
command.extend(["--context", context])
check = runner.invoke(main, command)
assert check.exit_code == 0, check.output
check_data = json.loads(check.output)
assert check_data["valid"] is True
assert check_data["document_path"].endswith(Path(document).name)
if context:
form_state = runner.invoke(
main,
[
"contract",
"form-state",
document,
"--contract",
contract,
"--context",
context,
"--format",
"json",
],
)
assert form_state.exit_code == 0, form_state.output
form_data = json.loads(form_state.output)
assert form_data["valid"] is True
assert form_data["fields"], label
@pytest.mark.parametrize(
("label", "count"),
[
("small", 1),
("typical", 5),
("large", 30),
],
)
def test_transform_compose_and_include_usecases_small_typical_large(
tmp_path: Path,
label: str,
count: int,
):
docs = tmp_path / "sources"
docs.mkdir()
for index in range(count):
(docs / f"source-{index:02d}.md").write_text(
f"""---
review:
state: draft
---
# Source {index}
## Decision
The {label} transform pipeline keeps source {index} reusable.
""",
encoding="utf-8",
)
runner = CliRunner()
transform = runner.invoke(
main,
[
"transform",
str(docs / "source-00.md"),
"--set",
"review.state=ready",
"--heading-delta",
"1",
],
)
assert transform.exit_code == 0, transform.output
assert "state: ready" in transform.output
assert "## Source 0" in transform.output
compose = runner.invoke(
main,
[
"compose",
*[str(path) for path in sorted(docs.glob("*.md"))],
"--title",
f"{label.title()} Corpus",
"--heading-delta",
"1",
"--format",
"markdown",
],
)
assert compose.exit_code == 0, compose.output
assert f"# {label.title()} Corpus" in compose.output
assert compose.output.count("### Decision") == count
include_file = tmp_path / "include.md"
include_file.write_text(
'<!-- mkt:include path="sources/source-00.md" selector="sections[heading=Decision]" heading_delta="1" -->',
encoding="utf-8",
)
include = runner.invoke(main, ["include", str(include_file), "--base-dir", str(tmp_path)])
assert include.exit_code == 0, include.output
assert "### Decision" in include.output
assert f"The {label} transform pipeline" in include.output
@pytest.mark.parametrize(
"workflow_file",
[
"examples/workflows/source-snippets.workflow.md",
"examples/workflows/adr-release-notes.workflow.md",
],
)
def test_workflow_and_refresh_planning_usecases(tmp_path: Path, workflow_file: str):
runner = CliRunner()
inspect = runner.invoke(main, ["workflow", "inspect", workflow_file, "--format", "json"])
assert inspect.exit_code == 0, inspect.output
assert json.loads(inspect.output)["valid"] is True
plan = runner.invoke(
main,
["workflow", "plan", workflow_file, "--output-dir", str(tmp_path / "out"), "--format", "json"],
)
assert plan.exit_code == 0, plan.output
plan_data = json.loads(plan.output)
assert plan_data["valid"] is True
assert plan_data["dry_run"] is True
refresh = runner.invoke(
main,
["backend", "refresh-plan", "examples/documents", "--root", ".", "--format", "json"],
)
assert refresh.exit_code == 1, refresh.output
refresh_data = json.loads(refresh.output)
assert refresh_data["dirty"] is True
assert refresh_data["counts"]["needs_parse"] >= 1
def test_large_corpus_performance_smoke_stays_bounded(tmp_path: Path):
docs = tmp_path / "docs"
docs.mkdir()
for index in range(120):
(docs / f"note-{index:03d}.md").write_text(
f"""# Note {index}
## Summary
This synthetic performance smoke document mentions adoption latency and search.
## Details
The index should stay responsive for local corpora without requiring services.
""",
encoding="utf-8",
)
runner = CliRunner()
started = time.perf_counter()
index = runner.invoke(main, ["cache", "index", str(docs), "--root", str(tmp_path), "--format", "json"])
index_seconds = time.perf_counter() - started
assert index.exit_code == 0, index.output
assert len(json.loads(index.output)["indexed"]) == 120
assert index_seconds < 30
started = time.perf_counter()
search = runner.invoke(
main,
["search", "adoption", "--root", str(tmp_path), "--limit", "10", "--format", "json"],
)
search_seconds = time.perf_counter() - started
assert search.exit_code == 0, search.output
assert json.loads(search.output)["count"] >= 1
assert search_seconds < 5

View File

@@ -3,7 +3,7 @@ id: MKTT-WP-0017
type: workplan
title: "CLI/API Polish And Practical Adoption"
domain: markitect
status: active
status: done
owner: markitect-tool
topic_slug: markitect
planning_priority: P1
@@ -123,7 +123,7 @@ Output: `docs/practical-usecase-relevance.md` and links from intent/README.
```task
id: MKTT-WP-0017-T005
status: in_progress
status: done
priority: high
state_hub_task_id: "1159ec82-00e6-45cd-8a2c-e61ae03e758a"
```
@@ -149,22 +149,22 @@ Required coverage:
Output: deterministic tests and fixtures with no network dependency.
Initial coverage has been added for:
Coverage has been added for:
- small, typical, and large synthetic corpus parse/query/cache/search/context
package flow
- example-backed contract, template, function, workflow, reference, processor,
policy, literate tangle, and explode/implode flows
Remaining work: expand the matrix into explicit small/typical/large variants
for every high-relevance workflow and add performance assertions where they are
stable.
- small, typical, and large contract/form-state validation flows
- small, typical, and large transform/compose/include flows
- workflow inspect/plan and backend refresh-planning flows
- large local cache/index/search performance smoke
## P17.6 - Run coverage and performance iteration
```task
id: MKTT-WP-0017-T006
status: todo
status: done
priority: high
state_hub_task_id: "8dd5afe5-cefa-4388-bf4c-a975ba5c77c3"
```
@@ -175,11 +175,15 @@ or performance issues exposed by the E2E matrix.
Output: passing test suite, documented coverage gaps, and targeted performance
notes for large synthetic corpora.
Implementation adds `docs/performance-notes.md` and a portable smoke test with
generous bounds for local index build and indexed search. Full-suite validation
is run before closing the workplan.
## P17.7 - Improve first-use accessibility
```task
id: MKTT-WP-0017-T007
status: todo
status: done
priority: medium
state_hub_task_id: "590af805-0cc0-4585-b85f-9e95679afa77"
```
@@ -198,6 +202,14 @@ Possible improvements:
Output: README/docs/examples polish and tests where executable.
Implementation adds:
- `docs/getting-started.md`
- `docs/command-cheatsheet.md`
- `docs/examples-index.md`
- `docs/performance-notes.md`
- README and intent links to the practical docs
## Exit Criteria
- Every implemented subsystem has CLI access, public API access, docs, examples,