diff --git a/INTENT.md b/INTENT.md index d345eff..80be1c6 100644 --- a/INTENT.md +++ b/INTENT.md @@ -20,6 +20,10 @@ The repository provides a **set of composable primitives** that: It turns markdown from plain text into a **programmable knowledge substrate**. +The current practical usecase matrix is maintained in +`docs/practical-usecase-relevance.md`. That document translates this intent into +adoption-oriented scenarios, relevance expectations, and E2E coverage targets. + --- ## Intended Users @@ -107,4 +111,3 @@ A mature version of this repository should: Changes to this file represent a **deliberate shift in the role of this repository as a foundational toolkit**. Such changes should be rare, as they define the contract relied upon by all higher-level systems. - diff --git a/README.md b/README.md index e243007..5c18605 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,9 @@ requirements documents in `wiki/`. - `wiki/ProductRequirementsDocument.md` - product scope and success criteria - `wiki/FunctionalRequirementsSpecification.md` - observable functional behavior - `docs/markitect-main-scope-assessment.md` - migration assessment from `markitect-main` +- `docs/practical-usecase-relevance.md` - practical adoption usecases and relevance expectations +- `docs/cli-reference.md` - generated `mkt` command reference +- `docs/api-reference.md` - generated public API reference - `workplans/` - source-of-truth implementation and migration plans ## State Hub @@ -34,3 +37,11 @@ Try the parser CLI from a checkout: ```bash PYTHONPATH=src python3 -m markitect_tool parse README.md --format tree ``` + +Generate shell completion and reference docs: + +```bash +mkt completion bash > ~/.mkt-complete.bash +mkt docs cli --output docs/cli-reference.md +mkt docs api --output docs/api-reference.md +``` diff --git a/docs/api-reference.md b/docs/api-reference.md new file mode 100644 index 0000000..94e903d --- /dev/null +++ b/docs/api-reference.md @@ -0,0 +1,363 @@ +# Markitect API Reference + +Generated from `markitect_tool.__all__`. + +## `markitect_tool` + +- `BACKEND_CAPABILITIES` - object. set() -> new empty set object +- `DEFAULT_BACKEND_PATHS` - object. Built-in immutable sequence. +- `DEFAULT_LOCAL_INDEX_PATH` - object. str(object='') -> str +- `EMPTY_PARSE_OPTIONS_HASH` - object. str(object='') -> str +- `EXPLODE_MANIFEST_NAME` - object. str(object='') -> str +- `LOCAL_INDEX_SCHEMA_VERSION` - object. str(object='') -> str + +## `markitect_tool.backend.engine` + +- `BackendCapabilityCheck(backend_id: 'str', required: 'list[str]', supported: 'list[str]', missing: 'list[str]') -> None` - class. Compatibility result for a backend against required capabilities. +- `BackendManifest(id: 'str', kind: 'str' = 'cache-backend', name: 'str | None' = None, version: 'str' = '1', capabilities: 'list[str]' = , storage: 'dict[str, Any]' = , policy: 'dict[str, Any]' = , description: 'str | None' = None, manifest_path: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Declarative manifest for an optional backend. +- `BackendRegistry(manifests: 'list[BackendManifest] | None' = None) -> 'None'` - class. Read-only registry of backend manifests. +- `BackendRegistryError` - class. Raised when backend manifests or registry operations fail. +- `DependencyEdge(source_id: 'str', target: 'str', kind: 'str', target_snapshot_id: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Dependency edge between a snapshot/unit and another addressable target. +- `DocumentSnapshot(identity: 'SnapshotIdentity', document: 'dict[str, Any]', units: 'list[dict[str, Any]]' = , dependencies: 'list[DependencyEdge]' = , provenance: 'ProvenanceEnvelope | None' = None) -> None` - class. A parsed document snapshot that optional backends may persist. +- `ProvenanceEnvelope(operation: 'str', snapshot_id: 'str | None' = None, source_path: 'str | None' = None, content_hash: 'str | None' = None, dependencies: 'list[DependencyEdge]' = , backend_id: 'str | None' = None, policy_decision_id: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Shared provenance metadata for backend-derived objects. +- `SnapshotIdentity(source_path: 'str', content_hash: 'str', parser: 'str' = 'markdown-it-py/commonmark', parser_version: 'str' = 'markitect-tool:1', parse_options_hash: 'str' = 'sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a', contract_hash: 'str | None' = None) -> None` - class. Content-addressed identity for a parsed document snapshot. +- `capability_check(manifest: 'BackendManifest', required: 'list[str]') -> 'BackendCapabilityCheck'` - function. +- `load_backend_manifest(path: 'str | Path') -> 'BackendManifest'` - function. Load one backend manifest from YAML or Markdown fenced YAML. +- `load_backend_registry(paths: 'list[str | Path] | None' = None) -> 'BackendRegistry'` - function. Load backend manifests from files or directories without importing backends. +- `snapshot_identity_for_file(path: 'str | Path', *, parse_options: 'dict[str, Any] | None' = None, contract_hash: 'str | None' = None) -> 'SnapshotIdentity'` - function. Build a content-addressed snapshot identity for a file. + +## `markitect_tool.backend.interfaces` + +- `AccessPolicyGateway(*args, **kwargs)` - class. Authorization and filtering gateway for backend results. +- `ContextPackageRegistry(*args, **kwargs)` - class. Agent-ready context package registry. +- `IndexBackend(*args, **kwargs)` - class. Derived index backend for snapshots. +- `ProcessorResultStore(*args, **kwargs)` - class. Optional store for deterministic or assisted processor outputs. +- `QueryAdapter(*args, **kwargs)` - class. Adapter from stable Markitect requests to backend execution. +- `SnapshotBackend(*args, **kwargs)` - class. Durable parsed-document snapshot backend. + +## `markitect_tool.backend.local_store` + +- `LocalIndexBuildResult(index_path: 'str', root: 'str', paths: 'list[str]', planned: 'dict[str, Any]', parsed: 'list[str]' = , indexed: 'list[str]' = , metadata_updated: 'list[str]' = , deleted: 'list[str]' = ) -> None` - class. Summary of a local index build or refresh. +- `LocalSearchResult(path: 'str', snapshot_id: 'str', unit_kind: 'str', unit_index: 'int', heading: 'str | None', text: 'str', rank: 'float', line_start: 'int | None' = None, line_end: 'int | None' = None) -> None` - class. One FTS search match from the local index. +- `LocalSnapshotStore(path: 'str | Path' = '.markitect/cache/index.sqlite3') -> 'None'` - class. SQLite-backed local snapshot store for parsed Markdown documents. +- `local_index_path_for(root: 'str | Path', index_path: 'str | Path | None' = None) -> 'Path'` - function. Return the local SQLite index path for a root and optional override. + +## `markitect_tool.backend.planning` + +- `SnapshotPlanEntry(path: 'str', actions: 'list[str]', reason: 'str', size: 'int | None' = None, mtime_ns: 'int | None' = None, previous_snapshot_id: 'str | None' = None, content_hash: 'str | None' = None, invalidated_by: 'list[str]' = ) -> None` - class. One source-path decision in a refresh plan. +- `SnapshotRefreshPlan(root: 'str', parser: 'str', parser_version: 'str', parse_options_hash: 'str', contract_hash: 'str | None', verify_hashes: 'bool', entries: 'list[SnapshotPlanEntry]') -> None` - class. A cheap-first plan for refreshing snapshots and derived indexes. +- `SnapshotState(path: 'str', size: 'int', mtime_ns: 'int', content_hash: 'str', snapshot_id: 'str', parser: 'str' = 'markdown-it-py/commonmark', parser_version: 'str' = 'markitect-tool:1', parse_options_hash: 'str' = 'sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a', contract_hash: 'str | None' = None, indexed: 'bool' = True, dependencies: 'list[DependencyEdge]' = ) -> None` - class. Previously known source state from a snapshot/index backend. +- `load_snapshot_state_file(path: 'str | Path') -> 'list[SnapshotState]'` - function. Load a portable snapshot-state fixture from JSON or YAML. +- `plan_snapshot_refresh(paths: 'list[str | Path]', *, previous: 'list[SnapshotState] | dict[str, SnapshotState] | None' = None, root: 'str | Path' = '.', recursive: 'bool' = True, parse_options: 'dict[str, Any] | None' = None, contract_hash: 'str | None' = None, verify_hashes: 'bool' = False) -> 'SnapshotRefreshPlan'` - function. Plan snapshot/index refresh work using cheap metadata before hashing. + +## `markitect_tool.cache.engine` + +- `CacheEntry(path: 'str', content_hash: 'str', size: 'int', mtime_ns: 'int', parser: 'str' = 'markdown-it-py/commonmark', headings: 'int' = 0, sections: 'int' = 0, blocks: 'int' = 0) -> None` - class. One cached source-file fingerprint and parse summary. +- `CacheManifest(schema_version: 'str' = '1.0', root: 'str' = '.', entries: 'dict[str, CacheEntry]' = ) -> None` - class. A file-backed cache manifest. +- `CacheStatus(new: 'list[str]' = , changed: 'list[str]' = , unchanged: 'list[str]' = , deleted: 'list[str]' = ) -> None` - class. Change detection result against a cache manifest. +- `build_cache(paths: 'list[str | Path]', *, root: 'str | Path' = '.', recursive: 'bool' = True) -> 'CacheManifest'` - function. Build a cache manifest for Markdown files under paths. +- `cache_path_for(root: 'str | Path', cache_path: 'str | Path | None' = None) -> 'Path'` - function. Return the manifest path for a root and optional cache path. +- `detect_changes(manifest: 'CacheManifest', paths: 'list[str | Path]', *, root: 'str | Path' = '.', recursive: 'bool' = True) -> 'CacheStatus'` - function. Compare current Markdown files against a cache manifest. +- `fingerprint_file(path: 'str | Path', *, root: 'str | Path | None' = None) -> 'CacheEntry'` - function. Fingerprint one Markdown file and record a small parse summary. +- `load_cache(path: 'str | Path') -> 'CacheManifest'` - function. Load a cache manifest. Missing manifests produce an empty manifest. +- `save_cache(manifest: 'CacheManifest', path: 'str | Path') -> 'None'` - function. Write a cache manifest. +- `scan_markdown_files(paths: 'list[str | Path]', *, recursive: 'bool' = True) -> 'list[Path]'` - function. Return Markdown files for a set of files or directories. + +## `markitect_tool.content_class.engine` + +- `ClassCompositionResult(class_name: 'str', linearization: 'list[str]', slots: 'dict[str, Any]', diagnostics: 'list[Diagnostic]' = ) -> None` - class. Resolved content class slots plus diagnostics. +- `ContentClass(name: 'str', extends: 'list[str]' = , slots: 'dict[str, Any]' = , merge_policies: 'dict[str, str]' = ) -> None` - class. A data-defined content class. +- `ContentClassRegistry(classes: 'dict[str, ContentClass] | None' = None) -> 'None'` - class. Registry and resolver for content classes. +- `ContentClassResolutionError` - class. Raised when content class definitions cannot be loaded. +- `load_content_class_file(path: 'str | Path') -> 'ContentClassRegistry'` - function. Load content class definitions from YAML. +- `load_content_classes(data: 'dict[str, Any]') -> 'ContentClassRegistry'` - function. Load content class definitions from a mapping. + +## `markitect_tool.contract.checker` + +- `ContractCheckResult(valid: 'bool', diagnostics: 'list[Diagnostic]', document_path: 'str | None' = None, contract_path: 'str | None' = None, metrics: 'dict[str, Any]' = , runtime: 'dict[str, Any]' = ) -> None` - class. Check result for one document and one contract. +- `ContractValidationResult(valid: 'bool', diagnostics: 'list[Diagnostic]', contract_path: 'str | None' = None) -> None` - class. Validation result for a contract definition. +- `check_document_contract(document: 'Document', contract: 'DocumentContract', *, runtime_context: 'RuntimeContext | None' = None) -> 'ContractCheckResult'` - function. Check a parsed Markdown document against a document contract. +- `check_markdown_file(markdown_path: 'str | Path', contract_path: 'str | Path', *, context_path: 'str | Path | None' = None, runtime_context: 'RuntimeContext | None' = None) -> 'ContractCheckResult'` - function. Parse and check a Markdown file against a contract file. +- `validate_contract(contract: 'DocumentContract') -> 'ContractValidationResult'` - function. Validate the contract definition itself. +- `validate_contract_file(contract_path: 'str | Path') -> 'ContractValidationResult'` - function. Load and validate a Markdown contract file. + +## `markitect_tool.contract.loader` + +- `load_contract_file(path: 'str | Path') -> 'DocumentContract'` - function. Load a Markdown-native document contract file. + +## `markitect_tool.contract.metrics` + +- `collect_metrics(document: 'Document') -> 'DocumentMetrics'` - function. Collect document-level and section-level metrics. + +## `markitect_tool.contract.model` + +- `DocumentContract(id: 'str | None', document_type: 'str | None', title: 'str | None' = None, version: 'str | None' = None, description: 'str | None' = None, sections: 'list[SectionSpec]' = , fields: 'list[FieldSpec]' = , metrics: 'list[MetricBand]' = , assertions: 'list[AssertionSpec]' = , forms: 'list[dict[str, Any]]' = , context: 'dict[str, Any]' = , rules: 'list[dict[str, Any]]' = , rubrics: 'list[dict[str, Any]]' = , metadata: 'dict[str, Any]' = , raw: 'dict[str, Any]' = , source_path: 'str | None' = None, source_line: 'int | None' = None) -> None` - class. A contract for a typed Markdown document. + +## `markitect_tool.core.document` + +- `ContentBlock(type: 'str', text: 'str', line_start: 'int | None' = None, line_end: 'int | None' = None, heading_level: 'int | None' = None) -> None` - class. A top-level Markdown content block. +- `Document(source_path: 'str | None', frontmatter: 'dict[str, Any]', body: 'str', blocks: 'list[ContentBlock]', headings: 'list[Heading]', sections: 'list[Section]', tokens: 'list[dict[str, Any]]') -> None` - class. Structured representation of a Markdown document. +- `Heading(level: 'int', text: 'str', line: 'int') -> None` - class. A Markdown heading with source location. +- `Section(heading: 'Heading', blocks: 'list[ContentBlock]' = ) -> None` - class. A heading-led section. + +## `markitect_tool.core.parser` + +- `MarkdownParseError` - class. Raised when Markdown metadata cannot be parsed safely. +- `parse_markdown(markdown: 'str', source_path: 'str | None' = None) -> 'Document'` - function. Parse Markdown text into frontmatter, blocks, headings, sections, and tokens. +- `parse_markdown_file(path: 'str | Path') -> 'Document'` - function. Parse a Markdown file into a structured document. + +## `markitect_tool.diagnostics` + +- `Diagnostic(severity: 'str', code: 'str', message: 'str', source: 'SourceLocation | None' = None, contract: 'SourceLocation | None' = None, rule_id: 'str | None' = None, guidance: 'str | None' = None, details: 'dict[str, Any]' = ) -> None` - class. A structured validation or assessment finding. +- `ProcessingDiagnostic(severity: 'str', code: 'str', message: 'str', source: 'SourceLocation | None' = None, contract: 'SourceLocation | None' = None, rule_id: 'str | None' = None, guidance: 'str | None' = None, details: 'dict[str, Any]' = ) -> None` - class. A structured validation or assessment finding. +- `SourceLocation(path: 'str | None' = None, line: 'int | None' = None, column: 'int | None' = None) -> None` - class. A source location inside a document or contract. + +## `markitect_tool.document_function` + +- `DocumentFunctionCall(function_id: 'str', args: 'list[Any]' = , kwargs: 'dict[str, Any]' = , body: 'str | None' = None, raw: 'str' = '', inline: 'bool' = True, line: 'int | None' = None, pipeline: "list['DocumentFunctionCall']" = ) -> None` - class. Parsed document function call. +- `DocumentFunctionDescriptor(id: 'str', summary: 'str', parameters: 'list[DocumentFunctionParameter]' = , output_type: 'str' = 'markdown', execution: 'str' = 'deterministic', capabilities: 'list[ProcessingCapability]' = , safety: 'dict[str, Any]' = , examples: 'list[str]' = , metadata: 'dict[str, Any]' = , implementation: 'FunctionImplementation | None' = None) -> None` - class. Inspectable descriptor for a document function. +- `DocumentFunctionError` - class. Raised when document function parsing or evaluation fails. +- `DocumentFunctionEvaluationResult(content: 'str', calls: 'list[DocumentFunctionRun]' = , diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. Result of expanding document functions in a Markdown document. +- `DocumentFunctionParameter(name: 'str', kind: 'str' = 'string', required: 'bool' = True, default: 'Any' = None, variadic: 'bool' = False, description: 'str | None' = None) -> None` - class. One declared document function parameter. +- `DocumentFunctionRegistry(descriptors: 'list[DocumentFunctionDescriptor] | None' = None) -> 'None'` - class. Registry and evaluator for document functions. +- `DocumentFunctionRun(call: 'DocumentFunctionCall', output: 'Any' = None, diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. One function call result. +- `default_document_function_registry() -> 'DocumentFunctionRegistry'` - function. Return built-in deterministic document functions. +- `parse_document_function_calls(text: 'str') -> 'list[DocumentFunctionCall]'` - function. Parse inline and fenced document function calls. +- `render_document_functions(text: 'str', *, registry: 'DocumentFunctionRegistry | None' = None, context: 'ProcessingContext | None' = None) -> 'DocumentFunctionEvaluationResult'` - function. Expand deterministic document functions in Markdown content. +- `validate_document_functions(text: 'str', *, registry: 'DocumentFunctionRegistry | None' = None, allowed: 'list[str] | None' = None, forbidden: 'list[str] | None' = None) -> 'DocumentFunctionEvaluationResult'` - function. Validate function calls without rendering the document. + +## `markitect_tool.explode.engine` + +- `ExplodeEntry(kind: 'str', file: 'str', order: 'int', unit_id: 'str', line_start: 'int', line_end: 'int', heading_level: 'int | None' = None, heading_text: 'str | None' = None, content_hash: 'str' = '') -> None` - class. One file entry in an exploded Markdown directory. +- `ExplodeError` - class. Raised when explode or implode cannot preserve a safe roundtrip. +- `ExplodeManifest(version: 'int', source_path: 'str', source_hash: 'str', variant: 'str', frontmatter_raw: 'str' = '', entries: 'list[ExplodeEntry]' = ) -> None` - class. Manifest used to implode an exploded Markdown directory. +- `ExplodeResult(manifest_path: 'str', output_dir: 'str', manifest: 'ExplodeManifest', written_files: 'list[str]') -> None` - class. Result of exploding a Markdown file into a directory. +- `ImplodeResult(markdown: 'str', manifest_path: 'str', source_hash: 'str', current_hash: 'str', entries: 'list[str]') -> None` - class. Result of rebuilding Markdown from an explode manifest. +- `explode_markdown_file(path: 'str | Path', output_dir: 'str | Path', *, variant: 'str' = 'flat', overwrite: 'bool' = False) -> 'ExplodeResult'` - function. Explode a Markdown file into section files plus a roundtrip manifest. +- `implode_markdown_directory(directory: 'str | Path', *, manifest_path: 'str | Path | None' = None) -> 'ImplodeResult'` - function. Implode a Markdown directory created by :func:`explode_markdown_file`. +- `load_explode_manifest(path: 'str | Path') -> 'ExplodeManifest'` - function. Load an explode manifest from YAML. + +## `markitect_tool.extension` + +- `builtin_extension_registry()` - function. Return built-in extension descriptors without import-cycle pressure. + +## `markitect_tool.extension.execution` + +- `ExtensionExecutor(registry: 'ExtensionRegistry', *, lifecycle: 'ExtensionLifecycle | None' = None) -> 'None'` - class. Execute registered extensions with deterministic lifecycle callbacks. +- `ExtensionLifecycle(before: 'list[BeforeCallback]' = , after_success: 'list[AfterCallback]' = , after_failure: 'list[AfterCallback]' = , after: 'list[AfterCallback]' = ) -> None` - class. Explicit callbacks around extension execution. + +## `markitect_tool.extension.processing` + +- `ProcessingCapability(id: 'str', kind: 'str' = 'feature', required: 'bool' = True, description: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. A declared capability or permission needed by an extension. +- `ProcessingContext(root: 'Path' = PosixPath('.'), source_path: 'Path | None' = None, namespaces: 'dict[str, str]' = , variables: 'dict[str, Any]' = , policy: 'dict[str, Any]' = , backend_handles: 'dict[str, Any]' = , caller: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Shared execution context available to extension implementations. +- `ProcessingProvenance(operation: 'str', source_path: 'str | None' = None, snapshot_id: 'str | None' = None, content_hash: 'str | None' = None, dependencies: 'list[str]' = , backend_id: 'str | None' = None, provider_id: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Cross-extension provenance envelope. +- `ProcessingRequest(operation: 'str', input: 'Any', context: 'ProcessingContext' = , options: 'dict[str, Any]' = , scope: 'str | None' = None, capabilities: 'list[ProcessingCapability]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Canonical request passed to an internal extension. +- `ProcessingResult(output: 'Any' = None, diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , dependencies: 'list[str]' = , trace: 'list[ProcessingTrace]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Canonical result returned by an internal extension. +- `ProcessingTrace(event: 'str', message: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. One optional execution trace event. + +## `markitect_tool.extension.registry` + +- `ExtensionDependencyCheck(extension_id: 'str', missing: 'list[str]' = , optional_missing: 'list[str]' = ) -> None` - class. Result of checking required extension dependencies. +- `ExtensionDescriptor(id: 'str', kind: 'str', version: 'str' = '1', summary: 'str | None' = None, factory: 'ExtensionFactory | None' = None, capabilities: 'list[ProcessingCapability]' = , optional_dependencies: 'list[OptionalDependency]' = , safety: 'dict[str, Any]' = , input_contract: 'str | None' = None, output_contract: 'str | None' = None, diagnostics_namespace: 'str | None' = None, provenance_prefix: 'str | None' = None, cli: 'dict[str, Any]' = , docs: 'list[str]' = , examples: 'list[str]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Inspectable descriptor for one internal extension. +- `ExtensionRegistry(descriptors: 'Iterable[ExtensionDescriptor] | None' = None) -> 'None'` - class. Registry of internal extension descriptors. +- `ExtensionRegistryError` - class. Raised when extension descriptors or registries are invalid. +- `OptionalDependency(name: 'str', package: 'str | None' = None, extra: 'str | None' = None, required: 'bool' = False, purpose: 'str | None' = None) -> None` - class. An optional runtime dependency declared by an extension. + +## `markitect_tool.generation.engine` + +- `GeneratedDocument(markdown: 'str', output_path: 'str | None' = None, source_template: 'str | None' = None, data: 'dict[str, Any]' = , missing_variables: 'list[str]' = ) -> None` - class. One generated Markdown document. +- `GenerationHookRequest(prompt: 'str', data: 'dict[str, Any]' = , template: 'str | None' = None, contract_id: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Provider-neutral request for optional assisted generation. +- `GenerationHookResult(markdown: 'str', provider: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Provider-neutral response from an assisted generation hook. +- `GenerationPlan(documents: 'list[dict[str, Any]]', source_path: 'str | None' = None) -> None` - class. Markdown/YAML rule-based generation plan. +- `GenerationResult(documents: 'list[GeneratedDocument]', plan_path: 'str | None' = None) -> None` - class. Result of a deterministic generation run. +- `generate_stub_from_contract(contract: 'DocumentContract', *, data: 'dict[str, Any] | None' = None, include_optional: 'bool' = False) -> 'GeneratedDocument'` - function. Generate a Markdown stub from a document contract. +- `generate_with_hook(request: 'GenerationHookRequest', hook: 'GenerationHook') -> 'GenerationHookResult'` - function. Run optional assisted generation through an external hook. +- `load_generation_plan_file(path: 'str | Path') -> 'GenerationPlan'` - function. Load a generation plan from a Markdown file with a fenced YAML block. +- `run_generation_plan(plan: 'GenerationPlan', *, base_dir: 'str | Path | None' = None, output_dir: 'str | Path | None' = None, dry_run: 'bool' = False) -> 'GenerationResult'` - function. Render every document described by a generation plan. + +## `markitect_tool.literate.engine` + +- `CodeChunk(chunk_id: 'str', content: 'str', language: 'str | None' = None, target_path: 'str | None' = None, references: 'list[str]' = , source_path: 'str | None' = None, line_start: 'int | None' = None, line_end: 'int | None' = None, content_hash: 'str' = '') -> None` - class. A named fenced code chunk. +- `LiterateFile(path: 'str', content: 'str', chunk_ids: 'list[str]') -> None` - class. One generated file from tangling. +- `TangleResult(files: 'list[LiterateFile]', chunks: 'list[CodeChunk]', diagnostics: 'list[Diagnostic]' = , provenance: 'list[OperationProvenance]' = ) -> None` - class. Result of tangling Markdown code chunks. +- `WeaveResult(markdown: 'str', chunks: 'list[CodeChunk]') -> None` - class. Result of weaving Markdown documentation with a chunk index. +- `discover_code_chunks(markdown: 'str', *, source_path: 'str | Path | None' = None) -> 'list[CodeChunk]'` - function. Discover named fenced code chunks in Markdown order. +- `tangle_markdown(markdown: 'str', *, source_path: 'str | Path | None' = None) -> 'TangleResult'` - function. Tangle named chunks into target files. +- `weave_markdown(markdown: 'str', *, source_path: 'str | Path | None' = None) -> 'WeaveResult'` - function. Append a deterministic chunk index to human-readable Markdown. +- `write_tangle_files(result: 'TangleResult', output_dir: 'str | Path') -> 'list[str]'` - function. Write tangled files under an output directory. + +## `markitect_tool.memory.engine` + +- `ContextActivation(id: 'str', package_id: 'str', status: 'str', target: 'str', content: 'str', items: 'list[ContextPackageItem]', summaries: 'list[SummaryLayer]' = , token_estimate: 'int' = 0, created_at: 'str' = , policy: 'dict[str, Any]' = , policy_decisions: 'list[dict[str, Any]]' = , diagnostics: 'list[dict[str, Any]]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Result of activating or deactivating a context package. +- `ContextBudget(max_tokens: 'int | None' = None, max_items: 'int | None' = None, reserve_tokens: 'int' = 0, strategy: 'str' = 'first-fit') -> None` - class. Token and item budget applied while creating or activating context. +- `ContextPackage(id: 'str', title: 'str', intent: 'str', namespace: 'MemoryNamespace', items: 'list[ContextPackageItem]', retrieval_recipes: 'list[RetrievalRecipe]' = , summaries: 'list[SummaryLayer]' = , budget: 'ContextBudget' = , activation_state: 'str' = 'inactive', created_at: 'str' = , updated_at: 'str' = , freshness: 'dict[str, Any]' = , policy: 'dict[str, Any]' = , provenance: 'list[dict[str, Any]]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Portable, inspectable context package for agent working memory. +- `ContextPackageError` - class. Raised when a context package cannot be created, loaded, or activated. +- `ContextPackageItem(id: 'str', source: 'SourceSpan', text: 'str', summary: 'str', token_estimate: 'int', score: 'float | None' = None, policy: 'dict[str, Any]' = , provenance: 'dict[str, Any]' = , metadata: 'dict[str, Any]' = ) -> None` - class. One item included in an agent-ready context package. +- `LocalContextPackageRegistry(root: 'str | Path' = '.') -> 'None'` - class. Filesystem-backed local context package registry. +- `MemoryNamespace(project: 'str | None' = None, user: 'str | None' = None, agent: 'str | None' = None, thread: 'str | None' = None, task: 'str | None' = None, custom: 'dict[str, Any]' = ) -> None` - class. Stable namespace coordinates for agent memory packages. +- `MemorySourceSpan(path: 'str', snapshot_id: 'str | None' = None, unit_kind: 'str | None' = None, unit_index: 'int | None' = None, line_start: 'int | None' = None, line_end: 'int | None' = None, selector: 'str | None' = None, engine: 'str | None' = None) -> None` - class. Source range represented in a memory package. +- `RetrievalRecipe(kind: 'str', query: 'str', engine: 'str' = 'selector', sources: 'list[str]' = , paths: 'list[str]' = , root: 'str' = '.', index_path: 'str | None' = None, limit: 'int | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Recipe that can refresh memory package contents. +- `SummaryLayer(name: 'str', kind: 'str', text: 'str', token_estimate: 'int', generated_by: 'str' = 'markitect.deterministic', metadata: 'dict[str, Any]' = ) -> None` - class. One deterministic or assisted summary layer. +- `activate_context_package(package: 'ContextPackage', *, target: 'str' = 'default', policy_gateway: 'LocalLabelPolicyGateway | None' = None, subject: 'str' = 'anonymous', action: 'str' = 'read', budget: 'ContextBudget | None' = None) -> 'ContextActivation'` - function. Activate a package into a Markdown context bundle. +- `create_context_package_from_index(query: 'str', *, root: 'str | Path' = '.', index_path: 'str | Path | None' = None, engine: 'str' = 'selector', paths: 'list[str] | None' = None, search: 'bool' = False, limit: 'int' = 20, title: 'str | None' = None, intent: 'str | None' = None, namespace: 'MemoryNamespace | None' = None, budget: 'ContextBudget | None' = None, policy_gateway: 'LocalLabelPolicyGateway | None' = None, subject: 'str' = 'anonymous', action: 'str' = 'package', package_id: 'str | None' = None) -> 'ContextPackage'` - function. Create a context package from the local SQLite index. +- `create_context_package_from_manifest(manifest_path: 'str | Path', *, root: 'str | Path' = '.', budget: 'ContextBudget | None' = None, policy_gateway: 'LocalLabelPolicyGateway | None' = None, subject: 'str' = 'anonymous', action: 'str' = 'package') -> 'ContextPackage'` - function. Create a context package from a YAML/JSON manifest file. +- `create_context_package_from_sources(query: 'str', sources: 'list[str | Path]', *, root: 'str | Path' = '.', engine: 'str' = 'selector', title: 'str | None' = None, intent: 'str | None' = None, namespace: 'MemoryNamespace | None' = None, budget: 'ContextBudget | None' = None, policy_gateway: 'LocalLabelPolicyGateway | None' = None, subject: 'str' = 'anonymous', action: 'str' = 'package', package_id: 'str | None' = None) -> 'ContextPackage'` - function. Create a context package by querying Markdown source files directly. +- `deactivate_context_package(activation: 'ContextActivation') -> 'ContextActivation'` - function. Return a deactivated activation envelope. +- `explain_context_package(package: 'ContextPackage') -> 'dict[str, Any]'` - function. Return provenance, budget, policy, namespace, and retrieval details. +- `load_context_package_file(path: 'str | Path') -> 'ContextPackage'` - function. Load a context package from YAML or JSON. +- `refresh_context_package(package: 'ContextPackage', *, policy_gateway: 'LocalLabelPolicyGateway | None' = None, subject: 'str' = 'anonymous', action: 'str' = 'package') -> 'ContextPackage'` - function. Refresh a package by re-running its retrieval recipes. + +## `markitect_tool.ops.engine` + +- `ComposeResult(markdown: 'str', sources: 'list[str]' = ) -> None` - class. Result of composing multiple Markdown sources. +- `IncludeError` - class. Raised when include resolution cannot continue. +- `IncludeResult(markdown: 'str', included_paths: 'list[str]' = , provenance: 'list[OperationProvenance]' = ) -> None` - class. Result of resolving include markers in Markdown. +- `OperationProvenance(operation: 'str', source_path: 'str | None' = None, line_start: 'int | None' = None, line_end: 'int | None' = None, target_path: 'str | None' = None, dependencies: 'list[str]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Structured provenance for deterministic Markdown operations. +- `TransformResult(markdown: 'str', operations: 'list[str]' = , provenance: 'list[OperationProvenance]' = ) -> None` - class. Result of a deterministic Markdown transform. +- `compose_files(paths: 'list[str | Path]', *, title: 'str | None' = None, heading_delta: 'int' = 0, include_frontmatter: 'bool' = False, separator: 'str' = '\n\n---\n\n') -> 'ComposeResult'` - function. Compose Markdown files into one Markdown output. +- `resolve_includes(markdown: 'str', *, base_dir: 'str | Path', current_path: 'str | Path | None' = None, max_depth: 'int' = 10) -> 'IncludeResult'` - function. Resolve Markdown include markers recursively. +- `transform_markdown(markdown: 'str', *, strip_frontmatter: 'bool' = False, set_frontmatter: 'dict[str, Any] | None' = None, heading_delta: 'int' = 0, extract_selector: 'str | None' = None, source_path: 'str | None' = None) -> 'TransformResult'` - function. Apply deterministic operations to one Markdown document. + +## `markitect_tool.policy.adapters` + +- `DecisionLogStore(*args, **kwargs)` - class. Persistent audit boundary for policy decisions. +- `DirectoryGroupResolution(groups: 'list[str]' = , source: 'str | None' = None, refreshed_at: 'str | None' = None, overage: 'bool' = False, metadata: 'dict[str, Any]' = ) -> None` - class. Directory group resolution result with freshness and source provenance. +- `DirectoryGroupResolutionRequest(subject_id: 'str', issuer: 'str', groups: 'list[str]' = , claims: 'dict[str, Any]' = , context: 'dict[str, Any]' = ) -> None` - class. Request for resolving group claims that are stale, partial, or overlarge. +- `DirectoryGroupResolver(*args, **kwargs)` - class. Adapter boundary for SCIM/LDAP/Graph/Keycloak group resolution. +- `EnterpriseIdentity(issuer: 'str', subject: 'str', identity_scheme: 'str' = 'oidc', principal_type: 'str' = 'human', audience: 'list[str]' = , authorized_party: 'str | None' = None, preferred_username: 'str | None' = None, roles: 'list[str]' = , scopes: 'list[str]' = , groups: 'list[str]' = , assurance: 'dict[str, Any]' = , directory: 'dict[str, Any]' = , claims: 'dict[str, Any]' = , provenance: 'dict[str, Any]' = ) -> None` - class. Verified enterprise identity claims normalized for policy mapping. +- `EnterprisePolicyMapRequest(identity: 'EnterpriseIdentity | dict[str, Any]', policy_map: 'dict[str, Any]' = , groups: 'list[str]' = , context: 'dict[str, Any]' = ) -> None` - class. Request to map enterprise claims onto Markitect policy vocabulary. +- `EnterprisePolicyMapper(*args, **kwargs)` - class. Adapter boundary for mapping IAM roles/groups/scopes to policy subjects. +- `IdentityClaimsAdapter(*args, **kwargs)` - class. Adapter boundary for OIDC/JWT/SAML verification and normalization. +- `RelationshipPolicyAdapter(*args, **kwargs)` - class. Adapter boundary for relationship authorization systems. +- `RelationshipPolicyRequest(subject: 'str', relation: 'str', object_id: 'str', namespace: 'str | None' = None, context: 'dict[str, Any]' = ) -> None` - class. Relationship-based authorization request. +- `RulePolicyAdapter(*args, **kwargs)` - class. Adapter boundary for rule/attribute policy systems. +- `RulePolicyRequest(subject: 'dict[str, Any]', action: 'str', object: 'dict[str, Any]', context: 'dict[str, Any]' = , policy_id: 'str | None' = None) -> None` - class. Attribute/rule policy evaluation request. + +## `markitect_tool.policy.enterprise` + +- `EnterprisePolicyError` - class. Raised when enterprise identity or policy mapping is invalid. +- `EnterprisePolicyMap(id: 'str' = 'enterprise-policy-map', issuer: 'str | None' = None, audiences: 'list[str]' = , defaults: 'dict[str, Any]' = , groups: 'dict[str, dict[str, Any]]' = , roles: 'dict[str, dict[str, Any]]' = , scopes: 'dict[str, dict[str, Any]]' = , trust_zones: 'dict[str, dict[str, Any]]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Versioned Markitect-side mapping from IAM claims to policy subjects. +- `FlexAuthResource(id: 'str', type: 'str', path: 'str | None' = None, parent: 'str | None' = None, labels: 'list[str]' = , trust_zone: 'str | None' = None, owner: 'str | None' = None, attributes: 'dict[str, Any]' = ) -> None` - class. Resource record Markitect can register with flex-auth. +- `FlexAuthResourceManifest(id: 'str', system: 'str' = 'markitect-tool', resources: 'list[FlexAuthResource]' = , actions: 'list[str]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Manifest of Markitect resources intended for flex-auth registration. +- `LocalDecisionLogStore(path: 'Path | None' = None) -> None` - class. Local JSONL decision sink for development and tests. +- `LocalEnterprisePolicyMapper(policy_map: 'EnterprisePolicyMap') -> None` - class. Map verified enterprise identity into Markitect's PolicySubject shape. +- `NetKingdomIdentityClaimsAdapter(issuer: 'str | None' = None, audiences: 'list[str]' = , clock_skew_seconds: 'int' = 60, reject_local_issuers_in_production: 'bool' = True) -> None` - class. Validate NetKingdom/key-cape-compatible claims and normalize identity. +- `StaticDirectoryGroupResolver(groups_by_subject: 'dict[str, list[str]]' = , source: 'str' = 'static', refreshed_at: 'str | None' = None) -> None` - class. Deterministic group resolver for fixtures and local development. +- `load_enterprise_identity_file(path: 'str | Path', *, issuer: 'str | None' = None, audiences: 'list[str] | None' = None, environment: 'str | None' = None) -> 'EnterpriseIdentity'` - function. Load and validate deterministic NetKingdom/key-cape-style claims. +- `load_enterprise_policy_subject(claims_file: 'str | Path', policy_map_file: 'str | Path', *, extra_groups: 'list[str] | None' = None, environment: 'str | None' = None) -> 'PolicySubject'` - function. Load claims and policy map files and return a gateway-ready subject. + +## `markitect_tool.policy.local` + +- `LocalLabelPolicy(id: 'str' = 'local-label-policy', mode: 'str' = 'enforce', default_labels: 'list[str]' = , default_trust_zone: 'str | None' = None, default_subject: 'str' = 'anonymous', on_denied: 'str' = 'drop', subjects: 'dict[str, PolicySubject]' = , path_rules: 'list[LocalPathPolicyRule]' = , metadata: 'dict[str, Any]' = ) -> None` - class. Declarative local policy for labels, trust zones, and path ACLs. +- `LocalLabelPolicyGateway(policy: 'LocalLabelPolicy | dict[str, Any] | None' = None, *, mode: 'str | None' = None) -> 'None'` - class. AccessPolicyGateway implementation for local label policies. +- `LocalPathPolicyRule(pattern: 'str', labels: 'list[str]' = , trust_zone: 'str | None' = None, deny: 'bool' = False, id: 'str | None' = None) -> None` - class. Path rule that can add labels, set trust zone, or deny directly. +- `policy_metadata_from_document(document: 'dict[str, Any]', *, path: 'str | None' = None) -> 'dict[str, Any]'` - function. Extract stable policy metadata from parsed document frontmatter. + +## `markitect_tool.policy.models` + +- `PolicyDecision(subject: 'str', action: 'str', object_id: 'str', effect: 'str', reason: 'str', mode: 'str' = 'enforce', rule_id: 'str | None' = None, labels: 'list[str]' = , trust_zone: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. Explainable policy decision for one subject/action/object tuple. +- `PolicyFilterResult(results: 'list[dict[str, Any]]', decisions: 'list[PolicyDecision]' = , diagnostics: 'list[Any]' = , mode: 'str' = 'enforce', subject: 'str | None' = None, action: 'str | None' = None) -> None` - class. Results after policy filtering plus decision and diagnostic metadata. +- `PolicyObject(id: 'str', path: 'str | None' = None, labels: 'list[str]' = , trust_zone: 'str | None' = None, attributes: 'dict[str, Any]' = ) -> None` - class. Knowledge object considered by a policy decision. +- `PolicySubject(id: 'str', allowed_labels: 'list[str]' = , trust_zones: 'list[str]' = , roles: 'list[str]' = , allowed_actions: 'list[str]' = , path_allow: 'list[str]' = , path_deny: 'list[str]' = , attributes: 'dict[str, Any]' = ) -> None` - class. Actor asking to read, query, search, or package knowledge. + +## `markitect_tool.processor.engine` + +- `FencedProcessorBlock(processor: 'str', content: 'str', unit_id: 'str', attrs: 'dict[str, str]', language: 'str | None' = None, source_path: 'str | None' = None, line_start: 'int | None' = None, line_end: 'int | None' = None, content_hash: 'str' = '') -> None` - class. A fenced Markdown block that opted into processor handling. +- `ProcessorContext(root: 'Path' = PosixPath('.'), current_path: 'Path | None' = None, namespaces: 'dict[str, str]' = , variables: 'dict[str, Any]' = , policy: 'dict[str, Any]' = ) -> None` - class. Execution context passed to deterministic processors. +- `ProcessorOutputFile(path: 'str', content: 'str') -> None` - class. A generated file requested by a processor. +- `ProcessorRegistry() -> 'None'` - class. Explicit registry for deterministic fenced-block processors. +- `ProcessorRequest(block: 'FencedProcessorBlock', context: 'ProcessorContext') -> None` - class. One processor invocation. +- `ProcessorResult(content: 'str | None' = None, files: 'list[ProcessorOutputFile]' = , diagnostics: 'list[Diagnostic]' = , dependencies: 'list[str]' = , provenance: 'list[OperationProvenance]' = ) -> None` - class. Deterministic processor result envelope. +- `ProcessorRun(source_path: 'str | None', blocks: 'list[FencedProcessorBlock]', results: 'list[ProcessorResult]') -> None` - class. Results from running all processor blocks in a document. +- `default_processor_registry() -> 'ProcessorRegistry'` - function. Create the default deterministic processor registry. +- `discover_fenced_processors(markdown: 'str', *, source_path: 'str | Path | None' = None) -> 'list[FencedProcessorBlock]'` - function. Discover fenced blocks that explicitly opt into processor handling. +- `run_fenced_processors(markdown: 'str', *, context: 'ProcessorContext', registry: 'ProcessorRegistry | None' = None, source_path: 'str | Path | None' = None) -> 'ProcessorRun'` - function. Run all processor-marked fenced blocks in document order. + +## `markitect_tool.query.engine` + +- `InvalidQueryError` - class. Raised when a selector cannot be parsed or evaluated. +- `QueryMatch(kind: 'str', path: 'str', value: 'Any', text: 'str | None' = None, line: 'int | None' = None) -> None` - class. One match returned by a selector. +- `extract_document(document: 'Document', selector: 'str') -> 'list[str]'` - function. Extract text content from query matches. +- `extract_document_jsonpath(document: 'Document', expression: 'str') -> 'list[str]'` - function. Extract textual JSONPath matches from a parsed document. +- `extract_document_with_engine(document: 'Document', selector: 'str', *, engine: 'str' = 'selector') -> 'list[str]'` - function. Extract textual query matches through a registered query engine. +- `query_document(document: 'Document', selector: 'str') -> 'list[QueryMatch]'` - function. Query a parsed document with a small Markitect selector. +- `query_document_jsonpath(document: 'Document', expression: 'str') -> 'list[QueryMatch]'` - function. Query a parsed document with JSONPath over ``Document.to_dict()``. +- `query_document_with_engine(document: 'Document', selector: 'str', *, engine: 'str' = 'selector') -> 'list[QueryMatch]'` - function. Query a parsed document through a registered query engine. + +## `markitect_tool.query.registry` + +- `QueryEngine(descriptor: 'ExtensionDescriptor', query: 'QueryCallable') -> None` - class. Registered query engine implementation. +- `QueryEngineRegistry(engines: 'list[QueryEngine] | None' = None) -> 'None'` - class. Registry of query engines keyed by short engine id. +- `default_query_engine_registry() -> 'QueryEngineRegistry'` - function. Return the built-in query engine registry. + +## `markitect_tool.reference.engine` + +- `ContentUnit(kind: 'str', unit_id: 'str', text: 'str', source_path: 'str', span: 'SourceSpan | None' = None, name: 'str | None' = None, content_hash: 'str' = '', metadata: 'dict[str, Any]' = ) -> None` - class. One addressable content unit resolved from Markdown. +- `ReferenceAddress(raw: 'str', namespace: 'str | None' = None, address: 'str' = '', fragment: 'str | None' = None, selector: 'str | None' = None) -> None` - class. Parsed content reference address. +- `ReferenceContext(root: 'Path' = PosixPath('.'), current_path: 'Path | None' = None, namespaces: 'dict[str, str]' = ) -> None` - class. Inputs used to resolve namespaced and relative content references. +- `ReferenceResolution(reference: 'ReferenceAddress', source_path: 'str', target_path: 'str', units: 'list[ContentUnit]') -> None` - class. Resolved content reference and its dependency edge. +- `ReferenceResolutionError` - class. Raised when a content reference cannot be resolved. +- `ReferenceSourceSpan(line_start: 'int | None' = None, line_end: 'int | None' = None) -> None` - class. Line span for a resolved unit in its source file. +- `load_namespaces(frontmatter: 'dict[str, Any]') -> 'dict[str, str]'` - function. Load namespace mappings from Markdown frontmatter. +- `parse_reference(reference: 'str') -> 'ReferenceAddress'` - function. Parse a compact Markitect content reference. +- `resolve_reference(reference: 'str | ReferenceAddress', *, context: 'ReferenceContext') -> 'ReferenceResolution'` - function. Resolve a content reference to one or more content units. + +## `markitect_tool.runtime.assessment` + +- `AssessmentRequest(contract_id: 'str | None', rule_id: 'str', scope: 'str', text: 'str', criteria: 'Any', context: 'dict[str, Any]' = , structured_inputs: 'dict[str, Any]' = , severity: 'str' = 'error', threshold: 'float | None' = None, provider: 'str | None' = None, model: 'str | None' = None, metadata: 'dict[str, Any]' = ) -> None` - class. A provider-neutral request for rubric assessment. +- `AssessmentResult(rule_id: 'str', passed: 'bool', score: 'float | None' = None, reason: 'str | None' = None, diagnostics: 'list[Diagnostic]' = , provider: 'str | None' = None, model: 'str | None' = None, cache_key: 'str | None' = None, cached: 'bool' = False, metadata: 'dict[str, Any]' = ) -> None` - class. Normalized result returned by an assessment adapter. +- `AssessmentRunResult(assessments: 'list[AssessmentResult]' = , diagnostics: 'list[Diagnostic]' = ) -> None` - class. Result of executing one or more rubric assessments. +- `AssessmentRunner(adapter: 'AssessmentAdapter', *, cache: 'AssessmentCache | None' = None) -> 'None'` - class. Invoke an injected assessment adapter and normalize diagnostics. +- `MemoryAssessmentCache() -> 'None'` - class. Transparent in-memory cache useful for tests and short workflow runs. +- `assessment_requests_for_contract(document: 'Document', contract: 'DocumentContract', runtime_context: 'RuntimeContext | None' = None) -> 'list[AssessmentRequest]'` - function. Create assessment requests for contract rubric declarations. +- `run_contract_assessments(document: 'Document', contract: 'DocumentContract', adapter: 'AssessmentAdapter', *, runtime_context: 'RuntimeContext | None' = None, cache: 'AssessmentCache | None' = None) -> 'AssessmentRunResult'` - function. Run all rubrics in a contract with an injected assessment adapter. + +## `markitect_tool.runtime.context` + +- `RuntimeContext(data: 'dict[str, Any]' = , metadata: 'dict[str, Any]' = , source_path: 'str | None' = None, sources: 'list[RuntimeContextSource]' = , schemas: 'dict[str, Any]' = , diagnostics: 'list[Diagnostic]' = ) -> None` - class. Named external data available to contract checks and generation. +- `RuntimeContextLoadResult(context: 'RuntimeContext') -> None` - class. Context load result that can carry diagnostics instead of raising. +- `RuntimeContextSource(name: 'str | None' = None, path: 'str | None' = None, kind: 'str' = 'file', metadata: 'dict[str, Any]' = ) -> None` - class. Origin metadata for one loaded context object. +- `load_runtime_context_file(path: 'str | Path') -> 'RuntimeContext'` - function. Load a runtime context from JSON or YAML. +- `load_runtime_context_file_result(path: 'str | Path') -> 'RuntimeContextLoadResult'` - function. Load a runtime context and represent malformed input as diagnostics. + +## `markitect_tool.runtime.forms` + +- `FieldState(id: 'str', path: 'str | None' = None, source: 'str | None' = None, type: 'str | None' = None, label: 'str | None' = None, description: 'str | None' = None, value: 'Any' = None, exists: 'bool' = False, origin: 'str' = 'missing', required: 'bool' = False, visible: 'bool' = True, enabled: 'bool' = True, allowed_values: 'list[Any] | None' = None, diagnostics: 'list[Diagnostic]' = , metadata: 'dict[str, Any]' = ) -> None` - class. UI-neutral runtime state for one contract field. +- `FormState(contract_id: 'str | None', document_path: 'str | None' = None, context_path: 'str | None' = None, fields: 'list[FieldState]' = , diagnostics: 'list[Diagnostic]' = , rules_applied: 'list[str]' = , metadata: 'dict[str, Any]' = ) -> None` - class. A complete runtime form state derived from a document contract. +- `evaluate_form_state(document: 'Document', contract: 'DocumentContract', runtime_context: 'RuntimeContext | None' = None) -> 'FormState'` - function. Evaluate contract fields, prefill sources, and dynamic runtime rules. + +## `markitect_tool.schema.loader` + +- `MarkdownSchema(schema: 'dict[str, Any]', metadata: 'dict[str, Any]', documentation: 'str', source_path: 'str | None' = None) -> None` - class. A JSON Schema loaded from a Markdown schema document. +- `load_schema_file(path: 'str | Path') -> 'MarkdownSchema'` - function. Load a Markdown schema file. + +## `markitect_tool.schema.validator` + +- `SchemaValidationResult(valid: 'bool', violations: 'list[ValidationViolation]', document_path: 'str | None' = None, schema_path: 'str | None' = None) -> None` - class. Validation result for one document and one schema. +- `ValidationViolation(path: 'str', message: 'str', schema_path: 'str') -> None` - class. A single schema validation violation. +- `validate_document(document: 'Document', schema: 'MarkdownSchema | dict[str, Any]') -> 'SchemaValidationResult'` - function. Validate a parsed document against a loaded or raw JSON Schema. +- `validate_markdown_file(markdown_path: 'str | Path', schema_path: 'str | Path') -> 'SchemaValidationResult'` - function. Parse and validate a Markdown file against a Markdown schema file. +- `validate_schema(schema: 'dict[str, Any]') -> 'SchemaValidationResult'` - function. Validate that a JSON Schema itself is well formed. + +## `markitect_tool.template.engine` + +- `MissingTemplateVariable` - class. Raised when strict rendering cannot resolve a variable. +- `TemplateAnalysis(variables: 'list[str]', root_variables: 'list[str]', nested_variables: 'list[str]', syntax_errors: 'list[str]', max_nesting_depth: 'int' = 0) -> None` - class. Variables and syntax diagnostics for one template. +- `TemplateError` - class. Raised when a template cannot be parsed or rendered. +- `TemplateRenderResult(markdown: 'str', variables: 'list[str]', missing_variables: 'list[str]', strict: 'bool' = True) -> None` - class. Rendered Markdown plus trace information. +- `analyze_template(template_text: 'str') -> 'TemplateAnalysis'` - function. Analyze variable usage and syntax in a template. +- `render_template(template_text: 'str', data: 'dict[str, Any]', *, strict: 'bool' = True) -> 'TemplateRenderResult'` - function. Render ``{{variable.path}}`` placeholders with data. + +## `markitect_tool.workflow.engine` + +- `WorkflowError` - class. Raised when a workflow definition cannot be loaded or executed. +- `WorkflowOutputRecord(id: 'str', path: 'str | None', content: 'str', written: 'bool' = False, artifact: 'str | None' = None) -> None` - class. One output considered or written by a workflow run. +- `WorkflowPlan(metadata: 'dict[str, Any]' = , intent: 'dict[str, Any]' = , inputs: 'dict[str, dict[str, Any]]' = , steps: 'list[dict[str, Any]]' = , outputs: 'dict[str, dict[str, Any]]' = , dependencies: 'list[Any]' = , conditions: 'dict[str, Any]' = , artifacts: 'dict[str, Any]' = , permissions: 'dict[str, Any]' = , resources: 'dict[str, Any]' = , timeouts: 'dict[str, Any]' = , retry_policies: 'dict[str, Any]' = , escalation_rules: 'dict[str, Any]' = , observability: 'dict[str, Any]' = , responsibilities: 'dict[str, Any]' = , extensions: 'dict[str, Any]' = , source_path: 'str | None' = None) -> None` - class. Loaded declarative workflow definition. +- `WorkflowRunResult(workflow_id: 'str', plan_path: 'str | None', dry_run: 'bool', sources: 'dict[str, Any]' = , steps: 'dict[str, Any]' = , outputs: 'list[WorkflowOutputRecord]' = , diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. Result envelope for workflow inspect/plan/run operations. +- `WorkflowRunner(plan: 'WorkflowPlan', *, base_dir: 'str | Path | None' = None, output_dir: 'str | Path | None' = None, assisted_hook: 'GenerationHook | None' = None) -> 'None'` - class. Execute deterministic Markitect workflows. +- `load_workflow_file(path: 'str | Path') -> 'WorkflowPlan'` - function. Load a YAML or Markdown-fenced workflow definition. +- `resolve_workflow_bindings(value: 'Any', context: 'dict[str, Any]') -> 'Any'` - function. Resolve `${...}` expressions recursively. + +## `typing` + +- `ExtensionRunner(*args, **kwargs)` - object. diff --git a/docs/cli-reference.md b/docs/cli-reference.md new file mode 100644 index 0000000..deb8b2c --- /dev/null +++ b/docs/cli-reference.md @@ -0,0 +1,1011 @@ +# Markitect CLI Reference + +Generated from the Click command tree. + +## `mkt` + +Markdown-native toolkit for structured knowledge artifacts. + +```text +mkt [OPTIONS] COMMAND [ARGS]... +``` + +Parameters: + +- `--version` - Show the version and exit. + +## `mkt ast` + +Inspect parsed Markdown ASTs and parser summaries. + +```text +ast [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt ast show` + +Show a parsed Markdown AST without requiring a cache. + +```text +show [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--format` - + +## `mkt ast stats` + +Summarize parsed Markdown AST shape and token distribution. + +```text +stats [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--format` - + +## `mkt backend` + +Inspect optional backend manifests and snapshot identities. + +```text +backend [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt backend inspect` + +Inspect one backend manifest and optional compatibility check. + +```text +inspect [OPTIONS] BACKEND_ID +``` + +Parameters: + +- `BACKEND_ID` - Required. +- `--path` - Backend manifest file or directory. Defaults to .markitect/backends and .markitect/backend.yaml. +- `--require` - Required capability to check. May be repeated. +- `--format` - + +## `mkt backend list` + +List registered optional backend manifests. + +```text +list [OPTIONS] +``` + +Parameters: + +- `--path` - Backend manifest file or directory. Defaults to .markitect/backends and .markitect/backend.yaml. +- `--capability` - Only show backends that declare this capability. +- `--format` - + +## `mkt backend refresh-plan` + +Plan cheap-first snapshot and index refresh work. + +```text +refresh-plan [OPTIONS] PATHS... +``` + +Parameters: + +- `PATHS` - Required. +- `--root` - Root used for relative source paths. +- `--state` - YAML/JSON snapshot state file from a previous backend run. +- `--no-recursive` - Do not recurse into directories. +- `--verify-hashes` - Hash metadata-changed files to avoid unnecessary parse/index work. +- `--parse-option` - Parse option included in the identity comparison. +- `--contract-hash` - Optional contract hash included in identity comparison. +- `--format` - + +## `mkt backend snapshot-id` + +Compute a read-only content-addressed snapshot identity for a file. + +```text +snapshot-id [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--parse-option` - Parse option included in the snapshot identity hash. +- `--contract-hash` - Optional contract hash included in the snapshot identity. +- `--format` - + +## `mkt cache` + +Fingerprint Markdown files and detect changed inputs. + +```text +cache [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt cache build` + +Build or refresh a lightweight Markdown cache manifest. + +```text +build [OPTIONS] PATHS... +``` + +Parameters: + +- `PATHS` - Required. +- `--root` - Root used for relative cache paths. +- `--cache-path` - Cache manifest path. Defaults to .markitect/cache/manifest.json under root. +- `--no-recursive` - Do not recurse into directories. +- `--dry-run` - Report manifest without writing it. +- `--format` - + +## `mkt cache fingerprint` + +Fingerprint one Markdown file. + +```text +fingerprint [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--root` - Root used for relative cache paths. +- `--format` - + +## `mkt cache index` + +Build or refresh the local SQLite snapshot/index store. + +```text +index [OPTIONS] PATHS... +``` + +Parameters: + +- `PATHS` - Required. +- `--root` - Root used for relative index paths. +- `--index-path` - SQLite index path. Defaults to .markitect/cache/index.sqlite3 under root. +- `--no-recursive` - Do not recurse into directories. +- `--no-verify-hashes` - Do not hash metadata-changed files before parsing. +- `--parse-option` - Parse option included in the snapshot identity hash. +- `--contract-hash` - Optional contract hash included in snapshot identity. +- `--format` - + +## `mkt cache init` + +Initialize the local SQLite snapshot/index store. + +```text +init [OPTIONS] +``` + +Parameters: + +- `--root` - Root used for the default local index path. +- `--index-path` - SQLite index path. Defaults to .markitect/cache/index.sqlite3 under root. +- `--format` - + +## `mkt cache query` + +Run a selector or JSONPath query over indexed document snapshots. + +```text +query [OPTIONS] SELECTOR +``` + +Parameters: + +- `SELECTOR` - Required. +- `--root` - Root used for the default local index path. +- `--index-path` - SQLite index path. Defaults to .markitect/cache/index.sqlite3 under root. +- `--path` - Restrict query to one or more indexed relative paths. +- `--policy` - Local label policy file used to filter results. +- `--subject` - Policy subject id. +- `--policy-mode` - Override policy mode for this query. +- `--engine` - Query engine to use. +- `--format` - + +## `mkt cache status` + +Report changed, new, unchanged, and deleted Markdown files. + +```text +status [OPTIONS] PATHS... +``` + +Parameters: + +- `PATHS` - Required. +- `--root` - Root used for relative cache paths. +- `--cache-path` - Cache manifest path. Defaults to .markitect/cache/manifest.json under root. +- `--no-recursive` - Do not recurse into directories. +- `--format` - + +## `mkt class` + +Resolve deterministic content classes. + +```text +class [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt class resolve` + +Resolve content class inheritance and merged slots. + +```text +resolve [OPTIONS] CLASS_FILE CLASS_NAME +``` + +Parameters: + +- `CLASS_FILE` - Required. +- `CLASS_NAME` - Required. +- `--format` - + +## `mkt completion` + +Generate shell completion for Bash, Zsh, or Fish. + +```text +completion [OPTIONS] {bash|zsh|fish} +``` + +Parameters: + +- `SHELL` - Required. +- `--program-name` - Installed executable name used by the shell completion script. +- `--instructions` - Print installation instructions instead of the completion script. + +## `mkt compose` + +Compose multiple Markdown files into one document. + +```text +compose [OPTIONS] FILES... +``` + +Parameters: + +- `FILES` - Required. +- `--title` - Add a top-level title before composed files. +- `--heading-delta` - Shift heading levels in each input before composing. +- `--include-frontmatter` - Keep each input file's frontmatter in the composed body. +- `--output` - Write composed Markdown to a file. +- `--format` - + +## `mkt context` + +Pack and activate agent working-memory context. + +```text +context [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt context activate` + +Activate a saved context package as Markdown working memory. + +```text +activate [OPTIONS] PACKAGE +``` + +Parameters: + +- `PACKAGE` - Required. +- `--root` - Root used for the local context registry. +- `--target` - Thread/workspace activation target. +- `--max-items` - Maximum context items to activate. +- `--max-tokens` - Approximate maximum activation tokens. +- `--reserve-tokens` - Token reserve inside max tokens. +- `--policy` - Local label policy file used to re-check items before activation. +- `--subject` - Policy subject id. +- `--policy-mode` - Override policy mode while activating. +- `--action` - Policy action used while activating. +- `--format` - + +## `mkt context deactivate` + +Deactivate a recorded context activation. + +```text +deactivate [OPTIONS] ACTIVATION_ID +``` + +Parameters: + +- `ACTIVATION_ID` - Required. +- `--root` - Root used for the local context registry. +- `--format` - + +## `mkt context explain` + +Explain context package contents, retrieval, budget, and policy metadata. + +```text +explain [OPTIONS] PACKAGE +``` + +Parameters: + +- `PACKAGE` - Required. +- `--root` - Root used for the local context registry. +- `--format` - + +## `mkt context list` + +List locally saved context packages. + +```text +list [OPTIONS] +``` + +Parameters: + +- `--root` - Root used for the local context registry. +- `--format` - + +## `mkt context pack` + +Create an inspectable context package from a query, search, or manifest. + +```text +pack [OPTIONS] QUERY_OR_MANIFEST +``` + +Parameters: + +- `QUERY_OR_MANIFEST` - Required. +- `--source` - Markdown source file to query directly. May be repeated. +- `--root` - Root used for relative paths and the local context registry. +- `--index-path` - SQLite index path. Defaults to .markitect/cache/index.sqlite3 under root. +- `--search` - Treat the argument as an FTS search query. +- `--path` - Restrict indexed query/search to relative paths. +- `--engine` - Query engine for selector packages. +- `--limit` - Maximum FTS search matches. +- `--title` - Package title. +- `--intent` - Package intent statement. +- `--project` - Project namespace. +- `--user` - User namespace. +- `--agent` - Agent namespace. +- `--thread` - Thread namespace. +- `--task` - Task namespace. +- `--namespace` - Custom namespace key. +- `--max-items` - Maximum context items to include. +- `--max-tokens` - Approximate maximum package tokens. +- `--reserve-tokens` - Token reserve inside max tokens. +- `--policy` - Local label policy file used to filter package items. +- `--subject` - Policy subject id. +- `--policy-mode` - Override policy mode while packing. +- `--action` - Policy action used while packing. +- `--output` - Write package to this file. +- `--no-save` - Do not save to the local context registry. +- `--format` - + +## `mkt context refresh` + +Refresh a package by re-running its retrieval recipes. + +```text +refresh [OPTIONS] PACKAGE +``` + +Parameters: + +- `PACKAGE` - Required. +- `--root` - Root used for the local context registry. +- `--policy` - Local label policy file used to filter refreshed package items. +- `--subject` - Policy subject id. +- `--policy-mode` - Override policy mode while refreshing. +- `--action` - Policy action used while refreshing. +- `--no-save` - Do not save the refreshed package. +- `--format` - + +## `mkt contract` + +Work with Markdown document contracts. + +```text +contract [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt contract check` + +Check a Markdown file against a Markdown document contract. + +```text +check [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--contract` - Required. +- `--format` - +- `--context` - YAML or JSON runtime context used for field prefill and dynamic rules. + +## `mkt contract form-state` + +Evaluate UI-neutral form state for a document contract. + +```text +form-state [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--contract` - Required. +- `--context` - YAML or JSON runtime context used for field prefill and dynamic rules. +- `--format` - + +## `mkt contract validate` + +Validate that a Markdown contract file is well formed. + +```text +validate [OPTIONS] CONTRACT_FILE +``` + +Parameters: + +- `CONTRACT_FILE` - Required. +- `--format` - + +## `mkt docs` + +Generate CLI and API reference documentation. + +```text +docs [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt docs api` + +Generate a compact Markdown API reference from public exports. + +```text +api [OPTIONS] +``` + +Parameters: + +- `--output` - Write generated Markdown reference to this file. + +## `mkt docs cli` + +Generate a Markdown command reference from the Click command tree. + +```text +cli [OPTIONS] +``` + +Parameters: + +- `--output` - Write generated Markdown reference to this file. + +## `mkt explode` + +Explode a Markdown file into reversible section files. + +```text +explode [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--output-dir` - Directory to write exploded Markdown files and manifest into. Required. +- `--variant` - +- `--force` - Allow writing into a non-empty output directory. +- `--format` - + +## `mkt extension` + +Inspect built-in extension descriptors and CLI affordances. + +```text +extension [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt extension commands` + +List CLI commands declared by built-in extension descriptors. + +```text +commands [OPTIONS] +``` + +Parameters: + +- `--format` - + +## `mkt extension inspect` + +Inspect one built-in extension descriptor. + +```text +inspect [OPTIONS] EXTENSION_ID +``` + +Parameters: + +- `EXTENSION_ID` - Required. +- `--format` - + +## `mkt extension list` + +List built-in extension descriptors. + +```text +list [OPTIONS] +``` + +Parameters: + +- `--kind` - Only show extensions of this kind. +- `--format` - + +## `mkt extract` + +Extract text or Markdown content from structured Markdown. + +```text +extract [OPTIONS] FILE SELECTOR +``` + +Parameters: + +- `FILE` - Required. +- `SELECTOR` - Required. +- `--engine` - Query engine to use. +- `--format` - + +## `mkt function` + +Inspect and execute deterministic document functions. + +```text +function [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt function check` + +Validate document function calls without rendering. + +```text +check [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--allow` - Only allow this function id. May be repeated. +- `--forbid` - Forbid this function id. May be repeated. +- `--format` - + +## `mkt function list` + +List registered document functions. + +```text +list [OPTIONS] +``` + +Parameters: + +- `--namespace` - Only list functions in one namespace. +- `--format` - + +## `mkt function render` + +Render deterministic document function calls in a Markdown file. + +```text +render [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--format` - + +## `mkt generate` + +Generate Markdown from contracts, rules, or external hooks. + +```text +generate [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt generate rules` + +Run a Markdown/YAML generation plan. + +```text +rules [OPTIONS] RULES_FILE +``` + +Parameters: + +- `RULES_FILE` - Required. +- `--output-dir` - Directory used for relative output paths in the plan. +- `--dry-run` - Render without writing output files. +- `--format` - + +## `mkt generate stub` + +Generate a Markdown stub from a document contract. + +```text +stub [OPTIONS] +``` + +Parameters: + +- `--contract` - Markdown document contract to generate from. Required. +- `--data` - Optional JSON/YAML data for frontmatter values. +- `--set` - Set generation data. Dot paths create nested mappings. +- `--include-optional` - Include optional contract sections. +- `--output` - Write generated Markdown to a file. +- `--format` - + +## `mkt implode` + +Implode a Markdown directory created by `mkt explode`. + +```text +implode [OPTIONS] DIRECTORY +``` + +Parameters: + +- `DIRECTORY` - Required. +- `--manifest` - Manifest path. Defaults to markitect-explode.yaml in the input directory. +- `--output` - Write imploded Markdown to a file. +- `--format` - + +## `mkt include` + +Resolve Markdown include markers in a document. + +```text +include [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--base-dir` - Directory includes must stay within. Defaults to the input file directory. +- `--max-depth` - Maximum recursive include depth. +- `--output` - Write resolved Markdown to a file. +- `--format` - + +## `mkt metrics` + +Report practical size and complexity metrics for a Markdown file. + +```text +metrics [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--format` - + +## `mkt parse` + +Parse a Markdown file into a structured representation. + +```text +parse [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--format` - + +## `mkt policy` + +Check local access policy decisions. + +```text +policy [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt policy check` + +Authorize one subject/action/object tuple with local label policy. + +```text +check [OPTIONS] SUBJECT ACTION OBJECT_ID +``` + +Parameters: + +- `SUBJECT` - Required. +- `ACTION` - Required. +- `OBJECT_ID` - Required. +- `--policy` - Local label policy file. +- `--label` - Object policy label. May be repeated. +- `--path` - Object path for path ACL and path-label rules. +- `--trust-zone` - Object trust zone. +- `--policy-mode` - Override policy mode for this check. +- `--format` - + +## `mkt policy resource-manifest` + +Inspect a Markitect flex-auth resource registration manifest. + +```text +resource-manifest [OPTIONS] MANIFEST_FILE +``` + +Parameters: + +- `MANIFEST_FILE` - Required. +- `--format` - + +## `mkt policy subject` + +Map enterprise identity claims into a Markitect policy subject. + +```text +subject [OPTIONS] CLAIMS_FILE +``` + +Parameters: + +- `CLAIMS_FILE` - Required. +- `--policy-map` - Enterprise policy map file. Required. +- `--group` - Additional resolved group. May be repeated. +- `--environment` - Validation environment for issuer safety checks. +- `--format` - + +## `mkt process` + +Run deterministic fenced-block processors in a Markdown file. + +```text +process [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--root` - Root used for relative processor references. +- `--format` - + +## `mkt query` + +Query structured Markdown content with a small selector. + +```text +query [OPTIONS] FILE SELECTOR +``` + +Parameters: + +- `FILE` - Required. +- `SELECTOR` - Required. +- `--engine` - Query engine to use. +- `--format` - + +## `mkt ref` + +Resolve namespaced Markdown content references. + +```text +ref [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt ref resolve` + +Resolve a content reference using a Markdown document as context. + +```text +resolve [OPTIONS] CONTEXT_FILE REFERENCE +``` + +Parameters: + +- `CONTEXT_FILE` - Required. +- `REFERENCE` - Required. +- `--root` - Root that relative paths and namespaces must stay within. +- `--format` - + +## `mkt schema` + +Work with Markdown schema files. + +```text +schema [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt schema validate` + +Validate that a Markdown schema contains a well-formed JSON Schema. + +```text +validate [OPTIONS] SCHEMA_FILE +``` + +Parameters: + +- `SCHEMA_FILE` - Required. +- `--format` - + +## `mkt search` + +Search the local SQLite index with FTS5. + +```text +search [OPTIONS] TEXT +``` + +Parameters: + +- `TEXT` - Required. +- `--root` - Root used for the default local index path. +- `--index-path` - SQLite index path. Defaults to .markitect/cache/index.sqlite3 under root. +- `--limit` - +- `--policy` - Local label policy file used to filter results. +- `--subject` - Policy subject id. +- `--policy-mode` - Override policy mode for this search. +- `--format` - + +## `mkt tangle` + +Tangle named Markdown code chunks into target files. + +```text +tangle [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--output-dir` - Write tangled files under this directory. Omit for dry JSON/YAML/text output. +- `--format` - + +## `mkt template` + +Render and inspect deterministic Markdown templates. + +```text +template [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt template inspect` + +Inspect variables required by a template. + +```text +inspect [OPTIONS] TEMPLATE_FILE +``` + +Parameters: + +- `TEMPLATE_FILE` - Required. +- `--format` - + +## `mkt template render` + +Render a Markdown template with structured data. + +```text +render [OPTIONS] TEMPLATE_FILE +``` + +Parameters: + +- `TEMPLATE_FILE` - Required. +- `--data` - JSON, YAML, or CSV data file. CSV must contain one record for render. +- `--set` - Set a template data value. Dot paths create nested mappings. +- `--lenient` - Keep unresolved placeholders instead of failing. +- `--output` - Write rendered Markdown to a file. +- `--format` - + +## `mkt transform` + +Apply deterministic transforms to a Markdown file. + +```text +transform [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--strip-frontmatter` - Remove YAML frontmatter. +- `--set` - Set a frontmatter value. Dot paths create nested mappings. +- `--heading-delta` - Shift ATX heading levels, clamped to 1..6. +- `--extract` - Replace content with selector output. +- `--output` - Write transformed Markdown to a file. +- `--format` - + +## `mkt validate` + +Validate a Markdown file against a Markdown schema file. + +```text +validate [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--schema` - Required. +- `--format` - + +## `mkt weave` + +Weave Markdown documentation with a deterministic chunk index. + +```text +weave [OPTIONS] FILE +``` + +Parameters: + +- `FILE` - Required. +- `--output` - Write woven Markdown to a file. +- `--format` - + +## `mkt workflow` + +Inspect, plan, and run declarative Markdown workflows. + +```text +workflow [OPTIONS] COMMAND [ARGS]... +``` + +## `mkt workflow inspect` + +Inspect a workflow definition without executing steps. + +```text +inspect [OPTIONS] WORKFLOW_FILE +``` + +Parameters: + +- `WORKFLOW_FILE` - Required. +- `--format` - + +## `mkt workflow plan` + +Run a workflow in dry-run mode and report planned outputs. + +```text +plan [OPTIONS] WORKFLOW_FILE +``` + +Parameters: + +- `WORKFLOW_FILE` - Required. +- `--output-dir` - Output root for path-safety checks. No files are written in plan mode. +- `--format` - + +## `mkt workflow run` + +Run a deterministic Markdown workflow. + +```text +run [OPTIONS] WORKFLOW_FILE +``` + +Parameters: + +- `WORKFLOW_FILE` - Required. +- `--output-dir` - Output root for workflow outputs. Defaults to the workflow directory. +- `--dry-run` - Execute without writing output files. +- `--format` - diff --git a/docs/internal-extension-framework.md b/docs/internal-extension-framework.md index 58f7061..05bd537 100644 --- a/docs/internal-extension-framework.md +++ b/docs/internal-extension-framework.md @@ -187,6 +187,18 @@ Extension registries are optimized for common lookup patterns: Kinds and capabilities are indexed at registration time, so large registries can avoid repeated full scans for basic discovery. +The same registry is exposed on the CLI for practical discovery: + +```bash +mkt extension list +mkt extension inspect memory.context-package +mkt extension commands +``` + +This makes the extension catalog part of the user-visible contract. When a +capability gains a CLI command, the descriptor should declare that command so +generated docs and audits can compare intent with the live surface. + ### Execution Lifecycle `ExtensionExecutor` wraps a descriptor factory with deterministic lifecycle diff --git a/docs/practical-usecase-relevance.md b/docs/practical-usecase-relevance.md new file mode 100644 index 0000000..a3ba9cb --- /dev/null +++ b/docs/practical-usecase-relevance.md @@ -0,0 +1,88 @@ +# Practical Usecase Relevance + +This document translates the project intent, PRD, FRS, and current +implementation into practical adoption usecases for `markitect-tool`. + +The relevance expectations are based on the local product scope plus a light web +research pass across Markdown documentation ecosystems, docs-as-code practice, +RAG/agent context tooling, policy-as-code systems, and Markdown publishing +systems. + +## Research Signals + +- CommonMark exists because Markdown needed a strongly defined, compatible + specification across many implementations: https://commonmark.org/ +- GitLab documents large handbook and product-doc workflows in Markdown, with + linting/style enforcement in the pipeline: + https://handbook.gitlab.com/docs/markdown-guide/ and + https://docs.gitlab.com/development/documentation/styleguide/ +- MkDocs positions itself directly as project documentation with Markdown and + uses Markdown files plus YAML configuration: + https://www.mkdocs.org/ +- Docusaurus uses Markdown/MDX as its main authoring format and highlights the + tension between CommonMark compatibility and richer MDX component behavior: + https://docusaurus.io/docs/3.2.1/markdown-features +- LangChain and Semantic Kernel both model RAG around loading, splitting, + storing, retrieving, and injecting text context: + https://docs.langchain.com/oss/python/langchain/rag and + https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-rag +- OPA and OpenFGA show that policy and authorization are commonly externalized + into dedicated engines, which supports Markitect's adapter-first boundary: + https://www.openpolicyagent.org/docs and https://openfga.dev/docs/fga +- Quarkdown demonstrates current interest in Markdown extended with functions, + scripting, and richer publishing semantics: + https://www.quarkdown.net/about +- Click already supports shell completion for Bash, Zsh, and Fish when exposed + through an installed entry point: + https://click.palletsprojects.com/en/stable/shell-completion/ +- pdoc and Sphinx show common expectations for automatic Python API + documentation from public APIs and docstrings: + https://pdoc.dev/ and + https://www.sphinx-doc.org/en/master/usage/extensions/apidoc.html + +## Usecase Matrix + +| Usecase | Relevance | Why It Matters | Current Support | E2E Variations | +| --- | --- | --- | --- | --- | +| Parse Markdown into structured ASTs | Foundational | Every higher-level workflow needs stable structure and source spans. | `mkt parse`, `mkt ast`, API parser primitives | Small single note, typical ADR/PRD, large multi-section document | +| Contract-check business documents | High | Teams need predictable structure, assertions, field validation, and diagnostics. | `mkt contract validate/check/form-state`, contract APIs | Small concept note, typical ADR/business letter, large PRD/FRS bundle | +| Query and extract reusable content | High | Docs-as-code and agent/RAG workflows need selectors over source Markdown. | `mkt query`, `mkt extract`, JSONPath, indexed query | Small file query, typical docs directory, large cached corpus | +| Transform, compose, include, transclude | High | Real document pipelines need deterministic reuse without manual copy/paste. | `mkt transform`, `compose`, `include`, references, processors | Small include, typical multi-source review, large composed handbook segment | +| Generate documents from templates/contracts | High | Practical adoption depends on producing useful Markdown, not only checking it. | `mkt template`, `mkt generate`, document functions | Small template, typical release notes, large generated document pack | +| Declarative Markdown workflows | High | Business usecases need repeatable pipelines with intent, inputs, outputs, and diagnostics. | `mkt workflow inspect/plan/run` | Small source-snippet workflow, typical ADR release notes, large multi-input pipeline | +| Local indexing, search, and incremental refresh | High | Large corpora require speed, change detection, and retrieval without reparsing everything. | `mkt cache`, `mkt backend refresh-plan`, `mkt search` | Small two-file index, typical docs tree, large synthetic corpus | +| Policy-aware knowledge access | Medium-high | Enterprise use needs safe filtering and explainable denial without making Markitect the IAM system. | `mkt policy`, policy-filtered `search`, policy-filtered `cache query` | Public/private notes, typical trust-zone docs, large labelled corpus | +| Literate weave/tangle workflows | Medium-high | Knuth-style authoring and modern docs-as-code both benefit from code/document roundtrips. | `mkt tangle`, `mkt weave` | Small code chunk, typical app skeleton, large multi-file tangle | +| Explode/implode document sections | Medium-high | Human editing and review often need splitting documents while preserving roundtrip integrity. | `mkt explode`, `mkt implode` | Small flat split, typical hierarchical doc, large section tree | +| Runtime forms and dynamic context prefill | Medium-high | Structured documents often originate as form-like workflows with validation and prefills. | `mkt contract form-state`, runtime context APIs | Small static form, typical prefilled letter, large dynamic contract | +| Agent working-memory context packages | Medium-high | Agents need bounded, inspectable context payloads with provenance and policy rechecks. | `mkt context pack/activate/explain/refresh/list` | Small source package, typical indexed package, large budget-limited package | +| Document function layer | Medium | Quarkdown-like function layers make Markdown more expressive while keeping deterministic boundaries. | `mkt function list/check/render` | Small inline function, typical function fence, large generated function document | +| Internal extension framework | Medium | Future growth requires extension isolation, descriptors, docs, safety, and CLI/API visibility. | `mkt extension list/inspect/commands`, extension APIs | Built-in registry, docs descriptor, future optional adapter descriptor | +| Generated CLI/API documentation | Medium | Accessibility improves when users can discover the live surface without reading code. | `mkt docs cli`, `mkt docs api`, generated docs files | Console output, committed Markdown, packaging docs build | + +## Coverage Expectations + +The adoption polish work should treat these usecases as the public contract: + +- Each usecase should have at least one API test and one CLI test. +- Each high-relevance usecase should have small, typical, and large E2E + fixtures. +- Large fixtures should be synthetic enough to stay fast but big enough to catch + obvious performance regressions. +- E2E tests should assert useful behavior, diagnostics, and output shape, not + only zero exit codes. +- Optional adapters should be tested through stable local fakes and must not + require network or live LLM calls. + +## Accessibility Expectations + +Markitect becomes more accessible when: + +- users can install one command and get `--help`, tab completion, and generated + command docs +- every architecture capability is discoverable through `mkt extension` +- API docs are generated from the public API and kept close to the source +- examples map directly to usecases and can be copied into local experiments +- diagnostics use stable codes, source locations, and concise remediation hints +- defaults support practical single-file work before asking users to configure a + full project diff --git a/docs/workplan-planning-map.md b/docs/workplan-planning-map.md index 509299c..d7c31db 100644 --- a/docs/workplan-planning-map.md +++ b/docs/workplan-planning-map.md @@ -41,6 +41,7 @@ and descriptions mirror the operational view. | `MKTT-WP-0014` | complete | done | `MKTT-WP-0009` | Markitect-side enterprise IAM access-control integration is complete: NetKingdom/key-cape-compatible identity claims, flex-auth resource/policy contract, directory group resolution fixtures, decision-log sink, workflow declarations, CLI commands, and external PDP request examples. | | `MKTT-WP-0012` | complete | done | `MKTT-WP-0004`, `MKTT-WP-0010`, `MKTT-WP-0011` | Document function layer is complete: deterministic Markdown-native function descriptors, registry, inline/fenced syntax, pipelines, context bindings, CLI, docs, examples, diagnostics, provenance, and extension descriptor. | | `MKTT-WP-0008` | complete | done | `MKTT-WP-0006`, `MKTT-WP-0007`, `MKTT-WP-0009` | Agent working-memory context cache is complete: context package schema, local registry, package creation from queries/search/manifests, deterministic summaries, namespaces, activation/deactivation/refresh/explain lifecycle, policy re-checks, CLI, docs, and examples. | +| `MKTT-WP-0017` | P1 | active | `MKTT-WP-0003`, `MKTT-WP-0013` | Current polish/adoption track: CLI/API completeness, shell completion, generated command/API docs, usecase relevance matrix, E2E fixture planning, accessibility, and test/performance iteration before more advanced feature work. | | `MKTT-WP-0015` | P2 | todo | `MKTT-WP-0010`, `MKTT-WP-0011`, `MKTT-WP-0012` | Future render and document-function extensions: typed values, richer syntax, document-local reusable functions, Quarkdown/export adapters, render-aware references, assets, and permission sandboxing. Defer unless publishing/export pressure becomes current. | | `MKTT-WP-0016` | P2 | todo | `MKTT-WP-0008`, `MKTT-WP-0007`, `MKTT-WP-0009`, `MKTT-WP-0013` | Follow-on agentic memory architecture: reasoning decision graphs, conversational paths, long-term knowledge graphs, memory service blueprints/profiles, graph-to-context-package compilation, and adapter boundaries. | @@ -117,6 +118,11 @@ as memory size, latency, retention, refresh, compaction, activation budgets, and policy reauthorization. It should compile graph neighborhoods into `ContextPackage` objects rather than replacing the WP-0008 package layer. +`MKTT-WP-0017` is the current practical-adoption polish track. It should run +before the remaining advanced extension work because users and agents need a +complete, documented, shell-friendly, test-backed public surface before the +tool grows further. + ## State Hub Mirror Native State Hub dependency edges should mirror the whole-workstream @@ -154,3 +160,5 @@ dependencies: - `MKTT-WP-0016 -> MKTT-WP-0007` - `MKTT-WP-0016 -> MKTT-WP-0009` - `MKTT-WP-0016 -> MKTT-WP-0013` +- `MKTT-WP-0017 -> MKTT-WP-0003` +- `MKTT-WP-0017 -> MKTT-WP-0013` diff --git a/pyproject.toml b/pyproject.toml index cb5e0c5..05f55d0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -26,6 +26,9 @@ query = [ tables = [ "tabulate>=0.9", ] +docs = [ + "pdoc>=16", +] llm = [ "llm-connect @ file:///home/worsch/llm-connect", ] diff --git a/src/markitect_tool/__init__.py b/src/markitect_tool/__init__.py index 5f031ac..d1c0f14 100644 --- a/src/markitect_tool/__init__.py +++ b/src/markitect_tool/__init__.py @@ -48,6 +48,8 @@ from markitect_tool.cache import ( from markitect_tool.backend import ( BACKEND_CAPABILITIES, DEFAULT_BACKEND_PATHS, + DEFAULT_LOCAL_INDEX_PATH, + LOCAL_INDEX_SCHEMA_VERSION, AccessPolicyGateway, BackendCapabilityCheck, BackendManifest, @@ -58,6 +60,9 @@ from markitect_tool.backend import ( DocumentSnapshot, EMPTY_PARSE_OPTIONS_HASH, IndexBackend, + LocalIndexBuildResult, + LocalSearchResult, + LocalSnapshotStore, ProcessorResultStore, ProvenanceEnvelope, QueryAdapter, @@ -70,6 +75,7 @@ from markitect_tool.backend import ( load_backend_manifest, load_backend_registry, load_snapshot_state_file, + local_index_path_for, plan_snapshot_refresh, snapshot_identity_for_file, ) @@ -82,6 +88,24 @@ from markitect_tool.content_class import ( load_content_classes, ) from markitect_tool.diagnostics import Diagnostic, SourceLocation +from markitect_tool.extension import ( + ExtensionDependencyCheck, + ExtensionDescriptor, + ExtensionExecutor, + ExtensionLifecycle, + ExtensionRegistry, + ExtensionRegistryError, + ExtensionRunner, + OptionalDependency, + ProcessingCapability, + ProcessingContext, + ProcessingDiagnostic, + ProcessingProvenance, + ProcessingRequest, + ProcessingResult, + ProcessingTrace, + builtin_extension_registry, +) from markitect_tool.explode import ( EXPLODE_MANIFEST_NAME, ExplodeEntry, @@ -157,9 +181,24 @@ from markitect_tool.processor import ( run_fenced_processors, ) from markitect_tool.policy import ( + DecisionLogStore, + DirectoryGroupResolution, + DirectoryGroupResolutionRequest, + DirectoryGroupResolver, + EnterpriseIdentity, + EnterprisePolicyError, + EnterprisePolicyMap, + EnterprisePolicyMapRequest, + EnterprisePolicyMapper, + FlexAuthResource, + FlexAuthResourceManifest, + IdentityClaimsAdapter, LocalLabelPolicy, LocalLabelPolicyGateway, LocalPathPolicyRule, + LocalDecisionLogStore, + LocalEnterprisePolicyMapper, + NetKingdomIdentityClaimsAdapter, PolicyDecision, PolicyFilterResult, PolicyObject, @@ -168,13 +207,23 @@ from markitect_tool.policy import ( RelationshipPolicyRequest, RulePolicyAdapter, RulePolicyRequest, + StaticDirectoryGroupResolver, + load_enterprise_identity_file, + load_enterprise_policy_subject, policy_metadata_from_document, ) from markitect_tool.query import ( InvalidQueryError, + QueryEngine, + QueryEngineRegistry, QueryMatch, + default_query_engine_registry, extract_document, + extract_document_jsonpath, + extract_document_with_engine, query_document, + query_document_jsonpath, + query_document_with_engine, ) from markitect_tool.reference import ( ContentUnit, @@ -211,6 +260,7 @@ from markitect_tool.schema import ( load_schema_file, validate_document, validate_markdown_file, + validate_schema, ) from markitect_tool.template import ( MissingTemplateVariable, @@ -244,6 +294,7 @@ __all__ = [ "load_schema_file", "validate_document", "validate_markdown_file", + "validate_schema", "ContractCheckResult", "ContractValidationResult", "DocumentContract", @@ -276,6 +327,8 @@ __all__ = [ "scan_markdown_files", "BACKEND_CAPABILITIES", "DEFAULT_BACKEND_PATHS", + "DEFAULT_LOCAL_INDEX_PATH", + "LOCAL_INDEX_SCHEMA_VERSION", "AccessPolicyGateway", "BackendCapabilityCheck", "BackendManifest", @@ -286,6 +339,9 @@ __all__ = [ "DocumentSnapshot", "EMPTY_PARSE_OPTIONS_HASH", "IndexBackend", + "LocalIndexBuildResult", + "LocalSearchResult", + "LocalSnapshotStore", "ProcessorResultStore", "ProvenanceEnvelope", "QueryAdapter", @@ -298,6 +354,7 @@ __all__ = [ "load_backend_manifest", "load_backend_registry", "load_snapshot_state_file", + "local_index_path_for", "plan_snapshot_refresh", "snapshot_identity_for_file", "ClassCompositionResult", @@ -308,6 +365,22 @@ __all__ = [ "load_content_classes", "Diagnostic", "SourceLocation", + "ExtensionDependencyCheck", + "ExtensionDescriptor", + "ExtensionExecutor", + "ExtensionLifecycle", + "ExtensionRegistry", + "ExtensionRegistryError", + "ExtensionRunner", + "OptionalDependency", + "ProcessingCapability", + "ProcessingContext", + "ProcessingDiagnostic", + "ProcessingProvenance", + "ProcessingRequest", + "ProcessingResult", + "ProcessingTrace", + "builtin_extension_registry", "EXPLODE_MANIFEST_NAME", "ExplodeEntry", "ExplodeError", @@ -370,9 +443,24 @@ __all__ = [ "default_processor_registry", "discover_fenced_processors", "run_fenced_processors", + "DecisionLogStore", + "DirectoryGroupResolution", + "DirectoryGroupResolutionRequest", + "DirectoryGroupResolver", + "EnterpriseIdentity", + "EnterprisePolicyError", + "EnterprisePolicyMap", + "EnterprisePolicyMapRequest", + "EnterprisePolicyMapper", + "FlexAuthResource", + "FlexAuthResourceManifest", + "IdentityClaimsAdapter", "LocalLabelPolicy", "LocalLabelPolicyGateway", "LocalPathPolicyRule", + "LocalDecisionLogStore", + "LocalEnterprisePolicyMapper", + "NetKingdomIdentityClaimsAdapter", "PolicyDecision", "PolicyFilterResult", "PolicyObject", @@ -381,11 +469,21 @@ __all__ = [ "RelationshipPolicyRequest", "RulePolicyAdapter", "RulePolicyRequest", + "StaticDirectoryGroupResolver", + "load_enterprise_identity_file", + "load_enterprise_policy_subject", "policy_metadata_from_document", "InvalidQueryError", + "QueryEngine", + "QueryEngineRegistry", "QueryMatch", + "default_query_engine_registry", "extract_document", + "extract_document_jsonpath", + "extract_document_with_engine", "query_document", + "query_document_jsonpath", + "query_document_with_engine", "ContentUnit", "ReferenceAddress", "ReferenceContext", diff --git a/src/markitect_tool/cli/main.py b/src/markitect_tool/cli/main.py index ab03050..a7abbca 100644 --- a/src/markitect_tool/cli/main.py +++ b/src/markitect_tool/cli/main.py @@ -3,10 +3,12 @@ from __future__ import annotations import json +import inspect from pathlib import Path import click import yaml +from click.shell_completion import get_completion_class from markitect_tool.cache import ( build_cache, @@ -43,7 +45,12 @@ from markitect_tool.document_function import ( render_document_functions, validate_document_functions, ) -from markitect_tool.extension import ProcessingContext +from markitect_tool.cli.extensions import collect_cli_command_specs +from markitect_tool.extension import ( + ExtensionRegistryError, + ProcessingContext, + builtin_extension_registry, +) from markitect_tool.explode import ( ExplodeError, explode_markdown_file, @@ -107,6 +114,118 @@ def main() -> None: """Markdown-native toolkit for structured knowledge artifacts.""" +@main.command("completion") +@click.argument("shell", type=click.Choice(["bash", "zsh", "fish"], case_sensitive=False)) +@click.option( + "--program-name", + default="mkt", + show_default=True, + help="Installed executable name used by the shell completion script.", +) +@click.option( + "--instructions", + is_flag=True, + help="Print installation instructions instead of the completion script.", +) +def completion(shell: str, program_name: str, instructions: bool) -> None: + """Generate shell completion for Bash, Zsh, or Fish.""" + + normalized_shell = shell.lower() + if instructions: + click.echo(_completion_instructions(normalized_shell, program_name)) + return + completion_class = get_completion_class(normalized_shell) + if completion_class is None: + raise click.ClickException(f"Unsupported completion shell `{shell}`") + complete = completion_class(main, {}, program_name, "_MKT_COMPLETE") + click.echo(complete.source()) + + +@main.group("extension") +def extension_group() -> None: + """Inspect built-in extension descriptors and CLI affordances.""" + + +@extension_group.command("list") +@click.option("--kind", help="Only show extensions of this kind.") +@click.option( + "--format", + "output_format", + type=click.Choice(["json", "yaml", "text"], case_sensitive=False), + default="text", + show_default=True, +) +def extension_list(kind: str | None, output_format: str) -> None: + """List built-in extension descriptors.""" + + registry = builtin_extension_registry() + extensions = [descriptor.to_dict() for descriptor in registry.list(kind=kind)] + _emit_extension_catalog({"count": len(extensions), "extensions": extensions}, output_format) + + +@extension_group.command("inspect") +@click.argument("extension_id") +@click.option( + "--format", + "output_format", + type=click.Choice(["json", "yaml", "text"], case_sensitive=False), + default="text", + show_default=True, +) +def extension_inspect(extension_id: str, output_format: str) -> None: + """Inspect one built-in extension descriptor.""" + + try: + descriptor = builtin_extension_registry().get(extension_id) + except ExtensionRegistryError as exc: + raise click.ClickException(str(exc)) from exc + _emit_extension_catalog({"extension": descriptor.to_dict()}, output_format) + + +@extension_group.command("commands") +@click.option( + "--format", + "output_format", + type=click.Choice(["json", "yaml", "text"], case_sensitive=False), + default="text", + show_default=True, +) +def extension_commands(output_format: str) -> None: + """List CLI commands declared by built-in extension descriptors.""" + + specs = [spec.to_dict() for spec in collect_cli_command_specs(builtin_extension_registry())] + _emit_extension_catalog({"count": len(specs), "commands": specs}, output_format) + + +@main.group("docs") +def docs_group() -> None: + """Generate CLI and API reference documentation.""" + + +@docs_group.command("cli") +@click.option( + "--output", + type=click.Path(dir_okay=False, path_type=Path), + help="Write generated Markdown reference to this file.", +) +def docs_cli(output: Path | None) -> None: + """Generate a Markdown command reference from the Click command tree.""" + + _emit_or_write_text(_generate_cli_reference(), output) + + +@docs_group.command("api") +@click.option( + "--output", + type=click.Path(dir_okay=False, path_type=Path), + help="Write generated Markdown reference to this file.", +) +def docs_api(output: Path | None) -> None: + """Generate a compact Markdown API reference from public exports.""" + + _emit_or_write_text(_generate_api_reference(), output) + + @main.command() @click.argument("file", type=click.Path(exists=True, dir_okay=False, path_type=Path)) @click.option( @@ -2743,6 +2862,36 @@ def _emit_tangle_result(data: dict, output_format: str) -> None: click.echo(f"written: {written}") +def _emit_extension_catalog(data: dict, output_format: str) -> None: + if output_format == "json": + click.echo(json.dumps(data, indent=2, ensure_ascii=False)) + elif output_format == "yaml": + click.echo(yaml.safe_dump(data, sort_keys=False)) + elif "extension" in data: + extension = data["extension"] + click.echo(f"{extension['id']} ({extension['kind']})") + if extension.get("summary"): + click.echo(extension["summary"]) + commands = extension.get("cli", {}).get("commands", []) + if commands: + click.echo("commands:") + for command in commands: + click.echo(f"- {command}") + docs = extension.get("docs", []) + if docs: + click.echo("docs:") + for doc in docs: + click.echo(f"- {doc}") + elif "commands" in data: + click.echo(f"command specs: {data['count']}") + for spec in data["commands"]: + click.echo(f"- {spec['command']} [{spec['extension_id']}]") + else: + click.echo(f"extensions: {data['count']}") + for extension in data["extensions"]: + click.echo(f"- {extension['id']} ({extension['kind']})") + + def _emit_jsonish(data: dict, output_format: str) -> None: if output_format == "yaml": click.echo(yaml.safe_dump(data, sort_keys=False)) @@ -2764,6 +2913,115 @@ def _emit_template_analysis(data: dict, output_format: str) -> None: click.echo(f"! {error}") +def _emit_or_write_text(text: str, output: Path | None) -> None: + if output: + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(text, encoding="utf-8") + else: + click.echo(text) + + +def _completion_instructions(shell: str, program_name: str) -> str: + command = f"_MKT_COMPLETE={shell}_source {program_name}" + if shell == "fish": + target = f"~/.config/fish/completions/{program_name}.fish" + return ( + f"Generate completion script:\n{command} > {target}\n\n" + f"Or load it for the current shell:\n{command} | source" + ) + target = f"~/.{program_name}-complete.{shell}" + rc_file = "~/.bashrc" if shell == "bash" else "~/.zshrc" + return ( + f"Generate completion script:\n{command} > {target}\n\n" + f"Then add this to {rc_file}:\n. {target}" + ) + + +def _generate_cli_reference() -> str: + lines = [ + "# Markitect CLI Reference", + "", + "Generated from the Click command tree.", + "", + ] + _append_command_reference(lines, main, "mkt") + return "\n".join(lines).rstrip() + "\n" + + +def _append_command_reference(lines: list[str], command: click.Command, path: str) -> None: + lines.extend([f"## `{path}`", ""]) + if command.help: + lines.extend([command.help, ""]) + with click.Context(command, info_name=path.split()[-1]) as ctx: + usage = command.get_usage(ctx).replace("Usage: ", "", 1) + lines.extend(["```text", usage, "```", ""]) + params = [ + param + for param in command.params + if not getattr(param, "hidden", False) and getattr(param, "name", None) + ] + if params: + lines.extend(["Parameters:", ""]) + for param in params: + if isinstance(param, click.Option): + names = ", ".join(param.opts + param.secondary_opts) + else: + names = param.human_readable_name + help_text = getattr(param, "help", None) or "" + if param.required: + help_text = (help_text + " Required.").strip() + lines.append(f"- `{names}` - {help_text}".rstrip()) + lines.append("") + if isinstance(command, click.Group): + for name, subcommand in sorted(command.commands.items()): + _append_command_reference(lines, subcommand, f"{path} {name}") + + +def _generate_api_reference() -> str: + import markitect_tool as api + + lines = [ + "# Markitect API Reference", + "", + "Generated from `markitect_tool.__all__`.", + "", + ] + grouped: dict[str, list[str]] = {} + for name in sorted(api.__all__): + obj = getattr(api, name, None) + module = getattr(obj, "__module__", "markitect_tool") + grouped.setdefault(module, []).append(name) + for module in sorted(grouped): + lines.extend([f"## `{module}`", ""]) + for name in grouped[module]: + obj = getattr(api, name, None) + lines.append(_api_reference_line(name, obj)) + lines.append("") + return "\n".join(lines).rstrip() + "\n" + + +def _api_reference_line(name: str, obj: object) -> str: + kind = "object" + if inspect.isclass(obj): + kind = "class" + elif inspect.isfunction(obj): + kind = "function" + elif inspect.isbuiltin(obj): + kind = "builtin" + signature = "" + if callable(obj): + try: + signature = str(inspect.signature(obj)) + except (TypeError, ValueError): + signature = "" + summary = "" + doc = inspect.getdoc(obj) + if doc: + summary = doc.splitlines()[0] + display = f"`{name}{signature}`" if signature else f"`{name}`" + return f"- {display} - {kind}. {summary}".rstrip() + + def _parse_key_value_options(items: tuple[str, ...]) -> dict[str, object]: values: dict[str, object] = {} for item in items: diff --git a/src/markitect_tool/extension/builtins.py b/src/markitect_tool/extension/builtins.py index fdc01e2..e71bc3d 100644 --- a/src/markitect_tool/extension/builtins.py +++ b/src/markitect_tool/extension/builtins.py @@ -12,6 +12,8 @@ def builtin_extension_registry() -> ExtensionRegistry: registry = default_query_engine_registry().extension_registry() for descriptor in _processor_descriptors() + [ + _extension_catalog_descriptor(), + _generated_docs_descriptor(), _local_sqlite_backend_descriptor(), _workflow_engine_descriptor(), _runtime_context_descriptor(), @@ -25,6 +27,51 @@ def builtin_extension_registry() -> ExtensionRegistry: return registry +def _extension_catalog_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="extension.catalog", + kind="extension-registry", + summary="Inspectable catalog of built-in extension descriptors and CLI affordances.", + capabilities=[ + ProcessingCapability(id="extensions", kind="inspect"), + ProcessingCapability(id="cli", kind="inspect"), + ], + safety={"reads_files": False, "writes_files": False, "network": False}, + input_contract="Built-in ExtensionRegistry", + output_contract="ExtensionDescriptor catalog | CliCommandSpec list", + diagnostics_namespace="extension", + provenance_prefix="extension.catalog", + cli={ + "commands": [ + "mkt extension list", + "mkt extension inspect", + "mkt extension commands", + ] + }, + docs=["docs/internal-extension-framework.md"], + ) + + +def _generated_docs_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="docs.generated-reference", + kind="documentation", + summary="Generated CLI and API reference documentation from live command and API surfaces.", + capabilities=[ + ProcessingCapability(id="docs", kind="generate"), + ProcessingCapability(id="cli", kind="inspect"), + ProcessingCapability(id="api", kind="inspect"), + ], + safety={"reads_files": False, "writes_files": True, "network": False}, + input_contract="Click command tree | markitect_tool.__all__", + output_contract="Markdown reference document", + diagnostics_namespace="docs", + provenance_prefix="docs.generated_reference", + cli={"commands": ["mkt docs cli", "mkt docs api"]}, + docs=["docs/cli-reference.md", "docs/api-reference.md"], + ) + + def _processor_descriptors() -> list[ExtensionDescriptor]: return [ ExtensionDescriptor( diff --git a/tests/test_builtin_extension_catalog.py b/tests/test_builtin_extension_catalog.py index e88f440..788d15a 100644 --- a/tests/test_builtin_extension_catalog.py +++ b/tests/test_builtin_extension_catalog.py @@ -12,6 +12,8 @@ def test_builtin_extension_registry_lists_query_processors_and_backend(): assert "processor.identity" in ids assert "processor.uppercase" in ids assert "processor.include" in ids + assert "extension.catalog" in ids + assert "docs.generated-reference" in ids assert "backend.local-sqlite" in ids assert "workflow.markdown-dataflow" in ids assert "runtime.context" in ids @@ -22,6 +24,19 @@ def test_builtin_extension_registry_lists_query_processors_and_backend(): assert "memory.context-package" in ids +def test_builtin_meta_descriptors_expose_extension_and_docs_accesspoints(): + registry = builtin_extension_registry() + + extension = registry.get("extension.catalog") + docs = registry.get("docs.generated-reference") + + assert extension.kind == "extension-registry" + assert "mkt extension commands" in extension.cli["commands"] + assert docs.kind == "documentation" + assert docs.safety["writes_files"] is True + assert docs.cli["commands"] == ["mkt docs cli", "mkt docs api"] + + def test_builtin_processor_descriptors_capture_safety_and_provenance(): registry = builtin_extension_registry() diff --git a/tests/test_cli_api_polish.py b/tests/test_cli_api_polish.py new file mode 100644 index 0000000..d6ee27f --- /dev/null +++ b/tests/test_cli_api_polish.py @@ -0,0 +1,65 @@ +from click.testing import CliRunner + +import markitect_tool as api +from markitect_tool.cli import main + + +def test_mkt_completion_generates_bash_script(): + result = CliRunner().invoke(main, ["completion", "bash"]) + + assert result.exit_code == 0, result.output + assert "_mkt_completion" in result.output + assert "_MKT_COMPLETE=bash_complete" in result.output + + +def test_mkt_completion_instructions_are_unix_shell_friendly(): + result = CliRunner().invoke(main, ["completion", "zsh", "--instructions"]) + + assert result.exit_code == 0, result.output + assert "_MKT_COMPLETE=zsh_source mkt" in result.output + assert "~/.zshrc" in result.output + + +def test_mkt_extension_commands_exposes_builtin_cli_affordances(): + result = CliRunner().invoke(main, ["extension", "commands", "--format", "json"]) + + assert result.exit_code == 0, result.output + assert '"command": "mkt query"' in result.output + assert '"command": "mkt extension commands"' in result.output + assert '"command": "mkt docs api"' in result.output + + +def test_mkt_extension_inspect_explains_descriptor(): + result = CliRunner().invoke(main, ["extension", "inspect", "memory.context-package"]) + + assert result.exit_code == 0, result.output + assert "memory.context-package" in result.output + assert "mkt context pack" in result.output + + +def test_mkt_docs_cli_generates_command_reference(): + result = CliRunner().invoke(main, ["docs", "cli"]) + + assert result.exit_code == 0, result.output + assert "# Markitect CLI Reference" in result.output + assert "## `mkt extension commands`" in result.output + assert "## `mkt docs api`" in result.output + + +def test_mkt_docs_api_generates_public_api_reference(): + result = CliRunner().invoke(main, ["docs", "api"]) + + assert result.exit_code == 0, result.output + assert "# Markitect API Reference" in result.output + assert "query_document_jsonpath" in result.output + assert "ExtensionDescriptor" in result.output + assert "LocalSnapshotStore" in result.output + + +def test_top_level_api_exports_newer_architecture_surfaces(): + assert api.query_document_jsonpath + assert api.extract_document_with_engine + assert api.LocalSnapshotStore + assert api.ExtensionDescriptor + assert api.builtin_extension_registry + assert api.validate_schema diff --git a/tests/test_practical_usecases_e2e.py b/tests/test_practical_usecases_e2e.py new file mode 100644 index 0000000..0245211 --- /dev/null +++ b/tests/test_practical_usecases_e2e.py @@ -0,0 +1,205 @@ +import json +from pathlib import Path + +import pytest +from click.testing import CliRunner + +from markitect_tool.cli import main + + +@pytest.mark.parametrize( + ("label", "count"), + [ + ("small", 1), + ("typical", 12), + ("large", 80), + ], +) +def test_corpus_usecase_pipeline_small_typical_large(tmp_path: Path, label: str, count: int): + docs = tmp_path / "docs" + docs.mkdir() + for index in range(count): + (docs / f"adr-{index:03d}.md").write_text( + f"""--- +title: Decision {index} +labels: [public] +trust_zone: public +--- + +# Decision {index} + +## Context + +The team needs predictable latency for document operations in the {label} corpus. + +## Decision + +Use deterministic Markdown contracts, selectors, and cache refresh planning. + +## Consequences + +Search and context packaging should stay fast as the corpus grows. +""", + encoding="utf-8", + ) + + runner = CliRunner() + + parse = runner.invoke(main, ["parse", str(docs / "adr-000.md"), "--format", "json"]) + assert parse.exit_code == 0, parse.output + assert json.loads(parse.output)["headings"][0]["text"] == "Decision 0" + + query = runner.invoke( + main, + ["query", str(docs / "adr-000.md"), "sections[heading=Decision]", "--format", "json"], + ) + assert query.exit_code == 0, query.output + assert json.loads(query.output)["count"] == 1 + + index = runner.invoke(main, ["cache", "index", str(docs), "--root", str(tmp_path), "--format", "json"]) + assert index.exit_code == 0, index.output + index_data = json.loads(index.output) + assert len(index_data["indexed"]) == count + + search = runner.invoke( + main, + ["search", "latency", "--root", str(tmp_path), "--limit", "5", "--format", "json"], + ) + assert search.exit_code == 0, search.output + assert json.loads(search.output)["count"] >= 1 + + context = runner.invoke( + main, + [ + "context", + "pack", + "latency", + "--search", + "--root", + str(tmp_path), + "--max-items", + "3", + "--no-save", + "--format", + "json", + ], + ) + assert context.exit_code == 0, context.output + assert 1 <= len(json.loads(context.output)["items"]) <= 3 + + +def test_document_pipeline_usecases_from_examples(tmp_path: Path): + runner = CliRunner() + + contract = runner.invoke( + main, + [ + "contract", + "check", + "examples/documents/adr-valid.md", + "--contract", + "examples/contracts/adr.contract.md", + ], + ) + assert contract.exit_code == 0, contract.output + assert "valid" in contract.output + + template = runner.invoke( + main, + [ + "template", + "render", + "examples/templates/adr-summary.template.md", + "--data", + "examples/templates/adr-summary.data.yaml", + ], + ) + assert template.exit_code == 0, template.output + assert "Use Deterministic Templates" in template.output + + function = runner.invoke(main, ["function", "render", "examples/functions/basic-functions.md"]) + assert function.exit_code == 0, function.output + assert "DRAFT" in function.output + assert "Generated Section" in function.output + + workflow = runner.invoke( + main, + [ + "workflow", + "plan", + "examples/workflows/source-snippets.workflow.md", + "--output-dir", + str(tmp_path / "workflow-out"), + ], + ) + assert workflow.exit_code == 0, workflow.output + assert "valid" in workflow.output + + +def test_structure_reuse_policy_and_literate_usecases(tmp_path: Path): + runner = CliRunner() + + reference = runner.invoke( + main, + [ + "ref", + "resolve", + "examples/references/context.md", + "std:clauses.md#payment-terms", + "--root", + "examples/references", + "--format", + "json", + ], + ) + assert reference.exit_code == 0, reference.output + assert json.loads(reference.output)["count"] == 1 + + process = runner.invoke( + main, + ["process", "examples/references/context.md", "--root", "examples/references"], + ) + assert process.exit_code == 0, process.output + assert "payment-example" in process.output + + policy = runner.invoke( + main, + [ + "policy", + "check", + "public-agent", + "read", + "examples/policy/private/internal-note.md", + "--policy", + "examples/policy/local-label-policy.yaml", + "--path", + "private/internal-note.md", + ], + ) + assert policy.exit_code == 1 + assert "denied" in policy.output + + tangle = runner.invoke( + main, + ["tangle", "examples/literate/app.md", "--output-dir", str(tmp_path / "tangle")], + ) + assert tangle.exit_code == 0, tangle.output + assert (tmp_path / "tangle" / "src" / "app.py").exists() + + exploded = tmp_path / "exploded" + explode = runner.invoke( + main, + [ + "explode", + "examples/documents/adr-valid.md", + "--output-dir", + str(exploded), + "--format", + "json", + ], + ) + assert explode.exit_code == 0, explode.output + + implode = runner.invoke(main, ["implode", str(exploded), "--format", "markdown"]) + assert implode.exit_code == 0, implode.output + assert "# Use Markdown Contracts" in implode.output diff --git a/workplans/MKTT-WP-0017-cli-api-polish-and-practical-adoption.md b/workplans/MKTT-WP-0017-cli-api-polish-and-practical-adoption.md new file mode 100644 index 0000000..5fcbf81 --- /dev/null +++ b/workplans/MKTT-WP-0017-cli-api-polish-and-practical-adoption.md @@ -0,0 +1,212 @@ +--- +id: MKTT-WP-0017 +type: workplan +title: "CLI/API Polish And Practical Adoption" +domain: markitect +status: active +owner: markitect-tool +topic_slug: markitect +planning_priority: P1 +planning_order: 40 +depends_on_workplans: + - MKTT-WP-0003 + - MKTT-WP-0013 +related_workplans: + - MKTT-WP-0004 + - MKTT-WP-0007 + - MKTT-WP-0008 + - MKTT-WP-0010 + - MKTT-WP-0011 +created: "2026-05-04" +updated: "2026-05-04" +state_hub_workstream_id: "077bc50c-360b-425a-a1ad-fd961f819422" +--- + +# MKTT-WP-0017: CLI/API Polish And Practical Adoption + +## Purpose + +Polish `markitect-tool` from a broad implementation toolkit into something that +is easy to discover, use, document, test, and adopt in practical Markdown +pipelines. + +This workplan intentionally pauses advanced future architecture work and focuses +on the public surface: + +- CLI completeness and consistency +- shell completion and Unix-style generated command documentation +- public API completeness and generated API documentation +- CLI/API alignment with the extension architecture +- usecase-driven E2E testing +- accessibility for practical first use + +## Decisions + +- Treat every implemented architecture capability as something that should be + discoverable through CLI, API, docs, and extension descriptors. +- Keep generated docs deterministic and local-first. +- Use web-researched practical usecases as adoption and E2E-test anchors. +- Prefer improving command/API discoverability before adding more advanced + features. + +## P17.1 - Audit CLI and API surface completeness + +```task +id: MKTT-WP-0017-T001 +status: done +priority: high +state_hub_task_id: "1827fec2-3c64-4dd0-9b24-e6a7c3a19912" +``` + +Review all implemented modules, built-in extensions, and docs. Identify missing +CLI accesspoints, missing public API exports, and mismatches between extension +descriptors and real commands. + +Initial implementation adds: + +- `mkt completion` +- `mkt extension list` +- `mkt extension inspect` +- `mkt extension commands` +- `mkt docs cli` +- `mkt docs api` +- top-level API exports for newer query, backend, extension, schema, and policy + surfaces + +Output: implementation, tests, and an updated audit note if gaps remain. + +## P17.2 - Provide shell completion and Unix-style command docs + +```task +id: MKTT-WP-0017-T002 +status: done +priority: high +state_hub_task_id: "052b9684-cc23-4215-926e-bc498fe10f4f" +``` + +Expose Bash, Zsh, and Fish completion scripts through the CLI and generate a +Markdown command reference from the live Click tree. + +Output: CLI command, generated `docs/cli-reference.md`, README installation +notes, and tests. + +## P17.3 - Provide generated API documentation + +```task +id: MKTT-WP-0017-T003 +status: done +priority: high +state_hub_task_id: "8874078a-e2f4-4a4e-a2d0-a7cbd5d62816" +``` + +Generate a compact public API reference from `markitect_tool.__all__` and make +`pdoc` available as an optional richer documentation path. + +Output: CLI command, generated `docs/api-reference.md`, optional docs extra, +README notes, and tests. + +## P17.4 - Compile practical usecases and relevance expectations + +```task +id: MKTT-WP-0017-T004 +status: done +priority: high +state_hub_task_id: "23fffff4-a1f0-498c-b4d7-00241d0bb29d" +``` + +Revisit `INTENT.md`, PRD, FRS, examples, and docs. Use web research to define a +practical usecase matrix with relevance expectations and adoption signals. + +Output: `docs/practical-usecase-relevance.md` and links from intent/README. + +## P17.5 - Build E2E fixture matrix + +```task +id: MKTT-WP-0017-T005 +status: in_progress +priority: high +state_hub_task_id: "1159ec82-00e6-45cd-8a2c-e61ae03e758a" +``` + +Create E2E tests for the practical usecases with small, typical, and large +variations where meaningful. + +Required coverage: + +- parse/AST/metrics +- schema and contract checks +- query/extract and indexed search +- transform/compose/include/references/processors +- templates/generation/functions +- workflow inspect/plan/run +- cache/index/refresh plans +- policy-aware query/search +- literate weave/tangle +- explode/implode +- runtime form-state +- context packages +- extension catalog and generated docs + +Output: deterministic tests and fixtures with no network dependency. + +Initial coverage has been added for: + +- small, typical, and large synthetic corpus parse/query/cache/search/context + package flow +- example-backed contract, template, function, workflow, reference, processor, + policy, literate tangle, and explode/implode flows + +Remaining work: expand the matrix into explicit small/typical/large variants +for every high-relevance workflow and add performance assertions where they are +stable. + +## P17.6 - Run coverage and performance iteration + +```task +id: MKTT-WP-0017-T006 +status: todo +priority: high +state_hub_task_id: "8dd5afe5-cefa-4388-bf4c-a975ba5c77c3" +``` + +Run full tests repeatedly, add coverage instrumentation if needed, and fix bugs +or performance issues exposed by the E2E matrix. + +Output: passing test suite, documented coverage gaps, and targeted performance +notes for large synthetic corpora. + +## P17.7 - Improve first-use accessibility + +```task +id: MKTT-WP-0017-T007 +status: todo +priority: medium +state_hub_task_id: "590af805-0cc0-4585-b85f-9e95679afa77" +``` + +Make the first useful experience obvious. + +Possible improvements: + +- concise README quickstart with common workflows +- generated examples index +- command cheat sheet +- install/completion instructions +- clearer diagnostics examples +- `mkt extension` discovery guidance +- optional docs generation workflow using `pdoc` + +Output: README/docs/examples polish and tests where executable. + +## Exit Criteria + +- Every implemented subsystem has CLI access, public API access, docs, examples, + and tests or a documented reason for omission. +- Shell completion works for Bash, Zsh, and Fish through `mkt completion`. +- `docs/cli-reference.md` and `docs/api-reference.md` can be regenerated from + the live implementation. +- Practical usecases are documented with relevance expectations and E2E coverage + targets. +- E2E tests cover small, typical, and large variations for the important + workflows. +- Full test suite passes.