diff --git a/docs/api-reference.md b/docs/api-reference.md index 1cc61c1..5df164e 100644 --- a/docs/api-reference.md +++ b/docs/api-reference.md @@ -7,9 +7,11 @@ Generated from `markitect_tool.__all__`. - `BACKEND_CAPABILITIES` - object. set() -> new empty set object - `DEFAULT_BACKEND_PATHS` - object. Built-in immutable sequence. - `DEFAULT_LOCAL_INDEX_PATH` - object. str(object='') -> str +- `DOCUMENT_VALUE_KINDS` - object. set() -> new empty set object - `EMPTY_PARSE_OPTIONS_HASH` - object. str(object='') -> str - `EXPLODE_MANIFEST_NAME` - object. str(object='') -> str - `LOCAL_INDEX_SCHEMA_VERSION` - object. str(object='') -> str +- `MAX_FUNCTION_PIPELINE_DEPTH` - object. int([x]) -> integer - `NORMALIZED_SOURCE_SCHEMA_VERSION` - object. str(object='') -> str - `SOURCE_ADAPTER_ENTRY_POINT_GROUP` - object. str(object='') -> str @@ -122,8 +124,12 @@ Generated from `markitect_tool.__all__`. - `DocumentFunctionEvaluationResult(content: 'str', calls: 'list[DocumentFunctionRun]' = , diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. Result of expanding document functions in a Markdown document. - `DocumentFunctionParameter(name: 'str', kind: 'str' = 'string', required: 'bool' = True, default: 'Any' = None, variadic: 'bool' = False, description: 'str | None' = None) -> None` - class. One declared document function parameter. - `DocumentFunctionRegistry(descriptors: 'list[DocumentFunctionDescriptor] | None' = None) -> 'None'` - class. Registry and evaluator for document functions. -- `DocumentFunctionRun(call: 'DocumentFunctionCall', output: 'Any' = None, diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. One function call result. +- `DocumentFunctionRun(call: 'DocumentFunctionCall', output: 'Any' = None, value: 'DocumentValue | None' = None, diagnostics: 'list[Diagnostic]' = , provenance: 'list[ProcessingProvenance]' = , trace: 'list[ProcessingTrace]' = ) -> None` - class. One function call result. +- `DocumentValue(kind: 'str', value: 'Any' = None, items: "list['DocumentValue']" = , fields: "dict[str, 'DocumentValue']" = , metadata: 'dict[str, Any]' = , provenance: 'list[dict[str, Any]]' = ) -> None` - class. Typed value produced by a document function. +- `coerce_document_value(value: 'Any', *, declared_kind: 'str' = 'dynamic') -> 'DocumentValue'` - function. Coerce a Python value into a typed document value. - `default_document_function_registry() -> 'DocumentFunctionRegistry'` - function. Return built-in deterministic document functions. +- `document_value_to_json(value: 'DocumentValue | Any') -> 'dict[str, Any]'` - function. Return the stable JSON-compatible representation of a document value. +- `format_document_value(value: 'DocumentValue | Any', *, inline: 'bool') -> 'str'` - function. Map a typed document value to deterministic Markdown text. - `parse_document_function_calls(text: 'str') -> 'list[DocumentFunctionCall]'` - function. Parse inline and fenced document function calls. - `render_document_functions(text: 'str', *, registry: 'DocumentFunctionRegistry | None' = None, context: 'ProcessingContext | None' = None) -> 'DocumentFunctionEvaluationResult'` - function. Expand deterministic document functions in Markdown content. - `validate_document_functions(text: 'str', *, registry: 'DocumentFunctionRegistry | None' = None, allowed: 'list[str] | None' = None, forbidden: 'list[str] | None' = None) -> 'DocumentFunctionEvaluationResult'` - function. Validate function calls without rendering the document. diff --git a/docs/document-functions.md b/docs/document-functions.md index 3cadad2..f7c5866 100644 --- a/docs/document-functions.md +++ b/docs/document-functions.md @@ -78,6 +78,38 @@ Initial deterministic functions: | `md.codeblock` | Create a fenced code block. | | `data.get` | Read a value from processing context variables. | +## Typed Values + +Document functions now expose typed result values in addition to their legacy +raw `output` field. Each `DocumentFunctionRun` includes a `value` object with a +stable `kind`, metadata, and optional provenance. + +Supported value kinds: + +| Kind | Markdown mapping | +| --- | --- | +| `string` | Inline or block text. | +| `number` | Decimal text. | +| `boolean` | `true` or `false`. | +| `none` | Empty text. | +| `markdown` | Markdown content passed through directly. | +| `list` | Comma-separated inline text or newline-separated block text. | +| `dictionary` | Stable JSON object text. | +| `record` | Stable JSON object text. | +| `table` | Deterministic Markdown table. | +| `reference` | Label/title/value text, with provenance required. | +| `content_unit` | Label/title/value text, with provenance required. | +| `unknown` | Diagnostic fallback for mismatched output values. | +| `dynamic` | Reserved for explicitly dynamic values. | + +Function descriptors declare `output_type`; execution validates the returned +value against that declaration. Mismatches produce +`function.output_type_mismatch`, while reference-like values without provenance +produce `function.provenance_missing`. + +The raw `output` field remains for compatibility. New callers should prefer +`value` for typed API use and use the mapper when Markdown output is needed. + ## CLI List functions: @@ -132,18 +164,44 @@ capabilities before execution. External policy services may provide decisions through adapters later, but deterministic function execution has no external service dependency. +## Syntax Boundary + +The supported syntax remains intentionally conservative: + +- inline calls with `{{mkt:...}}` +- fenced calls with `mkt-function` +- positional and named arguments parsed with shell-like quoting +- pipeline chaining with quoted pipe characters preserved +- `${name}` context variable lookup +- bounded pipeline depth to avoid accidental runaway expressions + +Deferred syntax: + +- nested function expressions +- document-local function definitions +- conditionals, loops, lambdas, or general scripting +- Quarkdown syntax compatibility in the core parser + ## Natural Extensions The deterministic layer deliberately stops before becoming a full publishing -language. Future extension work is captured in -`MKTT-WP-0015: Render And Document Function Extensions`. +language. The original broad render/function follow-up has been split into +native workplans: -That workplan should consider: +- `MKTT-WP-0015`: typed document-function value contracts. +- `MKTT-WP-0020`: render/export adapter contracts. +- `MKTT-WP-0021`: render reference and asset manifest contracts. +- `MQD-WP-0001`: concrete Quarkdown adapter implementation in + `markitect-quarkdown`. +- `MKTF-WP-0003`: read-side source attachment metadata compatibility in + `markitect-filter`. + +Those workplans should consider: - typed document values and value-to-Markdown mapping -- richer multiline and nested function syntax -- document-local reusable functions -- render/export adapters, including optional Quarkdown source export +- constrained parser compatibility improvements, while deferring nested + function expressions and document-local reusable functions +- render/export adapter contracts, including optional Quarkdown source export - render-aware numbering, references, tables, figures, equations, and code blocks - static asset and media manifests with checksums diff --git a/docs/examples-index.md b/docs/examples-index.md index bfce854..45992f5 100644 --- a/docs/examples-index.md +++ b/docs/examples-index.md @@ -18,7 +18,7 @@ This index maps example files to practical usecases and useful commands. | `examples/references/context.md` | Namespace, region, fence, and section references | `mkt ref resolve examples/references/context.md 'std:clauses.md#payment-terms' --root examples/references` | | `examples/references/standard/clauses.md` | Referenced reusable content | Use with `mkt ref resolve` or `mkt process` | | `examples/migration/legacy-path-include.md` | Migration-style include handling | `mkt include examples/migration/legacy-path-include.md --base-dir examples/migration` | -| `examples/functions/basic-functions.md` | Deterministic document functions | `mkt function render examples/functions/basic-functions.md` | +| `examples/functions/basic-functions.md`, `examples/functions/typed-values.md` | Deterministic document functions and typed value examples | `mkt function render examples/functions/basic-functions.md` | ## Templates, Generation, And Workflows @@ -46,7 +46,8 @@ This index maps example files to practical usecases and useful commands. | `examples/policy/local-label-policy.yaml` | Local policy gateway | `mkt policy check public-agent read note --policy examples/policy/local-label-policy.yaml --label public` | | `examples/policy/enterprise-policy-map.yaml` | Enterprise IAM mapping fixture | `mkt policy subject examples/policy/netkingdom-claims.yaml --policy-map examples/policy/enterprise-policy-map.yaml` | | `examples/memory/workplan-context.manifest.yaml` | Context package manifest | `mkt context pack examples/memory/workplan-context.manifest.yaml --root . --no-save` | -| `examples/memory/memory-profile.local.yaml`, `examples/memory/decision-graph*.yaml` | Memory graph/profile contract fixtures | `mkt memory graph pack examples/memory/decision-graph-selection.yaml --format yaml` | +| `examples/memory/memory-profile.local.yaml`, `examples/memory/*graph*.yaml`, `examples/memory/*path*.yaml`, `examples/memory/*neighborhood*.yaml` | Memory graph/profile contract fixtures | `mkt memory graph pack examples/memory/decision-graph-selection.yaml --format yaml` | +| `examples/memory/invalid-memory-*.yaml`, `examples/memory/runtime-adapter-boundaries.yaml` | Negative memory fixtures and runtime handoff descriptors | `mkt memory graph validate examples/memory/invalid-memory-graph.yaml --format text` | ## Literate And Migration diff --git a/docs/memory-graph-contract.md b/docs/memory-graph-contract.md index c830fc0..c941163 100644 --- a/docs/memory-graph-contract.md +++ b/docs/memory-graph-contract.md @@ -42,10 +42,10 @@ validation so that cross-repository contracts do not drift silently. Valid node kinds include `question`, `claim`, `assumption`, `evidence`, `decision`, `alternative`, `outcome`, `risk`, `follow_up`, `turn`, `plan`, -`tool_call`, `observation`, `edit`, `validation`, `task`, `topic`, `document`, -`entity`, `artifact`, `concept`, `capability`, `contract`, `policy`, -`preference`, `source_fact`, `episode`, `finding`, `constraint`, `profile`, -`context_package`, and `memory`. +`tool_call`, `observation`, `edit`, `validation`, `interruption`, +`activation`, `task`, `topic`, `document`, `entity`, `artifact`, `concept`, +`capability`, `contract`, `policy`, `preference`, `source_fact`, `episode`, +`finding`, `constraint`, `profile`, `context_package`, and `memory`. Valid edge kinds include `supports`, `contradicts`, `depends_on`, `derived_from`, `led_to`, `affects`, `references`, `relates_to`, `supersedes`, @@ -70,6 +70,42 @@ mkt memory graph validate examples/memory/decision-graph.yaml mkt memory graph pack examples/memory/decision-graph-selection.yaml --format yaml ``` +## Fixture Catalog + +The memory fixtures cover the three intended memory views: + +| Files | View | Purpose | +| --- | --- | --- | +| `examples/memory/decision-graph.yaml`, `examples/memory/decision-graph-selection.yaml` | reasoning | Decision and constraint path that pins the contract/runtime boundary. | +| `examples/memory/conversation-path.yaml`, `examples/memory/conversation-path-selection.yaml` | conversation | Short-term branch/plan/tool/observation episode with event activation. | +| `examples/memory/knowledge-neighborhood.yaml`, `examples/memory/knowledge-neighborhood-selection.yaml` | knowledge | Durable neighborhood around schema versions, docs, compiler capability, and runtime policy. | +| `examples/memory/memory-profile.local.yaml` | mixed profile | Local profile declaring reasoning, knowledge, and package stores plus activation limits. | +| `examples/memory/invalid-memory-graph.yaml`, `examples/memory/invalid-memory-profile.yaml` | invalid | Negative fixtures for validation diagnostics and handoff contract tests. | + +The valid selections compile to standard `ContextPackage` output. Invalid +fixtures are deliberately small so downstream runtimes can assert diagnostic +codes without needing Markitect-owned durable storage. + +## Runtime Adapter Handoff + +`examples/memory/runtime-adapter-boundaries.yaml` is a non-executing descriptor +catalog for external memory runtimes and stores. It names the contracts that +runtime packages should accept or emit while keeping Markitect limited to local +validation, planning, and package compilation. + +The descriptor catalog covers: + +| Boundary | Responsibility | +| --- | --- | +| `memory.runtime.kontextual-engine` | Durable graph snapshots, event append, selection resolution, and refresh execution. | +| `memory.runtime.phased-memory` | Future lifecycle policies such as compaction, retention intent, and observability. | +| `memory.store.external-graph` | External graph database path and neighborhood queries. | +| `memory.store.vector` | Embedding and vector retrieval flows that return graph selections. | +| `memory.extract.llm-assisted` | Optional graph extraction proposals from transcripts or source artifacts. | +| `memory.policy.enterprise-pdp` | Runtime policy authorization and activation reauthorization. | +| `memory.registry.remote` | Future remote profile/package registry interactions. | +| `memory.audit.sink` | Durable audit and policy decision event sinks. | + This keeps Markitect as the compiler/contract boundary. `kontextual-engine` should implement runtime stores and event production against these schemas, and `infospace-bench` should benchmark generated context packages and runtime diff --git a/docs/workplan-planning-map.md b/docs/workplan-planning-map.md index 0edc3a0..3c91d5c 100644 --- a/docs/workplan-planning-map.md +++ b/docs/workplan-planning-map.md @@ -138,6 +138,16 @@ extension descriptors, `mkt source` commands, API exports, fake adapter fixtures, and sibling-repo migration notes. Concrete EPUB3 extraction remains `markitect-filter` scope. +`MKTT-WP-0015` has been narrowed to document-function typed value contracts. +The former render-and-function extension scope is split across native +workplans: `MKTT-WP-0020` owns render/export adapter contracts, +`MKTT-WP-0021` owns passive render reference and asset manifests, +`MQD-WP-0001` owns concrete Quarkdown adapter implementation in +`markitect-quarkdown`, and `MKTF-WP-0003` owns read-side source attachment +metadata compatibility in `markitect-filter`. This preserves the +`markitect-filter` read-only boundary and avoids pulling renderer execution +into `markitect-tool`. + ## State Hub Mirror Native State Hub dependency edges should mirror the whole-workstream @@ -182,3 +192,8 @@ dependencies: - `MKTT-WP-0018 -> MKTT-WP-0013` - `MKTT-WP-0018 -> MKTT-WP-0017` - `MKTT-WP-0018 -> MKTT-WP-0019` +- `MKTT-WP-0020 -> MKTT-WP-0013` +- `MKTT-WP-0020 -> MKTT-WP-0015` +- `MKTT-WP-0021 -> MKTT-WP-0010` +- `MKTT-WP-0021 -> MKTT-WP-0015` +- `MKTT-WP-0021 -> MKTT-WP-0020` diff --git a/examples/functions/typed-values.md b/examples/functions/typed-values.md new file mode 100644 index 0000000..53bf352 --- /dev/null +++ b/examples/functions/typed-values.md @@ -0,0 +1,12 @@ +# Typed Document Function Values + +String value: {{mkt:text.upper "draft"}} + +Markdown value: + +```mkt-function md.heading level=3 +Typed Section +``` + +Dynamic context values can map lists, records, dictionaries, and tables through +the typed value mapper when supplied by an embedding application. diff --git a/examples/memory/conversation-path-selection.yaml b/examples/memory/conversation-path-selection.yaml new file mode 100644 index 0000000..1bb6fbe --- /dev/null +++ b/examples/memory/conversation-path-selection.yaml @@ -0,0 +1,21 @@ +schema_version: markitect.memory.selection.v1 +graph: conversation-path.yaml +profile: memory-profile.local.yaml +title: Conversation Episode Package +intent: Activate the short-term path that explains why MKTT-WP-0016 is the current implementation target. +namespace: + project: markitect-tool + task: MKTT-WP-0016 +node_ids: + - turn.user-correction + - interruption.direction-corrected + - plan.finish-memory-graph-profile + - observation.remaining-gap +event_ids: + - event.user-correction +budget: + max_items: 5 + max_tokens: 1400 +metadata: + memory_view: conversation + purpose: short-term-episode diff --git a/examples/memory/conversation-path.yaml b/examples/memory/conversation-path.yaml new file mode 100644 index 0000000..97c0bc2 --- /dev/null +++ b/examples/memory/conversation-path.yaml @@ -0,0 +1,86 @@ +schema_version: markitect.memory.graph.v1 +id: markitect-memory-conversation-path +title: Markitect Memory Conversation Path +intent: Preserve a short-term implementation episode with branch, tool, observation, and activation context. +namespace: + project: markitect-tool + task: MKTT-WP-0016 +nodes: + - id: turn.user-correction + kind: turn + text: The requester corrected the target from the Pragmatic State Hub workplan to MKTT-WP-0016. + metadata: + title: User correction + speaker: user + - id: interruption.direction-corrected + kind: interruption + text: The implementation path changed before any file edits were made. + metadata: + title: Direction corrected + - id: plan.finish-memory-graph-profile + kind: plan + text: Finish the memory graph profile workplan by closing fixture breadth and runtime handoff gaps. + metadata: + title: Finish profile workplan + - id: tool.inspect-workplan + kind: tool_call + text: Inspect the active MKTT-WP-0016 workplan and memory graph implementation files. + metadata: + title: Inspect workplan and implementation + command_family: rg/sed + - id: observation.remaining-gap + kind: observation + text: The core graph compiler and CLI are implemented; the remaining work is fixture breadth and adapter handoff guidance. + metadata: + title: Remaining gap + - id: activation.contract-context + kind: activation + text: Activate the memory graph contract context package for follow-on implementation and cross-repo handoff. + metadata: + title: Contract context activation +edges: + - id: edge.turn-to-interruption + kind: led_to + source: turn.user-correction + target: interruption.direction-corrected + - id: edge.interruption-to-plan + kind: led_to + source: interruption.direction-corrected + target: plan.finish-memory-graph-profile + - id: edge.plan-to-tool + kind: depends_on + source: plan.finish-memory-graph-profile + target: tool.inspect-workplan + - id: edge.tool-to-observation + kind: led_to + source: tool.inspect-workplan + target: observation.remaining-gap + - id: edge.observation-to-activation + kind: activates + source: observation.remaining-gap + target: activation.contract-context +events: + - id: event.user-correction + kind: branched + timestamp: "2026-05-15T08:15:00Z" + actor: user + thread: markitect-tool + task: MKTT-WP-0016 + node_updates: + - node_id: turn.user-correction + operation: create + - node_id: interruption.direction-corrected + operation: create + branch: + from: pragmatic-state-hub + to: MKTT-WP-0016 + metadata: + reason: user-correction + - id: event.contract-context-activated + kind: activated + timestamp: "2026-05-15T08:20:00Z" + actor: codex + task: MKTT-WP-0016 + node_updates: + - node_id: activation.contract-context + operation: create diff --git a/examples/memory/invalid-memory-graph.yaml b/examples/memory/invalid-memory-graph.yaml new file mode 100644 index 0000000..280408b --- /dev/null +++ b/examples/memory/invalid-memory-graph.yaml @@ -0,0 +1,17 @@ +schema_version: markitect.memory.graph.v1 +id: invalid-memory-graph +title: Invalid Memory Graph +intent: Exercise graph validation diagnostics for fixture consumers. +nodes: + - id: node.unknown-kind + kind: unknown_kind + text: This node kind is not part of the memory graph vocabulary. +edges: + - id: edge.unknown-target + kind: supports + source: node.unknown-kind + target: node.missing +events: + - id: event.unknown-kind + kind: unrecognized_event + timestamp: "2026-05-15T08:35:00Z" diff --git a/examples/memory/invalid-memory-profile.yaml b/examples/memory/invalid-memory-profile.yaml new file mode 100644 index 0000000..680d11b --- /dev/null +++ b/examples/memory/invalid-memory-profile.yaml @@ -0,0 +1,11 @@ +schema_version: markitect.memory.profile.v1 +id: invalid-memory-profile +title: Invalid Memory Profile +intent: Exercise profile validation diagnostics for fixture consumers. +memory_kinds: + - reasoning + - telemetry +stores: + reasoning: local-event-log +metadata: + fixture: invalid-profile diff --git a/examples/memory/knowledge-neighborhood-selection.yaml b/examples/memory/knowledge-neighborhood-selection.yaml new file mode 100644 index 0000000..4969d94 --- /dev/null +++ b/examples/memory/knowledge-neighborhood-selection.yaml @@ -0,0 +1,23 @@ +schema_version: markitect.memory.selection.v1 +graph: knowledge-neighborhood.yaml +profile: memory-profile.local.yaml +title: Knowledge Neighborhood Package +intent: Activate the durable contract and runtime-boundary knowledge around memory graph profiles. +namespace: + project: markitect-tool + task: MKTT-WP-0016 +node_ids: + - artifact.memory-graph-doc + - concept.graph-profile-contract + - capability.context-package-compile + - contract.schema-versions + - policy.runtime-boundary + - source_fact.workplan-out-of-scope +event_ids: + - event.knowledge-refresh +budget: + max_items: 7 + max_tokens: 1800 +metadata: + memory_view: knowledge + purpose: durable-neighborhood diff --git a/examples/memory/knowledge-neighborhood.yaml b/examples/memory/knowledge-neighborhood.yaml new file mode 100644 index 0000000..ca2407c --- /dev/null +++ b/examples/memory/knowledge-neighborhood.yaml @@ -0,0 +1,94 @@ +schema_version: markitect.memory.graph.v1 +id: markitect-memory-knowledge-neighborhood +title: Markitect Memory Knowledge Neighborhood +intent: Represent the durable knowledge neighborhood around memory graph contracts and runtime boundaries. +namespace: + project: markitect-tool + task: MKTT-WP-0016 +nodes: + - id: entity.markitect-tool + kind: entity + text: Markitect Tool owns deterministic Markdown-native contracts, validation, and context package compilation. + metadata: + title: markitect-tool + - id: artifact.memory-graph-doc + kind: artifact + text: docs/memory-graph-contract.md describes the graph, profile, selection, fixture, and runtime handoff contracts. + source_spans: + - path: docs/memory-graph-contract.md + unit_kind: document + selector: document + engine: selector + metadata: + title: Memory graph contract document + - id: concept.graph-profile-contract + kind: concept + text: A memory profile declares runtime intent while a memory graph records nodes, edges, and event envelopes. + metadata: + title: Graph/profile contract + - id: capability.context-package-compile + kind: capability + text: Selected graph nodes and events compile into regular ContextPackage objects with provenance metadata. + metadata: + title: Graph selection compiler + - id: contract.schema-versions + kind: contract + text: markitect.memory.profile.v1, markitect.memory.graph.v1, and markitect.memory.selection.v1 are the stable contract versions for this slice. + metadata: + title: Memory schema versions + - id: policy.runtime-boundary + kind: policy + text: Runtime storage, retrieval, compaction, policy enforcement, audit, and service operation stay outside markitect-tool. + metadata: + title: Runtime boundary policy + - id: preference.local-deterministic + kind: preference + text: Keep first-version memory contract behavior deterministic, local-first, and service-free. + metadata: + title: Local deterministic preference + - id: source_fact.workplan-out-of-scope + kind: source_fact + text: MKTT-WP-0016 explicitly excludes durable graph stores, runtime daemons, retention enforcement, and remote registries. + source_spans: + - path: workplans/MKTT-WP-0016-agentic-memory-graphs-and-service-blueprints.md + unit_kind: section + selector: heading[Out Of Scope] + engine: selector + metadata: + title: Out-of-scope source fact +edges: + - id: edge.owner-to-doc + kind: mentions + source: entity.markitect-tool + target: artifact.memory-graph-doc + - id: edge.doc-to-contract + kind: supports + source: artifact.memory-graph-doc + target: contract.schema-versions + - id: edge.contract-to-concept + kind: supports + source: contract.schema-versions + target: concept.graph-profile-contract + - id: edge.policy-governs-compiler + kind: governs + source: policy.runtime-boundary + target: capability.context-package-compile + - id: edge.preference-supports-policy + kind: supports + source: preference.local-deterministic + target: policy.runtime-boundary + - id: edge.fact-supports-policy + kind: supports + source: source_fact.workplan-out-of-scope + target: policy.runtime-boundary +events: + - id: event.knowledge-refresh + kind: refreshed + timestamp: "2026-05-15T08:30:00Z" + actor: codex + task: MKTT-WP-0016 + node_updates: + - node_id: artifact.memory-graph-doc + operation: refresh + - node_id: contract.schema-versions + operation: refresh diff --git a/examples/memory/runtime-adapter-boundaries.yaml b/examples/memory/runtime-adapter-boundaries.yaml new file mode 100644 index 0000000..fa8ef4a --- /dev/null +++ b/examples/memory/runtime-adapter-boundaries.yaml @@ -0,0 +1,114 @@ +schema_version: markitect.memory.runtime-adapter-boundaries.v1 +id: markitect-memory-runtime-adapter-boundaries +title: Memory Runtime Adapter Boundaries +intent: Document non-executing handoff descriptors for external memory runtimes and stores. +boundary: + markitect_tool: + owns: + - validate memory profiles, graph snapshots, events, and selections + - compile selected graph context into ContextPackage objects + - emit deterministic provenance and diagnostics + does_not_own: + - durable graph or event persistence + - vector search, embedding generation, or graph database queries + - runtime retention, deletion, compaction, policy enforcement, or audit sinks +descriptors: + - id: memory.runtime.kontextual-engine + kind: memory-runtime-adapter + target_repo: kontextual-engine + operations: + - persist_graph_snapshot + - append_memory_event + - resolve_graph_selection + - refresh_memory_profile + input_contracts: + - markitect.memory.profile.v1 + - markitect.memory.graph.v1 + - markitect.memory.selection.v1 + output_contracts: + - markitect.memory.graph.v1 + - markitect.memory.selection.v1 + notes: Durable graph and event persistence belongs here, not in markitect-tool. + - id: memory.runtime.phased-memory + kind: memory-runtime-adapter + target_repo: phased-memory + operations: + - compact_memory_window + - enforce_retention_intent + - produce_profile_observability + input_contracts: + - markitect.memory.profile.v1 + - markitect.memory.graph.v1 + output_contracts: + - markitect.memory.graph.v1 + notes: Future phased-memory runtimes may implement lifecycle policies described by profiles. + - id: memory.store.external-graph + kind: graph-store-adapter + operations: + - query_neighborhood + - query_path + - resolve_edges + input_contracts: + - markitect.memory.graph.v1 + output_contracts: + - markitect.memory.graph.v1 + notes: External graph databases are queried by runtime packages and returned as contract snapshots. + - id: memory.store.vector + kind: vector-store-adapter + operations: + - embed_text + - retrieve_similar_nodes + - return_ranked_selection + input_contracts: + - markitect.memory.profile.v1 + output_contracts: + - markitect.memory.selection.v1 + notes: Embeddings and vector retrieval are optional runtime concerns. + - id: memory.extract.llm-assisted + kind: extraction-adapter + operations: + - propose_nodes + - propose_edges + - propose_event_summary + input_contracts: + - conversational transcript + - source artifact + output_contracts: + - markitect.memory.graph.v1 + notes: LLM-assisted extraction must stay adapter-only and emit reviewable graph contracts. + - id: memory.policy.enterprise-pdp + kind: policy-adapter + operations: + - authorize_memory_read + - authorize_memory_write + - reauthorize_activation + input_contracts: + - profile policy block + - graph node and edge policy metadata + output_contracts: + - policy decision envelope + notes: Enterprise policy decisions are external; Markitect preserves policy metadata only. + - id: memory.registry.remote + kind: remote-registry-adapter + operations: + - publish_profile_descriptor + - resolve_remote_package + - list_available_memory_views + input_contracts: + - markitect.memory.profile.v1 + - ContextPackage metadata + output_contracts: + - markitect.memory.selection.v1 + notes: Remote registries are future scope and must not be required for local validation. + - id: memory.audit.sink + kind: audit-adapter + operations: + - record_memory_event + - record_policy_decision + - record_activation + input_contracts: + - markitect.memory.graph.v1 event envelope + - policy decision envelope + output_contracts: + - audit receipt + notes: Durable audit/event sinks belong to runtime or governance packages. diff --git a/src/markitect_tool/__init__.py b/src/markitect_tool/__init__.py index 45f20b2..0ea9882 100644 --- a/src/markitect_tool/__init__.py +++ b/src/markitect_tool/__init__.py @@ -21,6 +21,8 @@ from markitect_tool.contract import ( validate_contract_file, ) from markitect_tool.document_function import ( + DOCUMENT_VALUE_KINDS, + MAX_FUNCTION_PIPELINE_DEPTH, DocumentFunctionCall, DocumentFunctionDescriptor, DocumentFunctionError, @@ -28,7 +30,11 @@ from markitect_tool.document_function import ( DocumentFunctionParameter, DocumentFunctionRegistry, DocumentFunctionRun, + DocumentValue, + coerce_document_value, default_document_function_registry, + document_value_to_json, + format_document_value, parse_document_function_calls, render_document_functions, validate_document_functions, @@ -354,6 +360,8 @@ __all__ = [ "load_contract_file", "validate_contract", "validate_contract_file", + "DOCUMENT_VALUE_KINDS", + "MAX_FUNCTION_PIPELINE_DEPTH", "DocumentFunctionCall", "DocumentFunctionDescriptor", "DocumentFunctionError", @@ -361,7 +369,11 @@ __all__ = [ "DocumentFunctionParameter", "DocumentFunctionRegistry", "DocumentFunctionRun", + "DocumentValue", + "coerce_document_value", "default_document_function_registry", + "document_value_to_json", + "format_document_value", "parse_document_function_calls", "render_document_functions", "validate_document_functions", diff --git a/src/markitect_tool/document_function.py b/src/markitect_tool/document_function.py index c07670c..5c13aaf 100644 --- a/src/markitect_tool/document_function.py +++ b/src/markitect_tool/document_function.py @@ -23,14 +23,62 @@ FENCE_CALL_RE = re.compile( r"```(?P[^\n`]*)\n(?P.*?)\n```", re.DOTALL, ) +MAX_FUNCTION_PIPELINE_DEPTH = 12 FunctionImplementation = Callable[..., Any] +DOCUMENT_VALUE_KINDS = { + "string", + "number", + "boolean", + "none", + "markdown", + "list", + "dictionary", + "record", + "table", + "reference", + "content_unit", + "unknown", + "dynamic", +} + class DocumentFunctionError(ValueError): """Raised when document function parsing or evaluation fails.""" +@dataclass(frozen=True) +class DocumentValue: + """Typed value produced by a document function.""" + + kind: str + value: Any = None + items: list["DocumentValue"] = field(default_factory=list) + fields: dict[str, "DocumentValue"] = field(default_factory=dict) + metadata: dict[str, Any] = field(default_factory=dict) + provenance: list[dict[str, Any]] = field(default_factory=list) + + def __post_init__(self) -> None: + if self.kind not in DOCUMENT_VALUE_KINDS: + raise DocumentFunctionError(f"Unknown document value kind `{self.kind}`.") + + def to_dict(self) -> dict[str, Any]: + return _drop_empty( + { + "kind": self.kind, + "value": self.value, + "items": [item.to_dict() for item in self.items], + "fields": {key: value.to_dict() for key, value in self.fields.items()}, + "metadata": self.metadata, + "provenance": self.provenance, + } + ) + + def __str__(self) -> str: + return format_document_value(self, inline=True) + + @dataclass(frozen=True) class DocumentFunctionParameter: """One declared document function parameter.""" @@ -107,6 +155,7 @@ class DocumentFunctionRun: call: DocumentFunctionCall output: Any = None + value: DocumentValue | None = None diagnostics: list[Diagnostic] = field(default_factory=list) provenance: list[ProcessingProvenance] = field(default_factory=list) trace: list[ProcessingTrace] = field(default_factory=list) @@ -120,7 +169,8 @@ class DocumentFunctionRun: { "call": self.call.to_dict(), "valid": self.valid, - "output": self.output, + "output": _serialize_output(self.output), + "value": self.value.to_dict() if self.value else None, "diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics], "provenance": [event.to_dict() for event in self.provenance], "trace": [event.to_dict() for event in self.trace], @@ -180,6 +230,10 @@ class DocumentFunctionRegistry: raise DocumentFunctionError(f"Duplicate document function `{descriptor.id}`") if descriptor.implementation is None: raise DocumentFunctionError(f"Document function `{descriptor.id}` has no implementation") + if _normalize_output_type(descriptor.output_type) not in DOCUMENT_VALUE_KINDS: + raise DocumentFunctionError( + f"Document function `{descriptor.id}` declares unknown output type `{descriptor.output_type}`" + ) self._descriptors[descriptor.id] = descriptor def get(self, function_id: str) -> DocumentFunctionDescriptor: @@ -208,6 +262,7 @@ class DocumentFunctionRegistry: ) -> DocumentFunctionRun: context = context or ProcessingContext() output: Any = None + value: DocumentValue | None = None diagnostics: list[Diagnostic] = [] provenance: list[ProcessingProvenance] = [] trace: list[ProcessingTrace] = [] @@ -230,12 +285,15 @@ class DocumentFunctionRegistry: trace.extend(run.trace) if not run.valid: output = current.raw + value = run.value break output = run.output + value = run.value return DocumentFunctionRun( call=call, output=output, + value=value, diagnostics=diagnostics, provenance=provenance, trace=trace, @@ -281,6 +339,10 @@ class DocumentFunctionRegistry: else: assert descriptor.implementation is not None output = descriptor.implementation(*args, **kwargs) + value = coerce_document_value(output, declared_kind=descriptor.output_type) + value_diagnostic = _validate_output_value(descriptor, value, call, context) + if value_diagnostic is not None: + return DocumentFunctionRun(call=call, output=output, value=value, diagnostics=[value_diagnostic]) except Exception as exc: return _call_error(call, "function.evaluation_failed", str(exc), context) @@ -301,7 +363,83 @@ class DocumentFunctionRegistry: metadata={"function": descriptor.id, "line": call.line}, ) ] - return DocumentFunctionRun(call=call, output=output, provenance=provenance, trace=trace) + return DocumentFunctionRun(call=call, output=output, value=value, provenance=provenance, trace=trace) + + +def coerce_document_value(value: Any, *, declared_kind: str = "dynamic") -> DocumentValue: + """Coerce a Python value into a typed document value.""" + + normalized_kind = _normalize_output_type(declared_kind) + if isinstance(value, DocumentValue): + if normalized_kind in {"dynamic", "any"} or _value_matches_kind(value, normalized_kind): + return value + return DocumentValue(kind="unknown", value=value.to_dict(), metadata={"declared_kind": normalized_kind}) + if normalized_kind == "dynamic" or normalized_kind == "any": + return _infer_document_value(value) + if normalized_kind == "markdown": + if isinstance(value, str): + return DocumentValue(kind="markdown", value=value) + if normalized_kind == "string": + if isinstance(value, str): + return DocumentValue(kind="string", value=value) + if normalized_kind == "number": + if isinstance(value, (int, float)) and not isinstance(value, bool): + return DocumentValue(kind="number", value=value) + if normalized_kind == "boolean": + if isinstance(value, bool): + return DocumentValue(kind="boolean", value=value) + if normalized_kind == "none": + if value is None: + return DocumentValue(kind="none") + if normalized_kind == "list": + if isinstance(value, list): + return DocumentValue(kind="list", items=[coerce_document_value(item) for item in value]) + if normalized_kind in {"dictionary", "record"}: + if isinstance(value, dict): + return DocumentValue( + kind=normalized_kind, + fields={str(key): coerce_document_value(raw) for key, raw in value.items()}, + ) + if normalized_kind == "table": + if isinstance(value, list): + return DocumentValue(kind="table", items=[coerce_document_value(item, declared_kind="record") for item in value]) + if normalized_kind in {"reference", "content_unit", "unknown"}: + return DocumentValue(kind=normalized_kind, value=value) + return DocumentValue(kind="unknown", value=value, metadata={"declared_kind": normalized_kind}) + + +def format_document_value(value: DocumentValue | Any, *, inline: bool) -> str: + """Map a typed document value to deterministic Markdown text.""" + + document_value = value if isinstance(value, DocumentValue) else coerce_document_value(value) + if document_value.kind in {"markdown", "string"}: + return str(document_value.value or "") + if document_value.kind == "number": + return str(document_value.value) + if document_value.kind == "boolean": + return "true" if document_value.value else "false" + if document_value.kind == "none": + return "" + if document_value.kind == "list": + rendered = [format_document_value(item, inline=inline) for item in document_value.items] + return ", ".join(rendered) if inline else "\n".join(rendered) + if document_value.kind in {"dictionary", "record"}: + return json.dumps(_document_value_to_plain(document_value), sort_keys=True, ensure_ascii=False) + if document_value.kind == "table": + return _format_table_value(document_value) + if document_value.kind in {"reference", "content_unit"}: + label = document_value.metadata.get("label") or document_value.metadata.get("title") + return str(label or document_value.value or "") + if document_value.kind == "dynamic": + return format_document_value(coerce_document_value(document_value.value), inline=inline) + return "" if document_value.value is None else str(document_value.value) + + +def document_value_to_json(value: DocumentValue | Any) -> dict[str, Any]: + """Return the stable JSON-compatible representation of a document value.""" + + document_value = value if isinstance(value, DocumentValue) else coerce_document_value(value) + return document_value.to_dict() def default_document_function_registry() -> DocumentFunctionRegistry: @@ -314,6 +452,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: "Uppercase text.", _text_upper, [DocumentFunctionParameter("value")], + output_type="string", examples=['{{mkt:text.upper "draft"}}'], ), _descriptor( @@ -321,6 +460,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: "Lowercase text.", _text_lower, [DocumentFunctionParameter("value")], + output_type="string", examples=['{{mkt:text.lower "DRAFT"}}'], ), _descriptor( @@ -328,6 +468,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: "Title-case text.", _text_title, [DocumentFunctionParameter("value")], + output_type="string", examples=['{{mkt:text.title "release notes"}}'], ), _descriptor( @@ -335,6 +476,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: "Trim surrounding whitespace.", _text_trim, [DocumentFunctionParameter("value")], + output_type="string", examples=['{{mkt:text.trim " ok "}}'], ), _descriptor( @@ -346,6 +488,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: DocumentFunctionParameter("old"), DocumentFunctionParameter("new"), ], + output_type="string", examples=['{{mkt:text.replace "draft" draft final}}'], ), _descriptor( @@ -356,6 +499,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: DocumentFunctionParameter("items", variadic=True), DocumentFunctionParameter("sep", required=False, default=""), ], + output_type="string", examples=['{{mkt:text.join "A" "B" sep=", "}}'], ), _descriptor( @@ -398,6 +542,7 @@ def default_document_function_registry() -> DocumentFunctionRegistry: "Read a value from processing context variables.", _data_get, [DocumentFunctionParameter("key"), DocumentFunctionParameter("default", required=False, default="")], + output_type="dynamic", examples=["{{mkt:data.get title}}"], ), ] @@ -460,7 +605,7 @@ def render_document_functions( trace.extend(run.trace) if not run.valid: return match.group(0) - return _format_function_output(run.output, inline=True) + return _format_function_output(run.value or run.output, inline=True) content = INLINE_CALL_RE.sub(replace_inline, text) @@ -483,7 +628,7 @@ def render_document_functions( trace.extend(run.trace) if not run.valid: return match.group(0) - return _format_function_output(run.output, inline=False) + return _format_function_output(run.value or run.output, inline=False) content = FENCE_CALL_RE.sub(replace_fence, content) trace.append(ProcessingTrace(event="document_function.rendered", metadata={"calls": len(runs)})) @@ -554,6 +699,10 @@ def _parse_call_expression( pipeline_parts = _split_pipeline_expression(expression) if not pipeline_parts: raise DocumentFunctionError("Document function call is empty.") + if len(pipeline_parts) > MAX_FUNCTION_PIPELINE_DEPTH: + raise DocumentFunctionError( + f"Document function pipeline exceeds maximum depth {MAX_FUNCTION_PIPELINE_DEPTH}." + ) first = _parse_single_call(pipeline_parts[0], raw=raw, inline=inline, line=line, body=body) pipeline = [ _parse_single_call(part, raw=part, inline=inline, line=line) @@ -712,6 +861,109 @@ def _blocked_capabilities( return sorted(set(blocked)) +def _normalize_output_type(output_type: str | None) -> str: + aliases = { + None: "dynamic", + "": "dynamic", + "any": "dynamic", + "integer": "number", + "float": "number", + "bool": "boolean", + "dict": "dictionary", + "map": "dictionary", + "markdown_content": "markdown", + "content-unit": "content_unit", + } + normalized = str(output_type or "dynamic").strip().lower().replace("-", "_") + return aliases.get(normalized, normalized) + + +def _infer_document_value(value: Any) -> DocumentValue: + if isinstance(value, DocumentValue): + return value + if value is None: + return DocumentValue(kind="none") + if isinstance(value, bool): + return DocumentValue(kind="boolean", value=value) + if isinstance(value, (int, float)): + return DocumentValue(kind="number", value=value) + if isinstance(value, str): + return DocumentValue(kind="string", value=value) + if isinstance(value, list): + return DocumentValue(kind="list", items=[coerce_document_value(item) for item in value]) + if isinstance(value, dict): + return DocumentValue( + kind="dictionary", + fields={str(key): coerce_document_value(raw) for key, raw in value.items()}, + ) + return DocumentValue(kind="unknown", value=str(value), metadata={"python_type": type(value).__name__}) + + +def _value_matches_kind(value: DocumentValue, expected_kind: str) -> bool: + if expected_kind in {"dynamic", "any"}: + return True + if expected_kind == "markdown": + return value.kind in {"markdown", "string"} + if expected_kind == "dictionary": + return value.kind in {"dictionary", "record"} + return value.kind == expected_kind + + +def _validate_output_value( + descriptor: DocumentFunctionDescriptor, + value: DocumentValue, + call: DocumentFunctionCall, + context: ProcessingContext, +) -> Diagnostic | None: + expected = _normalize_output_type(descriptor.output_type) + if value.kind == "unknown": + return _output_diagnostic( + call, + "function.output_type_mismatch", + f"Function `{descriptor.id}` returned a value that does not match output type `{expected}`.", + context, + {"function": descriptor.id, "output_type": expected, "value_kind": value.kind}, + ) + if descriptor.execution == "deterministic" and value.kind == "dynamic": + return _output_diagnostic( + call, + "function.dynamic_output", + f"Function `{descriptor.id}` returned a dynamic value in a deterministic context.", + context, + {"function": descriptor.id, "output_type": expected}, + ) + if value.kind in {"reference", "content_unit"} and not value.provenance: + return _output_diagnostic( + call, + "function.provenance_missing", + f"Function `{descriptor.id}` returned `{value.kind}` without provenance.", + context, + {"function": descriptor.id, "value_kind": value.kind}, + ) + return None + + +def _output_diagnostic( + call: DocumentFunctionCall, + code: str, + message: str, + context: ProcessingContext, + details: dict[str, Any], +) -> Diagnostic: + return Diagnostic( + severity="error", + code=code, + message=message, + source=SourceLocation( + path=str(context.source_path) if context.source_path else None, + line=call.line, + ) + if context.source_path or call.line + else None, + details=details, + ) + + def _resolve_value(value: Any, context: ProcessingContext) -> Any: if isinstance(value, str): if value.startswith("${") and value.endswith("}"): @@ -721,13 +973,59 @@ def _resolve_value(value: Any, context: ProcessingContext) -> Any: def _format_function_output(value: Any, *, inline: bool) -> str: - if isinstance(value, str): - return value + return format_document_value(value, inline=inline) + + +def _document_value_to_plain(value: DocumentValue) -> Any: + if value.kind in {"string", "number", "boolean", "markdown", "reference", "content_unit", "unknown", "dynamic"}: + return value.value + if value.kind == "none": + return None + if value.kind in {"list", "table"}: + return [_document_value_to_plain(item) for item in value.items] + if value.kind in {"dictionary", "record"}: + return {key: _document_value_to_plain(raw) for key, raw in value.fields.items()} + return value.value + + +def _format_table_value(value: DocumentValue) -> str: + rows = [item for item in value.items if item.kind in {"record", "dictionary"}] + if not rows: + return "" + columns: list[str] = [] + for row in rows: + for key in row.fields: + if key not in columns: + columns.append(key) + if not columns: + return "" + header = "| " + " | ".join(_escape_table_cell(column) for column in columns) + " |" + separator = "| " + " | ".join("---" for _ in columns) + " |" + body = [] + for row in rows: + body.append( + "| " + + " | ".join( + _escape_table_cell(format_document_value(row.fields.get(column, DocumentValue(kind="none")), inline=True)) + for column in columns + ) + + " |" + ) + return "\n".join([header, separator, *body]) + + +def _escape_table_cell(value: str) -> str: + return value.replace("|", "\\|").replace("\n", " ").strip() + + +def _serialize_output(value: Any) -> Any: + if isinstance(value, DocumentValue): + return value.to_dict() if isinstance(value, list): - return ", ".join(str(item) for item in value) if inline else "\n".join(str(item) for item in value) + return [_serialize_output(item) for item in value] if isinstance(value, dict): - return json.dumps(value, sort_keys=True, ensure_ascii=False) - return "" if value is None else str(value) + return {str(key): _serialize_output(raw) for key, raw in value.items()} + return value def _parse_literal(value: str) -> Any: diff --git a/src/markitect_tool/extension/builtins.py b/src/markitect_tool/extension/builtins.py index d10b3a5..62dd766 100644 --- a/src/markitect_tool/extension/builtins.py +++ b/src/markitect_tool/extension/builtins.py @@ -26,6 +26,7 @@ def builtin_extension_registry() -> ExtensionRegistry: _local_label_policy_descriptor(), _document_function_descriptor(), _memory_graph_contract_descriptor(), + _memory_runtime_adapter_descriptor(), _agent_memory_descriptor(), source_adapter_registry_descriptor(), ]: @@ -319,6 +320,23 @@ def _document_function_descriptor() -> ExtensionDescriptor: metadata={ "execution": "deterministic-only", "external_policy_services_required": False, + "typed_values": True, + "value_kinds": [ + "string", + "number", + "boolean", + "none", + "markdown", + "list", + "dictionary", + "record", + "table", + "reference", + "content_unit", + "unknown", + "dynamic", + ], + "render_export_execution": False, }, ) @@ -412,6 +430,13 @@ def _memory_graph_contract_descriptor() -> ExtensionDescriptor: "examples/memory/memory-profile.local.yaml", "examples/memory/decision-graph.yaml", "examples/memory/decision-graph-selection.yaml", + "examples/memory/conversation-path.yaml", + "examples/memory/conversation-path-selection.yaml", + "examples/memory/knowledge-neighborhood.yaml", + "examples/memory/knowledge-neighborhood-selection.yaml", + "examples/memory/invalid-memory-graph.yaml", + "examples/memory/invalid-memory-profile.yaml", + "examples/memory/runtime-adapter-boundaries.yaml", ], metadata={ "schema_versions": [ @@ -422,7 +447,59 @@ def _memory_graph_contract_descriptor() -> ExtensionDescriptor: "runtime_execution_required": False, "runtime_handoff_repositories": [ "kontextual-engine", + "phased-memory", "infospace-bench", ], + "runtime_adapter_boundaries": [ + "memory.runtime.kontextual-engine", + "memory.runtime.phased-memory", + "memory.store.external-graph", + "memory.store.vector", + "memory.extract.llm-assisted", + "memory.policy.enterprise-pdp", + "memory.registry.remote", + "memory.audit.sink", + ], + }, + ) + + +def _memory_runtime_adapter_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="memory.runtime-adapter-boundary", + kind="memory-runtime-adapter", + summary="Non-executing handoff descriptors for external memory runtimes, stores, extraction, policy, and audit.", + capabilities=[ + ProcessingCapability(id="memory_runtime_adapters", kind="describe"), + ProcessingCapability(id="memory_graphs", kind="handoff"), + ProcessingCapability(id="memory_events", kind="handoff"), + ProcessingCapability(id="context_packages", kind="handoff"), + ProcessingCapability(id="policy_decisions", kind="handoff"), + ], + safety={ + "reads_files": False, + "writes_files": False, + "network": False, + "launches_services": False, + "runtime_execution": False, + }, + input_contract="MemoryProfile | MemoryGraph | MemoryEvent | MemoryGraphSelection | ContextPackage metadata", + output_contract="External runtime/store/policy/audit adapter descriptor", + diagnostics_namespace="memory.runtime_adapter", + provenance_prefix="memory.runtime_adapter_boundary", + docs=["docs/memory-graph-contract.md"], + examples=["examples/memory/runtime-adapter-boundaries.yaml"], + metadata={ + "descriptor_catalog": "examples/memory/runtime-adapter-boundaries.yaml", + "markitect_role": "contract-validation-and-context-package-compilation", + "external_runtime_roles": [ + "durable graph and event persistence", + "graph and vector retrieval", + "LLM-assisted graph extraction", + "policy enforcement and reauthorization", + "remote registry coordination", + "audit and event sinks", + ], + "services_launched_by_markitect_tool": False, }, ) diff --git a/src/markitect_tool/memory/graph.py b/src/markitect_tool/memory/graph.py index 1250e76..e7a6951 100644 --- a/src/markitect_tool/memory/graph.py +++ b/src/markitect_tool/memory/graph.py @@ -31,6 +31,7 @@ MEMORY_NODE_KINDS = { "assumption", "alternative", "artifact", + "activation", "capability", "claim", "concept", @@ -45,6 +46,7 @@ MEMORY_NODE_KINDS = { "evidence", "finding", "follow_up", + "interruption", "memory", "observation", "outcome", diff --git a/tests/test_builtin_extension_catalog.py b/tests/test_builtin_extension_catalog.py index c274fd3..9fdf6ba 100644 --- a/tests/test_builtin_extension_catalog.py +++ b/tests/test_builtin_extension_catalog.py @@ -21,6 +21,8 @@ def test_builtin_extension_registry_lists_query_processors_and_backend(): assert "runtime.assessment" in ids assert "policy.local-label" in ids assert "document.function" in ids + assert "memory.graph-contract" in ids + assert "memory.runtime-adapter-boundary" in ids assert "memory.context-package" in ids assert "source.adapter-registry" in ids @@ -131,6 +133,9 @@ def test_builtin_document_function_descriptor_exposes_deterministic_boundary(): assert descriptor.kind == "document-function" assert descriptor.safety["network"] is False assert descriptor.metadata["external_policy_services_required"] is False + assert descriptor.metadata["typed_values"] is True + assert "table" in descriptor.metadata["value_kinds"] + assert descriptor.metadata["render_export_execution"] is False assert {capability.id for capability in descriptor.capabilities} >= { "document_function", "deterministic", @@ -158,3 +163,22 @@ def test_builtin_memory_descriptor_exposes_local_optional_boundary(): } assert "mkt context pack" in descriptor.cli["commands"] assert "mkt context activate" in descriptor.cli["commands"] + + +def test_builtin_memory_graph_descriptor_exposes_runtime_handoff_boundaries(): + registry = builtin_extension_registry() + + graph = registry.get("memory.graph-contract") + adapters = registry.get("memory.runtime-adapter-boundary") + + assert graph.kind == "memory-contract" + assert graph.safety["external_memory_services"] is False + assert "mkt memory graph pack" in graph.cli["commands"] + assert "examples/memory/conversation-path.yaml" in graph.examples + assert "examples/memory/knowledge-neighborhood.yaml" in graph.examples + assert "memory.runtime.kontextual-engine" in graph.metadata["runtime_adapter_boundaries"] + assert "memory.audit.sink" in graph.metadata["runtime_adapter_boundaries"] + assert adapters.kind == "memory-runtime-adapter" + assert adapters.safety["runtime_execution"] is False + assert adapters.metadata["services_launched_by_markitect_tool"] is False + assert "examples/memory/runtime-adapter-boundaries.yaml" in adapters.examples diff --git a/tests/test_cli_extension_specs.py b/tests/test_cli_extension_specs.py index 92e0da5..4327083 100644 --- a/tests/test_cli_extension_specs.py +++ b/tests/test_cli_extension_specs.py @@ -19,6 +19,7 @@ def test_collect_cli_command_specs_from_builtin_registry(): assert ("backend.local-sqlite", "mkt cache index") in commands assert ("backend.local-sqlite", "mkt search") in commands assert ("document.function", "mkt function render") in commands + assert ("memory.graph-contract", "mkt memory graph pack") in commands assert ("memory.context-package", "mkt context pack") in commands diff --git a/tests/test_document_functions.py b/tests/test_document_functions.py index 95a7443..c64bf9e 100644 --- a/tests/test_document_functions.py +++ b/tests/test_document_functions.py @@ -8,7 +8,12 @@ from markitect_tool.document_function import ( DocumentFunctionDescriptor, DocumentFunctionParameter, DocumentFunctionRegistry, + DocumentValue, + MAX_FUNCTION_PIPELINE_DEPTH, + coerce_document_value, default_document_function_registry, + document_value_to_json, + format_document_value, parse_document_function_calls, render_document_functions, validate_document_functions, @@ -51,6 +56,20 @@ Decision assert "### Decision" in result.content assert len(result.calls) == 2 assert result.provenance[0].operation == "document_function.text.upper" + assert result.calls[0].value.kind == "string" + assert result.calls[1].value.kind == "markdown" + + +def test_document_values_coerce_and_map_to_markdown(): + table = coerce_document_value( + [{"name": "Ada", "role": "Architect"}, {"name": "Grace", "role": "Reviewer"}], + declared_kind="table", + ) + + assert document_value_to_json(table)["kind"] == "table" + assert format_document_value(DocumentValue(kind="boolean", value=True), inline=True) == "true" + assert "| name | role |" in format_document_value(table, inline=False) + assert "| Ada | Architect |" in format_document_value(table, inline=False) def test_pipeline_passes_previous_output_to_next_function(): @@ -67,6 +86,17 @@ def test_pipeline_separator_inside_quotes_is_literal(): assert result.content == "a/b" +def test_pipeline_depth_limit_reports_syntax_error(): + expression = " | ".join(["text.trim value"] * (MAX_FUNCTION_PIPELINE_DEPTH + 1)) + + try: + parse_document_function_calls("{{mkt:" + expression + "}}") + except Exception as exc: + assert "maximum depth" in str(exc) + else: + raise AssertionError("Expected pipeline depth diagnostic") + + def test_context_variables_can_be_used_in_function_arguments(): context = ProcessingContext(variables={"title": "Architecture Decision"}) @@ -75,6 +105,16 @@ def test_context_variables_can_be_used_in_function_arguments(): assert result.content == "## Architecture Decision" +def test_dynamic_context_values_render_through_typed_mapper(): + context = ProcessingContext(variables={"items": ["alpha", "beta"]}) + + result = render_document_functions("{{mkt:data.get items}}", context=context) + + assert result.valid + assert result.content == "alpha, beta" + assert result.calls[0].value.kind == "list" + + def test_validate_document_functions_reports_forbidden_calls(): result = validate_document_functions("{{mkt:text.upper draft}}", forbidden=["text.upper"]) @@ -106,6 +146,41 @@ def test_registry_can_expose_custom_function_without_core_rewrite(): assert result.content == "[ok]" +def test_descriptor_output_type_mismatch_is_diagnostic(): + registry = DocumentFunctionRegistry() + registry.register( + DocumentFunctionDescriptor( + id="demo.count", + summary="Return a count.", + output_type="number", + implementation=lambda: "not-a-number", + ) + ) + + result = render_document_functions("{{mkt:demo.count}}", registry=registry) + + assert not result.valid + assert result.content == "{{mkt:demo.count}}" + assert result.diagnostics[0].code == "function.output_type_mismatch" + + +def test_reference_values_require_provenance(): + registry = DocumentFunctionRegistry() + registry.register( + DocumentFunctionDescriptor( + id="demo.reference", + summary="Return a reference.", + output_type="reference", + implementation=lambda: DocumentValue(kind="reference", value="std:clause.md#payment"), + ) + ) + + result = render_document_functions("{{mkt:demo.reference}}", registry=registry) + + assert not result.valid + assert result.diagnostics[0].code == "function.provenance_missing" + + def test_unknown_function_is_left_in_place_with_diagnostic(): result = render_document_functions("{{mkt:nope.missing value}}") @@ -133,6 +208,17 @@ def test_mkt_function_render_outputs_expanded_markdown(tmp_path: Path): assert "**Important**" in result.output +def test_mkt_function_render_json_includes_typed_values(tmp_path: Path): + file = tmp_path / "functions.md" + file.write_text("{{mkt:text.upper draft}}\n", encoding="utf-8") + + result = CliRunner().invoke(main, ["function", "render", str(file), "--format", "json"]) + data = json.loads(result.output) + + assert result.exit_code == 0 + assert data["calls"][0]["value"] == {"kind": "string", "value": "DRAFT"} + + def test_mkt_function_check_can_restrict_allowed_functions(tmp_path: Path): file = tmp_path / "functions.md" file.write_text("{{mkt:text.upper draft}}\n", encoding="utf-8") @@ -148,3 +234,4 @@ def test_default_registry_serializes_without_implementations(): assert data["count"] >= 1 assert "implementation" not in data["functions"][0] + assert {function["id"]: function["output_type"] for function in data["functions"]}["data.get"] == "dynamic" diff --git a/tests/test_memory_graph_contracts.py b/tests/test_memory_graph_contracts.py index fd13d62..346dd83 100644 --- a/tests/test_memory_graph_contracts.py +++ b/tests/test_memory_graph_contracts.py @@ -48,6 +48,62 @@ def test_memory_graph_validation_and_context_package_compile(tmp_path: Path): assert package.retrieval_recipes[0].kind == "memory-graph-selection" +def test_memory_graph_example_fixtures_cover_reasoning_conversation_and_knowledge(): + examples = Path("examples/memory") + profile = load_memory_profile_file(examples / "memory-profile.local.yaml") + + for graph_name, selection_name, package_title, expected_node_count in [ + ( + "decision-graph.yaml", + "decision-graph-selection.yaml", + "Memory Contract Boundary Package", + 3, + ), + ( + "conversation-path.yaml", + "conversation-path-selection.yaml", + "Conversation Episode Package", + 4, + ), + ( + "knowledge-neighborhood.yaml", + "knowledge-neighborhood-selection.yaml", + "Knowledge Neighborhood Package", + 6, + ), + ]: + graph = load_memory_graph_file(examples / graph_name) + selection = load_memory_graph_selection_file(examples / selection_name) + validation = validate_memory_graph(graph, path=examples / graph_name) + package = compile_memory_graph_selection_to_context_package(graph, selection, profile=profile) + + assert validation.valid, validation.to_dict() + assert package.title == package_title + assert package.metadata["memory_graph"]["selected_nodes"] == selection.node_ids + assert len(selection.node_ids) == expected_node_count + assert package.metadata["memory_profile"]["id"] == "local-agent-memory" + + +def test_memory_graph_invalid_example_fixtures_emit_errors(): + examples = Path("examples/memory") + graph = load_memory_graph_file(examples / "invalid-memory-graph.yaml") + profile = load_memory_profile_file(examples / "invalid-memory-profile.yaml") + + graph_result = validate_memory_graph(graph, path=examples / "invalid-memory-graph.yaml") + profile_result = validate_memory_profile(profile, path=examples / "invalid-memory-profile.yaml") + + graph_codes = {diagnostic.code for diagnostic in graph_result.diagnostics} + profile_codes = {diagnostic.code for diagnostic in profile_result.diagnostics} + + assert not graph_result.valid + assert "memory.graph.node.unknown_kind" in graph_codes + assert "memory.graph.edge.unknown_target" in graph_codes + assert "memory.graph.event.unknown_kind" in graph_codes + assert not profile_result.valid + assert "memory.profile.unknown_kind" in profile_codes + assert "memory.profile.store_missing" in profile_codes + + def test_mkt_memory_blueprint_and_graph_cli(tmp_path: Path): _write_profile(tmp_path) _write_graph(tmp_path) diff --git a/workplans/MKTT-WP-0015-render-and-document-function-extensions.md b/workplans/MKTT-WP-0015-render-and-document-function-extensions.md index 7200e7f..9b48082 100644 --- a/workplans/MKTT-WP-0015-render-and-document-function-extensions.md +++ b/workplans/MKTT-WP-0015-render-and-document-function-extensions.md @@ -1,222 +1,221 @@ --- id: MKTT-WP-0015 type: workplan -title: "Render And Document Function Extensions" +title: "Document Function Value Contracts" domain: markitect -status: todo +status: done owner: markitect-tool topic_slug: markitect -planning_priority: P2 +planning_priority: complete planning_order: 130 depends_on_workplans: - MKTT-WP-0010 - MKTT-WP-0011 - MKTT-WP-0012 related_workplans: - - MKTT-WP-0007 - - MKTT-WP-0008 - - MKTT-WP-0009 - MKTT-WP-0013 - - KONT-WP-0017 - - IB-WP-0017 + - MKTT-WP-0020 + - MKTT-WP-0021 + - MKTF-WP-0003 + - MQD-WP-0001 created: "2026-05-04" -updated: "2026-05-04" +updated: "2026-05-15" state_hub_workstream_id: "a38f676a-0d0b-493c-9792-2e34480c3681" --- -# MKTT-WP-0015: Render And Document Function Extensions +# MKTT-WP-0015: Document Function Value Contracts ## Purpose -Capture the natural follow-on work from the Quarkdown comparison and the first -Markitect document-function layer. +Split the original render-and-function extension plan into a small, +Markitect-native core slice: typed document-function values and deterministic +value mapping. -The current function layer is intentionally small: deterministic functions, -Markdown-native explicit syntax, local context variables, diagnostics, -provenance, and capability metadata. This workplan should extend that model -only when the need is concrete, keeping the core framework clean and avoiding a -second workflow engine. +The current document-function layer is intentionally small: deterministic +functions, Markdown-native explicit syntax, local context variables, +diagnostics, provenance, and capability metadata. This workplan should make +function outputs more explicit without turning Markitect into a publishing +language, renderer, or second workflow engine. -## Background +## Split Decision - 2026-05-15 -Quarkdown shows the value of a document language where functions are not just -macros. They can return typed values, Markdown content, layout structures, -tables, dictionaries, booleans, and renderable nodes. Its compiler expands -function-call nodes, maps output values back to renderable nodes, and then -continues through traversal, rendering, and post-rendering stages. +The previous `MKTT-WP-0015` scope bundled typed values, syntax extensions, +document-local reusable functions, render/export adapters, render-aware +references/assets, and permission sandboxing. That was architecturally too +large for one workplan. -Markitect should not become a Quarkdown clone. The better fit is: +The split is now: -- keep Markitect as the contract, reference, processor, workflow, cache, - provenance, and policy framework -- make document functions an authoring surface over those primitives -- add render/export behavior as optional extensions -- use Quarkdown as an optional external publishing target where that is useful +- `MKTT-WP-0015`: typed document-function value contracts in `markitect-tool`. +- `MKTT-WP-0020`: render/export adapter contracts in `markitect-tool`. +- `MKTT-WP-0021`: render reference and asset manifest contracts in + `markitect-tool`. +- `MQD-WP-0001`: concrete Quarkdown render adapter work in + `markitect-quarkdown`. +- `MKTF-WP-0003`: read-side source attachment and asset metadata compatibility + in `markitect-filter`. -## Decision +`markitect-filter` remains a source normalization repository. It should not own +write/export adapters or renderer execution. Concrete Quarkdown invocation +belongs in `markitect-quarkdown`; durable render jobs, publication state, and +artifact storage belong outside `markitect-tool`. -Defer this work until after the current original successor work is stable, -unless a concrete document publishing, render provenance, or function-language -use case becomes urgent. +## Boundary -When picked up, treat this as an extension workplan. It may evolve framework -interfaces, but should not make Quarkdown, flex-auth, network access, live LLM -calls, filesystem writes, or external processes required for deterministic -Markitect parsing and function validation. +`markitect-tool` owns: -Boundary clarification: `markitect-tool` owns typed value contracts, -Markdown-compatible function syntax, render/export adapter interfaces, -provenance envelopes, diagnostics, and deterministic fake renderers. It does -not own durable rendered artifact storage, publication workflows, enterprise -authorization, real renderer operations, or concrete application reports. Those -belong in `kontextual-engine`, optional renderer packages, or -`infospace-bench` application workflows. +- typed document value contracts +- value-to-Markdown and value-to-JSON mapping +- function descriptor output validation +- deterministic diagnostics and provenance for value coercion +- compatibility notes for future syntax extensions -## P15.1 - Typed document values and value mapping +`markitect-tool` does not own in this workplan: + +- nested function expression parsing +- document-local reusable functions +- render/export adapter execution +- Quarkdown invocation +- filesystem-writing render jobs +- publication lifecycle or durable artifact storage +- concrete source-format filtering + +## Implementation Summary - 2026-05-15 + +Implemented the narrowed value-contract slice: + +- `DocumentValue` and `DOCUMENT_VALUE_KINDS`. +- Deterministic coercion and Markdown/JSON mapping helpers: + `coerce_document_value`, `format_document_value`, and + `document_value_to_json`. +- `DocumentFunctionRun.value` while preserving the legacy raw `output` field. +- Descriptor `output_type` validation for string, markdown, dynamic, table, + record, reference, content unit, and related value kinds. +- Diagnostics for output type mismatches and reference/content-unit values that + lack provenance. +- Conservative pipeline depth limit without adding nested function syntax. +- Docs, examples, API exports, extension descriptor metadata, and tests. + +Render/export contracts remain in `MKTT-WP-0020`; render references and asset +manifests remain in `MKTT-WP-0021`; concrete Quarkdown execution remains in +`MQD-WP-0001`; read-side source attachment metadata remains in `MKTF-WP-0003`. + +## P15.1 - Define typed document values ```task id: MKTT-WP-0015-T001 -status: todo +status: done priority: high state_hub_task_id: "995945c5-6cec-435c-8943-b8da0a9ff89d" ``` -Define a typed value model for document functions: +Define a small `DocumentValue` model for function results: -- string, number, boolean, none +- string, number, boolean, and none - Markdown content -- list and dictionary values +- lists and dictionaries +- records and tables - references and content units -- tables and records - diagnostics-friendly unknown or dynamic values -Define how each value maps back to Markdown or structured output. Keep the -mapper deterministic and inspectable. +The model should serialize cleanly and stay independent of renderer-specific +objects. -Output: value model, mapper API, tests, and documentation. +Output: value dataclasses or schema objects, serialization helpers, tests, and +documentation. -## P15.2 - Richer function syntax without losing Markdown compatibility +## P15.2 - Add deterministic value mapping ```task id: MKTT-WP-0015-T002 -status: todo -priority: medium +status: done +priority: high state_hub_task_id: "bfce1388-e123-4e91-a5ab-ba67d21c22b8" ``` -Evaluate syntax extensions that improve author ergonomics without turning -Markitect into a full compiler language: +Define how each `DocumentValue` maps to: -- multiline argument continuation -- nested function expressions -- clearer escaping rules -- block-body argument refinements -- source spans beyond line numbers -- cycle and depth limits for nested calls +- inline Markdown +- block Markdown +- JSON/YAML API output +- diagnostic fallback text for unknown or dynamic values -Output: syntax compatibility note, parser tests, and diagnostics examples. +Mapping must be deterministic, inspectable, and safe to use from +`mkt function render`. -## P15.3 - Document-local reusable functions +Output: mapper API, compatibility tests for existing function outputs, and +diagnostics examples. + +## P15.3 - Validate function descriptor output contracts ```task id: MKTT-WP-0015-T003 -status: todo -priority: medium +status: done +priority: high state_hub_task_id: "a8a8f017-3622-47f1-814e-0c71bd49a42f" ``` -Explore document-local reusable functions as a constrained, contract-aware -extension: +Extend `DocumentFunctionDescriptor` output metadata so functions can declare +their result type using the new value vocabulary. -- named reusable snippets -- parameter lists and default values -- body arguments -- provenance for expansions -- validation against allowed function namespaces +Validation should catch: -Avoid general-purpose Turing-complete scripting in core. If assisted or -external behavior is needed, route it through workflow steps and explicit -capability gates. +- declared output type mismatches +- values that cannot be mapped to Markdown in the requested context +- unstable or dynamic values in deterministic contexts +- missing provenance for reference/content-unit values -Output: design proposal and one deterministic prototype if justified. +Output: descriptor updates, evaluator checks, diagnostics, and tests. -## P15.4 - Quarkdown and render/export adapters +## P15.4 - Pin syntax extension boundaries ```task id: MKTT-WP-0015-T004 -status: todo -priority: high +status: done +priority: medium state_hub_task_id: "69e550a0-188b-4bc4-9658-47219b090904" ``` -Design optional render/export adapter contracts: +Document which syntax improvements are still acceptable without replacing the +current conservative parser: -- emit Quarkdown source from Markitect references, processors, templates, and - function calls -- support output profiles such as plain, docs, slides, paged, and static site -- invoke external renderers only through declared capabilities and fake - deterministic test adapters in this repo -- keep direct code reuse license-safe -- track source to rendered-artifact provenance +- clearer escaping rules +- fenced block-body argument refinements +- line/span reporting improvements +- cycle and depth limits for current pipeline evaluation -Output: adapter interface, Quarkdown export sketch, local policy model, and -tests with deterministic fake renderers. Durable render jobs, stored artifacts, -publication state, and application-specific reports are out of scope here. +Explicitly defer: -## P15.5 - Render-aware references, numbering, and assets +- nested function expressions +- document-local function definitions +- conditionals, loops, lambdas, or Turing-complete scripting +- Quarkdown syntax compatibility as a core parser requirement + +Output: syntax compatibility note and parser diagnostics tests for the retained +scope. + +## P15.5 - Update examples and extension descriptor metadata ```task id: MKTT-WP-0015-T005 -status: todo -priority: high +status: done +priority: medium state_hub_task_id: "53eb9f94-830b-4fdf-bb47-3f549048c82a" ``` -Extend the reference model for rendered documents: +Update document-function docs, examples, and extension descriptor metadata to +show typed outputs without advertising render/export behavior as core function +execution. -- figures, tables, equations, code blocks, and custom numbered units -- generated table of contents and cross-reference links -- static asset manifests -- media checksums and copy policies -- root output asset references - -Output: reference/asset manifest model and docs with examples. - -## P15.6 - Permission sandbox for non-core functions - -```task -id: MKTT-WP-0015-T006 -status: todo -priority: high -state_hub_task_id: "9ef2c516-2cd0-40ba-b270-abefbfd8fc40" -``` - -Add explicit local permission gates for functions that need: - -- filesystem reads or writes -- network access -- external processes -- native content inclusion -- assisted generation -- render/export side effects - -Use Markitect-local policy contracts first. flex-auth, OpenFGA, OPA, Cedar, -Keycloak, Entra, or similar systems may be optional adapters, but must not be -required for deterministic function parsing, validation, and rendering of pure -functions. - -Output: permission vocabulary, denied-operation diagnostics, and policy tests. -This is a syntax-layer capability contract, not a durable authorization service -or audit system. +Output: docs, examples, extension catalog assertions, and generated API/CLI +reference updates if the public surface changes. ## Exit Criteria -- Core deterministic document functions remain simple and dependency-light. -- Richer functions are optional extensions with declared capabilities. -- Render/export adapters can be tested without live external services. -- Quarkdown interoperability is conceptually supported without direct code - dependency. -- Typed values, render provenance, references, and assets have clear contracts. -- The extension does not duplicate the dataflow workflow engine. -- Durable render operations, publication lifecycle, and enterprise policy - enforcement remain outside `markitect-tool`. +- Core deterministic document functions remain dependency-light. +- Function outputs use explicit typed value contracts. +- Existing Markdown rendering behavior remains compatible. +- Renderer-specific objects and Quarkdown invocation are outside this + workplan. +- Richer syntax and reusable local functions are deferred until a concrete use + case justifies a parser/evaluator redesign. diff --git a/workplans/MKTT-WP-0016-agentic-memory-graphs-and-service-blueprints.md b/workplans/MKTT-WP-0016-agentic-memory-graphs-and-service-blueprints.md index f9b57c3..89cbc3d 100644 --- a/workplans/MKTT-WP-0016-agentic-memory-graphs-and-service-blueprints.md +++ b/workplans/MKTT-WP-0016-agentic-memory-graphs-and-service-blueprints.md @@ -3,7 +3,7 @@ id: MKTT-WP-0016 type: workplan title: "Memory Graph Profile Contract And Context Package Compiler" domain: markitect -status: active +status: done owner: markitect-tool topic_slug: markitect planning_priority: P2 @@ -119,9 +119,16 @@ mkt memory graph pack ``` Examples and documentation now live in `docs/memory-graph-contract.md` and -`examples/memory/*graph*.yaml`. The remaining refinement is fixture breadth: -add invalid fixtures and richer conversation/knowledge examples before closing -the workplan fully. +`examples/memory/`. The fixture catalog now covers reasoning decision graphs, +conversation paths, knowledge neighborhoods, mixed profiles, invalid graph and +profile examples, and non-executing runtime adapter handoff descriptors. + +## Closure Update - 2026-05-15 + +This workplan is complete. The final pass added deterministic conversation and +knowledge fixtures, invalid validation fixtures, a runtime adapter boundary +catalog, extension descriptor coverage, and tests that compile the valid graph +selections into `ContextPackage` objects. ## Out Of Scope @@ -216,7 +223,7 @@ retention, compaction, latency, or policy decisions. ```task id: MKTT-WP-0016-T004 -status: in_progress +status: done priority: medium state_hub_task_id: "afcd7ce9-e657-4779-b3b2-0ae3f8e2d66e" ``` @@ -286,7 +293,7 @@ Output: CLI commands, docs, and tests. ```task id: MKTT-WP-0016-T007 -status: in_progress +status: done priority: medium state_hub_task_id: "ff78d449-0738-4f1b-a018-2efed3d8e878" ``` diff --git a/workplans/MKTT-WP-0020-render-export-adapter-contract.md b/workplans/MKTT-WP-0020-render-export-adapter-contract.md new file mode 100644 index 0000000..6e8b7e8 --- /dev/null +++ b/workplans/MKTT-WP-0020-render-export-adapter-contract.md @@ -0,0 +1,174 @@ +--- +id: MKTT-WP-0020 +type: workplan +title: "Render Export Adapter Contract" +domain: markitect +status: todo +owner: markitect-tool +topic_slug: markitect +planning_priority: P2 +planning_order: 150 +depends_on_workplans: + - MKTT-WP-0013 + - MKTT-WP-0015 +related_workplans: + - MKTT-WP-0018 + - MKTT-WP-0021 + - MQD-WP-0001 +created: "2026-05-15" +updated: "2026-05-15" +state_hub_workstream_id: "19d1a377-848a-4156-b712-7b9febd836a6" +--- + +# MKTT-WP-0020: Render Export Adapter Contract + +## Purpose + +Define the `markitect-tool` contract for optional render/export adapters +without implementing real renderer operations in the core toolkit. + +This is the output-side sibling of the source adapter contract. Source +adapters normalize input formats into canonical Markdown; render/export +adapters transform Markitect-compatible Markdown into rendered artifacts or +renderer-specific sources. The two directions must stay separate now that +`markitect-filter` owns concrete source normalization. + +## Boundary + +`markitect-tool` owns: + +- render/export request and result envelopes +- adapter descriptors and extension catalog metadata +- declared output profiles such as plain, docs, slides, paged, static site, + and PDF +- deterministic fake renderers for tests +- diagnostics, provenance, source maps, and capability declarations + +`markitect-tool` does not own: + +- Quarkdown CLI invocation +- Pandoc, browser, PDF, or static-site generator execution +- filesystem-writing publication jobs beyond test fixtures +- durable artifact storage +- renderer-specific dependency installation +- concrete application reports + +Concrete Quarkdown integration belongs in `markitect-quarkdown`. + +## P20.1 - Define render adapter descriptors + +```task +id: MKTT-WP-0020-T001 +status: todo +priority: high +state_hub_task_id: "5b52b196-a7f5-4e4f-abc6-972febdc2638" +``` + +Define a `RenderExportAdapterDescriptor` shape that mirrors the source adapter +descriptor style while remaining output-oriented. + +Descriptor fields should include: + +- stable adapter id +- version and human-readable name +- operations such as `export-source`, `render-artifact`, and `inspect-profile` +- supported input contracts +- output profiles and artifact media types +- option schema +- optional dependencies +- safety and capability metadata +- docs/examples links + +Output: descriptor dataclass or schema, extension descriptor mapping, and +registry tests. + +## P20.2 - Define render request and result envelopes + +```task +id: MKTT-WP-0020-T002 +status: todo +priority: high +state_hub_task_id: "3d43b168-7e55-4885-ae17-fcea262f2641" +``` + +Define service-free request/result types: + +- `RenderExportRequest` +- `RenderExportResult` +- `RenderArtifact` +- `RenderDiagnostic` +- `RenderProvenance` + +Results should describe artifacts and source maps without requiring durable +artifact storage. + +Output: serializable models, round-trip tests, and docs. + +## P20.3 - Add deterministic fake renderer fixtures + +```task +id: MKTT-WP-0020-T003 +status: todo +priority: high +state_hub_task_id: "45748ad7-131c-4b75-9720-6958fde93208" +``` + +Add an in-tree fake renderer that proves the contract without invoking +Quarkdown or any external process. + +The fake renderer should support: + +- profile inspection +- source export into deterministic text +- artifact metadata emission +- source-to-render provenance +- denied-operation diagnostics for disabled capabilities + +Output: fake adapter, tests, and examples. + +## P20.4 - Define capability and policy gates + +```task +id: MKTT-WP-0020-T004 +status: todo +priority: medium +state_hub_task_id: "50aa8f00-eeba-495d-9d45-089f855fc3bd" +``` + +Pin a local capability vocabulary for render/export adapters: + +- filesystem read +- filesystem write +- external process +- network +- native renderer dependency +- assisted generation +- publication side effect + +This is a syntax-layer contract and local denial model, not a durable +authorization service. + +Output: capability constants, blocked-operation diagnostics, and tests. + +## P20.5 - Document Quarkdown handoff + +```task +id: MKTT-WP-0020-T005 +status: todo +priority: medium +state_hub_task_id: "173a75d0-d82c-4dcb-9ced-eb73e9438db2" +``` + +Document how `markitect-quarkdown` should implement a concrete Quarkdown +adapter against this contract. + +Output: handoff note linking `MQD-WP-0001`, example descriptor metadata, and +no direct Quarkdown dependency in `markitect-tool`. + +## Exit Criteria + +- Render/export adapters are discoverable through the extension framework. +- Contract tests pass with a fake deterministic renderer. +- Core Markitect remains usable without renderer dependencies. +- `markitect-filter` remains read-side only. +- Quarkdown interop is supported by contract, not by core dependency. diff --git a/workplans/MKTT-WP-0021-render-reference-asset-manifest.md b/workplans/MKTT-WP-0021-render-reference-asset-manifest.md new file mode 100644 index 0000000..82544da --- /dev/null +++ b/workplans/MKTT-WP-0021-render-reference-asset-manifest.md @@ -0,0 +1,147 @@ +--- +id: MKTT-WP-0021 +type: workplan +title: "Render Reference And Asset Manifest Contract" +domain: markitect +status: todo +owner: markitect-tool +topic_slug: markitect +planning_priority: P2 +planning_order: 155 +depends_on_workplans: + - MKTT-WP-0010 + - MKTT-WP-0015 + - MKTT-WP-0020 +related_workplans: + - MKTT-WP-0018 + - MKTF-WP-0003 + - MQD-WP-0001 +created: "2026-05-15" +updated: "2026-05-15" +state_hub_workstream_id: "c567c1a8-3029-4f9c-9587-e85fede64599" +--- + +# MKTT-WP-0021: Render Reference And Asset Manifest Contract + +## Purpose + +Define passive contracts for render-aware references, numbered units, and +static asset manifests. + +This workplan should make rendered document structure inspectable before and +after an optional renderer runs. It should not implement renderer layout, +final numbering, asset copying, or publication lifecycle. + +## Boundary + +`markitect-tool` owns: + +- stable render unit identities +- references to figures, tables, equations, code blocks, and custom numbered + units +- table-of-contents and cross-reference planning metadata +- static asset manifest fields +- media checksums and copy-policy declarations +- source-to-render provenance envelopes + +Renderer packages own: + +- final numbering and layout +- link rewriting +- asset copying +- output directory conventions +- artifact validation + +`markitect-filter` owns only read-side source asset and attachment metadata +needed by normalized Markdown inputs. + +## P21.1 - Define render unit references + +```task +id: MKTT-WP-0021-T001 +status: todo +priority: high +state_hub_task_id: "3d33d387-633e-4ffb-962e-1a5061d3db01" +``` + +Define stable identities for renderable units: + +- figures +- tables +- equations +- code blocks +- sections +- custom numbered units + +Output: render reference model, serialization tests, and examples. + +## P21.2 - Define cross-reference and TOC planning metadata + +```task +id: MKTT-WP-0021-T002 +status: todo +priority: medium +state_hub_task_id: "4a96e27b-9165-450c-899c-f7af484d9438" +``` + +Represent requested cross-reference links and table-of-contents entries before +renderer-specific numbering is known. + +Output: manifest model and tests that keep final numbering outside core. + +## P21.3 - Define static asset manifests + +```task +id: MKTT-WP-0021-T003 +status: todo +priority: high +state_hub_task_id: "ba917e45-1912-4bdb-bf3f-5946c20957b2" +``` + +Define an asset manifest with: + +- source URI/path +- media type and extension +- digest/checksum +- logical role +- copy policy declaration +- renderer output reference placeholder +- provenance back to source spans or source adapter attachments + +Output: model, examples, and compatibility note for `MKTF-WP-0003`. + +## P21.4 - Define source-to-render provenance maps + +```task +id: MKTT-WP-0021-T004 +status: todo +priority: high +state_hub_task_id: "dd9f1128-af1e-44a8-961b-3aba6104ec9a" +``` + +Define a provenance envelope that maps Markitect source spans and generated +function outputs to renderer-source units and artifact references. + +Output: source map model, fake-renderer fixture integration, and tests. + +## P21.5 - Add docs and examples + +```task +id: MKTT-WP-0021-T005 +status: todo +priority: medium +state_hub_task_id: "1d472c44-d970-4403-9f0b-18e6192da737" +``` + +Add examples showing render references and asset manifests without requiring a +real renderer. + +Output: docs, examples, and extension catalog metadata if needed. + +## Exit Criteria + +- Render-aware references can be represented before renderer execution. +- Asset manifests are deterministic and provenance-preserving. +- Core Markitect does not perform asset copying or final layout numbering. +- `markitect-filter` attachment metadata can feed the manifest without making + `markitect-filter` a renderer.