markitect-tool/workplans/MKTT-WP-0012-document-function-layer.md

id, type, title, domain, status, owner, topic_slug, planning_priority, planning_order, depends_on_workplans, related_workplans, created, updated, state_hub_workstream_id

id	type	title	domain	status	owner	topic_slug	planning_priority	planning_order	depends_on_workplans	related_workplans	created	updated	state_hub_workstream_id
MKTT-WP-0012	workplan	Document Function Layer	markitect	done	markitect-tool	markitect	complete	85	MKTT-WP-0004 MKTT-WP-0010 MKTT-WP-0011	MKTT-WP-0005 MKTT-WP-0006 MKTT-WP-0007 MKTT-WP-0009	2026-05-04	2026-05-04	f82e74c4-d349-4e9e-80cd-10c0c6d9545d

MKTT-WP-0012: Document Function Layer

Purpose

Design and implement an optional document function layer inspired by Quarkdown's rich function language, while keeping Markitect markdown-native, contract-aware, provenance-rich, and extension-oriented.

This layer should let authors and agents express reusable document operations as named functions over Markdown content, structured data, references, processors, contracts, workflows, and eventually assisted generation.

Implementation Summary

Implemented the first deterministic document function layer:

• DocumentFunctionDescriptor, DocumentFunctionParameter, DocumentFunctionCall, DocumentFunctionRegistry, run/evaluation result envelopes, diagnostics, provenance, and trace output.
• Conservative inline syntax: {{mkt:function.name ...}}.
• Conservative fenced syntax: mkt-function function.name ....
• Pipeline chaining with |, where the previous result becomes the next function's first argument.
• ProcessingContext.variables bindings through ${name} values.
• Built-in deterministic functions for text operations, Markdown headings, bold text, links, code blocks, and context value lookup.
• mkt function list, mkt function check, and mkt function render.
• Built-in extension descriptor document.function.
• Documentation and examples in docs/document-functions.md and examples/functions/basic-functions.md.

Assisted, filesystem, network, external-process, render/export, and live policy service functions remain future optional extensions gated by local capability and policy metadata.

Background

Quarkdown demonstrates that document authoring can benefit from a compact function language with:

• dot-style function calls
• positional and named arguments
• block-body arguments that remain readable in Markdown
• nested and chained calls
• variables and scoped bindings
• conditionals, loops, lambdas, and table/data operations
• rendering-aware helpers for layout, numbering, references, and document setup

For Markitect, the useful idea is not to clone Quarkdown syntax or compiler behavior. The useful idea is a disciplined function abstraction:

document function = named operation + declared inputs + Markdown-friendly body
                    + output type + capability/policy metadata
                    + provenance + diagnostics

The layer should sit above existing primitives rather than replace them: contracts define expectations, references identify content, processors perform deterministic transformations, workflows orchestrate dataflow, backend fabric tracks identity/provenance, and access policy gates risky operations.

Decision

The deterministic workflow layer, processor/reference stack, local backend, and internal extension framework are now in place. This workplan can be picked up without waiting for flex-auth or any other external authorization service. Treat it as an optional Markitect extension track that unifies processor ergonomics, workflow steps, contract-aware generation, and render/export adapters.

Recommended sequencing:

1. Use practical workflow examples to identify recurring author-facing step patterns.
2. Define the function model as a thin authoring surface over existing processors, references, templates, contracts, workflows, and extension descriptors.
3. Implement pure deterministic functions first.
4. Add assisted, filesystem, external-process, network, or export functions only after local capability and policy metadata are enforced.
5. Keep flex-auth integration optional through policy adapter protocols; do not require a live flex-auth service for function parsing, registration, or deterministic execution.

Current Code Fit

The workplan should build on current Markitect primitives:

• ExtensionDescriptor, ProcessingCapability, ProcessingRequest, ProcessingContext, ProcessingResult, provenance, and trace envelopes.
• Existing processors, references, selectors, JSONPath query engines, templates, contracts, runtime context, form state, and workflow steps.
• AccessPolicyGateway and policy decision envelopes for gating unsafe capabilities.

The function layer should be an authoring surface over these capabilities, not a second workflow engine or a dependency on flex-auth.

P12.1 - Define function model and vocabulary

id: MKTT-WP-0012-T001
status: done
priority: high
state_hub_task_id: "13ddfdbb-8fc1-4570-915d-038d40d489e1"

Define a document function descriptor with:

• stable function id and namespace
• summary and documentation fields
• positional and named parameters
• optional/default parameter behavior
• block-body parameter semantics
• accepted input types and output type
• deterministic/assisted/external execution kind
• capability requirements
• provenance and diagnostic contract

Output: design note, schema, and small examples.

P12.2 - Design Markdown-native call syntax

id: MKTT-WP-0012-T002
status: done
priority: high
state_hub_task_id: "58166792-457a-4844-96a7-27baf50c1d7e"

Evaluate a conservative syntax that stays close to Markdown and does not break CommonMark documents accidentally.

The design should consider:

• inline calls
• block calls with indented body content
• named arguments
• multiline arguments
• escaping/literal behavior
• fenced-block alternatives
• namespace-qualified function names
• diagnostics for ambiguous parses

Output: syntax proposal with accepted/rejected examples and parser impact.

P12.3 - Add function registry and adapter boundary

id: MKTT-WP-0012-T003
status: done
priority: high
state_hub_task_id: "06196bde-cc10-464e-9d1a-6b8acc616c06"

Create a registry that can expose existing Markitect capabilities as functions without moving their core implementation:

• reference resolution
• selectors and extraction
• deterministic processors
• template rendering
• contract validation
• workflow steps
• render/export adapters

Output: registry API, adapter protocol, and tests with fake functions.

P12.4 - Implement pure deterministic function execution

id: MKTT-WP-0012-T004
status: done
priority: high
state_hub_task_id: "986121f0-f824-46eb-af59-65ebf2389f34"

Implement an evaluator for pure deterministic functions only.

It should produce:

• expanded Markdown or structured values
• source spans and provenance envelopes
• typed diagnostics
• cycle/depth limits
• stable execution traces for caching and test fixtures

Output: minimal evaluator and CLI/library tests.

P12.5 - Add chaining and data binding semantics

id: MKTT-WP-0012-T005
status: done
priority: medium
state_hub_task_id: "94bb131c-cb4e-4391-8453-bb9de4f3834c"

Explore pipeline-style chaining where the previous result becomes the first argument of the next function, plus explicit bindings to workflow data products.

This should integrate with MKTT-WP-0011 expressions rather than create a second unrelated expression system.

Output: chaining rules, data binding rules, and diagnostic examples.

P12.6 - Add contract-aware function validation

id: MKTT-WP-0012-T006
status: done
priority: medium
state_hub_task_id: "899d361f-8eaa-4098-97d9-0fd33afc3304"

Allow contracts to declare which functions are permitted, expected, deprecated, or forbidden in a document or workflow.

Validation should cover:

• unknown functions
• wrong arity or argument names
• wrong input/output type
• unsafe capability use
• missing required provenance
• unstable assisted functions in deterministic contexts

Output: contract integration and actionable diagnostics.

P12.7 - Define capability and policy gates

id: MKTT-WP-0012-T007
status: done
priority: medium
state_hub_task_id: "2a51b42c-b46b-42cd-ba33-ab504100e653"

Integrate with the access-control and backend capability vocabulary before any function can read files, access network resources, invoke external processes, or call assisted-generation adapters.

Use Markitect-local AccessPolicyGateway/capability metadata as the required contract. External policy services, including flex-auth, may provide decisions through adapters later but must not be required.

Output: permission model, blocked-operation diagnostics, and policy examples.

P12.8 - Add author-facing examples

id: MKTT-WP-0012-T008
status: done
priority: medium
state_hub_task_id: "30358902-5564-48a2-b1e3-e400bfbe7d1a"

Create examples that show practical value:

• reusable document components
• contract-aware report sections
• Markdown table filtering/summarization
• reference-driven transclusion
• processor pipelines expressed as function chains
• Quarkdown export preparation without taking a dependency on Quarkdown

Output: examples and comparison notes against Quarkdown-inspired patterns.

Exit Criteria

• The function layer is optional and does not disturb existing Markdown parsing or deterministic processors.
• Functions are discoverable, typed enough for diagnostics, and namespace-aware.
• Pure deterministic functions can be executed and tested without external dependencies.
• Function calls preserve source provenance and can be explained.
• Risky capabilities are declared before execution and can be blocked.
• No live flex-auth or other external authorization service is required for deterministic function execution.
• The model composes naturally with workflows, contracts, references, processors, cache backends, and future assisted steps.