markitect-tool/docs/document-functions.md

# Document Function Layer

Date: 2026-05-04

## Purpose

Document functions are a Markdown-native authoring surface over existing
Markitect primitives. They let a document author write small deterministic
operations inline or in fenced blocks while preserving diagnostics,
provenance, trace events, capability metadata, and extension descriptors.

The first implementation is intentionally conservative:

- deterministic functions only
- no network access
- no filesystem access
- no external processes
- no provider or assisted-generation calls
- no live flex-auth or external authorization service required

Riskier functions can be added later as optional adapters once capability and
policy gates are explicit.

## Syntax

Inline calls use:

```markdown
{{mkt:text.upper "draft"}}
```

Fenced block calls use:

````markdown
```mkt-function md.codeblock lang=python
print("hello")
```
````

Names are namespace-qualified. Arguments may be positional or named:

```markdown
{{mkt:md.heading text="Decision" level=2}}
```

Pipeline calls pass the previous result as the first argument of the next
function:

```markdown
{{mkt:text.upper "draft" | text.replace DRAFT Final}}
```

Quoted pipe characters remain literal:

```markdown
{{mkt:text.replace "a|b" "|" "/"}}
```

Values of the form `${name}` are resolved from `ProcessingContext.variables`.
This keeps data binding aligned with workflow expression conventions without
creating a second workflow engine.

## Built-In Functions

Initial deterministic functions:

| Function | Purpose |
| --- | --- |
| `text.upper` | Uppercase text. |
| `text.lower` | Lowercase text. |
| `text.title` | Title-case text. |
| `text.trim` | Trim surrounding whitespace. |
| `text.replace` | Replace text. |
| `text.join` | Join values with an optional separator. |
| `md.heading` | Create a Markdown heading. |
| `md.bold` | Create bold Markdown text. |
| `md.link` | Create a Markdown link. |
| `md.codeblock` | Create a fenced code block. |
| `data.get` | Read a value from processing context variables. |

## Typed Values

Document functions now expose typed result values in addition to their legacy
raw `output` field. Each `DocumentFunctionRun` includes a `value` object with a
stable `kind`, metadata, and optional provenance.

Supported value kinds:

| Kind | Markdown mapping |
| --- | --- |
| `string` | Inline or block text. |
| `number` | Decimal text. |
| `boolean` | `true` or `false`. |
| `none` | Empty text. |
| `markdown` | Markdown content passed through directly. |
| `list` | Comma-separated inline text or newline-separated block text. |
| `dictionary` | Stable JSON object text. |
| `record` | Stable JSON object text. |
| `table` | Deterministic Markdown table. |
| `reference` | Label/title/value text, with provenance required. |
| `content_unit` | Label/title/value text, with provenance required. |
| `unknown` | Diagnostic fallback for mismatched output values. |
| `dynamic` | Reserved for explicitly dynamic values. |

Function descriptors declare `output_type`; execution validates the returned
value against that declaration. Mismatches produce
`function.output_type_mismatch`, while reference-like values without provenance
produce `function.provenance_missing`.

The raw `output` field remains for compatibility. New callers should prefer
`value` for typed API use and use the mapper when Markdown output is needed.

## CLI

List functions:

```text
mkt function list
```

Validate calls without rendering:

```text
mkt function check examples/functions/basic-functions.md
```

Render deterministic calls:

```text
mkt function render examples/functions/basic-functions.md
```

JSON and YAML outputs include calls, diagnostics, provenance, and trace data.

## Registry And Extension Fit

The function layer has its own `DocumentFunctionRegistry`. Functions are
described by `DocumentFunctionDescriptor`:

- stable id and namespace
- parameters
- output type
- execution kind
- capability declarations
- safety metadata
- examples

The built-in extension catalog exposes this layer as `document.function` with
kind `document-function`. This keeps it discoverable without replacing
processors, workflows, references, contracts, templates, or query engines.

## Policy And Capability Gates

The first evaluator blocks non-deterministic functions and supports local
capability blocking through `ProcessingContext.policy`, for example:

```python
ProcessingContext(policy={"blocked_capabilities": ["document_function"]})
```

Future functions that read files, access network resources, invoke external
processes, render exports, or call assisted generation must declare those
capabilities before execution. External policy services may provide decisions
through adapters later, but deterministic function execution has no external
service dependency.

## Syntax Boundary

The supported syntax remains intentionally conservative:

- inline calls with `{{mkt:...}}`
- fenced calls with `mkt-function`
- positional and named arguments parsed with shell-like quoting
- pipeline chaining with quoted pipe characters preserved
- `${name}` context variable lookup
- bounded pipeline depth to avoid accidental runaway expressions

Deferred syntax:

- nested function expressions
- document-local function definitions
- conditionals, loops, lambdas, or general scripting
- Quarkdown syntax compatibility in the core parser

## Natural Extensions

The deterministic layer deliberately stops before becoming a full publishing
language. The original broad render/function follow-up has been split into
native workplans:

- `MKTT-WP-0015`: typed document-function value contracts.
- `MKTT-WP-0020`: render/export adapter contracts.
- `MKTT-WP-0021`: render reference and asset manifest contracts.
- `MQD-WP-0001`: concrete Quarkdown adapter implementation in
  `markitect-quarkdown`.
- `MKTF-WP-0003`: read-side source attachment metadata compatibility in
  `markitect-filter`.

Those workplans should consider:

- typed document values and value-to-Markdown mapping
- constrained parser compatibility improvements, while deferring nested
  function expressions and document-local reusable functions
- render/export adapter contracts, including optional Quarkdown source export
- render-aware numbering, references, tables, figures, equations, and code
  blocks
- static asset and media manifests with checksums
- local permission gates for filesystem, network, external process, assisted,
  and render/export functions

## Design Rules

- Stay close to Markdown and preserve CommonMark documents unless function
  syntax is explicit.
- Keep deterministic execution useful without backends or providers.
- Surface diagnostics instead of silently deleting failed calls.
- Preserve source line information where available.
- Treat functions as an authoring surface over existing capabilities, not as a
  second workflow engine.