Quarkdown research and integration intent

wiki/MarkitectQuarkdownResearch.md

@@ -0,0 +1,94 @@
+**Short Take**
+Quarkdown is much closer to a programmable publishing/typesetting system than to what `markitect-tool` is becoming. It compiles Markdown-like source into polished HTML/PDF/slides/docs with a Turing-complete function language, layout standard library, media pipeline, live preview, and renderer/post-renderer stack.
+
+`markitect-tool` is better positioned as a document infrastructure and pipeline framework: contracts, validation, extraction, references, processors, literate workflows, cache/index backends, provenance, and future policy/context engines. Quarkdown could fit as an optional output/rendering target or inspiration source, but I would not fold its language model into Markitect core.
+
+**What Quarkdown 2.0 Adds**
+As of the Heise article on April 24, 2026 and the GitHub stable release on April 23, 2026, `v2.0.0` is the latest stable release I’d treat as relevant here. GitHub also shows a “Development build” pre-release, but given our current date of May 4, 2026, I would not base planning on that timestamp.
+
+Main 2.0 changes:
+
+- Permission system for compile-time access control: `project-read`, `global-read`, `network`, `native-content`, `all`, configured with `--allow` / `--deny`.
+- Fully offline HTML output: fonts, code themes, and optional libraries are bundled/copied instead of relying on CDNs, with exceptions for some font cases.
+- Parallel rendering across sibling elements.
+- Lighter media storage: copy by reference and checksum validation to avoid unnecessary copies.
+- Static `public/` assets copied to output root.
+- HTML `baseurl`, canonical links, and `sitemap.xml`.
+- Root output path symbol `@` for shared output-root assets.
+- `.image` function with fine-grained media behavior.
+- Linked `.ref` cross-references for headings, figures, tables, code blocks, equations, and custom numbered blocks.
+- Function-call line continuation and `.keybinding`.
+
+**What Is Novel Compared With Markitect**
+Quarkdown’s big novelty is the integrated authoring-to-rendering loop. It has one language that covers Markdown, functions, variables, loops, conditionals, layout, document metadata, includes, media handling, cross-references, themes, slides, docs, paged output, live preview, and post-rendering.
+
+Markitect currently has parts of this as infrastructure concepts, but not as one renderable authoring language.
+
+Notably novel or stronger in Quarkdown:
+
+- Rich function language inside Markdown.
+- Layout primitives: rows, columns, grids, boxes, alignment, page margins, slides config.
+- Numbering and references as rendered document semantics.
+- First-class media storage and static asset output.
+- Offline HTML packaging.
+- Live preview/compiler workflow.
+- Static-site-ish output support: sitemap, canonical links, `public/`.
+- User-facing document types: `plain`, `paged`, `slides`, `docs`.
+- A full compiler pipeline: lexing, parsing, function expansion, traversal, rendering, post-rendering.
+
+**What Markitect Already Has Conceptually**
+Markitect already has several “deeper infrastructure” equivalents, though often not renderer-facing:
+
+- Includes/transclusion: `mkt include`, reference-backed processor include.
+- References: namespaces, fragments, sections, regions, fenced blocks, tags, line ranges.
+- Cross-reference foundation: content units and IDs exist, though rendered links do not.
+- Processors: deterministic fenced-block processor registry.
+- Literate workflows: `mkt tangle`, `mkt weave`, named chunks.
+- Contracts: document expectations, metrics, fields, assertions, rubrics.
+- Templates/generation: deterministic templates and generation hooks.
+- Provenance: operation and backend provenance envelopes.
+- Cache/backend planning: snapshot identity, refresh planning, dependency invalidation.
+- Permission/policy concept: backend fabric has policy gateways, but not yet a Quarkdown-style execution sandbox.
+
+**Where Quarkdown Could Fit**
+Best fit: optional rendering/export backend.
+
+Possible integration shape:
+
+- `markitect-tool` validates, extracts, composes, resolves references, checks contracts, runs deterministic processors, and emits Quarkdown source.
+- Quarkdown compiles that source to HTML/PDF/slides/docs.
+- Markitect backend tracks Quarkdown source inputs, generated outputs, media dependencies, checksums, and provenance.
+- Markitect contracts could validate “pre-render” structure, while Quarkdown handles final layout/render semantics.
+
+This is cleaner than reimplementing Quarkdown’s renderer. Let Quarkdown be a publishing target; let Markitect remain the contract/pipeline/cache/provenance layer.
+
+**Gaps To Revisit Later**
+High-value gaps inspired by Quarkdown:
+
+- Render-aware cross-reference model: figures, tables, equations, code blocks, custom numbered units.
+- Asset/media pipeline: copied assets, checksums, public directory, root asset references.
+- Output profiles: `plain`, `docs`, `slides`, `paged` as generation targets.
+- Permission sandbox for processors and assisted/LLM execution.
+- Preview/watch mode, probably later and outside core.
+- Function-call style macro layer, but only if kept separate from core Markdown contracts.
+- Document numbering and generated ToC as structured operations.
+- Quarkdown export adapter: `mkt export quarkdown` or a dataflow step in WP-0011.
+- Provenance for rendered outputs: source unit -> generated page/slide/asset.
+
+**Caution**
+Quarkdown is GPL/AGPL-licensed depending on module/binary, so direct code reuse is probably not a good fit unless licensing is deliberately accepted. Conceptual interoperability or invoking it as an external tool is safer.
+
+**Recommendation**
+Do not pivot Markitect toward being “our own Quarkdown.” Instead, add a future workplan or WP-0011/WP-0005 notes for:
+
+1. Quarkdown export adapter.
+2. Render-aware reference/numbering model.
+3. Media/static asset pipeline.
+4. Processor permission sandbox.
+5. Output provenance for rendered artifacts.
+
+That would let Markitect gain Quarkdown’s practical publishing utility without corrupting the elegant infrastructure direction we’ve built.
+
+Sources: [Heise article](https://www.heise.de/en/news/Books-and-slides-from-Markdown-Quarkdown-2-0-is-here-11271511.html), [Quarkdown v2.0.0 release notes](https://github.com/iamgio/quarkdown/releases/tag/v2.0.0), [Quarkdown homepage](https://quarkdown.com/), [function syntax docs](https://quarkdown.com/wiki/syntax-of-a-function-call/), [include docs](https://quarkdown.com/wiki/including-other-quarkdown-files/), [media storage docs](https://quarkdown.com/wiki/media-storage/), [pipeline docs](https://quarkdown.com/wiki/pipeline---function-call-expansion/).
+
+My take: Quarkdown’s function language is a very good inspiration for a future Markitect “document function layer,” especially if we keep it optional, markdown-close, typed enough for contracts, and integrated with our provenance/cache/access-control fabric.

wiki/QuarkdownFunctionLanguage.md

@@ -0,0 +1,117 @@
+This is one of Quarkdown’s most interesting ideas. 
+It is not just “Markdown plus directives”; it is closer to a small document programming language embedded in Markdown.
+
+**Core Idea**
+
+In Quarkdown, function calls are the central extension mechanism. They use a dot-prefixed syntax:
+
+```quarkdown
+.sum {26} {16}
+```
+
+Arguments can be positional or named:
+
+```quarkdown
+.multiply {6} by:{3}
+```
+
+They can be inline, block-level, nested, chained, or given an indented body argument. Quarkdown explicitly presents functions as its key feature and groups them across layout, utility views, math/logical operations, control flow, file data, and document metadata. Source: [Quarkdown function-call syntax](https://quarkdown.com/wiki/syntax-of-a-function-call/).
+
+**What Makes It Rich**
+
+1. **Functions produce document content, not just scalar values**
+
+A function may return text, Markdown content, layout structures, booleans, numbers, tables, etc. This is crucial: document generation and computation live in the same language.
+
+```quarkdown
+.if {.isadult age:{20}}
+    You're an adult!
+```
+
+User-defined functions are declared with `.function`, and their body is a lambda-like block:
+
+```quarkdown
+.function {greet}
+    to from:
+    **Hello, .to** from .from!
+```
+
+Source: [Declaring functions](https://quarkdown.com/wiki/declaring-functions/).
+
+2. **Block arguments make functions feel Markdown-native**
+
+Instead of forcing everything into parenthesized expressions, Quarkdown lets the final argument be an indented block:
+
+```quarkdown
+.box {Example}
+    This is the content of the example.
+```
+
+That is a very elegant move. It keeps prose readable while making structured composition possible.
+
+3. **Chaining gives it pipeline ergonomics**
+
+This:
+
+```quarkdown
+.pow {3} {2}::subtract {1}::sum {2}
+```
+
+is transformed conceptually into nested calls where each result becomes the first argument of the next call. That gives Quarkdown a pleasant Unix-pipe / method-chain feel without turning the surface language into JavaScript. Source: [Chaining calls](https://quarkdown.com/wiki/syntax-of-a-function-call/).
+
+4. **Variables are functions too**
+
+Variables are defined with `.var`, then accessed like zero-argument functions:
+
+```quarkdown
+.var {name} {Quarkdown}
+
+Hello, .name!
+```
+
+They can hold simple values or block/layout content. There is also `.let` for scoped temporary variables. Sources: [Variables](https://quarkdown.com/wiki/variables/) and [Let](https://quarkdown.com/wiki/let/).
+
+5. **It has real control flow**
+
+Quarkdown has conditional functions like `.if` and `.ifnot`, plus loops like `.foreach` and `.repeat`:
+
+```quarkdown
+.foreach {1..3}
+    n:
+    The number is **.n**
+```
+
+The loop returns an iterable of evaluated results, so it behaves more like `map` than a pure side-effect loop. Source: [Loops](https://quarkdown.com/wiki/loops/).
+
+6. **It has higher-order behavior**
+
+Lambdas show up in custom functions, `let`, `foreach`, sorting, filtering, and table operations. Inline lambdas can be passed as arguments, for example in table filtering:
+
+```quarkdown
+.tablefilter {2} {@lambda x: .x::isgreater {20}}
+    ...
+```
+
+Source: [Table manipulation](https://quarkdown.com/wiki/table-manipulation/).
+
+7. **It includes typed-ish document values**
+
+The docs list value types such as string, number, Markdown content, boolean, none, iterable, dictionary, range, lambda, size, color, and dynamic values. It is weak/dynamic enough to remain author-friendly, but structured enough to support meaningful diagnostics and reusable functions.
+
+**Why This Matters For Markitect**
+
+For Markitect, the lesson is not necessarily “copy Quarkdown syntax.” The deeper lesson is this:
+
+Quarkdown treats document operations as composable functions with Markdown-native block bodies.
+
+That maps beautifully to our direction:
+
+- Markitect contracts could define allowed/expected processors as named document functions.
+- Our fenced-block processors could grow into a function registry with capability metadata.
+- Our dataflow pipelines could benefit from function chaining semantics.
+- Our template/generation layer could distinguish pure deterministic functions, context-bound functions, LLM-backed functions, and unsafe external functions.
+- Our diagnostics could be stronger than Quarkdown’s if every function call has provenance, declared inputs, permissions, and expected output type.
+
+My take: Quarkdown’s function language is a very good inspiration for a future Markitect “document function layer,” especially if we keep it optional, markdown-close, typed enough for contracts, and integrated with our provenance/cache/access-control fabric.
+
+