--- id: MKTT-WP-0005 type: workplan title: "Runtime Context, Form, and Assessment Engines" domain: markitect status: done owner: markitect-tool topic_slug: markitect planning_priority: complete planning_order: 70 depends_on_workplans: - MKTT-WP-0003 - MKTT-WP-0004 related_workplans: - MKTT-WP-0011 created: "2026-05-03" updated: "2026-05-04" state_hub_workstream_id: "7918687e-2364-46b1-ab7e-65aa77cb8449" --- # MKTT-WP-0005: Runtime Context, Form, and Assessment Engines ## Purpose Turn the contract framework extension points into executable runtime engines: context loading, field prefill, form state evaluation, dynamic rules, and provider-neutral LLM assessment execution. This workplan picks up the deferred runtime scope from `MKTT-WP-0004-practical-contract-framework.md`. ## Decision This workplan is implemented. It was picked up after the workflow and internal extension framework work made runtime form/context evaluation useful and cleanly integrable. The original trigger conditions were: - We are implementing template/generation flows that need reliable field prefill and pre-render validation. - We need document checks that depend on case-specific external context. - Deterministic assertions are no longer enough to assess whether sections do their semantic job. - A user-facing or agent-facing workflow needs structured form state, defaults, conditional requiredness, or guided repair. Implemented sequencing: 1. Implement context and form runtime first. 2. Add deterministic context-aware rules. 3. Add provider-neutral assessment execution after the diagnostic/caching boundary was stable. ## Background The deterministic contract framework already supports: - field declarations - deterministic section assertions - metric bands - provider-neutral rubric declarations as contract vocabulary - one shared diagnostic model It does not yet execute: - context resolvers - form state evaluation - dynamic requiredness or visibility - calculated values - prefill - provider-neutral LLM assessment requests - assessment result caching ## P5.1 - Define runtime context model ```task id: MKTT-WP-0005-T001 status: done priority: high state_hub_task_id: "e24e6238-efef-41c4-9f1e-ca677c1be89b" ``` Define how external context is supplied to contract checks and generation: - inline YAML/JSON files - named context objects - typed context schemas - explicit source paths - conflict behavior when frontmatter and context both provide a value - diagnostic behavior for missing or malformed context Expected output: design notes and tests for context loading. ## P5.2 - Implement context resolver API and CLI input ```task id: MKTT-WP-0005-T002 status: done priority: high state_hub_task_id: "d180bb6d-dae8-4305-88de-64c80b708b8a" ``` Add a small runtime API and CLI option such as: ```text mkt contract check --contract --context ``` Resolvers must be deterministic, local-first, and provider-neutral. Network or application-specific data access belongs in adapters outside the core package. ## P5.3 - Implement field prefill and validation runtime ```task id: MKTT-WP-0005-T003 status: done priority: high state_hub_task_id: "b954984a-6f67-4e5b-8744-35e3c4fcc992" ``` Evaluate field specs against document frontmatter and context: - required fields - defaults - source paths - enum/pattern/range validation - type coercion policy - diagnostics for missing, ambiguous, or conflicting values Expected utility: contracts become useful before generation, not only after a document exists. ## P5.4 - Implement form state model ```task id: MKTT-WP-0005-T004 status: done priority: medium state_hub_task_id: "cccdf868-2308-42a1-b564-8b54fccd3c8b" ``` Represent form state without binding to a UI framework: - field id - value - defaulted/prefilled/manual/calculated origin - visible/hidden - enabled/disabled - required/optional - validation diagnostics - display hints as metadata, not behavior This should support future UI adapters while remaining useful from the CLI. ## P5.5 - Implement dynamic rules ```task id: MKTT-WP-0005-T005 status: done priority: medium state_hub_task_id: "6e420e1e-2465-40d3-8e64-d8681a294e63" ``` Add deterministic dynamic rules for field and section behavior: - `if` / `then` / `else` - requiredness - visibility - allowed values - calculated values - context-dependent assertions Keep the expression language intentionally small. Prefer JSON/YAML paths and a small set of operators over embedding a general programming language. ## P5.6 - Define LLM assessment execution interface ```task id: MKTT-WP-0005-T006 status: done priority: medium state_hub_task_id: "24b22b3a-e89e-4946-81f4-94f971a11979" ``` Define provider-neutral request/response models for rubric execution: - contract id - rule id - scope: document, section, or field - text and structured inputs - context snapshot - rubric criteria - cache key material - pass/fail - score - reason - model/provider metadata - diagnostics Core package should define the protocol and result model, not a provider implementation. ## P5.7 - Add assessment runner and cache boundary ```task id: MKTT-WP-0005-T007 status: done priority: medium state_hub_task_id: "b09b77e2-59c0-4d31-b246-685b742d111f" ``` Implement a runner that can invoke an injected assessment adapter and normalize results into diagnostics. Add deterministic cache key calculation but keep storage pluggable. The default cache may be local file-based only if it remains transparent and easy to reset. ## P5.8 - Add examples and failure diagnostics ```task id: MKTT-WP-0005-T008 status: done priority: high state_hub_task_id: "2efb8233-3154-4824-a898-6fcde37330c5" ``` Create examples that show the practical value: - business letter with context-prefilled recipient/sender data - PRD/FRS with context-dependent product metadata - workplan where task requirements depend on status - concept note with LLM rubric declaration and mocked assessment output Each example should include expected diagnostics for missing context, ambiguous prefill, invalid dynamic rules, and assessment failures. ## Open Questions - Resolved: document/frontmatter values win over context; conflicts are diagnostics and can be escalated per field. - Resolved: dynamic rules use a smaller Markitect-native vocabulary over JSON/YAML paths instead of embedding JSON Schema conditionals. - Resolved: provider execution remains external; core defines request/result, runner, diagnostics, and cache boundary only. - Resolved for core: assessment cache keys include contract/rule/scope/text, criteria, context, structured inputs, provider/model metadata, threshold, and metadata. Persistent cache invalidation remains a backend concern. ## Implementation Notes - Added `markitect_tool.runtime` with context loading, path resolution, deterministic condition evaluation, form state, dynamic rules, assessment request/result models, runner, and in-memory cache boundary. - `mkt contract check` now accepts `--context`. - Added `mkt contract form-state`. - Workflow `contract_check` accepts context and workflow `form_state` exposes runtime field state as a step result. - Built-in extension descriptors now include `runtime.context`, `runtime.form-state`, and `runtime.assessment`. - Documentation lives in `docs/runtime-context-forms-assessments.md`. - Examples live in `examples/runtime/`. ## Exit Criteria - Runtime context can be supplied to contract checks. - Field prefill and validation produce unified diagnostics. - Form state can be rendered by a future adapter without changing core models. - Dynamic rules cover common requiredness, visibility, and context assertions. - LLM rubric execution has a provider-neutral protocol and mocked test adapter. - Examples demonstrate utility beyond static document validation.