markitect-tool/workplans/MKTT-WP-0005-runtime-context-and-assessment-engines.md

id, type, title, domain, status, owner, topic_slug, planning_priority, planning_order, depends_on_workplans, related_workplans, created, updated, state_hub_workstream_id

id	type	title	domain	status	owner	topic_slug	planning_priority	planning_order	depends_on_workplans	related_workplans	created	updated	state_hub_workstream_id
MKTT-WP-0005	workplan	Runtime Context, Form, and Assessment Engines	markitect	done	markitect-tool	markitect	complete	70	MKTT-WP-0003 MKTT-WP-0004	MKTT-WP-0011	2026-05-03	2026-05-04	7918687e-2364-46b1-ab7e-65aa77cb8449

MKTT-WP-0005: Runtime Context, Form, and Assessment Engines

Purpose

Turn the contract framework extension points into executable runtime engines: context loading, field prefill, form state evaluation, dynamic rules, and provider-neutral LLM assessment execution.

This workplan picks up the deferred runtime scope from MKTT-WP-0004-practical-contract-framework.md.

Decision

This workplan is implemented. It was picked up after the workflow and internal extension framework work made runtime form/context evaluation useful and cleanly integrable.

The original trigger conditions were:

• We are implementing template/generation flows that need reliable field prefill and pre-render validation.
• We need document checks that depend on case-specific external context.
• Deterministic assertions are no longer enough to assess whether sections do their semantic job.
• A user-facing or agent-facing workflow needs structured form state, defaults, conditional requiredness, or guided repair.

Implemented sequencing:

1. Implement context and form runtime first.
2. Add deterministic context-aware rules.
3. Add provider-neutral assessment execution after the diagnostic/caching boundary was stable.

Background

The deterministic contract framework already supports:

• field declarations
• deterministic section assertions
• metric bands
• provider-neutral rubric declarations as contract vocabulary
• one shared diagnostic model

It does not yet execute:

• context resolvers
• form state evaluation
• dynamic requiredness or visibility
• calculated values
• prefill
• provider-neutral LLM assessment requests
• assessment result caching

P5.1 - Define runtime context model

id: MKTT-WP-0005-T001
status: done
priority: high
state_hub_task_id: "e24e6238-efef-41c4-9f1e-ca677c1be89b"

Define how external context is supplied to contract checks and generation:

• inline YAML/JSON files
• named context objects
• typed context schemas
• explicit source paths
• conflict behavior when frontmatter and context both provide a value
• diagnostic behavior for missing or malformed context

Expected output: design notes and tests for context loading.

P5.2 - Implement context resolver API and CLI input

id: MKTT-WP-0005-T002
status: done
priority: high
state_hub_task_id: "d180bb6d-dae8-4305-88de-64c80b708b8a"

Add a small runtime API and CLI option such as:

mkt contract check <document.md> --contract <contract.md> --context <context.yaml>

Resolvers must be deterministic, local-first, and provider-neutral. Network or application-specific data access belongs in adapters outside the core package.

P5.3 - Implement field prefill and validation runtime

id: MKTT-WP-0005-T003
status: done
priority: high
state_hub_task_id: "b954984a-6f67-4e5b-8744-35e3c4fcc992"

Evaluate field specs against document frontmatter and context:

• required fields
• defaults
• source paths
• enum/pattern/range validation
• type coercion policy
• diagnostics for missing, ambiguous, or conflicting values

Expected utility: contracts become useful before generation, not only after a document exists.

P5.4 - Implement form state model

id: MKTT-WP-0005-T004
status: done
priority: medium
state_hub_task_id: "cccdf868-2308-42a1-b564-8b54fccd3c8b"

Represent form state without binding to a UI framework:

• field id
• value
• defaulted/prefilled/manual/calculated origin
• visible/hidden
• enabled/disabled
• required/optional
• validation diagnostics
• display hints as metadata, not behavior

This should support future UI adapters while remaining useful from the CLI.

P5.5 - Implement dynamic rules

id: MKTT-WP-0005-T005
status: done
priority: medium
state_hub_task_id: "6e420e1e-2465-40d3-8e64-d8681a294e63"

Add deterministic dynamic rules for field and section behavior:

• if / then / else
• requiredness
• visibility
• allowed values
• calculated values
• context-dependent assertions

Keep the expression language intentionally small. Prefer JSON/YAML paths and a small set of operators over embedding a general programming language.

P5.6 - Define LLM assessment execution interface

id: MKTT-WP-0005-T006
status: done
priority: medium
state_hub_task_id: "24b22b3a-e89e-4946-81f4-94f971a11979"

Define provider-neutral request/response models for rubric execution:

• contract id
• rule id
• scope: document, section, or field
• text and structured inputs
• context snapshot
• rubric criteria
• cache key material
• pass/fail
• score
• reason
• model/provider metadata
• diagnostics

Core package should define the protocol and result model, not a provider implementation.

P5.7 - Add assessment runner and cache boundary

id: MKTT-WP-0005-T007
status: done
priority: medium
state_hub_task_id: "b09b77e2-59c0-4d31-b246-685b742d111f"

Implement a runner that can invoke an injected assessment adapter and normalize results into diagnostics.

Add deterministic cache key calculation but keep storage pluggable. The default cache may be local file-based only if it remains transparent and easy to reset.

P5.8 - Add examples and failure diagnostics

id: MKTT-WP-0005-T008
status: done
priority: high
state_hub_task_id: "2efb8233-3154-4824-a898-6fcde37330c5"

Create examples that show the practical value:

• business letter with context-prefilled recipient/sender data
• PRD/FRS with context-dependent product metadata
• workplan where task requirements depend on status
• concept note with LLM rubric declaration and mocked assessment output

Each example should include expected diagnostics for missing context, ambiguous prefill, invalid dynamic rules, and assessment failures.

Open Questions

• Resolved: document/frontmatter values win over context; conflicts are diagnostics and can be escalated per field.
• Resolved: dynamic rules use a smaller Markitect-native vocabulary over JSON/YAML paths instead of embedding JSON Schema conditionals.
• Resolved: provider execution remains external; core defines request/result, runner, diagnostics, and cache boundary only.
• Resolved for core: assessment cache keys include contract/rule/scope/text, criteria, context, structured inputs, provider/model metadata, threshold, and metadata. Persistent cache invalidation remains a backend concern.

Implementation Notes

• Added markitect_tool.runtime with context loading, path resolution, deterministic condition evaluation, form state, dynamic rules, assessment request/result models, runner, and in-memory cache boundary.
• mkt contract check now accepts --context.
• Added mkt contract form-state.
• Workflow contract_check accepts context and workflow form_state exposes runtime field state as a step result.
• Built-in extension descriptors now include runtime.context, runtime.form-state, and runtime.assessment.
• Documentation lives in docs/runtime-context-forms-assessments.md.
• Examples live in examples/runtime/.

Exit Criteria

• Runtime context can be supplied to contract checks.
• Field prefill and validation produce unified diagnostics.
• Form state can be rendered by a future adapter without changing core models.
• Dynamic rules cover common requiredness, visibility, and context assertions.
• LLM rubric execution has a provider-neutral protocol and mocked test adapter.
• Examples demonstrate utility beyond static document validation.