generated from coulomb/repo-seed
context loading, path resolution, form state, dynamic rules, and provider-neutral assessment runner/cache boundary
This commit is contained in:
@@ -9,9 +9,9 @@ Markdown as the authoring surface and uses fenced YAML as a structured extension
|
|||||||
for rules that need machine interpretation.
|
for rules that need machine interpretation.
|
||||||
|
|
||||||
The first implementation is deterministic. It checks document type, fields,
|
The first implementation is deterministic. It checks document type, fields,
|
||||||
sections, ordering, metric bands, and text assertions. Forms, context, and LLM
|
sections, ordering, metric bands, and text assertions. Runtime context, forms,
|
||||||
rubrics are represented in the contract vocabulary as extension points before
|
dynamic rules, and provider-neutral assessment requests are implemented as
|
||||||
runtime adapters are added.
|
extensions around the same contract vocabulary.
|
||||||
|
|
||||||
## Contract File Shape
|
## Contract File Shape
|
||||||
|
|
||||||
@@ -112,10 +112,10 @@ fields, and metric bands. This is the bridge to later LLM rubrics: semantic
|
|||||||
checks can become additional assessments without changing how failures are
|
checks can become additional assessments without changing how failures are
|
||||||
reported.
|
reported.
|
||||||
|
|
||||||
## Forms And Context
|
## Forms, Context, And Runtime Rules
|
||||||
|
|
||||||
Field specs are the first step toward form-backed Markdown generation. Runtime
|
Field specs are the foundation for form-backed Markdown generation and
|
||||||
form handling should build on the same field vocabulary:
|
context-aware checks. Runtime form handling uses the same field vocabulary:
|
||||||
|
|
||||||
- `id`
|
- `id`
|
||||||
- `type`
|
- `type`
|
||||||
@@ -128,15 +128,25 @@ form handling should build on the same field vocabulary:
|
|||||||
- `min` / `max`
|
- `min` / `max`
|
||||||
- `min_length` / `max_length`
|
- `min_length` / `max_length`
|
||||||
|
|
||||||
Dynamic requiredness, visibility, calculations, and prefill should be declared
|
Runtime context can be supplied as local YAML or JSON:
|
||||||
as context-aware rules in later work. The contract should remain the source of
|
|
||||||
truth, while UI and generation layers act as adapters.
|
```text
|
||||||
|
mkt contract check <document.md> --contract <contract.md> --context <context.yaml>
|
||||||
|
mkt contract form-state <document.md> --contract <contract.md> --context <context.yaml>
|
||||||
|
```
|
||||||
|
|
||||||
|
The runtime resolves fields in this order: document value, context source,
|
||||||
|
default, missing. Document values win over context and conflicts are diagnostics.
|
||||||
|
Dynamic rules support small deterministic `if` / `then` / `else` expressions
|
||||||
|
for requiredness, visibility, allowed values, calculated values, context
|
||||||
|
assertions, and dynamic section presence. See
|
||||||
|
`docs/runtime-context-forms-assessments.md`.
|
||||||
|
|
||||||
## LLM Assessment Extension
|
## LLM Assessment Extension
|
||||||
|
|
||||||
LLM-assisted checks should be declared as rubrics, scoped to document or section
|
LLM-assisted checks are declared as rubrics, scoped to document or section roles.
|
||||||
roles. Core Markitect should not call a provider directly. A future adapter
|
Core Markitect does not call a provider directly. It creates provider-neutral
|
||||||
should accept a provider-neutral request:
|
assessment requests for injected adapters:
|
||||||
|
|
||||||
- contract id and rule id
|
- contract id and rule id
|
||||||
- document or section text
|
- document or section text
|
||||||
@@ -152,26 +162,11 @@ It should return:
|
|||||||
- model/provider metadata
|
- model/provider metadata
|
||||||
- diagnostics using the shared diagnostic model
|
- diagnostics using the shared diagnostic model
|
||||||
|
|
||||||
## Deferred Runtime Work
|
|
||||||
|
|
||||||
The deterministic contract framework is ready now. The runtime engines are
|
|
||||||
deferred to `MKTT-WP-0005-runtime-context-and-assessment-engines.md`.
|
|
||||||
|
|
||||||
Pick that work up when one of these becomes true:
|
|
||||||
|
|
||||||
- contract checks need external user, project, or entity context
|
|
||||||
- generation needs reliable field prefill before rendering
|
|
||||||
- a UI or agent workflow needs form state, defaults, and dynamic requiredness
|
|
||||||
- deterministic section assertions are not enough and rubric-based semantic
|
|
||||||
assessment becomes necessary
|
|
||||||
|
|
||||||
The intended order is context and form runtime first, deterministic dynamic
|
|
||||||
rules second, LLM assessment execution third.
|
|
||||||
|
|
||||||
## CLI
|
## CLI
|
||||||
|
|
||||||
```text
|
```text
|
||||||
mkt contract validate <contract.md>
|
mkt contract validate <contract.md>
|
||||||
mkt contract check <document.md> --contract <contract.md>
|
mkt contract check <document.md> --contract <contract.md> [--context <context.yaml>]
|
||||||
|
mkt contract form-state <document.md> --contract <contract.md> [--context <context.yaml>]
|
||||||
mkt metrics <document.md>
|
mkt metrics <document.md>
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ framework organizes how Markitect itself exposes and composes capabilities.
|
|||||||
| `backend` | local SQLite index | snapshots/index/search storage |
|
| `backend` | local SQLite index | snapshots/index/search storage |
|
||||||
| `reference-provider` | section, region, fence, line | address in, content units out |
|
| `reference-provider` | section, region, fence, line | address in, content units out |
|
||||||
| `validator` | schema, contract, section assertion | document/context in, diagnostics out |
|
| `validator` | schema, contract, section assertion | document/context in, diagnostics out |
|
||||||
|
| `runtime` | context loader, form state, dynamic rules | document/contract/context in, diagnostics and state out |
|
||||||
|
| `assessment-runner` | provider-neutral rubric execution | assessment request in, normalized result out |
|
||||||
| `template-engine` | deterministic templates | template/data in, Markdown out |
|
| `template-engine` | deterministic templates | template/data in, Markdown out |
|
||||||
| `generation-adapter` | provider-neutral assisted generation | request in, generated candidate out |
|
| `generation-adapter` | provider-neutral assisted generation | request in, generated candidate out |
|
||||||
| `cli-group` | cache, backend, ref, class | command descriptors or registration hook |
|
| `cli-group` | cache, backend, ref, class | command descriptors or registration hook |
|
||||||
|
|||||||
@@ -72,7 +72,8 @@ This makes workflows useful without provider dependencies.
|
|||||||
| `transform` | transformed `markdown`, operations, provenance. |
|
| `transform` | transformed `markdown`, operations, provenance. |
|
||||||
| `include` | include-resolved `markdown`, included paths, provenance. |
|
| `include` | include-resolved `markdown`, included paths, provenance. |
|
||||||
| `contract_stub` | generated contract stub Markdown. |
|
| `contract_stub` | generated contract stub Markdown. |
|
||||||
| `contract_check` | contract diagnostics and metrics. |
|
| `contract_check` | contract diagnostics, metrics, and optional runtime context results. |
|
||||||
|
| `form_state` | field values, origins, dynamic rule results, and diagnostics. |
|
||||||
| `assisted` | generated Markdown if a hook is supplied, otherwise skipped/diagnostic. |
|
| `assisted` | generated Markdown if a hook is supplied, otherwise skipped/diagnostic. |
|
||||||
|
|
||||||
## Data Bindings
|
## Data Bindings
|
||||||
|
|||||||
220
docs/runtime-context-forms-assessments.md
Normal file
220
docs/runtime-context-forms-assessments.md
Normal file
@@ -0,0 +1,220 @@
|
|||||||
|
# Runtime Context, Forms, Rules, And Assessments
|
||||||
|
|
||||||
|
Date: 2026-05-04
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The runtime layer turns contract extension points into executable behavior while
|
||||||
|
keeping the deterministic contract framework intact. Static checks still handle
|
||||||
|
document type, sections, assertions, and metric bands. Runtime checks add
|
||||||
|
external context, field prefill, UI-neutral form state, dynamic rules, and a
|
||||||
|
provider-neutral assessment protocol.
|
||||||
|
|
||||||
|
The layer is deliberately local-first. Core Markitect reads YAML or JSON context
|
||||||
|
files and runs deterministic rules. Network calls, application lookups, and LLM
|
||||||
|
providers belong behind adapters.
|
||||||
|
|
||||||
|
## Context Files
|
||||||
|
|
||||||
|
Runtime context can be a raw YAML/JSON mapping:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
recipient:
|
||||||
|
name: Ada Lovelace
|
||||||
|
sender:
|
||||||
|
name: Markitect Team
|
||||||
|
```
|
||||||
|
|
||||||
|
or an envelope with metadata and schema:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
metadata:
|
||||||
|
case_id: case-42
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [recipient, sender]
|
||||||
|
context:
|
||||||
|
recipient:
|
||||||
|
name: Ada Lovelace
|
||||||
|
sender:
|
||||||
|
name: Markitect Team
|
||||||
|
```
|
||||||
|
|
||||||
|
The value under `context` is bound as `context` in field sources and dynamic
|
||||||
|
rules. `schema` validates the full context object. `schemas` can validate named
|
||||||
|
objects individually.
|
||||||
|
|
||||||
|
Malformed context and schema failures produce normal diagnostics:
|
||||||
|
|
||||||
|
- `runtime.context.malformed`
|
||||||
|
- `runtime.context.schema_invalid`
|
||||||
|
- `runtime.context.schema_target_missing`
|
||||||
|
- `runtime.context.schema`
|
||||||
|
|
||||||
|
## Field Runtime
|
||||||
|
|
||||||
|
Field specs continue to live in the contract:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
fields:
|
||||||
|
recipient_name:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
source: context.recipient.name
|
||||||
|
delivery_channel:
|
||||||
|
type: string
|
||||||
|
default: email
|
||||||
|
enum: [email, print]
|
||||||
|
```
|
||||||
|
|
||||||
|
Runtime resolution order is:
|
||||||
|
|
||||||
|
1. Manual document value from `path`, usually frontmatter.
|
||||||
|
2. Context value from `source` or `sources`.
|
||||||
|
3. Contract `default`.
|
||||||
|
4. Missing.
|
||||||
|
|
||||||
|
Manual document values win over context. If both exist and differ, Markitect
|
||||||
|
emits `runtime.field.conflict` as a warning by default. A field can set
|
||||||
|
`conflict: error` to make that stricter. Multiple context sources with distinct
|
||||||
|
values produce `runtime.field.ambiguous`.
|
||||||
|
|
||||||
|
`mkt contract check` uses runtime evaluation only when `--context` is supplied:
|
||||||
|
|
||||||
|
```text
|
||||||
|
mkt contract check document.md --contract contract.md --context context.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
`mkt contract form-state` always emits the UI-neutral runtime view:
|
||||||
|
|
||||||
|
```text
|
||||||
|
mkt contract form-state document.md --contract contract.md --context context.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Form State
|
||||||
|
|
||||||
|
Form state is not a UI framework. It is a stable contract that future UIs,
|
||||||
|
agents, generators, and workflow steps can render:
|
||||||
|
|
||||||
|
- field id
|
||||||
|
- value
|
||||||
|
- origin: `manual`, `prefilled`, `defaulted`, `calculated`, or `missing`
|
||||||
|
- required/optional
|
||||||
|
- visible/hidden
|
||||||
|
- enabled/disabled
|
||||||
|
- allowed values
|
||||||
|
- diagnostics
|
||||||
|
- metadata
|
||||||
|
|
||||||
|
Hidden fields are not required unless a future adapter explicitly asks for
|
||||||
|
hidden validation. This matches practical form behavior and avoids punishing
|
||||||
|
authors for data that the current context made irrelevant.
|
||||||
|
|
||||||
|
## Dynamic Rules
|
||||||
|
|
||||||
|
Rules are deterministic YAML. They use a deliberately small condition language:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
rules:
|
||||||
|
- id: postal-address-for-print
|
||||||
|
if:
|
||||||
|
path: fields.delivery_channel.value
|
||||||
|
equals: print
|
||||||
|
then:
|
||||||
|
required: [postal_address]
|
||||||
|
visible:
|
||||||
|
postal_address: true
|
||||||
|
else:
|
||||||
|
hidden: [postal_address]
|
||||||
|
```
|
||||||
|
|
||||||
|
Supported condition operators:
|
||||||
|
|
||||||
|
- `exists`
|
||||||
|
- `equals` / `eq`
|
||||||
|
- `not_equals`
|
||||||
|
- `in`
|
||||||
|
- `contains`
|
||||||
|
- `matches`
|
||||||
|
- `gt`, `gte`, `lt`, `lte`
|
||||||
|
- `all`, `any`, `not`
|
||||||
|
|
||||||
|
Supported actions:
|
||||||
|
|
||||||
|
- `required` / `optional`
|
||||||
|
- `visible` / `hidden`
|
||||||
|
- `enabled` / `disabled`
|
||||||
|
- `allowed_values`
|
||||||
|
- `set`
|
||||||
|
- `assert`
|
||||||
|
- `sections`
|
||||||
|
|
||||||
|
Calculated values can reference runtime paths:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
then:
|
||||||
|
set:
|
||||||
|
contact_label: "${fields.sender_name.value} <${context.sender.email}>"
|
||||||
|
```
|
||||||
|
|
||||||
|
Context assertions use the same condition vocabulary:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
assert:
|
||||||
|
path: context.sender.email
|
||||||
|
matches: "@example\\.com$"
|
||||||
|
message: Sender email must come from example.com.
|
||||||
|
severity: warning
|
||||||
|
```
|
||||||
|
|
||||||
|
Dynamic section rules are intentionally narrow. They can require, recommend,
|
||||||
|
discourage, or forbid section specs already declared in the contract.
|
||||||
|
|
||||||
|
## Assessment Protocol
|
||||||
|
|
||||||
|
Rubrics remain provider-neutral contract declarations:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
rubrics:
|
||||||
|
- id: tone-fit
|
||||||
|
scope: section.body
|
||||||
|
criteria: The body should match the recipient relationship.
|
||||||
|
threshold: 0.75
|
||||||
|
```
|
||||||
|
|
||||||
|
Core Markitect turns rubrics into `AssessmentRequest` objects and normalizes
|
||||||
|
adapter results into `AssessmentResult` and diagnostics. It does not call an LLM
|
||||||
|
provider directly. The cache key includes contract id, rule id, scope, text,
|
||||||
|
criteria, context, structured inputs, threshold, provider, model, and metadata.
|
||||||
|
|
||||||
|
Adapters can be injected from workflows, applications, or tests. A transparent
|
||||||
|
in-memory cache exists for tests and short runs; persistent storage remains a
|
||||||
|
backend concern.
|
||||||
|
|
||||||
|
## Workflow Integration
|
||||||
|
|
||||||
|
Workflow `contract_check` steps accept `context`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- id: check-letter
|
||||||
|
kind: contract_check
|
||||||
|
document: letter.md
|
||||||
|
contract: letter.contract.md
|
||||||
|
context: letter.context.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Workflow `form_state` steps expose the runtime state as a step result:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
steps:
|
||||||
|
- id: form
|
||||||
|
kind: form_state
|
||||||
|
document: letter.md
|
||||||
|
contract: letter.contract.md
|
||||||
|
context: letter.context.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
This keeps workflow orchestration separate from the runtime engine. The runtime
|
||||||
|
engine answers "what does this contract imply in this context"; the workflow
|
||||||
|
engine decides when to run it and where to send the output.
|
||||||
@@ -188,7 +188,8 @@ First implementation step kinds:
|
|||||||
| `transform` | Apply deterministic Markdown transforms. |
|
| `transform` | Apply deterministic Markdown transforms. |
|
||||||
| `include` | Resolve include markers in Markdown. |
|
| `include` | Resolve include markers in Markdown. |
|
||||||
| `contract_stub` | Generate a Markdown stub from a contract. |
|
| `contract_stub` | Generate a Markdown stub from a contract. |
|
||||||
| `contract_check` | Check a Markdown document against a contract. |
|
| `contract_check` | Check a Markdown document against a contract, optionally with runtime context. |
|
||||||
|
| `form_state` | Evaluate UI-neutral field prefill, validation, and dynamic rules. |
|
||||||
| `assisted` | Provider-neutral assisted step boundary, optional by default. |
|
| `assisted` | Provider-neutral assisted step boundary, optional by default. |
|
||||||
|
|
||||||
## Data Bindings
|
## Data Bindings
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ and descriptions mirror the operational view.
|
|||||||
| `MKTT-WP-0010` | complete | done | `MKTT-WP-0004`; task-level trigger: `MKTT-WP-0003-T006` | Content references, processors, explode/implode, weave/tangle, content classes, and migration examples are complete as the first WP-0010 extension layer. |
|
| `MKTT-WP-0010` | complete | done | `MKTT-WP-0004`; task-level trigger: `MKTT-WP-0003-T006` | Content references, processors, explode/implode, weave/tangle, content classes, and migration examples are complete as the first WP-0010 extension layer. |
|
||||||
| `MKTT-WP-0007` | complete | done | `MKTT-WP-0006` | Advanced query and local index backend is complete: AST inspection, optional JSONPath, SQLite snapshots/metadata, FTS5 search, incremental refresh, and local index CLI. |
|
| `MKTT-WP-0007` | complete | done | `MKTT-WP-0006` | Advanced query and local index backend is complete: AST inspection, optional JSONPath, SQLite snapshots/metadata, FTS5 search, incremental refresh, and local index CLI. |
|
||||||
| `MKTT-WP-0013` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0004`, `MKTT-WP-0006`, `MKTT-WP-0007`, `MKTT-WP-0010` | Internal extension framework is complete: characterization tests, canonical processing model, descriptors, registries, lifecycle callbacks, query-engine registry, built-in extension catalog, CLI command specs, and authoring guide. |
|
| `MKTT-WP-0013` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0004`, `MKTT-WP-0006`, `MKTT-WP-0007`, `MKTT-WP-0010` | Internal extension framework is complete: characterization tests, canonical processing model, descriptors, registries, lifecycle callbacks, query-engine registry, built-in extension catalog, CLI command specs, and authoring guide. |
|
||||||
| `MKTT-WP-0005` | P2 | todo | `MKTT-WP-0003`, `MKTT-WP-0004` | Pick up when generation/form/context or semantic assessment pressure appears. |
|
| `MKTT-WP-0005` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0004` | Runtime context, form state, dynamic rules, workflow integration, and provider-neutral assessment boundary are complete. |
|
||||||
| `MKTT-WP-0011` | complete | done | `MKTT-WP-0003`; task-level triggers: `MKTT-WP-0010-T001`, `MKTT-WP-0010-T005` | Markdown dataflow workflow layer is complete: workflow standard, source collectors, binding model, deterministic steps, assisted boundary, safe outputs, CLI, docs, and examples. |
|
| `MKTT-WP-0011` | complete | done | `MKTT-WP-0003`; task-level triggers: `MKTT-WP-0010-T001`, `MKTT-WP-0010-T005` | Markdown dataflow workflow layer is complete: workflow standard, source collectors, binding model, deterministic steps, assisted boundary, safe outputs, CLI, docs, and examples. |
|
||||||
| `MKTT-WP-0009` | P2 | todo | `MKTT-WP-0006` | Establish access-control gateway before security-sensitive cache/context use. |
|
| `MKTT-WP-0009` | P2 | todo | `MKTT-WP-0006` | Establish access-control gateway before security-sensitive cache/context use. |
|
||||||
| `MKTT-WP-0012` | P3 | todo | `MKTT-WP-0004`, `MKTT-WP-0010`, `MKTT-WP-0011` | Future Quarkdown-inspired document function layer: reusable Markdown-native function calls over processors, references, contracts, workflows, and later assisted steps. |
|
| `MKTT-WP-0012` | P3 | todo | `MKTT-WP-0004`, `MKTT-WP-0010`, `MKTT-WP-0011` | Future Quarkdown-inspired document function layer: reusable Markdown-native function calls over processors, references, contracts, workflows, and later assisted steps. |
|
||||||
|
|||||||
23
examples/runtime/business-letter-prefill.md
Normal file
23
examples/runtime/business-letter-prefill.md
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
---
|
||||||
|
document_type: business-letter
|
||||||
|
---
|
||||||
|
|
||||||
|
# Follow-Up Letter
|
||||||
|
|
||||||
|
## Greeting
|
||||||
|
|
||||||
|
Dear Ada Lovelace,
|
||||||
|
|
||||||
|
## Body
|
||||||
|
|
||||||
|
Thank you for the thoughtful discussion about structured Markdown generation.
|
||||||
|
We reviewed the requirements and will send a concise proposal that separates
|
||||||
|
document contracts, field prefill, validation diagnostics, and optional semantic
|
||||||
|
assessment. This keeps the implementation practical while leaving room for
|
||||||
|
future automation.
|
||||||
|
|
||||||
|
## Closing
|
||||||
|
|
||||||
|
Kind regards,
|
||||||
|
|
||||||
|
Markitect Team
|
||||||
19
examples/runtime/business-letter.context.yaml
Normal file
19
examples/runtime/business-letter.context.yaml
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
metadata:
|
||||||
|
case_id: letter-ada-001
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [recipient, sender]
|
||||||
|
properties:
|
||||||
|
recipient:
|
||||||
|
type: object
|
||||||
|
required: [name]
|
||||||
|
sender:
|
||||||
|
type: object
|
||||||
|
required: [name, email]
|
||||||
|
context:
|
||||||
|
recipient:
|
||||||
|
name: Ada Lovelace
|
||||||
|
relationship: research collaborator
|
||||||
|
sender:
|
||||||
|
name: Markitect Team
|
||||||
|
email: hello@example.com
|
||||||
27
examples/runtime/concept-note-assessment.contract.md
Normal file
27
examples/runtime/concept-note-assessment.contract.md
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
# Concept Note Assessment Contract
|
||||||
|
|
||||||
|
```yaml contract
|
||||||
|
id: concept-note-assessment-v1
|
||||||
|
document:
|
||||||
|
type: concept-note
|
||||||
|
fields:
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
enum: [draft, review, accepted]
|
||||||
|
sections:
|
||||||
|
- id: concept
|
||||||
|
title: Concept
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
- id: utility
|
||||||
|
title: Utility
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
rubrics:
|
||||||
|
- id: utility-is-practical
|
||||||
|
scope: section.utility
|
||||||
|
criteria: The utility section should explain who benefits, what changes in practice, and how success can be recognized.
|
||||||
|
threshold: 0.75
|
||||||
|
severity: warning
|
||||||
|
```
|
||||||
13
examples/runtime/prd-frs.context.yaml
Normal file
13
examples/runtime/prd-frs.context.yaml
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
metadata:
|
||||||
|
release: runtime-context-alpha
|
||||||
|
context:
|
||||||
|
product:
|
||||||
|
name: Markitect Tool
|
||||||
|
owner: Platform Architecture
|
||||||
|
regulatory_tier: internal
|
||||||
|
project:
|
||||||
|
target_date: "2026-06-30"
|
||||||
|
stakeholders:
|
||||||
|
- Documentation
|
||||||
|
- Architecture
|
||||||
|
- Product Operations
|
||||||
25
examples/runtime/runtime-expected-diagnostics.md
Normal file
25
examples/runtime/runtime-expected-diagnostics.md
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
# Runtime Expected Diagnostics
|
||||||
|
|
||||||
|
These examples are meant for manual inspection and future snapshot fixtures.
|
||||||
|
|
||||||
|
```text
|
||||||
|
mkt contract check examples/runtime/workplan-done-missing-decision.md \
|
||||||
|
--contract examples/runtime/workplan-dynamic.contract.md \
|
||||||
|
--format text
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected diagnostic:
|
||||||
|
|
||||||
|
- `runtime.section.missing`: status `done` requires a `Decision Point` section.
|
||||||
|
|
||||||
|
```text
|
||||||
|
mkt contract check examples/runtime/business-letter-prefill.md \
|
||||||
|
--contract examples/contracts/business-letter.contract.md \
|
||||||
|
--context examples/runtime/business-letter.context.yaml \
|
||||||
|
--format text
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected runtime utility:
|
||||||
|
|
||||||
|
- `recipient_name` and `sender_name` are prefilled from context.
|
||||||
|
- No field-missing diagnostic is emitted for those fields.
|
||||||
18
examples/runtime/workplan-done-missing-decision.md
Normal file
18
examples/runtime/workplan-done-missing-decision.md
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
---
|
||||||
|
document_type: workplan
|
||||||
|
id: MKTT-WP-EXAMPLE
|
||||||
|
status: done
|
||||||
|
---
|
||||||
|
|
||||||
|
# Example Workplan
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Show how a dynamic section rule reports missing completion context.
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: MKTT-WP-EXAMPLE-T001
|
||||||
|
status: done
|
||||||
|
```
|
||||||
46
examples/runtime/workplan-dynamic.contract.md
Normal file
46
examples/runtime/workplan-dynamic.contract.md
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
# Dynamic Workplan Contract
|
||||||
|
|
||||||
|
```yaml contract
|
||||||
|
id: dynamic-workplan-contract-v1
|
||||||
|
document:
|
||||||
|
type: workplan
|
||||||
|
fields:
|
||||||
|
id:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
enum: [proposed, active, done, deferred]
|
||||||
|
owner:
|
||||||
|
type: string
|
||||||
|
source: context.workplan.owner
|
||||||
|
sections:
|
||||||
|
- id: purpose
|
||||||
|
title: Purpose
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
- id: tasks
|
||||||
|
title: Tasks
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
- id: decision-point
|
||||||
|
title: Decision Point
|
||||||
|
presence: optional
|
||||||
|
level: 2
|
||||||
|
rules:
|
||||||
|
- id: require-decision-point-when-done
|
||||||
|
if:
|
||||||
|
path: fields.status.value
|
||||||
|
equals: done
|
||||||
|
then:
|
||||||
|
sections:
|
||||||
|
decision-point:
|
||||||
|
presence: required
|
||||||
|
- id: owner-needed-for-active-work
|
||||||
|
if:
|
||||||
|
path: fields.status.value
|
||||||
|
equals: active
|
||||||
|
then:
|
||||||
|
required: [owner]
|
||||||
|
```
|
||||||
@@ -140,6 +140,23 @@ from markitect_tool.reference import (
|
|||||||
parse_reference,
|
parse_reference,
|
||||||
resolve_reference,
|
resolve_reference,
|
||||||
)
|
)
|
||||||
|
from markitect_tool.runtime import (
|
||||||
|
AssessmentRequest,
|
||||||
|
AssessmentResult,
|
||||||
|
AssessmentRunResult,
|
||||||
|
AssessmentRunner,
|
||||||
|
FieldState,
|
||||||
|
FormState,
|
||||||
|
MemoryAssessmentCache,
|
||||||
|
RuntimeContext,
|
||||||
|
RuntimeContextLoadResult,
|
||||||
|
RuntimeContextSource,
|
||||||
|
assessment_requests_for_contract,
|
||||||
|
evaluate_form_state,
|
||||||
|
load_runtime_context_file,
|
||||||
|
load_runtime_context_file_result,
|
||||||
|
run_contract_assessments,
|
||||||
|
)
|
||||||
from markitect_tool.schema import (
|
from markitect_tool.schema import (
|
||||||
MarkdownSchema,
|
MarkdownSchema,
|
||||||
SchemaValidationResult,
|
SchemaValidationResult,
|
||||||
@@ -290,6 +307,21 @@ __all__ = [
|
|||||||
"load_namespaces",
|
"load_namespaces",
|
||||||
"parse_reference",
|
"parse_reference",
|
||||||
"resolve_reference",
|
"resolve_reference",
|
||||||
|
"AssessmentRequest",
|
||||||
|
"AssessmentResult",
|
||||||
|
"AssessmentRunResult",
|
||||||
|
"AssessmentRunner",
|
||||||
|
"FieldState",
|
||||||
|
"FormState",
|
||||||
|
"MemoryAssessmentCache",
|
||||||
|
"RuntimeContext",
|
||||||
|
"RuntimeContextLoadResult",
|
||||||
|
"RuntimeContextSource",
|
||||||
|
"assessment_requests_for_contract",
|
||||||
|
"evaluate_form_state",
|
||||||
|
"load_runtime_context_file",
|
||||||
|
"load_runtime_context_file_result",
|
||||||
|
"run_contract_assessments",
|
||||||
"MissingTemplateVariable",
|
"MissingTemplateVariable",
|
||||||
"TemplateAnalysis",
|
"TemplateAnalysis",
|
||||||
"TemplateError",
|
"TemplateError",
|
||||||
|
|||||||
@@ -65,6 +65,7 @@ from markitect_tool.reference import (
|
|||||||
load_namespaces,
|
load_namespaces,
|
||||||
resolve_reference,
|
resolve_reference,
|
||||||
)
|
)
|
||||||
|
from markitect_tool.runtime import evaluate_form_state, load_runtime_context_file
|
||||||
from markitect_tool.schema import load_schema_file, validate_markdown_file, validate_schema
|
from markitect_tool.schema import load_schema_file, validate_markdown_file, validate_schema
|
||||||
from markitect_tool.template import (
|
from markitect_tool.template import (
|
||||||
MissingTemplateVariable,
|
MissingTemplateVariable,
|
||||||
@@ -1466,17 +1467,68 @@ def contract_validate(contract_file: Path, output_format: str) -> None:
|
|||||||
default="text",
|
default="text",
|
||||||
show_default=True,
|
show_default=True,
|
||||||
)
|
)
|
||||||
def contract_check(file: Path, contract_file: Path, output_format: str) -> None:
|
@click.option(
|
||||||
|
"--context",
|
||||||
|
"context_file",
|
||||||
|
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||||
|
help="YAML or JSON runtime context used for field prefill and dynamic rules.",
|
||||||
|
)
|
||||||
|
def contract_check(
|
||||||
|
file: Path,
|
||||||
|
contract_file: Path,
|
||||||
|
output_format: str,
|
||||||
|
context_file: Path | None,
|
||||||
|
) -> None:
|
||||||
"""Check a Markdown file against a Markdown document contract."""
|
"""Check a Markdown file against a Markdown document contract."""
|
||||||
|
|
||||||
try:
|
try:
|
||||||
result = check_markdown_file(file, contract_file)
|
result = check_markdown_file(file, contract_file, context_path=context_file)
|
||||||
except ContractLoaderError as exc:
|
except ContractLoaderError as exc:
|
||||||
raise click.ClickException(str(exc)) from exc
|
raise click.ClickException(str(exc)) from exc
|
||||||
_emit_diagnostic_result(result.to_dict(), output_format)
|
_emit_diagnostic_result(result.to_dict(), output_format)
|
||||||
raise click.exceptions.Exit(0 if result.valid else 1)
|
raise click.exceptions.Exit(0 if result.valid else 1)
|
||||||
|
|
||||||
|
|
||||||
|
@contract.command("form-state")
|
||||||
|
@click.argument("file", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||||
|
@click.option(
|
||||||
|
"--contract",
|
||||||
|
"contract_file",
|
||||||
|
required=True,
|
||||||
|
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||||
|
)
|
||||||
|
@click.option(
|
||||||
|
"--context",
|
||||||
|
"context_file",
|
||||||
|
type=click.Path(exists=True, dir_okay=False, path_type=Path),
|
||||||
|
help="YAML or JSON runtime context used for field prefill and dynamic rules.",
|
||||||
|
)
|
||||||
|
@click.option(
|
||||||
|
"--format",
|
||||||
|
"output_format",
|
||||||
|
type=click.Choice(["json", "yaml", "text"], case_sensitive=False),
|
||||||
|
default="text",
|
||||||
|
show_default=True,
|
||||||
|
)
|
||||||
|
def contract_form_state(
|
||||||
|
file: Path,
|
||||||
|
contract_file: Path,
|
||||||
|
context_file: Path | None,
|
||||||
|
output_format: str,
|
||||||
|
) -> None:
|
||||||
|
"""Evaluate UI-neutral form state for a document contract."""
|
||||||
|
|
||||||
|
try:
|
||||||
|
document = parse_markdown_file(file)
|
||||||
|
contract_definition = load_contract_file(contract_file)
|
||||||
|
context = load_runtime_context_file(context_file) if context_file else None
|
||||||
|
form_state = evaluate_form_state(document, contract_definition, context)
|
||||||
|
except ContractLoaderError as exc:
|
||||||
|
raise click.ClickException(str(exc)) from exc
|
||||||
|
_emit_form_state(form_state.to_dict(), output_format)
|
||||||
|
raise click.exceptions.Exit(0 if form_state.valid else 1)
|
||||||
|
|
||||||
|
|
||||||
def _emit_result(data: dict, output_format: str) -> None:
|
def _emit_result(data: dict, output_format: str) -> None:
|
||||||
if output_format == "json":
|
if output_format == "json":
|
||||||
click.echo(json.dumps(data, indent=2, ensure_ascii=False))
|
click.echo(json.dumps(data, indent=2, ensure_ascii=False))
|
||||||
@@ -1511,6 +1563,31 @@ def _emit_diagnostic_result(data: dict, output_format: str) -> None:
|
|||||||
click.echo(f" guidance: {diagnostic['guidance']}")
|
click.echo(f" guidance: {diagnostic['guidance']}")
|
||||||
|
|
||||||
|
|
||||||
|
def _emit_form_state(data: dict, output_format: str) -> None:
|
||||||
|
if output_format == "json":
|
||||||
|
click.echo(json.dumps(data, indent=2, ensure_ascii=False))
|
||||||
|
elif output_format == "yaml":
|
||||||
|
click.echo(yaml.safe_dump(data, sort_keys=False))
|
||||||
|
else:
|
||||||
|
click.echo("valid" if data.get("valid") else "invalid")
|
||||||
|
for field in data.get("fields", []):
|
||||||
|
value = field.get("value", "<missing>") if field.get("exists") else "<missing>"
|
||||||
|
flags = []
|
||||||
|
if field.get("required"):
|
||||||
|
flags.append("required")
|
||||||
|
if field.get("visible") is False:
|
||||||
|
flags.append("hidden")
|
||||||
|
if field.get("enabled") is False:
|
||||||
|
flags.append("disabled")
|
||||||
|
suffix = f" ({', '.join(flags)})" if flags else ""
|
||||||
|
click.echo(f"- {field['id']}: {value} [{field.get('origin', 'unknown')}]{suffix}")
|
||||||
|
for diagnostic in data.get("diagnostics", []):
|
||||||
|
click.echo(
|
||||||
|
f" [{diagnostic['severity']}] {diagnostic['code']}: "
|
||||||
|
f"{diagnostic['message']}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _emit_metrics(data: dict, output_format: str) -> None:
|
def _emit_metrics(data: dict, output_format: str) -> None:
|
||||||
if output_format == "json":
|
if output_format == "json":
|
||||||
click.echo(json.dumps(data, indent=2, ensure_ascii=False))
|
click.echo(json.dumps(data, indent=2, ensure_ascii=False))
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ from markitect_tool.diagnostics import (
|
|||||||
has_error,
|
has_error,
|
||||||
valid_severity,
|
valid_severity,
|
||||||
)
|
)
|
||||||
|
from markitect_tool.runtime import RuntimeContext, evaluate_form_state, load_runtime_context_file
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -55,6 +56,7 @@ class ContractCheckResult:
|
|||||||
document_path: str | None = None
|
document_path: str | None = None
|
||||||
contract_path: str | None = None
|
contract_path: str | None = None
|
||||||
metrics: dict[str, Any] = field(default_factory=dict)
|
metrics: dict[str, Any] = field(default_factory=dict)
|
||||||
|
runtime: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
def to_dict(self) -> dict[str, Any]:
|
def to_dict(self) -> dict[str, Any]:
|
||||||
data = {
|
data = {
|
||||||
@@ -63,6 +65,7 @@ class ContractCheckResult:
|
|||||||
"document_path": self.document_path,
|
"document_path": self.document_path,
|
||||||
"contract_path": self.contract_path,
|
"contract_path": self.contract_path,
|
||||||
"metrics": self.metrics or None,
|
"metrics": self.metrics or None,
|
||||||
|
"runtime": self.runtime or None,
|
||||||
}
|
}
|
||||||
return {key: value for key, value in data.items() if value is not None}
|
return {key: value for key, value in data.items() if value is not None}
|
||||||
|
|
||||||
@@ -131,26 +134,48 @@ def validate_contract(contract: DocumentContract) -> ContractValidationResult:
|
|||||||
|
|
||||||
|
|
||||||
def check_markdown_file(
|
def check_markdown_file(
|
||||||
markdown_path: str | Path, contract_path: str | Path
|
markdown_path: str | Path,
|
||||||
|
contract_path: str | Path,
|
||||||
|
*,
|
||||||
|
context_path: str | Path | None = None,
|
||||||
|
runtime_context: RuntimeContext | None = None,
|
||||||
) -> ContractCheckResult:
|
) -> ContractCheckResult:
|
||||||
"""Parse and check a Markdown file against a contract file."""
|
"""Parse and check a Markdown file against a contract file."""
|
||||||
|
|
||||||
document = parse_markdown_file(markdown_path)
|
document = parse_markdown_file(markdown_path)
|
||||||
contract = load_contract_file(contract_path)
|
contract = load_contract_file(contract_path)
|
||||||
return check_document_contract(document, contract)
|
context = runtime_context
|
||||||
|
if context_path is not None:
|
||||||
|
context = load_runtime_context_file(context_path)
|
||||||
|
return check_document_contract(document, contract, runtime_context=context)
|
||||||
|
|
||||||
|
|
||||||
def check_document_contract(
|
def check_document_contract(
|
||||||
document: Document, contract: DocumentContract
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
*,
|
||||||
|
runtime_context: RuntimeContext | None = None,
|
||||||
) -> ContractCheckResult:
|
) -> ContractCheckResult:
|
||||||
"""Check a parsed Markdown document against a document contract."""
|
"""Check a parsed Markdown document against a document contract."""
|
||||||
|
|
||||||
contract_validation = validate_contract(contract)
|
contract_validation = validate_contract(contract)
|
||||||
document_metrics = collect_metrics(document)
|
document_metrics = collect_metrics(document)
|
||||||
diagnostics = list(contract_validation.diagnostics)
|
diagnostics = list(contract_validation.diagnostics)
|
||||||
|
runtime: dict[str, Any] = {}
|
||||||
if contract_validation.valid:
|
if contract_validation.valid:
|
||||||
diagnostics.extend(_check_document_type(document, contract))
|
diagnostics.extend(_check_document_type(document, contract))
|
||||||
diagnostics.extend(_check_fields(document, contract))
|
if runtime_context is None and not contract.rules:
|
||||||
|
diagnostics.extend(_check_fields(document, contract))
|
||||||
|
else:
|
||||||
|
form_state = evaluate_form_state(
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
runtime_context or RuntimeContext.empty(),
|
||||||
|
)
|
||||||
|
diagnostics.extend(form_state.diagnostics)
|
||||||
|
context = runtime_context or RuntimeContext.empty()
|
||||||
|
runtime["context"] = context.to_dict()
|
||||||
|
runtime["form_state"] = form_state.to_dict()
|
||||||
diagnostics.extend(_check_document_metrics(document, contract, document_metrics))
|
diagnostics.extend(_check_document_metrics(document, contract, document_metrics))
|
||||||
diagnostics.extend(_check_assertions(document.body, contract.assertions, document, contract))
|
diagnostics.extend(_check_assertions(document.body, contract.assertions, document, contract))
|
||||||
diagnostics.extend(_check_sections(document, contract, document_metrics))
|
diagnostics.extend(_check_sections(document, contract, document_metrics))
|
||||||
@@ -161,6 +186,7 @@ def check_document_contract(
|
|||||||
document_path=document.source_path,
|
document_path=document.source_path,
|
||||||
contract_path=contract.source_path,
|
contract_path=contract.source_path,
|
||||||
metrics=document_metrics.to_dict(),
|
metrics=document_metrics.to_dict(),
|
||||||
|
runtime=runtime,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -120,6 +120,7 @@ def _looks_like_contract(data: dict[str, Any]) -> bool:
|
|||||||
"metric_bands",
|
"metric_bands",
|
||||||
"assertions",
|
"assertions",
|
||||||
"forms",
|
"forms",
|
||||||
|
"rules",
|
||||||
"rubrics",
|
"rubrics",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -223,6 +223,7 @@ class DocumentContract:
|
|||||||
assertions: list[AssertionSpec] = field(default_factory=list)
|
assertions: list[AssertionSpec] = field(default_factory=list)
|
||||||
forms: list[dict[str, Any]] = field(default_factory=list)
|
forms: list[dict[str, Any]] = field(default_factory=list)
|
||||||
context: dict[str, Any] = field(default_factory=dict)
|
context: dict[str, Any] = field(default_factory=dict)
|
||||||
|
rules: list[dict[str, Any]] = field(default_factory=list)
|
||||||
rubrics: list[dict[str, Any]] = field(default_factory=list)
|
rubrics: list[dict[str, Any]] = field(default_factory=list)
|
||||||
metadata: dict[str, Any] = field(default_factory=dict)
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
raw: dict[str, Any] = field(default_factory=dict)
|
raw: dict[str, Any] = field(default_factory=dict)
|
||||||
@@ -264,6 +265,7 @@ class DocumentContract:
|
|||||||
assertions=assertions_from_value(raw.get("assertions")),
|
assertions=assertions_from_value(raw.get("assertions")),
|
||||||
forms=raw.get("forms") if isinstance(raw.get("forms"), list) else [],
|
forms=raw.get("forms") if isinstance(raw.get("forms"), list) else [],
|
||||||
context=raw.get("context") if isinstance(raw.get("context"), dict) else {},
|
context=raw.get("context") if isinstance(raw.get("context"), dict) else {},
|
||||||
|
rules=raw.get("rules") if isinstance(raw.get("rules"), list) else [],
|
||||||
rubrics=raw.get("rubrics") if isinstance(raw.get("rubrics"), list) else [],
|
rubrics=raw.get("rubrics") if isinstance(raw.get("rubrics"), list) else [],
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
raw=raw,
|
raw=raw,
|
||||||
@@ -284,6 +286,7 @@ class DocumentContract:
|
|||||||
"assertions": [assertion.raw for assertion in self.assertions],
|
"assertions": [assertion.raw for assertion in self.assertions],
|
||||||
"forms": self.forms,
|
"forms": self.forms,
|
||||||
"context": self.context,
|
"context": self.context,
|
||||||
|
"rules": self.rules,
|
||||||
"rubrics": self.rubrics,
|
"rubrics": self.rubrics,
|
||||||
"source_path": self.source_path,
|
"source_path": self.source_path,
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,6 +14,9 @@ def builtin_extension_registry() -> ExtensionRegistry:
|
|||||||
for descriptor in _processor_descriptors() + [
|
for descriptor in _processor_descriptors() + [
|
||||||
_local_sqlite_backend_descriptor(),
|
_local_sqlite_backend_descriptor(),
|
||||||
_workflow_engine_descriptor(),
|
_workflow_engine_descriptor(),
|
||||||
|
_runtime_context_descriptor(),
|
||||||
|
_runtime_form_state_descriptor(),
|
||||||
|
_runtime_assessment_descriptor(),
|
||||||
]:
|
]:
|
||||||
registry.register(descriptor)
|
registry.register(descriptor)
|
||||||
return registry
|
return registry
|
||||||
@@ -120,3 +123,68 @@ def _workflow_engine_descriptor() -> ExtensionDescriptor:
|
|||||||
docs=["docs/workflow-definition-standard.md"],
|
docs=["docs/workflow-definition-standard.md"],
|
||||||
examples=["examples/workflows/adr-release-notes.workflow.md"],
|
examples=["examples/workflows/adr-release-notes.workflow.md"],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime_context_descriptor() -> ExtensionDescriptor:
|
||||||
|
return ExtensionDescriptor(
|
||||||
|
id="runtime.context",
|
||||||
|
kind="runtime",
|
||||||
|
summary="YAML/JSON runtime context loader for document contracts.",
|
||||||
|
capabilities=[
|
||||||
|
ProcessingCapability(id="context", kind="read"),
|
||||||
|
ProcessingCapability(id="json-schema", kind="validate"),
|
||||||
|
ProcessingCapability(id="diagnostics", kind="emit"),
|
||||||
|
],
|
||||||
|
safety={"reads_files": True, "network": False},
|
||||||
|
input_contract="YAML/JSON runtime context file",
|
||||||
|
output_contract="RuntimeContext",
|
||||||
|
diagnostics_namespace="runtime.context",
|
||||||
|
provenance_prefix="runtime.context",
|
||||||
|
cli={"commands": ["mkt contract check --context", "mkt contract form-state"]},
|
||||||
|
docs=["docs/runtime-context-forms-assessments.md"],
|
||||||
|
examples=["examples/runtime/business-letter.context.yaml"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime_form_state_descriptor() -> ExtensionDescriptor:
|
||||||
|
return ExtensionDescriptor(
|
||||||
|
id="runtime.form-state",
|
||||||
|
kind="runtime",
|
||||||
|
summary="UI-neutral field prefill, validation, and dynamic rule engine.",
|
||||||
|
capabilities=[
|
||||||
|
ProcessingCapability(id="forms", kind="evaluate"),
|
||||||
|
ProcessingCapability(id="rules", kind="evaluate"),
|
||||||
|
ProcessingCapability(id="contracts", kind="validate"),
|
||||||
|
ProcessingCapability(id="diagnostics", kind="emit"),
|
||||||
|
],
|
||||||
|
safety={"reads_files": False, "network": False},
|
||||||
|
input_contract="Document + DocumentContract + RuntimeContext",
|
||||||
|
output_contract="FormState",
|
||||||
|
diagnostics_namespace="runtime",
|
||||||
|
provenance_prefix="runtime.form_state",
|
||||||
|
cli={"commands": ["mkt contract form-state"]},
|
||||||
|
docs=["docs/runtime-context-forms-assessments.md"],
|
||||||
|
examples=["examples/runtime/workplan-dynamic.contract.md"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _runtime_assessment_descriptor() -> ExtensionDescriptor:
|
||||||
|
return ExtensionDescriptor(
|
||||||
|
id="runtime.assessment",
|
||||||
|
kind="assessment-runner",
|
||||||
|
summary="Provider-neutral rubric assessment request, result, and cache boundary.",
|
||||||
|
capabilities=[
|
||||||
|
ProcessingCapability(id="assessment", kind="execute"),
|
||||||
|
ProcessingCapability(id="rubrics", kind="read"),
|
||||||
|
ProcessingCapability(id="cache-key", kind="compute"),
|
||||||
|
ProcessingCapability(id="diagnostics", kind="emit"),
|
||||||
|
],
|
||||||
|
safety={"network": "adapter-only", "provider_calls": "adapter-only"},
|
||||||
|
input_contract="AssessmentRequest",
|
||||||
|
output_contract="AssessmentResult",
|
||||||
|
diagnostics_namespace="runtime.assessment",
|
||||||
|
provenance_prefix="runtime.assessment",
|
||||||
|
docs=["docs/runtime-context-forms-assessments.md"],
|
||||||
|
examples=["examples/runtime/concept-note-assessment.contract.md"],
|
||||||
|
metadata={"provider_implementation": "external adapter required"},
|
||||||
|
)
|
||||||
|
|||||||
53
src/markitect_tool/runtime/__init__.py
Normal file
53
src/markitect_tool/runtime/__init__.py
Normal file
@@ -0,0 +1,53 @@
|
|||||||
|
"""Runtime context, form state, rules, and assessment extension APIs."""
|
||||||
|
|
||||||
|
from markitect_tool.runtime.assessment import (
|
||||||
|
AssessmentAdapter,
|
||||||
|
AssessmentCache,
|
||||||
|
AssessmentRequest,
|
||||||
|
AssessmentResult,
|
||||||
|
AssessmentRunResult,
|
||||||
|
AssessmentRunner,
|
||||||
|
MemoryAssessmentCache,
|
||||||
|
assessment_requests_for_contract,
|
||||||
|
run_contract_assessments,
|
||||||
|
)
|
||||||
|
from markitect_tool.runtime.context import (
|
||||||
|
RuntimeContext,
|
||||||
|
RuntimeContextLoadResult,
|
||||||
|
RuntimeContextSource,
|
||||||
|
load_runtime_context_file,
|
||||||
|
load_runtime_context_file_result,
|
||||||
|
)
|
||||||
|
from markitect_tool.runtime.forms import (
|
||||||
|
FieldState,
|
||||||
|
FormState,
|
||||||
|
build_runtime_bindings,
|
||||||
|
evaluate_form_state,
|
||||||
|
)
|
||||||
|
from markitect_tool.runtime.paths import comparable_value, resolve_path
|
||||||
|
from markitect_tool.runtime.rules import ConditionResult, evaluate_condition
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AssessmentAdapter",
|
||||||
|
"AssessmentCache",
|
||||||
|
"AssessmentRequest",
|
||||||
|
"AssessmentResult",
|
||||||
|
"AssessmentRunResult",
|
||||||
|
"AssessmentRunner",
|
||||||
|
"ConditionResult",
|
||||||
|
"FieldState",
|
||||||
|
"FormState",
|
||||||
|
"MemoryAssessmentCache",
|
||||||
|
"RuntimeContext",
|
||||||
|
"RuntimeContextLoadResult",
|
||||||
|
"RuntimeContextSource",
|
||||||
|
"assessment_requests_for_contract",
|
||||||
|
"build_runtime_bindings",
|
||||||
|
"comparable_value",
|
||||||
|
"evaluate_condition",
|
||||||
|
"evaluate_form_state",
|
||||||
|
"load_runtime_context_file",
|
||||||
|
"load_runtime_context_file_result",
|
||||||
|
"resolve_path",
|
||||||
|
"run_contract_assessments",
|
||||||
|
]
|
||||||
309
src/markitect_tool/runtime/assessment.py
Normal file
309
src/markitect_tool/runtime/assessment.py
Normal file
@@ -0,0 +1,309 @@
|
|||||||
|
"""Provider-neutral assessment protocol for rubric-backed contract checks."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from typing import Any, Protocol
|
||||||
|
|
||||||
|
from markitect_tool.contract.model import DocumentContract
|
||||||
|
from markitect_tool.core import Document, Section
|
||||||
|
from markitect_tool.diagnostics import Diagnostic, SourceLocation, has_error
|
||||||
|
from markitect_tool.runtime.context import RuntimeContext
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AssessmentRequest:
|
||||||
|
"""A provider-neutral request for rubric assessment."""
|
||||||
|
|
||||||
|
contract_id: str | None
|
||||||
|
rule_id: str
|
||||||
|
scope: str
|
||||||
|
text: str
|
||||||
|
criteria: Any
|
||||||
|
context: dict[str, Any] = field(default_factory=dict)
|
||||||
|
structured_inputs: dict[str, Any] = field(default_factory=dict)
|
||||||
|
severity: str = "error"
|
||||||
|
threshold: float | None = None
|
||||||
|
provider: str | None = None
|
||||||
|
model: str | None = None
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def cache_key(self) -> str:
|
||||||
|
payload = {
|
||||||
|
"contract_id": self.contract_id,
|
||||||
|
"rule_id": self.rule_id,
|
||||||
|
"scope": self.scope,
|
||||||
|
"text": self.text,
|
||||||
|
"criteria": self.criteria,
|
||||||
|
"context": self.context,
|
||||||
|
"structured_inputs": self.structured_inputs,
|
||||||
|
"threshold": self.threshold,
|
||||||
|
"provider": self.provider,
|
||||||
|
"model": self.model,
|
||||||
|
"metadata": self.metadata,
|
||||||
|
}
|
||||||
|
return "assessment:" + hashlib.sha256(
|
||||||
|
json.dumps(payload, sort_keys=True, ensure_ascii=False, default=str).encode("utf-8")
|
||||||
|
).hexdigest()
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = asdict(self)
|
||||||
|
data["cache_key"] = self.cache_key
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AssessmentResult:
|
||||||
|
"""Normalized result returned by an assessment adapter."""
|
||||||
|
|
||||||
|
rule_id: str
|
||||||
|
passed: bool
|
||||||
|
score: float | None = None
|
||||||
|
reason: str | None = None
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
provider: str | None = None
|
||||||
|
model: str | None = None
|
||||||
|
cache_key: str | None = None
|
||||||
|
cached: bool = False
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return self.passed and not has_error(self.diagnostics)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = {
|
||||||
|
"rule_id": self.rule_id,
|
||||||
|
"passed": self.passed,
|
||||||
|
"score": self.score,
|
||||||
|
"reason": self.reason,
|
||||||
|
"diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics],
|
||||||
|
"provider": self.provider,
|
||||||
|
"model": self.model,
|
||||||
|
"cache_key": self.cache_key,
|
||||||
|
"cached": self.cached,
|
||||||
|
"metadata": self.metadata,
|
||||||
|
}
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
|
||||||
|
class AssessmentAdapter(Protocol):
|
||||||
|
"""Adapter boundary for an LLM or other semantic grader."""
|
||||||
|
|
||||||
|
def assess(self, request: AssessmentRequest) -> AssessmentResult | dict[str, Any]:
|
||||||
|
"""Assess a request and return a normalized result or mapping."""
|
||||||
|
|
||||||
|
|
||||||
|
class AssessmentCache(Protocol):
|
||||||
|
"""Minimal pluggable assessment cache."""
|
||||||
|
|
||||||
|
def get(self, cache_key: str) -> AssessmentResult | None:
|
||||||
|
"""Return a cached assessment result if available."""
|
||||||
|
|
||||||
|
def set(self, cache_key: str, result: AssessmentResult) -> None:
|
||||||
|
"""Store an assessment result."""
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryAssessmentCache:
|
||||||
|
"""Transparent in-memory cache useful for tests and short workflow runs."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._results: dict[str, AssessmentResult] = {}
|
||||||
|
|
||||||
|
def get(self, cache_key: str) -> AssessmentResult | None:
|
||||||
|
return self._results.get(cache_key)
|
||||||
|
|
||||||
|
def set(self, cache_key: str, result: AssessmentResult) -> None:
|
||||||
|
self._results[cache_key] = result
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AssessmentRunResult:
|
||||||
|
"""Result of executing one or more rubric assessments."""
|
||||||
|
|
||||||
|
assessments: list[AssessmentResult] = field(default_factory=list)
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return not has_error(self.diagnostics)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = {
|
||||||
|
"valid": self.valid,
|
||||||
|
"assessments": [assessment.to_dict() for assessment in self.assessments],
|
||||||
|
"diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics],
|
||||||
|
}
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
|
||||||
|
class AssessmentRunner:
|
||||||
|
"""Invoke an injected assessment adapter and normalize diagnostics."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
adapter: AssessmentAdapter,
|
||||||
|
*,
|
||||||
|
cache: AssessmentCache | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.adapter = adapter
|
||||||
|
self.cache = cache
|
||||||
|
|
||||||
|
def assess(self, request: AssessmentRequest) -> AssessmentResult:
|
||||||
|
if self.cache:
|
||||||
|
cached = self.cache.get(request.cache_key)
|
||||||
|
if cached:
|
||||||
|
return replace(cached, cached=True)
|
||||||
|
|
||||||
|
raw_result = self.adapter.assess(request)
|
||||||
|
result = _normalize_assessment_result(raw_result, request)
|
||||||
|
if self.cache:
|
||||||
|
self.cache.set(request.cache_key, result)
|
||||||
|
return result
|
||||||
|
|
||||||
|
def assess_all(self, requests: list[AssessmentRequest]) -> AssessmentRunResult:
|
||||||
|
assessments = [self.assess(request) for request in requests]
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
for request, assessment in zip(requests, assessments, strict=True):
|
||||||
|
diagnostics.extend(_diagnostics_for_result(request, assessment))
|
||||||
|
return AssessmentRunResult(assessments=assessments, diagnostics=diagnostics)
|
||||||
|
|
||||||
|
|
||||||
|
def assessment_requests_for_contract(
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
runtime_context: RuntimeContext | None = None,
|
||||||
|
) -> list[AssessmentRequest]:
|
||||||
|
"""Create assessment requests for contract rubric declarations."""
|
||||||
|
|
||||||
|
context = runtime_context or RuntimeContext.empty()
|
||||||
|
requests: list[AssessmentRequest] = []
|
||||||
|
for index, rubric in enumerate(contract.rubrics):
|
||||||
|
if not isinstance(rubric, dict):
|
||||||
|
continue
|
||||||
|
rule_id = str(rubric.get("id") or f"rubric-{index + 1}")
|
||||||
|
scope = str(rubric.get("scope") or "document")
|
||||||
|
text = _text_for_scope(document, contract, scope)
|
||||||
|
requests.append(
|
||||||
|
AssessmentRequest(
|
||||||
|
contract_id=contract.id,
|
||||||
|
rule_id=rule_id,
|
||||||
|
scope=scope,
|
||||||
|
text=text,
|
||||||
|
criteria=rubric.get("criteria") or rubric.get("prompt") or rubric,
|
||||||
|
context=context.binding(),
|
||||||
|
structured_inputs={
|
||||||
|
"frontmatter": document.frontmatter,
|
||||||
|
"contract": contract.to_dict(),
|
||||||
|
},
|
||||||
|
severity=str(rubric.get("severity", "error")),
|
||||||
|
threshold=rubric.get("threshold"),
|
||||||
|
provider=rubric.get("provider"),
|
||||||
|
model=rubric.get("model"),
|
||||||
|
metadata={"rubric": rubric},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return requests
|
||||||
|
|
||||||
|
|
||||||
|
def run_contract_assessments(
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
adapter: AssessmentAdapter,
|
||||||
|
*,
|
||||||
|
runtime_context: RuntimeContext | None = None,
|
||||||
|
cache: AssessmentCache | None = None,
|
||||||
|
) -> AssessmentRunResult:
|
||||||
|
"""Run all rubrics in a contract with an injected assessment adapter."""
|
||||||
|
|
||||||
|
runner = AssessmentRunner(adapter, cache=cache)
|
||||||
|
return runner.assess_all(
|
||||||
|
assessment_requests_for_contract(document, contract, runtime_context)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_assessment_result(
|
||||||
|
raw_result: AssessmentResult | dict[str, Any],
|
||||||
|
request: AssessmentRequest,
|
||||||
|
) -> AssessmentResult:
|
||||||
|
if isinstance(raw_result, AssessmentResult):
|
||||||
|
result = raw_result
|
||||||
|
else:
|
||||||
|
result = AssessmentResult(
|
||||||
|
rule_id=str(raw_result.get("rule_id") or request.rule_id),
|
||||||
|
passed=bool(raw_result.get("passed")),
|
||||||
|
score=raw_result.get("score"),
|
||||||
|
reason=raw_result.get("reason"),
|
||||||
|
diagnostics=list(raw_result.get("diagnostics", [])),
|
||||||
|
provider=raw_result.get("provider"),
|
||||||
|
model=raw_result.get("model"),
|
||||||
|
metadata=raw_result.get("metadata", {}),
|
||||||
|
)
|
||||||
|
return replace(
|
||||||
|
result,
|
||||||
|
rule_id=result.rule_id or request.rule_id,
|
||||||
|
cache_key=request.cache_key,
|
||||||
|
provider=result.provider or request.provider,
|
||||||
|
model=result.model or request.model,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _diagnostics_for_result(
|
||||||
|
request: AssessmentRequest, assessment: AssessmentResult
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
diagnostics = list(assessment.diagnostics)
|
||||||
|
if not assessment.passed:
|
||||||
|
diagnostics.append(
|
||||||
|
Diagnostic(
|
||||||
|
severity=request.severity,
|
||||||
|
code="runtime.assessment.failed",
|
||||||
|
message=assessment.reason or f"Assessment `{request.rule_id}` failed.",
|
||||||
|
rule_id=request.rule_id,
|
||||||
|
details={
|
||||||
|
"scope": request.scope,
|
||||||
|
"score": assessment.score,
|
||||||
|
"threshold": request.threshold,
|
||||||
|
"provider": assessment.provider,
|
||||||
|
"model": assessment.model,
|
||||||
|
"cache_key": assessment.cache_key,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _text_for_scope(document: Document, contract: DocumentContract, scope: str) -> str:
|
||||||
|
if scope == "document":
|
||||||
|
return document.body
|
||||||
|
if scope.startswith("section."):
|
||||||
|
section_id = scope.split(".", 1)[1]
|
||||||
|
for section_spec in contract.sections:
|
||||||
|
if section_spec.id != section_id:
|
||||||
|
continue
|
||||||
|
section = _matching_section(document, section_spec.headings)
|
||||||
|
if section:
|
||||||
|
return "\n".join(block.text for block in section.blocks if block.text)
|
||||||
|
if scope.startswith("field."):
|
||||||
|
field_id = scope.split(".", 1)[1]
|
||||||
|
value = document.frontmatter.get(field_id)
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
return document.body
|
||||||
|
|
||||||
|
|
||||||
|
def _matching_section(document: Document, headings: list[str]) -> Section | None:
|
||||||
|
expected = {heading.strip().lower() for heading in headings}
|
||||||
|
for section in document.sections:
|
||||||
|
if section.heading.text.strip().lower() in expected:
|
||||||
|
return section
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_empty(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
key: value
|
||||||
|
for key, value in data.items()
|
||||||
|
if value not in (None, [], {}, "")
|
||||||
|
}
|
||||||
297
src/markitect_tool/runtime/context.py
Normal file
297
src/markitect_tool/runtime/context.py
Normal file
@@ -0,0 +1,297 @@
|
|||||||
|
"""Runtime context loading and validation for contract execution."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from dataclasses import asdict, dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
from jsonschema import Draft202012Validator, SchemaError, ValidationError
|
||||||
|
|
||||||
|
from markitect_tool.diagnostics import Diagnostic, SourceLocation, has_error
|
||||||
|
|
||||||
|
|
||||||
|
CONTEXT_RESERVED_KEYS = {"context", "metadata", "schema", "schemas", "sources"}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RuntimeContextSource:
|
||||||
|
"""Origin metadata for one loaded context object."""
|
||||||
|
|
||||||
|
name: str | None = None
|
||||||
|
path: str | None = None
|
||||||
|
kind: str = "file"
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
return _drop_empty(asdict(self))
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RuntimeContext:
|
||||||
|
"""Named external data available to contract checks and generation."""
|
||||||
|
|
||||||
|
data: dict[str, Any] = field(default_factory=dict)
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
source_path: str | None = None
|
||||||
|
sources: list[RuntimeContextSource] = field(default_factory=list)
|
||||||
|
schemas: dict[str, Any] = field(default_factory=dict)
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return not has_error(self.diagnostics)
|
||||||
|
|
||||||
|
def binding(self) -> dict[str, Any]:
|
||||||
|
"""Return the object bound at `context` in runtime expressions."""
|
||||||
|
|
||||||
|
return self.data
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = {
|
||||||
|
"valid": self.valid,
|
||||||
|
"source_path": self.source_path,
|
||||||
|
"data": self.data,
|
||||||
|
"metadata": self.metadata,
|
||||||
|
"sources": [source.to_dict() for source in self.sources],
|
||||||
|
"schemas": self.schemas,
|
||||||
|
"diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics],
|
||||||
|
}
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def empty(cls) -> "RuntimeContext":
|
||||||
|
return cls()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_mapping(
|
||||||
|
cls,
|
||||||
|
raw: dict[str, Any],
|
||||||
|
*,
|
||||||
|
source_path: str | None = None,
|
||||||
|
) -> "RuntimeContext":
|
||||||
|
if "context" in raw:
|
||||||
|
context_data = raw.get("context") or {}
|
||||||
|
if not isinstance(context_data, dict):
|
||||||
|
return cls(
|
||||||
|
source_path=source_path,
|
||||||
|
diagnostics=[
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.invalid",
|
||||||
|
"`context` must be a mapping.",
|
||||||
|
source_path=source_path,
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
context_data = {
|
||||||
|
key: value for key, value in raw.items() if key not in CONTEXT_RESERVED_KEYS
|
||||||
|
}
|
||||||
|
|
||||||
|
metadata = raw.get("metadata") if isinstance(raw.get("metadata"), dict) else {}
|
||||||
|
schemas: dict[str, Any] = {}
|
||||||
|
if isinstance(raw.get("schema"), dict):
|
||||||
|
schemas["$context"] = raw["schema"]
|
||||||
|
if isinstance(raw.get("schemas"), dict):
|
||||||
|
schemas.update(raw["schemas"])
|
||||||
|
|
||||||
|
sources = _sources_from_raw(raw.get("sources"), source_path)
|
||||||
|
diagnostics = _validate_context_schemas(context_data, schemas, source_path)
|
||||||
|
return cls(
|
||||||
|
data=context_data,
|
||||||
|
metadata=metadata,
|
||||||
|
source_path=source_path,
|
||||||
|
sources=sources,
|
||||||
|
schemas=schemas,
|
||||||
|
diagnostics=diagnostics,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RuntimeContextLoadResult:
|
||||||
|
"""Context load result that can carry diagnostics instead of raising."""
|
||||||
|
|
||||||
|
context: RuntimeContext
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return self.context.valid
|
||||||
|
|
||||||
|
@property
|
||||||
|
def diagnostics(self) -> list[Diagnostic]:
|
||||||
|
return self.context.diagnostics
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
return self.context.to_dict()
|
||||||
|
|
||||||
|
|
||||||
|
def load_runtime_context_file(path: str | Path) -> RuntimeContext:
|
||||||
|
"""Load a runtime context from JSON or YAML."""
|
||||||
|
|
||||||
|
return load_runtime_context_file_result(path).context
|
||||||
|
|
||||||
|
|
||||||
|
def load_runtime_context_file_result(path: str | Path) -> RuntimeContextLoadResult:
|
||||||
|
"""Load a runtime context and represent malformed input as diagnostics."""
|
||||||
|
|
||||||
|
context_path = Path(path)
|
||||||
|
try:
|
||||||
|
text = context_path.read_text(encoding="utf-8")
|
||||||
|
except OSError as exc:
|
||||||
|
return RuntimeContextLoadResult(
|
||||||
|
RuntimeContext(
|
||||||
|
source_path=str(context_path),
|
||||||
|
diagnostics=[
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.read_failed",
|
||||||
|
f"Cannot read runtime context: {exc}",
|
||||||
|
source_path=str(context_path),
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
raw = _load_mapping(text, context_path)
|
||||||
|
except ValueError as exc:
|
||||||
|
return RuntimeContextLoadResult(
|
||||||
|
RuntimeContext(
|
||||||
|
source_path=str(context_path),
|
||||||
|
diagnostics=[
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.malformed",
|
||||||
|
str(exc),
|
||||||
|
source_path=str(context_path),
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return RuntimeContextLoadResult(
|
||||||
|
RuntimeContext.from_mapping(raw, source_path=str(context_path))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_mapping(text: str, path: Path) -> dict[str, Any]:
|
||||||
|
suffix = path.suffix.lower()
|
||||||
|
try:
|
||||||
|
if suffix == ".json":
|
||||||
|
data = json.loads(text) if text.strip() else {}
|
||||||
|
else:
|
||||||
|
data = yaml.safe_load(text) if text.strip() else {}
|
||||||
|
except (json.JSONDecodeError, yaml.YAMLError) as exc:
|
||||||
|
raise ValueError(f"Invalid context file `{path}`: {exc}") from exc
|
||||||
|
if data is None:
|
||||||
|
data = {}
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError("Runtime context file must contain a mapping.")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _sources_from_raw(
|
||||||
|
raw_sources: Any, source_path: str | None
|
||||||
|
) -> list[RuntimeContextSource]:
|
||||||
|
if not raw_sources:
|
||||||
|
return [RuntimeContextSource(path=source_path)] if source_path else []
|
||||||
|
if isinstance(raw_sources, dict):
|
||||||
|
return [
|
||||||
|
RuntimeContextSource(
|
||||||
|
name=str(name),
|
||||||
|
path=str(raw.get("path")) if isinstance(raw, dict) and raw.get("path") else None,
|
||||||
|
kind=str(raw.get("kind", "file")) if isinstance(raw, dict) else "file",
|
||||||
|
metadata=raw.get("metadata", {}) if isinstance(raw, dict) else {},
|
||||||
|
)
|
||||||
|
for name, raw in raw_sources.items()
|
||||||
|
]
|
||||||
|
if isinstance(raw_sources, list):
|
||||||
|
sources: list[RuntimeContextSource] = []
|
||||||
|
for item in raw_sources:
|
||||||
|
if isinstance(item, dict):
|
||||||
|
sources.append(
|
||||||
|
RuntimeContextSource(
|
||||||
|
name=item.get("name"),
|
||||||
|
path=item.get("path"),
|
||||||
|
kind=str(item.get("kind", "file")),
|
||||||
|
metadata=item.get("metadata", {}),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return sources
|
||||||
|
return [RuntimeContextSource(path=source_path)] if source_path else []
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_context_schemas(
|
||||||
|
data: dict[str, Any], schemas: dict[str, Any], source_path: str | None
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
for name, schema in schemas.items():
|
||||||
|
if not isinstance(schema, dict):
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.schema_invalid",
|
||||||
|
f"Context schema `{name}` must be a mapping.",
|
||||||
|
source_path=source_path,
|
||||||
|
rule_id=str(name),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
target = data if name == "$context" else data.get(name)
|
||||||
|
if name != "$context" and name not in data:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.schema_target_missing",
|
||||||
|
f"Context schema `{name}` has no matching context object.",
|
||||||
|
source_path=source_path,
|
||||||
|
rule_id=str(name),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
Draft202012Validator.check_schema(schema)
|
||||||
|
Draft202012Validator(schema).validate(target)
|
||||||
|
except SchemaError as exc:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.schema_invalid",
|
||||||
|
f"Invalid context schema `{name}`: {exc.message}",
|
||||||
|
source_path=source_path,
|
||||||
|
rule_id=str(name),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except ValidationError as exc:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.context.schema",
|
||||||
|
f"Context object `{name}` does not match its schema: {exc.message}",
|
||||||
|
source_path=source_path,
|
||||||
|
rule_id=str(name),
|
||||||
|
details={"path": list(exc.absolute_path)},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _diagnostic(
|
||||||
|
code: str,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
source_path: str | None = None,
|
||||||
|
rule_id: str | None = None,
|
||||||
|
details: dict[str, Any] | None = None,
|
||||||
|
) -> Diagnostic:
|
||||||
|
return Diagnostic(
|
||||||
|
severity="error",
|
||||||
|
code=code,
|
||||||
|
message=message,
|
||||||
|
source=SourceLocation(path=source_path) if source_path else None,
|
||||||
|
rule_id=rule_id,
|
||||||
|
details=details or {},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_empty(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
key: value
|
||||||
|
for key, value in data.items()
|
||||||
|
if value not in (None, [], {}, "")
|
||||||
|
}
|
||||||
839
src/markitect_tool/runtime/forms.py
Normal file
839
src/markitect_tool/runtime/forms.py
Normal file
@@ -0,0 +1,839 @@
|
|||||||
|
"""Form state and deterministic runtime rule evaluation."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from markitect_tool.contract.model import DocumentContract, FieldSpec, SectionSpec
|
||||||
|
from markitect_tool.core import Document
|
||||||
|
from markitect_tool.diagnostics import (
|
||||||
|
Diagnostic,
|
||||||
|
SourceLocation,
|
||||||
|
has_error,
|
||||||
|
valid_severity,
|
||||||
|
)
|
||||||
|
from markitect_tool.runtime.context import RuntimeContext
|
||||||
|
from markitect_tool.runtime.paths import comparable_value, resolve_path
|
||||||
|
from markitect_tool.runtime.rules import evaluate_condition
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FieldState:
|
||||||
|
"""UI-neutral runtime state for one contract field."""
|
||||||
|
|
||||||
|
id: str
|
||||||
|
path: str | None = None
|
||||||
|
source: str | None = None
|
||||||
|
type: str | None = None
|
||||||
|
label: str | None = None
|
||||||
|
description: str | None = None
|
||||||
|
value: Any = None
|
||||||
|
exists: bool = False
|
||||||
|
origin: str = "missing"
|
||||||
|
required: bool = False
|
||||||
|
visible: bool = True
|
||||||
|
enabled: bool = True
|
||||||
|
allowed_values: list[Any] | None = None
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return not has_error(self.diagnostics)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = asdict(self)
|
||||||
|
data["diagnostics"] = [diagnostic.to_dict() for diagnostic in self.diagnostics]
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FormState:
|
||||||
|
"""A complete runtime form state derived from a document contract."""
|
||||||
|
|
||||||
|
contract_id: str | None
|
||||||
|
document_path: str | None = None
|
||||||
|
context_path: str | None = None
|
||||||
|
fields: list[FieldState] = field(default_factory=list)
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
rules_applied: list[str] = field(default_factory=list)
|
||||||
|
metadata: dict[str, Any] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def valid(self) -> bool:
|
||||||
|
return not has_error(self.diagnostics)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def field_values(self) -> dict[str, Any]:
|
||||||
|
return {field.id: field.value for field in self.fields if field.exists}
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, Any]:
|
||||||
|
data = {
|
||||||
|
"valid": self.valid,
|
||||||
|
"contract_id": self.contract_id,
|
||||||
|
"document_path": self.document_path,
|
||||||
|
"context_path": self.context_path,
|
||||||
|
"field_values": self.field_values,
|
||||||
|
"fields": [field.to_dict() for field in self.fields],
|
||||||
|
"diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics],
|
||||||
|
"rules_applied": self.rules_applied,
|
||||||
|
"metadata": self.metadata,
|
||||||
|
}
|
||||||
|
return _drop_empty(data)
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_form_state(
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
runtime_context: RuntimeContext | None = None,
|
||||||
|
) -> FormState:
|
||||||
|
"""Evaluate contract fields, prefill sources, and dynamic runtime rules."""
|
||||||
|
|
||||||
|
context = runtime_context or RuntimeContext.empty()
|
||||||
|
bindings = build_runtime_bindings(document, contract, context)
|
||||||
|
fields = [
|
||||||
|
_resolve_field_state(field_spec, document, contract, bindings)
|
||||||
|
for field_spec in contract.fields
|
||||||
|
]
|
||||||
|
diagnostics: list[Diagnostic] = list(context.diagnostics)
|
||||||
|
rules_applied: list[str] = []
|
||||||
|
|
||||||
|
fields_by_id = {field.id: field for field in fields}
|
||||||
|
bindings = build_runtime_bindings(document, contract, context, fields_by_id)
|
||||||
|
fields_by_id, rule_diagnostics, rules_applied = _apply_dynamic_rules(
|
||||||
|
fields_by_id,
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
context,
|
||||||
|
bindings,
|
||||||
|
)
|
||||||
|
diagnostics.extend(rule_diagnostics)
|
||||||
|
|
||||||
|
validated_fields: list[FieldState] = []
|
||||||
|
for field_spec in contract.fields:
|
||||||
|
field_state = fields_by_id[_field_key(field_spec)]
|
||||||
|
field_diagnostics = [
|
||||||
|
*field_state.diagnostics,
|
||||||
|
*_validate_field_state(field_spec, field_state, document, contract),
|
||||||
|
]
|
||||||
|
validated_fields.append(replace(field_state, diagnostics=field_diagnostics))
|
||||||
|
diagnostics.extend(field_diagnostics)
|
||||||
|
|
||||||
|
return FormState(
|
||||||
|
contract_id=contract.id,
|
||||||
|
document_path=document.source_path,
|
||||||
|
context_path=context.source_path,
|
||||||
|
fields=validated_fields,
|
||||||
|
diagnostics=diagnostics,
|
||||||
|
rules_applied=rules_applied,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_runtime_bindings(
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
context: RuntimeContext,
|
||||||
|
fields_by_id: dict[str, FieldState] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Build deterministic bindings used by field sources and rule conditions."""
|
||||||
|
|
||||||
|
field_states = fields_by_id or {}
|
||||||
|
return {
|
||||||
|
"document": document.to_dict(),
|
||||||
|
"frontmatter": document.frontmatter,
|
||||||
|
"context": context.binding(),
|
||||||
|
"contract": contract.to_dict(),
|
||||||
|
"fields": {
|
||||||
|
field_id: field.to_dict() | {"value": field.value}
|
||||||
|
for field_id, field in field_states.items()
|
||||||
|
},
|
||||||
|
"field_values": {
|
||||||
|
field_id: field.value
|
||||||
|
for field_id, field in field_states.items()
|
||||||
|
if field.exists
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_field_state(
|
||||||
|
field_spec: FieldSpec,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
bindings: dict[str, Any],
|
||||||
|
) -> FieldState:
|
||||||
|
field_id = field_spec.id or "<missing>"
|
||||||
|
path = field_spec.path or (f"frontmatter.{field_id}" if field_spec.id else None)
|
||||||
|
source_candidates = _source_candidates(field_spec)
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
|
||||||
|
document_value, document_exists = resolve_path(bindings, path)
|
||||||
|
source_values = [
|
||||||
|
(source, value)
|
||||||
|
for source in source_candidates
|
||||||
|
for value, exists in [resolve_path(bindings, source)]
|
||||||
|
if exists
|
||||||
|
]
|
||||||
|
|
||||||
|
if document_exists:
|
||||||
|
origin = "manual"
|
||||||
|
value = document_value
|
||||||
|
exists = True
|
||||||
|
for source, source_value in source_values:
|
||||||
|
if comparable_value(source_value) != comparable_value(document_value):
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.field.conflict",
|
||||||
|
(
|
||||||
|
f"Field `{field_id}` is provided by the document and "
|
||||||
|
f"context source `{source}` with different values."
|
||||||
|
),
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_id,
|
||||||
|
severity=_conflict_severity(field_spec),
|
||||||
|
details={
|
||||||
|
"path": path,
|
||||||
|
"source": source,
|
||||||
|
"document_value": comparable_value(document_value),
|
||||||
|
"context_value": comparable_value(source_value),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
elif source_values:
|
||||||
|
origin = "prefilled"
|
||||||
|
value = source_values[0][1]
|
||||||
|
exists = True
|
||||||
|
distinct = {
|
||||||
|
repr(comparable_value(item_value))
|
||||||
|
for _source, item_value in source_values
|
||||||
|
}
|
||||||
|
if len(distinct) > 1:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.field.ambiguous",
|
||||||
|
f"Field `{field_id}` has multiple source values.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_id,
|
||||||
|
details={"sources": [source for source, _value in source_values]},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
elif field_spec.default is not None:
|
||||||
|
origin = "defaulted"
|
||||||
|
value = field_spec.default
|
||||||
|
exists = True
|
||||||
|
else:
|
||||||
|
origin = "missing"
|
||||||
|
value = None
|
||||||
|
exists = False
|
||||||
|
|
||||||
|
return FieldState(
|
||||||
|
id=field_id,
|
||||||
|
path=path,
|
||||||
|
source=source_candidates[0] if source_candidates else field_spec.source,
|
||||||
|
type=field_spec.type,
|
||||||
|
label=field_spec.label,
|
||||||
|
description=field_spec.description,
|
||||||
|
value=value,
|
||||||
|
exists=exists,
|
||||||
|
origin=origin,
|
||||||
|
required=field_spec.required,
|
||||||
|
allowed_values=field_spec.enum,
|
||||||
|
diagnostics=diagnostics,
|
||||||
|
metadata={
|
||||||
|
"sources": source_candidates,
|
||||||
|
"coerce": bool(field_spec.raw.get("coerce", False))
|
||||||
|
if isinstance(field_spec.raw, dict)
|
||||||
|
else False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _source_candidates(field_spec: FieldSpec) -> list[str]:
|
||||||
|
raw = field_spec.raw if isinstance(field_spec.raw, dict) else {}
|
||||||
|
candidates: list[str] = []
|
||||||
|
source = raw.get("source", field_spec.source)
|
||||||
|
if isinstance(source, list):
|
||||||
|
candidates.extend(str(item) for item in source if item)
|
||||||
|
elif source:
|
||||||
|
candidates.append(str(source))
|
||||||
|
sources = raw.get("sources")
|
||||||
|
if isinstance(sources, list):
|
||||||
|
candidates.extend(str(item) for item in sources if item)
|
||||||
|
return list(dict.fromkeys(candidates))
|
||||||
|
|
||||||
|
|
||||||
|
def _field_key(field_spec: FieldSpec) -> str:
|
||||||
|
return field_spec.id or "<missing>"
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_dynamic_rules(
|
||||||
|
fields_by_id: dict[str, FieldState],
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
context: RuntimeContext,
|
||||||
|
bindings: dict[str, Any],
|
||||||
|
) -> tuple[dict[str, FieldState], list[Diagnostic], list[str]]:
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
applied: list[str] = []
|
||||||
|
rules = contract.rules
|
||||||
|
for index, rule in enumerate(rules):
|
||||||
|
if not isinstance(rule, dict):
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.invalid",
|
||||||
|
"Dynamic rule must be a mapping.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
rule_id = str(rule.get("id") or f"rule-{index + 1}")
|
||||||
|
result = evaluate_condition(rule.get("if"), bindings, rule_id=rule_id)
|
||||||
|
diagnostics.extend(
|
||||||
|
_with_contract_location(item, document=document, contract=contract)
|
||||||
|
for item in result.diagnostics
|
||||||
|
)
|
||||||
|
action = rule.get("then") if result.matched else rule.get("else")
|
||||||
|
if result.matched:
|
||||||
|
applied.append(rule_id)
|
||||||
|
if action is not None:
|
||||||
|
fields_by_id, action_diagnostics = _apply_rule_action(
|
||||||
|
fields_by_id,
|
||||||
|
action,
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
context,
|
||||||
|
rule_id,
|
||||||
|
)
|
||||||
|
diagnostics.extend(action_diagnostics)
|
||||||
|
bindings = build_runtime_bindings(document, contract, context, fields_by_id)
|
||||||
|
if "assert" in rule and result.matched:
|
||||||
|
diagnostics.extend(
|
||||||
|
_evaluate_runtime_assertions(
|
||||||
|
rule["assert"],
|
||||||
|
bindings,
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
rule_id,
|
||||||
|
rule,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return fields_by_id, diagnostics, applied
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_rule_action(
|
||||||
|
fields_by_id: dict[str, FieldState],
|
||||||
|
action: Any,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
context: RuntimeContext,
|
||||||
|
rule_id: str,
|
||||||
|
) -> tuple[dict[str, FieldState], list[Diagnostic]]:
|
||||||
|
if not isinstance(action, dict):
|
||||||
|
return fields_by_id, [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.action_invalid",
|
||||||
|
f"Dynamic rule `{rule_id}` action must be a mapping.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
updated = dict(fields_by_id)
|
||||||
|
bindings = build_runtime_bindings(document, contract, context, updated)
|
||||||
|
|
||||||
|
for field_id in _field_id_list(action.get("required")):
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"required": True}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
for field_id in _field_id_list(action.get("optional")):
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"required": False}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
for field_id, visible in _field_bool_mapping(action.get("visible")).items():
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"visible": visible}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
for field_id in _field_id_list(action.get("hidden")):
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"visible": False}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
for field_id, enabled in _field_bool_mapping(action.get("enabled")).items():
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"enabled": enabled}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
for field_id in _field_id_list(action.get("disabled")):
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated, diagnostics, field_id, {"enabled": False}, document, contract, rule_id
|
||||||
|
)
|
||||||
|
if isinstance(action.get("allowed_values"), dict):
|
||||||
|
for field_id, allowed in action["allowed_values"].items():
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated,
|
||||||
|
diagnostics,
|
||||||
|
str(field_id),
|
||||||
|
{"allowed_values": allowed if isinstance(allowed, list) else [allowed]},
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
rule_id,
|
||||||
|
)
|
||||||
|
if isinstance(action.get("set"), dict):
|
||||||
|
for field_id, raw_value in action["set"].items():
|
||||||
|
value = _resolve_template_value(raw_value, bindings)
|
||||||
|
updated, diagnostics = _set_field_attr(
|
||||||
|
updated,
|
||||||
|
diagnostics,
|
||||||
|
str(field_id),
|
||||||
|
{"value": value, "exists": True, "origin": "calculated"},
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
rule_id,
|
||||||
|
)
|
||||||
|
if "assert" in action:
|
||||||
|
diagnostics.extend(
|
||||||
|
_evaluate_runtime_assertions(
|
||||||
|
action["assert"],
|
||||||
|
build_runtime_bindings(document, contract, context, updated),
|
||||||
|
document,
|
||||||
|
contract,
|
||||||
|
rule_id,
|
||||||
|
action,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if "sections" in action:
|
||||||
|
diagnostics.extend(_check_dynamic_sections(action["sections"], document, contract, rule_id))
|
||||||
|
return updated, diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _set_field_attr(
|
||||||
|
fields_by_id: dict[str, FieldState],
|
||||||
|
diagnostics: list[Diagnostic],
|
||||||
|
field_id: str,
|
||||||
|
updates: dict[str, Any],
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
rule_id: str,
|
||||||
|
) -> tuple[dict[str, FieldState], list[Diagnostic]]:
|
||||||
|
if field_id not in fields_by_id:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.unknown_field",
|
||||||
|
f"Dynamic rule `{rule_id}` references unknown field `{field_id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return fields_by_id, diagnostics
|
||||||
|
fields_by_id[field_id] = replace(fields_by_id[field_id], **updates)
|
||||||
|
return fields_by_id, diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _evaluate_runtime_assertions(
|
||||||
|
raw_assertions: Any,
|
||||||
|
bindings: dict[str, Any],
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
rule_id: str,
|
||||||
|
rule: dict[str, Any],
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
assertions = raw_assertions if isinstance(raw_assertions, list) else [raw_assertions]
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
for assertion in assertions:
|
||||||
|
result = evaluate_condition(assertion, bindings, rule_id=rule_id)
|
||||||
|
diagnostics.extend(
|
||||||
|
_with_contract_location(item, document=document, contract=contract)
|
||||||
|
for item in result.diagnostics
|
||||||
|
)
|
||||||
|
if not result.matched:
|
||||||
|
severity = _severity_from_mapping(assertion, rule)
|
||||||
|
message = (
|
||||||
|
assertion.get("message")
|
||||||
|
if isinstance(assertion, dict) and assertion.get("message")
|
||||||
|
else rule.get("message")
|
||||||
|
or f"Runtime assertion `{rule_id}` was not satisfied."
|
||||||
|
)
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.assertion_failed",
|
||||||
|
str(message),
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
severity=severity,
|
||||||
|
details={"assertion": assertion if isinstance(assertion, dict) else None},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _check_dynamic_sections(
|
||||||
|
raw_sections: Any,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
rule_id: str,
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
if not isinstance(raw_sections, dict):
|
||||||
|
return [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.sections_invalid",
|
||||||
|
"`sections` action must be a mapping.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
section_specs = {section.id: section for section in contract.sections if section.id}
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
for section_id, raw in raw_sections.items():
|
||||||
|
section_spec = section_specs.get(str(section_id))
|
||||||
|
if section_spec is None:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.unknown_section",
|
||||||
|
f"Dynamic rule `{rule_id}` references unknown section `{section_id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
presence = raw.get("presence") if isinstance(raw, dict) else raw
|
||||||
|
diagnostics.extend(_check_dynamic_section_presence(document, contract, section_spec, str(presence), rule_id))
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _check_dynamic_section_presence(
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
section_spec: SectionSpec,
|
||||||
|
presence: str,
|
||||||
|
rule_id: str,
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
matches = _matching_section_lines(document, section_spec)
|
||||||
|
if presence == "required" and not matches:
|
||||||
|
return [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.section.missing",
|
||||||
|
f"Dynamic rule requires section `{section_spec.id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
guidance=f"Add a {'#' * (section_spec.level or 2)} {section_spec.title or section_spec.id} section.",
|
||||||
|
)
|
||||||
|
]
|
||||||
|
if presence == "recommended" and not matches:
|
||||||
|
return [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.section.recommended_missing",
|
||||||
|
f"Dynamic rule recommends section `{section_spec.id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
severity="warning",
|
||||||
|
)
|
||||||
|
]
|
||||||
|
if presence == "forbidden" and matches:
|
||||||
|
return [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.section.forbidden",
|
||||||
|
f"Dynamic rule forbids section `{section_spec.id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
source_line=matches[0],
|
||||||
|
)
|
||||||
|
]
|
||||||
|
if presence == "discouraged" and matches:
|
||||||
|
return [
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.section.discouraged",
|
||||||
|
f"Dynamic rule discourages section `{section_spec.id}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=rule_id,
|
||||||
|
source_line=matches[0],
|
||||||
|
severity="warning",
|
||||||
|
)
|
||||||
|
]
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_field_state(
|
||||||
|
field_spec: FieldSpec,
|
||||||
|
field_state: FieldState,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
) -> list[Diagnostic]:
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
if field_state.required and field_state.visible and not field_state.exists:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.missing",
|
||||||
|
f"Required field `{field_state.id}` is missing.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
guidance=f"Provide `{field_state.path}` in the document or context.",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return diagnostics
|
||||||
|
if not field_state.exists:
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
value = field_state.value
|
||||||
|
if field_state.metadata.get("coerce"):
|
||||||
|
value, coerced, coercion_error = _coerce_value(value, field_state.type)
|
||||||
|
if coercion_error:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.field.coercion_failed",
|
||||||
|
f"Field `{field_state.id}` could not be coerced to `{field_state.type}`.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
elif coerced:
|
||||||
|
object.__setattr__(field_state, "value", value)
|
||||||
|
|
||||||
|
if field_state.type and not _value_matches_type(value, field_state.type):
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.type_mismatch",
|
||||||
|
(
|
||||||
|
f"Field `{field_state.id}` must be `{field_state.type}`, "
|
||||||
|
f"got `{type(value).__name__}`."
|
||||||
|
),
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
allowed_values = field_state.allowed_values
|
||||||
|
if allowed_values is not None and value not in allowed_values:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.enum",
|
||||||
|
f"Field `{field_state.id}` must be one of {allowed_values}.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if field_spec.pattern and isinstance(value, str) and not re.search(field_spec.pattern, value):
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.pattern",
|
||||||
|
f"Field `{field_state.id}` does not match its required pattern.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if field_spec.min_length is not None and hasattr(value, "__len__") and len(value) < field_spec.min_length:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.min_length",
|
||||||
|
f"Field `{field_state.id}` is shorter than {field_spec.min_length}.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if field_spec.max_length is not None and hasattr(value, "__len__") and len(value) > field_spec.max_length:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.max_length",
|
||||||
|
f"Field `{field_state.id}` is longer than {field_spec.max_length}.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if field_spec.min is not None and isinstance(value, int | float) and value < field_spec.min:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.min",
|
||||||
|
f"Field `{field_state.id}` is below {field_spec.min}.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if field_spec.max is not None and isinstance(value, int | float) and value > field_spec.max:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"contract.field.max",
|
||||||
|
f"Field `{field_state.id}` is above {field_spec.max}.",
|
||||||
|
document=document,
|
||||||
|
contract=contract,
|
||||||
|
rule_id=field_state.id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return diagnostics
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_value(value: Any, expected_type: str | None) -> tuple[Any, bool, bool]:
|
||||||
|
if expected_type == "string" and not isinstance(value, str):
|
||||||
|
return str(value), True, False
|
||||||
|
if expected_type == "integer" and isinstance(value, str):
|
||||||
|
try:
|
||||||
|
return int(value), True, False
|
||||||
|
except ValueError:
|
||||||
|
return value, False, True
|
||||||
|
if expected_type == "number" and isinstance(value, str):
|
||||||
|
try:
|
||||||
|
return float(value), True, False
|
||||||
|
except ValueError:
|
||||||
|
return value, False, True
|
||||||
|
if expected_type == "boolean" and isinstance(value, str):
|
||||||
|
normalized = value.strip().lower()
|
||||||
|
if normalized in {"true", "yes", "1"}:
|
||||||
|
return True, True, False
|
||||||
|
if normalized in {"false", "no", "0"}:
|
||||||
|
return False, True, False
|
||||||
|
return value, False, True
|
||||||
|
return value, False, False
|
||||||
|
|
||||||
|
|
||||||
|
def _value_matches_type(value: Any, expected_type: str) -> bool:
|
||||||
|
if expected_type == "string":
|
||||||
|
return isinstance(value, str)
|
||||||
|
if expected_type == "number":
|
||||||
|
return isinstance(value, int | float) and not isinstance(value, bool)
|
||||||
|
if expected_type == "integer":
|
||||||
|
return isinstance(value, int) and not isinstance(value, bool)
|
||||||
|
if expected_type == "boolean":
|
||||||
|
return isinstance(value, bool)
|
||||||
|
if expected_type == "array":
|
||||||
|
return isinstance(value, list)
|
||||||
|
if expected_type == "object":
|
||||||
|
return isinstance(value, dict)
|
||||||
|
if expected_type == "date":
|
||||||
|
return isinstance(value, str)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def _field_id_list(value: Any) -> list[str]:
|
||||||
|
if value is None:
|
||||||
|
return []
|
||||||
|
if isinstance(value, str):
|
||||||
|
return [value]
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [str(item) for item in value if item]
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return [str(key) for key, enabled in value.items() if enabled]
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _field_bool_mapping(value: Any) -> dict[str, bool]:
|
||||||
|
if value is None:
|
||||||
|
return {}
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {str(key): bool(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, str):
|
||||||
|
return {value: True}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return {str(item): True for item in value if item}
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def _resolve_template_value(value: Any, bindings: dict[str, Any]) -> Any:
|
||||||
|
if isinstance(value, str):
|
||||||
|
full = re.fullmatch(r"\$\{([^}]+)\}", value.strip())
|
||||||
|
if full:
|
||||||
|
resolved, exists = resolve_path(bindings, full.group(1))
|
||||||
|
return resolved if exists else value
|
||||||
|
|
||||||
|
def replace_match(match: re.Match[str]) -> str:
|
||||||
|
resolved, exists = resolve_path(bindings, match.group(1))
|
||||||
|
return str(resolved) if exists else match.group(0)
|
||||||
|
|
||||||
|
return re.sub(r"\$\{([^}]+)\}", replace_match, value)
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [_resolve_template_value(item, bindings) for item in value]
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {key: _resolve_template_value(item, bindings) for key, item in value.items()}
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _conflict_severity(field_spec: FieldSpec) -> str:
|
||||||
|
raw = field_spec.raw if isinstance(field_spec.raw, dict) else {}
|
||||||
|
severity = raw.get("conflict") or raw.get("conflict_severity") or "warning"
|
||||||
|
return str(severity) if valid_severity(str(severity)) else "warning"
|
||||||
|
|
||||||
|
|
||||||
|
def _severity_from_mapping(*items: Any) -> str:
|
||||||
|
for item in items:
|
||||||
|
if isinstance(item, dict) and valid_severity(str(item.get("severity"))):
|
||||||
|
return str(item["severity"])
|
||||||
|
return "error"
|
||||||
|
|
||||||
|
|
||||||
|
def _matching_section_lines(document: Document, section_spec: SectionSpec) -> list[int]:
|
||||||
|
expected = {_normalize_heading(value) for value in section_spec.headings}
|
||||||
|
return [
|
||||||
|
section.heading.line
|
||||||
|
for section in document.sections
|
||||||
|
if _normalize_heading(section.heading.text) in expected
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_heading(text: str) -> str:
|
||||||
|
return re.sub(r"\s+", " ", text.strip().lower())
|
||||||
|
|
||||||
|
|
||||||
|
def _with_contract_location(
|
||||||
|
diagnostic: Diagnostic,
|
||||||
|
*,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
) -> Diagnostic:
|
||||||
|
return Diagnostic(
|
||||||
|
severity=diagnostic.severity,
|
||||||
|
code=diagnostic.code,
|
||||||
|
message=diagnostic.message,
|
||||||
|
source=diagnostic.source or SourceLocation(path=document.source_path),
|
||||||
|
contract=diagnostic.contract or SourceLocation(path=contract.source_path, line=contract.source_line),
|
||||||
|
rule_id=diagnostic.rule_id,
|
||||||
|
guidance=diagnostic.guidance,
|
||||||
|
details=diagnostic.details,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _diagnostic(
|
||||||
|
code: str,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
document: Document,
|
||||||
|
contract: DocumentContract,
|
||||||
|
rule_id: str | None = None,
|
||||||
|
severity: str = "error",
|
||||||
|
guidance: str | None = None,
|
||||||
|
details: dict[str, Any] | None = None,
|
||||||
|
source_line: int | None = 1,
|
||||||
|
) -> Diagnostic:
|
||||||
|
return Diagnostic(
|
||||||
|
severity=severity,
|
||||||
|
code=code,
|
||||||
|
message=message,
|
||||||
|
source=SourceLocation(path=document.source_path, line=source_line),
|
||||||
|
contract=SourceLocation(path=contract.source_path, line=contract.source_line),
|
||||||
|
rule_id=rule_id,
|
||||||
|
guidance=guidance,
|
||||||
|
details=details or {},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _drop_empty(data: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
key: value
|
||||||
|
for key, value in data.items()
|
||||||
|
if value not in (None, [], {}, "")
|
||||||
|
}
|
||||||
45
src/markitect_tool/runtime/paths.py
Normal file
45
src/markitect_tool/runtime/paths.py
Normal file
@@ -0,0 +1,45 @@
|
|||||||
|
"""Small path helpers for runtime context and rule evaluation."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_path(data: Any, path: str | None) -> tuple[Any, bool]:
|
||||||
|
"""Resolve a dotted path against dictionaries and lists.
|
||||||
|
|
||||||
|
The runtime path vocabulary is intentionally small: `context.user.name`,
|
||||||
|
`frontmatter.status`, `fields.owner.value`, and numeric list indexes are
|
||||||
|
enough for contract checks, form state, and deterministic rules without
|
||||||
|
embedding a general expression language.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not path:
|
||||||
|
return None, False
|
||||||
|
current = data
|
||||||
|
for part in _path_parts(path):
|
||||||
|
if isinstance(current, dict) and part in current:
|
||||||
|
current = current[part]
|
||||||
|
continue
|
||||||
|
if isinstance(current, list) and part.isdigit():
|
||||||
|
index = int(part)
|
||||||
|
if index < len(current):
|
||||||
|
current = current[index]
|
||||||
|
continue
|
||||||
|
return None, False
|
||||||
|
return current, True
|
||||||
|
|
||||||
|
|
||||||
|
def comparable_value(value: Any) -> Any:
|
||||||
|
"""Return a stable scalar-ish value for diagnostics and equality checks."""
|
||||||
|
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {key: comparable_value(value[key]) for key in sorted(value)}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [comparable_value(item) for item in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _path_parts(path: str) -> list[str]:
|
||||||
|
normalized = str(path).strip().removeprefix("$.")
|
||||||
|
return [part for part in normalized.split(".") if part]
|
||||||
162
src/markitect_tool/runtime/rules.py
Normal file
162
src/markitect_tool/runtime/rules.py
Normal file
@@ -0,0 +1,162 @@
|
|||||||
|
"""Deterministic rule conditions for runtime forms and checks."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from markitect_tool.diagnostics import Diagnostic
|
||||||
|
from markitect_tool.runtime.paths import resolve_path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ConditionResult:
|
||||||
|
"""Result of evaluating one deterministic condition."""
|
||||||
|
|
||||||
|
matched: bool
|
||||||
|
diagnostics: list[Diagnostic] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_condition(
|
||||||
|
condition: Any,
|
||||||
|
bindings: dict[str, Any],
|
||||||
|
*,
|
||||||
|
rule_id: str | None = None,
|
||||||
|
) -> ConditionResult:
|
||||||
|
"""Evaluate a small Markitect-native condition mapping."""
|
||||||
|
|
||||||
|
if condition is None:
|
||||||
|
return ConditionResult(True)
|
||||||
|
if isinstance(condition, bool):
|
||||||
|
return ConditionResult(condition)
|
||||||
|
if isinstance(condition, list):
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
results = [evaluate_condition(item, bindings, rule_id=rule_id) for item in condition]
|
||||||
|
for result in results:
|
||||||
|
diagnostics.extend(result.diagnostics)
|
||||||
|
return ConditionResult(all(result.matched for result in results), diagnostics)
|
||||||
|
if not isinstance(condition, dict):
|
||||||
|
return ConditionResult(
|
||||||
|
False,
|
||||||
|
[
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.condition_invalid",
|
||||||
|
"Rule condition must be a mapping, list, or boolean.",
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
if "all" in condition:
|
||||||
|
return _combine_conditions(condition["all"], bindings, rule_id, all)
|
||||||
|
if "any" in condition:
|
||||||
|
return _combine_conditions(condition["any"], bindings, rule_id, any)
|
||||||
|
if "not" in condition:
|
||||||
|
result = evaluate_condition(condition["not"], bindings, rule_id=rule_id)
|
||||||
|
return ConditionResult(not result.matched, result.diagnostics)
|
||||||
|
|
||||||
|
path = condition.get("path")
|
||||||
|
if not path:
|
||||||
|
return ConditionResult(
|
||||||
|
False,
|
||||||
|
[
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.condition_missing_path",
|
||||||
|
"Rule condition must declare `path`.",
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
value, exists = resolve_path(bindings, str(path))
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
matched = True
|
||||||
|
if "exists" in condition:
|
||||||
|
matched = matched and (exists is bool(condition["exists"]))
|
||||||
|
elif not exists:
|
||||||
|
matched = False
|
||||||
|
|
||||||
|
if exists:
|
||||||
|
matched = matched and _operator_matches(value, condition, diagnostics, rule_id)
|
||||||
|
return ConditionResult(matched, diagnostics)
|
||||||
|
|
||||||
|
|
||||||
|
def _combine_conditions(
|
||||||
|
raw_conditions: Any,
|
||||||
|
bindings: dict[str, Any],
|
||||||
|
rule_id: str | None,
|
||||||
|
combiner: Any,
|
||||||
|
) -> ConditionResult:
|
||||||
|
conditions = raw_conditions if isinstance(raw_conditions, list) else [raw_conditions]
|
||||||
|
results = [evaluate_condition(item, bindings, rule_id=rule_id) for item in conditions]
|
||||||
|
diagnostics: list[Diagnostic] = []
|
||||||
|
for result in results:
|
||||||
|
diagnostics.extend(result.diagnostics)
|
||||||
|
return ConditionResult(combiner(result.matched for result in results), diagnostics)
|
||||||
|
|
||||||
|
|
||||||
|
def _operator_matches(
|
||||||
|
value: Any,
|
||||||
|
condition: dict[str, Any],
|
||||||
|
diagnostics: list[Diagnostic],
|
||||||
|
rule_id: str | None,
|
||||||
|
) -> bool:
|
||||||
|
matched = True
|
||||||
|
if "equals" in condition:
|
||||||
|
matched = matched and value == condition["equals"]
|
||||||
|
if "eq" in condition:
|
||||||
|
matched = matched and value == condition["eq"]
|
||||||
|
if "not_equals" in condition:
|
||||||
|
matched = matched and value != condition["not_equals"]
|
||||||
|
if "in" in condition:
|
||||||
|
expected = condition["in"]
|
||||||
|
matched = matched and isinstance(expected, list) and value in expected
|
||||||
|
if "contains" in condition:
|
||||||
|
expected = condition["contains"]
|
||||||
|
matched = matched and _contains(value, expected)
|
||||||
|
if "matches" in condition:
|
||||||
|
pattern = str(condition["matches"])
|
||||||
|
try:
|
||||||
|
matched = matched and re.search(pattern, str(value)) is not None
|
||||||
|
except re.error as exc:
|
||||||
|
diagnostics.append(
|
||||||
|
_diagnostic(
|
||||||
|
"runtime.rule.regex_invalid",
|
||||||
|
f"Invalid rule regular expression `{pattern}`: {exc}",
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
matched = False
|
||||||
|
for key, predicate in {
|
||||||
|
"gt": lambda actual, expected: actual > expected,
|
||||||
|
"gte": lambda actual, expected: actual >= expected,
|
||||||
|
"lt": lambda actual, expected: actual < expected,
|
||||||
|
"lte": lambda actual, expected: actual <= expected,
|
||||||
|
}.items():
|
||||||
|
if key not in condition:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
matched = matched and predicate(value, condition[key])
|
||||||
|
except TypeError:
|
||||||
|
matched = False
|
||||||
|
return matched
|
||||||
|
|
||||||
|
|
||||||
|
def _contains(value: Any, expected: Any) -> bool:
|
||||||
|
if isinstance(value, str):
|
||||||
|
return str(expected) in value
|
||||||
|
if isinstance(value, list | tuple | set):
|
||||||
|
return expected in value
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return expected in value
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _diagnostic(code: str, message: str, *, rule_id: str | None = None) -> Diagnostic:
|
||||||
|
return Diagnostic(
|
||||||
|
severity="error",
|
||||||
|
code=code,
|
||||||
|
message=message,
|
||||||
|
rule_id=rule_id,
|
||||||
|
)
|
||||||
@@ -25,6 +25,7 @@ from markitect_tool.query import (
|
|||||||
extract_document_with_engine,
|
extract_document_with_engine,
|
||||||
query_document_with_engine,
|
query_document_with_engine,
|
||||||
)
|
)
|
||||||
|
from markitect_tool.runtime import evaluate_form_state, load_runtime_context_file
|
||||||
from markitect_tool.template import MissingTemplateVariable, TemplateError, render_template
|
from markitect_tool.template import MissingTemplateVariable, TemplateError, render_template
|
||||||
|
|
||||||
|
|
||||||
@@ -322,6 +323,8 @@ class WorkflowRunner:
|
|||||||
return self._step_contract_stub(step)
|
return self._step_contract_stub(step)
|
||||||
if kind == "contract_check":
|
if kind == "contract_check":
|
||||||
return self._step_contract_check(step)
|
return self._step_contract_check(step)
|
||||||
|
if kind == "form_state":
|
||||||
|
return self._step_form_state(step)
|
||||||
if kind == "assisted":
|
if kind == "assisted":
|
||||||
return self._step_assisted(step)
|
return self._step_assisted(step)
|
||||||
raise WorkflowError(f"Unsupported workflow step kind `{kind}`")
|
raise WorkflowError(f"Unsupported workflow step kind `{kind}`")
|
||||||
@@ -404,9 +407,28 @@ class WorkflowRunner:
|
|||||||
def _step_contract_check(self, step: dict[str, Any]) -> dict[str, Any]:
|
def _step_contract_check(self, step: dict[str, Any]) -> dict[str, Any]:
|
||||||
document_path = _safe_input_path(self.base_dir, step.get("document"))
|
document_path = _safe_input_path(self.base_dir, step.get("document"))
|
||||||
contract_path = _safe_input_path(self.base_dir, step.get("contract"))
|
contract_path = _safe_input_path(self.base_dir, step.get("contract"))
|
||||||
result = check_markdown_file(document_path, contract_path)
|
context_path = (
|
||||||
|
_safe_input_path(self.base_dir, step.get("context"))
|
||||||
|
if step.get("context")
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
result = check_markdown_file(document_path, contract_path, context_path=context_path)
|
||||||
return result.to_dict() | {"kind": "contract_check"}
|
return result.to_dict() | {"kind": "contract_check"}
|
||||||
|
|
||||||
|
def _step_form_state(self, step: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
document_path = _safe_input_path(self.base_dir, step.get("document"))
|
||||||
|
contract_path = _safe_input_path(self.base_dir, step.get("contract"))
|
||||||
|
context_path = (
|
||||||
|
_safe_input_path(self.base_dir, step.get("context"))
|
||||||
|
if step.get("context")
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
document = parse_markdown_file(document_path)
|
||||||
|
contract = load_contract_file(contract_path)
|
||||||
|
context = load_runtime_context_file(context_path) if context_path else None
|
||||||
|
result = evaluate_form_state(document, contract, context)
|
||||||
|
return result.to_dict() | {"kind": "form_state"}
|
||||||
|
|
||||||
def _step_assisted(self, step: dict[str, Any]) -> dict[str, Any]:
|
def _step_assisted(self, step: dict[str, Any]) -> dict[str, Any]:
|
||||||
optional = bool(step.get("optional", True))
|
optional = bool(step.get("optional", True))
|
||||||
if self.assisted_hook is None:
|
if self.assisted_hook is None:
|
||||||
|
|||||||
@@ -14,6 +14,9 @@ def test_builtin_extension_registry_lists_query_processors_and_backend():
|
|||||||
assert "processor.include" in ids
|
assert "processor.include" in ids
|
||||||
assert "backend.local-sqlite" in ids
|
assert "backend.local-sqlite" in ids
|
||||||
assert "workflow.markdown-dataflow" in ids
|
assert "workflow.markdown-dataflow" in ids
|
||||||
|
assert "runtime.context" in ids
|
||||||
|
assert "runtime.form-state" in ids
|
||||||
|
assert "runtime.assessment" in ids
|
||||||
|
|
||||||
|
|
||||||
def test_builtin_processor_descriptors_capture_safety_and_provenance():
|
def test_builtin_processor_descriptors_capture_safety_and_provenance():
|
||||||
@@ -64,3 +67,17 @@ def test_builtin_workflow_descriptor_exposes_cli_and_safety():
|
|||||||
"mkt workflow plan",
|
"mkt workflow plan",
|
||||||
"mkt workflow run",
|
"mkt workflow run",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_builtin_runtime_descriptors_expose_boundaries():
|
||||||
|
registry = builtin_extension_registry()
|
||||||
|
|
||||||
|
context = registry.get("runtime.context")
|
||||||
|
form_state = registry.get("runtime.form-state")
|
||||||
|
assessment = registry.get("runtime.assessment")
|
||||||
|
|
||||||
|
assert context.safety["reads_files"] is True
|
||||||
|
assert "mkt contract check --context" in context.cli["commands"]
|
||||||
|
assert {capability.id for capability in form_state.capabilities} >= {"forms", "rules"}
|
||||||
|
assert assessment.kind == "assessment-runner"
|
||||||
|
assert assessment.safety["provider_calls"] == "adapter-only"
|
||||||
|
|||||||
388
tests/test_runtime_context_forms_assessment.py
Normal file
388
tests/test_runtime_context_forms_assessment.py
Normal file
@@ -0,0 +1,388 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from click.testing import CliRunner
|
||||||
|
|
||||||
|
from markitect_tool.cli import main
|
||||||
|
from markitect_tool.contract import check_markdown_file, load_contract_file
|
||||||
|
from markitect_tool.core import parse_markdown, parse_markdown_file
|
||||||
|
from markitect_tool.runtime import (
|
||||||
|
AssessmentResult,
|
||||||
|
AssessmentRunner,
|
||||||
|
MemoryAssessmentCache,
|
||||||
|
assessment_requests_for_contract,
|
||||||
|
evaluate_form_state,
|
||||||
|
load_runtime_context_file,
|
||||||
|
)
|
||||||
|
from markitect_tool.workflow import WorkflowRunner, load_workflow_file
|
||||||
|
|
||||||
|
|
||||||
|
LETTER_CONTRACT = """# Letter Contract
|
||||||
|
|
||||||
|
```yaml contract
|
||||||
|
id: letter-runtime-v1
|
||||||
|
document:
|
||||||
|
type: business-letter
|
||||||
|
fields:
|
||||||
|
recipient_name:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
source: context.recipient.name
|
||||||
|
sender_name:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
source: context.sender.name
|
||||||
|
sender_email:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
source: context.sender.email
|
||||||
|
pattern: "@example\\\\.com$"
|
||||||
|
delivery_channel:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
default: email
|
||||||
|
enum: [email, print]
|
||||||
|
postal_address:
|
||||||
|
type: string
|
||||||
|
contact_label:
|
||||||
|
type: string
|
||||||
|
rules:
|
||||||
|
- id: postal-address-for-print
|
||||||
|
if:
|
||||||
|
path: fields.delivery_channel.value
|
||||||
|
equals: print
|
||||||
|
then:
|
||||||
|
required: [postal_address]
|
||||||
|
visible:
|
||||||
|
postal_address: true
|
||||||
|
else:
|
||||||
|
hidden: [postal_address]
|
||||||
|
- id: calculate-contact-label
|
||||||
|
then:
|
||||||
|
set:
|
||||||
|
contact_label: "${fields.sender_name.value} <${fields.sender_email.value}>"
|
||||||
|
- id: sender-email-domain
|
||||||
|
assert:
|
||||||
|
path: context.sender.email
|
||||||
|
matches: "@example\\\\.com$"
|
||||||
|
message: Sender email must come from example.com.
|
||||||
|
severity: warning
|
||||||
|
sections:
|
||||||
|
- id: greeting
|
||||||
|
title: Greeting
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
- id: body
|
||||||
|
title: Body
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
- id: closing
|
||||||
|
title: Closing
|
||||||
|
presence: required
|
||||||
|
level: 2
|
||||||
|
rubrics:
|
||||||
|
- id: tone-fit
|
||||||
|
scope: section.body
|
||||||
|
criteria: The body should match the recipient relationship.
|
||||||
|
```
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
LETTER_DOC = """---
|
||||||
|
document_type: business-letter
|
||||||
|
---
|
||||||
|
|
||||||
|
# Follow Up
|
||||||
|
|
||||||
|
## Greeting
|
||||||
|
|
||||||
|
Dear Ada,
|
||||||
|
|
||||||
|
## Body
|
||||||
|
|
||||||
|
Thank you for the productive discussion. We will follow up with a concise
|
||||||
|
proposal and next steps for the Markdown workflow.
|
||||||
|
|
||||||
|
## Closing
|
||||||
|
|
||||||
|
Kind regards
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
LETTER_CONTEXT = """metadata:
|
||||||
|
case_id: case-42
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [recipient, sender]
|
||||||
|
properties:
|
||||||
|
recipient:
|
||||||
|
type: object
|
||||||
|
required: [name]
|
||||||
|
sender:
|
||||||
|
type: object
|
||||||
|
required: [name, email]
|
||||||
|
context:
|
||||||
|
recipient:
|
||||||
|
name: Ada Lovelace
|
||||||
|
sender:
|
||||||
|
name: Markitect Team
|
||||||
|
email: hello@example.com
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_context_loads_yaml_and_validates_schema(tmp_path: Path):
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
|
||||||
|
context = load_runtime_context_file(context_file)
|
||||||
|
|
||||||
|
assert context.valid is True
|
||||||
|
assert context.data["recipient"]["name"] == "Ada Lovelace"
|
||||||
|
assert context.metadata["case_id"] == "case-42"
|
||||||
|
|
||||||
|
|
||||||
|
def test_runtime_context_reports_schema_failure(tmp_path: Path):
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
context_file.write_text(
|
||||||
|
"schema:\n type: object\n required: [recipient]\ncontext: {}\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
context = load_runtime_context_file(context_file)
|
||||||
|
|
||||||
|
assert context.valid is False
|
||||||
|
assert context.diagnostics[0].code == "runtime.context.schema"
|
||||||
|
|
||||||
|
|
||||||
|
def test_form_state_prefills_defaults_hides_fields_and_calculates_values(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
document = parse_markdown(LETTER_DOC, source_path="letter.md")
|
||||||
|
|
||||||
|
form_state = evaluate_form_state(
|
||||||
|
document,
|
||||||
|
load_contract_file(contract_file),
|
||||||
|
load_runtime_context_file(context_file),
|
||||||
|
)
|
||||||
|
fields = {field.id: field for field in form_state.fields}
|
||||||
|
|
||||||
|
assert form_state.valid is True
|
||||||
|
assert fields["recipient_name"].origin == "prefilled"
|
||||||
|
assert fields["delivery_channel"].origin == "defaulted"
|
||||||
|
assert fields["postal_address"].visible is False
|
||||||
|
assert fields["contact_label"].origin == "calculated"
|
||||||
|
assert fields["contact_label"].value == "Markitect Team <hello@example.com>"
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_conflict_keeps_manual_document_value_as_warning(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
document = parse_markdown(
|
||||||
|
LETTER_DOC.replace(
|
||||||
|
"document_type: business-letter",
|
||||||
|
"document_type: business-letter\nrecipient_name: Grace Hopper",
|
||||||
|
),
|
||||||
|
source_path="letter.md",
|
||||||
|
)
|
||||||
|
|
||||||
|
form_state = evaluate_form_state(
|
||||||
|
document,
|
||||||
|
load_contract_file(contract_file),
|
||||||
|
load_runtime_context_file(context_file),
|
||||||
|
)
|
||||||
|
recipient = next(field for field in form_state.fields if field.id == "recipient_name")
|
||||||
|
|
||||||
|
assert form_state.valid is True
|
||||||
|
assert recipient.value == "Grace Hopper"
|
||||||
|
assert "runtime.field.conflict" in {diagnostic.code for diagnostic in form_state.diagnostics}
|
||||||
|
|
||||||
|
|
||||||
|
def test_dynamic_rule_requires_print_address(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
document = parse_markdown(
|
||||||
|
LETTER_DOC.replace(
|
||||||
|
"document_type: business-letter",
|
||||||
|
"document_type: business-letter\ndelivery_channel: print",
|
||||||
|
),
|
||||||
|
source_path="letter.md",
|
||||||
|
)
|
||||||
|
|
||||||
|
form_state = evaluate_form_state(
|
||||||
|
document,
|
||||||
|
load_contract_file(contract_file),
|
||||||
|
load_runtime_context_file(context_file),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert form_state.valid is False
|
||||||
|
assert "contract.field.missing" in {diagnostic.code for diagnostic in form_state.diagnostics}
|
||||||
|
|
||||||
|
|
||||||
|
def test_dynamic_section_rule_can_require_section(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "workplan.contract.md"
|
||||||
|
contract_file.write_text(
|
||||||
|
"""# Workplan Contract
|
||||||
|
|
||||||
|
```yaml contract
|
||||||
|
id: dynamic-workplan-v1
|
||||||
|
document:
|
||||||
|
type: workplan
|
||||||
|
fields:
|
||||||
|
status:
|
||||||
|
type: string
|
||||||
|
required: true
|
||||||
|
sections:
|
||||||
|
- id: purpose
|
||||||
|
title: Purpose
|
||||||
|
presence: required
|
||||||
|
- id: decision-point
|
||||||
|
title: Decision Point
|
||||||
|
presence: optional
|
||||||
|
rules:
|
||||||
|
- id: require-decision-when-done
|
||||||
|
if:
|
||||||
|
path: fields.status.value
|
||||||
|
equals: done
|
||||||
|
then:
|
||||||
|
sections:
|
||||||
|
decision-point:
|
||||||
|
presence: required
|
||||||
|
```
|
||||||
|
""",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
document = parse_markdown(
|
||||||
|
"---\ndocument_type: workplan\nstatus: done\n---\n# WP\n\n## Purpose\n\nDone.\n",
|
||||||
|
source_path="workplan.md",
|
||||||
|
)
|
||||||
|
|
||||||
|
form_state = evaluate_form_state(document, load_contract_file(contract_file))
|
||||||
|
|
||||||
|
assert form_state.valid is False
|
||||||
|
assert "runtime.section.missing" in {diagnostic.code for diagnostic in form_state.diagnostics}
|
||||||
|
|
||||||
|
|
||||||
|
def test_contract_check_uses_runtime_context(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
document_file = tmp_path / "letter.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
document_file.write_text(LETTER_DOC, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
|
||||||
|
result = check_markdown_file(document_file, contract_file, context_path=context_file)
|
||||||
|
|
||||||
|
assert result.valid is True
|
||||||
|
assert result.runtime["form_state"]["field_values"]["recipient_name"] == "Ada Lovelace"
|
||||||
|
|
||||||
|
|
||||||
|
def test_contract_cli_accepts_context_and_reports_form_state(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
document_file = tmp_path / "letter.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
document_file.write_text(LETTER_DOC, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
|
||||||
|
check = CliRunner().invoke(
|
||||||
|
main,
|
||||||
|
[
|
||||||
|
"contract",
|
||||||
|
"check",
|
||||||
|
str(document_file),
|
||||||
|
"--contract",
|
||||||
|
str(contract_file),
|
||||||
|
"--context",
|
||||||
|
str(context_file),
|
||||||
|
"--format",
|
||||||
|
"json",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
form_state = CliRunner().invoke(
|
||||||
|
main,
|
||||||
|
[
|
||||||
|
"contract",
|
||||||
|
"form-state",
|
||||||
|
str(document_file),
|
||||||
|
"--contract",
|
||||||
|
str(contract_file),
|
||||||
|
"--context",
|
||||||
|
str(context_file),
|
||||||
|
"--format",
|
||||||
|
"text",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert check.exit_code == 0
|
||||||
|
assert '"runtime"' in check.output
|
||||||
|
assert form_state.exit_code == 0
|
||||||
|
assert "recipient_name: Ada Lovelace [prefilled]" in form_state.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_assessment_runner_normalizes_cache_and_failure_diagnostics(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
document = parse_markdown_file(_write_file(tmp_path / "letter.md", LETTER_DOC))
|
||||||
|
requests = assessment_requests_for_contract(document, load_contract_file(contract_file))
|
||||||
|
|
||||||
|
class Adapter:
|
||||||
|
calls = 0
|
||||||
|
|
||||||
|
def assess(self, request):
|
||||||
|
self.calls += 1
|
||||||
|
return AssessmentResult(
|
||||||
|
rule_id=request.rule_id,
|
||||||
|
passed=False,
|
||||||
|
score=0.2,
|
||||||
|
reason="Tone is too vague.",
|
||||||
|
provider="mock",
|
||||||
|
model="mock-grader",
|
||||||
|
)
|
||||||
|
|
||||||
|
adapter = Adapter()
|
||||||
|
runner = AssessmentRunner(adapter, cache=MemoryAssessmentCache())
|
||||||
|
first = runner.assess(requests[0])
|
||||||
|
second = runner.assess(requests[0])
|
||||||
|
run = runner.assess_all(requests)
|
||||||
|
|
||||||
|
assert first.cached is False
|
||||||
|
assert second.cached is True
|
||||||
|
assert adapter.calls == 1
|
||||||
|
assert "runtime.assessment.failed" in {diagnostic.code for diagnostic in run.diagnostics}
|
||||||
|
|
||||||
|
|
||||||
|
def test_workflow_form_state_step_uses_context(tmp_path: Path):
|
||||||
|
contract_file = tmp_path / "letter.contract.md"
|
||||||
|
document_file = tmp_path / "letter.md"
|
||||||
|
context_file = tmp_path / "context.yaml"
|
||||||
|
workflow_file = tmp_path / "workflow.yaml"
|
||||||
|
contract_file.write_text(LETTER_CONTRACT, encoding="utf-8")
|
||||||
|
document_file.write_text(LETTER_DOC, encoding="utf-8")
|
||||||
|
context_file.write_text(LETTER_CONTEXT, encoding="utf-8")
|
||||||
|
workflow_file.write_text(
|
||||||
|
"""metadata:
|
||||||
|
id: runtime-workflow
|
||||||
|
steps:
|
||||||
|
- id: form
|
||||||
|
kind: form_state
|
||||||
|
document: letter.md
|
||||||
|
contract: letter.contract.md
|
||||||
|
context: context.yaml
|
||||||
|
""",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
result = WorkflowRunner(load_workflow_file(workflow_file)).run()
|
||||||
|
|
||||||
|
assert result.valid is True
|
||||||
|
assert result.steps["form"]["field_values"]["recipient_name"] == "Ada Lovelace"
|
||||||
|
|
||||||
|
|
||||||
|
def _write_file(path: Path, text: str) -> Path:
|
||||||
|
path.write_text(text, encoding="utf-8")
|
||||||
|
return path
|
||||||
@@ -3,10 +3,10 @@ id: MKTT-WP-0005
|
|||||||
type: workplan
|
type: workplan
|
||||||
title: "Runtime Context, Form, and Assessment Engines"
|
title: "Runtime Context, Form, and Assessment Engines"
|
||||||
domain: markitect
|
domain: markitect
|
||||||
status: todo
|
status: done
|
||||||
owner: markitect-tool
|
owner: markitect-tool
|
||||||
topic_slug: markitect
|
topic_slug: markitect
|
||||||
planning_priority: P2
|
planning_priority: complete
|
||||||
planning_order: 70
|
planning_order: 70
|
||||||
depends_on_workplans:
|
depends_on_workplans:
|
||||||
- MKTT-WP-0003
|
- MKTT-WP-0003
|
||||||
@@ -31,7 +31,11 @@ This workplan picks up the deferred runtime scope from
|
|||||||
|
|
||||||
## Decision
|
## Decision
|
||||||
|
|
||||||
Do not start this immediately unless one of these is true:
|
This workplan is implemented. It was picked up after the workflow and internal
|
||||||
|
extension framework work made runtime form/context evaluation useful and cleanly
|
||||||
|
integrable.
|
||||||
|
|
||||||
|
The original trigger conditions were:
|
||||||
|
|
||||||
- We are implementing template/generation flows that need reliable field
|
- We are implementing template/generation flows that need reliable field
|
||||||
prefill and pre-render validation.
|
prefill and pre-render validation.
|
||||||
@@ -41,16 +45,12 @@ Do not start this immediately unless one of these is true:
|
|||||||
- A user-facing or agent-facing workflow needs structured form state, defaults,
|
- A user-facing or agent-facing workflow needs structured form state, defaults,
|
||||||
conditional requiredness, or guided repair.
|
conditional requiredness, or guided repair.
|
||||||
|
|
||||||
Recommended sequencing:
|
Implemented sequencing:
|
||||||
|
|
||||||
1. Implement context and form runtime first.
|
1. Implement context and form runtime first.
|
||||||
2. Add deterministic context-aware rules.
|
2. Add deterministic context-aware rules.
|
||||||
3. Add LLM assessment execution only after the diagnostic/caching boundary is
|
3. Add provider-neutral assessment execution after the diagnostic/caching
|
||||||
stable.
|
boundary was stable.
|
||||||
|
|
||||||
This is probably not the next immediate implementation if we want to first
|
|
||||||
finish core query/extraction and deterministic transform primitives. It should
|
|
||||||
come before serious generation pipelines or any LLM review loop.
|
|
||||||
|
|
||||||
## Background
|
## Background
|
||||||
|
|
||||||
@@ -76,7 +76,7 @@ It does not yet execute:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T001
|
id: MKTT-WP-0005-T001
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "e24e6238-efef-41c4-9f1e-ca677c1be89b"
|
state_hub_task_id: "e24e6238-efef-41c4-9f1e-ca677c1be89b"
|
||||||
```
|
```
|
||||||
@@ -96,7 +96,7 @@ Expected output: design notes and tests for context loading.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T002
|
id: MKTT-WP-0005-T002
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "d180bb6d-dae8-4305-88de-64c80b708b8a"
|
state_hub_task_id: "d180bb6d-dae8-4305-88de-64c80b708b8a"
|
||||||
```
|
```
|
||||||
@@ -114,7 +114,7 @@ application-specific data access belongs in adapters outside the core package.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T003
|
id: MKTT-WP-0005-T003
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "b954984a-6f67-4e5b-8744-35e3c4fcc992"
|
state_hub_task_id: "b954984a-6f67-4e5b-8744-35e3c4fcc992"
|
||||||
```
|
```
|
||||||
@@ -135,7 +135,7 @@ document exists.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T004
|
id: MKTT-WP-0005-T004
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "cccdf868-2308-42a1-b564-8b54fccd3c8b"
|
state_hub_task_id: "cccdf868-2308-42a1-b564-8b54fccd3c8b"
|
||||||
```
|
```
|
||||||
@@ -157,7 +157,7 @@ This should support future UI adapters while remaining useful from the CLI.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T005
|
id: MKTT-WP-0005-T005
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "6e420e1e-2465-40d3-8e64-d8681a294e63"
|
state_hub_task_id: "6e420e1e-2465-40d3-8e64-d8681a294e63"
|
||||||
```
|
```
|
||||||
@@ -178,7 +178,7 @@ small set of operators over embedding a general programming language.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T006
|
id: MKTT-WP-0005-T006
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "24b22b3a-e89e-4946-81f4-94f971a11979"
|
state_hub_task_id: "24b22b3a-e89e-4946-81f4-94f971a11979"
|
||||||
```
|
```
|
||||||
@@ -205,7 +205,7 @@ implementation.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T007
|
id: MKTT-WP-0005-T007
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "b09b77e2-59c0-4d31-b246-685b742d111f"
|
state_hub_task_id: "b09b77e2-59c0-4d31-b246-685b742d111f"
|
||||||
```
|
```
|
||||||
@@ -220,7 +220,7 @@ cache may be local file-based only if it remains transparent and easy to reset.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: MKTT-WP-0005-T008
|
id: MKTT-WP-0005-T008
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "2efb8233-3154-4824-a898-6fcde37330c5"
|
state_hub_task_id: "2efb8233-3154-4824-a898-6fcde37330c5"
|
||||||
```
|
```
|
||||||
@@ -237,14 +237,29 @@ prefill, invalid dynamic rules, and assessment failures.
|
|||||||
|
|
||||||
## Open Questions
|
## Open Questions
|
||||||
|
|
||||||
- Should context values override frontmatter, or should conflicts always be
|
- Resolved: document/frontmatter values win over context; conflicts are
|
||||||
diagnostics until explicitly resolved?
|
diagnostics and can be escalated per field.
|
||||||
- Should the first dynamic rule syntax reuse JSON Schema conditionals or define
|
- Resolved: dynamic rules use a smaller Markitect-native vocabulary over
|
||||||
a smaller Markitect-native rule vocabulary?
|
JSON/YAML paths instead of embedding JSON Schema conditionals.
|
||||||
- Should LLM assessment execution live behind an optional extra, or only in
|
- Resolved: provider execution remains external; core defines request/result,
|
||||||
external adapters?
|
runner, diagnostics, and cache boundary only.
|
||||||
- What cache invalidation metadata is sufficient for assessment reproducibility
|
- Resolved for core: assessment cache keys include contract/rule/scope/text,
|
||||||
without pretending model judgments are deterministic?
|
criteria, context, structured inputs, provider/model metadata, threshold, and
|
||||||
|
metadata. Persistent cache invalidation remains a backend concern.
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
|
||||||
|
- Added `markitect_tool.runtime` with context loading, path resolution,
|
||||||
|
deterministic condition evaluation, form state, dynamic rules, assessment
|
||||||
|
request/result models, runner, and in-memory cache boundary.
|
||||||
|
- `mkt contract check` now accepts `--context`.
|
||||||
|
- Added `mkt contract form-state`.
|
||||||
|
- Workflow `contract_check` accepts context and workflow `form_state` exposes
|
||||||
|
runtime field state as a step result.
|
||||||
|
- Built-in extension descriptors now include `runtime.context`,
|
||||||
|
`runtime.form-state`, and `runtime.assessment`.
|
||||||
|
- Documentation lives in `docs/runtime-context-forms-assessments.md`.
|
||||||
|
- Examples live in `examples/runtime/`.
|
||||||
|
|
||||||
## Exit Criteria
|
## Exit Criteria
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user