From 8361f9ea457ad92a6d7b1c2d197f1308f1ad57dc Mon Sep 17 00:00:00 2001 From: tegwick Date: Mon, 4 May 2026 13:52:29 +0200 Subject: [PATCH] context loading, path resolution, form state, dynamic rules, and provider-neutral assessment runner/cache boundary --- docs/contract-framework.md | 53 +- docs/internal-extension-framework.md | 2 + docs/markdown-workflows.md | 3 +- docs/runtime-context-forms-assessments.md | 220 +++++ docs/workflow-definition-standard.md | 3 +- docs/workplan-planning-map.md | 2 +- examples/runtime/business-letter-prefill.md | 23 + examples/runtime/business-letter.context.yaml | 19 + .../concept-note-assessment.contract.md | 27 + examples/runtime/prd-frs.context.yaml | 13 + .../runtime/runtime-expected-diagnostics.md | 25 + .../runtime/workplan-done-missing-decision.md | 18 + examples/runtime/workplan-dynamic.contract.md | 46 + src/markitect_tool/__init__.py | 32 + src/markitect_tool/cli/main.py | 81 +- src/markitect_tool/contract/checker.py | 34 +- src/markitect_tool/contract/loader.py | 1 + src/markitect_tool/contract/model.py | 3 + src/markitect_tool/extension/builtins.py | 68 ++ src/markitect_tool/runtime/__init__.py | 53 ++ src/markitect_tool/runtime/assessment.py | 309 +++++++ src/markitect_tool/runtime/context.py | 297 +++++++ src/markitect_tool/runtime/forms.py | 839 ++++++++++++++++++ src/markitect_tool/runtime/paths.py | 45 + src/markitect_tool/runtime/rules.py | 162 ++++ src/markitect_tool/workflow/engine.py | 24 +- tests/test_builtin_extension_catalog.py | 17 + .../test_runtime_context_forms_assessment.py | 388 ++++++++ ...-runtime-context-and-assessment-engines.md | 67 +- 29 files changed, 2809 insertions(+), 65 deletions(-) create mode 100644 docs/runtime-context-forms-assessments.md create mode 100644 examples/runtime/business-letter-prefill.md create mode 100644 examples/runtime/business-letter.context.yaml create mode 100644 examples/runtime/concept-note-assessment.contract.md create mode 100644 examples/runtime/prd-frs.context.yaml create mode 100644 examples/runtime/runtime-expected-diagnostics.md create mode 100644 examples/runtime/workplan-done-missing-decision.md create mode 100644 examples/runtime/workplan-dynamic.contract.md create mode 100644 src/markitect_tool/runtime/__init__.py create mode 100644 src/markitect_tool/runtime/assessment.py create mode 100644 src/markitect_tool/runtime/context.py create mode 100644 src/markitect_tool/runtime/forms.py create mode 100644 src/markitect_tool/runtime/paths.py create mode 100644 src/markitect_tool/runtime/rules.py create mode 100644 tests/test_runtime_context_forms_assessment.py diff --git a/docs/contract-framework.md b/docs/contract-framework.md index dd691e5..8e1f7fd 100644 --- a/docs/contract-framework.md +++ b/docs/contract-framework.md @@ -9,9 +9,9 @@ Markdown as the authoring surface and uses fenced YAML as a structured extension for rules that need machine interpretation. The first implementation is deterministic. It checks document type, fields, -sections, ordering, metric bands, and text assertions. Forms, context, and LLM -rubrics are represented in the contract vocabulary as extension points before -runtime adapters are added. +sections, ordering, metric bands, and text assertions. Runtime context, forms, +dynamic rules, and provider-neutral assessment requests are implemented as +extensions around the same contract vocabulary. ## Contract File Shape @@ -112,10 +112,10 @@ fields, and metric bands. This is the bridge to later LLM rubrics: semantic checks can become additional assessments without changing how failures are reported. -## Forms And Context +## Forms, Context, And Runtime Rules -Field specs are the first step toward form-backed Markdown generation. Runtime -form handling should build on the same field vocabulary: +Field specs are the foundation for form-backed Markdown generation and +context-aware checks. Runtime form handling uses the same field vocabulary: - `id` - `type` @@ -128,15 +128,25 @@ form handling should build on the same field vocabulary: - `min` / `max` - `min_length` / `max_length` -Dynamic requiredness, visibility, calculations, and prefill should be declared -as context-aware rules in later work. The contract should remain the source of -truth, while UI and generation layers act as adapters. +Runtime context can be supplied as local YAML or JSON: + +```text +mkt contract check --contract --context +mkt contract form-state --contract --context +``` + +The runtime resolves fields in this order: document value, context source, +default, missing. Document values win over context and conflicts are diagnostics. +Dynamic rules support small deterministic `if` / `then` / `else` expressions +for requiredness, visibility, allowed values, calculated values, context +assertions, and dynamic section presence. See +`docs/runtime-context-forms-assessments.md`. ## LLM Assessment Extension -LLM-assisted checks should be declared as rubrics, scoped to document or section -roles. Core Markitect should not call a provider directly. A future adapter -should accept a provider-neutral request: +LLM-assisted checks are declared as rubrics, scoped to document or section roles. +Core Markitect does not call a provider directly. It creates provider-neutral +assessment requests for injected adapters: - contract id and rule id - document or section text @@ -152,26 +162,11 @@ It should return: - model/provider metadata - diagnostics using the shared diagnostic model -## Deferred Runtime Work - -The deterministic contract framework is ready now. The runtime engines are -deferred to `MKTT-WP-0005-runtime-context-and-assessment-engines.md`. - -Pick that work up when one of these becomes true: - -- contract checks need external user, project, or entity context -- generation needs reliable field prefill before rendering -- a UI or agent workflow needs form state, defaults, and dynamic requiredness -- deterministic section assertions are not enough and rubric-based semantic - assessment becomes necessary - -The intended order is context and form runtime first, deterministic dynamic -rules second, LLM assessment execution third. - ## CLI ```text mkt contract validate -mkt contract check --contract +mkt contract check --contract [--context ] +mkt contract form-state --contract [--context ] mkt metrics ``` diff --git a/docs/internal-extension-framework.md b/docs/internal-extension-framework.md index e77af7a..5f37ed3 100644 --- a/docs/internal-extension-framework.md +++ b/docs/internal-extension-framework.md @@ -34,6 +34,8 @@ framework organizes how Markitect itself exposes and composes capabilities. | `backend` | local SQLite index | snapshots/index/search storage | | `reference-provider` | section, region, fence, line | address in, content units out | | `validator` | schema, contract, section assertion | document/context in, diagnostics out | +| `runtime` | context loader, form state, dynamic rules | document/contract/context in, diagnostics and state out | +| `assessment-runner` | provider-neutral rubric execution | assessment request in, normalized result out | | `template-engine` | deterministic templates | template/data in, Markdown out | | `generation-adapter` | provider-neutral assisted generation | request in, generated candidate out | | `cli-group` | cache, backend, ref, class | command descriptors or registration hook | diff --git a/docs/markdown-workflows.md b/docs/markdown-workflows.md index 626ff09..f0a407d 100644 --- a/docs/markdown-workflows.md +++ b/docs/markdown-workflows.md @@ -72,7 +72,8 @@ This makes workflows useful without provider dependencies. | `transform` | transformed `markdown`, operations, provenance. | | `include` | include-resolved `markdown`, included paths, provenance. | | `contract_stub` | generated contract stub Markdown. | -| `contract_check` | contract diagnostics and metrics. | +| `contract_check` | contract diagnostics, metrics, and optional runtime context results. | +| `form_state` | field values, origins, dynamic rule results, and diagnostics. | | `assisted` | generated Markdown if a hook is supplied, otherwise skipped/diagnostic. | ## Data Bindings diff --git a/docs/runtime-context-forms-assessments.md b/docs/runtime-context-forms-assessments.md new file mode 100644 index 0000000..8295840 --- /dev/null +++ b/docs/runtime-context-forms-assessments.md @@ -0,0 +1,220 @@ +# Runtime Context, Forms, Rules, And Assessments + +Date: 2026-05-04 + +## Purpose + +The runtime layer turns contract extension points into executable behavior while +keeping the deterministic contract framework intact. Static checks still handle +document type, sections, assertions, and metric bands. Runtime checks add +external context, field prefill, UI-neutral form state, dynamic rules, and a +provider-neutral assessment protocol. + +The layer is deliberately local-first. Core Markitect reads YAML or JSON context +files and runs deterministic rules. Network calls, application lookups, and LLM +providers belong behind adapters. + +## Context Files + +Runtime context can be a raw YAML/JSON mapping: + +```yaml +recipient: + name: Ada Lovelace +sender: + name: Markitect Team +``` + +or an envelope with metadata and schema: + +```yaml +metadata: + case_id: case-42 +schema: + type: object + required: [recipient, sender] +context: + recipient: + name: Ada Lovelace + sender: + name: Markitect Team +``` + +The value under `context` is bound as `context` in field sources and dynamic +rules. `schema` validates the full context object. `schemas` can validate named +objects individually. + +Malformed context and schema failures produce normal diagnostics: + +- `runtime.context.malformed` +- `runtime.context.schema_invalid` +- `runtime.context.schema_target_missing` +- `runtime.context.schema` + +## Field Runtime + +Field specs continue to live in the contract: + +```yaml +fields: + recipient_name: + type: string + required: true + source: context.recipient.name + delivery_channel: + type: string + default: email + enum: [email, print] +``` + +Runtime resolution order is: + +1. Manual document value from `path`, usually frontmatter. +2. Context value from `source` or `sources`. +3. Contract `default`. +4. Missing. + +Manual document values win over context. If both exist and differ, Markitect +emits `runtime.field.conflict` as a warning by default. A field can set +`conflict: error` to make that stricter. Multiple context sources with distinct +values produce `runtime.field.ambiguous`. + +`mkt contract check` uses runtime evaluation only when `--context` is supplied: + +```text +mkt contract check document.md --contract contract.md --context context.yaml +``` + +`mkt contract form-state` always emits the UI-neutral runtime view: + +```text +mkt contract form-state document.md --contract contract.md --context context.yaml +``` + +## Form State + +Form state is not a UI framework. It is a stable contract that future UIs, +agents, generators, and workflow steps can render: + +- field id +- value +- origin: `manual`, `prefilled`, `defaulted`, `calculated`, or `missing` +- required/optional +- visible/hidden +- enabled/disabled +- allowed values +- diagnostics +- metadata + +Hidden fields are not required unless a future adapter explicitly asks for +hidden validation. This matches practical form behavior and avoids punishing +authors for data that the current context made irrelevant. + +## Dynamic Rules + +Rules are deterministic YAML. They use a deliberately small condition language: + +```yaml +rules: + - id: postal-address-for-print + if: + path: fields.delivery_channel.value + equals: print + then: + required: [postal_address] + visible: + postal_address: true + else: + hidden: [postal_address] +``` + +Supported condition operators: + +- `exists` +- `equals` / `eq` +- `not_equals` +- `in` +- `contains` +- `matches` +- `gt`, `gte`, `lt`, `lte` +- `all`, `any`, `not` + +Supported actions: + +- `required` / `optional` +- `visible` / `hidden` +- `enabled` / `disabled` +- `allowed_values` +- `set` +- `assert` +- `sections` + +Calculated values can reference runtime paths: + +```yaml +then: + set: + contact_label: "${fields.sender_name.value} <${context.sender.email}>" +``` + +Context assertions use the same condition vocabulary: + +```yaml +assert: + path: context.sender.email + matches: "@example\\.com$" + message: Sender email must come from example.com. + severity: warning +``` + +Dynamic section rules are intentionally narrow. They can require, recommend, +discourage, or forbid section specs already declared in the contract. + +## Assessment Protocol + +Rubrics remain provider-neutral contract declarations: + +```yaml +rubrics: + - id: tone-fit + scope: section.body + criteria: The body should match the recipient relationship. + threshold: 0.75 +``` + +Core Markitect turns rubrics into `AssessmentRequest` objects and normalizes +adapter results into `AssessmentResult` and diagnostics. It does not call an LLM +provider directly. The cache key includes contract id, rule id, scope, text, +criteria, context, structured inputs, threshold, provider, model, and metadata. + +Adapters can be injected from workflows, applications, or tests. A transparent +in-memory cache exists for tests and short runs; persistent storage remains a +backend concern. + +## Workflow Integration + +Workflow `contract_check` steps accept `context`: + +```yaml +steps: + - id: check-letter + kind: contract_check + document: letter.md + contract: letter.contract.md + context: letter.context.yaml +``` + +Workflow `form_state` steps expose the runtime state as a step result: + +```yaml +steps: + - id: form + kind: form_state + document: letter.md + contract: letter.contract.md + context: letter.context.yaml +``` + +This keeps workflow orchestration separate from the runtime engine. The runtime +engine answers "what does this contract imply in this context"; the workflow +engine decides when to run it and where to send the output. diff --git a/docs/workflow-definition-standard.md b/docs/workflow-definition-standard.md index b80908e..a5412c6 100644 --- a/docs/workflow-definition-standard.md +++ b/docs/workflow-definition-standard.md @@ -188,7 +188,8 @@ First implementation step kinds: | `transform` | Apply deterministic Markdown transforms. | | `include` | Resolve include markers in Markdown. | | `contract_stub` | Generate a Markdown stub from a contract. | -| `contract_check` | Check a Markdown document against a contract. | +| `contract_check` | Check a Markdown document against a contract, optionally with runtime context. | +| `form_state` | Evaluate UI-neutral field prefill, validation, and dynamic rules. | | `assisted` | Provider-neutral assisted step boundary, optional by default. | ## Data Bindings diff --git a/docs/workplan-planning-map.md b/docs/workplan-planning-map.md index 8fd9de2..5c305f3 100644 --- a/docs/workplan-planning-map.md +++ b/docs/workplan-planning-map.md @@ -35,7 +35,7 @@ and descriptions mirror the operational view. | `MKTT-WP-0010` | complete | done | `MKTT-WP-0004`; task-level trigger: `MKTT-WP-0003-T006` | Content references, processors, explode/implode, weave/tangle, content classes, and migration examples are complete as the first WP-0010 extension layer. | | `MKTT-WP-0007` | complete | done | `MKTT-WP-0006` | Advanced query and local index backend is complete: AST inspection, optional JSONPath, SQLite snapshots/metadata, FTS5 search, incremental refresh, and local index CLI. | | `MKTT-WP-0013` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0004`, `MKTT-WP-0006`, `MKTT-WP-0007`, `MKTT-WP-0010` | Internal extension framework is complete: characterization tests, canonical processing model, descriptors, registries, lifecycle callbacks, query-engine registry, built-in extension catalog, CLI command specs, and authoring guide. | -| `MKTT-WP-0005` | P2 | todo | `MKTT-WP-0003`, `MKTT-WP-0004` | Pick up when generation/form/context or semantic assessment pressure appears. | +| `MKTT-WP-0005` | complete | done | `MKTT-WP-0003`, `MKTT-WP-0004` | Runtime context, form state, dynamic rules, workflow integration, and provider-neutral assessment boundary are complete. | | `MKTT-WP-0011` | complete | done | `MKTT-WP-0003`; task-level triggers: `MKTT-WP-0010-T001`, `MKTT-WP-0010-T005` | Markdown dataflow workflow layer is complete: workflow standard, source collectors, binding model, deterministic steps, assisted boundary, safe outputs, CLI, docs, and examples. | | `MKTT-WP-0009` | P2 | todo | `MKTT-WP-0006` | Establish access-control gateway before security-sensitive cache/context use. | | `MKTT-WP-0012` | P3 | todo | `MKTT-WP-0004`, `MKTT-WP-0010`, `MKTT-WP-0011` | Future Quarkdown-inspired document function layer: reusable Markdown-native function calls over processors, references, contracts, workflows, and later assisted steps. | diff --git a/examples/runtime/business-letter-prefill.md b/examples/runtime/business-letter-prefill.md new file mode 100644 index 0000000..30a4ccb --- /dev/null +++ b/examples/runtime/business-letter-prefill.md @@ -0,0 +1,23 @@ +--- +document_type: business-letter +--- + +# Follow-Up Letter + +## Greeting + +Dear Ada Lovelace, + +## Body + +Thank you for the thoughtful discussion about structured Markdown generation. +We reviewed the requirements and will send a concise proposal that separates +document contracts, field prefill, validation diagnostics, and optional semantic +assessment. This keeps the implementation practical while leaving room for +future automation. + +## Closing + +Kind regards, + +Markitect Team diff --git a/examples/runtime/business-letter.context.yaml b/examples/runtime/business-letter.context.yaml new file mode 100644 index 0000000..22b7934 --- /dev/null +++ b/examples/runtime/business-letter.context.yaml @@ -0,0 +1,19 @@ +metadata: + case_id: letter-ada-001 +schema: + type: object + required: [recipient, sender] + properties: + recipient: + type: object + required: [name] + sender: + type: object + required: [name, email] +context: + recipient: + name: Ada Lovelace + relationship: research collaborator + sender: + name: Markitect Team + email: hello@example.com diff --git a/examples/runtime/concept-note-assessment.contract.md b/examples/runtime/concept-note-assessment.contract.md new file mode 100644 index 0000000..5e4fac6 --- /dev/null +++ b/examples/runtime/concept-note-assessment.contract.md @@ -0,0 +1,27 @@ +# Concept Note Assessment Contract + +```yaml contract +id: concept-note-assessment-v1 +document: + type: concept-note +fields: + status: + type: string + required: true + enum: [draft, review, accepted] +sections: + - id: concept + title: Concept + presence: required + level: 2 + - id: utility + title: Utility + presence: required + level: 2 +rubrics: + - id: utility-is-practical + scope: section.utility + criteria: The utility section should explain who benefits, what changes in practice, and how success can be recognized. + threshold: 0.75 + severity: warning +``` diff --git a/examples/runtime/prd-frs.context.yaml b/examples/runtime/prd-frs.context.yaml new file mode 100644 index 0000000..c162eef --- /dev/null +++ b/examples/runtime/prd-frs.context.yaml @@ -0,0 +1,13 @@ +metadata: + release: runtime-context-alpha +context: + product: + name: Markitect Tool + owner: Platform Architecture + regulatory_tier: internal + project: + target_date: "2026-06-30" + stakeholders: + - Documentation + - Architecture + - Product Operations diff --git a/examples/runtime/runtime-expected-diagnostics.md b/examples/runtime/runtime-expected-diagnostics.md new file mode 100644 index 0000000..197ea01 --- /dev/null +++ b/examples/runtime/runtime-expected-diagnostics.md @@ -0,0 +1,25 @@ +# Runtime Expected Diagnostics + +These examples are meant for manual inspection and future snapshot fixtures. + +```text +mkt contract check examples/runtime/workplan-done-missing-decision.md \ + --contract examples/runtime/workplan-dynamic.contract.md \ + --format text +``` + +Expected diagnostic: + +- `runtime.section.missing`: status `done` requires a `Decision Point` section. + +```text +mkt contract check examples/runtime/business-letter-prefill.md \ + --contract examples/contracts/business-letter.contract.md \ + --context examples/runtime/business-letter.context.yaml \ + --format text +``` + +Expected runtime utility: + +- `recipient_name` and `sender_name` are prefilled from context. +- No field-missing diagnostic is emitted for those fields. diff --git a/examples/runtime/workplan-done-missing-decision.md b/examples/runtime/workplan-done-missing-decision.md new file mode 100644 index 0000000..90d0633 --- /dev/null +++ b/examples/runtime/workplan-done-missing-decision.md @@ -0,0 +1,18 @@ +--- +document_type: workplan +id: MKTT-WP-EXAMPLE +status: done +--- + +# Example Workplan + +## Purpose + +Show how a dynamic section rule reports missing completion context. + +## Tasks + +```task +id: MKTT-WP-EXAMPLE-T001 +status: done +``` diff --git a/examples/runtime/workplan-dynamic.contract.md b/examples/runtime/workplan-dynamic.contract.md new file mode 100644 index 0000000..35a67f2 --- /dev/null +++ b/examples/runtime/workplan-dynamic.contract.md @@ -0,0 +1,46 @@ +# Dynamic Workplan Contract + +```yaml contract +id: dynamic-workplan-contract-v1 +document: + type: workplan +fields: + id: + type: string + required: true + status: + type: string + required: true + enum: [proposed, active, done, deferred] + owner: + type: string + source: context.workplan.owner +sections: + - id: purpose + title: Purpose + presence: required + level: 2 + - id: tasks + title: Tasks + presence: required + level: 2 + - id: decision-point + title: Decision Point + presence: optional + level: 2 +rules: + - id: require-decision-point-when-done + if: + path: fields.status.value + equals: done + then: + sections: + decision-point: + presence: required + - id: owner-needed-for-active-work + if: + path: fields.status.value + equals: active + then: + required: [owner] +``` diff --git a/src/markitect_tool/__init__.py b/src/markitect_tool/__init__.py index 2e72e46..6e4cc03 100644 --- a/src/markitect_tool/__init__.py +++ b/src/markitect_tool/__init__.py @@ -140,6 +140,23 @@ from markitect_tool.reference import ( parse_reference, resolve_reference, ) +from markitect_tool.runtime import ( + AssessmentRequest, + AssessmentResult, + AssessmentRunResult, + AssessmentRunner, + FieldState, + FormState, + MemoryAssessmentCache, + RuntimeContext, + RuntimeContextLoadResult, + RuntimeContextSource, + assessment_requests_for_contract, + evaluate_form_state, + load_runtime_context_file, + load_runtime_context_file_result, + run_contract_assessments, +) from markitect_tool.schema import ( MarkdownSchema, SchemaValidationResult, @@ -290,6 +307,21 @@ __all__ = [ "load_namespaces", "parse_reference", "resolve_reference", + "AssessmentRequest", + "AssessmentResult", + "AssessmentRunResult", + "AssessmentRunner", + "FieldState", + "FormState", + "MemoryAssessmentCache", + "RuntimeContext", + "RuntimeContextLoadResult", + "RuntimeContextSource", + "assessment_requests_for_contract", + "evaluate_form_state", + "load_runtime_context_file", + "load_runtime_context_file_result", + "run_contract_assessments", "MissingTemplateVariable", "TemplateAnalysis", "TemplateError", diff --git a/src/markitect_tool/cli/main.py b/src/markitect_tool/cli/main.py index 9894c70..9ccf7a4 100644 --- a/src/markitect_tool/cli/main.py +++ b/src/markitect_tool/cli/main.py @@ -65,6 +65,7 @@ from markitect_tool.reference import ( load_namespaces, resolve_reference, ) +from markitect_tool.runtime import evaluate_form_state, load_runtime_context_file from markitect_tool.schema import load_schema_file, validate_markdown_file, validate_schema from markitect_tool.template import ( MissingTemplateVariable, @@ -1466,17 +1467,68 @@ def contract_validate(contract_file: Path, output_format: str) -> None: default="text", show_default=True, ) -def contract_check(file: Path, contract_file: Path, output_format: str) -> None: +@click.option( + "--context", + "context_file", + type=click.Path(exists=True, dir_okay=False, path_type=Path), + help="YAML or JSON runtime context used for field prefill and dynamic rules.", +) +def contract_check( + file: Path, + contract_file: Path, + output_format: str, + context_file: Path | None, +) -> None: """Check a Markdown file against a Markdown document contract.""" try: - result = check_markdown_file(file, contract_file) + result = check_markdown_file(file, contract_file, context_path=context_file) except ContractLoaderError as exc: raise click.ClickException(str(exc)) from exc _emit_diagnostic_result(result.to_dict(), output_format) raise click.exceptions.Exit(0 if result.valid else 1) +@contract.command("form-state") +@click.argument("file", type=click.Path(exists=True, dir_okay=False, path_type=Path)) +@click.option( + "--contract", + "contract_file", + required=True, + type=click.Path(exists=True, dir_okay=False, path_type=Path), +) +@click.option( + "--context", + "context_file", + type=click.Path(exists=True, dir_okay=False, path_type=Path), + help="YAML or JSON runtime context used for field prefill and dynamic rules.", +) +@click.option( + "--format", + "output_format", + type=click.Choice(["json", "yaml", "text"], case_sensitive=False), + default="text", + show_default=True, +) +def contract_form_state( + file: Path, + contract_file: Path, + context_file: Path | None, + output_format: str, +) -> None: + """Evaluate UI-neutral form state for a document contract.""" + + try: + document = parse_markdown_file(file) + contract_definition = load_contract_file(contract_file) + context = load_runtime_context_file(context_file) if context_file else None + form_state = evaluate_form_state(document, contract_definition, context) + except ContractLoaderError as exc: + raise click.ClickException(str(exc)) from exc + _emit_form_state(form_state.to_dict(), output_format) + raise click.exceptions.Exit(0 if form_state.valid else 1) + + def _emit_result(data: dict, output_format: str) -> None: if output_format == "json": click.echo(json.dumps(data, indent=2, ensure_ascii=False)) @@ -1511,6 +1563,31 @@ def _emit_diagnostic_result(data: dict, output_format: str) -> None: click.echo(f" guidance: {diagnostic['guidance']}") +def _emit_form_state(data: dict, output_format: str) -> None: + if output_format == "json": + click.echo(json.dumps(data, indent=2, ensure_ascii=False)) + elif output_format == "yaml": + click.echo(yaml.safe_dump(data, sort_keys=False)) + else: + click.echo("valid" if data.get("valid") else "invalid") + for field in data.get("fields", []): + value = field.get("value", "") if field.get("exists") else "" + flags = [] + if field.get("required"): + flags.append("required") + if field.get("visible") is False: + flags.append("hidden") + if field.get("enabled") is False: + flags.append("disabled") + suffix = f" ({', '.join(flags)})" if flags else "" + click.echo(f"- {field['id']}: {value} [{field.get('origin', 'unknown')}]{suffix}") + for diagnostic in data.get("diagnostics", []): + click.echo( + f" [{diagnostic['severity']}] {diagnostic['code']}: " + f"{diagnostic['message']}" + ) + + def _emit_metrics(data: dict, output_format: str) -> None: if output_format == "json": click.echo(json.dumps(data, indent=2, ensure_ascii=False)) diff --git a/src/markitect_tool/contract/checker.py b/src/markitect_tool/contract/checker.py index 167d2c1..e40badc 100644 --- a/src/markitect_tool/contract/checker.py +++ b/src/markitect_tool/contract/checker.py @@ -27,6 +27,7 @@ from markitect_tool.diagnostics import ( has_error, valid_severity, ) +from markitect_tool.runtime import RuntimeContext, evaluate_form_state, load_runtime_context_file @dataclass(frozen=True) @@ -55,6 +56,7 @@ class ContractCheckResult: document_path: str | None = None contract_path: str | None = None metrics: dict[str, Any] = field(default_factory=dict) + runtime: dict[str, Any] = field(default_factory=dict) def to_dict(self) -> dict[str, Any]: data = { @@ -63,6 +65,7 @@ class ContractCheckResult: "document_path": self.document_path, "contract_path": self.contract_path, "metrics": self.metrics or None, + "runtime": self.runtime or None, } return {key: value for key, value in data.items() if value is not None} @@ -131,26 +134,48 @@ def validate_contract(contract: DocumentContract) -> ContractValidationResult: def check_markdown_file( - markdown_path: str | Path, contract_path: str | Path + markdown_path: str | Path, + contract_path: str | Path, + *, + context_path: str | Path | None = None, + runtime_context: RuntimeContext | None = None, ) -> ContractCheckResult: """Parse and check a Markdown file against a contract file.""" document = parse_markdown_file(markdown_path) contract = load_contract_file(contract_path) - return check_document_contract(document, contract) + context = runtime_context + if context_path is not None: + context = load_runtime_context_file(context_path) + return check_document_contract(document, contract, runtime_context=context) def check_document_contract( - document: Document, contract: DocumentContract + document: Document, + contract: DocumentContract, + *, + runtime_context: RuntimeContext | None = None, ) -> ContractCheckResult: """Check a parsed Markdown document against a document contract.""" contract_validation = validate_contract(contract) document_metrics = collect_metrics(document) diagnostics = list(contract_validation.diagnostics) + runtime: dict[str, Any] = {} if contract_validation.valid: diagnostics.extend(_check_document_type(document, contract)) - diagnostics.extend(_check_fields(document, contract)) + if runtime_context is None and not contract.rules: + diagnostics.extend(_check_fields(document, contract)) + else: + form_state = evaluate_form_state( + document, + contract, + runtime_context or RuntimeContext.empty(), + ) + diagnostics.extend(form_state.diagnostics) + context = runtime_context or RuntimeContext.empty() + runtime["context"] = context.to_dict() + runtime["form_state"] = form_state.to_dict() diagnostics.extend(_check_document_metrics(document, contract, document_metrics)) diagnostics.extend(_check_assertions(document.body, contract.assertions, document, contract)) diagnostics.extend(_check_sections(document, contract, document_metrics)) @@ -161,6 +186,7 @@ def check_document_contract( document_path=document.source_path, contract_path=contract.source_path, metrics=document_metrics.to_dict(), + runtime=runtime, ) diff --git a/src/markitect_tool/contract/loader.py b/src/markitect_tool/contract/loader.py index 0636b7c..0e30e13 100644 --- a/src/markitect_tool/contract/loader.py +++ b/src/markitect_tool/contract/loader.py @@ -120,6 +120,7 @@ def _looks_like_contract(data: dict[str, Any]) -> bool: "metric_bands", "assertions", "forms", + "rules", "rubrics", } ) diff --git a/src/markitect_tool/contract/model.py b/src/markitect_tool/contract/model.py index 2c518fc..cd6980f 100644 --- a/src/markitect_tool/contract/model.py +++ b/src/markitect_tool/contract/model.py @@ -223,6 +223,7 @@ class DocumentContract: assertions: list[AssertionSpec] = field(default_factory=list) forms: list[dict[str, Any]] = field(default_factory=list) context: dict[str, Any] = field(default_factory=dict) + rules: list[dict[str, Any]] = field(default_factory=list) rubrics: list[dict[str, Any]] = field(default_factory=list) metadata: dict[str, Any] = field(default_factory=dict) raw: dict[str, Any] = field(default_factory=dict) @@ -264,6 +265,7 @@ class DocumentContract: assertions=assertions_from_value(raw.get("assertions")), forms=raw.get("forms") if isinstance(raw.get("forms"), list) else [], context=raw.get("context") if isinstance(raw.get("context"), dict) else {}, + rules=raw.get("rules") if isinstance(raw.get("rules"), list) else [], rubrics=raw.get("rubrics") if isinstance(raw.get("rubrics"), list) else [], metadata=metadata, raw=raw, @@ -284,6 +286,7 @@ class DocumentContract: "assertions": [assertion.raw for assertion in self.assertions], "forms": self.forms, "context": self.context, + "rules": self.rules, "rubrics": self.rubrics, "source_path": self.source_path, } diff --git a/src/markitect_tool/extension/builtins.py b/src/markitect_tool/extension/builtins.py index f021b46..8477209 100644 --- a/src/markitect_tool/extension/builtins.py +++ b/src/markitect_tool/extension/builtins.py @@ -14,6 +14,9 @@ def builtin_extension_registry() -> ExtensionRegistry: for descriptor in _processor_descriptors() + [ _local_sqlite_backend_descriptor(), _workflow_engine_descriptor(), + _runtime_context_descriptor(), + _runtime_form_state_descriptor(), + _runtime_assessment_descriptor(), ]: registry.register(descriptor) return registry @@ -120,3 +123,68 @@ def _workflow_engine_descriptor() -> ExtensionDescriptor: docs=["docs/workflow-definition-standard.md"], examples=["examples/workflows/adr-release-notes.workflow.md"], ) + + +def _runtime_context_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="runtime.context", + kind="runtime", + summary="YAML/JSON runtime context loader for document contracts.", + capabilities=[ + ProcessingCapability(id="context", kind="read"), + ProcessingCapability(id="json-schema", kind="validate"), + ProcessingCapability(id="diagnostics", kind="emit"), + ], + safety={"reads_files": True, "network": False}, + input_contract="YAML/JSON runtime context file", + output_contract="RuntimeContext", + diagnostics_namespace="runtime.context", + provenance_prefix="runtime.context", + cli={"commands": ["mkt contract check --context", "mkt contract form-state"]}, + docs=["docs/runtime-context-forms-assessments.md"], + examples=["examples/runtime/business-letter.context.yaml"], + ) + + +def _runtime_form_state_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="runtime.form-state", + kind="runtime", + summary="UI-neutral field prefill, validation, and dynamic rule engine.", + capabilities=[ + ProcessingCapability(id="forms", kind="evaluate"), + ProcessingCapability(id="rules", kind="evaluate"), + ProcessingCapability(id="contracts", kind="validate"), + ProcessingCapability(id="diagnostics", kind="emit"), + ], + safety={"reads_files": False, "network": False}, + input_contract="Document + DocumentContract + RuntimeContext", + output_contract="FormState", + diagnostics_namespace="runtime", + provenance_prefix="runtime.form_state", + cli={"commands": ["mkt contract form-state"]}, + docs=["docs/runtime-context-forms-assessments.md"], + examples=["examples/runtime/workplan-dynamic.contract.md"], + ) + + +def _runtime_assessment_descriptor() -> ExtensionDescriptor: + return ExtensionDescriptor( + id="runtime.assessment", + kind="assessment-runner", + summary="Provider-neutral rubric assessment request, result, and cache boundary.", + capabilities=[ + ProcessingCapability(id="assessment", kind="execute"), + ProcessingCapability(id="rubrics", kind="read"), + ProcessingCapability(id="cache-key", kind="compute"), + ProcessingCapability(id="diagnostics", kind="emit"), + ], + safety={"network": "adapter-only", "provider_calls": "adapter-only"}, + input_contract="AssessmentRequest", + output_contract="AssessmentResult", + diagnostics_namespace="runtime.assessment", + provenance_prefix="runtime.assessment", + docs=["docs/runtime-context-forms-assessments.md"], + examples=["examples/runtime/concept-note-assessment.contract.md"], + metadata={"provider_implementation": "external adapter required"}, + ) diff --git a/src/markitect_tool/runtime/__init__.py b/src/markitect_tool/runtime/__init__.py new file mode 100644 index 0000000..b6f69ee --- /dev/null +++ b/src/markitect_tool/runtime/__init__.py @@ -0,0 +1,53 @@ +"""Runtime context, form state, rules, and assessment extension APIs.""" + +from markitect_tool.runtime.assessment import ( + AssessmentAdapter, + AssessmentCache, + AssessmentRequest, + AssessmentResult, + AssessmentRunResult, + AssessmentRunner, + MemoryAssessmentCache, + assessment_requests_for_contract, + run_contract_assessments, +) +from markitect_tool.runtime.context import ( + RuntimeContext, + RuntimeContextLoadResult, + RuntimeContextSource, + load_runtime_context_file, + load_runtime_context_file_result, +) +from markitect_tool.runtime.forms import ( + FieldState, + FormState, + build_runtime_bindings, + evaluate_form_state, +) +from markitect_tool.runtime.paths import comparable_value, resolve_path +from markitect_tool.runtime.rules import ConditionResult, evaluate_condition + +__all__ = [ + "AssessmentAdapter", + "AssessmentCache", + "AssessmentRequest", + "AssessmentResult", + "AssessmentRunResult", + "AssessmentRunner", + "ConditionResult", + "FieldState", + "FormState", + "MemoryAssessmentCache", + "RuntimeContext", + "RuntimeContextLoadResult", + "RuntimeContextSource", + "assessment_requests_for_contract", + "build_runtime_bindings", + "comparable_value", + "evaluate_condition", + "evaluate_form_state", + "load_runtime_context_file", + "load_runtime_context_file_result", + "resolve_path", + "run_contract_assessments", +] diff --git a/src/markitect_tool/runtime/assessment.py b/src/markitect_tool/runtime/assessment.py new file mode 100644 index 0000000..53fc394 --- /dev/null +++ b/src/markitect_tool/runtime/assessment.py @@ -0,0 +1,309 @@ +"""Provider-neutral assessment protocol for rubric-backed contract checks.""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import asdict, dataclass, field, replace +from typing import Any, Protocol + +from markitect_tool.contract.model import DocumentContract +from markitect_tool.core import Document, Section +from markitect_tool.diagnostics import Diagnostic, SourceLocation, has_error +from markitect_tool.runtime.context import RuntimeContext + + +@dataclass(frozen=True) +class AssessmentRequest: + """A provider-neutral request for rubric assessment.""" + + contract_id: str | None + rule_id: str + scope: str + text: str + criteria: Any + context: dict[str, Any] = field(default_factory=dict) + structured_inputs: dict[str, Any] = field(default_factory=dict) + severity: str = "error" + threshold: float | None = None + provider: str | None = None + model: str | None = None + metadata: dict[str, Any] = field(default_factory=dict) + + @property + def cache_key(self) -> str: + payload = { + "contract_id": self.contract_id, + "rule_id": self.rule_id, + "scope": self.scope, + "text": self.text, + "criteria": self.criteria, + "context": self.context, + "structured_inputs": self.structured_inputs, + "threshold": self.threshold, + "provider": self.provider, + "model": self.model, + "metadata": self.metadata, + } + return "assessment:" + hashlib.sha256( + json.dumps(payload, sort_keys=True, ensure_ascii=False, default=str).encode("utf-8") + ).hexdigest() + + def to_dict(self) -> dict[str, Any]: + data = asdict(self) + data["cache_key"] = self.cache_key + return _drop_empty(data) + + +@dataclass(frozen=True) +class AssessmentResult: + """Normalized result returned by an assessment adapter.""" + + rule_id: str + passed: bool + score: float | None = None + reason: str | None = None + diagnostics: list[Diagnostic] = field(default_factory=list) + provider: str | None = None + model: str | None = None + cache_key: str | None = None + cached: bool = False + metadata: dict[str, Any] = field(default_factory=dict) + + @property + def valid(self) -> bool: + return self.passed and not has_error(self.diagnostics) + + def to_dict(self) -> dict[str, Any]: + data = { + "rule_id": self.rule_id, + "passed": self.passed, + "score": self.score, + "reason": self.reason, + "diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics], + "provider": self.provider, + "model": self.model, + "cache_key": self.cache_key, + "cached": self.cached, + "metadata": self.metadata, + } + return _drop_empty(data) + + +class AssessmentAdapter(Protocol): + """Adapter boundary for an LLM or other semantic grader.""" + + def assess(self, request: AssessmentRequest) -> AssessmentResult | dict[str, Any]: + """Assess a request and return a normalized result or mapping.""" + + +class AssessmentCache(Protocol): + """Minimal pluggable assessment cache.""" + + def get(self, cache_key: str) -> AssessmentResult | None: + """Return a cached assessment result if available.""" + + def set(self, cache_key: str, result: AssessmentResult) -> None: + """Store an assessment result.""" + + +class MemoryAssessmentCache: + """Transparent in-memory cache useful for tests and short workflow runs.""" + + def __init__(self) -> None: + self._results: dict[str, AssessmentResult] = {} + + def get(self, cache_key: str) -> AssessmentResult | None: + return self._results.get(cache_key) + + def set(self, cache_key: str, result: AssessmentResult) -> None: + self._results[cache_key] = result + + +@dataclass(frozen=True) +class AssessmentRunResult: + """Result of executing one or more rubric assessments.""" + + assessments: list[AssessmentResult] = field(default_factory=list) + diagnostics: list[Diagnostic] = field(default_factory=list) + + @property + def valid(self) -> bool: + return not has_error(self.diagnostics) + + def to_dict(self) -> dict[str, Any]: + data = { + "valid": self.valid, + "assessments": [assessment.to_dict() for assessment in self.assessments], + "diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics], + } + return _drop_empty(data) + + +class AssessmentRunner: + """Invoke an injected assessment adapter and normalize diagnostics.""" + + def __init__( + self, + adapter: AssessmentAdapter, + *, + cache: AssessmentCache | None = None, + ) -> None: + self.adapter = adapter + self.cache = cache + + def assess(self, request: AssessmentRequest) -> AssessmentResult: + if self.cache: + cached = self.cache.get(request.cache_key) + if cached: + return replace(cached, cached=True) + + raw_result = self.adapter.assess(request) + result = _normalize_assessment_result(raw_result, request) + if self.cache: + self.cache.set(request.cache_key, result) + return result + + def assess_all(self, requests: list[AssessmentRequest]) -> AssessmentRunResult: + assessments = [self.assess(request) for request in requests] + diagnostics: list[Diagnostic] = [] + for request, assessment in zip(requests, assessments, strict=True): + diagnostics.extend(_diagnostics_for_result(request, assessment)) + return AssessmentRunResult(assessments=assessments, diagnostics=diagnostics) + + +def assessment_requests_for_contract( + document: Document, + contract: DocumentContract, + runtime_context: RuntimeContext | None = None, +) -> list[AssessmentRequest]: + """Create assessment requests for contract rubric declarations.""" + + context = runtime_context or RuntimeContext.empty() + requests: list[AssessmentRequest] = [] + for index, rubric in enumerate(contract.rubrics): + if not isinstance(rubric, dict): + continue + rule_id = str(rubric.get("id") or f"rubric-{index + 1}") + scope = str(rubric.get("scope") or "document") + text = _text_for_scope(document, contract, scope) + requests.append( + AssessmentRequest( + contract_id=contract.id, + rule_id=rule_id, + scope=scope, + text=text, + criteria=rubric.get("criteria") or rubric.get("prompt") or rubric, + context=context.binding(), + structured_inputs={ + "frontmatter": document.frontmatter, + "contract": contract.to_dict(), + }, + severity=str(rubric.get("severity", "error")), + threshold=rubric.get("threshold"), + provider=rubric.get("provider"), + model=rubric.get("model"), + metadata={"rubric": rubric}, + ) + ) + return requests + + +def run_contract_assessments( + document: Document, + contract: DocumentContract, + adapter: AssessmentAdapter, + *, + runtime_context: RuntimeContext | None = None, + cache: AssessmentCache | None = None, +) -> AssessmentRunResult: + """Run all rubrics in a contract with an injected assessment adapter.""" + + runner = AssessmentRunner(adapter, cache=cache) + return runner.assess_all( + assessment_requests_for_contract(document, contract, runtime_context) + ) + + +def _normalize_assessment_result( + raw_result: AssessmentResult | dict[str, Any], + request: AssessmentRequest, +) -> AssessmentResult: + if isinstance(raw_result, AssessmentResult): + result = raw_result + else: + result = AssessmentResult( + rule_id=str(raw_result.get("rule_id") or request.rule_id), + passed=bool(raw_result.get("passed")), + score=raw_result.get("score"), + reason=raw_result.get("reason"), + diagnostics=list(raw_result.get("diagnostics", [])), + provider=raw_result.get("provider"), + model=raw_result.get("model"), + metadata=raw_result.get("metadata", {}), + ) + return replace( + result, + rule_id=result.rule_id or request.rule_id, + cache_key=request.cache_key, + provider=result.provider or request.provider, + model=result.model or request.model, + ) + + +def _diagnostics_for_result( + request: AssessmentRequest, assessment: AssessmentResult +) -> list[Diagnostic]: + diagnostics = list(assessment.diagnostics) + if not assessment.passed: + diagnostics.append( + Diagnostic( + severity=request.severity, + code="runtime.assessment.failed", + message=assessment.reason or f"Assessment `{request.rule_id}` failed.", + rule_id=request.rule_id, + details={ + "scope": request.scope, + "score": assessment.score, + "threshold": request.threshold, + "provider": assessment.provider, + "model": assessment.model, + "cache_key": assessment.cache_key, + }, + ) + ) + return diagnostics + + +def _text_for_scope(document: Document, contract: DocumentContract, scope: str) -> str: + if scope == "document": + return document.body + if scope.startswith("section."): + section_id = scope.split(".", 1)[1] + for section_spec in contract.sections: + if section_spec.id != section_id: + continue + section = _matching_section(document, section_spec.headings) + if section: + return "\n".join(block.text for block in section.blocks if block.text) + if scope.startswith("field."): + field_id = scope.split(".", 1)[1] + value = document.frontmatter.get(field_id) + return "" if value is None else str(value) + return document.body + + +def _matching_section(document: Document, headings: list[str]) -> Section | None: + expected = {heading.strip().lower() for heading in headings} + for section in document.sections: + if section.heading.text.strip().lower() in expected: + return section + return None + + +def _drop_empty(data: dict[str, Any]) -> dict[str, Any]: + return { + key: value + for key, value in data.items() + if value not in (None, [], {}, "") + } diff --git a/src/markitect_tool/runtime/context.py b/src/markitect_tool/runtime/context.py new file mode 100644 index 0000000..220f9ce --- /dev/null +++ b/src/markitect_tool/runtime/context.py @@ -0,0 +1,297 @@ +"""Runtime context loading and validation for contract execution.""" + +from __future__ import annotations + +import json +from dataclasses import asdict, dataclass, field +from pathlib import Path +from typing import Any + +import yaml +from jsonschema import Draft202012Validator, SchemaError, ValidationError + +from markitect_tool.diagnostics import Diagnostic, SourceLocation, has_error + + +CONTEXT_RESERVED_KEYS = {"context", "metadata", "schema", "schemas", "sources"} + + +@dataclass(frozen=True) +class RuntimeContextSource: + """Origin metadata for one loaded context object.""" + + name: str | None = None + path: str | None = None + kind: str = "file" + metadata: dict[str, Any] = field(default_factory=dict) + + def to_dict(self) -> dict[str, Any]: + return _drop_empty(asdict(self)) + + +@dataclass(frozen=True) +class RuntimeContext: + """Named external data available to contract checks and generation.""" + + data: dict[str, Any] = field(default_factory=dict) + metadata: dict[str, Any] = field(default_factory=dict) + source_path: str | None = None + sources: list[RuntimeContextSource] = field(default_factory=list) + schemas: dict[str, Any] = field(default_factory=dict) + diagnostics: list[Diagnostic] = field(default_factory=list) + + @property + def valid(self) -> bool: + return not has_error(self.diagnostics) + + def binding(self) -> dict[str, Any]: + """Return the object bound at `context` in runtime expressions.""" + + return self.data + + def to_dict(self) -> dict[str, Any]: + data = { + "valid": self.valid, + "source_path": self.source_path, + "data": self.data, + "metadata": self.metadata, + "sources": [source.to_dict() for source in self.sources], + "schemas": self.schemas, + "diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics], + } + return _drop_empty(data) + + @classmethod + def empty(cls) -> "RuntimeContext": + return cls() + + @classmethod + def from_mapping( + cls, + raw: dict[str, Any], + *, + source_path: str | None = None, + ) -> "RuntimeContext": + if "context" in raw: + context_data = raw.get("context") or {} + if not isinstance(context_data, dict): + return cls( + source_path=source_path, + diagnostics=[ + _diagnostic( + "runtime.context.invalid", + "`context` must be a mapping.", + source_path=source_path, + ) + ], + ) + else: + context_data = { + key: value for key, value in raw.items() if key not in CONTEXT_RESERVED_KEYS + } + + metadata = raw.get("metadata") if isinstance(raw.get("metadata"), dict) else {} + schemas: dict[str, Any] = {} + if isinstance(raw.get("schema"), dict): + schemas["$context"] = raw["schema"] + if isinstance(raw.get("schemas"), dict): + schemas.update(raw["schemas"]) + + sources = _sources_from_raw(raw.get("sources"), source_path) + diagnostics = _validate_context_schemas(context_data, schemas, source_path) + return cls( + data=context_data, + metadata=metadata, + source_path=source_path, + sources=sources, + schemas=schemas, + diagnostics=diagnostics, + ) + + +@dataclass(frozen=True) +class RuntimeContextLoadResult: + """Context load result that can carry diagnostics instead of raising.""" + + context: RuntimeContext + + @property + def valid(self) -> bool: + return self.context.valid + + @property + def diagnostics(self) -> list[Diagnostic]: + return self.context.diagnostics + + def to_dict(self) -> dict[str, Any]: + return self.context.to_dict() + + +def load_runtime_context_file(path: str | Path) -> RuntimeContext: + """Load a runtime context from JSON or YAML.""" + + return load_runtime_context_file_result(path).context + + +def load_runtime_context_file_result(path: str | Path) -> RuntimeContextLoadResult: + """Load a runtime context and represent malformed input as diagnostics.""" + + context_path = Path(path) + try: + text = context_path.read_text(encoding="utf-8") + except OSError as exc: + return RuntimeContextLoadResult( + RuntimeContext( + source_path=str(context_path), + diagnostics=[ + _diagnostic( + "runtime.context.read_failed", + f"Cannot read runtime context: {exc}", + source_path=str(context_path), + ) + ], + ) + ) + + try: + raw = _load_mapping(text, context_path) + except ValueError as exc: + return RuntimeContextLoadResult( + RuntimeContext( + source_path=str(context_path), + diagnostics=[ + _diagnostic( + "runtime.context.malformed", + str(exc), + source_path=str(context_path), + ) + ], + ) + ) + return RuntimeContextLoadResult( + RuntimeContext.from_mapping(raw, source_path=str(context_path)) + ) + + +def _load_mapping(text: str, path: Path) -> dict[str, Any]: + suffix = path.suffix.lower() + try: + if suffix == ".json": + data = json.loads(text) if text.strip() else {} + else: + data = yaml.safe_load(text) if text.strip() else {} + except (json.JSONDecodeError, yaml.YAMLError) as exc: + raise ValueError(f"Invalid context file `{path}`: {exc}") from exc + if data is None: + data = {} + if not isinstance(data, dict): + raise ValueError("Runtime context file must contain a mapping.") + return data + + +def _sources_from_raw( + raw_sources: Any, source_path: str | None +) -> list[RuntimeContextSource]: + if not raw_sources: + return [RuntimeContextSource(path=source_path)] if source_path else [] + if isinstance(raw_sources, dict): + return [ + RuntimeContextSource( + name=str(name), + path=str(raw.get("path")) if isinstance(raw, dict) and raw.get("path") else None, + kind=str(raw.get("kind", "file")) if isinstance(raw, dict) else "file", + metadata=raw.get("metadata", {}) if isinstance(raw, dict) else {}, + ) + for name, raw in raw_sources.items() + ] + if isinstance(raw_sources, list): + sources: list[RuntimeContextSource] = [] + for item in raw_sources: + if isinstance(item, dict): + sources.append( + RuntimeContextSource( + name=item.get("name"), + path=item.get("path"), + kind=str(item.get("kind", "file")), + metadata=item.get("metadata", {}), + ) + ) + return sources + return [RuntimeContextSource(path=source_path)] if source_path else [] + + +def _validate_context_schemas( + data: dict[str, Any], schemas: dict[str, Any], source_path: str | None +) -> list[Diagnostic]: + diagnostics: list[Diagnostic] = [] + for name, schema in schemas.items(): + if not isinstance(schema, dict): + diagnostics.append( + _diagnostic( + "runtime.context.schema_invalid", + f"Context schema `{name}` must be a mapping.", + source_path=source_path, + rule_id=str(name), + ) + ) + continue + target = data if name == "$context" else data.get(name) + if name != "$context" and name not in data: + diagnostics.append( + _diagnostic( + "runtime.context.schema_target_missing", + f"Context schema `{name}` has no matching context object.", + source_path=source_path, + rule_id=str(name), + ) + ) + continue + try: + Draft202012Validator.check_schema(schema) + Draft202012Validator(schema).validate(target) + except SchemaError as exc: + diagnostics.append( + _diagnostic( + "runtime.context.schema_invalid", + f"Invalid context schema `{name}`: {exc.message}", + source_path=source_path, + rule_id=str(name), + ) + ) + except ValidationError as exc: + diagnostics.append( + _diagnostic( + "runtime.context.schema", + f"Context object `{name}` does not match its schema: {exc.message}", + source_path=source_path, + rule_id=str(name), + details={"path": list(exc.absolute_path)}, + ) + ) + return diagnostics + + +def _diagnostic( + code: str, + message: str, + *, + source_path: str | None = None, + rule_id: str | None = None, + details: dict[str, Any] | None = None, +) -> Diagnostic: + return Diagnostic( + severity="error", + code=code, + message=message, + source=SourceLocation(path=source_path) if source_path else None, + rule_id=rule_id, + details=details or {}, + ) + + +def _drop_empty(data: dict[str, Any]) -> dict[str, Any]: + return { + key: value + for key, value in data.items() + if value not in (None, [], {}, "") + } diff --git a/src/markitect_tool/runtime/forms.py b/src/markitect_tool/runtime/forms.py new file mode 100644 index 0000000..691e5f0 --- /dev/null +++ b/src/markitect_tool/runtime/forms.py @@ -0,0 +1,839 @@ +"""Form state and deterministic runtime rule evaluation.""" + +from __future__ import annotations + +import re +from dataclasses import asdict, dataclass, field, replace +from typing import Any + +from markitect_tool.contract.model import DocumentContract, FieldSpec, SectionSpec +from markitect_tool.core import Document +from markitect_tool.diagnostics import ( + Diagnostic, + SourceLocation, + has_error, + valid_severity, +) +from markitect_tool.runtime.context import RuntimeContext +from markitect_tool.runtime.paths import comparable_value, resolve_path +from markitect_tool.runtime.rules import evaluate_condition + + +@dataclass(frozen=True) +class FieldState: + """UI-neutral runtime state for one contract field.""" + + id: str + path: str | None = None + source: str | None = None + type: str | None = None + label: str | None = None + description: str | None = None + value: Any = None + exists: bool = False + origin: str = "missing" + required: bool = False + visible: bool = True + enabled: bool = True + allowed_values: list[Any] | None = None + diagnostics: list[Diagnostic] = field(default_factory=list) + metadata: dict[str, Any] = field(default_factory=dict) + + @property + def valid(self) -> bool: + return not has_error(self.diagnostics) + + def to_dict(self) -> dict[str, Any]: + data = asdict(self) + data["diagnostics"] = [diagnostic.to_dict() for diagnostic in self.diagnostics] + return _drop_empty(data) + + +@dataclass(frozen=True) +class FormState: + """A complete runtime form state derived from a document contract.""" + + contract_id: str | None + document_path: str | None = None + context_path: str | None = None + fields: list[FieldState] = field(default_factory=list) + diagnostics: list[Diagnostic] = field(default_factory=list) + rules_applied: list[str] = field(default_factory=list) + metadata: dict[str, Any] = field(default_factory=dict) + + @property + def valid(self) -> bool: + return not has_error(self.diagnostics) + + @property + def field_values(self) -> dict[str, Any]: + return {field.id: field.value for field in self.fields if field.exists} + + def to_dict(self) -> dict[str, Any]: + data = { + "valid": self.valid, + "contract_id": self.contract_id, + "document_path": self.document_path, + "context_path": self.context_path, + "field_values": self.field_values, + "fields": [field.to_dict() for field in self.fields], + "diagnostics": [diagnostic.to_dict() for diagnostic in self.diagnostics], + "rules_applied": self.rules_applied, + "metadata": self.metadata, + } + return _drop_empty(data) + + +def evaluate_form_state( + document: Document, + contract: DocumentContract, + runtime_context: RuntimeContext | None = None, +) -> FormState: + """Evaluate contract fields, prefill sources, and dynamic runtime rules.""" + + context = runtime_context or RuntimeContext.empty() + bindings = build_runtime_bindings(document, contract, context) + fields = [ + _resolve_field_state(field_spec, document, contract, bindings) + for field_spec in contract.fields + ] + diagnostics: list[Diagnostic] = list(context.diagnostics) + rules_applied: list[str] = [] + + fields_by_id = {field.id: field for field in fields} + bindings = build_runtime_bindings(document, contract, context, fields_by_id) + fields_by_id, rule_diagnostics, rules_applied = _apply_dynamic_rules( + fields_by_id, + document, + contract, + context, + bindings, + ) + diagnostics.extend(rule_diagnostics) + + validated_fields: list[FieldState] = [] + for field_spec in contract.fields: + field_state = fields_by_id[_field_key(field_spec)] + field_diagnostics = [ + *field_state.diagnostics, + *_validate_field_state(field_spec, field_state, document, contract), + ] + validated_fields.append(replace(field_state, diagnostics=field_diagnostics)) + diagnostics.extend(field_diagnostics) + + return FormState( + contract_id=contract.id, + document_path=document.source_path, + context_path=context.source_path, + fields=validated_fields, + diagnostics=diagnostics, + rules_applied=rules_applied, + ) + + +def build_runtime_bindings( + document: Document, + contract: DocumentContract, + context: RuntimeContext, + fields_by_id: dict[str, FieldState] | None = None, +) -> dict[str, Any]: + """Build deterministic bindings used by field sources and rule conditions.""" + + field_states = fields_by_id or {} + return { + "document": document.to_dict(), + "frontmatter": document.frontmatter, + "context": context.binding(), + "contract": contract.to_dict(), + "fields": { + field_id: field.to_dict() | {"value": field.value} + for field_id, field in field_states.items() + }, + "field_values": { + field_id: field.value + for field_id, field in field_states.items() + if field.exists + }, + } + + +def _resolve_field_state( + field_spec: FieldSpec, + document: Document, + contract: DocumentContract, + bindings: dict[str, Any], +) -> FieldState: + field_id = field_spec.id or "" + path = field_spec.path or (f"frontmatter.{field_id}" if field_spec.id else None) + source_candidates = _source_candidates(field_spec) + diagnostics: list[Diagnostic] = [] + + document_value, document_exists = resolve_path(bindings, path) + source_values = [ + (source, value) + for source in source_candidates + for value, exists in [resolve_path(bindings, source)] + if exists + ] + + if document_exists: + origin = "manual" + value = document_value + exists = True + for source, source_value in source_values: + if comparable_value(source_value) != comparable_value(document_value): + diagnostics.append( + _diagnostic( + "runtime.field.conflict", + ( + f"Field `{field_id}` is provided by the document and " + f"context source `{source}` with different values." + ), + document=document, + contract=contract, + rule_id=field_id, + severity=_conflict_severity(field_spec), + details={ + "path": path, + "source": source, + "document_value": comparable_value(document_value), + "context_value": comparable_value(source_value), + }, + ) + ) + elif source_values: + origin = "prefilled" + value = source_values[0][1] + exists = True + distinct = { + repr(comparable_value(item_value)) + for _source, item_value in source_values + } + if len(distinct) > 1: + diagnostics.append( + _diagnostic( + "runtime.field.ambiguous", + f"Field `{field_id}` has multiple source values.", + document=document, + contract=contract, + rule_id=field_id, + details={"sources": [source for source, _value in source_values]}, + ) + ) + elif field_spec.default is not None: + origin = "defaulted" + value = field_spec.default + exists = True + else: + origin = "missing" + value = None + exists = False + + return FieldState( + id=field_id, + path=path, + source=source_candidates[0] if source_candidates else field_spec.source, + type=field_spec.type, + label=field_spec.label, + description=field_spec.description, + value=value, + exists=exists, + origin=origin, + required=field_spec.required, + allowed_values=field_spec.enum, + diagnostics=diagnostics, + metadata={ + "sources": source_candidates, + "coerce": bool(field_spec.raw.get("coerce", False)) + if isinstance(field_spec.raw, dict) + else False, + }, + ) + + +def _source_candidates(field_spec: FieldSpec) -> list[str]: + raw = field_spec.raw if isinstance(field_spec.raw, dict) else {} + candidates: list[str] = [] + source = raw.get("source", field_spec.source) + if isinstance(source, list): + candidates.extend(str(item) for item in source if item) + elif source: + candidates.append(str(source)) + sources = raw.get("sources") + if isinstance(sources, list): + candidates.extend(str(item) for item in sources if item) + return list(dict.fromkeys(candidates)) + + +def _field_key(field_spec: FieldSpec) -> str: + return field_spec.id or "" + + +def _apply_dynamic_rules( + fields_by_id: dict[str, FieldState], + document: Document, + contract: DocumentContract, + context: RuntimeContext, + bindings: dict[str, Any], +) -> tuple[dict[str, FieldState], list[Diagnostic], list[str]]: + diagnostics: list[Diagnostic] = [] + applied: list[str] = [] + rules = contract.rules + for index, rule in enumerate(rules): + if not isinstance(rule, dict): + diagnostics.append( + _diagnostic( + "runtime.rule.invalid", + "Dynamic rule must be a mapping.", + document=document, + contract=contract, + ) + ) + continue + rule_id = str(rule.get("id") or f"rule-{index + 1}") + result = evaluate_condition(rule.get("if"), bindings, rule_id=rule_id) + diagnostics.extend( + _with_contract_location(item, document=document, contract=contract) + for item in result.diagnostics + ) + action = rule.get("then") if result.matched else rule.get("else") + if result.matched: + applied.append(rule_id) + if action is not None: + fields_by_id, action_diagnostics = _apply_rule_action( + fields_by_id, + action, + document, + contract, + context, + rule_id, + ) + diagnostics.extend(action_diagnostics) + bindings = build_runtime_bindings(document, contract, context, fields_by_id) + if "assert" in rule and result.matched: + diagnostics.extend( + _evaluate_runtime_assertions( + rule["assert"], + bindings, + document, + contract, + rule_id, + rule, + ) + ) + return fields_by_id, diagnostics, applied + + +def _apply_rule_action( + fields_by_id: dict[str, FieldState], + action: Any, + document: Document, + contract: DocumentContract, + context: RuntimeContext, + rule_id: str, +) -> tuple[dict[str, FieldState], list[Diagnostic]]: + if not isinstance(action, dict): + return fields_by_id, [ + _diagnostic( + "runtime.rule.action_invalid", + f"Dynamic rule `{rule_id}` action must be a mapping.", + document=document, + contract=contract, + rule_id=rule_id, + ) + ] + + diagnostics: list[Diagnostic] = [] + updated = dict(fields_by_id) + bindings = build_runtime_bindings(document, contract, context, updated) + + for field_id in _field_id_list(action.get("required")): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"required": True}, document, contract, rule_id + ) + for field_id in _field_id_list(action.get("optional")): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"required": False}, document, contract, rule_id + ) + for field_id, visible in _field_bool_mapping(action.get("visible")).items(): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"visible": visible}, document, contract, rule_id + ) + for field_id in _field_id_list(action.get("hidden")): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"visible": False}, document, contract, rule_id + ) + for field_id, enabled in _field_bool_mapping(action.get("enabled")).items(): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"enabled": enabled}, document, contract, rule_id + ) + for field_id in _field_id_list(action.get("disabled")): + updated, diagnostics = _set_field_attr( + updated, diagnostics, field_id, {"enabled": False}, document, contract, rule_id + ) + if isinstance(action.get("allowed_values"), dict): + for field_id, allowed in action["allowed_values"].items(): + updated, diagnostics = _set_field_attr( + updated, + diagnostics, + str(field_id), + {"allowed_values": allowed if isinstance(allowed, list) else [allowed]}, + document, + contract, + rule_id, + ) + if isinstance(action.get("set"), dict): + for field_id, raw_value in action["set"].items(): + value = _resolve_template_value(raw_value, bindings) + updated, diagnostics = _set_field_attr( + updated, + diagnostics, + str(field_id), + {"value": value, "exists": True, "origin": "calculated"}, + document, + contract, + rule_id, + ) + if "assert" in action: + diagnostics.extend( + _evaluate_runtime_assertions( + action["assert"], + build_runtime_bindings(document, contract, context, updated), + document, + contract, + rule_id, + action, + ) + ) + if "sections" in action: + diagnostics.extend(_check_dynamic_sections(action["sections"], document, contract, rule_id)) + return updated, diagnostics + + +def _set_field_attr( + fields_by_id: dict[str, FieldState], + diagnostics: list[Diagnostic], + field_id: str, + updates: dict[str, Any], + document: Document, + contract: DocumentContract, + rule_id: str, +) -> tuple[dict[str, FieldState], list[Diagnostic]]: + if field_id not in fields_by_id: + diagnostics.append( + _diagnostic( + "runtime.rule.unknown_field", + f"Dynamic rule `{rule_id}` references unknown field `{field_id}`.", + document=document, + contract=contract, + rule_id=rule_id, + ) + ) + return fields_by_id, diagnostics + fields_by_id[field_id] = replace(fields_by_id[field_id], **updates) + return fields_by_id, diagnostics + + +def _evaluate_runtime_assertions( + raw_assertions: Any, + bindings: dict[str, Any], + document: Document, + contract: DocumentContract, + rule_id: str, + rule: dict[str, Any], +) -> list[Diagnostic]: + assertions = raw_assertions if isinstance(raw_assertions, list) else [raw_assertions] + diagnostics: list[Diagnostic] = [] + for assertion in assertions: + result = evaluate_condition(assertion, bindings, rule_id=rule_id) + diagnostics.extend( + _with_contract_location(item, document=document, contract=contract) + for item in result.diagnostics + ) + if not result.matched: + severity = _severity_from_mapping(assertion, rule) + message = ( + assertion.get("message") + if isinstance(assertion, dict) and assertion.get("message") + else rule.get("message") + or f"Runtime assertion `{rule_id}` was not satisfied." + ) + diagnostics.append( + _diagnostic( + "runtime.rule.assertion_failed", + str(message), + document=document, + contract=contract, + rule_id=rule_id, + severity=severity, + details={"assertion": assertion if isinstance(assertion, dict) else None}, + ) + ) + return diagnostics + + +def _check_dynamic_sections( + raw_sections: Any, + document: Document, + contract: DocumentContract, + rule_id: str, +) -> list[Diagnostic]: + if not isinstance(raw_sections, dict): + return [ + _diagnostic( + "runtime.rule.sections_invalid", + "`sections` action must be a mapping.", + document=document, + contract=contract, + rule_id=rule_id, + ) + ] + section_specs = {section.id: section for section in contract.sections if section.id} + diagnostics: list[Diagnostic] = [] + for section_id, raw in raw_sections.items(): + section_spec = section_specs.get(str(section_id)) + if section_spec is None: + diagnostics.append( + _diagnostic( + "runtime.rule.unknown_section", + f"Dynamic rule `{rule_id}` references unknown section `{section_id}`.", + document=document, + contract=contract, + rule_id=rule_id, + ) + ) + continue + presence = raw.get("presence") if isinstance(raw, dict) else raw + diagnostics.extend(_check_dynamic_section_presence(document, contract, section_spec, str(presence), rule_id)) + return diagnostics + + +def _check_dynamic_section_presence( + document: Document, + contract: DocumentContract, + section_spec: SectionSpec, + presence: str, + rule_id: str, +) -> list[Diagnostic]: + matches = _matching_section_lines(document, section_spec) + if presence == "required" and not matches: + return [ + _diagnostic( + "runtime.section.missing", + f"Dynamic rule requires section `{section_spec.id}`.", + document=document, + contract=contract, + rule_id=rule_id, + guidance=f"Add a {'#' * (section_spec.level or 2)} {section_spec.title or section_spec.id} section.", + ) + ] + if presence == "recommended" and not matches: + return [ + _diagnostic( + "runtime.section.recommended_missing", + f"Dynamic rule recommends section `{section_spec.id}`.", + document=document, + contract=contract, + rule_id=rule_id, + severity="warning", + ) + ] + if presence == "forbidden" and matches: + return [ + _diagnostic( + "runtime.section.forbidden", + f"Dynamic rule forbids section `{section_spec.id}`.", + document=document, + contract=contract, + rule_id=rule_id, + source_line=matches[0], + ) + ] + if presence == "discouraged" and matches: + return [ + _diagnostic( + "runtime.section.discouraged", + f"Dynamic rule discourages section `{section_spec.id}`.", + document=document, + contract=contract, + rule_id=rule_id, + source_line=matches[0], + severity="warning", + ) + ] + return [] + + +def _validate_field_state( + field_spec: FieldSpec, + field_state: FieldState, + document: Document, + contract: DocumentContract, +) -> list[Diagnostic]: + diagnostics: list[Diagnostic] = [] + if field_state.required and field_state.visible and not field_state.exists: + diagnostics.append( + _diagnostic( + "contract.field.missing", + f"Required field `{field_state.id}` is missing.", + document=document, + contract=contract, + rule_id=field_state.id, + guidance=f"Provide `{field_state.path}` in the document or context.", + ) + ) + return diagnostics + if not field_state.exists: + return diagnostics + + value = field_state.value + if field_state.metadata.get("coerce"): + value, coerced, coercion_error = _coerce_value(value, field_state.type) + if coercion_error: + diagnostics.append( + _diagnostic( + "runtime.field.coercion_failed", + f"Field `{field_state.id}` could not be coerced to `{field_state.type}`.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + elif coerced: + object.__setattr__(field_state, "value", value) + + if field_state.type and not _value_matches_type(value, field_state.type): + diagnostics.append( + _diagnostic( + "contract.field.type_mismatch", + ( + f"Field `{field_state.id}` must be `{field_state.type}`, " + f"got `{type(value).__name__}`." + ), + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + allowed_values = field_state.allowed_values + if allowed_values is not None and value not in allowed_values: + diagnostics.append( + _diagnostic( + "contract.field.enum", + f"Field `{field_state.id}` must be one of {allowed_values}.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + if field_spec.pattern and isinstance(value, str) and not re.search(field_spec.pattern, value): + diagnostics.append( + _diagnostic( + "contract.field.pattern", + f"Field `{field_state.id}` does not match its required pattern.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + if field_spec.min_length is not None and hasattr(value, "__len__") and len(value) < field_spec.min_length: + diagnostics.append( + _diagnostic( + "contract.field.min_length", + f"Field `{field_state.id}` is shorter than {field_spec.min_length}.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + if field_spec.max_length is not None and hasattr(value, "__len__") and len(value) > field_spec.max_length: + diagnostics.append( + _diagnostic( + "contract.field.max_length", + f"Field `{field_state.id}` is longer than {field_spec.max_length}.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + if field_spec.min is not None and isinstance(value, int | float) and value < field_spec.min: + diagnostics.append( + _diagnostic( + "contract.field.min", + f"Field `{field_state.id}` is below {field_spec.min}.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + if field_spec.max is not None and isinstance(value, int | float) and value > field_spec.max: + diagnostics.append( + _diagnostic( + "contract.field.max", + f"Field `{field_state.id}` is above {field_spec.max}.", + document=document, + contract=contract, + rule_id=field_state.id, + ) + ) + return diagnostics + + +def _coerce_value(value: Any, expected_type: str | None) -> tuple[Any, bool, bool]: + if expected_type == "string" and not isinstance(value, str): + return str(value), True, False + if expected_type == "integer" and isinstance(value, str): + try: + return int(value), True, False + except ValueError: + return value, False, True + if expected_type == "number" and isinstance(value, str): + try: + return float(value), True, False + except ValueError: + return value, False, True + if expected_type == "boolean" and isinstance(value, str): + normalized = value.strip().lower() + if normalized in {"true", "yes", "1"}: + return True, True, False + if normalized in {"false", "no", "0"}: + return False, True, False + return value, False, True + return value, False, False + + +def _value_matches_type(value: Any, expected_type: str) -> bool: + if expected_type == "string": + return isinstance(value, str) + if expected_type == "number": + return isinstance(value, int | float) and not isinstance(value, bool) + if expected_type == "integer": + return isinstance(value, int) and not isinstance(value, bool) + if expected_type == "boolean": + return isinstance(value, bool) + if expected_type == "array": + return isinstance(value, list) + if expected_type == "object": + return isinstance(value, dict) + if expected_type == "date": + return isinstance(value, str) + return True + + +def _field_id_list(value: Any) -> list[str]: + if value is None: + return [] + if isinstance(value, str): + return [value] + if isinstance(value, list): + return [str(item) for item in value if item] + if isinstance(value, dict): + return [str(key) for key, enabled in value.items() if enabled] + return [] + + +def _field_bool_mapping(value: Any) -> dict[str, bool]: + if value is None: + return {} + if isinstance(value, dict): + return {str(key): bool(item) for key, item in value.items()} + if isinstance(value, str): + return {value: True} + if isinstance(value, list): + return {str(item): True for item in value if item} + return {} + + +def _resolve_template_value(value: Any, bindings: dict[str, Any]) -> Any: + if isinstance(value, str): + full = re.fullmatch(r"\$\{([^}]+)\}", value.strip()) + if full: + resolved, exists = resolve_path(bindings, full.group(1)) + return resolved if exists else value + + def replace_match(match: re.Match[str]) -> str: + resolved, exists = resolve_path(bindings, match.group(1)) + return str(resolved) if exists else match.group(0) + + return re.sub(r"\$\{([^}]+)\}", replace_match, value) + if isinstance(value, list): + return [_resolve_template_value(item, bindings) for item in value] + if isinstance(value, dict): + return {key: _resolve_template_value(item, bindings) for key, item in value.items()} + return value + + +def _conflict_severity(field_spec: FieldSpec) -> str: + raw = field_spec.raw if isinstance(field_spec.raw, dict) else {} + severity = raw.get("conflict") or raw.get("conflict_severity") or "warning" + return str(severity) if valid_severity(str(severity)) else "warning" + + +def _severity_from_mapping(*items: Any) -> str: + for item in items: + if isinstance(item, dict) and valid_severity(str(item.get("severity"))): + return str(item["severity"]) + return "error" + + +def _matching_section_lines(document: Document, section_spec: SectionSpec) -> list[int]: + expected = {_normalize_heading(value) for value in section_spec.headings} + return [ + section.heading.line + for section in document.sections + if _normalize_heading(section.heading.text) in expected + ] + + +def _normalize_heading(text: str) -> str: + return re.sub(r"\s+", " ", text.strip().lower()) + + +def _with_contract_location( + diagnostic: Diagnostic, + *, + document: Document, + contract: DocumentContract, +) -> Diagnostic: + return Diagnostic( + severity=diagnostic.severity, + code=diagnostic.code, + message=diagnostic.message, + source=diagnostic.source or SourceLocation(path=document.source_path), + contract=diagnostic.contract or SourceLocation(path=contract.source_path, line=contract.source_line), + rule_id=diagnostic.rule_id, + guidance=diagnostic.guidance, + details=diagnostic.details, + ) + + +def _diagnostic( + code: str, + message: str, + *, + document: Document, + contract: DocumentContract, + rule_id: str | None = None, + severity: str = "error", + guidance: str | None = None, + details: dict[str, Any] | None = None, + source_line: int | None = 1, +) -> Diagnostic: + return Diagnostic( + severity=severity, + code=code, + message=message, + source=SourceLocation(path=document.source_path, line=source_line), + contract=SourceLocation(path=contract.source_path, line=contract.source_line), + rule_id=rule_id, + guidance=guidance, + details=details or {}, + ) + + +def _drop_empty(data: dict[str, Any]) -> dict[str, Any]: + return { + key: value + for key, value in data.items() + if value not in (None, [], {}, "") + } diff --git a/src/markitect_tool/runtime/paths.py b/src/markitect_tool/runtime/paths.py new file mode 100644 index 0000000..86d6047 --- /dev/null +++ b/src/markitect_tool/runtime/paths.py @@ -0,0 +1,45 @@ +"""Small path helpers for runtime context and rule evaluation.""" + +from __future__ import annotations + +from typing import Any + + +def resolve_path(data: Any, path: str | None) -> tuple[Any, bool]: + """Resolve a dotted path against dictionaries and lists. + + The runtime path vocabulary is intentionally small: `context.user.name`, + `frontmatter.status`, `fields.owner.value`, and numeric list indexes are + enough for contract checks, form state, and deterministic rules without + embedding a general expression language. + """ + + if not path: + return None, False + current = data + for part in _path_parts(path): + if isinstance(current, dict) and part in current: + current = current[part] + continue + if isinstance(current, list) and part.isdigit(): + index = int(part) + if index < len(current): + current = current[index] + continue + return None, False + return current, True + + +def comparable_value(value: Any) -> Any: + """Return a stable scalar-ish value for diagnostics and equality checks.""" + + if isinstance(value, dict): + return {key: comparable_value(value[key]) for key in sorted(value)} + if isinstance(value, list): + return [comparable_value(item) for item in value] + return value + + +def _path_parts(path: str) -> list[str]: + normalized = str(path).strip().removeprefix("$.") + return [part for part in normalized.split(".") if part] diff --git a/src/markitect_tool/runtime/rules.py b/src/markitect_tool/runtime/rules.py new file mode 100644 index 0000000..9840be2 --- /dev/null +++ b/src/markitect_tool/runtime/rules.py @@ -0,0 +1,162 @@ +"""Deterministic rule conditions for runtime forms and checks.""" + +from __future__ import annotations + +import re +from dataclasses import dataclass, field +from typing import Any + +from markitect_tool.diagnostics import Diagnostic +from markitect_tool.runtime.paths import resolve_path + + +@dataclass(frozen=True) +class ConditionResult: + """Result of evaluating one deterministic condition.""" + + matched: bool + diagnostics: list[Diagnostic] = field(default_factory=list) + + +def evaluate_condition( + condition: Any, + bindings: dict[str, Any], + *, + rule_id: str | None = None, +) -> ConditionResult: + """Evaluate a small Markitect-native condition mapping.""" + + if condition is None: + return ConditionResult(True) + if isinstance(condition, bool): + return ConditionResult(condition) + if isinstance(condition, list): + diagnostics: list[Diagnostic] = [] + results = [evaluate_condition(item, bindings, rule_id=rule_id) for item in condition] + for result in results: + diagnostics.extend(result.diagnostics) + return ConditionResult(all(result.matched for result in results), diagnostics) + if not isinstance(condition, dict): + return ConditionResult( + False, + [ + _diagnostic( + "runtime.rule.condition_invalid", + "Rule condition must be a mapping, list, or boolean.", + rule_id=rule_id, + ) + ], + ) + + if "all" in condition: + return _combine_conditions(condition["all"], bindings, rule_id, all) + if "any" in condition: + return _combine_conditions(condition["any"], bindings, rule_id, any) + if "not" in condition: + result = evaluate_condition(condition["not"], bindings, rule_id=rule_id) + return ConditionResult(not result.matched, result.diagnostics) + + path = condition.get("path") + if not path: + return ConditionResult( + False, + [ + _diagnostic( + "runtime.rule.condition_missing_path", + "Rule condition must declare `path`.", + rule_id=rule_id, + ) + ], + ) + + value, exists = resolve_path(bindings, str(path)) + diagnostics: list[Diagnostic] = [] + matched = True + if "exists" in condition: + matched = matched and (exists is bool(condition["exists"])) + elif not exists: + matched = False + + if exists: + matched = matched and _operator_matches(value, condition, diagnostics, rule_id) + return ConditionResult(matched, diagnostics) + + +def _combine_conditions( + raw_conditions: Any, + bindings: dict[str, Any], + rule_id: str | None, + combiner: Any, +) -> ConditionResult: + conditions = raw_conditions if isinstance(raw_conditions, list) else [raw_conditions] + results = [evaluate_condition(item, bindings, rule_id=rule_id) for item in conditions] + diagnostics: list[Diagnostic] = [] + for result in results: + diagnostics.extend(result.diagnostics) + return ConditionResult(combiner(result.matched for result in results), diagnostics) + + +def _operator_matches( + value: Any, + condition: dict[str, Any], + diagnostics: list[Diagnostic], + rule_id: str | None, +) -> bool: + matched = True + if "equals" in condition: + matched = matched and value == condition["equals"] + if "eq" in condition: + matched = matched and value == condition["eq"] + if "not_equals" in condition: + matched = matched and value != condition["not_equals"] + if "in" in condition: + expected = condition["in"] + matched = matched and isinstance(expected, list) and value in expected + if "contains" in condition: + expected = condition["contains"] + matched = matched and _contains(value, expected) + if "matches" in condition: + pattern = str(condition["matches"]) + try: + matched = matched and re.search(pattern, str(value)) is not None + except re.error as exc: + diagnostics.append( + _diagnostic( + "runtime.rule.regex_invalid", + f"Invalid rule regular expression `{pattern}`: {exc}", + rule_id=rule_id, + ) + ) + matched = False + for key, predicate in { + "gt": lambda actual, expected: actual > expected, + "gte": lambda actual, expected: actual >= expected, + "lt": lambda actual, expected: actual < expected, + "lte": lambda actual, expected: actual <= expected, + }.items(): + if key not in condition: + continue + try: + matched = matched and predicate(value, condition[key]) + except TypeError: + matched = False + return matched + + +def _contains(value: Any, expected: Any) -> bool: + if isinstance(value, str): + return str(expected) in value + if isinstance(value, list | tuple | set): + return expected in value + if isinstance(value, dict): + return expected in value + return False + + +def _diagnostic(code: str, message: str, *, rule_id: str | None = None) -> Diagnostic: + return Diagnostic( + severity="error", + code=code, + message=message, + rule_id=rule_id, + ) diff --git a/src/markitect_tool/workflow/engine.py b/src/markitect_tool/workflow/engine.py index 2f0a5a7..86b2409 100644 --- a/src/markitect_tool/workflow/engine.py +++ b/src/markitect_tool/workflow/engine.py @@ -25,6 +25,7 @@ from markitect_tool.query import ( extract_document_with_engine, query_document_with_engine, ) +from markitect_tool.runtime import evaluate_form_state, load_runtime_context_file from markitect_tool.template import MissingTemplateVariable, TemplateError, render_template @@ -322,6 +323,8 @@ class WorkflowRunner: return self._step_contract_stub(step) if kind == "contract_check": return self._step_contract_check(step) + if kind == "form_state": + return self._step_form_state(step) if kind == "assisted": return self._step_assisted(step) raise WorkflowError(f"Unsupported workflow step kind `{kind}`") @@ -404,9 +407,28 @@ class WorkflowRunner: def _step_contract_check(self, step: dict[str, Any]) -> dict[str, Any]: document_path = _safe_input_path(self.base_dir, step.get("document")) contract_path = _safe_input_path(self.base_dir, step.get("contract")) - result = check_markdown_file(document_path, contract_path) + context_path = ( + _safe_input_path(self.base_dir, step.get("context")) + if step.get("context") + else None + ) + result = check_markdown_file(document_path, contract_path, context_path=context_path) return result.to_dict() | {"kind": "contract_check"} + def _step_form_state(self, step: dict[str, Any]) -> dict[str, Any]: + document_path = _safe_input_path(self.base_dir, step.get("document")) + contract_path = _safe_input_path(self.base_dir, step.get("contract")) + context_path = ( + _safe_input_path(self.base_dir, step.get("context")) + if step.get("context") + else None + ) + document = parse_markdown_file(document_path) + contract = load_contract_file(contract_path) + context = load_runtime_context_file(context_path) if context_path else None + result = evaluate_form_state(document, contract, context) + return result.to_dict() | {"kind": "form_state"} + def _step_assisted(self, step: dict[str, Any]) -> dict[str, Any]: optional = bool(step.get("optional", True)) if self.assisted_hook is None: diff --git a/tests/test_builtin_extension_catalog.py b/tests/test_builtin_extension_catalog.py index ca7a037..d4dc619 100644 --- a/tests/test_builtin_extension_catalog.py +++ b/tests/test_builtin_extension_catalog.py @@ -14,6 +14,9 @@ def test_builtin_extension_registry_lists_query_processors_and_backend(): assert "processor.include" in ids assert "backend.local-sqlite" in ids assert "workflow.markdown-dataflow" in ids + assert "runtime.context" in ids + assert "runtime.form-state" in ids + assert "runtime.assessment" in ids def test_builtin_processor_descriptors_capture_safety_and_provenance(): @@ -64,3 +67,17 @@ def test_builtin_workflow_descriptor_exposes_cli_and_safety(): "mkt workflow plan", "mkt workflow run", ] + + +def test_builtin_runtime_descriptors_expose_boundaries(): + registry = builtin_extension_registry() + + context = registry.get("runtime.context") + form_state = registry.get("runtime.form-state") + assessment = registry.get("runtime.assessment") + + assert context.safety["reads_files"] is True + assert "mkt contract check --context" in context.cli["commands"] + assert {capability.id for capability in form_state.capabilities} >= {"forms", "rules"} + assert assessment.kind == "assessment-runner" + assert assessment.safety["provider_calls"] == "adapter-only" diff --git a/tests/test_runtime_context_forms_assessment.py b/tests/test_runtime_context_forms_assessment.py new file mode 100644 index 0000000..36221b5 --- /dev/null +++ b/tests/test_runtime_context_forms_assessment.py @@ -0,0 +1,388 @@ +from pathlib import Path + +from click.testing import CliRunner + +from markitect_tool.cli import main +from markitect_tool.contract import check_markdown_file, load_contract_file +from markitect_tool.core import parse_markdown, parse_markdown_file +from markitect_tool.runtime import ( + AssessmentResult, + AssessmentRunner, + MemoryAssessmentCache, + assessment_requests_for_contract, + evaluate_form_state, + load_runtime_context_file, +) +from markitect_tool.workflow import WorkflowRunner, load_workflow_file + + +LETTER_CONTRACT = """# Letter Contract + +```yaml contract +id: letter-runtime-v1 +document: + type: business-letter +fields: + recipient_name: + type: string + required: true + source: context.recipient.name + sender_name: + type: string + required: true + source: context.sender.name + sender_email: + type: string + required: true + source: context.sender.email + pattern: "@example\\\\.com$" + delivery_channel: + type: string + required: true + default: email + enum: [email, print] + postal_address: + type: string + contact_label: + type: string +rules: + - id: postal-address-for-print + if: + path: fields.delivery_channel.value + equals: print + then: + required: [postal_address] + visible: + postal_address: true + else: + hidden: [postal_address] + - id: calculate-contact-label + then: + set: + contact_label: "${fields.sender_name.value} <${fields.sender_email.value}>" + - id: sender-email-domain + assert: + path: context.sender.email + matches: "@example\\\\.com$" + message: Sender email must come from example.com. + severity: warning +sections: + - id: greeting + title: Greeting + presence: required + level: 2 + - id: body + title: Body + presence: required + level: 2 + - id: closing + title: Closing + presence: required + level: 2 +rubrics: + - id: tone-fit + scope: section.body + criteria: The body should match the recipient relationship. +``` +""" + + +LETTER_DOC = """--- +document_type: business-letter +--- + +# Follow Up + +## Greeting + +Dear Ada, + +## Body + +Thank you for the productive discussion. We will follow up with a concise +proposal and next steps for the Markdown workflow. + +## Closing + +Kind regards +""" + + +LETTER_CONTEXT = """metadata: + case_id: case-42 +schema: + type: object + required: [recipient, sender] + properties: + recipient: + type: object + required: [name] + sender: + type: object + required: [name, email] +context: + recipient: + name: Ada Lovelace + sender: + name: Markitect Team + email: hello@example.com +""" + + +def test_runtime_context_loads_yaml_and_validates_schema(tmp_path: Path): + context_file = tmp_path / "context.yaml" + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + + context = load_runtime_context_file(context_file) + + assert context.valid is True + assert context.data["recipient"]["name"] == "Ada Lovelace" + assert context.metadata["case_id"] == "case-42" + + +def test_runtime_context_reports_schema_failure(tmp_path: Path): + context_file = tmp_path / "context.yaml" + context_file.write_text( + "schema:\n type: object\n required: [recipient]\ncontext: {}\n", + encoding="utf-8", + ) + + context = load_runtime_context_file(context_file) + + assert context.valid is False + assert context.diagnostics[0].code == "runtime.context.schema" + + +def test_form_state_prefills_defaults_hides_fields_and_calculates_values(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + context_file = tmp_path / "context.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + document = parse_markdown(LETTER_DOC, source_path="letter.md") + + form_state = evaluate_form_state( + document, + load_contract_file(contract_file), + load_runtime_context_file(context_file), + ) + fields = {field.id: field for field in form_state.fields} + + assert form_state.valid is True + assert fields["recipient_name"].origin == "prefilled" + assert fields["delivery_channel"].origin == "defaulted" + assert fields["postal_address"].visible is False + assert fields["contact_label"].origin == "calculated" + assert fields["contact_label"].value == "Markitect Team " + + +def test_context_conflict_keeps_manual_document_value_as_warning(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + context_file = tmp_path / "context.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + document = parse_markdown( + LETTER_DOC.replace( + "document_type: business-letter", + "document_type: business-letter\nrecipient_name: Grace Hopper", + ), + source_path="letter.md", + ) + + form_state = evaluate_form_state( + document, + load_contract_file(contract_file), + load_runtime_context_file(context_file), + ) + recipient = next(field for field in form_state.fields if field.id == "recipient_name") + + assert form_state.valid is True + assert recipient.value == "Grace Hopper" + assert "runtime.field.conflict" in {diagnostic.code for diagnostic in form_state.diagnostics} + + +def test_dynamic_rule_requires_print_address(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + context_file = tmp_path / "context.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + document = parse_markdown( + LETTER_DOC.replace( + "document_type: business-letter", + "document_type: business-letter\ndelivery_channel: print", + ), + source_path="letter.md", + ) + + form_state = evaluate_form_state( + document, + load_contract_file(contract_file), + load_runtime_context_file(context_file), + ) + + assert form_state.valid is False + assert "contract.field.missing" in {diagnostic.code for diagnostic in form_state.diagnostics} + + +def test_dynamic_section_rule_can_require_section(tmp_path: Path): + contract_file = tmp_path / "workplan.contract.md" + contract_file.write_text( + """# Workplan Contract + +```yaml contract +id: dynamic-workplan-v1 +document: + type: workplan +fields: + status: + type: string + required: true +sections: + - id: purpose + title: Purpose + presence: required + - id: decision-point + title: Decision Point + presence: optional +rules: + - id: require-decision-when-done + if: + path: fields.status.value + equals: done + then: + sections: + decision-point: + presence: required +``` +""", + encoding="utf-8", + ) + document = parse_markdown( + "---\ndocument_type: workplan\nstatus: done\n---\n# WP\n\n## Purpose\n\nDone.\n", + source_path="workplan.md", + ) + + form_state = evaluate_form_state(document, load_contract_file(contract_file)) + + assert form_state.valid is False + assert "runtime.section.missing" in {diagnostic.code for diagnostic in form_state.diagnostics} + + +def test_contract_check_uses_runtime_context(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + document_file = tmp_path / "letter.md" + context_file = tmp_path / "context.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + document_file.write_text(LETTER_DOC, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + + result = check_markdown_file(document_file, contract_file, context_path=context_file) + + assert result.valid is True + assert result.runtime["form_state"]["field_values"]["recipient_name"] == "Ada Lovelace" + + +def test_contract_cli_accepts_context_and_reports_form_state(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + document_file = tmp_path / "letter.md" + context_file = tmp_path / "context.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + document_file.write_text(LETTER_DOC, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + + check = CliRunner().invoke( + main, + [ + "contract", + "check", + str(document_file), + "--contract", + str(contract_file), + "--context", + str(context_file), + "--format", + "json", + ], + ) + form_state = CliRunner().invoke( + main, + [ + "contract", + "form-state", + str(document_file), + "--contract", + str(contract_file), + "--context", + str(context_file), + "--format", + "text", + ], + ) + + assert check.exit_code == 0 + assert '"runtime"' in check.output + assert form_state.exit_code == 0 + assert "recipient_name: Ada Lovelace [prefilled]" in form_state.output + + +def test_assessment_runner_normalizes_cache_and_failure_diagnostics(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + document = parse_markdown_file(_write_file(tmp_path / "letter.md", LETTER_DOC)) + requests = assessment_requests_for_contract(document, load_contract_file(contract_file)) + + class Adapter: + calls = 0 + + def assess(self, request): + self.calls += 1 + return AssessmentResult( + rule_id=request.rule_id, + passed=False, + score=0.2, + reason="Tone is too vague.", + provider="mock", + model="mock-grader", + ) + + adapter = Adapter() + runner = AssessmentRunner(adapter, cache=MemoryAssessmentCache()) + first = runner.assess(requests[0]) + second = runner.assess(requests[0]) + run = runner.assess_all(requests) + + assert first.cached is False + assert second.cached is True + assert adapter.calls == 1 + assert "runtime.assessment.failed" in {diagnostic.code for diagnostic in run.diagnostics} + + +def test_workflow_form_state_step_uses_context(tmp_path: Path): + contract_file = tmp_path / "letter.contract.md" + document_file = tmp_path / "letter.md" + context_file = tmp_path / "context.yaml" + workflow_file = tmp_path / "workflow.yaml" + contract_file.write_text(LETTER_CONTRACT, encoding="utf-8") + document_file.write_text(LETTER_DOC, encoding="utf-8") + context_file.write_text(LETTER_CONTEXT, encoding="utf-8") + workflow_file.write_text( + """metadata: + id: runtime-workflow +steps: + - id: form + kind: form_state + document: letter.md + contract: letter.contract.md + context: context.yaml +""", + encoding="utf-8", + ) + + result = WorkflowRunner(load_workflow_file(workflow_file)).run() + + assert result.valid is True + assert result.steps["form"]["field_values"]["recipient_name"] == "Ada Lovelace" + + +def _write_file(path: Path, text: str) -> Path: + path.write_text(text, encoding="utf-8") + return path diff --git a/workplans/MKTT-WP-0005-runtime-context-and-assessment-engines.md b/workplans/MKTT-WP-0005-runtime-context-and-assessment-engines.md index f68f358..f4255af 100644 --- a/workplans/MKTT-WP-0005-runtime-context-and-assessment-engines.md +++ b/workplans/MKTT-WP-0005-runtime-context-and-assessment-engines.md @@ -3,10 +3,10 @@ id: MKTT-WP-0005 type: workplan title: "Runtime Context, Form, and Assessment Engines" domain: markitect -status: todo +status: done owner: markitect-tool topic_slug: markitect -planning_priority: P2 +planning_priority: complete planning_order: 70 depends_on_workplans: - MKTT-WP-0003 @@ -31,7 +31,11 @@ This workplan picks up the deferred runtime scope from ## Decision -Do not start this immediately unless one of these is true: +This workplan is implemented. It was picked up after the workflow and internal +extension framework work made runtime form/context evaluation useful and cleanly +integrable. + +The original trigger conditions were: - We are implementing template/generation flows that need reliable field prefill and pre-render validation. @@ -41,16 +45,12 @@ Do not start this immediately unless one of these is true: - A user-facing or agent-facing workflow needs structured form state, defaults, conditional requiredness, or guided repair. -Recommended sequencing: +Implemented sequencing: 1. Implement context and form runtime first. 2. Add deterministic context-aware rules. -3. Add LLM assessment execution only after the diagnostic/caching boundary is - stable. - -This is probably not the next immediate implementation if we want to first -finish core query/extraction and deterministic transform primitives. It should -come before serious generation pipelines or any LLM review loop. +3. Add provider-neutral assessment execution after the diagnostic/caching + boundary was stable. ## Background @@ -76,7 +76,7 @@ It does not yet execute: ```task id: MKTT-WP-0005-T001 -status: todo +status: done priority: high state_hub_task_id: "e24e6238-efef-41c4-9f1e-ca677c1be89b" ``` @@ -96,7 +96,7 @@ Expected output: design notes and tests for context loading. ```task id: MKTT-WP-0005-T002 -status: todo +status: done priority: high state_hub_task_id: "d180bb6d-dae8-4305-88de-64c80b708b8a" ``` @@ -114,7 +114,7 @@ application-specific data access belongs in adapters outside the core package. ```task id: MKTT-WP-0005-T003 -status: todo +status: done priority: high state_hub_task_id: "b954984a-6f67-4e5b-8744-35e3c4fcc992" ``` @@ -135,7 +135,7 @@ document exists. ```task id: MKTT-WP-0005-T004 -status: todo +status: done priority: medium state_hub_task_id: "cccdf868-2308-42a1-b564-8b54fccd3c8b" ``` @@ -157,7 +157,7 @@ This should support future UI adapters while remaining useful from the CLI. ```task id: MKTT-WP-0005-T005 -status: todo +status: done priority: medium state_hub_task_id: "6e420e1e-2465-40d3-8e64-d8681a294e63" ``` @@ -178,7 +178,7 @@ small set of operators over embedding a general programming language. ```task id: MKTT-WP-0005-T006 -status: todo +status: done priority: medium state_hub_task_id: "24b22b3a-e89e-4946-81f4-94f971a11979" ``` @@ -205,7 +205,7 @@ implementation. ```task id: MKTT-WP-0005-T007 -status: todo +status: done priority: medium state_hub_task_id: "b09b77e2-59c0-4d31-b246-685b742d111f" ``` @@ -220,7 +220,7 @@ cache may be local file-based only if it remains transparent and easy to reset. ```task id: MKTT-WP-0005-T008 -status: todo +status: done priority: high state_hub_task_id: "2efb8233-3154-4824-a898-6fcde37330c5" ``` @@ -237,14 +237,29 @@ prefill, invalid dynamic rules, and assessment failures. ## Open Questions -- Should context values override frontmatter, or should conflicts always be - diagnostics until explicitly resolved? -- Should the first dynamic rule syntax reuse JSON Schema conditionals or define - a smaller Markitect-native rule vocabulary? -- Should LLM assessment execution live behind an optional extra, or only in - external adapters? -- What cache invalidation metadata is sufficient for assessment reproducibility - without pretending model judgments are deterministic? +- Resolved: document/frontmatter values win over context; conflicts are + diagnostics and can be escalated per field. +- Resolved: dynamic rules use a smaller Markitect-native vocabulary over + JSON/YAML paths instead of embedding JSON Schema conditionals. +- Resolved: provider execution remains external; core defines request/result, + runner, diagnostics, and cache boundary only. +- Resolved for core: assessment cache keys include contract/rule/scope/text, + criteria, context, structured inputs, provider/model metadata, threshold, and + metadata. Persistent cache invalidation remains a backend concern. + +## Implementation Notes + +- Added `markitect_tool.runtime` with context loading, path resolution, + deterministic condition evaluation, form state, dynamic rules, assessment + request/result models, runner, and in-memory cache boundary. +- `mkt contract check` now accepts `--context`. +- Added `mkt contract form-state`. +- Workflow `contract_check` accepts context and workflow `form_state` exposes + runtime field state as a step result. +- Built-in extension descriptors now include `runtime.context`, + `runtime.form-state`, and `runtime.assessment`. +- Documentation lives in `docs/runtime-context-forms-assessments.md`. +- Examples live in `examples/runtime/`. ## Exit Criteria