--- id: KONT-WP-0008 type: workplan title: "Traceable Transformations And Workflow Jobs" domain: markitect repo: kontextual-engine status: todo owner: codex topic_slug: markitect planning_priority: high planning_order: 8 created: "2026-05-05" updated: "2026-05-05" state_hub_workstream_id: "1b7a6b04-7879-4862-bb3e-817f7f20fc59" --- # KONT-WP-0008: Traceable Transformations And Workflow Jobs ## Purpose Implement the operations layer that turns knowledge assets into traceable outputs: transformation operations, derived artifacts, workflow templates, workflow runs, job execution state, retries, cancellation, review gates, exception queues, and operation audit. ## Requirement Coverage Primary: FR-080 to FR-090 and FR-100 to FR-110. Supporting: FR-083 to FR-085, FR-106, FR-144 to FR-145, FR-165, FR-200 to FR-202. ## Architecture Constraint Implement transformations and workflows through operation registries, workflow services, repository ports, event ports, policy checks, and audit events described in `docs/architecture-blueprint.md`. Execution may start embedded, but contracts must allow later queue or workflow-engine adapters. ## O8.1 - Implement transformation operation registry ```task id: KONT-WP-0008-T001 status: todo priority: high state_hub_task_id: "ee2471b1-fab3-48f5-8b2d-d8f624abfc35" ``` Create a registry for transformation operations such as summarize, extract, classify, compose, validate, generate report, and produce structured view. Acceptance: - Operations declare inputs, outputs, parameters, required permissions, and supported asset types. - Provider-specific LLM behavior remains behind adapters. - Unsupported operations return structured capability errors. ## O8.2 - Implement transformation runs with parameters actors and policy context ```task id: KONT-WP-0008-T002 status: todo priority: high state_hub_task_id: "1eac7b47-8cff-4736-9f7d-599123218bad" ``` Represent each transformation as a run with source assets, source versions, operation type, parameters, actor, policy context, timestamps, and status. Acceptance: - Transformations can be queued, run, completed, failed, retried, or canceled. - Transformation permissions are checked before reading sources or writing outputs. - Parameters needed to interpret or reproduce the run are preserved. ## O8.3 - Persist derived artifacts and source lineage ```task id: KONT-WP-0008-T003 status: todo priority: high state_hub_task_id: "837ad793-2e9a-41f0-bce6-0a75815b5c15" ``` Persist summaries, extracts, reports, structured representations, generated artifacts, and composed outputs as governed derived artifacts. Acceptance: - Derived artifacts have stable identity and lineage to source assets. - Lineage includes transformation run, source versions, actor, parameters, policy context, and output representation. - Re-runs create new traceable records rather than silently overwriting outputs. ## O8.4 - Implement workflow templates steps dependencies and preconditions ```task id: KONT-WP-0008-T004 status: todo priority: high state_hub_task_id: "2c55c5dd-f07b-466b-85a5-f229e41fd124" ``` Define reusable workflow templates containing steps, dependencies, inputs, outputs, preconditions, policy checks, and failure behavior. Acceptance: - Templates can be created and invoked programmatically. - Step dependencies prevent unsafe or premature execution. - Workflow inputs can be assets, collections, queries, source events, or submitted payloads. ## O8.5 - Implement job runner status retry resume and cancel behavior ```task id: KONT-WP-0008-T005 status: todo priority: high state_hub_task_id: "5f4d6c88-904d-4369-90d5-eaa4d27e3010" ``` Implement a simple MVP job runner for workflows and transformations. Acceptance: - Runs expose queued, running, waiting, completed, failed, retried, canceled, and partially completed states. - Safe retry, resume, and cancellation behavior is defined per operation. - Recovery actions do not require direct storage edits. ## O8.6 - Implement review gates human tasks and exception queues ```task id: KONT-WP-0008-T006 status: todo priority: medium state_hub_task_id: "5fae9005-4d64-4fca-8c51-a19405512377" ``` Add workflow primitives for review, approval, correction, rejection, low-confidence handling, policy conflicts, and blocked exceptions. Acceptance: - Sensitive or high-impact outputs can pause for human review. - Exception queues expose failed, blocked, low-confidence, policy-conflicted, or review-required items. - Review decisions continue, reject, correct, retry, or escalate runs. ## O8.7 - Audit workflow and transformation operations ```task id: KONT-WP-0008-T007 status: todo priority: medium state_hub_task_id: "9e06aa46-3988-4389-99ec-0a934c68af1b" ``` Audit template changes, run starts, step executions, retries, cancellations, approvals, failures, outputs, and derived artifact changes. Acceptance: - A workflow run can be reconstructed from run records and audit events. - Audit records include actor, operation, target, outcome, correlation ID, and policy context. - Derived artifact audit events connect to source lineage. ## Definition Of Done - Transformations and workflows produce inspectable run records and audit events. - Derived artifacts are persistent, governed, and lineage-linked. - Transformation and workflow run models follow `docs/architecture-blueprint.md`. - `python3 -m pytest` passes.