Implemented durable workflow/job foundation

docs/asset-registry-implementation.md

@@ -65,6 +65,7 @@ and SQLite repositories are adapters behind those ports.
 - Idempotency-key reuse with a different payload raises a validation error.
 - Transformation run and derived lineage persistence for traceable derived
   artifact creation.
+- Workflow template and workflow run persistence for durable job execution.
 - In-memory repository for deterministic tests.
 - SQLite repository for local-first durable asset registry state.
 - SQLite foreign-key enforcement for representation and metadata asset
@@ -89,12 +90,14 @@ and SQLite repositories are adapters behind those ports.
 - `ingestion_jobs`
 - `transformation_runs`
 - `derived_lineage`
+- `workflow_templates`
+- `workflow_runs`
 
 Payloads are stored as compact JSON envelopes while indexed columns carry
 stable lookup fields such as asset ID, lifecycle, representation kind, digest,
 sequence, relationship source/target, actor ID, target, correlation ID,
-idempotency key, transformation status, operation ID, and derived output asset
-ID.
+idempotency key, transformation status, operation ID, derived output asset ID,
+workflow template ID, workflow version, and workflow run status.
 
 ## Not Yet Implemented
 

docs/workflow-jobs-implementation.md

@@ -0,0 +1,104 @@
+# Workflow Jobs Implementation Note
+
+Date: 2026-05-06
+
+Status: active implementation note for `KONT-WP-0008`.
+
+## Purpose
+
+This note records the first durable workflow and job-runner foundation. It
+extends traceable transformations into reusable workflow templates with
+dependency-ordered execution and recoverable run state.
+
+## Implemented Package Shape
+
+```text
+src/kontextual_engine/
+  core/
+    workflow_jobs.py
+  services/
+    workflow_service.py
+  ports/
+    repositories.py
+  adapters/
+    memory/asset_registry.py
+    sqlite/asset_registry.py
+```
+
+The workflow service uses the existing repository and policy ports. Executable
+steps delegate to `TransformationService`, so workflow runs do not bypass asset
+governance, transformation run records, derived lineage, or audit.
+
+## Implemented Capabilities
+
+- `WorkflowTemplate` for reusable template identity, version, metadata,
+  inputs, steps, policy checks, and failure behavior.
+- `WorkflowInputDefinition` supports asset, collection, query, source event,
+  and payload inputs.
+- `WorkflowStepDefinition` captures step kind, operation ID, dependencies,
+  input/output bindings, preconditions, failure behavior, and metadata.
+- Template registration validates duplicate step IDs, missing dependencies,
+  dependency cycles, and unsupported failure behavior.
+- `WorkflowRun` and `WorkflowStepRun` persist queued, running, waiting,
+  completed, partially completed, failed, retried, and canceled states.
+- `WorkflowService` can register templates, queue runs, invoke runs, resume
+  runs, retry runs, and cancel runs programmatically.
+- The MVP runner executes transformation-backed steps in dependency order.
+- Step inputs can resolve source asset IDs from invocation inputs or completed
+  upstream step outputs.
+- Retry and repeated invocation avoid silent overwrite by choosing a fresh
+  output asset ID when a fixed template output ID already exists.
+- Workflow template and run persistence are implemented for memory and SQLite.
+- Audit events are emitted for template registration, run queue/start/final
+  states, step start/final states, retry, and cancellation.
+
+## Current Boundaries
+
+The runner executes only `kind="transformation"` steps. Non-transformation
+steps are represented and fail with structured diagnostics until a later
+adapter or job worker handles them.
+
+Workflow inputs preserve collection, query, source event, and payload bindings,
+but the MVP runner only interprets asset bindings for transformation execution.
+Query expansion, source-event ingestion, human tasks, and exception queues stay
+in later WP-0008 tasks.
+
+Markdown-specific transformations remain adapter-backed through markitect-tool.
+Workflow orchestration may invoke those operations once the adapter boundary is
+wired, but the engine does not reimplement Markdown semantics.
+
+## Current SQLite Tables
+
+WP-0008 adds shared registry persistence for:
+
+- `workflow_templates`
+- `workflow_runs`
+
+The tables store compact JSON payloads with indexed lookup columns for template
+ID, template version, template name, run ID, run status, actor ID, correlation
+ID, queued timestamp, and updated timestamp.
+
+## Not Yet Implemented
+
+- Human review gates and approval tasks.
+- Exception queues for blocked, failed, low-confidence, or policy-conflicted
+  items.
+- Queue-worker adapters beyond embedded execution.
+- Rich retry policies by operation type.
+- Query/input expansion into dynamic asset sets.
+- Full workflow reconstruction views across all audit and lineage records.
+
+These remain in open tasks `KONT-WP-0008-T006` and `KONT-WP-0008-T007`.
+
+## Test Coverage
+
+`tests/test_workflow_service.py` covers:
+
+- template registration, input-kind persistence, missing dependency rejection,
+  and dependency-cycle rejection,
+- dependency-ordered execution of two transformation steps,
+- queue, cancel, resume denial, and retry without direct storage edits,
+- partial completion when a continue-on-failure step fails and another step
+  succeeds,
+- SQLite reload of workflow templates, workflow runs, step state, and derived
+  output representation state.