# Transformation Implementation Note

Date: 2026-05-06

Status: active implementation note for `KONT-WP-0008`.

## Purpose

This note records the first traceable transformation foundation. The goal is to
turn governed source assets into governed derived assets without losing actor,
policy, run, version, and lineage context.

## Implemented Package Shape

```text
src/kontextual_engine/
  core/
    transformations.py
    provenance.py
  services/
    transformation_service.py
  ports/
    repositories.py
  adapters/
    memory/asset_registry.py
    sqlite/asset_registry.py
```

The transformation service depends on the asset registry repository and policy
gateway ports. Derived outputs are persisted through `AssetRegistryService`
rather than by bypassing existing asset governance.

## Implemented Capabilities

- `TransformationOperation` descriptors with input specs, output specs,
  parameter schema, required permissions, supported asset types, adapter
  references, and metadata.
- `TransformationOperationRegistry` with a default operation catalog.
- Executable engine-owned `structured_view` operation that produces a JSON
  derived representation from source asset metadata and representation refs.
- Adapter-backed operation descriptors for `summarize`, `classify`,
  `generate_report`, and Markdown operations.
- Markdown compose/include/transform/validate operations are registered with
  `adapter_ref="markitect-tool"` and no engine-local handler.
- Structured capability diagnostics for unsupported operations, unsupported
  asset types, missing parameters, missing executable adapters, policy denial,
  and execution failure.
- `TransformationRun` records with queued, running, waiting, completed,
  partially completed, failed, retried, and canceled states.
- Run persistence in memory and SQLite repositories.
- Source-read and run-execute policy decisions captured in run policy context
  before transformation handler execution.
- Derived artifact creation through the governed asset registry service.
- Derived output versions use `VersionChangeType.DERIVED_OUTPUT`, store the
  transformation run ID, and link to the first source parent version when
  available.
- `DerivedArtifactLineage` persistence and lookup by output asset, source
  asset, or transformation run.
- `derived_artifact.lineage.linked` audit events connect output assets to
  transformation runs, source assets, source versions, and output
  representations.
- Audit events for queued, started, completed, failed, denied, and canceled
  transformation runs.

## markitect-tool Boundary

Kontextual Engine owns run state, governance, derived artifact identity,
lineage, policy context, and audit. Markitect owns Markdown-specific semantics.

The engine therefore registers Markdown operations as adapter-backed
capabilities but does not implement Markdown syntax processing itself. Once the
adapter integration is wired, these operations should call Markitect functions
through a stable boundary and keep the resulting run/output/lineage state in
Kontextual Engine.

## Current SQLite Tables

WP-0008 adds shared registry persistence for:

- `transformation_runs`
- `derived_lineage`

The tables store compact JSON payloads with indexed lookup columns for run ID,
operation ID, status, actor ID, correlation ID, queued/updated timestamps,
output asset ID, and transformation run lineage.

## Not Yet Implemented

- Workflow template persistence and invocation.
- Step dependencies, preconditions, and workflow input binding.
- Durable job runner execution model with resume semantics.
- Rich retry policy by operation type.
- Review gates, human tasks, and exception queues.
- Full workflow reconstruction across template, step, run, derived output, and
  audit records.
- Concrete adapter execution for Markitect and provider-backed operations.

These remain in open tasks `KONT-WP-0008-T004` through
`KONT-WP-0008-T007`.

## Test Coverage

`tests/test_transformation_service.py` covers:

- default operation registration and the Markitect adapter boundary,
- structured view execution with derived asset, version, lineage, and audit
  persistence,
- SQLite reload of transformation runs and derived lineage,
- adapter-backed operation failure without local Markdown reimplementation,
- unsupported operation diagnostics without creating a run,
- source-read policy denial before handler execution or output creation.