generated from coulomb/repo-seed
112 lines
4.3 KiB
Markdown
112 lines
4.3 KiB
Markdown
# Transformation Implementation Note
|
|
|
|
Date: 2026-05-06
|
|
|
|
Status: active implementation note for `KONT-WP-0008`.
|
|
|
|
## Purpose
|
|
|
|
This note records the first traceable transformation foundation. The goal is to
|
|
turn governed source assets into governed derived assets without losing actor,
|
|
policy, run, version, and lineage context.
|
|
|
|
## Implemented Package Shape
|
|
|
|
```text
|
|
src/kontextual_engine/
|
|
core/
|
|
transformations.py
|
|
provenance.py
|
|
services/
|
|
transformation_service.py
|
|
ports/
|
|
repositories.py
|
|
adapters/
|
|
memory/asset_registry.py
|
|
sqlite/asset_registry.py
|
|
```
|
|
|
|
The transformation service depends on the asset registry repository and policy
|
|
gateway ports. Derived outputs are persisted through `AssetRegistryService`
|
|
rather than by bypassing existing asset governance.
|
|
|
|
## Implemented Capabilities
|
|
|
|
- `TransformationOperation` descriptors with input specs, output specs,
|
|
parameter schema, required permissions, supported asset types, adapter
|
|
references, and metadata.
|
|
- `TransformationOperationRegistry` with a default operation catalog.
|
|
- Executable engine-owned `structured_view` operation that produces a JSON
|
|
derived representation from source asset metadata and representation refs.
|
|
- Adapter-backed operation descriptors for `summarize`, `classify`,
|
|
`generate_report`, and Markdown operations.
|
|
- Markdown compose/include/transform/validate operations are registered with
|
|
`adapter_ref="markitect-tool"` and no engine-local handler.
|
|
- Structured capability diagnostics for unsupported operations, unsupported
|
|
asset types, missing parameters, missing executable adapters, policy denial,
|
|
and execution failure.
|
|
- `TransformationRun` records with queued, running, waiting, completed,
|
|
partially completed, failed, retried, and canceled states.
|
|
- Run persistence in memory and SQLite repositories.
|
|
- Source-read and run-execute policy decisions captured in run policy context
|
|
before transformation handler execution.
|
|
- Derived artifact creation through the governed asset registry service.
|
|
- Derived output versions use `VersionChangeType.DERIVED_OUTPUT`, store the
|
|
transformation run ID, and link to the first source parent version when
|
|
available.
|
|
- `DerivedArtifactLineage` persistence and lookup by output asset, source
|
|
asset, or transformation run.
|
|
- `derived_artifact.lineage.linked` audit events connect output assets to
|
|
transformation runs, source assets, source versions, and output
|
|
representations.
|
|
- Audit events for queued, started, completed, failed, denied, and canceled
|
|
transformation runs.
|
|
|
|
## markitect-tool Boundary
|
|
|
|
Kontextual Engine owns run state, governance, derived artifact identity,
|
|
lineage, policy context, and audit. Markitect owns Markdown-specific semantics.
|
|
|
|
The engine therefore registers Markdown operations as adapter-backed
|
|
capabilities but does not implement Markdown syntax processing itself. Once the
|
|
adapter integration is wired, these operations should call Markitect functions
|
|
through a stable boundary and keep the resulting run/output/lineage state in
|
|
Kontextual Engine.
|
|
|
|
## Current SQLite Tables
|
|
|
|
WP-0008 adds shared registry persistence for:
|
|
|
|
- `transformation_runs`
|
|
- `derived_lineage`
|
|
|
|
The tables store compact JSON payloads with indexed lookup columns for run ID,
|
|
operation ID, status, actor ID, correlation ID, queued/updated timestamps,
|
|
output asset ID, and transformation run lineage.
|
|
|
|
## Not Yet Implemented
|
|
|
|
- Workflow template persistence and invocation.
|
|
- Step dependencies, preconditions, and workflow input binding.
|
|
- Durable job runner execution model with resume semantics.
|
|
- Rich retry policy by operation type.
|
|
- Review gates, human tasks, and exception queues.
|
|
- Full workflow reconstruction across template, step, run, derived output, and
|
|
audit records.
|
|
- Concrete adapter execution for Markitect and provider-backed operations.
|
|
|
|
These remain in open tasks `KONT-WP-0008-T004` through
|
|
`KONT-WP-0008-T007`.
|
|
|
|
## Test Coverage
|
|
|
|
`tests/test_transformation_service.py` covers:
|
|
|
|
- default operation registration and the Markitect adapter boundary,
|
|
- structured view execution with derived asset, version, lineage, and audit
|
|
persistence,
|
|
- SQLite reload of transformation runs and derived lineage,
|
|
- adapter-backed operation failure without local Markdown reimplementation,
|
|
- unsupported operation diagnostics without creating a run,
|
|
- source-read policy denial before handler execution or output creation.
|