kontextual-engine/docs/system-layer-migration-backlog.md

System-Layer Migration Backlog

Date: 2026-05-05

This backlog is the output of KONT-WP-0002. It feeds KONT-WP-0003: Headless Knowledge Engine Implementation.

Migration Strategies

• migrate-test: port or rewrite old tests first.
• reimplement: build a new implementation around the kontextual-engine PRD/FRS.
• adapter: call another repo, especially markitect-tool or llm-connect.
• defer: keep documented, but do not build in the first implementation slice.
• out-of-scope: leave in legacy or another repo.

P0 - Contract Baseline

Item	Strategy	Source	Output
Define engine model names and module layout	reimplement	PRD/FRS, scope assessment	kontextual_engine/artifacts, collections, storage, relationships, workflows, query, context, integrations
Structured error envelope	reimplement	infrastructure/exceptions.py, production/error_handler.py	Common KontextualError / diagnostic model
Adapter boundary to markitect-tool	adapter	docs/markitect-tool-reuse-boundary.md	Markdown ingestion adapter protocol

P1 - Artifacts, Collections, And Persistence

Item	Strategy	Source	Acceptance direction
Artifact with id, collection id, name, type, digest, size, metadata, timestamps	migrate-test + reimplement	markitect/prompts/models.py, test_artifact_models.py	Deterministic digest and serialization tests pass.
ArtifactReference / address parser	migrate-test + reimplement	ArtifactReference.parse tests	Supports local, collection-qualified, and versioned refs.
Collection model replacing legacy InformationSpace assumptions	reimplement	infospace config/state concepts	Supports nested or related collections without filesystem coupling.
Storage repository protocol	migrate-test + reimplement	prompt repository tests, SQL migrations	CRUD, duplicate detection, lookup by id/name/digest.
First in-memory repository	reimplement	none	Enables fast unit tests before SQLite decision.
SQLite schema design note	reimplement	prompt migrations 001-006	Decide whether to use SQLAlchemy or direct SQLite in later slice.

P2 - Relationships And Graphs

Item	Strategy	Source	Acceptance direction
Relationship model with source, target, predicate, type, evidence, provenance	migrate-test + reimplement	RelationMeta, DependencyEdge	Relationship roundtrip and edge tuple tests.
Artifact dependency graph	migrate-test + reimplement	DependencyGraph, dependency tests	Successor/predecessor, cycle detection, topological sort.
Collection dependency and stale mapping model	reimplement	composition.py, discipline status/stale mapping	Can detect missing target relationships.
Graph query surface	reimplement	EntityGraph, query result envelope	Return nodes/edges with provenance, not Mermaid/DOT.
Graph export	adapter/defer	graph_export.py, markitect-tool docs	Defer visual export; engine returns data.

P3 - Workflow Runs And Manifests

Item	Strategy	Source	Acceptance direction
OperationRun / WorkflowRun lifecycle	migrate-test + reimplement	PromptRun, test_execution_models.py	Stage transitions, success/fail/skip semantics.
InputBundle idempotency hash	migrate-test + reimplement	InputBundle, RunManifestSchema.md	Hash independent of dict ordering.
Run manifest model	migrate-test + reimplement	RunManifestSchema.md, migration 003	Captures inputs, outputs, dependencies, validation, impact debt, timing.
Nested operation runs	migrate-test + reimplement	parent run/depth tests	Parent/child links and max-depth checks.
Workflow step adapter registry	adapter + reimplement	markitect-tool workflow standard	Steps call adapters; engine persists run state.
Prompt execution operation kind	adapter/defer	prompt execution engine	Defer provider execution until llm-connect adapter is explicit.

P4 - Ingestion And Normalization

Item	Strategy	Source	Acceptance direction
Ingestion adapter protocol	reimplement	PRD/FRS FR-020/021	Accepts bytes/text/path plus media type and returns normalized artifact(s).
Markdown ingestion adapter	adapter	markitect_tool.core.parser, backend snapshots	Calls markitect-tool; does not parse markdown internally.
Infospace fixture adapter	migrate-test + adapter	examples/infospace-with-history	Import a small fixture subset as collections/artifacts/relationships.
Multi-format placeholder interfaces	reimplement/defer	PRD/FRS	Design now; implement non-markdown later.

P5 - Query And Retrieval

Item	Strategy	Source	Acceptance direction
Engine QueryResult envelope	migrate-test + reimplement	query paradigm QueryResult	Includes query, count, results, metadata, diagnostics.
Query by id/name/digest/metadata	reimplement	storage requirements	Deterministic repository-backed tests.
Query relationships and provenance	reimplement	graph/dependency tests	Finds neighbors, dependents, ancestors, producing runs.
Markdown selector passthrough	adapter	markitect-tool query/extract	Only available for markdown artifacts with source snapshots.
FTS/vector search	adapter/defer	markitect-tool local index, FTS plugin	Defer durable FTS until storage backend is chosen.

P6 - Quality, Assessment, And Impact

Item	Strategy	Source	Acceptance direction
Assessment snapshot model	migrate-test + reimplement	EvaluationSnapshot, infospace tests	Attach scores/metrics to artifact, collection, or run.
Quality gate records	migrate-test + reimplement	quality models/tests, migration 006	Persist pass/fail/skipped with diagnostics.
Halting policy model	migrate-test + reimplement	quality policy tests	Uses gate status, iteration limits, budgets, improvement.
Artifact change records	migrate-test + reimplement	ArtifactChange, migration 005	Created/modified/deleted with old/new digest.
Impact debt records	migrate-test + reimplement	ImpactDebt, integration tests	Records suppressed recompute with magnitude and reason.
LLM assessed impact	adapter/defer	prompt FRS	Use llm-connect later; deterministic metrics first.

P7 - State Hub And Documentation Follow-Up

Item	Strategy	Source	Acceptance direction
Update KONT-WP-0003 from this backlog	reimplement	this doc	Ensure implementation workplan tasks line up with backlog order.
Add architectural ADR for storage/backend decision	defer	docs/stack-decision.md	Decide after in-memory contracts stabilize.
Add SBOM/lockfile	defer	foundation DoI	Generate once dependencies are installed with a lockfile.

Explicitly Out Of Scope

• Porting markitect/core, markitect/schema, explode/implode, transforms, includes, document contracts, or selector internals. Use markitect-tool.
• Porting provider-specific markitect/llm code. Use llm-connect.
• Porting finance, issue tracker, profile, release, Gitea, browser, rendering, or UI plugin code.
• Porting GraphQL as a default API. Use it only as historical evidence.