kontextual-engine/workplans/KONT-WP-0004-durable-persistence.md

id, type, title, domain, repo, status, owner, topic_slug, created, updated, state_hub_workstream_id

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
KONT-WP-0004	workplan	Durable Persistence Foundation	markitect	kontextual-engine	todo	codex	markitect	2026-05-05	2026-05-05	e177f2dc-a2a0-41a4-b5cd-82e8f9f12f34

KONT-WP-0004: Durable Persistence Foundation

Purpose

Close the persistence gap identified after KONT-WP-0003 by turning the current in-memory repository contract into a durable, local-first storage foundation for knowledge artifacts, collections, relationships, workflow state, and context references.

This workplan deliberately does not implement phase-memory behavior. It uses docs/phase-memory-boundary.md as the boundary: kontextual-engine persists durable knowledge runtime state; phase-memory owns agentic memory phases, profiles, compaction, retention, and activation planning.

Persistence Scope

In scope:

• Durable storage for collections, artifacts, artifact revisions, and relationships.
• Durable storage for workflow runs and run manifests.
• Explicit update and delete behavior for artifacts and relationships.
• Change records that make artifact evolution inspectable.
• Query support for identifiers, names, digests, metadata, content text, and relationships.
• Local-first SQLite backend with a repository interface that can later be backed by service, PostgreSQL, graph, or object-storage adapters.
• Tests proving data survives repository re-instantiation.

Out of scope:

• Memory-phase lifecycle behavior from phase-memory.
• Vector search, embedding storage, and memory activation planning.
• Markdown parsing or markdown transformations; use markitect-tool adapters.
• LLM provider execution; use future llm-connect adapters.
• Remote multi-tenant deployment concerns beyond schema choices that do not block later migration.

P4.1 - Finalize persistence boundary and ADR

id: KONT-WP-0004-T001
status: todo
priority: high
state_hub_task_id: "6b665ab1-cc8e-473b-824a-d953b598bb72"

Promote the storage decision from deferred to explicit: local-first SQLite for the first durable backend, wrapped by repository contracts. Decide whether the implementation uses direct sqlite3 or SQLAlchemy for this slice.

Output: update docs/stack-decision.md or add an ADR under docs/.

Acceptance:

• The backend choice is explicit and justified.
• The decision references docs/phase-memory-boundary.md.
• Future service-backed storage remains possible.

P4.2 - Complete repository contract semantics

id: KONT-WP-0004-T002
status: todo
priority: high
state_hub_task_id: "eed4f0b5-9080-4c76-9ae6-841459edbab6"

Extend KnowledgeRepository from create/list/get into a durable lifecycle contract. Define update, delete, revision, and transaction semantics without binding callers to a specific backend.

Output: src/kontextual_engine/storage.py and focused tests.

Acceptance:

• Artifact update produces explicit revision/change semantics.
• Artifact delete behavior is defined for relationships and query results.
• Duplicate-name and referential-integrity behavior remains deterministic.
• Existing in-memory tests continue to pass.

P4.3 - Design durable schema and migrations

id: KONT-WP-0004-T003
status: todo
priority: high
state_hub_task_id: "7f34e36f-4e9b-40ab-bbe9-afaee4553a9f"

Create the first durable schema for collections, artifacts, revisions, relationships, workflow runs, run manifests, and change records.

Output: schema/migration files under src/kontextual_engine/storage/ or an equivalent package-owned location.

Acceptance:

• Schema stores content digest, artifact type, size, metadata, timestamps, and provenance.
• Relationships enforce valid source and target artifacts.
• JSON metadata is preserved roundtrip.
• Migrations can initialize an empty local database deterministically.

P4.4 - Implement SQLite repository backend

id: KONT-WP-0004-T004
status: todo
priority: high
state_hub_task_id: "6d20a457-7246-4380-943f-c6d726506356"

Implement SQLiteKnowledgeRepository behind the same repository contract used by the in-memory backend.

Output: durable repository implementation and tests.

Acceptance:

• Collections, artifacts, relationships, and metadata survive closing and reopening the repository.
• Query behavior matches the in-memory repository for supported filters.
• Tests cover duplicate artifact names, missing relationship endpoints, and deterministic ordering.
• No markdown or memory-runtime logic is introduced.

P4.5 - Persist artifact evolution

id: KONT-WP-0004-T005
status: todo
priority: high
state_hub_task_id: "e4e6f188-9ac3-4daf-9633-f11d812e50fa"

Add artifact revision and change-record support so persistent knowledge can be versioned and audited over time.

Output: model additions, repository methods, and tests.

Acceptance:

• Updating an artifact records old and new digests.
• Revision history can be retrieved by artifact id.
• Deletion is traceable through a change record.
• Change records are backend-neutral at the programmatic API boundary.

P4.6 - Persist workflow run state

id: KONT-WP-0004-T006
status: todo
priority: medium
state_hub_task_id: "d0a9e9d4-12eb-406b-b32c-5b45f931f18c"

Persist OperationRun, WorkflowStep, InputBundle, and RunManifest records so orchestration can resume and inspect prior execution.

Output: repository methods and persistence tests for workflow records.

Acceptance:

• Run status transitions survive repository re-instantiation.
• Run manifests roundtrip with inputs, outputs, diagnostics, and timestamps.
• Artifact outputs can be linked to producing runs.

P4.7 - Add context and phase-memory reference hooks

id: KONT-WP-0004-T007
status: todo
priority: medium
state_hub_task_id: "965738a5-9538-45f6-98bb-7987aba62904"

Add lightweight persistence for context-package references and external memory references without implementing memory lifecycle behavior.

Output: context reference model and tests.

Acceptance:

• Context packages can refer to artifacts, relationships, runs, and external memory records.
• External memory references are opaque and provenance-tagged.
• No retention, decay, compaction, activation planning, or preference-memory behavior is added to this repo.

P4.8 - Add import/export and smoke verification

id: KONT-WP-0004-T008
status: todo
priority: medium
state_hub_task_id: "ea7313ce-fb1f-49b1-b5da-66a036893a04"

Provide a deterministic import/export path for repository snapshots so early users and agents can inspect or migrate local state.

Output: programmatic snapshot helpers and tests.

Acceptance:

• A repository snapshot can be exported and imported into a fresh backend.
• Imported data preserves ids, digests, metadata, relationships, revisions, and run links.
• Snapshot format does not become a replacement for the service API.

Definition Of Done

• python3 -m pytest passes.
• Existing in-memory behavior remains compatible unless explicitly revised.
• SQLite-backed tests prove durable behavior across repository re-instantiation.
• Persistence docs explain what is durable now and what remains deferred.
• docs/phase-memory-boundary.md remains the boundary for memory-specific behavior.