kontextual-engine/workplans/KONT-WP-0004-durable-persistence.md

---
id: KONT-WP-0004
type: workplan
title: "Durable Persistence Foundation"
domain: markitect
repo: kontextual-engine
status: todo
owner: codex
topic_slug: markitect
created: "2026-05-05"
updated: "2026-05-05"
state_hub_workstream_id: "e177f2dc-a2a0-41a4-b5cd-82e8f9f12f34"
---

# KONT-WP-0004: Durable Persistence Foundation

## Purpose

Close the persistence gap identified after `KONT-WP-0003` by turning the current
in-memory repository contract into a durable, local-first storage foundation for
knowledge artifacts, collections, relationships, workflow state, and context
references.

This workplan deliberately does not implement `phase-memory` behavior. It uses
`docs/phase-memory-boundary.md` as the boundary: `kontextual-engine` persists
durable knowledge runtime state; `phase-memory` owns agentic memory phases,
profiles, compaction, retention, and activation planning.

## Persistence Scope

In scope:

- Durable storage for collections, artifacts, artifact revisions, and
  relationships.
- Durable storage for workflow runs and run manifests.
- Explicit update and delete behavior for artifacts and relationships.
- Change records that make artifact evolution inspectable.
- Query support for identifiers, names, digests, metadata, content text, and
  relationships.
- Local-first SQLite backend with a repository interface that can later be
  backed by service, PostgreSQL, graph, or object-storage adapters.
- Tests proving data survives repository re-instantiation.

Out of scope:

- Memory-phase lifecycle behavior from `phase-memory`.
- Vector search, embedding storage, and memory activation planning.
- Markdown parsing or markdown transformations; use `markitect-tool` adapters.
- LLM provider execution; use future `llm-connect` adapters.
- Remote multi-tenant deployment concerns beyond schema choices that do not
  block later migration.

## P4.1 - Finalize persistence boundary and ADR

```task
id: KONT-WP-0004-T001
status: todo
priority: high
state_hub_task_id: "6b665ab1-cc8e-473b-824a-d953b598bb72"
```

Promote the storage decision from deferred to explicit: local-first SQLite for
the first durable backend, wrapped by repository contracts. Decide whether the
implementation uses direct `sqlite3` or SQLAlchemy for this slice.

Output: update `docs/stack-decision.md` or add an ADR under `docs/`.

Acceptance:

- The backend choice is explicit and justified.
- The decision references `docs/phase-memory-boundary.md`.
- Future service-backed storage remains possible.

## P4.2 - Complete repository contract semantics

```task
id: KONT-WP-0004-T002
status: todo
priority: high
state_hub_task_id: "eed4f0b5-9080-4c76-9ae6-841459edbab6"
```

Extend `KnowledgeRepository` from create/list/get into a durable lifecycle
contract. Define update, delete, revision, and transaction semantics without
binding callers to a specific backend.

Output: `src/kontextual_engine/storage.py` and focused tests.

Acceptance:

- Artifact update produces explicit revision/change semantics.
- Artifact delete behavior is defined for relationships and query results.
- Duplicate-name and referential-integrity behavior remains deterministic.
- Existing in-memory tests continue to pass.

## P4.3 - Design durable schema and migrations

```task
id: KONT-WP-0004-T003
status: todo
priority: high
state_hub_task_id: "7f34e36f-4e9b-40ab-bbe9-afaee4553a9f"
```

Create the first durable schema for collections, artifacts, revisions,
relationships, workflow runs, run manifests, and change records.

Output: schema/migration files under `src/kontextual_engine/storage/` or an
equivalent package-owned location.

Acceptance:

- Schema stores content digest, artifact type, size, metadata, timestamps, and
  provenance.
- Relationships enforce valid source and target artifacts.
- JSON metadata is preserved roundtrip.
- Migrations can initialize an empty local database deterministically.

## P4.4 - Implement SQLite repository backend

```task
id: KONT-WP-0004-T004
status: todo
priority: high
state_hub_task_id: "6d20a457-7246-4380-943f-c6d726506356"
```

Implement `SQLiteKnowledgeRepository` behind the same repository contract used
by the in-memory backend.

Output: durable repository implementation and tests.

Acceptance:

- Collections, artifacts, relationships, and metadata survive closing and
  reopening the repository.
- Query behavior matches the in-memory repository for supported filters.
- Tests cover duplicate artifact names, missing relationship endpoints, and
  deterministic ordering.
- No markdown or memory-runtime logic is introduced.

## P4.5 - Persist artifact evolution

```task
id: KONT-WP-0004-T005
status: todo
priority: high
state_hub_task_id: "e4e6f188-9ac3-4daf-9633-f11d812e50fa"
```

Add artifact revision and change-record support so persistent knowledge can be
versioned and audited over time.

Output: model additions, repository methods, and tests.

Acceptance:

- Updating an artifact records old and new digests.
- Revision history can be retrieved by artifact id.
- Deletion is traceable through a change record.
- Change records are backend-neutral at the programmatic API boundary.

## P4.6 - Persist workflow run state

```task
id: KONT-WP-0004-T006
status: todo
priority: medium
state_hub_task_id: "d0a9e9d4-12eb-406b-b32c-5b45f931f18c"
```

Persist `OperationRun`, `WorkflowStep`, `InputBundle`, and `RunManifest`
records so orchestration can resume and inspect prior execution.

Output: repository methods and persistence tests for workflow records.

Acceptance:

- Run status transitions survive repository re-instantiation.
- Run manifests roundtrip with inputs, outputs, diagnostics, and timestamps.
- Artifact outputs can be linked to producing runs.

## P4.7 - Add context and phase-memory reference hooks

```task
id: KONT-WP-0004-T007
status: todo
priority: medium
state_hub_task_id: "965738a5-9538-45f6-98bb-7987aba62904"
```

Add lightweight persistence for context-package references and external memory
references without implementing memory lifecycle behavior.

Output: context reference model and tests.

Acceptance:

- Context packages can refer to artifacts, relationships, runs, and external
  memory records.
- External memory references are opaque and provenance-tagged.
- No retention, decay, compaction, activation planning, or preference-memory
  behavior is added to this repo.

## P4.8 - Add import/export and smoke verification

```task
id: KONT-WP-0004-T008
status: todo
priority: medium
state_hub_task_id: "ea7313ce-fb1f-49b1-b5da-66a036893a04"
```

Provide a deterministic import/export path for repository snapshots so early
users and agents can inspect or migrate local state.

Output: programmatic snapshot helpers and tests.

Acceptance:

- A repository snapshot can be exported and imported into a fresh backend.
- Imported data preserves ids, digests, metadata, relationships, revisions, and
  run links.
- Snapshot format does not become a replacement for the service API.

## Definition Of Done

- `python3 -m pytest` passes.
- Existing in-memory behavior remains compatible unless explicitly revised.
- SQLite-backed tests prove durable behavior across repository
  re-instantiation.
- Persistence docs explain what is durable now and what remains deferred.
- `docs/phase-memory-boundary.md` remains the boundary for memory-specific
  behavior.