generated from coulomb/repo-seed
System boundry and persistance workplan
This commit is contained in:
@@ -12,6 +12,7 @@ Start here:
|
||||
- `docs/stack-decision.md`
|
||||
- `docs/markitect-main-scope-assessment.md`
|
||||
- `docs/markitect-tool-reuse-boundary.md`
|
||||
- `docs/phase-memory-boundary.md`
|
||||
- `docs/system-layer-extraction-inventory.md`
|
||||
- `docs/system-layer-migration-backlog.md`
|
||||
- `docs/service-api-boundary.md`
|
||||
|
||||
107
docs/phase-memory-boundary.md
Normal file
107
docs/phase-memory-boundary.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# Phase Memory Boundary
|
||||
|
||||
Date: 2026-05-05
|
||||
|
||||
This note defines how `kontextual-engine` should relate to the sibling
|
||||
`phase-memory` project while closing the current persistence gap.
|
||||
|
||||
## Source Scope
|
||||
|
||||
`phase-memory` exists to provide profile-driven memory infrastructure for
|
||||
agentic systems. Its intent covers memory phases, memory graphs, event paths,
|
||||
stores, compaction, retention, service profiles, activation planning, and
|
||||
adapters for advanced memory backends.
|
||||
|
||||
`kontextual-engine` exists to provide an AI-first, headless knowledge and
|
||||
content engine. Its intent covers persistent knowledge artifacts, collections,
|
||||
relationships, ingestion, workflow state, query, service access, and agent-
|
||||
operable knowledge processes.
|
||||
|
||||
The projects overlap around durable state, graph-shaped records, context
|
||||
packages, provenance, and agent operation. They must stay separated by purpose.
|
||||
|
||||
## Ownership Boundary
|
||||
|
||||
`kontextual-engine` owns:
|
||||
|
||||
- Persistent knowledge collections, artifacts, relationships, and revisions.
|
||||
- Content identity, artifact metadata, source provenance, and lifecycle state.
|
||||
- Durable workflow run records and operation manifests.
|
||||
- Query and retrieval over knowledge artifacts and relationships.
|
||||
- Service and programmatic APIs for operating knowledge systems.
|
||||
- Adapter surfaces that can call `markitect-tool`, `phase-memory`, storage
|
||||
backends, and future workflow engines.
|
||||
|
||||
`phase-memory` owns:
|
||||
|
||||
- Ephemeral, fluid, stabilized, and rigid memory phase semantics.
|
||||
- Reasoning graphs, conversational paths, activation memory, identity memory,
|
||||
preference memory, and reviewed memory records.
|
||||
- Memory profiles with latency, token, byte, retention, decay, compaction,
|
||||
freshness, merge, conflict, and policy parameters.
|
||||
- Memory-specific lifecycle enforcement, including retention, deletion,
|
||||
stabilization, compaction, and summarization.
|
||||
- Memory retrieval planning and activation-package compilation.
|
||||
- Adapters to graph databases, vector stores, audit sinks, policy systems, and
|
||||
memory-specific backend infrastructure.
|
||||
|
||||
## Interop Boundary
|
||||
|
||||
`kontextual-engine` may persist memory-related artifacts when they are treated
|
||||
as knowledge assets, such as:
|
||||
|
||||
- reviewed decisions,
|
||||
- source-backed facts,
|
||||
- durable project context packages,
|
||||
- workflow outputs,
|
||||
- policy-visible provenance records,
|
||||
- exported memory graph slices.
|
||||
|
||||
It should not implement memory-specific behavior such as:
|
||||
|
||||
- conversational path branching and merge semantics,
|
||||
- memory decay and retention policy execution,
|
||||
- compaction or summarization strategies,
|
||||
- activation planning heuristics,
|
||||
- preference or identity memory governance,
|
||||
- graph/vector memory backend selection,
|
||||
- memory service profile validation.
|
||||
|
||||
Those behaviors belong to `phase-memory`. `kontextual-engine` should integrate
|
||||
with them through explicit adapters and stable exchange formats.
|
||||
|
||||
## Persistence Implication
|
||||
|
||||
The persistence gap in `kontextual-engine` should be closed as durable knowledge
|
||||
runtime storage, not as a general memory runtime.
|
||||
|
||||
The first durable backend should persist:
|
||||
|
||||
- collections,
|
||||
- artifacts,
|
||||
- artifact revisions,
|
||||
- relationships,
|
||||
- workflow runs,
|
||||
- run manifests,
|
||||
- change records,
|
||||
- queryable metadata,
|
||||
- context-package references.
|
||||
|
||||
It should not persist phase-memory internals beyond opaque references or
|
||||
exported artifacts unless an integration adapter explicitly maps them.
|
||||
|
||||
## Integration Shape
|
||||
|
||||
The preferred integration shape is:
|
||||
|
||||
1. Core `kontextual-engine` repository contracts remain independent.
|
||||
2. A future `phase-memory` adapter can read/write durable knowledge artifacts
|
||||
through those contracts.
|
||||
3. Context packages can move between the systems as inspectable payloads.
|
||||
4. Memory-specific policies stay in `phase-memory`; knowledge persistence
|
||||
policy stays in `kontextual-engine`.
|
||||
5. Cross-repo references should use stable identifiers, content digests, and
|
||||
provenance metadata rather than implicit filesystem coupling.
|
||||
|
||||
This keeps `kontextual-engine` focused on persistent, operable knowledge while
|
||||
leaving sophisticated agentic memory behavior to the project designed for it.
|
||||
226
workplans/KONT-WP-0004-durable-persistence.md
Normal file
226
workplans/KONT-WP-0004-durable-persistence.md
Normal file
@@ -0,0 +1,226 @@
|
||||
---
|
||||
id: KONT-WP-0004
|
||||
type: workplan
|
||||
title: "Durable Persistence Foundation"
|
||||
domain: markitect
|
||||
repo: kontextual-engine
|
||||
status: planned
|
||||
owner: codex
|
||||
topic_slug: markitect
|
||||
created: "2026-05-05"
|
||||
updated: "2026-05-05"
|
||||
state_hub_workstream_id: pending
|
||||
---
|
||||
|
||||
# KONT-WP-0004: Durable Persistence Foundation
|
||||
|
||||
## Purpose
|
||||
|
||||
Close the persistence gap identified after `KONT-WP-0003` by turning the current
|
||||
in-memory repository contract into a durable, local-first storage foundation for
|
||||
knowledge artifacts, collections, relationships, workflow state, and context
|
||||
references.
|
||||
|
||||
This workplan deliberately does not implement `phase-memory` behavior. It uses
|
||||
`docs/phase-memory-boundary.md` as the boundary: `kontextual-engine` persists
|
||||
durable knowledge runtime state; `phase-memory` owns agentic memory phases,
|
||||
profiles, compaction, retention, and activation planning.
|
||||
|
||||
## Persistence Scope
|
||||
|
||||
In scope:
|
||||
|
||||
- Durable storage for collections, artifacts, artifact revisions, and
|
||||
relationships.
|
||||
- Durable storage for workflow runs and run manifests.
|
||||
- Explicit update and delete behavior for artifacts and relationships.
|
||||
- Change records that make artifact evolution inspectable.
|
||||
- Query support for identifiers, names, digests, metadata, content text, and
|
||||
relationships.
|
||||
- Local-first SQLite backend with a repository interface that can later be
|
||||
backed by service, PostgreSQL, graph, or object-storage adapters.
|
||||
- Tests proving data survives repository re-instantiation.
|
||||
|
||||
Out of scope:
|
||||
|
||||
- Memory-phase lifecycle behavior from `phase-memory`.
|
||||
- Vector search, embedding storage, and memory activation planning.
|
||||
- Markdown parsing or markdown transformations; use `markitect-tool` adapters.
|
||||
- LLM provider execution; use future `llm-connect` adapters.
|
||||
- Remote multi-tenant deployment concerns beyond schema choices that do not
|
||||
block later migration.
|
||||
|
||||
## P4.1 - Finalize persistence boundary and ADR
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T001
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Promote the storage decision from deferred to explicit: local-first SQLite for
|
||||
the first durable backend, wrapped by repository contracts. Decide whether the
|
||||
implementation uses direct `sqlite3` or SQLAlchemy for this slice.
|
||||
|
||||
Output: update `docs/stack-decision.md` or add an ADR under `docs/`.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- The backend choice is explicit and justified.
|
||||
- The decision references `docs/phase-memory-boundary.md`.
|
||||
- Future service-backed storage remains possible.
|
||||
|
||||
## P4.2 - Complete repository contract semantics
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T002
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Extend `KnowledgeRepository` from create/list/get into a durable lifecycle
|
||||
contract. Define update, delete, revision, and transaction semantics without
|
||||
binding callers to a specific backend.
|
||||
|
||||
Output: `src/kontextual_engine/storage.py` and focused tests.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Artifact update produces explicit revision/change semantics.
|
||||
- Artifact delete behavior is defined for relationships and query results.
|
||||
- Duplicate-name and referential-integrity behavior remains deterministic.
|
||||
- Existing in-memory tests continue to pass.
|
||||
|
||||
## P4.3 - Design durable schema and migrations
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T003
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Create the first durable schema for collections, artifacts, revisions,
|
||||
relationships, workflow runs, run manifests, and change records.
|
||||
|
||||
Output: schema/migration files under `src/kontextual_engine/storage/` or an
|
||||
equivalent package-owned location.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Schema stores content digest, artifact type, size, metadata, timestamps, and
|
||||
provenance.
|
||||
- Relationships enforce valid source and target artifacts.
|
||||
- JSON metadata is preserved roundtrip.
|
||||
- Migrations can initialize an empty local database deterministically.
|
||||
|
||||
## P4.4 - Implement SQLite repository backend
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T004
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Implement `SQLiteKnowledgeRepository` behind the same repository contract used
|
||||
by the in-memory backend.
|
||||
|
||||
Output: durable repository implementation and tests.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Collections, artifacts, relationships, and metadata survive closing and
|
||||
reopening the repository.
|
||||
- Query behavior matches the in-memory repository for supported filters.
|
||||
- Tests cover duplicate artifact names, missing relationship endpoints, and
|
||||
deterministic ordering.
|
||||
- No markdown or memory-runtime logic is introduced.
|
||||
|
||||
## P4.5 - Persist artifact evolution
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T005
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Add artifact revision and change-record support so persistent knowledge can be
|
||||
versioned and audited over time.
|
||||
|
||||
Output: model additions, repository methods, and tests.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Updating an artifact records old and new digests.
|
||||
- Revision history can be retrieved by artifact id.
|
||||
- Deletion is traceable through a change record.
|
||||
- Change records are backend-neutral at the programmatic API boundary.
|
||||
|
||||
## P4.6 - Persist workflow run state
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T006
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Persist `OperationRun`, `WorkflowStep`, `InputBundle`, and `RunManifest`
|
||||
records so orchestration can resume and inspect prior execution.
|
||||
|
||||
Output: repository methods and persistence tests for workflow records.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Run status transitions survive repository re-instantiation.
|
||||
- Run manifests roundtrip with inputs, outputs, diagnostics, and timestamps.
|
||||
- Artifact outputs can be linked to producing runs.
|
||||
|
||||
## P4.7 - Add context and phase-memory reference hooks
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T007
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Add lightweight persistence for context-package references and external memory
|
||||
references without implementing memory lifecycle behavior.
|
||||
|
||||
Output: context reference model and tests.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- Context packages can refer to artifacts, relationships, runs, and external
|
||||
memory records.
|
||||
- External memory references are opaque and provenance-tagged.
|
||||
- No retention, decay, compaction, activation planning, or preference-memory
|
||||
behavior is added to this repo.
|
||||
|
||||
## P4.8 - Add import/export and smoke verification
|
||||
|
||||
```task
|
||||
id: KONT-WP-0004-T008
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Provide a deterministic import/export path for repository snapshots so early
|
||||
users and agents can inspect or migrate local state.
|
||||
|
||||
Output: programmatic snapshot helpers and tests.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- A repository snapshot can be exported and imported into a fresh backend.
|
||||
- Imported data preserves ids, digests, metadata, relationships, revisions, and
|
||||
run links.
|
||||
- Snapshot format does not become a replacement for the service API.
|
||||
|
||||
## Definition Of Done
|
||||
|
||||
- `python3 -m pytest` passes.
|
||||
- Existing in-memory behavior remains compatible unless explicitly revised.
|
||||
- SQLite-backed tests prove durable behavior across repository
|
||||
re-instantiation.
|
||||
- Persistence docs explain what is durable now and what remains deferred.
|
||||
- `docs/phase-memory-boundary.md` remains the boundary for memory-specific
|
||||
behavior.
|
||||
Reference in New Issue
Block a user