coulomb/kontextual-engine
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

State-hub connect and initial workplans

Browse Source
This commit is contained in:
Bernd Worsch
2026-05-03 20:43:50 +02:00
parent b40ac33d11
commit 50d6866c64
10 changed files with 675 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.custodian-brief.md
View File

@@ -2,7 +2,7 @@
# Custodian Brief — kontextual-engine
**Domain:** markitect
**Last synced:** 2026-05-03 18:02 UTC
**Last synced:** 2026-05-03 18:05 UTC
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
## Active Workstreams
@@ -31,10 +31,9 @@ Progress: 1/6 done | workstream_id: `e46d0962-7451-4b6c-b39f-461e35ba6a76`
- · S2.6 - Produce migration backlog `54a7e7a7`
### Repository Foundation And State Hub Integration
Progress: 3/5 done | workstream_id: `52ff9c49-edcf-4150-8895-c6b31f5aa075`
Progress: 4/5 done | workstream_id: `52ff9c49-edcf-4150-8895-c6b31f5aa075`
**Open tasks:**
- · F1.4 - Run State Hub consistency sync `6fefec5f`
- · F1.5 - Decide initial implementation stack `616e0476`
---

84
CLAUDE.md Normal file
View File

@@ -0,0 +1,84 @@
# kontextual-engine Agent Guide
This repository is tracked in the Custodian State Hub as:
- Domain: `markitect`
- Repo slug: `kontextual-engine`
- Topic ID: `5571d954-0d30-4950-980d-7bcaaad8e3e2`
- Workplan prefix: `KONT-WP`
At session start, orient from:
1. `.custodian-brief.md`
2. `INTENT.md`
3. `wiki/ProductRequirementsDocument.md`
4. `wiki/FunctionalRequirementsSpecification.md`
5. `docs/markitect-main-scope-assessment.md`
6. Active files in `workplans/`
## State Hub
This repo is registered with State Hub through the local Custodian service.
State Hub is an index/cache for coordination state; authoritative work items
live in this repository as Markdown workplans.
Local API:
```bash
curl -s http://127.0.0.1:8000/state/health
curl -s http://127.0.0.1:8000/repos/kontextual-engine | python3 -m json.tool
```
At session close, record notable progress:
```bash
curl -s -X POST http://127.0.0.1:8000/progress/ \
-H "Content-Type: application/json" \
-d '{"summary":"what changed","event_type":"note","author":"codex"}'
```
After workplan file changes, ask the custodian operator to run from
`/home/worsch/the-custodian/state-hub`:
```bash
make fix-consistency REPO=kontextual-engine
```
## Boundary
`kontextual-engine` is the system-layer successor to the runtime/platform parts
of `markitect-main`.
It owns:
- Persistent knowledge artifacts and collections.
- Artifact metadata and relationships.
- Ingestion and normalization interfaces.
- Query, retrieval, and composition service contracts.
- Workflow orchestration and operation tracking.
- Agent-operable context and action surfaces.
- Integration boundaries to `markitect-tool` and `llm-connect`.
It does not own:
- Markdown syntax primitives or document-level schema tooling.
- End-user visual UI applications or rendering plugins.
- Domain-specific knowledge content.
- Provider-specific LLM adapters.
- Legacy finance, issue, profile, release, or project-management utilities.
## Development Posture
Prefer clean reimplementation around the new PRD/FRS. Use `markitect-main` as
reference material for behavior, tests, and domain vocabulary, not as an
architecture to copy wholesale.
## Workplans
Workplans live in `workplans/` and follow the Custodian ADR-001 convention:
- Frontmatter declares `type: workplan`, `domain: markitect`, `repo:
kontextual-engine`, and `owner: codex`.
- Tasks are embedded as headed sections with fenced `task` blocks.
- State Hub may index these files, but the files remain authoritative.

14
README.md
View File

@@ -1,3 +1,13 @@
# repo-seed
# kontextual-engine
A git repository template to bootstrap coulomb projects from.
AI-first, headless knowledge engine for persistent, operable structured
knowledge.
Start here:
- `INTENT.md`
- `wiki/ProductRequirementsDocument.md`
- `wiki/FunctionalRequirementsSpecification.md`
- `SCOPE.md`
- `docs/markitect-main-scope-assessment.md`
- `workplans/`

147
SCOPE.md Normal file
View File

@@ -0,0 +1,147 @@
# SCOPE
> This file helps agents and humans quickly understand what this repository is
> about, when it is relevant, and when it is not.
---
## One-liner
AI-first, headless knowledge engine that makes structured knowledge persistent,
queryable, orchestratable, and operable across formats.
---
## Core Idea
`kontextual-engine` is the system-layer successor to the platform portions of
`markitect-main`. It should preserve the useful ideas around infospaces,
knowledge artifacts, relationships, retrieval, workflow execution, and agent
context, while avoiding the old repo's mixed ownership of markdown primitives,
UI, provider integrations, and project/domain content.
The engine owns the runtime contract for persistent knowledge systems. Lower
level syntax operations belong in `markitect-tool`; concrete domain workspaces
belong in `infospace-bench`.
---
## In Scope
- Persistent storage and lifecycle management for knowledge artifacts.
- Collections/domains, metadata, and relationships between artifacts.
- Multi-format ingestion interfaces and normalized internal representations.
- Query, retrieval, indexing, and composition APIs.
- Workflow orchestration for transformation, generation, and analysis.
- Agent-facing context continuity and operation surfaces.
- Integration adapters for lower-layer tools such as `markitect-tool`.
- Structured errors, deterministic behavior where applicable, and auditable
state transitions.
---
## Out of Scope
- Low-level markdown parsing, schema primitives, document transforms, or CLI
tooling that belongs in `markitect-tool`.
- Visual UI applications, rendering plugins, or WYSIWYG editing.
- Domain-specific infospace content or benchmark corpora that belong in
`infospace-bench`.
- Direct ownership of LLM provider adapters; use `llm-connect` or equivalent.
- Finance, issue tracking, profile management, release tooling, and other
legacy `markitect-main` utilities unrelated to the engine contract.
- A CLI-first product posture; any CLI should remain an administrative or
development convenience over service/programmatic APIs.
---
## Relevant When
- A project needs durable knowledge artifacts instead of one-off file parsing.
- Agents need stable context and retrievable state across sessions.
- Workflows must ingest, normalize, transform, compose, and query knowledge.
- Multiple formats and external tooling need a common runtime layer.
- A higher-level application needs a headless knowledge service.
---
## Not Relevant When
- The task is only markdown syntax manipulation or schema validation.
- The primary need is an end-user visual application.
- The work is domain-specific corpus curation without runtime needs.
- Provider-specific LLM client behavior is the main concern.
---
## Current State
- Status: scoping / foundation.
- Implementation: documentation and workplans only.
- Stability: evolving.
- Usage: successor planning for the in-scope system-layer parts of
`markitect-main`.
---
## How It Fits
- Upstream dependencies: `markitect-tool` for syntax-layer primitives,
`llm-connect` for provider-neutral LLM access, storage backends to be chosen.
- Downstream consumers: `infospace-bench`, future knowledge services, agents,
and automation systems.
- Often used with: State Hub for planning/coordination, markitect ecosystem
repos for adjacent responsibilities.
---
## Terminology
- Preferred terms: knowledge artifact, collection, relationship, ingestion,
normalization, workflow, context, operation.
- Also known as: Kontextual Engine, knowledge runtime, headless knowledge
engine.
- Potentially confusing terms: "infospace" is a conceptual collection pattern
inherited from `markitect-main`, not necessarily a project directory or a
UI-facing workspace.
---
## Related / Overlapping Repositories
- `markitect-main` — legacy mixed platform; source for candidate behavior and
tests, not the target architecture.
- `markitect-tool` — syntax layer for markdown and structured document
primitives.
- `infospace-bench` — application/project layer for concrete knowledge spaces.
- `llm-connect` — LLM provider abstraction that this repo may call but should
not replace.
- `the-custodian/state-hub` — coordination and repo/workplan index.
---
## Getting Oriented
- Start with: `INTENT.md`, `wiki/ProductRequirementsDocument.md`,
`wiki/FunctionalRequirementsSpecification.md`.
- Key files / directories: `docs/`, `workplans/`, `SCOPE.md`, `CLAUDE.md`.
- Entry points: none yet; implementation starts from the workplans.
---
## Provided Capabilities
```capability
type: service
title: Persistent knowledge runtime
description: Provides the planned system layer for storing, querying, transforming, and orchestrating structured knowledge artifacts across formats.
keywords: [knowledge, runtime, persistence, orchestration, retrieval]
```
```capability
type: automation
title: Agent-operable knowledge workflows
description: Provides planned APIs and workflow surfaces that let agents access context, trigger transformations, and operate over durable knowledge state.
keywords: [agent, workflow, context, automation, knowledge]
```

73
docs/markitect-main-scope-assessment.md Normal file
View File

@@ -0,0 +1,73 @@
# markitect-main Scope Assessment For kontextual-engine
This assessment compares `/home/worsch/markitect-main` with the
`kontextual-engine` PRD, FRS, and intent documents.
## Summary
`markitect-main` contains the seed ideas for this repo, but they are mixed with
syntax tooling, UI, plugins, provider adapters, finance, issue tracking, and
project-specific utilities. `kontextual-engine` should migrate concepts and
tests selectively, then reimplement the runtime contract as a headless service
and programmatic API.
The most important inheritance is not old module structure. It is the concept
of a durable infospace-like knowledge environment with typed artifacts,
relationships, evaluation/composition workflows, and agent-operable context.
## In-Scope Candidates
| FRS area | markitect-main evidence | Recommendation |
| --- | --- | --- |
| FR-001 to FR-004 knowledge persistence | `infrastructure/repositories/`, `infrastructure/connection_manager.py`, `docs/WORKSPACE_AND_DATABASES.md`, migrations under `migrations/prompts/` | Reimplement a storage abstraction early. Reuse lessons from filesystem/SQLite split, but do not inherit workspace-local assumptions blindly. |
| FR-010 to FR-011 organization and relationships | `markitect/infospace/models.py`, `relation_models.py`, `relation_parser.py`, `graph_export.py`, examples under `examples/infospace-with-history/` | Migrate vocabulary and relationship tests where generic. Keep domain-specific example content as fixtures only. |
| FR-020 to FR-021 ingestion and normalization | `markitect/infospace/pipeline.py`, `entity_parser.py`, `classifier.py`, packaging/proxy docs, asset/document managers | Define ingestion interfaces that call external format tools. Markdown-specific parsing should route through `markitect-tool`. |
| FR-030 to FR-031 query and retrieval | `markitect/infospace/evaluate.py`, `evaluation_io.py`, `classification_io.py`, search plugin docs | Reimplement query as service/programmatic contract over persisted artifacts, metadata, and relationships. Avoid CLI/search-plugin coupling. |
| FR-040 to FR-041 transformation and composition | `markitect/infospace/composition.py`, `markitect/packaging/transclusion/`, `docs/composition-guide.md`, prompt dependency resolution roadmap | Keep composition/workflow ideas. Delegate document-level transforms to `markitect-tool`; engine tracks operation state and derived artifacts. |
| FR-050 to FR-052 workflow orchestration | `roadmap/prompt-dependency-resolution/`, `migrations/prompts/`, quality tables, run manifests, batch processor | Reimplement workflow model around explicit runs, steps, dependencies, inputs, outputs, and structured errors. |
| FR-060 to FR-061 AI interaction/context | `markitect/helper/knowledge.py`, `markitect/llm/`, evaluation/classification modules | Preserve agent use cases and context needs. Use `llm-connect` for providers and keep prompts/workflow state auditable. |
| FR-070 to FR-071 external tooling | `markitect/plugins/`, capabilities architecture, `pyproject.toml` file dependencies | Build adapter boundaries, not embedded capability code. First adapter should likely target `markitect-tool`. |
| FR-080 to FR-091 API and errors | `markitect/query_paradigms/`, production error handling, GraphQL docs | Define stable Python API first, then service API. Structured errors should be part of the first implementation slice. |
## Out Of Scope For kontextual-engine
- `markitect/core/`, `markitect/schema/`, `markitect/explode_variants/`, and
document transform primitives: move/reimplement in `markitect-tool`.
- `src/*.js`, `testdrive-jsui`, rendering plugins, static assets, browser
integration: not headless engine scope.
- `markitect/llm/*` provider implementations: use `llm-connect`.
- `markitect/finance/`, `issue_tracker/`, `profile/`, `tddai`, Gitea tooling,
and release-management helpers: unrelated product scope.
- Domain content from examples: useful as fixtures only when testing generic
engine behavior.
- GraphQL as a default interface: old docs are evidence of API needs, not a
commitment to GraphQL.
## Migration Principles
1. Start from the `kontextual-engine` PRD/FRS, not legacy package names.
2. Migrate tests and fixtures before code when behavior is clearly in scope.
3. Treat legacy code as reference material unless a module is already isolated
and aligned with the new runtime boundary.
4. Keep syntax, provider, UI, and domain layers outside this repo.
5. Make persistence and operation state explicit before adding AI workflows.
6. Prefer API/programmatic contracts first; CLI can be administrative later.
## Initial Architecture Target
```text
kontextual_engine/
artifacts/ artifact model, metadata, lifecycle operations
collections/ grouping, domain/collection membership, relationships
storage/ repository interfaces and backends
ingest/ format-agnostic ingestion adapters and normalization
query/ retrieval over content, metadata, and relationships
workflows/ runs, steps, dependencies, derived outputs
context/ agent context assembly and operation surfaces
integrations/ markitect-tool, llm-connect, storage adapters
api/ programmatic and service-facing contracts
```
The first implementation workplan should validate this shape against migrated
tests before committing to a framework or storage backend.

49
docs/state-hub-integration.md Normal file
View File

@@ -0,0 +1,49 @@
# State Hub Integration
## Registration
`kontextual-engine` belongs to the State Hub `markitect` domain.
Expected repo record:
- repo slug: `kontextual-engine`
- domain: `markitect`
- topic id: `5571d954-0d30-4950-980d-7bcaaad8e3e2`
- local path: `/home/worsch/kontextual-engine`
- primary host path: `/home/worsch/kontextual-engine`
Verify:
```bash
curl -s http://127.0.0.1:8000/repos/kontextual-engine | python3 -m json.tool
```
## Source Of Truth
Workplans in this repository are authoritative. State Hub is the operational
index/cache used for cross-repo coordination, dashboards, and task dispatch.
When workplans change, sync from the State Hub repo:
```bash
cd /home/worsch/the-custodian/state-hub
make fix-consistency REPO=kontextual-engine
```
## Integration Tier
Core integration requires:
- repo registered in State Hub
- `SCOPE.md` present
- `CLAUDE.md` present
- workplans follow ADR-001 file format
- `tpsc.yaml` declares external services, even if empty
Full integration later needs:
- active repo goals in State Hub
- ingested capabilities from `SCOPE.md`
- clean consistency check
- SBOM once an implementation stack and lockfile exist

5
tpsc.yaml Normal file
View File

@@ -0,0 +1,5 @@
# tpsc.yaml - Third-Party Services Catalog declarations for kontextual-engine
# Ingest: cd /home/worsch/the-custodian/state-hub && make ingest-tpsc REPO=kontextual-engine
services: []

86
workplans/KONT-WP-0001-repo-foundation.md Normal file
View File

@@ -0,0 +1,86 @@
---
id: KONT-WP-0001
type: workplan
title: "Repository Foundation And State Hub Integration"
domain: markitect
repo: kontextual-engine
status: active
owner: codex
topic_slug: markitect
created: "2026-05-03"
updated: "2026-05-03"
state_hub_workstream_id: "52ff9c49-edcf-4150-8895-c6b31f5aa075"
---
# KONT-WP-0001: Repository Foundation And State Hub Integration
## Purpose
Establish `kontextual-engine` as a properly scoped, State Hub connected
successor repo before implementation begins.
## F1.1 - Register repo with State Hub
```task
id: KONT-WP-0001-T001
status: done
priority: high
state_hub_task_id: "f11fbf85-6841-464d-b076-73811a161a8e"
```
Create or verify the State Hub repo record for `kontextual-engine` in the
`markitect` domain with the local host path `/home/worsch/kontextual-engine`.
## F1.2 - Add State Hub repo integration files
```task
id: KONT-WP-0001-T002
status: done
priority: high
state_hub_task_id: "7a56ea42-1690-4bd6-9640-29c3863717a6"
```
Add `CLAUDE.md`, `.custodian-brief.md`, `SCOPE.md`, `tpsc.yaml`, and
`docs/state-hub-integration.md` so future sessions can orient consistently.
## F1.3 - Establish authoritative workplan set
```task
id: KONT-WP-0001-T003
status: done
priority: high
state_hub_task_id: "950f43a4-e815-42b5-a167-eb48f0706d4d"
```
Create initial workplans for foundation, legacy scope extraction, and first
implementation slices using the ADR-001 file-backed convention.
## F1.4 - Run State Hub consistency sync
```task
id: KONT-WP-0001-T004
status: done
priority: high
state_hub_task_id: "6fefec5f-eff8-402a-ad87-bf4821468a3d"
```
From `/home/worsch/the-custodian/state-hub`, run:
```bash
make fix-consistency REPO=kontextual-engine
```
Verify the repo has no consistency failures after State Hub indexes the
workplan files.
## F1.5 - Decide initial implementation stack
```task
id: KONT-WP-0001-T005
status: todo
priority: medium
state_hub_task_id: "616e0476-7b65-4079-b545-13e0fc9436f7"
```
Choose the first implementation stack, dependency manager, test command, and
service framework. Record the decision before adding application code.

97
workplans/KONT-WP-0002-markitect-main-system-scope.md Normal file
View File

@@ -0,0 +1,97 @@
---
id: KONT-WP-0002
type: workplan
title: "markitect-main System-Layer Extraction"
domain: markitect
repo: kontextual-engine
status: active
owner: codex
topic_slug: markitect
created: "2026-05-03"
updated: "2026-05-03"
state_hub_workstream_id: "e46d0962-7451-4b6c-b39f-461e35ba6a76"
---
# KONT-WP-0002: markitect-main System-Layer Extraction
## Purpose
Compare `markitect-main` against the `kontextual-engine` PRD/FRS and extract
only the behavior, tests, and concepts that belong in the headless knowledge
runtime layer.
## S2.1 - Complete first-pass scope assessment
```task
id: KONT-WP-0002-T001
status: done
priority: high
state_hub_task_id: "1d7609a6-605d-425b-ab51-9a77a82ff73d"
```
Document the first-pass migration/reimplementation assessment in
`docs/markitect-main-scope-assessment.md`.
## S2.2 - Inventory persistence and repository code
```task
id: KONT-WP-0002-T002
status: todo
priority: high
state_hub_task_id: "86a1bf90-db72-44a0-a5ad-6374e6de8454"
```
Review legacy filesystem/SQLite repositories, workspace database docs, prompt
run migrations, and related tests. Classify each item as migrate test,
reimplement concept, defer, or out of scope.
## S2.3 - Inventory infospace models and relationships
```task
id: KONT-WP-0002-T003
status: todo
priority: high
state_hub_task_id: "8b88b3fa-a905-44aa-a25f-993cc9d50f2c"
```
Review `markitect/infospace/` models, relationship parsing, graph export, and
example fixtures. Extract generic artifact, collection, relationship, and
evaluation concepts without importing project-layer assumptions.
## S2.4 - Inventory orchestration and run-manifest material
```task
id: KONT-WP-0002-T004
status: todo
priority: high
state_hub_task_id: "1f15f603-4f86-41f8-8a24-95c0e9c825f7"
```
Review prompt dependency resolution roadmap, run manifests, quality tables,
batch processor behavior, and workflow-related migrations. Produce a candidate
workflow model for engine implementation.
## S2.5 - Inventory API and query experiments
```task
id: KONT-WP-0002-T005
status: todo
priority: medium
state_hub_task_id: "0a1e5a4b-f64d-4228-8f0f-e174475253da"
```
Review query paradigms, GraphQL docs, search/indexing experiments, and error
handling. Decide which API/query ideas deserve new tests or design notes.
## S2.6 - Produce migration backlog
```task
id: KONT-WP-0002-T006
status: todo
priority: high
state_hub_task_id: "54a7e7a7-bf26-4f71-a8a5-9da48f5018c2"
```
Create a structured backlog of candidate tests, fixtures, modules, and
behaviors for `KONT-WP-0003`, grouped by FRS section and migration strategy.

120
workplans/KONT-WP-0003-engine-implementation.md Normal file
View File

@@ -0,0 +1,120 @@
---
id: KONT-WP-0003
type: workplan
title: "Headless Knowledge Engine Implementation"
domain: markitect
repo: kontextual-engine
status: active
owner: codex
topic_slug: markitect
created: "2026-05-03"
updated: "2026-05-03"
state_hub_workstream_id: "0fd08391-e8c9-4f1b-ace4-06439f958e88"
---
# KONT-WP-0003: Headless Knowledge Engine Implementation
## Purpose
Build the first usable `kontextual-engine` runtime around the PRD/FRS contract:
persistent knowledge artifacts, collections, relationships, ingestion,
retrieval, workflows, and agent-operable context.
## I3.1 - Scaffold package and test harness
```task
id: KONT-WP-0003-T001
status: todo
priority: high
state_hub_task_id: "3d3400a5-63ee-4c64-8bfb-fd3caa1ce787"
```
Create the initial Python package, dependency metadata, test harness, and CI
commands after the stack decision in `KONT-WP-0001`.
## I3.2 - Implement artifact and collection model
```task
id: KONT-WP-0003-T002
status: todo
priority: high
state_hub_task_id: "638ce4d0-b96b-4d8f-8bd9-4c77acad0a59"
```
Implement core models for artifacts, metadata, collections, domains, and
relationships. Cover FR-001 through FR-011 with focused unit tests.
## I3.3 - Implement storage abstraction
```task
id: KONT-WP-0003-T003
status: todo
priority: high
state_hub_task_id: "41c84f25-96b0-40eb-b2b4-834938cf9bb9"
```
Define repository interfaces and the first backend. Start with the simplest
backend that can support deterministic tests, then document the path to SQLite
or service-backed persistence.
## I3.4 - Implement ingestion and normalization interfaces
```task
id: KONT-WP-0003-T004
status: todo
priority: high
state_hub_task_id: "0be5fff9-f4a2-4aee-86b9-b84feef477f7"
```
Define format-agnostic ingestion contracts. Add a markdown adapter boundary
that can later call `markitect-tool`, without embedding markdown primitives in
this repo.
## I3.5 - Implement query and retrieval API
```task
id: KONT-WP-0003-T005
status: todo
priority: high
state_hub_task_id: "c8841fab-6dd1-47e1-a794-798bef1663e7"
```
Provide programmatic query over identifiers, metadata, content references, and
relationships. Cover FR-030 and FR-031 with tests.
## I3.6 - Implement workflow run model
```task
id: KONT-WP-0003-T006
status: todo
priority: high
state_hub_task_id: "0d62f47c-339a-4d86-9ab0-737bd4ab553c"
```
Represent workflow runs, steps, dependencies, inputs, outputs, structured
errors, and derived artifacts. Cover FR-040 through FR-052.
## I3.7 - Implement agent context surface
```task
id: KONT-WP-0003-T007
status: todo
priority: medium
state_hub_task_id: "96689817-e8e9-42e4-9af3-f03021e3ae4e"
```
Provide a provider-neutral context assembly API so agents can inspect
knowledge state, trigger operations, and receive traceable results.
## I3.8 - Define service API boundary
```task
id: KONT-WP-0003-T008
status: todo
priority: medium
state_hub_task_id: "0f4748f9-934a-438c-ac2d-e9796ad98818"
```
Decide and document the first HTTP/RPC service boundary after the programmatic
API is stable enough to avoid framework-driven architecture.