generated from coulomb/repo-seed
Reestablishing intent based goals and workplans
This commit is contained in:
224
docs/markitect-tool-and-infospace-replacement-assessment.md
Normal file
224
docs/markitect-tool-and-infospace-replacement-assessment.md
Normal file
@@ -0,0 +1,224 @@
|
||||
# Markitect Tool And Infospace Replacement Assessment
|
||||
|
||||
Date: 2026-05-14
|
||||
|
||||
## Questions
|
||||
|
||||
1. Does `infospace-bench` use `markitect-tool`?
|
||||
2. Can `infospace-bench` replace the infospace features that were present in
|
||||
`markitect-project` / `markitect-main`?
|
||||
|
||||
## Short Answer
|
||||
|
||||
`infospace-bench` does **not** currently use `markitect-tool`.
|
||||
|
||||
`infospace-bench` can replace the **foundation** of the old infospace feature
|
||||
set, but it cannot yet replace the full `markitect-project` infospace surface.
|
||||
It has a clean file-backed lifecycle, artifact manifest, evaluation result
|
||||
models, basic deterministic collection checks, viability thresholding,
|
||||
relationship summaries, and a small reference pilot. The legacy project still
|
||||
has substantially more mature Markdown parsing, entity extraction, schema
|
||||
validation, LLM pipeline execution, classification, metrics history, relation
|
||||
parsing, graph export, and CLI views.
|
||||
|
||||
## Current `infospace-bench` State
|
||||
|
||||
Implemented and tested:
|
||||
|
||||
- `create_infospace()`, `load_infospace()`, `add_artifact()`
|
||||
- `infospace.yaml` plus `artifacts/index.yaml`
|
||||
- JSON CLI commands: `create`, `inspect`, `add-artifact`, `export`
|
||||
- Structured `InfospaceError`
|
||||
- Evaluation objects: `ScoreEntry`, `EntityEvaluation`, `MetricValue`,
|
||||
`EvaluationSnapshot`, snapshot diffs
|
||||
- Baseline collection metrics:
|
||||
- `redundancy_ratio`
|
||||
- `coverage_ratio`
|
||||
- `coherence_components`
|
||||
- `consistency_cycles`
|
||||
- `granularity_entropy`
|
||||
- Viability threshold checks
|
||||
- Relationship summaries and Mermaid export
|
||||
- `infospaces/bootstrap-pilot`
|
||||
|
||||
Current test status: `14 passed`.
|
||||
|
||||
Current dependency declaration:
|
||||
|
||||
```toml
|
||||
dependencies = ["PyYAML>=6"]
|
||||
```
|
||||
|
||||
There is no import of `markitect_tool`, no package dependency on
|
||||
`markitect-tool`, and no adapter layer around its APIs.
|
||||
|
||||
## Current `markitect-tool` Relevance
|
||||
|
||||
`markitect-tool` is the syntax-layer successor to the Markdown tooling from
|
||||
`markitect-main`. It already exposes several surfaces that `infospace-bench`
|
||||
should use instead of reimplementing:
|
||||
|
||||
- Markdown parsing:
|
||||
- `parse_markdown()`
|
||||
- `parse_markdown_file()`
|
||||
- `Document`, `Heading`, `Section`, `ContentBlock`
|
||||
- Schema validation:
|
||||
- `validate_markdown_file()`
|
||||
- `validate_document()`
|
||||
- `load_schema_file()`
|
||||
- Contracts and diagnostics:
|
||||
- document contracts
|
||||
- validation diagnostics
|
||||
- metric collection
|
||||
- Query, cache, reference, processor, workflow, and generation primitives
|
||||
|
||||
This matches the intended boundary in `INTENT.md`: `markitect-tool` makes
|
||||
Markdown structured and manipulable; `infospace-bench` makes applied knowledge
|
||||
spaces concrete and meaningful.
|
||||
|
||||
## Legacy `markitect-project` Infospace Surface
|
||||
|
||||
The legacy infospace package in `markitect-main/markitect/infospace/` includes:
|
||||
|
||||
- Entity metadata parsing from Markdown entity files
|
||||
- Entity schema and compliance validation
|
||||
- Infospace config loading/saving, including schema registry, pipeline config,
|
||||
competency questions, directories, and viability thresholds
|
||||
- Per-entity LLM evaluation prompt construction, response parsing, and batch
|
||||
execution
|
||||
- Evaluation file I/O, snapshot history, and history diffing
|
||||
- Collection checks for redundancy, coverage, coherence, consistency, and
|
||||
granularity
|
||||
- Metrics file writing and history tracking
|
||||
- Classification models and classification summaries
|
||||
- Relation triplet parsing
|
||||
- Entity/relation graph construction and export
|
||||
- Source processing pipeline over `infospace.yaml` stages
|
||||
- CLI commands including status, entities, chapters, evaluate, eval-summary,
|
||||
relations, classify, classify-summary, classify-links, viability, check,
|
||||
history, history-diff, bind-discipline, process, stale-mappings, and graph
|
||||
- The large `examples/infospace-with-history` reference experiment
|
||||
|
||||
## Capability Comparison
|
||||
|
||||
| Capability | `infospace-bench` today | Legacy `markitect-project` | Replacement status |
|
||||
| --- | --- | --- | --- |
|
||||
| File-backed infospace layout | Yes | Partial/different | Replaceable |
|
||||
| Config model | Basic | Rich | Partial |
|
||||
| Artifact manifest | Yes | Entity/output directories | New baseline, not equivalent |
|
||||
| Entity Markdown parsing | No | Yes | Missing |
|
||||
| Markdown syntax parsing via `markitect-tool` | No | Legacy `markitect.core` | Missing integration |
|
||||
| Schema validation | No | Yes | Missing; should use `markitect-tool` |
|
||||
| Structured errors | Yes | Mixed CLI exits/exceptions | Improved baseline |
|
||||
| Evaluation result models | Yes | Yes | Mostly replaceable |
|
||||
| Evaluation I/O/history | No | Yes | Missing |
|
||||
| LLM evaluation pipeline | No | Yes | Missing; should route through lower-layer adapters |
|
||||
| Collection metrics | Basic deterministic | Richer entity/graph based checks | Partial |
|
||||
| Viability thresholds | Yes | Yes | Basic replacement |
|
||||
| Relationship summary | Basic manifest edges | Relation files, classifications, graph export | Partial |
|
||||
| Mermaid graph export | Basic | Richer typed graph export | Partial |
|
||||
| Classification | No | Yes | Missing |
|
||||
| Source processing pipeline | No | Yes | Missing |
|
||||
| Reference pilot | Small bootstrap pilot | Large Wealth of Nations/VSM pilot | Partial |
|
||||
| CLI surface | Minimal | Broad | Not replaceable yet |
|
||||
|
||||
## Assessment
|
||||
|
||||
`infospace-bench` is correctly positioned as the successor, but it is still in
|
||||
the early replacement phase. It is valuable because it has cleaner boundaries
|
||||
than `markitect-project`: it does not carry old Markdown parser code, prompt
|
||||
runtime infrastructure, asset management, GraphQL experiments, or platform
|
||||
concerns.
|
||||
|
||||
That boundary clarity is also why it is not a full replacement yet. The missing
|
||||
work is not just code volume; it is the application-layer integration between
|
||||
real Markdown artifacts, schemas/contracts, workflow execution, evaluation
|
||||
history, and graph inspection.
|
||||
|
||||
## Recommended Replacement Path
|
||||
|
||||
### Phase 1: Integrate `markitect-tool`
|
||||
|
||||
Add an explicit dependency on `markitect-tool` and introduce a thin adapter
|
||||
module in `infospace-bench`, for example:
|
||||
|
||||
```text
|
||||
src/infospace_bench/markdown_adapter.py
|
||||
```
|
||||
|
||||
The adapter should expose only application-level operations needed by
|
||||
infospaces:
|
||||
|
||||
- parse artifact Markdown into a structured document
|
||||
- extract heading-led sections
|
||||
- validate an artifact against a configured schema/contract
|
||||
- return diagnostics in `InfospaceError`-compatible shapes
|
||||
|
||||
Do not expose the whole `markitect-tool` API through `infospace-bench`.
|
||||
|
||||
### Phase 2: Replace Legacy Entity Parsing And Validation
|
||||
|
||||
Reimplement the useful parts of `markitect/infospace/entity_parser.py`,
|
||||
`schema.py`, and `validator.py` on top of `markitect-tool` parsed documents and
|
||||
schema/contract validation.
|
||||
|
||||
Acceptance target:
|
||||
|
||||
- migrate a small entity fixture from `examples/infospace-with-history`
|
||||
- parse entity metadata
|
||||
- validate required sections
|
||||
- emit structured diagnostics
|
||||
|
||||
### Phase 3: Add Evaluation History I/O
|
||||
|
||||
Promote the current evaluation dataclasses into persistent artifacts:
|
||||
|
||||
- write/read entity evaluation files
|
||||
- write/read snapshots
|
||||
- append/read history
|
||||
- diff snapshots from disk
|
||||
- merge collection metrics with per-artifact evaluation scores
|
||||
|
||||
### Phase 4: Rebuild Relation And Graph Features
|
||||
|
||||
Reimplement relation parsing and typed graph export, again using
|
||||
`markitect-tool` for Markdown structure and keeping graph semantics in
|
||||
`infospace-bench`.
|
||||
|
||||
Acceptance target:
|
||||
|
||||
- parse relation artifacts
|
||||
- build an artifact/entity graph
|
||||
- export Mermaid/DOT
|
||||
- filter by relationship type or artifact id
|
||||
|
||||
### Phase 5: Reintroduce Workflow Execution
|
||||
|
||||
Rebuild the old `SourcePipeline` as an application workflow over explicit
|
||||
artifact manifests and workflow definitions. Deterministic Markdown transforms
|
||||
should use `markitect-tool`; LLM/provider behavior should go through
|
||||
`llm-connect` or the eventual engine boundary, not direct legacy prompt code.
|
||||
|
||||
### Phase 6: Migrate A Pruned Legacy Pilot
|
||||
|
||||
Migrate a small, representative slice of
|
||||
`markitect-main/examples/infospace-with-history`:
|
||||
|
||||
- one or two source chapters
|
||||
- schemas/contracts
|
||||
- a few generated entities
|
||||
- relation examples
|
||||
- expected evaluation/metrics outputs
|
||||
|
||||
This should become the real proof that `infospace-bench` can replace the old
|
||||
infospace features.
|
||||
|
||||
## Conclusion
|
||||
|
||||
`infospace-bench` should become the replacement for `markitect-project`
|
||||
infospace features, but the replacement is not complete today.
|
||||
|
||||
The next decisive step is to make `markitect-tool` the Markdown/syntax
|
||||
dependency of `infospace-bench`. After that, replacement should proceed through
|
||||
entity parsing, schema/contract validation, evaluation history, relation graph
|
||||
features, workflow execution, and finally migration of a pruned legacy pilot.
|
||||
93
docs/orthogonal-successor-roadmap.md
Normal file
93
docs/orthogonal-successor-roadmap.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Orthogonal Successor Roadmap
|
||||
|
||||
Date: 2026-05-14
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap explains how `infospace-bench`, `markitect-tool`, and
|
||||
`kontextual-engine` should together replace the relevant functionality of the
|
||||
original `markitect-project` without recreating a monolith.
|
||||
|
||||
## Orthogonal Roles
|
||||
|
||||
| Repository | Layer | Owns | Must not own |
|
||||
| --- | --- | --- | --- |
|
||||
| `markitect-tool` | Syntax | Markdown parsing, document structure, contracts, validation, references, transformations, local query/cache primitives, CLI/library tooling | Concrete knowledge-space projects, durable operational state, domain workflows |
|
||||
| `kontextual-engine` | System | Persistent asset identity, ingestion, metadata, relationships, retrieval, governed transformation, workflow execution, APIs, permissions, auditability | Markdown-specific syntax rules, one domain application, final knowledge-space methodology |
|
||||
| `infospace-bench` | Application | Concrete infospaces, artifact manifests, evaluation methodology, inspection reports, reference experiments, applied workflows, migration pilots | Low-level Markdown tooling, general runtime/orchestration platform, reusable engine primitives |
|
||||
|
||||
## Replacement Principle
|
||||
|
||||
The old `markitect-project` combined syntax, runtime, application, examples,
|
||||
and experiments in one repo. The successor architecture should preserve the
|
||||
useful behaviors by relocating them to the correct layer:
|
||||
|
||||
- Markdown document mechanics move to `markitect-tool`.
|
||||
- Persistence, retrieval, orchestration, governance, and agent-safe operations
|
||||
move to `kontextual-engine`.
|
||||
- Infospace definitions, experiments, quality methodology, concrete pilots,
|
||||
and applied inspection workflows live in `infospace-bench`.
|
||||
|
||||
`infospace-bench` should therefore replace the old infospace feature surface as
|
||||
an **application composition** of the two lower layers, not as a direct code
|
||||
copy.
|
||||
|
||||
## Legacy Feature Placement
|
||||
|
||||
| Old `markitect-project` feature | New home | Notes |
|
||||
| --- | --- | --- |
|
||||
| Markdown AST parsing and section extraction | `markitect-tool` | `infospace-bench` consumes via adapter. |
|
||||
| Entity/relation Markdown shape validation | `markitect-tool` + `infospace-bench` | Generic contract validation in tool; infospace-specific contract selection in bench. |
|
||||
| Infospace config and project layout | `infospace-bench` | Concrete application artifact. |
|
||||
| Entity and relation domain models | `infospace-bench` | Application-level knowledge semantics. |
|
||||
| Collection checks and viability thresholds | `infospace-bench` | Methodology belongs with applied infospaces. |
|
||||
| Metrics history and evaluation reports | `infospace-bench` initially; `kontextual-engine` later for durable storage | File-backed first, engine-backed when needed. |
|
||||
| Source processing pipelines | `infospace-bench` definitions; `markitect-tool` transforms; `kontextual-engine` orchestration | Keep workflow intent separate from runtime. |
|
||||
| LLM-assisted evaluation/generation | `infospace-bench` workflow contracts; provider abstraction outside or lower layer | No vendor lock-in. |
|
||||
| Asset identity, provenance, retrieval, workflow runs | `kontextual-engine` | `infospace-bench` should integrate, not reimplement. |
|
||||
| Reference experiments such as Wealth of Nations/VSM | `infospace-bench` | Concrete pilot and acceptance corpus. |
|
||||
|
||||
## Workplan Set
|
||||
|
||||
The following `infospace-bench` workplans establish the path:
|
||||
|
||||
1. `IB-WP-0005` — Orthogonal successor architecture and feature map
|
||||
2. `IB-WP-0006` — `markitect-tool` adapter and Markdown artifact validation
|
||||
3. `IB-WP-0007` — Entity and relation model migration
|
||||
4. `IB-WP-0008` — Evaluation history, metrics, and viability parity
|
||||
5. `IB-WP-0009` — Applied workflow and generation pipeline
|
||||
6. `IB-WP-0010` — `kontextual-engine` integration boundary
|
||||
7. `IB-WP-0011` — Pruned legacy reference pilot migration
|
||||
8. `IB-WP-0012` — Replacement readiness and CLI parity gate
|
||||
|
||||
## Sequencing
|
||||
|
||||
```text
|
||||
IB-WP-0005
|
||||
-> IB-WP-0006
|
||||
-> IB-WP-0007
|
||||
-> IB-WP-0008
|
||||
-> IB-WP-0009
|
||||
-> IB-WP-0010
|
||||
-> IB-WP-0011
|
||||
-> IB-WP-0012
|
||||
```
|
||||
|
||||
The sequence is intentionally conservative: first prove boundaries, then add
|
||||
syntax-layer integration, then migrate application semantics, then introduce
|
||||
runtime/engine integration and final replacement gates.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
The successor split is working when:
|
||||
|
||||
- `infospace-bench` can run meaningful infospaces without internal Markdown
|
||||
parser code.
|
||||
- `infospace-bench` can optionally persist and operate infospaces through
|
||||
`kontextual-engine` without becoming an engine itself.
|
||||
- A pruned legacy MarkiTect infospace can be migrated and evaluated with
|
||||
traceable outputs.
|
||||
- Users can see which old `markitect-project` infospace commands are replaced,
|
||||
reframed, delegated, or intentionally retired.
|
||||
- The three repos remain independently understandable from their own
|
||||
`INTENT.md` files.
|
||||
Reference in New Issue
Block a user