infospace-bench/docs/markitect-tool-and-infospace-replacement-assessment.md

# Markitect Tool And Infospace Replacement Assessment

Date: 2026-05-14

## Questions

1. Does `infospace-bench` use `markitect-tool`?
2. Can `infospace-bench` replace the infospace features that were present in
   `markitect-project` / `markitect-main`?

## Short Answer

`infospace-bench` does **not** currently use `markitect-tool`.

`infospace-bench` can replace the **foundation** of the old infospace feature
set, but it cannot yet replace the full `markitect-project` infospace surface.
It has a clean file-backed lifecycle, artifact manifest, evaluation result
models, basic deterministic collection checks, viability thresholding,
relationship summaries, and a small reference pilot. The legacy project still
has substantially more mature Markdown parsing, entity extraction, schema
validation, LLM pipeline execution, classification, metrics history, relation
parsing, graph export, and CLI views.

## Current `infospace-bench` State

Implemented and tested:

- `create_infospace()`, `load_infospace()`, `add_artifact()`
- `infospace.yaml` plus `artifacts/index.yaml`
- JSON CLI commands: `create`, `inspect`, `add-artifact`, `export`
- Structured `InfospaceError`
- Evaluation objects: `ScoreEntry`, `EntityEvaluation`, `MetricValue`,
  `EvaluationSnapshot`, snapshot diffs
- Baseline collection metrics:
  - `redundancy_ratio`
  - `coverage_ratio`
  - `coherence_components`
  - `consistency_cycles`
  - `granularity_entropy`
- Viability threshold checks
- Relationship summaries and Mermaid export
- `infospaces/bootstrap-pilot`

Current test status: `14 passed`.

Current dependency declaration:

```toml
dependencies = ["PyYAML>=6"]
```

There is no import of `markitect_tool`, no package dependency on
`markitect-tool`, and no adapter layer around its APIs.

## Current `markitect-tool` Relevance

`markitect-tool` is the syntax-layer successor to the Markdown tooling from
`markitect-main`. It already exposes several surfaces that `infospace-bench`
should use instead of reimplementing:

- Markdown parsing:
  - `parse_markdown()`
  - `parse_markdown_file()`
  - `Document`, `Heading`, `Section`, `ContentBlock`
- Schema validation:
  - `validate_markdown_file()`
  - `validate_document()`
  - `load_schema_file()`
- Contracts and diagnostics:
  - document contracts
  - validation diagnostics
  - metric collection
- Query, cache, reference, processor, workflow, and generation primitives

This matches the intended boundary in `INTENT.md`: `markitect-tool` makes
Markdown structured and manipulable; `infospace-bench` makes applied knowledge
spaces concrete and meaningful.

## Legacy `markitect-project` Infospace Surface

The legacy infospace package in `markitect-main/markitect/infospace/` includes:

- Entity metadata parsing from Markdown entity files
- Entity schema and compliance validation
- Infospace config loading/saving, including schema registry, pipeline config,
  competency questions, directories, and viability thresholds
- Per-entity LLM evaluation prompt construction, response parsing, and batch
  execution
- Evaluation file I/O, snapshot history, and history diffing
- Collection checks for redundancy, coverage, coherence, consistency, and
  granularity
- Metrics file writing and history tracking
- Classification models and classification summaries
- Relation triplet parsing
- Entity/relation graph construction and export
- Source processing pipeline over `infospace.yaml` stages
- CLI commands including status, entities, chapters, evaluate, eval-summary,
  relations, classify, classify-summary, classify-links, viability, check,
  history, history-diff, bind-discipline, process, stale-mappings, and graph
- The large `examples/infospace-with-history` reference experiment

## Capability Comparison

| Capability | `infospace-bench` today | Legacy `markitect-project` | Replacement status |
| --- | --- | --- | --- |
| File-backed infospace layout | Yes | Partial/different | Replaceable |
| Config model | Basic | Rich | Partial |
| Artifact manifest | Yes | Entity/output directories | New baseline, not equivalent |
| Entity Markdown parsing | No | Yes | Missing |
| Markdown syntax parsing via `markitect-tool` | No | Legacy `markitect.core` | Missing integration |
| Schema validation | No | Yes | Missing; should use `markitect-tool` |
| Structured errors | Yes | Mixed CLI exits/exceptions | Improved baseline |
| Evaluation result models | Yes | Yes | Mostly replaceable |
| Evaluation I/O/history | No | Yes | Missing |
| LLM evaluation pipeline | No | Yes | Missing; should route through lower-layer adapters |
| Collection metrics | Basic deterministic | Richer entity/graph based checks | Partial |
| Viability thresholds | Yes | Yes | Basic replacement |
| Relationship summary | Basic manifest edges | Relation files, classifications, graph export | Partial |
| Mermaid graph export | Basic | Richer typed graph export | Partial |
| Classification | No | Yes | Missing |
| Source processing pipeline | No | Yes | Missing |
| Reference pilot | Small bootstrap pilot | Large Wealth of Nations/VSM pilot | Partial |
| CLI surface | Minimal | Broad | Not replaceable yet |

## Assessment

`infospace-bench` is correctly positioned as the successor, but it is still in
the early replacement phase. It is valuable because it has cleaner boundaries
than `markitect-project`: it does not carry old Markdown parser code, prompt
runtime infrastructure, asset management, GraphQL experiments, or platform
concerns.

That boundary clarity is also why it is not a full replacement yet. The missing
work is not just code volume; it is the application-layer integration between
real Markdown artifacts, schemas/contracts, workflow execution, evaluation
history, and graph inspection.

## Recommended Replacement Path

### Phase 1: Integrate `markitect-tool`

Add an explicit dependency on `markitect-tool` and introduce a thin adapter
module in `infospace-bench`, for example:

```text
src/infospace_bench/markdown_adapter.py
```

The adapter should expose only application-level operations needed by
infospaces:

- parse artifact Markdown into a structured document
- extract heading-led sections
- validate an artifact against a configured schema/contract
- return diagnostics in `InfospaceError`-compatible shapes

Do not expose the whole `markitect-tool` API through `infospace-bench`.

### Phase 2: Replace Legacy Entity Parsing And Validation

Reimplement the useful parts of `markitect/infospace/entity_parser.py`,
`schema.py`, and `validator.py` on top of `markitect-tool` parsed documents and
schema/contract validation.

Acceptance target:

- migrate a small entity fixture from `examples/infospace-with-history`
- parse entity metadata
- validate required sections
- emit structured diagnostics

### Phase 3: Add Evaluation History I/O

Promote the current evaluation dataclasses into persistent artifacts:

- write/read entity evaluation files
- write/read snapshots
- append/read history
- diff snapshots from disk
- merge collection metrics with per-artifact evaluation scores

### Phase 4: Rebuild Relation And Graph Features

Reimplement relation parsing and typed graph export, again using
`markitect-tool` for Markdown structure and keeping graph semantics in
`infospace-bench`.

Acceptance target:

- parse relation artifacts
- build an artifact/entity graph
- export Mermaid/DOT
- filter by relationship type or artifact id

### Phase 5: Reintroduce Workflow Execution

Rebuild the old `SourcePipeline` as an application workflow over explicit
artifact manifests and workflow definitions. Deterministic Markdown transforms
should use `markitect-tool`; LLM/provider behavior should go through
`llm-connect` or the eventual engine boundary, not direct legacy prompt code.

### Phase 6: Migrate A Pruned Legacy Pilot

Migrate a small, representative slice of
`markitect-main/examples/infospace-with-history`:

- one or two source chapters
- schemas/contracts
- a few generated entities
- relation examples
- expected evaluation/metrics outputs

This should become the real proof that `infospace-bench` can replace the old
infospace features.

## Conclusion

`infospace-bench` should become the replacement for `markitect-project`
infospace features, but the replacement is not complete today.

The next decisive step is to make `markitect-tool` the Markdown/syntax
dependency of `infospace-bench`. After that, replacement should proceed through
entity parsing, schema/contract validation, evaluation history, relation graph
features, workflow execution, and finally migration of a pruned legacy pilot.