infospace-bench/docs/markitect-tool-and-infospace-replacement-assessment.md

Markitect Tool And Infospace Replacement Assessment

Date: 2026-05-14

Questions

1. Does infospace-bench use markitect-tool?
2. Can infospace-bench replace the infospace features that were present in markitect-project / markitect-main?

Short Answer

infospace-bench does not currently use markitect-tool.

infospace-bench can replace the foundation of the old infospace feature set, but it cannot yet replace the full markitect-project infospace surface. It has a clean file-backed lifecycle, artifact manifest, evaluation result models, basic deterministic collection checks, viability thresholding, relationship summaries, and a small reference pilot. The legacy project still has substantially more mature Markdown parsing, entity extraction, schema validation, LLM pipeline execution, classification, metrics history, relation parsing, graph export, and CLI views.

Current infospace-bench State

Implemented and tested:

• create_infospace(), load_infospace(), add_artifact()
• infospace.yaml plus artifacts/index.yaml
• JSON CLI commands: create, inspect, add-artifact, export
• Structured InfospaceError
• Evaluation objects: ScoreEntry, EntityEvaluation, MetricValue, EvaluationSnapshot, snapshot diffs
• Baseline collection metrics:
  • redundancy_ratio
  • coverage_ratio
  • coherence_components
  • consistency_cycles
  • granularity_entropy
• Viability threshold checks
• Relationship summaries and Mermaid export
• infospaces/bootstrap-pilot

Current test status: 14 passed.

Current dependency declaration:

dependencies = ["PyYAML>=6"]

There is no import of markitect_tool, no package dependency on markitect-tool, and no adapter layer around its APIs.

Current markitect-tool Relevance

markitect-tool is the syntax-layer successor to the Markdown tooling from markitect-main. It already exposes several surfaces that infospace-bench should use instead of reimplementing:

• Markdown parsing:
  • parse_markdown()
  • parse_markdown_file()
  • Document, Heading, Section, ContentBlock
• Schema validation:
  • validate_markdown_file()
  • validate_document()
  • load_schema_file()
• Contracts and diagnostics:
  • document contracts
  • validation diagnostics
  • metric collection
• Query, cache, reference, processor, workflow, and generation primitives

This matches the intended boundary in INTENT.md: markitect-tool makes Markdown structured and manipulable; infospace-bench makes applied knowledge spaces concrete and meaningful.

Legacy markitect-project Infospace Surface

The legacy infospace package in markitect-main/markitect/infospace/ includes:

• Entity metadata parsing from Markdown entity files
• Entity schema and compliance validation
• Infospace config loading/saving, including schema registry, pipeline config, competency questions, directories, and viability thresholds
• Per-entity LLM evaluation prompt construction, response parsing, and batch execution
• Evaluation file I/O, snapshot history, and history diffing
• Collection checks for redundancy, coverage, coherence, consistency, and granularity
• Metrics file writing and history tracking
• Classification models and classification summaries
• Relation triplet parsing
• Entity/relation graph construction and export
• Source processing pipeline over infospace.yaml stages
• CLI commands including status, entities, chapters, evaluate, eval-summary, relations, classify, classify-summary, classify-links, viability, check, history, history-diff, bind-discipline, process, stale-mappings, and graph
• The large examples/infospace-with-history reference experiment

Capability Comparison

Capability	infospace-bench today	Legacy markitect-project	Replacement status
File-backed infospace layout	Yes	Partial/different	Replaceable
Config model	Basic	Rich	Partial
Artifact manifest	Yes	Entity/output directories	New baseline, not equivalent
Entity Markdown parsing	No	Yes	Missing
Markdown syntax parsing via markitect-tool	No	Legacy markitect.core	Missing integration
Schema validation	No	Yes	Missing; should use markitect-tool
Structured errors	Yes	Mixed CLI exits/exceptions	Improved baseline
Evaluation result models	Yes	Yes	Mostly replaceable
Evaluation I/O/history	No	Yes	Missing
LLM evaluation pipeline	No	Yes	Missing; should route through lower-layer adapters
Collection metrics	Basic deterministic	Richer entity/graph based checks	Partial
Viability thresholds	Yes	Yes	Basic replacement
Relationship summary	Basic manifest edges	Relation files, classifications, graph export	Partial
Mermaid graph export	Basic	Richer typed graph export	Partial
Classification	No	Yes	Missing
Source processing pipeline	No	Yes	Missing
Reference pilot	Small bootstrap pilot	Large Wealth of Nations/VSM pilot	Partial
CLI surface	Minimal	Broad	Not replaceable yet

Assessment

infospace-bench is correctly positioned as the successor, but it is still in the early replacement phase. It is valuable because it has cleaner boundaries than markitect-project: it does not carry old Markdown parser code, prompt runtime infrastructure, asset management, GraphQL experiments, or platform concerns.

That boundary clarity is also why it is not a full replacement yet. The missing work is not just code volume; it is the application-layer integration between real Markdown artifacts, schemas/contracts, workflow execution, evaluation history, and graph inspection.

Recommended Replacement Path

Phase 1: Integrate markitect-tool

Add an explicit dependency on markitect-tool and introduce a thin adapter module in infospace-bench, for example:

src/infospace_bench/markdown_adapter.py

The adapter should expose only application-level operations needed by infospaces:

• parse artifact Markdown into a structured document
• extract heading-led sections
• validate an artifact against a configured schema/contract
• return diagnostics in InfospaceError-compatible shapes

Do not expose the whole markitect-tool API through infospace-bench.

Phase 2: Replace Legacy Entity Parsing And Validation

Reimplement the useful parts of markitect/infospace/entity_parser.py, schema.py, and validator.py on top of markitect-tool parsed documents and schema/contract validation.

Acceptance target:

• migrate a small entity fixture from examples/infospace-with-history
• parse entity metadata
• validate required sections
• emit structured diagnostics

Phase 3: Add Evaluation History I/O

Promote the current evaluation dataclasses into persistent artifacts:

• write/read entity evaluation files
• write/read snapshots
• append/read history
• diff snapshots from disk
• merge collection metrics with per-artifact evaluation scores

Phase 4: Rebuild Relation And Graph Features

Reimplement relation parsing and typed graph export, again using markitect-tool for Markdown structure and keeping graph semantics in infospace-bench.

Acceptance target:

• parse relation artifacts
• build an artifact/entity graph
• export Mermaid/DOT
• filter by relationship type or artifact id

Phase 5: Reintroduce Workflow Execution

Rebuild the old SourcePipeline as an application workflow over explicit artifact manifests and workflow definitions. Deterministic Markdown transforms should use markitect-tool; LLM/provider behavior should go through llm-connect or the eventual engine boundary, not direct legacy prompt code.

Phase 6: Migrate A Pruned Legacy Pilot

Migrate a small, representative slice of markitect-main/examples/infospace-with-history:

• one or two source chapters
• schemas/contracts
• a few generated entities
• relation examples
• expected evaluation/metrics outputs

This should become the real proof that infospace-bench can replace the old infospace features.

Conclusion

infospace-bench should become the replacement for markitect-project infospace features, but the replacement is not complete today.

The next decisive step is to make markitect-tool the Markdown/syntax dependency of infospace-bench. After that, replacement should proceed through entity parsing, schema/contract validation, evaluation history, relation graph features, workflow execution, and finally migration of a pruned legacy pilot.