markitect-main/TODO.md

Todofile

This is a "to do next" file, particularly useful to keep the human and a coding assistant in sync.

The format is based on Keep a Todofile V0.0.1.

The structure organizes future tasks by their impact, just as a changelog organizes past changes by their impact.

---

[Unreleased] - Active Vibe-Coding State 💡

This section is for tasks currently being discussed with or worked on by the coding assistant. These are the ephemeral, flow-of-thought tasks.

Schema-of-Schemas Implementation (Active - Phase 4)

Status: Completed (All 6 Phases ✅) Workplan: See history/2026-01-05-schema-of-schemas/WORKPLAN.md (archived)

Current Goals:

1. ✅ Establish naming convention: {domain}-schema-v{major}.{minor}.md
2. ✅ Implement filename validation logic
3. ✅ Create markdown schema loader
4. ✅ Create example markdown schema
5. ✅ Build schema-for-schemas metaschema
6. ✅ Migrate existing schemas to new format

Phase 1 Tasks (Completed ✅):

• Write markitect/schema_naming.py with validation logic
• Add unit tests for filename validation (50 tests, 100% passing)
• Create SCHEMA_NAMING_SPEC.md documentation

Phase 2 Tasks (Completed ✅):

• Implement MarkdownSchemaLoader class (markitect/schema_loader.py, 515 lines)
• Add frontmatter extraction (YAML)
• Add JSON code block extraction with section preference
• Add metadata merging with x-markitect-source tracking
• Write comprehensive unit tests (35 tests, 100% passing)
• Create example markdown schema (manpage-schema-v1.0.md)
• Create SCHEMA_LOADER_GUIDE.md documentation

Phase 3 Tasks (Completed ✅):

• Design schema-for-schemas metaschema (schema-schema-v1.0.md)
• Implement metaschema with validation rules for MarkiTect conventions
• Add schema-validate CLI command with detailed error reporting
• Write comprehensive unit tests (12 tests, 100% passing)
• Test metaschema self-validation
• Validate existing schemas against metaschema

Phase 4 Tasks (Completed ✅):

• Create migration script (scripts/migrate_schemas.py)
• Migrate terminology-schema.json → terminology-schema-v1.0.md
• Migrate api-documentation → api-documentation-schema-v1.0.md
• Delete duplicate schemas (markdown-manpage, markdown-manpage-schema.json)
• Delete replaced schema (enhanced-manpage)
• Update schema-ingest CLI to support markdown files
• Validate all migrated schemas
• Ingest all markdown schemas into database

Phase 5 Tasks (Completed ✅):

• Add numbered references to schema-list (all output formats)
• Implement schema selection parser (numbers, ranges, lists)
• Implement schema resolution logic (registry with filesystem fallback)
• Enhance schema-validate command with multiple selection support
• Add --all flag for batch validation
• Implement batch output formatting with summary table
• Test all selection methods (1, 1-3, 1,3,5, all, filename, ./path)
• Maintain backward compatibility with single-file validation

Phase 6 Tasks (Completed ✅):

• Run complete test suite - all 97 tests passing (50 naming + 35 loader + 12 metaschema)
• Perform end-to-end integration testing of complete schema workflow
• Test schema creation, validation, ingestion, listing, and batch operations
• Create comprehensive usage documentation (SCHEMA_MANAGEMENT_GUIDE.md)
• Document all commands, workflows, and best practices
• Verify no regressions in existing functionality

Schema-of-Schemas Implementation: COMPLETE ✅

All 6 phases completed successfully. The schema management system is fully functional with comprehensive testing and documentation.

---

Extract Capability-Capability from Issue-Facade (Paused)

Context: Issue-facade currently provides two capabilities:

1. issue-tracking (explicit in CAPABILITY-issue-tracking.yaml) - Issue management across platforms
2. capability-capability (implicit) - Patterns and tools for creating/managing capabilities

The capability-capability includes:

• Feedback pattern (feedback/ directory, .capability/feedback CLI tool, documentation)
• Detachment facility (.capability/detach script for clean capability removal)
• Integration pattern (.capability/integrate.sh for project integration)
• CAPABILITY-*.yaml specification format
• ReusableCapabilitiesArchitecture.md (complete specification)
• Directory conventions (_family/implementation, visible/hidden patterns)

Goal: Extract capability-capability to separate reusable-capability repository so it can be used by any capability in the markitect ecosystem.

Approach: Step-by-step extraction, starting with specification.

Phase 1: Specification & Planning (Current)

• Create CAPABILITY-capability.yaml in issue-facade to explicitly declare the implicit capability
• Define what belongs to capability-capability family vs issue-tracking family
• Document the capability-capability API surface (what tools/patterns it provides)
• Identify all files/directories to extract
• Plan extraction strategy (copy vs move, how to maintain during transition)

Phase 2: Repository Creation

• Create reusable-capability repository structure
• Extract ReusableCapabilitiesArchitecture.md to new repo
• Extract feedback pattern (directory structure, CLI tool, README)
• Extract detachment facility (.capability/detach)
• Extract integration scripts (.capability/integrate.sh, integration-checklist.md)
• Create CAPABILITY-capability.yaml in new repo (canonical version)
• Add README.md for reusable-capability repo

Phase 3: Integration & Testing

• Update issue-facade to depend on reusable-capability (as integrated capability)
• Integrate reusable-capability into issue-facade using _capability/reusable-capability pattern
• Test that issue-facade still works with extracted capability
• Update issue-facade documentation to reference both capabilities it provides/uses
• Verify feedback system still works
• Verify detachment still works

Phase 4: Dogfooding & Validation

• Choose another markitect capability for dogfooding
• Integrate reusable-capability into that capability
• Add feedback system to new capability
• Add detachment facility to new capability
• Document learnings and refine reusable-capability based on real-world usage
• Update ReusableCapabilitiesArchitecture.md with insights

Current Step: Phase 1, Task 1 - Create CAPABILITY-capability.yaml

---

Completed Tasks

Recent completed tasks have been documented in _issue-tracking/issue-facade/CHANGELOG.md following Keep a Changelog format.

2026-01-05 - Phase 6: Integration Testing and Final Documentation

• ✅ Ran complete test suite - all 97 tests passing (50 naming + 35 loader + 12 metaschema)
• ✅ Performed end-to-end integration testing:
  • Schema creation and validation
  • Schema ingestion into registry
  • Numbered schema listing
  • Single schema validation (by number, filename, path)
  • Batch validation (ranges, lists, --all)
  • Schema deletion
• ✅ Created comprehensive SCHEMA_MANAGEMENT_GUIDE.md with:
  • Quick start guide and templates
  • Complete command reference
  • Common workflows and examples
  • Best practices and troubleshooting
  • Advanced usage patterns

Schema-of-Schemas Implementation Complete:

• 6 phases completed over 2 days
• 97 unit tests (100% passing)
• End-to-end integration verified
• Comprehensive documentation delivered
• Fully functional schema management system

2026-01-05 - Phase 5: Enhanced Schema Validation with Multiple Selection

• ✅ Enhanced schema-list command with numbered references in all formats
• ✅ Implemented schema selection parser supporting:
  • Single number: markitect schema-validate 1
  • Number range: markitect schema-validate 1-3
  • Number list: markitect schema-validate 1,3,5
  • Keyword: markitect schema-validate --all or all
  • Filename: markitect schema-validate schema.md
  • Filesystem path: markitect schema-validate ./schema.md
• ✅ Implemented schema resolution with registry precedence and filesystem fallback
• ✅ Added batch validation with summary table output
• ✅ Added ValidationResult dataclass for structured results
• ✅ Created helper functions: parse_schema_selector, resolve_schema_source, is_filesystem_path, format_validation_summary
• ✅ Maintained full backward compatibility with existing single-file validation
• ✅ Tested all selection methods successfully

Key Features Delivered:

• Number-based schema selection for quick validation
• Batch validation results displayed as clear summary table
• Registry schemas take precedence over filesystem paths
• Helpful error messages with usage examples
• Exit code 0 for success, 1 for validation failures
• Support for future wildcard/globbing expansion

2026-01-04 - Phase 2: Schema Refinement Tools & Terminology Example

• ✅ Implemented schema-analyze command to detect rigidity issues
• ✅ Implemented schema-refine command with automatic loosening logic
• ✅ Added interactive mode to schema-refine for fine-grained control
• ✅ Created comprehensive test suite (33 unit tests, 100% passing)
• ✅ Wrote user guide documentation with examples and workflows
• ✅ Successfully tested on example schemas (reduced rigidity from 60/100 to 24/100)
• ✅ Integrated into CLI with proper exit codes and error handling
• ✅ Moved SCHEMA_EVOLUTION_WORKPLAN.md to todo/ directory
• ✅ Created terminology validation example (examples/terminology/)

Key Features Delivered:

• Rigidity score calculation (0-100 scale)
• Automatic detection of exact counts, const values, overly specific numbers
• Path navigation for nested schema properties
• Dry-run mode for previewing changes
• Interactive approval workflow
• Comprehensive reporting (normal and verbose modes)

Terminology Example:

• Complete terminology document structure (terminology-example.md)
• JSON schema with MarkiTect extensions (terminology-schema.json)
• Demonstrates schema usage for non-manpage documents
• Validates term definitions, synonyms, related terms, examples
• Includes content control and validation rules
• Full documentation and usage examples (README.md)

2026-01-04 - Phase 2: Markdown Schema Loader

• ✅ Implemented MarkdownSchemaLoader class (markitect/schema_loader.py, 515 lines)
• ✅ YAML frontmatter extraction with validation
• ✅ JSON code block extraction with "Schema Definition" section preference
• ✅ Metadata merging with x-markitect-source tracking
• ✅ Schema saving with template support and round-trip capability
• ✅ Comprehensive test suite (35 unit tests, 100% passing)
• ✅ Created example markdown schema (manpage-schema-v1.0.md)
• ✅ Created SCHEMA_LOADER_GUIDE.md with complete usage documentation

Key Features Delivered:

• Markdown-first schema format with embedded JSON
• Frontmatter metadata merges into schema ($id, version, status)
• Automatic detection of multiple JSON blocks
• Schema structure validation helper
• Error handling for binary files and invalid formats
• List JSON blocks helper for debugging
• Full round-trip save/load capability

Example Markdown Schema:

• manpage-schema-v1.0.md demonstrating complete format
• Includes frontmatter, documentation, and JSON schema
• Shows section classification and content control
• Follows naming convention: {domain}-schema-v{major}.{minor}.md

2026-01-04 - Phase 3: Schema-for-Schemas Metaschema

• ✅ Created schema-schema-v1.0.md metaschema (650+ lines)
• ✅ Validates core JSON Schema fields ($schema, $id, title, description)
• ✅ Validates MarkiTect version field (SemVer: major.minor.patch)
• ✅ Validates $id URL format (HTTPS with version)
• ✅ Validates MarkiTect extensions (x-markitect-sections, x-markitect-content-control, x-markitect-metadata)
• ✅ Implemented schema-validate CLI command with detailed error reporting
• ✅ Comprehensive test suite (12 unit tests, 100% passing)
• ✅ Metaschema self-validation successful

Key Features Delivered:

• Complete metaschema for validating all MarkiTect schemas
• Section classification validation (required, recommended, optional, discouraged, improper)
• Content control pattern validation
• Version format enforcement (SemVer)
• $id URL format enforcement (HTTPS with version)
• CLI command for easy schema validation
• Detailed error messages with schema paths

Validation Results:

• ✅ Metaschema validates itself
• ✅ Manpage schema validates successfully
• ⚠️ Terminology schema needs migration (missing version field, incorrect $id format)

2026-01-05 - Phase 4: Schema Migration

• ✅ Created migration script (scripts/migrate_schemas.py, 240 lines)
• ✅ Migrated 2 schemas to markdown format
• ✅ Deleted 3 duplicate/replaced schemas from database
• ✅ Updated schema-ingest CLI to support markdown files (.md)
• ✅ All 4 schemas now in markdown format following naming convention

Schemas Migrated:

• terminology-schema.json → terminology-schema-v1.0.md
• api-documentation → api-documentation-schema-v1.0.md

Schemas Deleted:

• markdown-manpage (duplicate)
• markdown-manpage-schema.json (duplicate)
• enhanced-manpage (replaced by manpage-schema-v1.0.md)

Final Schema Registry:

• ✅ terminology-schema-v1.0.md
• ✅ api-documentation-schema-v1.0.md
• ✅ manpage-schema-v1.0.md
• ✅ schema-schema-v1.0.md (metaschema)

All schemas validate successfully against the metaschema!

2025-12-17 - Architecture Refactoring

• ✅ Implemented ReusableCapabilitiesArchitecture v0.1
• ✅ Added feedback capability to issue-facade
• ✅ Created detachment facility
• ✅ Refactored to family-based directory structure (_issue-tracking/issue-facade)
• ✅ Made feedback directory visible (feedback/ not .feedback/)
• ✅ Renamed to explicit family declaration (CAPABILITY-issue-tracking.yaml)
• ✅ Created CHANGELOG.md documenting v1.0.0