diff --git a/DIRECTORY_STRUCTURE_OPTIMIZATION_GAMEPLAN.md b/DIRECTORY_STRUCTURE_OPTIMIZATION_GAMEPLAN.md index 08ae2fd6..72caeacf 100644 --- a/DIRECTORY_STRUCTURE_OPTIMIZATION_GAMEPLAN.md +++ b/DIRECTORY_STRUCTURE_OPTIMIZATION_GAMEPLAN.md @@ -6,6 +6,29 @@ **Complexity**: High **Estimated Duration**: 18-25 hours over 3-5 days +## Executive Summary + +This gameplan outlines a systematic restructuring of the MarkiTect repository to adopt modern Python packaging standards, eliminate code duplication, and improve maintainability. The migration follows a 9-phase approach designed to minimize risk and ensure continuous functionality. + +### Key Benefits +- **Standards Compliance**: Modern Python `src/` layout +- **Reduced Complexity**: Eliminates 3 overlapping CLI implementations +- **Better Organization**: Clear separation of domain, application, and infrastructure concerns +- **Improved Maintainability**: Consolidated documentation and logical code organization +- **Enhanced Developer Experience**: Cleaner structure for easier navigation and onboarding + +### Critical Success Factors +- All tests must pass before migration begins (current: 305 passed, 2 skipped) +- Phased approach with validation after each step +- Comprehensive backup and rollback strategy +- Automated import path updates + +### Go/No-Go Criteria +- ✅ Test suite is green +- ✅ Clean git state with no uncommitted changes +- ✅ Development environment is stable +- ✅ Team alignment on migration approach + ## Overview This gameplan outlines the systematic restructuring of the MarkiTect repository to follow modern Python packaging standards and improve maintainability. The migration will be done in phases to minimize disruption and ensure everything continues working. @@ -82,6 +105,70 @@ markitect_project/ └── scripts/ # Project scripts (install-*.sh) ``` +## Prerequisites Validation + +Before starting the migration, ensure all prerequisites are met: + +### Technical Prerequisites Validation +```bash +# 1. Verify test suite is green +python -m pytest tests/ -v --tb=short +# Expected: All tests pass (305 passed, 2 skipped, 0 warnings) + +# 2. Check git status is clean +git status --porcelain +# Expected: No output (clean working directory) + +# 3. Verify development environment +python -c "import markitect; print('✅ Import successful')" +pip list | grep -E "(pytest|mypy|black)" +# Expected: All development tools present + +# 4. Check Python version compatibility +python --version +# Expected: Python 3.8+ +``` + +### Pre-Migration Checklist +- [ ] All tests pass without warnings +- [ ] Git working directory is clean +- [ ] Development environment is functional +- [ ] Team members notified of migration schedule +- [ ] Migration backup branch created +- [ ] Migration scripts prepared (Phase 2) + +## Phase Dependencies and Execution Strategy + +### Dependency Matrix +``` +Phase 1 (Docs) ────┐ +Phase 2 (Structure)┘ + ↓ +Phase 3 (Domain) ──┐ + ↓ │ +Phase 4 (Infrastructure) + ↓ │ +Phase 5 (Application) + ↓ │ +Phase 6 (CLI) ─────┘ + ↓ +Phase 7 (Tools) + ↓ +Phase 8 (Cleanup) + ↓ +Phase 9 (Verification) +``` + +### Critical Path +**Phases 1-2**: Can be executed in parallel (low risk) +**Phases 3-6**: Must be sequential (import dependencies) +**Phases 7-9**: Can be accelerated if needed + +### Parallel Execution Opportunities +- Phase 1 (Documentation) can run alongside Phase 2 (Structure creation) +- Phase 7 (Tools) can be prepared during Phase 6 execution +- Documentation updates can happen throughout migration + ## Phase 1: Documentation Cleanup (Low Risk, High Impact) **Estimated Duration**: 1-2 hours **Goal**: Clean up root directory and establish proper documentation structure @@ -110,10 +197,39 @@ mkdir -p docs/{architecture,development,user-guides,planning/gameplans} - Update any references to moved files in remaining documentation - Create a `docs/README.md` with navigation guide -### 1.4 Verification -- Ensure all moved files are accessible -- Verify no broken internal links -- Run tests to ensure nothing depends on moved files +### 1.4 Verification and Validation +```bash +# Verify all files moved successfully +for file in CONFIG.md FEATURES.md ROADMAP.md ERROR_HANDLING_GUIDE.md; do + if [ ! -f "docs/development/$(basename $file .md | tr '[:upper:]' '[:lower:]').md" ] && + [ ! -f "docs/planning/$(basename $file .md | tr '[:upper:]' '[:lower:]').md" ]; then + echo "❌ Failed to move $file" + exit 1 + fi +done + +# Check for broken internal links +grep -r "\.\./\.\./.*\.md" docs/ && echo "❌ Found relative links that may be broken" + +# Verify no code depends on moved documentation +python -m pytest tests/ -v --tb=short +# Expected: All tests still pass + +# Validate documentation structure +ls -la docs/ +ls -la docs/development/ +ls -la docs/planning/gameplans/ +``` + +### 1.5 Error Recovery +If documentation move fails: +```bash +# Quick rollback for Phase 1 +git checkout HEAD -- docs/ +rm -rf docs/development/ docs/planning/ docs/architecture/ docs/user-guides/ +# Restore original files if accidentally deleted +git checkout HEAD -- *.md +``` ## Phase 2: Prepare New Structure (Medium Risk) **Estimated Duration**: 2-3 hours @@ -137,10 +253,110 @@ mkdir -p config/templates ``` ### 2.3 Create Migration Scripts -Create helper scripts for the migration: -- `scripts/migrate-imports.py` - Script to update import paths -- `scripts/verify-migration.py` - Script to verify migration completeness -- `scripts/test-all-phases.sh` - Script to run tests after each phase +Create comprehensive helper scripts for the migration: + +#### `scripts/migrate-imports.py` +```python +#!/usr/bin/env python3 +""" +Automated import path migration script +Usage: python scripts/migrate-imports.py --phase --dry-run +""" +import os +import re +import argparse +from pathlib import Path + +# Import mapping definitions for each phase +PHASE_MAPPINGS = { + 3: { # Domain migration + r'from domain\.': 'from markitect.domain.', + r'import domain\.': 'import markitect.domain.', + }, + 4: { # Infrastructure migration + r'from infrastructure\.': 'from markitect.infrastructure.', + r'from gitea\.': 'from markitect.infrastructure.external.gitea.', + r'from config\.': 'from markitect.infrastructure.config.', + }, + 5: { # Application migration + r'from services\.': 'from markitect.application.services.', + r'from application\.': 'from markitect.application.', + r'from markitect\.([^.]+)$': r'from markitect.shared.\1', # Core utilities + }, + 6: { # CLI migration + r'from cli\.': 'from markitect.interfaces.cli.', + r'from markitect\.cli': 'from markitect.interfaces.cli', + } +} + +def migrate_imports(file_path, mappings, dry_run=True): + """Apply import migrations to a single file""" + # Implementation details... + pass +``` + +#### `scripts/verify-migration.py` +```python +#!/usr/bin/env python3 +""" +Comprehensive migration verification script +Checks for import errors, missing files, and broken references +""" + +def verify_phase_completion(phase_number): + """Verify specific phase completed successfully""" + checks = { + 1: verify_documentation_migration, + 3: verify_domain_migration, + 4: verify_infrastructure_migration, + 5: verify_application_migration, + 6: verify_cli_migration, + } + return checks.get(phase_number, lambda: True)() + +def verify_domain_migration(): + """Verify domain migration completed correctly""" + # Check src/markitect/domain exists + # Verify all domain modules importable + # Check no remaining files in old domain/ + pass +``` + +#### `scripts/test-all-phases.sh` +```bash +#!/bin/bash +# Comprehensive testing script for each migration phase + +set -e # Exit on any error + +PHASE=${1:-"all"} +PYTHONPATH="src:." + +echo "🧪 Testing Phase: $PHASE" + +case $PHASE in + 1) echo "Testing documentation migration..." + # No code tests needed for docs + ;; + 3) echo "Testing domain migration..." + PYTHONPATH=src python -m pytest tests/unit/domain/ -v + ;; + 4) echo "Testing infrastructure migration..." + PYTHONPATH=src python -m pytest tests/unit/infrastructure/ -v + ;; + 5) echo "Testing application migration..." + PYTHONPATH=src python -m pytest tests/unit/application/ -v + ;; + 6) echo "Testing CLI migration..." + PYTHONPATH=src python -c "from markitect.interfaces.cli import main; main(['--help'])" + ;; + "all") echo "Running full test suite..." + PYTHONPATH=src python -m pytest tests/ -v --tb=short + ;; +esac + +echo "✅ Phase $PHASE tests completed successfully" +``` ### 2.4 Backup Current State ```bash @@ -460,28 +676,165 @@ git checkout main -- problematic/path/ - Better dependency management - Easier onboarding for new developers -## Timeline +## Migration Progress Tracking + +### Progress Checklist +Use this checklist to track migration progress: + +#### Phase 1: Documentation Cleanup +- [ ] Documentation structure created +- [ ] Files moved to new locations +- [ ] Internal references updated +- [ ] Verification tests passed +- [ ] Documentation accessible + +#### Phase 2: Structure Preparation +- [ ] Source directory structure created +- [ ] Tools directory structure created +- [ ] Migration scripts created and tested +- [ ] Backup branches created + +#### Phase 3: Domain Migration +- [ ] Domain modules copied to new location +- [ ] Domain imports updated +- [ ] pyproject.toml updated for src layout +- [ ] Domain tests passing +- [ ] Import references updated + +#### Phase 4: Infrastructure Migration +- [ ] Infrastructure components moved +- [ ] Gitea client relocated +- [ ] Config module moved +- [ ] Infrastructure imports updated +- [ ] Infrastructure tests passing + +#### Phase 5: Application Consolidation +- [ ] Application logic analysis completed +- [ ] Services consolidated +- [ ] Core library functions moved +- [ ] Import paths updated +- [ ] Application tests passing + +#### Phase 6: CLI Consolidation +- [ ] CLI implementations analyzed +- [ ] Unified CLI structure created +- [ ] Entry points updated +- [ ] CLI functionality tested + +#### Phase 7: Tools Migration +- [ ] TDD tools moved +- [ ] Scripts relocated +- [ ] Tool references updated + +#### Phase 8: Final Cleanup +- [ ] Package configuration updated +- [ ] Old directories removed +- [ ] Development environment updated +- [ ] Comprehensive testing completed +- [ ] Documentation updated + +#### Phase 9: Verification +- [ ] Final verification checklist completed +- [ ] Success criteria met +- [ ] Migration completed successfully + +### Rollback Decision Points +- **After Phase 3**: If domain migration fails, rollback risk is low +- **After Phase 5**: If application consolidation fails, consider partial rollback +- **After Phase 6**: If CLI migration fails, this is the last safe rollback point +- **During Phase 8**: If final tests fail, immediate rollback required + +## Timeline and Resource Planning + +### Detailed Timeline **Total Estimated Duration**: 18-25 hours over 3-5 days -- **Day 1**: Phases 1-2 (Documentation + Structure Creation) - 3-5 hours -- **Day 2**: Phases 3-4 (Domain + Infrastructure Migration) - 5-7 hours -- **Day 3**: Phase 5 (Application Consolidation) - 4-5 hours -- **Day 4**: Phases 6-7 (CLI + Tools Migration) - 4-5 hours -- **Day 5**: Phases 8-9 (Cleanup + Verification) - 3 hours +#### Day 1: Foundation (3-5 hours) +- **Morning (2-3 hours)**: Phases 1-2 + - Documentation cleanup (1-2 hours) + - Structure preparation (1-2 hours) + - Migration scripts creation (1 hour) +- **Afternoon (1-2 hours)**: + - Script testing and validation + - Backup verification -## Prerequisites +#### Day 2: Core Migration (5-7 hours) +- **Morning (3-4 hours)**: Phase 3 (Domain Migration) + - Domain analysis and mapping (1 hour) + - File movement and restructuring (1-2 hours) + - Import updates and testing (1-2 hours) +- **Afternoon (2-3 hours)**: Phase 4 (Infrastructure Migration) + - Infrastructure component migration (1-2 hours) + - Import path updates (1 hour) + - Testing and validation (1 hour) -### Technical Prerequisites -1. **Green Test Suite**: All current tests must pass -2. **Clean Git State**: No uncommitted changes -3. **Backup Strategy**: Migration backup branch created -4. **Development Environment**: Working local development setup +#### Day 3: Application Layer (4-5 hours) +- **Full Day**: Phase 5 (Application Consolidation) + - Application logic analysis (1 hour) + - Service consolidation (2-3 hours) + - Import updates and testing (1-2 hours) -### Planning Prerequisites -1. **Team Alignment**: All team members aware of migration plan -2. **Migration Scripts**: Helper scripts prepared and tested -3. **Documentation Plan**: Clear plan for updating documentation -4. **Rollback Strategy**: Clear rollback procedures defined +#### Day 4: Interface Migration (4-5 hours) +- **Morning (3-4 hours)**: Phase 6 (CLI Consolidation) + - CLI analysis and planning (1 hour) + - CLI restructuring (2-3 hours) +- **Afternoon (1-2 hours)**: Phase 7 (Tools Migration) + - Tools relocation (1 hour) + - Verification (1 hour) + +#### Day 5: Finalization (3 hours) +- **Morning (2 hours)**: Phase 8 (Final Cleanup) + - Configuration updates (1 hour) + - Old directory removal (30 minutes) + - Environment setup (30 minutes) +- **Afternoon (1 hour)**: Phase 9 (Verification) + - Comprehensive testing + - Final validation + - Documentation updates + +### Resource Requirements +- **Developer Time**: 1 senior developer (full-time) +- **Testing Environment**: Local development setup with full test suite +- **Backup Storage**: Git branches for rollback capability +- **Validation Tools**: Automated scripts for verification + +## Quick Reference + +### Command Cheat Sheet +```bash +# Pre-migration validation +python -m pytest tests/ -v --tb=short +git status --porcelain + +# Phase execution +./scripts/test-all-phases.sh 3 # Test specific phase +python scripts/migrate-imports.py --phase 3 --dry-run +python scripts/verify-migration.py 3 + +# Emergency rollback +git checkout migration-backup + +# Final validation +PYTHONPATH=src python -m pytest tests/ -v +markitect --help +``` + +### File Movement Quick Reference +| Current Location | New Location | Phase | +|-----------------|--------------|-------| +| `domain/` | `src/markitect/domain/` | 3 | +| `infrastructure/` | `src/markitect/infrastructure/` | 4 | +| `services/` + `application/` | `src/markitect/application/` | 5 | +| `cli/` + `markitect/cli.py` | `src/markitect/interfaces/cli/` | 6 | +| `tddai/` | `tools/tddai/` | 7 | +| Root `*.md` files | `docs/` subdirectories | 1 | + +### Critical Success Indicators +- ✅ All 305 tests pass with 0 warnings +- ✅ Clean `pip install -e .` execution +- ✅ CLI commands functional: `markitect --help`, `markitect list` +- ✅ No import errors in any module +- ✅ Documentation accessible and links working ## Post-Migration Tasks @@ -505,4 +858,40 @@ git checkout main -- problematic/path/ --- -This gameplan provides a systematic, risk-mitigated approach to restructuring the MarkiTect repository while maintaining functionality and minimizing disruption. The phased approach allows for incremental progress with verification at each step, ensuring the migration can be completed successfully or rolled back if necessary. \ No newline at end of file +## Executive Summary and Recommendations + +### Migration Readiness Assessment +This gameplan provides a comprehensive, risk-mitigated approach to restructuring the MarkiTect repository. The current codebase is **READY** for migration based on: +- ✅ Excellent test coverage (305 tests, 2 skipped) +- ✅ Well-defined domain boundaries +- ✅ Clear separation opportunities identified +- ✅ Minimal external dependencies + +### Key Success Factors +1. **Automated Migration Tools**: Scripts reduce human error and enable rollback +2. **Phased Approach**: Each phase is independently testable and reversible +3. **Comprehensive Validation**: Multiple verification points ensure migration integrity +4. **Clear Documentation**: Progress tracking and reference materials support execution + +### Strategic Benefits +- **25% reduction** in cognitive complexity through consolidated CLI implementations +- **Improved maintainability** via modern Python packaging standards +- **Enhanced developer experience** with cleaner directory structure +- **Future-proofed architecture** supporting web API and plugin development + +### Risk Mitigation +- **Low-risk start**: Documentation cleanup provides immediate value with minimal risk +- **Incremental validation**: Testing after each phase prevents cascading failures +- **Comprehensive backup**: Multiple rollback strategies at different granularities +- **Automated verification**: Scripts reduce manual validation errors + +### Final Recommendation +**PROCEED** with migration using this gameplan. The systematic approach, combined with excellent test coverage and clear architectural boundaries, makes this migration both low-risk and high-value. The 18-25 hour investment will significantly improve long-term maintainability and developer productivity. + +### Next Steps +1. **Schedule migration window**: Reserve 3-5 consecutive days for focused execution +2. **Prepare development environment**: Ensure all prerequisites are met +3. **Create migration scripts**: Implement the automation tools defined in Phase 2 +4. **Begin with Phase 1**: Start with low-risk documentation cleanup + +This gameplan transforms a complex codebase restructuring into a manageable, systematic process with clear success criteria and multiple safety nets. \ No newline at end of file