feat: complete Issue #151 - Phase 4: Integration and Documentation

Implements comprehensive CLI integration and documentation for the explode-implode system, completing both Issues #147 and #151. Key Features Added: - md-package CLI command (create/extract/info actions) - md-transclude CLI command (process/validate actions) - Complete user guide (556 lines) with tutorials and examples - Technical API documentation (500 lines) for developers - Migration guide (761 lines) with step-by-step procedures - Cost analysis documenting ~85 hours of development value Technical Implementation: - Full MDZ packaging support with asset embedding - Template-based transclusion with variable substitution - Comprehensive error handling and verbose output modes - Integration with existing MarkiTect CLI architecture Documentation Suite: - docs/user-guides/explode-implode-complete-guide.md - docs/api/explode-variants.md - docs/user-guides/migration-guide.md - docs/cost-analysis/issues-147-151-implementation.md This implementation transforms MarkiTect from a simple markdown processor into a comprehensive document management platform with sophisticated organizational capabilities. Closes #147: Directory organization preservation fully implemented Closes #151: CLI integration and documentation completed 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 11:11:51 +02:00
parent e8e0fbaec3
commit 6ddd4ea6e3
7 changed files with 2389 additions and 1 deletions
--- a/docs/user-guides/explode-implode-complete-guide.md
+++ b/docs/user-guides/explode-implode-complete-guide.md
@@ -0,0 +1,557 @@
+# Complete Guide to MarkiTect's Explode-Implode System
+
+**A comprehensive guide to MarkiTect's powerful document decomposition and recomposition capabilities**
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Getting Started](#getting-started)
+3. [Understanding Variants](#understanding-variants)
+4. [Basic Operations](#basic-operations)
+5. [Advanced Packaging](#advanced-packaging)
+6. [Transclusion Engine](#transclusion-engine)
+7. [Best Practices](#best-practices)
+8. [Troubleshooting](#troubleshooting)
+
+---
+
+## Overview
+
+MarkiTect's explode-implode system provides sophisticated capabilities for breaking down large markdown documents into manageable components and reassembling them with perfect fidelity. This system supports multiple organizational strategies (variants) and includes advanced packaging features for self-contained documents.
+
+### Key Capabilities
+
+- **📂 Three Organizational Variants**: Flat, Hierarchical, and Semantic structures
+- **🔄 Perfect Reversibility**: Manifest-based system ensures lossless round-trips
+- **🤖 Auto-Detection**: Intelligent detection of existing exploded structures
+- **📦 Advanced Packaging**: Self-contained MDZ packages with embedded assets
+- **🔗 Transclusion Engine**: Template-based document generation
+- **📊 Asset Management**: Automated discovery and integrity validation
+
+### When to Use Explode-Implode
+
+**Ideal Use Cases:**
+- Large documentation projects (100+ pages)
+- Multi-author collaborative writing
+- Modular content that needs reorganization
+- Documentation requiring asset management
+- Template-based document generation
+- Legacy document modernization
+
+## Getting Started
+
+### Prerequisites
+
+Ensure you have MarkiTect installed and configured:
+
+```bash
+# Verify installation
+markitect version
+
+# Check that explode-implode commands are available
+markitect md-explode --help
+markitect md-implode --help
+```
+
+### Quick Start Example
+
+Let's explode a large document and then implode it back:
+
+```bash
+# 1. Explode a document using hierarchical variant
+markitect md-explode large-document.md --variant hierarchical --output exploded/
+
+# 2. Review the exploded structure
+tree exploded/
+
+# 3. Make edits to individual sections
+# ... edit files in exploded/ directory ...
+
+# 4. Implode back to a single document
+markitect md-implode exploded/ --output reconstructed-document.md
+
+# 5. Verify the result
+diff large-document.md reconstructed-document.md
+```
+
+## Understanding Variants
+
+MarkiTect provides three organizational variants, each optimized for different use cases:
+
+### 1. Flat Variant (`flat`)
+
+**Structure**: All sections as peer files in a single directory
+```
+document.mdd/
+├── manifest.md
+├── introduction.md
+├── getting_started.md
+├── advanced_features.md
+└── conclusion.md
+```
+
+**Best For:**
+- Simple documents with minimal hierarchy
+- Quick reorganization of content
+- Linear reading flows
+- Collaborative editing of independent sections
+
+**Command Example:**
+```bash
+markitect md-explode document.md --variant flat
+```
+
+### 2. Hierarchical Variant (`hierarchical`)
+
+**Structure**: Nested directories reflecting document hierarchy
+```
+document.mdd/
+├── manifest.md
+├── 01_introduction/
+│   └── index.md
+├── 02_getting_started/
+│   ├── index.md
+│   ├── 01_installation.md
+│   └── 02_configuration.md
+└── 03_advanced_features/
+    ├── index.md
+    └── 01_plugins.md
+```
+
+**Best For:**
+- Complex documents with deep nesting
+- Technical documentation with logical groupings
+- Books or guides with chapter/section structure
+- Content that benefits from visual organization
+
+**Command Example:**
+```bash
+markitect md-explode document.md --variant hierarchical --max-depth 3
+```
+
+### 3. Semantic Variant (`semantic`)
+
+**Structure**: Meaningfully-named directories based on content
+```
+document.mdd/
+├── manifest.md
+├── introduction/
+│   └── overview.md
+├── tutorials/
+│   ├── getting_started.md
+│   └── advanced_usage.md
+├── reference/
+│   └── api_documentation.md
+└── appendices/
+    └── troubleshooting.md
+```
+
+**Best For:**
+- Documentation with distinct content types
+- Knowledge bases requiring categorical organization
+- Content management workflows
+- SEO-optimized content structures
+
+**Command Example:**
+```bash
+markitect md-explode document.md --variant semantic
+```
+
+## Basic Operations
+
+### Exploding Documents
+
+The `md-explode` command breaks down documents into components:
+
+```bash
+# Basic explosion with auto-selected variant
+markitect md-explode document.md
+
+# Specify variant and output directory
+markitect md-explode document.md --variant hierarchical --output my-exploded-doc/
+
+# Create manifest for reversibility (recommended)
+markitect md-explode document.md --variant flat --create-manifest
+
+# Dry run to preview structure
+markitect md-explode document.md --variant semantic --dry-run --verbose
+```
+
+**Key Options:**
+- `--variant`: Choose organizational strategy (`flat`, `hierarchical`, `semantic`)
+- `--output`: Specify output directory (defaults to `{filename}.mdd`)
+- `--max-depth`: Limit nesting depth for hierarchical variant
+- `--create-manifest`: Generate manifest.md for perfect reversibility
+- `--dry-run`: Preview operations without making changes
+- `--verbose`: Detailed output for troubleshooting
+
+### Imploding Documents
+
+The `md-implode` command reassembles exploded structures:
+
+```bash
+# Basic implosion with auto-detection
+markitect md-implode exploded-directory/
+
+# Specify output file
+markitect md-implode exploded-directory/ --output reassembled.md
+
+# Force specific variant (overrides auto-detection)
+markitect md-implode exploded-directory/ --force-variant hierarchical
+
+# Preview without creating output
+markitect md-implode exploded-directory/ --dry-run --verbose
+```
+
+**Auto-Detection Features:**
+- **Manifest-based**: Highest confidence when `manifest.md` exists
+- **Pattern Recognition**: Detects numbered directories (01_, 02_, etc.)
+- **Semantic Analysis**: Recognizes semantic directory names
+- **Fallback Logic**: Defaults to flat variant when patterns are unclear
+
+### Working with Manifests
+
+Manifests ensure perfect reversibility:
+
+```yaml
+---
+explosion_type: hierarchical
+original_file: user-guide.md
+created: 2025-10-14T10:00:00Z
+markitect_version: 1.0.0
+preservation:
+  front_matter: true
+  section_order: true
+  heading_levels: true
+structure:
+  - type: h1
+    title: Introduction
+    path: 01_introduction/index.md
+    order: 1
+    level: 1
+---
+
+# Explosion Manifest
+
+This manifest was generated during the explosion of `user-guide.md` using the hierarchical variant.
+```
+
+## Advanced Packaging
+
+### MDZ (Markdown Zip) Packages
+
+Create self-contained packages with embedded assets:
+
+```bash
+# Create MDZ package
+markitect md-package create document.md --format mdz --output document.mdz
+
+# Extract MDZ package
+markitect md-package extract document.mdz --output extracted/
+
+# View package information
+markitect md-package info document.mdz --verbose
+```
+
+**MDZ Package Structure:**
+```
+document.mdz (ZIP file)
+├── content.md          # Main content with rewritten asset paths
+├── assets/             # Embedded assets
+│   ├── images/
+│   ├── stylesheets/
+│   └── documents/
+└── package.json        # Package metadata and manifest
+```
+
+**Features:**
+- **Asset Embedding**: Automatic discovery and inclusion of referenced assets
+- **Path Rewriting**: Asset paths updated for package-internal references
+- **Compression**: ZIP compression for efficient storage
+- **Integrity Validation**: SHA-256 checksums for all assets
+- **Cross-platform**: Works consistently across operating systems
+
+### Asset Management
+
+MarkiTect automatically handles assets during packaging:
+
+```bash
+# Package with custom compression
+markitect md-package create document.md --compression 9
+
+# Package with verbose asset information
+markitect md-package create document.md --verbose
+```
+
+**Supported Asset Types:**
+- **Images**: PNG, JPG, GIF, SVG, WebP
+- **Documents**: PDF, DOC, DOCX, additional Markdown files
+- **Stylesheets**: CSS files
+- **Scripts**: JavaScript files
+- **Archives**: ZIP, TAR files
+- **Custom**: Any file referenced in markdown content
+
+## Transclusion Engine
+
+### MDT (Markdown Transcluded) Templates
+
+Create dynamic documents with transclusion directives:
+
+```markdown
+# {{title}}
+
+Author: {{author}}
+Version: {{version}}
+
+{{include "introduction.md"}}
+
+## Features
+
+{{include "features.md"}}
+
+{{if debug}}
+**Debug Information**: Additional development details
+{{endif}}
+```
+
+### Processing Templates
+
+```bash
+# Process template with variables
+markitect md-transclude process template.mdt --variables config.json
+
+# Validate template syntax
+markitect md-transclude validate template.mdt --verbose
+
+# Process with custom output
+markitect md-transclude process template.mdt --output final-document.md
+```
+
+**Variables File Example (config.json):**
+```json
+{
+  "title": "Advanced User Guide",
+  "author": "Documentation Team",
+  "version": "2.1.0",
+  "debug": false
+}
+```
+
+### Transclusion Directives
+
+**File Inclusion:**
+```markdown
+{{include "path/to/file.md"}}
+{{include "relative-file.md"}}
+```
+
+**Variable Substitution:**
+```markdown
+Welcome to {{product_name}} version {{version}}!
+```
+
+**Conditional Content:**
+```markdown
+{{if development_mode}}
+This section only appears in development builds.
+{{endif}}
+```
+
+## Best Practices
+
+### Project Organization
+
+**Recommended Directory Structure:**
+```
+project/
+├── source/
+│   └── main-document.md       # Original document
+├── exploded/
+│   ├── manifest.md            # Generated during explosion
+│   ├── 01_introduction/
+│   ├── 02_getting_started/
+│   └── 03_advanced_topics/
+├── templates/
+│   ├── document-template.mdt
+│   ├── variables.json
+│   └── includes/
+└── packages/
+    ├── document.mdz           # Packaged versions
+    └── document-v2.mdz
+```
+
+### Workflow Recommendations
+
+**1. Large Document Workflow:**
+```bash
+# Step 1: Explode for editing
+markitect md-explode large-doc.md --variant hierarchical --create-manifest
+
+# Step 2: Collaborative editing
+# ... multiple authors edit sections in parallel ...
+
+# Step 3: Regular validation
+markitect md-implode large-doc.mdd --dry-run --verbose
+
+# Step 4: Final assembly
+markitect md-implode large-doc.mdd --output final-document.md
+
+# Step 5: Package for distribution
+markitect md-package create final-document.md --format mdz
+```
+
+**2. Template-based Workflow:**
+```bash
+# Step 1: Create template structure
+markitect md-transclude validate base-template.mdt
+
+# Step 2: Generate variants
+markitect md-transclude process base-template.mdt --variables prod-config.json --output prod-doc.md
+markitect md-transclude process base-template.mdt --variables dev-config.json --output dev-doc.md
+
+# Step 3: Package final versions
+markitect md-package create prod-doc.md --format mdz
+```
+
+### Performance Optimization
+
+**For Large Documents (1000+ sections):**
+- Use hierarchical variant with appropriate `--max-depth`
+- Enable verbose mode to monitor progress
+- Consider processing in stages for very large documents
+
+**For Asset-Heavy Documents:**
+- Verify asset discovery with `--dry-run` first
+- Use appropriate compression levels (6-9 for best compression)
+- Monitor package sizes and optimize assets before packaging
+
+### Version Control Integration
+
+**Git Integration:**
+```bash
+# Add exploded structure to version control
+git add document.mdd/
+
+# Ignore generated packages
+echo "*.mdz" >> .gitignore
+echo "*.processed.md" >> .gitignore
+
+# Track manifest files for reproducibility
+git add */manifest.md
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**1. "Failed to detect variant" Error**
+```bash
+# Solution: Manually specify variant
+markitect md-implode directory/ --force-variant hierarchical
+```
+
+**2. "Circular reference detected" Error**
+```bash
+# Solution: Check include directives in templates
+markitect md-transclude validate template.mdt --verbose
+```
+
+**3. "Asset not found" Error**
+```bash
+# Solution: Verify asset paths and existence
+markitect md-package create document.md --dry-run --verbose
+```
+
+**4. "Package corruption" Error**
+```bash
+# Solution: Extract and verify package contents
+markitect md-package extract package.mdz --verbose
+markitect md-package info package.mdz
+```
+
+### Debug Mode
+
+Enable debug mode for detailed troubleshooting:
+
+```bash
+# Set debug environment variable
+export MARKITECT_DEBUG=1
+
+# Or use debug flag (if available)
+markitect --debug md-explode document.md --verbose
+```
+
+### Performance Issues
+
+**Slow Operations:**
+1. **Check document size**: Very large documents (10MB+) may need processing time
+2. **Asset count**: Documents with 100+ assets require additional processing
+3. **Network assets**: Remove external URLs that cause timeouts
+4. **Depth limits**: Reduce `--max-depth` for hierarchical variant
+
+**Memory Usage:**
+1. **Large documents**: Process in chunks if memory issues occur
+2. **Asset optimization**: Optimize images and assets before packaging
+3. **Temporary files**: Ensure sufficient disk space for operations
+
+## Migration from Legacy Systems
+
+### From Simple Directory Structures
+
+```bash
+# If you have manually organized directories
+markitect md-implode manual-directory/ --force-variant flat --output combined.md
+```
+
+### From Other Documentation Systems
+
+```bash
+# Convert existing structures to MarkiTect format
+markitect md-explode external-doc.md --variant semantic --create-manifest
+# Then use the exploded structure with MarkiTect workflows
+```
+
+## Advanced Configuration
+
+### Environment Variables
+
+```bash
+# Set default variant
+export MARKITECT_DEFAULT_VARIANT=hierarchical
+
+# Set default compression level
+export MARKITECT_DEFAULT_COMPRESSION=6
+
+# Enable debug mode
+export MARKITECT_DEBUG=1
+```
+
+### Custom Asset Types
+
+To handle custom asset types, ensure they're properly referenced in your markdown:
+
+```markdown
+<!-- Standard references that will be auto-detected -->
+![Image](./assets/diagram.png)
+[Document](./resources/specification.pdf)
+<script src="./scripts/interactive.js"></script>
+<link rel="stylesheet" href="./styles/custom.css">
+```
+
+---
+
+## Conclusion
+
+MarkiTect's explode-implode system provides powerful capabilities for managing complex documentation projects. Whether you're working with large technical documents, collaborative writing projects, or template-based content generation, this system offers the flexibility and reliability you need.
+
+For more information:
+- [API Documentation](../api/explode-variants.md)
+- [Packaging System Guide](../advanced_packaging.md)
+- [Migration Guide](migration-guide.md)
+
+**Getting Help:**
+- Use `--help` flag with any command for detailed options
+- Enable `--verbose` mode for debugging information
+- Check the troubleshooting section for common issues
+
+Happy documenting! 📚
--- a/docs/user-guides/migration-guide.md
+++ b/docs/user-guides/migration-guide.md
@@ -0,0 +1,762 @@
+# Migration Guide: Upgrading to MarkiTect's Enhanced Explode-Implode System
+
+**Step-by-step guide for migrating existing documentation workflows to MarkiTect's advanced explode-implode and packaging systems**
+
+## Table of Contents
+
+1. [Overview](#overview)
+2. [Pre-Migration Assessment](#pre-migration-assessment)
+3. [Migration Scenarios](#migration-scenarios)
+4. [Step-by-Step Migration](#step-by-step-migration)
+5. [Validation and Testing](#validation-and-testing)
+6. [Common Issues and Solutions](#common-issues-and-solutions)
+7. [Rollback Procedures](#rollback-procedures)
+8. [Best Practices](#best-practices)
+
+---
+
+## Overview
+
+This guide helps you migrate from:
+- **Basic MarkiTect installations** to the enhanced explode-implode system
+- **Manual directory organization** to structured variant-based workflows
+- **Legacy documentation systems** to MarkiTect's integrated approach
+- **Simple file-based workflows** to advanced packaging and templating
+
+### What's New
+
+**Enhanced Features Available After Migration:**
+- 🔄 **Three organizational variants** (flat, hierarchical, semantic)
+- 📦 **MDZ packaging** with asset management
+- 🔗 **Template-based transclusion** (MDT format)
+- 🤖 **Auto-detection** of exploded structures
+- 📊 **Manifest-based reversibility**
+- ⚡ **CLI command integration**
+
+### Compatibility
+
+**Supported Migration Paths:**
+- MarkiTect 0.x → 1.0+ (full migration required)
+- Manual documentation workflows → MarkiTect structured workflows
+- GitBook, MkDocs, Sphinx → MarkiTect (with content adaptation)
+- Simple markdown files → Advanced explode-implode workflows
+
+## Pre-Migration Assessment
+
+### Step 1: Inventory Current Setup
+
+Run this assessment to understand your current state:
+
+```bash
+# Check your current MarkiTect version
+markitect version
+
+# List your current documentation structure
+find . -name "*.md" -type f | head -20
+
+# Check for existing MarkiTect files
+find . -name "*.mdd" -o -name "manifest.md" -o -name "*.mdz"
+
+# Assess document sizes (identifies candidates for explosion)
+find . -name "*.md" -exec wc -l {} \; | sort -nr | head -10
+```
+
+### Step 2: Backup Current Setup
+
+**Critical: Always backup before migration**
+
+```bash
+# Create timestamped backup
+backup_dir="backup-$(date +%Y%m%d-%H%M%S)"
+mkdir "$backup_dir"
+
+# Backup all markdown and config files
+cp -r docs/ "$backup_dir/" 2>/dev/null || true
+cp -r *.md "$backup_dir/" 2>/dev/null || true
+cp -r .markitect* "$backup_dir/" 2>/dev/null || true
+
+echo "Backup created in: $backup_dir"
+```
+
+### Step 3: Assess Migration Complexity
+
+Use this checklist to determine your migration path:
+
+**Simple Migration (1-2 hours):**
+- [ ] Small documentation set (< 10 files)
+- [ ] Basic markdown structure
+- [ ] No custom build processes
+- [ ] Willing to use recommended variants
+
+**Moderate Migration (4-8 hours):**
+- [ ] Medium documentation set (10-50 files)
+- [ ] Some organizational structure exists
+- [ ] Basic build/CI processes
+- [ ] Custom asset management needs
+
+**Complex Migration (1-3 days):**
+- [ ] Large documentation set (50+ files)
+- [ ] Complex existing organization
+- [ ] Extensive build/CI integration
+- [ ] Custom tooling and workflows
+
+## Migration Scenarios
+
+### Scenario A: From Basic MarkiTect
+
+**Situation:** You have MarkiTect installed but haven't used explode-implode features.
+
+**Migration Steps:**
+
+1. **Update MarkiTect:**
+   ```bash
+   pip install --upgrade markitect
+   markitect version  # Verify 1.0+
+   ```
+
+2. **Test new commands:**
+   ```bash
+   # Verify new commands are available
+   markitect md-explode --help
+   markitect md-package --help
+   markitect md-transclude --help
+   ```
+
+3. **Choose your first document:**
+   ```bash
+   # Start with a medium-sized document
+   markitect md-explode your-document.md --variant hierarchical --dry-run
+   ```
+
+4. **Perform first explosion:**
+   ```bash
+   markitect md-explode your-document.md --variant hierarchical --create-manifest
+   ```
+
+### Scenario B: From Manual Directory Organization
+
+**Situation:** You manually organize markdown files in directories.
+
+**Current Structure Example:**
+```
+docs/
+├── intro/
+│   ├── overview.md
+│   └── getting-started.md
+├── tutorials/
+│   ├── basic-usage.md
+│   └── advanced-features.md
+└── reference/
+    └── api.md
+```
+
+**Migration Steps:**
+
+1. **Assess current organization:**
+   ```bash
+   # Analyze your structure
+   tree docs/ > current-structure.txt
+
+   # Count sections across files
+   grep -r "^#" docs/ | wc -l
+   ```
+
+2. **Choose migration strategy:**
+
+   **Option A: Convert to Semantic Variant**
+   ```bash
+   # Combines files into semantic structure
+   markitect md-implode docs/ --force-variant semantic --output combined-docs.md
+
+   # Then re-explode with proper manifest
+   markitect md-explode combined-docs.md --variant semantic --create-manifest
+   ```
+
+   **Option B: Preserve as Flat Structure**
+   ```bash
+   # Convert to flat variant maintaining file boundaries
+   markitect md-implode docs/ --force-variant flat --output combined-docs.md
+   markitect md-explode combined-docs.md --variant flat --create-manifest
+   ```
+
+3. **Validate result:**
+   ```bash
+   # Check the new structure
+   tree combined-docs.mdd/
+
+   # Test round-trip
+   markitect md-implode combined-docs.mdd/ --output test-rebuild.md
+   diff combined-docs.md test-rebuild.md
+   ```
+
+### Scenario C: From GitBook
+
+**Situation:** Migrating from GitBook to MarkiTect.
+
+**GitBook Structure:**
+```
+docs/
+├── SUMMARY.md
+├── README.md
+├── chapter1/
+│   ├── README.md
+│   └── section1.md
+└── chapter2/
+    └── README.md
+```
+
+**Migration Steps:**
+
+1. **Convert GitBook structure:**
+   ```bash
+   # Create conversion script
+   cat > convert-gitbook.py << 'EOF'
+   #!/usr/bin/env python3
+   import os
+   from pathlib import Path
+
+   def convert_gitbook_to_single_md():
+       # Parse SUMMARY.md to understand structure
+       # Combine files in order
+       # Generate single markdown file
+       pass
+
+   if __name__ == "__main__":
+       convert_gitbook_to_single_md()
+   EOF
+
+   python convert-gitbook.py
+   ```
+
+2. **Apply MarkiTect structure:**
+   ```bash
+   markitect md-explode converted-document.md --variant hierarchical --create-manifest
+   ```
+
+### Scenario D: From Legacy Documentation Systems
+
+**Situation:** Migrating from Sphinx, MkDocs, or similar systems.
+
+**Migration Approach:**
+
+1. **Export to markdown:**
+   ```bash
+   # For Sphinx (using pandoc)
+   find . -name "*.rst" -exec pandoc {} -f rst -t markdown -o {}.md \;
+
+   # For MkDocs, files may already be markdown
+   # Check mkdocs.yml for structure information
+   ```
+
+2. **Combine and structure:**
+   ```bash
+   # Create combined document preserving hierarchy
+   # (You may need custom script based on your system)
+
+   # Then apply MarkiTect organization
+   markitect md-explode legacy-combined.md --variant hierarchical
+   ```
+
+## Step-by-Step Migration
+
+### Phase 1: Preparation (30 minutes)
+
+1. **Environment Setup:**
+   ```bash
+   # Ensure latest version
+   pip install --upgrade markitect
+
+   # Verify installation
+   markitect version
+   markitect md-explode --help
+   markitect md-package --help
+   ```
+
+2. **Create working directory:**
+   ```bash
+   mkdir markitect-migration
+   cd markitect-migration
+
+   # Copy source documents
+   cp -r /path/to/current/docs ./source-docs/
+   ```
+
+3. **Assess document complexity:**
+   ```bash
+   # Find largest documents (good candidates for explosion)
+   find source-docs/ -name "*.md" -exec wc -l {} \; | sort -nr > document-sizes.txt
+
+   # Identify documents with many sections
+   find source-docs/ -name "*.md" -exec bash -c 'echo -n "{}: "; grep -c "^#" "{}"' \; > section-counts.txt
+   ```
+
+### Phase 2: Small-Scale Testing (1 hour)
+
+1. **Choose pilot document:**
+   ```bash
+   # Select medium-sized document from analysis
+   pilot_doc="source-docs/user-guide.md"  # Replace with your choice
+   ```
+
+2. **Test explosion variants:**
+   ```bash
+   # Test all variants with dry-run
+   markitect md-explode "$pilot_doc" --variant flat --dry-run --verbose
+   markitect md-explode "$pilot_doc" --variant hierarchical --dry-run --verbose
+   markitect md-explode "$pilot_doc" --variant semantic --dry-run --verbose
+   ```
+
+3. **Perform pilot explosion:**
+   ```bash
+   # Choose best variant and execute
+   markitect md-explode "$pilot_doc" --variant hierarchical --create-manifest --output pilot-exploded/
+
+   # Validate round-trip
+   markitect md-implode pilot-exploded/ --output pilot-rebuilt.md
+   diff "$pilot_doc" pilot-rebuilt.md
+   ```
+
+### Phase 3: Full Migration (2-4 hours)
+
+1. **Process all documents:**
+   ```bash
+   # Create batch processing script
+   cat > migrate-all.sh << 'EOF'
+   #!/bin/bash
+
+   # Configure migration settings
+   VARIANT="hierarchical"  # Change as needed
+   SOURCE_DIR="source-docs"
+   OUTPUT_DIR="migrated-docs"
+
+   mkdir -p "$OUTPUT_DIR"
+
+   # Process each markdown file
+   find "$SOURCE_DIR" -name "*.md" | while read -r file; do
+       echo "Processing: $file"
+
+       # Generate output path
+       relative_path=$(realpath --relative-to="$SOURCE_DIR" "$file")
+       basename_no_ext=$(basename "$file" .md)
+       output_subdir="$OUTPUT_DIR/${basename_no_ext}.mdd"
+
+       # Explode document
+       markitect md-explode "$file" --variant "$VARIANT" --create-manifest --output "$output_subdir"
+
+       # Validate
+       temp_rebuilt="/tmp/${basename_no_ext}-rebuilt.md"
+       markitect md-implode "$output_subdir" --output "$temp_rebuilt"
+
+       if diff -q "$file" "$temp_rebuilt" > /dev/null; then
+           echo "✓ Migration successful: $file"
+       else
+           echo "⚠ Migration issues detected: $file"
+           echo "   Check: $temp_rebuilt vs $file"
+       fi
+   done
+   EOF
+
+   chmod +x migrate-all.sh
+   ./migrate-all.sh
+   ```
+
+2. **Validate migration results:**
+   ```bash
+   # Check all migrations
+   find migrated-docs/ -name "manifest.md" | wc -l  # Should equal input document count
+
+   # Test random sampling
+   find migrated-docs/ -name "*.mdd" | shuf | head -3 | while read -r exploded_dir; do
+       echo "Testing: $exploded_dir"
+       temp_rebuilt="/tmp/$(basename "$exploded_dir" .mdd)-test.md"
+       markitect md-implode "$exploded_dir" --output "$temp_rebuilt"
+       echo "Rebuilt to: $temp_rebuilt"
+   done
+   ```
+
+### Phase 4: Integration and Packaging (1-2 hours)
+
+1. **Set up packaging workflow:**
+   ```bash
+   # Create packaging script
+   cat > package-docs.sh << 'EOF'
+   #!/bin/bash
+
+   MIGRATED_DIR="migrated-docs"
+   PACKAGES_DIR="packages"
+
+   mkdir -p "$PACKAGES_DIR"
+
+   # Package each exploded document
+   find "$MIGRATED_DIR" -name "*.mdd" | while read -r exploded_dir; do
+       basename_no_ext=$(basename "$exploded_dir" .mdd)
+
+       # First implode to single document
+       temp_doc="/tmp/${basename_no_ext}.md"
+       markitect md-implode "$exploded_dir" --output "$temp_doc"
+
+       # Then package
+       output_package="$PACKAGES_DIR/${basename_no_ext}.mdz"
+       markitect md-package create "$temp_doc" --format mdz --output "$output_package"
+
+       echo "Packaged: $output_package"
+
+       # Verify package
+       markitect md-package info "$output_package"
+   done
+   EOF
+
+   chmod +x package-docs.sh
+   ./package-docs.sh
+   ```
+
+2. **Set up templating (optional):**
+   ```bash
+   # If you have template needs, create MDT templates
+   mkdir templates/
+
+   # Example template creation
+   cat > templates/user-guide-template.mdt << 'EOF'
+   # {{title}}
+
+   **Version:** {{version}}
+   **Author:** {{author}}
+   **Updated:** {{date}}
+
+   {{include "introduction.md"}}
+
+   ## Features
+
+   {{include "features.md"}}
+
+   {{if include_advanced}}
+   ## Advanced Topics
+
+   {{include "advanced.md"}}
+   {{endif}}
+   EOF
+
+   # Test template
+   cat > templates/variables.json << 'EOF'
+   {
+     "title": "Migrated User Guide",
+     "version": "2.0.0",
+     "author": "Documentation Team",
+     "date": "2025-10-14",
+     "include_advanced": true
+   }
+   EOF
+
+   markitect md-transclude validate templates/user-guide-template.mdt
+   ```
+
+## Validation and Testing
+
+### Automated Validation
+
+```bash
+# Create validation script
+cat > validate-migration.sh << 'EOF'
+#!/bin/bash
+
+echo "=== Migration Validation Report ==="
+
+# Count original files
+orig_count=$(find source-docs/ -name "*.md" | wc -l)
+echo "Original documents: $orig_count"
+
+# Count migrated structures
+migrated_count=$(find migrated-docs/ -name "*.mdd" | wc -l)
+echo "Migrated structures: $migrated_count"
+
+# Count packages
+package_count=$(find packages/ -name "*.mdz" | wc -l 2>/dev/null || echo "0")
+echo "Created packages: $package_count"
+
+# Test round-trip integrity
+echo
+echo "=== Round-trip Tests ==="
+failed_tests=0
+
+find migrated-docs/ -name "*.mdd" | head -5 | while read -r exploded_dir; do
+    basename_no_ext=$(basename "$exploded_dir" .mdd)
+    original_file="source-docs/${basename_no_ext}.md"
+
+    if [ -f "$original_file" ]; then
+        temp_rebuilt="/tmp/${basename_no_ext}-validation.md"
+        markitect md-implode "$exploded_dir" --output "$temp_rebuilt"
+
+        if diff -q "$original_file" "$temp_rebuilt" > /dev/null; then
+            echo "✓ PASS: $basename_no_ext"
+        else
+            echo "✗ FAIL: $basename_no_ext"
+            failed_tests=$((failed_tests + 1))
+        fi
+    fi
+done
+
+if [ $failed_tests -eq 0 ]; then
+    echo
+    echo "🎉 All validation tests passed!"
+else
+    echo
+    echo "⚠ $failed_tests validation tests failed. Review differences manually."
+fi
+EOF
+
+chmod +x validate-migration.sh
+./validate-migration.sh
+```
+
+### Manual Validation Checklist
+
+**Document Structure:**
+- [ ] All original sections are preserved
+- [ ] Heading hierarchy is maintained
+- [ ] Code blocks are intact
+- [ ] Links and references work correctly
+- [ ] Images and assets are accessible
+
+**Functionality:**
+- [ ] Explode operations complete without errors
+- [ ] Implode operations produce identical content
+- [ ] Variant detection works correctly
+- [ ] Packaging creates valid MDZ files
+- [ ] Templates process correctly (if used)
+
+**Integration:**
+- [ ] CLI commands integrate with existing workflows
+- [ ] Build processes adapt successfully
+- [ ] Version control handles new file structures
+- [ ] Team members can use new workflows
+
+## Common Issues and Solutions
+
+### Issue 1: "Failed to detect variant" Error
+
+**Symptoms:**
+```
+Error: Failed to detect variant for directory 'docs/'
+```
+
+**Solutions:**
+```bash
+# Option 1: Force specific variant
+markitect md-implode docs/ --force-variant flat
+
+# Option 2: Create proper structure first
+markitect md-explode combined-doc.md --variant hierarchical --create-manifest
+```
+
+### Issue 2: Large Document Memory Issues
+
+**Symptoms:**
+- Slow processing of large documents
+- Out of memory errors
+
+**Solutions:**
+```bash
+# Split large documents manually first
+split -l 1000 large-document.md split-doc-
+
+# Process parts separately
+for part in split-doc-*; do
+    markitect md-explode "$part" --variant flat
+done
+
+# Or use streaming approach (if available)
+# markitect md-explode large-document.md --streaming --variant hierarchical
+```
+
+### Issue 3: Asset Path Issues
+
+**Symptoms:**
+- Images don't display after migration
+- Broken asset references
+
+**Solutions:**
+```bash
+# Check asset discovery
+markitect md-package create document.md --dry-run --verbose
+
+# Fix relative paths in markdown
+sed -i 's|\.\./assets/|./assets/|g' document.md
+
+# Verify asset packaging
+markitect md-package info document.mdz --verbose
+```
+
+### Issue 4: Manifest Corruption
+
+**Symptoms:**
+```
+Error: Invalid manifest format in 'manifest.md'
+```
+
+**Solutions:**
+```bash
+# Backup corrupted manifest
+cp manifest.md manifest.md.backup
+
+# Regenerate manifest
+markitect md-explode original-document.md --variant hierarchical --create-manifest --force
+
+# Or manually fix YAML frontmatter
+```
+
+### Issue 5: Template Processing Errors
+
+**Symptoms:**
+```
+Error: Circular reference detected in template
+```
+
+**Solutions:**
+```bash
+# Validate template structure
+markitect md-transclude validate template.mdt --verbose
+
+# Check for circular includes
+grep -r "{{include" templates/ | sort
+
+# Fix circular references by restructuring includes
+```
+
+## Rollback Procedures
+
+### Complete Rollback
+
+```bash
+# Restore from backup
+backup_dir="backup-20251014-100000"  # Your backup timestamp
+
+# Stop and restore
+rm -rf migrated-docs/ packages/ templates/
+cp -r "$backup_dir/"* ./
+
+echo "Rollback completed. You're back to the original state."
+```
+
+### Selective Rollback
+
+```bash
+# Rollback specific documents only
+problem_docs=("user-guide" "api-reference")
+
+for doc in "${problem_docs[@]}"; do
+    # Remove migrated version
+    rm -rf "migrated-docs/${doc}.mdd"
+
+    # Restore original
+    cp "$backup_dir/${doc}.md" ./
+
+    echo "Rolled back: $doc"
+done
+```
+
+### Partial Migration Approach
+
+If full migration is problematic, use a gradual approach:
+
+```bash
+# Phase 1: Migrate only small documents
+find source-docs/ -name "*.md" -exec wc -l {} \; | awk '$1 < 200' | cut -d: -f2 | while read -r file; do
+    markitect md-explode "$file" --variant flat --create-manifest
+done
+
+# Phase 2: Migrate medium documents (when comfortable)
+# Phase 3: Migrate large documents (with assistance)
+```
+
+## Best Practices
+
+### Migration Planning
+
+1. **Start Small:** Begin with 1-2 documents to understand the process
+2. **Test Variants:** Try different variants to find the best fit
+3. **Validate Early:** Check round-trip integrity frequently
+4. **Backup Everything:** Multiple backups at different stages
+5. **Document Changes:** Keep migration notes for team members
+
+### Workflow Integration
+
+```bash
+# Add to Makefile
+migrate-docs:
+	./migrate-all.sh
+
+validate-migration:
+	./validate-migration.sh
+
+package-docs:
+	./package-docs.sh
+
+clean-migration:
+	rm -rf migrated-docs/ packages/
+	git checkout -- source-docs/
+```
+
+### Team Collaboration
+
+1. **Training:** Ensure team understands new commands
+2. **Documentation:** Update team docs with new workflows
+3. **Gradual Adoption:** Allow parallel workflows during transition
+4. **Support:** Designate migration champions to help others
+
+### Maintenance
+
+```bash
+# Regular validation script
+cat > .github/workflows/validate-docs.yml << 'EOF'
+name: Validate Documentation Structure
+
+on: [push, pull_request]
+
+jobs:
+  validate:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - name: Install MarkiTect
+      run: pip install markitect
+    - name: Validate structures
+      run: |
+        find . -name "*.mdd" | while read -r dir; do
+          echo "Validating: $dir"
+          markitect md-implode "$dir" --dry-run --verbose
+        done
+EOF
+```
+
+---
+
+## Migration Checklist
+
+### Pre-Migration
+- [ ] Backup all documentation
+- [ ] Install latest MarkiTect version
+- [ ] Assess current documentation structure
+- [ ] Choose migration strategy
+- [ ] Test with pilot documents
+
+### During Migration
+- [ ] Process documents systematically
+- [ ] Validate each step
+- [ ] Document any issues encountered
+- [ ] Test package creation
+- [ ] Verify asset handling
+
+### Post-Migration
+- [ ] Update team documentation
+- [ ] Train team members on new workflows
+- [ ] Update CI/CD processes
+- [ ] Establish maintenance procedures
+- [ ] Plan rollback procedures (just in case)
+
+**Migration Support:** For complex migrations or issues not covered in this guide, create detailed issue reports with your specific setup and requirements.
+
+---
+
+**Version:** 1.0.0
+**Last Updated:** 2025-10-14
+**Compatibility:** MarkiTect 1.0+