markitect-main/docs/user-guides/explode-implode-complete-guide.md

Complete Guide to MarkiTect's Explode-Implode System

A comprehensive guide to MarkiTect's powerful document decomposition and recomposition capabilities

Table of Contents

1. Overview
2. Getting Started
3. Understanding Variants
4. Basic Operations
5. Advanced Packaging
6. Transclusion Engine
7. Best Practices
8. Troubleshooting

---

Overview

MarkiTect's explode-implode system provides sophisticated capabilities for breaking down large markdown documents into manageable components and reassembling them with perfect fidelity. This system supports multiple organizational strategies (variants) and includes advanced packaging features for self-contained documents.

Key Capabilities

• 📂 Three Organizational Variants: Flat, Hierarchical, and Semantic structures
• 🔄 Perfect Reversibility: Manifest-based system ensures lossless round-trips
• 🤖 Auto-Detection: Intelligent detection of existing exploded structures
• 📦 Advanced Packaging: Self-contained MDZ packages with embedded assets
• 🔗 Transclusion Engine: Template-based document generation
• 📊 Asset Management: Automated discovery and integrity validation

When to Use Explode-Implode

Ideal Use Cases:

• Large documentation projects (100+ pages)
• Multi-author collaborative writing
• Modular content that needs reorganization
• Documentation requiring asset management
• Template-based document generation
• Legacy document modernization

Getting Started

Prerequisites

Ensure you have MarkiTect installed and configured:

# Verify installation
markitect version

# Check that explode-implode commands are available
markitect md-explode --help
markitect md-implode --help

Quick Start Example

Let's explode a large document and then implode it back:

# 1. Explode a document using hierarchical variant
markitect md-explode large-document.md --variant hierarchical --output exploded/

# 2. Review the exploded structure
tree exploded/

# 3. Make edits to individual sections
# ... edit files in exploded/ directory ...

# 4. Implode back to a single document
markitect md-implode exploded/ --output reconstructed-document.md

# 5. Verify the result
diff large-document.md reconstructed-document.md

Understanding Variants

MarkiTect provides three organizational variants, each optimized for different use cases:

1. Flat Variant (flat)

Structure: All sections as peer files in a single directory

document.mdd/
├── manifest.md
├── introduction.md
├── getting_started.md
├── advanced_features.md
└── conclusion.md

Best For:

• Simple documents with minimal hierarchy
• Quick reorganization of content
• Linear reading flows
• Collaborative editing of independent sections

Command Example:

markitect md-explode document.md --variant flat

2. Hierarchical Variant (hierarchical)

Structure: Nested directories reflecting document hierarchy

document.mdd/
├── manifest.md
├── 01_introduction/
│   └── index.md
├── 02_getting_started/
│   ├── index.md
│   ├── 01_installation.md
│   └── 02_configuration.md
└── 03_advanced_features/
    ├── index.md
    └── 01_plugins.md

Best For:

• Complex documents with deep nesting
• Technical documentation with logical groupings
• Books or guides with chapter/section structure
• Content that benefits from visual organization

Command Example:

markitect md-explode document.md --variant hierarchical --max-depth 3

3. Semantic Variant (semantic)

Structure: Meaningfully-named directories based on content

document.mdd/
├── manifest.md
├── introduction/
│   └── overview.md
├── tutorials/
│   ├── getting_started.md
│   └── advanced_usage.md
├── reference/
│   └── api_documentation.md
└── appendices/
    └── troubleshooting.md

Best For:

• Documentation with distinct content types
• Knowledge bases requiring categorical organization
• Content management workflows
• SEO-optimized content structures

Command Example:

markitect md-explode document.md --variant semantic

Basic Operations

Exploding Documents

The md-explode command breaks down documents into components:

# Basic explosion with auto-selected variant
markitect md-explode document.md

# Specify variant and output directory
markitect md-explode document.md --variant hierarchical --output my-exploded-doc/

# Create manifest for reversibility (recommended)
markitect md-explode document.md --variant flat --create-manifest

# Dry run to preview structure
markitect md-explode document.md --variant semantic --dry-run --verbose

Key Options:

• --variant: Choose organizational strategy (flat, hierarchical, semantic)
• --output: Specify output directory (defaults to {filename}.mdd)
• --max-depth: Limit nesting depth for hierarchical variant
• --create-manifest: Generate manifest.md for perfect reversibility
• --dry-run: Preview operations without making changes
• --verbose: Detailed output for troubleshooting

Imploding Documents

The md-implode command reassembles exploded structures:

# Basic implosion with auto-detection
markitect md-implode exploded-directory/

# Specify output file
markitect md-implode exploded-directory/ --output reassembled.md

# Force specific variant (overrides auto-detection)
markitect md-implode exploded-directory/ --force-variant hierarchical

# Preview without creating output
markitect md-implode exploded-directory/ --dry-run --verbose

Auto-Detection Features:

• Manifest-based: Highest confidence when manifest.md exists
• Pattern Recognition: Detects numbered directories (01_, 02_, etc.)
• Semantic Analysis: Recognizes semantic directory names
• Fallback Logic: Defaults to flat variant when patterns are unclear

Working with Manifests

Manifests ensure perfect reversibility:

---
explosion_type: hierarchical
original_file: user-guide.md
created: 2025-10-14T10:00:00Z
markitect_version: 1.0.0
preservation:
  front_matter: true
  section_order: true
  heading_levels: true
structure:
  - type: h1
    title: Introduction
    path: 01_introduction/index.md
    order: 1
    level: 1
---

# Explosion Manifest

This manifest was generated during the explosion of `user-guide.md` using the hierarchical variant.

Advanced Packaging

MDZ (Markdown Zip) Packages

Create self-contained packages with embedded assets:

# Create MDZ package
markitect md-package create document.md --format mdz --output document.mdz

# Extract MDZ package
markitect md-package extract document.mdz --output extracted/

# View package information
markitect md-package info document.mdz --verbose

MDZ Package Structure:

document.mdz (ZIP file)
├── content.md          # Main content with rewritten asset paths
├── assets/             # Embedded assets
│   ├── images/
│   ├── stylesheets/
│   └── documents/
└── package.json        # Package metadata and manifest

Features:

• Asset Embedding: Automatic discovery and inclusion of referenced assets
• Path Rewriting: Asset paths updated for package-internal references
• Compression: ZIP compression for efficient storage
• Integrity Validation: SHA-256 checksums for all assets
• Cross-platform: Works consistently across operating systems

Asset Management

MarkiTect automatically handles assets during packaging:

# Package with custom compression
markitect md-package create document.md --compression 9

# Package with verbose asset information
markitect md-package create document.md --verbose

Supported Asset Types:

• Images: PNG, JPG, GIF, SVG, WebP
• Documents: PDF, DOC, DOCX, additional Markdown files
• Stylesheets: CSS files
• Scripts: JavaScript files
• Archives: ZIP, TAR files
• Custom: Any file referenced in markdown content

Transclusion Engine

MDT (Markdown Transcluded) Templates

Create dynamic documents with transclusion directives:

# {{title}}

Author: {{author}}
Version: {{version}}

{{include "introduction.md"}}

## Features

{{include "features.md"}}

{{if debug}}
**Debug Information**: Additional development details
{{endif}}

Processing Templates

# Process template with variables
markitect md-transclude process template.mdt --variables config.json

# Validate template syntax
markitect md-transclude validate template.mdt --verbose

# Process with custom output
markitect md-transclude process template.mdt --output final-document.md

Variables File Example (config.json):

{
  "title": "Advanced User Guide",
  "author": "Documentation Team",
  "version": "2.1.0",
  "debug": false
}

Transclusion Directives

File Inclusion:

{{include "path/to/file.md"}}
{{include "relative-file.md"}}

Variable Substitution:

Welcome to {{product_name}} version {{version}}!

Conditional Content:

{{if development_mode}}
This section only appears in development builds.
{{endif}}

Best Practices

Project Organization

Recommended Directory Structure:

project/
├── source/
│   └── main-document.md       # Original document
├── exploded/
│   ├── manifest.md            # Generated during explosion
│   ├── 01_introduction/
│   ├── 02_getting_started/
│   └── 03_advanced_topics/
├── templates/
│   ├── document-template.mdt
│   ├── variables.json
│   └── includes/
└── packages/
    ├── document.mdz           # Packaged versions
    └── document-v2.mdz

Workflow Recommendations

1. Large Document Workflow:

# Step 1: Explode for editing
markitect md-explode large-doc.md --variant hierarchical --create-manifest

# Step 2: Collaborative editing
# ... multiple authors edit sections in parallel ...

# Step 3: Regular validation
markitect md-implode large-doc.mdd --dry-run --verbose

# Step 4: Final assembly
markitect md-implode large-doc.mdd --output final-document.md

# Step 5: Package for distribution
markitect md-package create final-document.md --format mdz

2. Template-based Workflow:

# Step 1: Create template structure
markitect md-transclude validate base-template.mdt

# Step 2: Generate variants
markitect md-transclude process base-template.mdt --variables prod-config.json --output prod-doc.md
markitect md-transclude process base-template.mdt --variables dev-config.json --output dev-doc.md

# Step 3: Package final versions
markitect md-package create prod-doc.md --format mdz

Performance Optimization

For Large Documents (1000+ sections):

• Use hierarchical variant with appropriate --max-depth
• Enable verbose mode to monitor progress
• Consider processing in stages for very large documents

For Asset-Heavy Documents:

• Verify asset discovery with --dry-run first
• Use appropriate compression levels (6-9 for best compression)
• Monitor package sizes and optimize assets before packaging

Version Control Integration

Git Integration:

# Add exploded structure to version control
git add document.mdd/

# Ignore generated packages
echo "*.mdz" >> .gitignore
echo "*.processed.md" >> .gitignore

# Track manifest files for reproducibility
git add */manifest.md

Troubleshooting

Common Issues

1. "Failed to detect variant" Error

# Solution: Manually specify variant
markitect md-implode directory/ --force-variant hierarchical

2. "Circular reference detected" Error

# Solution: Check include directives in templates
markitect md-transclude validate template.mdt --verbose

3. "Asset not found" Error

# Solution: Verify asset paths and existence
markitect md-package create document.md --dry-run --verbose

4. "Package corruption" Error

# Solution: Extract and verify package contents
markitect md-package extract package.mdz --verbose
markitect md-package info package.mdz

Debug Mode

Enable debug mode for detailed troubleshooting:

# Set debug environment variable
export MARKITECT_DEBUG=1

# Or use debug flag (if available)
markitect --debug md-explode document.md --verbose

Performance Issues

Slow Operations:

1. Check document size: Very large documents (10MB+) may need processing time
2. Asset count: Documents with 100+ assets require additional processing
3. Network assets: Remove external URLs that cause timeouts
4. Depth limits: Reduce --max-depth for hierarchical variant

Memory Usage:

1. Large documents: Process in chunks if memory issues occur
2. Asset optimization: Optimize images and assets before packaging
3. Temporary files: Ensure sufficient disk space for operations

Migration from Legacy Systems

From Simple Directory Structures

# If you have manually organized directories
markitect md-implode manual-directory/ --force-variant flat --output combined.md

From Other Documentation Systems

# Convert existing structures to MarkiTect format
markitect md-explode external-doc.md --variant semantic --create-manifest
# Then use the exploded structure with MarkiTect workflows

Advanced Configuration

Environment Variables

# Set default variant
export MARKITECT_DEFAULT_VARIANT=hierarchical

# Set default compression level
export MARKITECT_DEFAULT_COMPRESSION=6

# Enable debug mode
export MARKITECT_DEBUG=1

Custom Asset Types

To handle custom asset types, ensure they're properly referenced in your markdown:

<!-- Standard references that will be auto-detected -->
![Image](./assets/diagram.png)
[Document](./resources/specification.pdf)
<script src="./scripts/interactive.js"></script>
<link rel="stylesheet" href="./styles/custom.css">

---

Conclusion

MarkiTect's explode-implode system provides powerful capabilities for managing complex documentation projects. Whether you're working with large technical documents, collaborative writing projects, or template-based content generation, this system offers the flexibility and reliability you need.

For more information:

• API Documentation
• Packaging System Guide
• Migration Guide

Getting Help:

• Use --help flag with any command for detailed options
• Enable --verbose mode for debugging information
• Check the troubleshooting section for common issues

Happy documenting! 📚