markitect-main/CLI_TUTORIAL.md

MarkiTect CLI Tutorial: Clever Command-Line Usage

Table of Contents

1. Getting Started
2. Core Workflow Patterns
3. Document Processing
4. Template & Schema Workflows
5. Data Analysis & Querying
6. Advanced Techniques
7. Business Document Automation
8. Troubleshooting & Optimization

---

Getting Started

Installation & First Steps

# Check MarkiTect is properly installed
markitect --help

# View system statistics
markitect stats

# Check database status
markitect db-stats

Essential Setup Commands

# Initialize workspace - process your first document
markitect ingest README.md

# List all processed files
markitect list

# Check specific file status
markitect stats README.md

---

Core Workflow Patterns

1. Document Analysis Workflow

Scenario: Analyze and understand a markdown document structure

# Step 1: Ingest the document
markitect ingest document.md

# Step 2: View document metadata
markitect metadata document.md

# Step 3: Check frontmatter
markitect frontmatter-keys document.md
markitect frontmatter-get document.md title

# Step 4: Analyze AST structure
markitect ast-show document.md --format tree

# Step 5: Generate schema from structure
markitect schema-generate document.md --output document-schema.json

2. Content Extraction Workflow

Scenario: Extract specific content types from documents

# Extract pure content (no frontmatter/tailmatter)
markitect content-get document.md

# Get specific frontmatter values
markitect frontmatter-get document.md author
markitect frontmatter-get document.md config.theme  # nested values

# Extract contentmatter (MultiMarkdown key-value pairs)
markitect contentmatter-keys document.md
markitect contentmatter-get document.md project_id

# Check tailmatter (QA checklists, metadata)
markitect tailmatter-keys document.md
markitect tailmatter-get document.md qa.reviewed

3. Schema-Driven Development

Scenario: Use schemas to validate and generate documents

# Generate schema from example document
markitect schema-generate example.md --output project-schema.json

# Store schema in database
markitect schema-ingest project-schema.json

# Validate documents against schema
markitect validate document.md project-schema.json

# Generate stub from schema
markitect generate-stub project-schema.json --output new-document.md

# Generate multiple drafts
markitect generate-drafts project-schema.json data-source.json --output-dir ./drafts/

---

Document Processing

Batch Processing Techniques

# Process multiple files efficiently
for file in *.md; do
    markitect ingest "$file"
    echo "Processed: $file"
done

# Bulk validation
for file in docs/*.md; do
    markitect validate "$file" schema.json || echo "Validation failed: $file"
done

# Extract frontmatter from all files
markitect list --format json | jq -r '.[].filename' | while read file; do
    echo "=== $file ==="
    markitect frontmatter-keys "$file"
done

Content Modification Workflows

# Add sections to existing documents
markitect modify document.md --add-section "New Section" --section-content "Content here"

# Update frontmatter programmatically
markitect frontmatter-set document.md last_updated="$(date)"
markitect frontmatter-set document.md version=2.1

# Set contentmatter values
markitect contentmatter-set document.md status=reviewed
markitect contentmatter-set document.md project.phase=complete

---

Template & Schema Workflows

Template-Driven Document Generation

Scenario: Generate business documents from templates

# Create invoice from template
markitect template-render invoice-template.md customer-data.json \
    --output "invoice-$(date +%Y%m%d).md" \
    --validate --check-data

# Generate report with YAML data
markitect template-render report-template.md quarterly-data.yaml \
    --format yaml --lenient --output quarterly-report.md

# Batch generate documents
for customer in customers/*.json; do
    customer_name=$(basename "$customer" .json)
    markitect template-render invoice-template.md "$customer" \
        --output "invoices/invoice-$customer_name.md"
done

Schema Management

# List all stored schemas
markitect schema-list --format table

# Export schema for sharing
markitect schema-get project-schema --output exported-schema.json

# Update schema in database
markitect schema-delete old-schema
markitect schema-ingest updated-schema.json

# Validate schema compliance
markitect validate document.md schema-name --detailed-errors

---

Data Analysis & Querying

Database Queries

# View database schema
markitect db-schema

# Query processed files
markitect db-query "SELECT filename, processed_at FROM files WHERE processed_at > '2025-01-01'"

# Advanced frontmatter queries
markitect db-query "SELECT filename, frontmatter FROM files WHERE JSON_EXTRACT(frontmatter, '$.author') = 'John Doe'"

# Content statistics
markitect db-query "SELECT AVG(JSON_EXTRACT(metadata, '$.word_count')) as avg_words FROM files"

AST Analysis

# Query AST structure with JSONPath
markitect ast-query document.md "$.children[?(@.type=='heading')].children[0].value"

# Find all links in document
markitect ast-query document.md "$..children[?(@.type=='link')].url"

# Extract code blocks
markitect ast-query document.md "$..children[?(@.type=='code')].value"

# Analyze heading structure
markitect ast-query document.md "$.children[?(@.type=='heading')].depth" --format json

Statistical Analysis

# Document statistics
markitect content-stats document.md

# Frontmatter analysis across all files
markitect frontmatter-stats

# Contentmatter usage patterns
markitect contentmatter-stats

# System performance metrics
markitect cache-stats
markitect ast-stats

---

Advanced Techniques

Command Chaining & Pipelines

# Extract and process frontmatter
markitect frontmatter-get document.md title | tr '[:lower:]' '[:upper:]'

# Combine with standard tools
markitect list --format json | jq '.[] | select(.word_count > 1000) | .filename'

# Template generation pipeline
markitect schema-generate source.md | \
    markitect generate-stub --stdin | \
    markitect template-render --stdin data.json

Conditional Processing

# Process only if file changed
if [ document.md -nt last-processed.timestamp ]; then
    markitect ingest document.md
    touch last-processed.timestamp
fi

# Validate before publishing
if markitect validate document.md schema.json --quiet; then
    echo "✅ Document valid - ready for publish"
    markitect template-render publish-template.md document-data.json
else
    echo "❌ Validation failed - fix errors first"
    markitect validate document.md schema.json --detailed-errors
fi

Output Format Optimization

# Machine-readable output
markitect list --format json > files.json
markitect stats --format yaml > stats.yaml

# Human-readable reports
markitect list --format table --names-only
markitect db-stats --format simple

# Export for external tools
markitect db-query "SELECT * FROM files" --format json | jq '.[] | .filename'

---

Business Document Automation

Invoice Generation Workflow

# Setup: Create invoice template and customer database
# invoice-template.md contains {{customer.name}}, {{items}}, {{total}} etc.
# customers.json contains customer data array

# Generate monthly invoices
markitect template-render templates/invoice.md data/customer-001.json \
    --output "invoices/$(date +%Y-%m)/customer-001-invoice.md" \
    --validate --check-data

# Batch invoice generation
for customer in data/customers/*.json; do
    customer_id=$(basename "$customer" .json)
    markitect template-render templates/invoice.md "$customer" \
        --output "invoices/$(date +%Y-%m)/$customer_id-invoice.md" \
        --strict
done

Report Generation Pipeline

# Generate quarterly business report
markitect template-render templates/quarterly-report.md data/q1-2025.yaml \
    --format yaml \
    --output "reports/Q1-2025-Business-Report.md" \
    --validate

# Validate report against company standards
markitect validate "reports/Q1-2025-Business-Report.md" schemas/report-schema.json

# Extract key metrics for dashboard
markitect frontmatter-get "reports/Q1-2025-Business-Report.md" metrics.revenue
markitect contentmatter-get "reports/Q1-2025-Business-Report.md" kpi.growth_rate

Content Management Workflows

# Blog post publishing pipeline
markitect ingest drafts/new-post.md
markitect validate drafts/new-post.md schemas/blog-post.json
markitect frontmatter-set drafts/new-post.md published_date="$(date)"
markitect frontmatter-set drafts/new-post.md status=published

# Documentation maintenance
markitect schema-generate docs/api-reference.md --output schemas/api-doc.json
markitect generate-stub schemas/api-doc.json --output templates/api-template.md

# Quality assurance checks
markitect tailmatter-check document.md  # Run QA checklist
markitect validate document.md company-standards.json --detailed-errors

---

Troubleshooting & Optimization

Performance Optimization

# Check cache effectiveness
markitect cache-stats

# Clean cache if needed
markitect cache-clean

# Invalidate specific file cache
markitect cache-invalidate problematic-file.md

# Monitor database performance
markitect db-stats --format json | jq '.performance'

Debugging Workflows

# Verbose output for debugging
markitect --verbose ingest document.md

# Check file processing status
markitect metadata document.md --format json | jq '.processing_errors'

# Validate template syntax
markitect template-render template.md data.json --validate

# Debug AST issues
markitect ast-show document.md --format json | jq '.errors'

Database Maintenance

# Backup database
cp markitect.db markitect-backup-$(date +%Y%m%d).db

# Clean up orphaned records
markitect db-query "DELETE FROM files WHERE filename NOT IN (SELECT DISTINCT filename FROM current_files)"

# Optimize database
markitect db-query "VACUUM"

# Check database integrity
markitect db-query "PRAGMA integrity_check"

Configuration Management

# Check configuration
markitect config-stats

# Use custom config file
markitect --config custom-config.yaml list

# Use different database
markitect --database project-specific.db ingest document.md

---

Pro Tips & Best Practices

1. Workflow Automation

# Create alias for common operations
alias md-process='markitect ingest'
alias md-validate='markitect validate'
alias md-extract='markitect frontmatter-get'

# Setup environment variables
export MARKITECT_DB="/path/to/project.db"
export MARKITECT_CONFIG="/path/to/config.yaml"

2. Error Handling in Scripts

#!/bin/bash
# Robust document processing script

process_document() {
    local file="$1"

    # Check file exists
    if [[ ! -f "$file" ]]; then
        echo "Error: File $file not found" >&2
        return 1
    fi

    # Process with error handling
    if markitect ingest "$file"; then
        echo "✅ Processed: $file"

        # Validate if schema exists
        if [[ -f "schema.json" ]]; then
            if markitect validate "$file" schema.json --quiet; then
                echo "✅ Validated: $file"
            else
                echo "⚠️  Validation failed: $file" >&2
                markitect validate "$file" schema.json --detailed-errors >&2
            fi
        fi
    else
        echo "❌ Processing failed: $file" >&2
        return 1
    fi
}

# Process all markdown files
for file in *.md; do
    process_document "$file" || echo "Skipping $file due to errors"
done

3. Integration with Other Tools

# Combine with git hooks
# .git/hooks/pre-commit
markitect validate changed-docs/*.md schemas/doc-standard.json --quiet || {
    echo "Documentation validation failed"
    exit 1
}

# Integration with CI/CD
markitect list --format json | jq -r '.[] | select(.validation_status != "valid") | .filename' | while read file; do
    echo "::error file=$file::Document validation failed"
done

# Export for external analytics
markitect db-query "SELECT filename, JSON_EXTRACT(metadata, '$.word_count') as words FROM files" \
    --format json | jq -r '.[] | "\(.filename),\(.words)"' > document-metrics.csv

---

Quick Reference

Most Common Commands

# Basic document processing
markitect ingest document.md
markitect list
markitect stats document.md

# Content extraction
markitect frontmatter-get document.md key
markitect content-get document.md

# Template processing
markitect template-render template.md data.json

# Schema operations
markitect schema-generate document.md
markitect validate document.md schema.json

# Database queries
markitect db-query "SQL_QUERY"
markitect list --format json

Output Formats

• --format table - Human-readable tables
• --format json - Machine-readable JSON
• --format yaml - YAML format
• --format simple - Plain text
• --format compact - Condensed output

Global Options

• --verbose - Detailed output
• --config CONFIG_FILE - Custom configuration
• --database DB_FILE - Custom database
• --help - Command help

---

🎯 Pro Tip: Start with basic ingest and list commands, then gradually explore advanced features. Use --help on any command to see all available options!

📚 Remember: MarkiTect is designed for powerful document automation - combine commands creatively to build sophisticated workflows that match your specific needs.