diff --git a/CLI_TUTORIAL.md b/CLI_TUTORIAL.md new file mode 100644 index 00000000..231c7374 --- /dev/null +++ b/CLI_TUTORIAL.md @@ -0,0 +1,521 @@ +# MarkiTect CLI Tutorial: Clever Command-Line Usage + +## Table of Contents +1. [Getting Started](#getting-started) +2. [Core Workflow Patterns](#core-workflow-patterns) +3. [Document Processing](#document-processing) +4. [Template & Schema Workflows](#template--schema-workflows) +5. [Data Analysis & Querying](#data-analysis--querying) +6. [Advanced Techniques](#advanced-techniques) +7. [Business Document Automation](#business-document-automation) +8. [Troubleshooting & Optimization](#troubleshooting--optimization) + +--- + +## Getting Started + +### Installation & First Steps +```bash +# Check MarkiTect is properly installed +markitect --help + +# View system statistics +markitect stats + +# Check database status +markitect db-stats +``` + +### Essential Setup Commands +```bash +# Initialize workspace - process your first document +markitect ingest README.md + +# List all processed files +markitect list + +# Check specific file status +markitect stats README.md +``` + +--- + +## Core Workflow Patterns + +### 1. Document Analysis Workflow +**Scenario**: Analyze and understand a markdown document structure + +```bash +# Step 1: Ingest the document +markitect ingest document.md + +# Step 2: View document metadata +markitect metadata document.md + +# Step 3: Check frontmatter +markitect frontmatter-keys document.md +markitect frontmatter-get document.md title + +# Step 4: Analyze AST structure +markitect ast-show document.md --format tree + +# Step 5: Generate schema from structure +markitect schema-generate document.md --output document-schema.json +``` + +### 2. Content Extraction Workflow +**Scenario**: Extract specific content types from documents + +```bash +# Extract pure content (no frontmatter/tailmatter) +markitect content-get document.md + +# Get specific frontmatter values +markitect frontmatter-get document.md author +markitect frontmatter-get document.md config.theme # nested values + +# Extract contentmatter (MultiMarkdown key-value pairs) +markitect contentmatter-keys document.md +markitect contentmatter-get document.md project_id + +# Check tailmatter (QA checklists, metadata) +markitect tailmatter-keys document.md +markitect tailmatter-get document.md qa.reviewed +``` + +### 3. Schema-Driven Development +**Scenario**: Use schemas to validate and generate documents + +```bash +# Generate schema from example document +markitect schema-generate example.md --output project-schema.json + +# Store schema in database +markitect schema-ingest project-schema.json + +# Validate documents against schema +markitect validate document.md project-schema.json + +# Generate stub from schema +markitect generate-stub project-schema.json --output new-document.md + +# Generate multiple drafts +markitect generate-drafts project-schema.json data-source.json --output-dir ./drafts/ +``` + +--- + +## Document Processing + +### Batch Processing Techniques +```bash +# Process multiple files efficiently +for file in *.md; do + markitect ingest "$file" + echo "Processed: $file" +done + +# Bulk validation +for file in docs/*.md; do + markitect validate "$file" schema.json || echo "Validation failed: $file" +done + +# Extract frontmatter from all files +markitect list --format json | jq -r '.[].filename' | while read file; do + echo "=== $file ===" + markitect frontmatter-keys "$file" +done +``` + +### Content Modification Workflows +```bash +# Add sections to existing documents +markitect modify document.md --add-section "New Section" --section-content "Content here" + +# Update frontmatter programmatically +markitect frontmatter-set document.md last_updated="$(date)" +markitect frontmatter-set document.md version=2.1 + +# Set contentmatter values +markitect contentmatter-set document.md status=reviewed +markitect contentmatter-set document.md project.phase=complete +``` + +--- + +## Template & Schema Workflows + +### Template-Driven Document Generation +**Scenario**: Generate business documents from templates + +```bash +# Create invoice from template +markitect template-render invoice-template.md customer-data.json \ + --output "invoice-$(date +%Y%m%d).md" \ + --validate --check-data + +# Generate report with YAML data +markitect template-render report-template.md quarterly-data.yaml \ + --format yaml --lenient --output quarterly-report.md + +# Batch generate documents +for customer in customers/*.json; do + customer_name=$(basename "$customer" .json) + markitect template-render invoice-template.md "$customer" \ + --output "invoices/invoice-$customer_name.md" +done +``` + +### Schema Management +```bash +# List all stored schemas +markitect schema-list --format table + +# Export schema for sharing +markitect schema-get project-schema --output exported-schema.json + +# Update schema in database +markitect schema-delete old-schema +markitect schema-ingest updated-schema.json + +# Validate schema compliance +markitect validate document.md schema-name --detailed-errors +``` + +--- + +## Data Analysis & Querying + +### Database Queries +```bash +# View database schema +markitect db-schema + +# Query processed files +markitect db-query "SELECT filename, processed_at FROM files WHERE processed_at > '2025-01-01'" + +# Advanced frontmatter queries +markitect db-query "SELECT filename, frontmatter FROM files WHERE JSON_EXTRACT(frontmatter, '$.author') = 'John Doe'" + +# Content statistics +markitect db-query "SELECT AVG(JSON_EXTRACT(metadata, '$.word_count')) as avg_words FROM files" +``` + +### AST Analysis +```bash +# Query AST structure with JSONPath +markitect ast-query document.md "$.children[?(@.type=='heading')].children[0].value" + +# Find all links in document +markitect ast-query document.md "$..children[?(@.type=='link')].url" + +# Extract code blocks +markitect ast-query document.md "$..children[?(@.type=='code')].value" + +# Analyze heading structure +markitect ast-query document.md "$.children[?(@.type=='heading')].depth" --format json +``` + +### Statistical Analysis +```bash +# Document statistics +markitect content-stats document.md + +# Frontmatter analysis across all files +markitect frontmatter-stats + +# Contentmatter usage patterns +markitect contentmatter-stats + +# System performance metrics +markitect cache-stats +markitect ast-stats +``` + +--- + +## Advanced Techniques + +### Command Chaining & Pipelines +```bash +# Extract and process frontmatter +markitect frontmatter-get document.md title | tr '[:lower:]' '[:upper:]' + +# Combine with standard tools +markitect list --format json | jq '.[] | select(.word_count > 1000) | .filename' + +# Template generation pipeline +markitect schema-generate source.md | \ + markitect generate-stub --stdin | \ + markitect template-render --stdin data.json +``` + +### Conditional Processing +```bash +# Process only if file changed +if [ document.md -nt last-processed.timestamp ]; then + markitect ingest document.md + touch last-processed.timestamp +fi + +# Validate before publishing +if markitect validate document.md schema.json --quiet; then + echo "✅ Document valid - ready for publish" + markitect template-render publish-template.md document-data.json +else + echo "❌ Validation failed - fix errors first" + markitect validate document.md schema.json --detailed-errors +fi +``` + +### Output Format Optimization +```bash +# Machine-readable output +markitect list --format json > files.json +markitect stats --format yaml > stats.yaml + +# Human-readable reports +markitect list --format table --names-only +markitect db-stats --format simple + +# Export for external tools +markitect db-query "SELECT * FROM files" --format json | jq '.[] | .filename' +``` + +--- + +## Business Document Automation + +### Invoice Generation Workflow +```bash +# Setup: Create invoice template and customer database +# invoice-template.md contains {{customer.name}}, {{items}}, {{total}} etc. +# customers.json contains customer data array + +# Generate monthly invoices +markitect template-render templates/invoice.md data/customer-001.json \ + --output "invoices/$(date +%Y-%m)/customer-001-invoice.md" \ + --validate --check-data + +# Batch invoice generation +for customer in data/customers/*.json; do + customer_id=$(basename "$customer" .json) + markitect template-render templates/invoice.md "$customer" \ + --output "invoices/$(date +%Y-%m)/$customer_id-invoice.md" \ + --strict +done +``` + +### Report Generation Pipeline +```bash +# Generate quarterly business report +markitect template-render templates/quarterly-report.md data/q1-2025.yaml \ + --format yaml \ + --output "reports/Q1-2025-Business-Report.md" \ + --validate + +# Validate report against company standards +markitect validate "reports/Q1-2025-Business-Report.md" schemas/report-schema.json + +# Extract key metrics for dashboard +markitect frontmatter-get "reports/Q1-2025-Business-Report.md" metrics.revenue +markitect contentmatter-get "reports/Q1-2025-Business-Report.md" kpi.growth_rate +``` + +### Content Management Workflows +```bash +# Blog post publishing pipeline +markitect ingest drafts/new-post.md +markitect validate drafts/new-post.md schemas/blog-post.json +markitect frontmatter-set drafts/new-post.md published_date="$(date)" +markitect frontmatter-set drafts/new-post.md status=published + +# Documentation maintenance +markitect schema-generate docs/api-reference.md --output schemas/api-doc.json +markitect generate-stub schemas/api-doc.json --output templates/api-template.md + +# Quality assurance checks +markitect tailmatter-check document.md # Run QA checklist +markitect validate document.md company-standards.json --detailed-errors +``` + +--- + +## Troubleshooting & Optimization + +### Performance Optimization +```bash +# Check cache effectiveness +markitect cache-stats + +# Clean cache if needed +markitect cache-clean + +# Invalidate specific file cache +markitect cache-invalidate problematic-file.md + +# Monitor database performance +markitect db-stats --format json | jq '.performance' +``` + +### Debugging Workflows +```bash +# Verbose output for debugging +markitect --verbose ingest document.md + +# Check file processing status +markitect metadata document.md --format json | jq '.processing_errors' + +# Validate template syntax +markitect template-render template.md data.json --validate + +# Debug AST issues +markitect ast-show document.md --format json | jq '.errors' +``` + +### Database Maintenance +```bash +# Backup database +cp markitect.db markitect-backup-$(date +%Y%m%d).db + +# Clean up orphaned records +markitect db-query "DELETE FROM files WHERE filename NOT IN (SELECT DISTINCT filename FROM current_files)" + +# Optimize database +markitect db-query "VACUUM" + +# Check database integrity +markitect db-query "PRAGMA integrity_check" +``` + +### Configuration Management +```bash +# Check configuration +markitect config-stats + +# Use custom config file +markitect --config custom-config.yaml list + +# Use different database +markitect --database project-specific.db ingest document.md +``` + +--- + +## Pro Tips & Best Practices + +### 1. Workflow Automation +```bash +# Create alias for common operations +alias md-process='markitect ingest' +alias md-validate='markitect validate' +alias md-extract='markitect frontmatter-get' + +# Setup environment variables +export MARKITECT_DB="/path/to/project.db" +export MARKITECT_CONFIG="/path/to/config.yaml" +``` + +### 2. Error Handling in Scripts +```bash +#!/bin/bash +# Robust document processing script + +process_document() { + local file="$1" + + # Check file exists + if [[ ! -f "$file" ]]; then + echo "Error: File $file not found" >&2 + return 1 + fi + + # Process with error handling + if markitect ingest "$file"; then + echo "✅ Processed: $file" + + # Validate if schema exists + if [[ -f "schema.json" ]]; then + if markitect validate "$file" schema.json --quiet; then + echo "✅ Validated: $file" + else + echo "⚠️ Validation failed: $file" >&2 + markitect validate "$file" schema.json --detailed-errors >&2 + fi + fi + else + echo "❌ Processing failed: $file" >&2 + return 1 + fi +} + +# Process all markdown files +for file in *.md; do + process_document "$file" || echo "Skipping $file due to errors" +done +``` + +### 3. Integration with Other Tools +```bash +# Combine with git hooks +# .git/hooks/pre-commit +markitect validate changed-docs/*.md schemas/doc-standard.json --quiet || { + echo "Documentation validation failed" + exit 1 +} + +# Integration with CI/CD +markitect list --format json | jq -r '.[] | select(.validation_status != "valid") | .filename' | while read file; do + echo "::error file=$file::Document validation failed" +done + +# Export for external analytics +markitect db-query "SELECT filename, JSON_EXTRACT(metadata, '$.word_count') as words FROM files" \ + --format json | jq -r '.[] | "\(.filename),\(.words)"' > document-metrics.csv +``` + +--- + +## Quick Reference + +### Most Common Commands +```bash +# Basic document processing +markitect ingest document.md +markitect list +markitect stats document.md + +# Content extraction +markitect frontmatter-get document.md key +markitect content-get document.md + +# Template processing +markitect template-render template.md data.json + +# Schema operations +markitect schema-generate document.md +markitect validate document.md schema.json + +# Database queries +markitect db-query "SQL_QUERY" +markitect list --format json +``` + +### Output Formats +- `--format table` - Human-readable tables +- `--format json` - Machine-readable JSON +- `--format yaml` - YAML format +- `--format simple` - Plain text +- `--format compact` - Condensed output + +### Global Options +- `--verbose` - Detailed output +- `--config CONFIG_FILE` - Custom configuration +- `--database DB_FILE` - Custom database +- `--help` - Command help + +--- + +**🎯 Pro Tip**: Start with basic `ingest` and `list` commands, then gradually explore advanced features. Use `--help` on any command to see all available options! + +**📚 Remember**: MarkiTect is designed for powerful document automation - combine commands creatively to build sophisticated workflows that match your specific needs. \ No newline at end of file