chore: history cleanup
This commit is contained in:
521
docs/CLI_TUTORIAL.md
Normal file
521
docs/CLI_TUTORIAL.md
Normal file
@@ -0,0 +1,521 @@
|
||||
# MarkiTect CLI Tutorial: Clever Command-Line Usage
|
||||
|
||||
## Table of Contents
|
||||
1. [Getting Started](#getting-started)
|
||||
2. [Core Workflow Patterns](#core-workflow-patterns)
|
||||
3. [Document Processing](#document-processing)
|
||||
4. [Template & Schema Workflows](#template--schema-workflows)
|
||||
5. [Data Analysis & Querying](#data-analysis--querying)
|
||||
6. [Advanced Techniques](#advanced-techniques)
|
||||
7. [Business Document Automation](#business-document-automation)
|
||||
8. [Troubleshooting & Optimization](#troubleshooting--optimization)
|
||||
|
||||
---
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Installation & First Steps
|
||||
```bash
|
||||
# Check MarkiTect is properly installed
|
||||
markitect --help
|
||||
|
||||
# View system statistics
|
||||
markitect stats
|
||||
|
||||
# Check database status
|
||||
markitect db-stats
|
||||
```
|
||||
|
||||
### Essential Setup Commands
|
||||
```bash
|
||||
# Initialize workspace - process your first document
|
||||
markitect ingest README.md
|
||||
|
||||
# List all processed files
|
||||
markitect list
|
||||
|
||||
# Check specific file status
|
||||
markitect stats README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Workflow Patterns
|
||||
|
||||
### 1. Document Analysis Workflow
|
||||
**Scenario**: Analyze and understand a markdown document structure
|
||||
|
||||
```bash
|
||||
# Step 1: Ingest the document
|
||||
markitect ingest document.md
|
||||
|
||||
# Step 2: View document metadata
|
||||
markitect metadata document.md
|
||||
|
||||
# Step 3: Check frontmatter
|
||||
markitect frontmatter-keys document.md
|
||||
markitect frontmatter-get document.md title
|
||||
|
||||
# Step 4: Analyze AST structure
|
||||
markitect ast-show document.md --format tree
|
||||
|
||||
# Step 5: Generate schema from structure
|
||||
markitect schema-generate document.md --output document-schema.json
|
||||
```
|
||||
|
||||
### 2. Content Extraction Workflow
|
||||
**Scenario**: Extract specific content types from documents
|
||||
|
||||
```bash
|
||||
# Extract pure content (no frontmatter/tailmatter)
|
||||
markitect content-get document.md
|
||||
|
||||
# Get specific frontmatter values
|
||||
markitect frontmatter-get document.md author
|
||||
markitect frontmatter-get document.md config.theme # nested values
|
||||
|
||||
# Extract contentmatter (MultiMarkdown key-value pairs)
|
||||
markitect contentmatter-keys document.md
|
||||
markitect contentmatter-get document.md project_id
|
||||
|
||||
# Check tailmatter (QA checklists, metadata)
|
||||
markitect tailmatter-keys document.md
|
||||
markitect tailmatter-get document.md qa.reviewed
|
||||
```
|
||||
|
||||
### 3. Schema-Driven Development
|
||||
**Scenario**: Use schemas to validate and generate documents
|
||||
|
||||
```bash
|
||||
# Generate schema from example document
|
||||
markitect schema-generate example.md --output project-schema.json
|
||||
|
||||
# Store schema in database
|
||||
markitect schema-ingest project-schema.json
|
||||
|
||||
# Validate documents against schema
|
||||
markitect validate document.md project-schema.json
|
||||
|
||||
# Generate stub from schema
|
||||
markitect generate-stub project-schema.json --output new-document.md
|
||||
|
||||
# Generate multiple drafts
|
||||
markitect generate-drafts project-schema.json data-source.json --output-dir ./drafts/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Document Processing
|
||||
|
||||
### Batch Processing Techniques
|
||||
```bash
|
||||
# Process multiple files efficiently
|
||||
for file in *.md; do
|
||||
markitect ingest "$file"
|
||||
echo "Processed: $file"
|
||||
done
|
||||
|
||||
# Bulk validation
|
||||
for file in docs/*.md; do
|
||||
markitect validate "$file" schema.json || echo "Validation failed: $file"
|
||||
done
|
||||
|
||||
# Extract frontmatter from all files
|
||||
markitect list --format json | jq -r '.[].filename' | while read file; do
|
||||
echo "=== $file ==="
|
||||
markitect frontmatter-keys "$file"
|
||||
done
|
||||
```
|
||||
|
||||
### Content Modification Workflows
|
||||
```bash
|
||||
# Add sections to existing documents
|
||||
markitect modify document.md --add-section "New Section" --section-content "Content here"
|
||||
|
||||
# Update frontmatter programmatically
|
||||
markitect frontmatter-set document.md last_updated="$(date)"
|
||||
markitect frontmatter-set document.md version=2.1
|
||||
|
||||
# Set contentmatter values
|
||||
markitect contentmatter-set document.md status=reviewed
|
||||
markitect contentmatter-set document.md project.phase=complete
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Template & Schema Workflows
|
||||
|
||||
### Template-Driven Document Generation
|
||||
**Scenario**: Generate business documents from templates
|
||||
|
||||
```bash
|
||||
# Create invoice from template
|
||||
markitect template-render invoice-template.md customer-data.json \
|
||||
--output "invoice-$(date +%Y%m%d).md" \
|
||||
--validate --check-data
|
||||
|
||||
# Generate report with YAML data
|
||||
markitect template-render report-template.md quarterly-data.yaml \
|
||||
--format yaml --lenient --output quarterly-report.md
|
||||
|
||||
# Batch generate documents
|
||||
for customer in customers/*.json; do
|
||||
customer_name=$(basename "$customer" .json)
|
||||
markitect template-render invoice-template.md "$customer" \
|
||||
--output "invoices/invoice-$customer_name.md"
|
||||
done
|
||||
```
|
||||
|
||||
### Schema Management
|
||||
```bash
|
||||
# List all stored schemas
|
||||
markitect schema-list --format table
|
||||
|
||||
# Export schema for sharing
|
||||
markitect schema-get project-schema --output exported-schema.json
|
||||
|
||||
# Update schema in database
|
||||
markitect schema-delete old-schema
|
||||
markitect schema-ingest updated-schema.json
|
||||
|
||||
# Validate schema compliance
|
||||
markitect validate document.md schema-name --detailed-errors
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Analysis & Querying
|
||||
|
||||
### Database Queries
|
||||
```bash
|
||||
# View database schema
|
||||
markitect db-schema
|
||||
|
||||
# Query processed files
|
||||
markitect db-query "SELECT filename, processed_at FROM files WHERE processed_at > '2025-01-01'"
|
||||
|
||||
# Advanced frontmatter queries
|
||||
markitect db-query "SELECT filename, frontmatter FROM files WHERE JSON_EXTRACT(frontmatter, '$.author') = 'John Doe'"
|
||||
|
||||
# Content statistics
|
||||
markitect db-query "SELECT AVG(JSON_EXTRACT(metadata, '$.word_count')) as avg_words FROM files"
|
||||
```
|
||||
|
||||
### AST Analysis
|
||||
```bash
|
||||
# Query AST structure with JSONPath
|
||||
markitect ast-query document.md "$.children[?(@.type=='heading')].children[0].value"
|
||||
|
||||
# Find all links in document
|
||||
markitect ast-query document.md "$..children[?(@.type=='link')].url"
|
||||
|
||||
# Extract code blocks
|
||||
markitect ast-query document.md "$..children[?(@.type=='code')].value"
|
||||
|
||||
# Analyze heading structure
|
||||
markitect ast-query document.md "$.children[?(@.type=='heading')].depth" --format json
|
||||
```
|
||||
|
||||
### Statistical Analysis
|
||||
```bash
|
||||
# Document statistics
|
||||
markitect content-stats document.md
|
||||
|
||||
# Frontmatter analysis across all files
|
||||
markitect frontmatter-stats
|
||||
|
||||
# Contentmatter usage patterns
|
||||
markitect contentmatter-stats
|
||||
|
||||
# System performance metrics
|
||||
markitect cache-stats
|
||||
markitect ast-stats
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Advanced Techniques
|
||||
|
||||
### Command Chaining & Pipelines
|
||||
```bash
|
||||
# Extract and process frontmatter
|
||||
markitect frontmatter-get document.md title | tr '[:lower:]' '[:upper:]'
|
||||
|
||||
# Combine with standard tools
|
||||
markitect list --format json | jq '.[] | select(.word_count > 1000) | .filename'
|
||||
|
||||
# Template generation pipeline
|
||||
markitect schema-generate source.md | \
|
||||
markitect generate-stub --stdin | \
|
||||
markitect template-render --stdin data.json
|
||||
```
|
||||
|
||||
### Conditional Processing
|
||||
```bash
|
||||
# Process only if file changed
|
||||
if [ document.md -nt last-processed.timestamp ]; then
|
||||
markitect ingest document.md
|
||||
touch last-processed.timestamp
|
||||
fi
|
||||
|
||||
# Validate before publishing
|
||||
if markitect validate document.md schema.json --quiet; then
|
||||
echo "✅ Document valid - ready for publish"
|
||||
markitect template-render publish-template.md document-data.json
|
||||
else
|
||||
echo "❌ Validation failed - fix errors first"
|
||||
markitect validate document.md schema.json --detailed-errors
|
||||
fi
|
||||
```
|
||||
|
||||
### Output Format Optimization
|
||||
```bash
|
||||
# Machine-readable output
|
||||
markitect list --format json > files.json
|
||||
markitect stats --format yaml > stats.yaml
|
||||
|
||||
# Human-readable reports
|
||||
markitect list --format table --names-only
|
||||
markitect db-stats --format simple
|
||||
|
||||
# Export for external tools
|
||||
markitect db-query "SELECT * FROM files" --format json | jq '.[] | .filename'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Business Document Automation
|
||||
|
||||
### Invoice Generation Workflow
|
||||
```bash
|
||||
# Setup: Create invoice template and customer database
|
||||
# invoice-template.md contains {{customer.name}}, {{items}}, {{total}} etc.
|
||||
# customers.json contains customer data array
|
||||
|
||||
# Generate monthly invoices
|
||||
markitect template-render templates/invoice.md data/customer-001.json \
|
||||
--output "invoices/$(date +%Y-%m)/customer-001-invoice.md" \
|
||||
--validate --check-data
|
||||
|
||||
# Batch invoice generation
|
||||
for customer in data/customers/*.json; do
|
||||
customer_id=$(basename "$customer" .json)
|
||||
markitect template-render templates/invoice.md "$customer" \
|
||||
--output "invoices/$(date +%Y-%m)/$customer_id-invoice.md" \
|
||||
--strict
|
||||
done
|
||||
```
|
||||
|
||||
### Report Generation Pipeline
|
||||
```bash
|
||||
# Generate quarterly business report
|
||||
markitect template-render templates/quarterly-report.md data/q1-2025.yaml \
|
||||
--format yaml \
|
||||
--output "reports/Q1-2025-Business-Report.md" \
|
||||
--validate
|
||||
|
||||
# Validate report against company standards
|
||||
markitect validate "reports/Q1-2025-Business-Report.md" schemas/report-schema.json
|
||||
|
||||
# Extract key metrics for dashboard
|
||||
markitect frontmatter-get "reports/Q1-2025-Business-Report.md" metrics.revenue
|
||||
markitect contentmatter-get "reports/Q1-2025-Business-Report.md" kpi.growth_rate
|
||||
```
|
||||
|
||||
### Content Management Workflows
|
||||
```bash
|
||||
# Blog post publishing pipeline
|
||||
markitect ingest drafts/new-post.md
|
||||
markitect validate drafts/new-post.md schemas/blog-post.json
|
||||
markitect frontmatter-set drafts/new-post.md published_date="$(date)"
|
||||
markitect frontmatter-set drafts/new-post.md status=published
|
||||
|
||||
# Documentation maintenance
|
||||
markitect schema-generate docs/api-reference.md --output schemas/api-doc.json
|
||||
markitect generate-stub schemas/api-doc.json --output templates/api-template.md
|
||||
|
||||
# Quality assurance checks
|
||||
markitect tailmatter-check document.md # Run QA checklist
|
||||
markitect validate document.md company-standards.json --detailed-errors
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting & Optimization
|
||||
|
||||
### Performance Optimization
|
||||
```bash
|
||||
# Check cache effectiveness
|
||||
markitect cache-stats
|
||||
|
||||
# Clean cache if needed
|
||||
markitect cache-clean
|
||||
|
||||
# Invalidate specific file cache
|
||||
markitect cache-invalidate problematic-file.md
|
||||
|
||||
# Monitor database performance
|
||||
markitect db-stats --format json | jq '.performance'
|
||||
```
|
||||
|
||||
### Debugging Workflows
|
||||
```bash
|
||||
# Verbose output for debugging
|
||||
markitect --verbose ingest document.md
|
||||
|
||||
# Check file processing status
|
||||
markitect metadata document.md --format json | jq '.processing_errors'
|
||||
|
||||
# Validate template syntax
|
||||
markitect template-render template.md data.json --validate
|
||||
|
||||
# Debug AST issues
|
||||
markitect ast-show document.md --format json | jq '.errors'
|
||||
```
|
||||
|
||||
### Database Maintenance
|
||||
```bash
|
||||
# Backup database
|
||||
cp markitect.db markitect-backup-$(date +%Y%m%d).db
|
||||
|
||||
# Clean up orphaned records
|
||||
markitect db-query "DELETE FROM files WHERE filename NOT IN (SELECT DISTINCT filename FROM current_files)"
|
||||
|
||||
# Optimize database
|
||||
markitect db-query "VACUUM"
|
||||
|
||||
# Check database integrity
|
||||
markitect db-query "PRAGMA integrity_check"
|
||||
```
|
||||
|
||||
### Configuration Management
|
||||
```bash
|
||||
# Check configuration
|
||||
markitect config-stats
|
||||
|
||||
# Use custom config file
|
||||
markitect --config custom-config.yaml list
|
||||
|
||||
# Use different database
|
||||
markitect --database project-specific.db ingest document.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pro Tips & Best Practices
|
||||
|
||||
### 1. Workflow Automation
|
||||
```bash
|
||||
# Create alias for common operations
|
||||
alias md-process='markitect ingest'
|
||||
alias md-validate='markitect validate'
|
||||
alias md-extract='markitect frontmatter-get'
|
||||
|
||||
# Setup environment variables
|
||||
export MARKITECT_DB="/path/to/project.db"
|
||||
export MARKITECT_CONFIG="/path/to/config.yaml"
|
||||
```
|
||||
|
||||
### 2. Error Handling in Scripts
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# Robust document processing script
|
||||
|
||||
process_document() {
|
||||
local file="$1"
|
||||
|
||||
# Check file exists
|
||||
if [[ ! -f "$file" ]]; then
|
||||
echo "Error: File $file not found" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# Process with error handling
|
||||
if markitect ingest "$file"; then
|
||||
echo "✅ Processed: $file"
|
||||
|
||||
# Validate if schema exists
|
||||
if [[ -f "schema.json" ]]; then
|
||||
if markitect validate "$file" schema.json --quiet; then
|
||||
echo "✅ Validated: $file"
|
||||
else
|
||||
echo "⚠️ Validation failed: $file" >&2
|
||||
markitect validate "$file" schema.json --detailed-errors >&2
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo "❌ Processing failed: $file" >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# Process all markdown files
|
||||
for file in *.md; do
|
||||
process_document "$file" || echo "Skipping $file due to errors"
|
||||
done
|
||||
```
|
||||
|
||||
### 3. Integration with Other Tools
|
||||
```bash
|
||||
# Combine with git hooks
|
||||
# .git/hooks/pre-commit
|
||||
markitect validate changed-docs/*.md schemas/doc-standard.json --quiet || {
|
||||
echo "Documentation validation failed"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Integration with CI/CD
|
||||
markitect list --format json | jq -r '.[] | select(.validation_status != "valid") | .filename' | while read file; do
|
||||
echo "::error file=$file::Document validation failed"
|
||||
done
|
||||
|
||||
# Export for external analytics
|
||||
markitect db-query "SELECT filename, JSON_EXTRACT(metadata, '$.word_count') as words FROM files" \
|
||||
--format json | jq -r '.[] | "\(.filename),\(.words)"' > document-metrics.csv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Most Common Commands
|
||||
```bash
|
||||
# Basic document processing
|
||||
markitect ingest document.md
|
||||
markitect list
|
||||
markitect stats document.md
|
||||
|
||||
# Content extraction
|
||||
markitect frontmatter-get document.md key
|
||||
markitect content-get document.md
|
||||
|
||||
# Template processing
|
||||
markitect template-render template.md data.json
|
||||
|
||||
# Schema operations
|
||||
markitect schema-generate document.md
|
||||
markitect validate document.md schema.json
|
||||
|
||||
# Database queries
|
||||
markitect db-query "SQL_QUERY"
|
||||
markitect list --format json
|
||||
```
|
||||
|
||||
### Output Formats
|
||||
- `--format table` - Human-readable tables
|
||||
- `--format json` - Machine-readable JSON
|
||||
- `--format yaml` - YAML format
|
||||
- `--format simple` - Plain text
|
||||
- `--format compact` - Condensed output
|
||||
|
||||
### Global Options
|
||||
- `--verbose` - Detailed output
|
||||
- `--config CONFIG_FILE` - Custom configuration
|
||||
- `--database DB_FILE` - Custom database
|
||||
- `--help` - Command help
|
||||
|
||||
---
|
||||
|
||||
**🎯 Pro Tip**: Start with basic `ingest` and `list` commands, then gradually explore advanced features. Use `--help` on any command to see all available options!
|
||||
|
||||
**📚 Remember**: MarkiTect is designed for powerful document automation - combine commands creatively to build sophisticated workflows that match your specific needs.
|
||||
Reference in New Issue
Block a user