markitect-main/docs/user-guides/cache-management.md

# Cache Management Guide

MarkiTect's caching system provides significant performance improvements by storing parsed AST representations of your markdown files. This guide explains how to monitor, maintain, and optimize your cache usage.

## Overview

The cache system automatically manages performance optimization, but provides CLI tools for monitoring and maintenance when needed.

## Cache Commands

### `markitect cache-info`

Display detailed information about your current cache status.

```bash
markitect cache-info
```

**Example Output:**
```
Cache Directory: /home/user/project/.ast_cache
Total Files: 42
Cache Size: 2.1 MB
```

**What it shows:**
- **Cache Directory**: Where cache files are stored
- **Total Files**: Number of documents currently cached
- **Cache Size**: Total disk space used by cache

### `markitect cache-clean`

Remove all cached files to free disk space or force fresh parsing.

```bash
markitect cache-clean
```

**Example Output:**
```
Cache cleaned successfully - removed 42 file(s).
```

**When to use:**
- Free up disk space
- Force fresh parsing of all documents
- Clear potentially corrupted cache
- Development debugging

### `markitect cache-invalidate <file>`

Remove cache for a specific file, forcing it to be re-parsed next time.

```bash
markitect cache-invalidate docs/architecture.md
```

**Example Output:**
```
Cache invalidated for architecture.md.
```

**When to use:**
- File was modified outside of MarkiTect
- Testing parsing behavior
- Troubleshooting specific document issues

## Understanding Cache Behavior

### Automatic Cache Management

The cache system handles most operations automatically:

1. **First Access**: File is parsed and cached
2. **Subsequent Access**: Cache is loaded (60-85% faster)
3. **File Modification**: Cache is automatically invalidated
4. **Next Access**: File is re-parsed and re-cached

### Cache Directory Structure

```
your-project/
├── docs/
│   ├── guide.md              # Your source files
│   └── api.md
├── .ast_cache/               # Auto-created cache directory
│   ├── guide.md.ast.json     # Cached AST for guide.md
│   └── api.md.ast.json       # Cached AST for api.md
└── .gitignore                # Should include .ast_cache/
```

## Performance Optimization

### Monitoring Cache Effectiveness

Use `cache-info` regularly to monitor cache usage:

```bash
# Check current cache status
markitect cache-info

# Process some files
markitect ingest docs/*.md
markitect query "SELECT COUNT(*) FROM markdown_files"

# Check cache growth
markitect cache-info
```

### Cache Performance Characteristics

| File Size | First Parse | Cached Load | Improvement |
|-----------|-------------|-------------|-------------|
| Small (< 1KB) | ~10ms | ~3ms | 70% |
| Medium (1-10KB) | ~50ms | ~15ms | 70% |
| Large (> 10KB) | ~200ms | ~25ms | 85% |

### Best Practices

#### For Daily Usage

1. **Let it work automatically** - No manual intervention needed
2. **Monitor disk usage** - Use `cache-info` periodically
3. **Clean when needed** - Use `cache-clean` if disk space is limited

#### For Development

1. **Add to .gitignore** - Cache files shouldn't be version controlled
2. **Clean during debugging** - Use `cache-invalidate` for specific issues
3. **Performance testing** - Monitor cache effectiveness with `cache-info`

#### For Production

1. **Plan disk space** - Cache grows with content
2. **Backup strategy** - Source files matter, cache is regenerable
3. **Monitoring** - Include cache metrics in system monitoring

## Troubleshooting

### Common Issues

**"Cache directory does not exist - nothing to clean"**
- Normal when no files have been processed yet
- Cache directory is created automatically on first use

**"No cache found for filename.md - nothing to invalidate"**
- File hasn't been processed by MarkiTect yet
- Use `markitect ingest filename.md` first

**Poor cache performance**
- Check if files are being modified frequently
- Verify cache directory is on fast storage (SSD recommended)
- Monitor cache hit rates with repeated `cache-info` calls

### Advanced Diagnostics

```bash
# Check if cache is being used effectively
markitect cache-info
markitect status docs/large-file.md  # Should be fast if cached
markitect cache-info  # File count should be same (cache hit)

# Force fresh parsing for comparison
markitect cache-invalidate docs/large-file.md
time markitect status docs/large-file.md  # Measure parse time
time markitect status docs/large-file.md  # Measure cache load time
```

## Integration with Other Features

### Database Queries
Cache improves performance of database operations that access document content:
```bash
markitect query "SELECT filename, title FROM markdown_files WHERE content LIKE '%architecture%'"
```

### Batch Operations
Cache provides significant benefits for batch processing:
```bash
markitect ingest docs/*.md  # First run: parse + cache
markitect query "SELECT COUNT(*) FROM markdown_files"  # Subsequent: cache only
```

## Technical Details

For detailed technical information about cache implementation, see:
- [Architecture: Caching System](../architecture/caching-system.md)
- [Development: Performance Testing](../development/performance-testing.md) *(coming soon)*

---

The cache system is designed to be invisible during normal usage while providing powerful tools for monitoring and optimization when needed.