markitect-main/examples/content-generator/README.md

# Content Generator Example

**Demonstrates: Prompt Dependency Resolution for Systematic Content Generation**

This example shows how to use MarkiTect's Prompt Dependency Resolution infrastructure to generate InfoTech primers with full dependency tracking, provenance tracing, and quality validation.

## Quick Start

```bash
# Run the example
python generate_primers.py

# Or if executable:
./generate_primers.py
```

## What This Example Demonstrates

1. **Artifact Management**: Store templates, topics, and guidelines as versioned artifacts
2. **Dependency Resolution**: Automatically resolve `@{macro}` references in templates
3. **Provenance Tracking**: Trace generated content back to its inputs
4. **Incremental Updates**: Detect changes and regenerate only affected outputs
5. **Quality Validation**: Apply gates to ensure output meets standards
6. **Visualization**: Export dependency graphs to Mermaid or DOT format

## Files

```
content-generator/
├── README.md                    # This file
├── TUTORIAL.md                  # Comprehensive tutorial (START HERE)
├── generate_primers.py          # Example implementation
│
├── templates/
│   └── generate-primer.md       # PromptTemplate for primer generation
│
├── artifacts/
│   ├── topics/                  # Topic definitions
│   │   ├── etl.md
│   │   └── microservices.md
│   │
│   └── guidelines/              # Authoring standards
│       ├── authoring-rules.md
│       └── research-prompt.md
│
└── prepdr/                      # Original manual system (preserved)
    ├── README.md
    ├── ETL.md
    ├── AuthoringRules.md
    ├── AssistentPrompt.md
    └── GeneratePrimerTemplate.md
```

## Before vs After

### Before (prepdr/ - Manual System)

```markdown
# Manual macro substitution
{{ETL}}
{{AuthoringRules}}
```

Problems:
- No automation
- No version tracking
- No dependency awareness
- Can't detect changes
- No traceability

### After (With Infrastructure)

```markdown
# Automatic resolution
@{topic}
@{authoring_rules}
```

Benefits:
- ✅ Automatic macro resolution
- ✅ SHA-256 content digests for change detection
- ✅ Full dependency graph
- ✅ Incremental recomputation
- ✅ Complete provenance traces
- ✅ CLI tools for inspection

## Tutorial

**[Read the full tutorial →](TUTORIAL.md)**

The tutorial covers:
- Architecture and core concepts
- Step-by-step implementation walkthrough
- Advanced features (incremental execution, quality gates, visualization)
- CLI usage examples
- Best practices

## CLI Commands

After running the example, you can use CLI commands to inspect the system:

```bash
# Trace provenance
markitect prompt trace <artifact-id> --database primers.db

# Visualize dependencies
markitect prompt graph <artifact-id> --format mermaid --database primers.db

# List runs
markitect prompt runs --database primers.db

# Show impact debt (stale artifacts)
markitect prompt debt --database primers.db

# Graph statistics
markitect prompt stats --database primers.db
```

## Architecture

```mermaid
graph TB
    A[Topic: ETL] -->|requires| D[Run: generate-etl]
    B[Authoring Rules] -->|requires| D
    C[Research Prompt] -->|requires| D
    D -->|generates| E[Output: ETL Primer]

    style E fill:#90EE90
    style D fill:#87CEEB
```

## Key Features

### 1. Content-Based Addressing

Every artifact has a SHA-256 digest that changes when content changes:

```python
artifact.content_digest  # "9c4d6e8a..."
artifact.has_changed(new_digest)  # True if modified
```

### 2. Dependency Graph

Full graph construction enables:
- **Impact analysis**: "What needs regeneration?"
- **Provenance**: "How was this produced?"
- **Build order**: Topological sort for correct execution

### 3. Incremental Execution

Only regenerate what's affected by changes:

```python
config = RecomputeConfig(
    max_depth=2,              # Traverse 2 levels
    impact_threshold=0.1,     # Skip minor changes
    max_recomputes=10,        # Budget limit
)
```

### 4. Quality Validation

Apply automated checks:

```python
gate = PatternValidationGate(
    required_patterns=[r"## Definition", r"## Context"],
    forbidden_patterns=[r"TODO", r"FIXME"],
)
```

## Extending the Example

### Add New Topics

```bash
# 1. Create artifact file
cat > artifacts/topics/oauth.md << 'EOF'
---
id: topic-oauth
name: OAuth
artifact_type: content
---

# OAuth 2.0

An authorization framework that enables...
EOF

# 2. Run generator
python generate_primers.py
# Automatically picks up new topic
```

### Custom Quality Gates

```python
from markitect.prompts.quality.gates.schema_gate import SchemaValidationGate

# Validate primer structure
schema = {
    "type": "object",
    "required": ["definition", "context", "scope"],
}

gate = SchemaValidationGate(schema=schema, gate_id="primer-schema")
```

### Integrate with LLM

```python
# Replace mock generation with real LLM call
from openai import OpenAI

client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": compiled.content}]
)

output_content = response.choices[0].message.content
```

## Performance

Database operations are optimized:
- Indexed lookups on artifact ID, space, digest
- Indexed dependency queries by source, target, run
- Efficient graph traversal with BFS/DFS
- Content digest comparison (no full content comparison)

## Troubleshooting

### "Artifact not found"
- Ensure artifact files exist in correct directories
- Check file extension is `.md`
- Verify space IDs match between template and config

### "Resolution failed"
- Check macro names match artifact names
- Verify spaces are configured in ResolutionConfig
- Use `ResolutionStrategy.FIRST_MATCH` for simplicity

### "Circular dependency detected"
- Review dependency edges in database
- Use `detect_circular_dependencies()` to find cycles
- Refactor template structure to break cycles

## Related Documentation

- **[TUTORIAL.md](TUTORIAL.md)** - Comprehensive walkthrough
- **[Prompt Dependency Resolution Spec](../../roadmap/prompt-dependency-resolution/)** - Design documentation
- **[Phase 8 Implementation](../../markitect/prompts/)** - Source code

## License

Same as MarkiTect project.