feat(examples): add content-generator example demonstrating Prompt Dependency Resolution
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
This example demonstrates the full workflow of generating InfoTech primers
using MarkiTect's Prompt Dependency Resolution infrastructure.
Features demonstrated:
- Artifact creation and storage with content-based addressing
- PromptTemplate with @{macro} resolution across multiple spaces
- Automatic dependency tracking and graph construction
- Provenance tracing from outputs back to inputs
- Visualization export (Mermaid format)
- Incremental execution with change detection
Files added:
- generate_primers.py: Complete working example
- README.md: Quick start guide and architecture overview
- TUTORIAL.md: Comprehensive 500+ line tutorial
- templates/generate-primer.md: Template with macros
- artifacts/topics/: ETL and Microservices topic definitions
- artifacts/guidelines/: Authoring rules and research protocol
- prepdr/: Original manual system (preserved for reference)
Example output:
- Generates 2 primers (ETL, Microservices)
- Creates 8 artifacts across 4 information spaces
- Records 8 dependency edges in SQLite database
- Exports dependency graph visualization
Run with: cd examples/content-generator && python generate_primers.py
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
258
examples/content-generator/README.md
Normal file
258
examples/content-generator/README.md
Normal file
@@ -0,0 +1,258 @@
|
||||
# Content Generator Example
|
||||
|
||||
**Demonstrates: Prompt Dependency Resolution for Systematic Content Generation**
|
||||
|
||||
This example shows how to use MarkiTect's Prompt Dependency Resolution infrastructure to generate InfoTech primers with full dependency tracking, provenance tracing, and quality validation.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Run the example
|
||||
python generate_primers.py
|
||||
|
||||
# Or if executable:
|
||||
./generate_primers.py
|
||||
```
|
||||
|
||||
## What This Example Demonstrates
|
||||
|
||||
1. **Artifact Management**: Store templates, topics, and guidelines as versioned artifacts
|
||||
2. **Dependency Resolution**: Automatically resolve `@{macro}` references in templates
|
||||
3. **Provenance Tracking**: Trace generated content back to its inputs
|
||||
4. **Incremental Updates**: Detect changes and regenerate only affected outputs
|
||||
5. **Quality Validation**: Apply gates to ensure output meets standards
|
||||
6. **Visualization**: Export dependency graphs to Mermaid or DOT format
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
content-generator/
|
||||
├── README.md # This file
|
||||
├── TUTORIAL.md # Comprehensive tutorial (START HERE)
|
||||
├── generate_primers.py # Example implementation
|
||||
│
|
||||
├── templates/
|
||||
│ └── generate-primer.md # PromptTemplate for primer generation
|
||||
│
|
||||
├── artifacts/
|
||||
│ ├── topics/ # Topic definitions
|
||||
│ │ ├── etl.md
|
||||
│ │ └── microservices.md
|
||||
│ │
|
||||
│ └── guidelines/ # Authoring standards
|
||||
│ ├── authoring-rules.md
|
||||
│ └── research-prompt.md
|
||||
│
|
||||
└── prepdr/ # Original manual system (preserved)
|
||||
├── README.md
|
||||
├── ETL.md
|
||||
├── AuthoringRules.md
|
||||
├── AssistentPrompt.md
|
||||
└── GeneratePrimerTemplate.md
|
||||
```
|
||||
|
||||
## Before vs After
|
||||
|
||||
### Before (prepdr/ - Manual System)
|
||||
|
||||
```markdown
|
||||
# Manual macro substitution
|
||||
{{ETL}}
|
||||
{{AuthoringRules}}
|
||||
```
|
||||
|
||||
Problems:
|
||||
- No automation
|
||||
- No version tracking
|
||||
- No dependency awareness
|
||||
- Can't detect changes
|
||||
- No traceability
|
||||
|
||||
### After (With Infrastructure)
|
||||
|
||||
```markdown
|
||||
# Automatic resolution
|
||||
@{topic}
|
||||
@{authoring_rules}
|
||||
```
|
||||
|
||||
Benefits:
|
||||
- ✅ Automatic macro resolution
|
||||
- ✅ SHA-256 content digests for change detection
|
||||
- ✅ Full dependency graph
|
||||
- ✅ Incremental recomputation
|
||||
- ✅ Complete provenance traces
|
||||
- ✅ CLI tools for inspection
|
||||
|
||||
## Tutorial
|
||||
|
||||
**[Read the full tutorial →](TUTORIAL.md)**
|
||||
|
||||
The tutorial covers:
|
||||
- Architecture and core concepts
|
||||
- Step-by-step implementation walkthrough
|
||||
- Advanced features (incremental execution, quality gates, visualization)
|
||||
- CLI usage examples
|
||||
- Best practices
|
||||
|
||||
## CLI Commands
|
||||
|
||||
After running the example, you can use CLI commands to inspect the system:
|
||||
|
||||
```bash
|
||||
# Trace provenance
|
||||
markitect prompt trace <artifact-id> --database primers.db
|
||||
|
||||
# Visualize dependencies
|
||||
markitect prompt graph <artifact-id> --format mermaid --database primers.db
|
||||
|
||||
# List runs
|
||||
markitect prompt runs --database primers.db
|
||||
|
||||
# Show impact debt (stale artifacts)
|
||||
markitect prompt debt --database primers.db
|
||||
|
||||
# Graph statistics
|
||||
markitect prompt stats --database primers.db
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
A[Topic: ETL] -->|requires| D[Run: generate-etl]
|
||||
B[Authoring Rules] -->|requires| D
|
||||
C[Research Prompt] -->|requires| D
|
||||
D -->|generates| E[Output: ETL Primer]
|
||||
|
||||
style E fill:#90EE90
|
||||
style D fill:#87CEEB
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Content-Based Addressing
|
||||
|
||||
Every artifact has a SHA-256 digest that changes when content changes:
|
||||
|
||||
```python
|
||||
artifact.content_digest # "9c4d6e8a..."
|
||||
artifact.has_changed(new_digest) # True if modified
|
||||
```
|
||||
|
||||
### 2. Dependency Graph
|
||||
|
||||
Full graph construction enables:
|
||||
- **Impact analysis**: "What needs regeneration?"
|
||||
- **Provenance**: "How was this produced?"
|
||||
- **Build order**: Topological sort for correct execution
|
||||
|
||||
### 3. Incremental Execution
|
||||
|
||||
Only regenerate what's affected by changes:
|
||||
|
||||
```python
|
||||
config = RecomputeConfig(
|
||||
max_depth=2, # Traverse 2 levels
|
||||
impact_threshold=0.1, # Skip minor changes
|
||||
max_recomputes=10, # Budget limit
|
||||
)
|
||||
```
|
||||
|
||||
### 4. Quality Validation
|
||||
|
||||
Apply automated checks:
|
||||
|
||||
```python
|
||||
gate = PatternValidationGate(
|
||||
required_patterns=[r"## Definition", r"## Context"],
|
||||
forbidden_patterns=[r"TODO", r"FIXME"],
|
||||
)
|
||||
```
|
||||
|
||||
## Extending the Example
|
||||
|
||||
### Add New Topics
|
||||
|
||||
```bash
|
||||
# 1. Create artifact file
|
||||
cat > artifacts/topics/oauth.md << 'EOF'
|
||||
---
|
||||
id: topic-oauth
|
||||
name: OAuth
|
||||
artifact_type: content
|
||||
---
|
||||
|
||||
# OAuth 2.0
|
||||
|
||||
An authorization framework that enables...
|
||||
EOF
|
||||
|
||||
# 2. Run generator
|
||||
python generate_primers.py
|
||||
# Automatically picks up new topic
|
||||
```
|
||||
|
||||
### Custom Quality Gates
|
||||
|
||||
```python
|
||||
from markitect.prompts.quality.gates.schema_gate import SchemaValidationGate
|
||||
|
||||
# Validate primer structure
|
||||
schema = {
|
||||
"type": "object",
|
||||
"required": ["definition", "context", "scope"],
|
||||
}
|
||||
|
||||
gate = SchemaValidationGate(schema=schema, gate_id="primer-schema")
|
||||
```
|
||||
|
||||
### Integrate with LLM
|
||||
|
||||
```python
|
||||
# Replace mock generation with real LLM call
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI()
|
||||
response = client.chat.completions.create(
|
||||
model="gpt-4",
|
||||
messages=[{"role": "user", "content": compiled.content}]
|
||||
)
|
||||
|
||||
output_content = response.choices[0].message.content
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
Database operations are optimized:
|
||||
- Indexed lookups on artifact ID, space, digest
|
||||
- Indexed dependency queries by source, target, run
|
||||
- Efficient graph traversal with BFS/DFS
|
||||
- Content digest comparison (no full content comparison)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Artifact not found"
|
||||
- Ensure artifact files exist in correct directories
|
||||
- Check file extension is `.md`
|
||||
- Verify space IDs match between template and config
|
||||
|
||||
### "Resolution failed"
|
||||
- Check macro names match artifact names
|
||||
- Verify spaces are configured in ResolutionConfig
|
||||
- Use `ResolutionStrategy.FIRST_MATCH` for simplicity
|
||||
|
||||
### "Circular dependency detected"
|
||||
- Review dependency edges in database
|
||||
- Use `detect_circular_dependencies()` to find cycles
|
||||
- Refactor template structure to break cycles
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **[TUTORIAL.md](TUTORIAL.md)** - Comprehensive walkthrough
|
||||
- **[Prompt Dependency Resolution Spec](../../roadmap/prompt-dependency-resolution/)** - Design documentation
|
||||
- **[Phase 8 Implementation](../../markitect/prompts/)** - Source code
|
||||
|
||||
## License
|
||||
|
||||
Same as MarkiTect project.
|
||||
Reference in New Issue
Block a user