tegwick
360c3b1de2
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
This example demonstrates the full workflow of generating InfoTech primers
using MarkiTect's Prompt Dependency Resolution infrastructure.
Features demonstrated:
- Artifact creation and storage with content-based addressing
- PromptTemplate with @{macro} resolution across multiple spaces
- Automatic dependency tracking and graph construction
- Provenance tracing from outputs back to inputs
- Visualization export (Mermaid format)
- Incremental execution with change detection
Files added:
- generate_primers.py: Complete working example
- README.md: Quick start guide and architecture overview
- TUTORIAL.md: Comprehensive 500+ line tutorial
- templates/generate-primer.md: Template with macros
- artifacts/topics/: ETL and Microservices topic definitions
- artifacts/guidelines/: Authoring rules and research protocol
- prepdr/: Original manual system (preserved for reference)
Example output:
- Generates 2 primers (ETL, Microservices)
- Creates 8 artifacts across 4 information spaces
- Records 8 dependency edges in SQLite database
- Exports dependency graph visualization
Run with: cd examples/content-generator && python generate_primers.py
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
32 lines
1.0 KiB
Markdown
32 lines
1.0 KiB
Markdown
---
|
|
id: topic-etl
|
|
name: ETL
|
|
artifact_type: content
|
|
description: Topic definition for ETL (Extract, Transform, Load)
|
|
version: 1.0.0
|
|
tags:
|
|
- data-engineering
|
|
- data-integration
|
|
- topic
|
|
---
|
|
|
|
# ETL (Extract, Transform, Load)
|
|
|
|
A three-phase computing process where data is extracted from source systems, transformed (including validation, cleaning, enrichment, and aggregation), and loaded into a target data store or data warehouse.
|
|
|
|
ETL is a fundamental pattern in data integration and analytics pipelines, enabling organizations to consolidate data from heterogeneous sources into a unified format suitable for analysis and reporting.
|
|
|
|
**Key Characteristics:**
|
|
- Sequential batch-oriented processing
|
|
- Data quality enforcement during transformation
|
|
- Schema mapping and normalization
|
|
- Support for diverse source and target systems
|
|
- Typically scheduled and automated
|
|
|
|
**Common Use Cases:**
|
|
- Data warehouse population
|
|
- Business intelligence reporting
|
|
- Data migration projects
|
|
- Master data management
|
|
- Regulatory compliance reporting
|