Files
markitect-main/tests/integration/prompts/test_prompt_cli.py
tegwick 7b4bd461c9 feat(prompts): implement Phase 8 - Observability & Traceability (FR-11)
Complete implementation of Phase 8, the final phase of prompt dependency
resolution infrastructure, adding full observability and traceability.

## Features (FR-11)

### FR-11.1: Complete Artifact Provenance Tracing
- TraceabilityService: composition layer for full artifact lineage
- Trace any artifact to producing PromptTemplate, input artifacts,
  generator runs, and quality validation results
- ProvenanceTrace model with complete dependency chain reconstruction
- RunSummary and ArtifactLineage models for structured trace output

### FR-11.2: Recomputation Query Infrastructure
- PromptQueryService: cross-service complex queries
- Run history queries with template and status filters
- Stale artifact detection via impact debt analysis
- Dependency graph statistics (nodes, edges, cycles, roots, leaves)
- Content-based artifact lookups by digest

### Visualization Support
- GraphExporter: DOT (Graphviz) and Mermaid format export
- Supports all edge types (requires, generates, includes)
- Handles isolated nodes, linear chains, diamonds, and complex graphs

### CLI Commands (prompt group)
- `prompt trace <artifact_id>` - Full provenance trace as JSON
- `prompt graph <artifact_id>` - Dependency graph (DOT/Mermaid)
- `prompt runs` - List execution runs with filters
- `prompt debt` - Show impact debt and stale artifacts
- `prompt stats` - Dependency graph statistics

## Implementation

Source files (8):
- markitect/prompts/traceability/models.py - Trace data models
- markitect/prompts/traceability/service.py - TraceabilityService
- markitect/prompts/visualization/graph.py - Graph export
- markitect/prompts/queries/operations.py - PromptQueryService
- markitect/prompts/cli.py - Click CLI commands
- Package __init__.py files (3)

Tests (64 total, all passing):
- tests/unit/prompts/test_traceability_service.py (21 tests)
- tests/unit/prompts/test_visualization.py (14 tests)
- tests/unit/prompts/test_query_operations.py (12 tests)
- tests/integration/prompts/test_traceability_workflow.py (7 tests)
- tests/integration/prompts/test_prompt_cli.py (10 tests)

## Architecture

TraceabilityService is a composition layer that delegates to:
- DependencyQueryService (transitive dependency lookups)
- QualityValidator (validation history)
- IncrementalExecutionEngine (impact debt queries)
- Direct repository access (artifacts, edges)

No duplicate data storage - all data comes from existing Phase 1-7
infrastructure (artifact repo, dependency repo, validation DB, debt DB).

## Verification

All 2250 tests pass with 0 regressions.
Phase 8 completes the full 8-phase implementation roadmap.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 20:32:18 +01:00

180 lines
5.7 KiB
Python

"""
Integration tests for prompt CLI commands.
Tests Click CLI commands with CliRunner and a real SQLite database.
"""
import json
import pytest
import tempfile
from pathlib import Path
from click.testing import CliRunner
from markitect.prompts.cli import prompt_commands
from markitect.prompts.dependencies.models import DependencyEdge, EdgeType
from markitect.prompts.dependencies.repository import SQLiteDependencyRepository
from markitect.prompts.models import Artifact, ArtifactType
from markitect.prompts.repositories.sqlite import SQLiteArtifactRepository
@pytest.fixture
def temp_db():
"""Create temporary database for testing."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
db_path = f.name
yield db_path
Path(db_path).unlink(missing_ok=True)
@pytest.fixture
def runner():
"""Click CLI test runner."""
return CliRunner()
@pytest.fixture
def seeded_db(temp_db):
"""Seed database with test data and return path."""
art_repo = SQLiteArtifactRepository(temp_db)
dep_repo = SQLiteDependencyRepository(temp_db)
a = Artifact.create(space_id="s", name="a", content="content-a",
artifact_type=ArtifactType.CONTENT)
b = Artifact.create(space_id="s", name="b", content="content-b",
artifact_type=ArtifactType.CONTENT)
a = art_repo.create(a)
b = art_repo.create(b)
edge = DependencyEdge.create(
source_artifact_id=a.id,
target_artifact_id=b.id,
run_id="run-1",
edge_type=EdgeType.REQUIRES,
)
dep_repo.create(edge)
return temp_db, a.id, b.id
class TestTraceCommand:
"""Tests for the 'trace' command."""
def test_trace_existing_artifact(self, runner, seeded_db):
"""Test tracing an existing artifact."""
db_path, art_a, art_b = seeded_db
result = runner.invoke(
prompt_commands, ["trace", art_a, "--database", db_path]
)
assert result.exit_code == 0
data = json.loads(result.output)
assert data["artifact_id"] == art_a
def test_trace_nonexistent_artifact(self, runner, temp_db):
"""Test tracing a non-existent artifact."""
# Initialize tables
SQLiteArtifactRepository(temp_db)
SQLiteDependencyRepository(temp_db)
result = runner.invoke(
prompt_commands, ["trace", "nonexistent", "--database", temp_db]
)
assert result.exit_code != 0
class TestGraphCommand:
"""Tests for the 'graph' command."""
def test_graph_mermaid(self, runner, seeded_db):
"""Test graph export in mermaid format."""
db_path, art_a, _ = seeded_db
result = runner.invoke(
prompt_commands,
["graph", art_a, "--format", "mermaid", "--database", db_path],
)
assert result.exit_code == 0
assert "graph LR" in result.output
def test_graph_dot(self, runner, seeded_db):
"""Test graph export in DOT format."""
db_path, art_a, _ = seeded_db
result = runner.invoke(
prompt_commands,
["graph", art_a, "--format", "dot", "--database", db_path],
)
assert result.exit_code == 0
assert "digraph" in result.output
class TestRunsCommand:
"""Tests for the 'runs' command."""
def test_runs_empty(self, runner, seeded_db):
"""Test listing runs when none are registered."""
db_path, _, _ = seeded_db
result = runner.invoke(
prompt_commands, ["runs", "--database", db_path]
)
assert result.exit_code == 0
assert "No runs found" in result.output
def test_runs_with_limit(self, runner, seeded_db):
"""Test listing runs with limit option."""
db_path, _, _ = seeded_db
result = runner.invoke(
prompt_commands, ["runs", "--limit", "5", "--database", db_path]
)
assert result.exit_code == 0
class TestDebtCommand:
"""Tests for the 'debt' command."""
def test_debt_no_stale(self, runner, seeded_db):
"""Test debt when no stale artifacts exist."""
db_path, _, _ = seeded_db
result = runner.invoke(
prompt_commands, ["debt", "--database", db_path]
)
assert result.exit_code == 0
assert "No stale artifacts" in result.output
def test_debt_for_artifact(self, runner, seeded_db):
"""Test debt for a specific artifact."""
db_path, art_a, _ = seeded_db
result = runner.invoke(
prompt_commands,
["debt", "--artifact", art_a, "--database", db_path],
)
assert result.exit_code == 0
assert "No impact debt" in result.output
class TestStatsCommand:
"""Tests for the 'stats' command."""
def test_stats_with_data(self, runner, seeded_db):
"""Test stats with real graph data."""
db_path, _, _ = seeded_db
result = runner.invoke(
prompt_commands, ["stats", "--database", db_path]
)
assert result.exit_code == 0
data = json.loads(result.output)
assert data["total_nodes"] == 2
assert data["total_edges"] == 1
assert data["has_cycles"] is False
def test_stats_empty_db(self, runner, temp_db):
"""Test stats on empty database."""
SQLiteArtifactRepository(temp_db)
SQLiteDependencyRepository(temp_db)
result = runner.invoke(
prompt_commands, ["stats", "--database", temp_db]
)
assert result.exit_code == 0
data = json.loads(result.output)
assert data["total_nodes"] == 0
assert data["total_edges"] == 0