Files
markitect-main/tests/unit/prompts/test_query_operations.py
tegwick 7b4bd461c9 feat(prompts): implement Phase 8 - Observability & Traceability (FR-11)
Complete implementation of Phase 8, the final phase of prompt dependency
resolution infrastructure, adding full observability and traceability.

## Features (FR-11)

### FR-11.1: Complete Artifact Provenance Tracing
- TraceabilityService: composition layer for full artifact lineage
- Trace any artifact to producing PromptTemplate, input artifacts,
  generator runs, and quality validation results
- ProvenanceTrace model with complete dependency chain reconstruction
- RunSummary and ArtifactLineage models for structured trace output

### FR-11.2: Recomputation Query Infrastructure
- PromptQueryService: cross-service complex queries
- Run history queries with template and status filters
- Stale artifact detection via impact debt analysis
- Dependency graph statistics (nodes, edges, cycles, roots, leaves)
- Content-based artifact lookups by digest

### Visualization Support
- GraphExporter: DOT (Graphviz) and Mermaid format export
- Supports all edge types (requires, generates, includes)
- Handles isolated nodes, linear chains, diamonds, and complex graphs

### CLI Commands (prompt group)
- `prompt trace <artifact_id>` - Full provenance trace as JSON
- `prompt graph <artifact_id>` - Dependency graph (DOT/Mermaid)
- `prompt runs` - List execution runs with filters
- `prompt debt` - Show impact debt and stale artifacts
- `prompt stats` - Dependency graph statistics

## Implementation

Source files (8):
- markitect/prompts/traceability/models.py - Trace data models
- markitect/prompts/traceability/service.py - TraceabilityService
- markitect/prompts/visualization/graph.py - Graph export
- markitect/prompts/queries/operations.py - PromptQueryService
- markitect/prompts/cli.py - Click CLI commands
- Package __init__.py files (3)

Tests (64 total, all passing):
- tests/unit/prompts/test_traceability_service.py (21 tests)
- tests/unit/prompts/test_visualization.py (14 tests)
- tests/unit/prompts/test_query_operations.py (12 tests)
- tests/integration/prompts/test_traceability_workflow.py (7 tests)
- tests/integration/prompts/test_prompt_cli.py (10 tests)

## Architecture

TraceabilityService is a composition layer that delegates to:
- DependencyQueryService (transitive dependency lookups)
- QualityValidator (validation history)
- IncrementalExecutionEngine (impact debt queries)
- Direct repository access (artifacts, edges)

No duplicate data storage - all data comes from existing Phase 1-7
infrastructure (artifact repo, dependency repo, validation DB, debt DB).

## Verification

All 2250 tests pass with 0 regressions.
Phase 8 completes the full 8-phase implementation roadmap.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 20:32:18 +01:00

185 lines
6.2 KiB
Python

"""
Unit tests for PromptQueryService.
Tests run history, impact analysis queries, dependency stats,
and artifact digest lookups.
"""
import pytest
import tempfile
from pathlib import Path
from markitect.prompts.dependencies.models import DependencyEdge, EdgeType
from markitect.prompts.dependencies.repository import SQLiteDependencyRepository
from markitect.prompts.execution.models import PromptRun, RunStatus
from markitect.prompts.models import Artifact, ArtifactType
from markitect.prompts.repositories.sqlite import SQLiteArtifactRepository
from markitect.prompts.queries.operations import PromptQueryService
@pytest.fixture
def temp_db():
"""Create temporary database for testing."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
db_path = f.name
yield db_path
Path(db_path).unlink(missing_ok=True)
@pytest.fixture
def artifact_repo(temp_db):
return SQLiteArtifactRepository(temp_db)
@pytest.fixture
def dep_repo(temp_db):
return SQLiteDependencyRepository(temp_db)
@pytest.fixture
def query_service(artifact_repo, dep_repo, temp_db):
return PromptQueryService(artifact_repo, dep_repo, db_path=temp_db)
def _make_artifact(repo, space_id, name, content="content"):
artifact = Artifact.create(
space_id=space_id, name=name, content=content,
artifact_type=ArtifactType.CONTENT,
)
return repo.create(artifact)
def _make_run(template_id, status=RunStatus.SUCCESS):
run = PromptRun.create(
template_id=template_id,
input_bundle_hash="hash-abc",
)
if status == RunStatus.SUCCESS:
run.mark_complete()
elif status == RunStatus.FAILED:
run.mark_failed("error")
return run
def _make_edge(repo, source, target, run_id):
edge = DependencyEdge.create(
source_artifact_id=source,
target_artifact_id=target,
run_id=run_id,
edge_type=EdgeType.REQUIRES,
)
return repo.create(edge)
class TestRunHistory:
"""Tests for get_run_history."""
def test_returns_all_runs(self, query_service):
"""Test returning all registered runs."""
run1 = _make_run("tmpl-1")
run2 = _make_run("tmpl-2")
query_service.register_run(run1)
query_service.register_run(run2)
history = query_service.get_run_history()
assert len(history) == 2
def test_filter_by_template(self, query_service):
"""Test filtering by template_id."""
run1 = _make_run("tmpl-1")
run2 = _make_run("tmpl-2")
query_service.register_run(run1)
query_service.register_run(run2)
history = query_service.get_run_history(template_id="tmpl-1")
assert len(history) == 1
assert history[0]["template_id"] == "tmpl-1"
def test_filter_by_status(self, query_service):
"""Test filtering by status."""
run_ok = _make_run("tmpl-1", RunStatus.SUCCESS)
run_fail = _make_run("tmpl-1", RunStatus.FAILED)
query_service.register_run(run_ok)
query_service.register_run(run_fail)
history = query_service.get_run_history(status="success")
assert len(history) == 1
assert history[0]["status"] == "success"
def test_limit(self, query_service):
"""Test limit parameter."""
for i in range(10):
run = _make_run(f"tmpl-{i}")
query_service.register_run(run)
history = query_service.get_run_history(limit=3)
assert len(history) == 3
def test_empty_history(self, query_service):
"""Test empty run history."""
assert query_service.get_run_history() == []
class TestStaleArtifacts:
"""Tests for get_stale_artifacts."""
def test_no_debt_returns_empty(self, query_service):
"""Test returns empty when no debt exists."""
assert query_service.get_stale_artifacts() == []
def test_no_engine_returns_empty(self, artifact_repo, dep_repo):
"""Test returns empty without db_path."""
svc = PromptQueryService(artifact_repo, dep_repo, db_path=None)
assert svc.get_stale_artifacts() == []
class TestDependencyStats:
"""Tests for get_dependency_stats."""
def test_empty_graph(self, query_service):
"""Test stats for empty graph."""
stats = query_service.get_dependency_stats()
assert stats["total_nodes"] == 0
assert stats["total_edges"] == 0
assert stats["has_cycles"] is False
def test_simple_graph(self, query_service, artifact_repo, dep_repo):
"""Test stats for a simple graph."""
a = _make_artifact(artifact_repo, "s", "a")
b = _make_artifact(artifact_repo, "s", "b")
c = _make_artifact(artifact_repo, "s", "c")
_make_edge(dep_repo, a.id, b.id, "r1")
_make_edge(dep_repo, b.id, c.id, "r1")
stats = query_service.get_dependency_stats()
assert stats["total_nodes"] == 3
assert stats["total_edges"] == 2
assert stats["root_count"] == 1 # 'a' is root
assert stats["leaf_count"] == 1 # 'c' is leaf
assert stats["has_cycles"] is False
class TestFindArtifactsByDigest:
"""Tests for find_artifacts_by_digest."""
def test_finds_matching(self, query_service, artifact_repo):
"""Test finding artifacts by content digest."""
art = _make_artifact(artifact_repo, "s", "test-art", "hello")
results = query_service.find_artifacts_by_digest(art.content_digest)
assert len(results) == 1
assert results[0]["artifact_id"] == art.id
def test_no_match(self, query_service):
"""Test returns empty for non-matching digest."""
assert query_service.find_artifacts_by_digest("nonexistent") == []
def test_multiple_matches(self, query_service, artifact_repo):
"""Test finding multiple artifacts with same digest."""
_make_artifact(artifact_repo, "s1", "a", "same-content")
_make_artifact(artifact_repo, "s2", "a", "same-content")
art = _make_artifact(artifact_repo, "s3", "a", "same-content")
results = query_service.find_artifacts_by_digest(art.content_digest)
assert len(results) == 3