markitect-main/TEST_ARCHITECTURE.md

MarkiTect Test Architecture & Execution Strategy

Architectural test organization and reverse dependency execution order

This document defines the test architecture based on the 7-layer system architecture, establishing clear naming conventions and optimal execution order for faster feedback and reduced debugging time.

🏗️ Test Architecture Mapping

Layer-Based Test Organization

Tests are organized by architectural layer with execution order optimized for dependency flow:

Test Execution Order (Foundation → Presentation):
Foundation → Infrastructure → Integration → Domain → Service → Application → Presentation
    ↑               ↑              ↑          ↑         ↑           ↑             ↑
  Fastest       Cache Tests   Gitea API   Business   Service    Workflows    CLI Tests
  Feedback        DB Tests      Tests      Logic     Tests      Tests        E2E Tests

---

📋 Current Test Files → New Architecture Mapping

Layer 7: Foundation Tests (Execute First - Fastest Feedback)

Core technology and utility validation

Current Test File	New Test File	Test Focus	Execution Priority
test_parser.py	test_l7_foundation_markdown_parsing.py	Core markdown parsing engine	1
test_issue_1_database_initialization.py	test_l7_foundation_database_core.py	SQLite database initialization	2

Rationale: Foundation layer failures break everything downstream. Test these first for immediate feedback.

---

Layer 5: Infrastructure Tests (Execute Second - Core Systems)

Technical capabilities and system resources

Current Test File	New Test File	Test Focus	Execution Priority
test_issue_2_file_ingestion.py	test_l5_infrastructure_ast_processing.py	AST processing and caching	3
test_issue_13_cache_commands.py	test_l5_infrastructure_cache_management.py	Cache system operations	4
test_issue_13_cache_info_command.py	test_l5_infrastructure_cache_monitoring.py	Cache statistics and monitoring	5
test_issue_14_query_commands.py	test_l5_infrastructure_database_queries.py	Database query interface	6
test_config_cli_commands.py	test_l5_infrastructure_configuration.py	Configuration management	7
unit/infrastructure/test_testing_infrastructure.py	test_l5_infrastructure_test_framework.py	Testing infrastructure	8

Rationale: Infrastructure provides technical foundation for all business logic. Failures here cascade widely.

---

Layer 6: Integration Tests (Execute Third - External Dependencies)

External system integration and APIs

Current Test File	New Test File	Test Focus	Execution Priority
test_gitea_facade.py	test_l6_integration_gitea_api.py	Gitea API client functionality	9
test_issue_creator.py	test_l6_integration_issue_creation.py	Issue creation via external APIs	10

Rationale: Integration failures are often environmental. Test after core systems are validated.

---

Layer 3: Domain Tests (Execute Fourth - Business Logic)

Pure business logic validation

Current Test File	New Test File	Test Focus	Execution Priority
unit/domain/issues/test_issue_models.py	test_l3_domain_issue_models.py	Issue domain models and business rules	11
unit/domain/issues/test_issue_services.py	test_l3_domain_issue_services.py	Issue business logic services	12
unit/domain/projects/test_project_models.py	test_l3_domain_project_models.py	Project domain models and logic	13
e2e/performance/test_domain_performance.py	test_l3_domain_performance_validation.py	Domain logic performance characteristics	14

Rationale: Domain logic is independent of technical concerns. Test after infrastructure is stable.

---

Layer 4: Service Tests (Execute Fifth - Application Services)

Cross-cutting concerns and service coordination

Current Test File	New Test File	Test Focus	Execution Priority
test_issue_4_retrieve_all_files.py	test_l4_service_document_management.py	Document service operations	15
test_issue_2_get_modify_commands.py	test_l4_service_document_modification.py	Document modification services	16
test_issue_15_ast_commands.py	test_l4_service_ast_analysis.py	AST analysis services	17
test_issue_14_output_formatting.py	test_l4_service_output_formatting.py	Output formatting services	18

Rationale: Services coordinate domain and infrastructure. Test after both layers are stable.

---

Layer 2: Application Tests (Execute Sixth - Use Cases & Workflows)

Workflow orchestration and use case validation

Current Test File	New Test File	Test Focus	Execution Priority
test_issue_11_workspace_creation.py	test_l2_application_workspace_workflows.py	TDD workspace management workflows	19
test_issue_11_workspace_creation_validation.py	test_l2_application_workspace_validation.py	Workspace workflow validation	20
test_issue_11_workflow_integration.py	test_l2_application_tdd_workflows.py	Complete TDD workflow integration	21
test_issue_11_feature.py	test_l2_application_feature_workflows.py	Feature development workflows	22

Rationale: Applications orchestrate multiple services. Test after service layer is validated.

---

Layer 1: Presentation Tests (Execute Last - User Interface)

CLI and user interaction validation

Current Test File	New Test File	Test Focus	Execution Priority
e2e/cli/test_issue_commands_e2e.py	test_l1_presentation_cli_interface.py	End-to-end CLI command testing	23

Rationale: Presentation layer depends on all other layers. Test last for comprehensive integration validation.

---

🚀 Test Execution Configuration

Reverse Dependency Order Execution

# Execute in dependency order for optimal feedback
pytest tests/test_l7_foundation_*.py           # Foundation (fastest)
pytest tests/test_l5_infrastructure_*.py       # Infrastructure
pytest tests/test_l6_integration_*.py          # Integration
pytest tests/test_l3_domain_*.py               # Domain
pytest tests/test_l4_service_*.py              # Service
pytest tests/test_l2_application_*.py          # Application
pytest tests/test_l1_presentation_*.py         # Presentation (slowest)

pytest.ini Configuration

Create optimized test execution configuration:

[tool:pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

# Test execution order optimization
collect_ignore = ["setup.py"]
addopts =
    --strict-markers
    --strict-config
    --verbose
    --tb=short
    --durations=10

# Custom test execution order
markers =
    foundation: Foundation layer tests (execute first)
    infrastructure: Infrastructure layer tests
    integration: Integration layer tests
    domain: Domain layer tests
    service: Service layer tests
    application: Application layer tests
    presentation: Presentation layer tests (execute last)
    slow: Slow tests requiring longer timeout
    performance: Performance validation tests

Custom Test Runner Script

#!/usr/bin/env python3
"""
Architectural test runner with dependency-optimized execution order.
"""

import subprocess
import sys
from pathlib import Path

def run_layer_tests(layer_name: str, pattern: str) -> bool:
    """Run tests for a specific architectural layer."""
    print(f"\n🧪 Testing {layer_name} Layer...")
    print("=" * 50)

    cmd = [
        "python", "-m", "pytest",
        f"tests/{pattern}",
        "-v", "--tb=short",
        f"--durations=5"
    ]

    result = subprocess.run(cmd)
    success = result.returncode == 0

    if success:
        print(f"✅ {layer_name} layer tests PASSED")
    else:
        print(f"❌ {layer_name} layer tests FAILED")

    return success

def main():
    """Execute tests in reverse dependency order."""
    layers = [
        ("Foundation", "test_l7_foundation_*.py"),
        ("Infrastructure", "test_l5_infrastructure_*.py"),
        ("Integration", "test_l6_integration_*.py"),
        ("Domain", "test_l3_domain_*.py"),
        ("Service", "test_l4_service_*.py"),
        ("Application", "test_l2_application_*.py"),
        ("Presentation", "test_l1_presentation_*.py")
    ]

    print("🏗️  MarkiTect Architectural Test Runner")
    print("Executing tests in reverse dependency order...")

    failed_layers = []

    for layer_name, pattern in layers:
        if not run_layer_tests(layer_name, pattern):
            failed_layers.append(layer_name)
            print(f"\n⚠️  Stopping at {layer_name} layer failure")
            break

    if failed_layers:
        print(f"\n❌ Test execution stopped at layer: {failed_layers[0]}")
        print("Fix foundation issues before testing dependent layers.")
        sys.exit(1)
    else:
        print("\n✅ All architectural layers passed!")
        print("🎉 System is architecturally sound!")

if __name__ == "__main__":
    main()

---

📊 Test Execution Benefits

Performance Optimization

Strategy	Time Savings	Debugging Efficiency
Foundation First	60-80% faster feedback	Fix root causes immediately
Layer Isolation	40-60% reduced test time	Clear failure boundaries
Dependency Order	70-90% fewer cascade failures	Targeted fixes
Early Termination	80%+ time saved on foundation failures	Stop at first architectural break

Development Workflow Benefits

1. Immediate Feedback: Foundation failures caught in < 5 seconds
2. Targeted Debugging: Clear layer-specific failure isolation
3. Reduced Context Switching: Fix architectural layers systematically
4. Confidence Building: Green foundation = stable architecture
5. Parallel Development: Teams can work on different layers safely

---

🔄 Migration Plan

Phase 1: Test File Renaming (1-2 days)

# Foundation Layer
mv test_parser.py test_l7_foundation_markdown_parsing.py
mv test_issue_1_database_initialization.py test_l7_foundation_database_core.py

# Infrastructure Layer
mv test_issue_2_file_ingestion.py test_l5_infrastructure_ast_processing.py
mv test_issue_13_cache_commands.py test_l5_infrastructure_cache_management.py
mv test_config_cli_commands.py test_l5_infrastructure_configuration.py

# Integration Layer
mv test_gitea_facade.py test_l6_integration_gitea_api.py
mv test_issue_creator.py test_l6_integration_issue_creation.py

# Domain Layer
mv unit/domain/issues/test_issue_models.py test_l3_domain_issue_models.py
mv unit/domain/issues/test_issue_services.py test_l3_domain_issue_services.py
mv unit/domain/projects/test_project_models.py test_l3_domain_project_models.py

# Service Layer
mv test_issue_4_retrieve_all_files.py test_l4_service_document_management.py
mv test_issue_15_ast_commands.py test_l4_service_ast_analysis.py

# Application Layer
mv test_issue_11_workspace_creation.py test_l2_application_workspace_workflows.py
mv test_issue_11_workflow_integration.py test_l2_application_tdd_workflows.py

# Presentation Layer
mv e2e/cli/test_issue_commands_e2e.py test_l1_presentation_cli_interface.py

Phase 2: Test Configuration Setup (1 day)

1. Update pytest.ini with execution order markers
2. Create architectural test runner script
3. Update CI/CD pipeline configuration
4. Create test execution documentation

Phase 3: Team Training (1 day)

1. Document new test architecture
2. Train team on execution strategy
3. Update development workflow docs
4. Create architectural test guidelines

---

🎯 Quality Gates

Layer-Specific Quality Requirements

Layer	Coverage Target	Performance Target	Failure Rate Target
Foundation	98%+	< 5s execution	< 0.1% failure rate
Infrastructure	95%+	< 15s execution	< 0.5% failure rate
Integration	85%+	< 30s execution	< 2% failure rate
Domain	98%+	< 10s execution	< 0.2% failure rate
Service	90%+	< 20s execution	< 1% failure rate
Application	85%+	< 45s execution	< 2% failure rate
Presentation	80%+	< 60s execution	< 5% failure rate

Architectural Health Metrics

1. Foundation Stability: > 99.5% pass rate
2. Layer Isolation: Zero upward dependency failures
3. Execution Time: < 3 minutes for full architectural test suite
4. Debugging Efficiency: Average fix time < 15 minutes per layer

---

📝 Architectural Test Patterns

Foundation Layer Pattern

# test_l7_foundation_markdown_parsing.py
class TestFoundationMarkdownParsing:
    """Foundation layer: Core markdown parsing engine validation."""

    def test_markdown_parser_converts_heading_and_paragraph_to_ast_tokens(self):
        """FOUNDATION: Validate core parsing capability."""
        # Fast, isolated, no dependencies

Infrastructure Layer Pattern

# test_l5_infrastructure_cache_management.py
class TestInfrastructureCacheManagement:
    """Infrastructure layer: Cache system technical capabilities."""

    def test_cache_info_command_works_with_empty_and_populated_cache(self):
        """INFRASTRUCTURE: Validate cache system operations."""
        # Technical capability, depends on foundation

Domain Layer Pattern

# test_l3_domain_issue_models.py
class TestDomainIssueModels:
    """Domain layer: Pure business logic validation."""

    def test_issue_creation_with_valid_data(self):
        """DOMAIN: Validate business rules and logic."""
        # No infrastructure dependencies, pure logic

---

🚦 Continuous Integration Integration

GitHub Actions Workflow

name: Architectural Test Suite

on: [push, pull_request]

jobs:
  architectural-tests:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.12'

    - name: Install dependencies
      run: |
        pip install -r requirements.txt
        pip install -r requirements-test.txt

    - name: Run Foundation Tests (Layer 7)
      run: pytest tests/test_l7_foundation_*.py -v

    - name: Run Infrastructure Tests (Layer 5)
      run: pytest tests/test_l5_infrastructure_*.py -v
      if: success()

    - name: Run Integration Tests (Layer 6)
      run: pytest tests/test_l6_integration_*.py -v
      if: success()

    - name: Run Domain Tests (Layer 3)
      run: pytest tests/test_l3_domain_*.py -v
      if: success()

    - name: Run Service Tests (Layer 4)
      run: pytest tests/test_l4_service_*.py -v
      if: success()

    - name: Run Application Tests (Layer 2)
      run: pytest tests/test_l2_application_*.py -v
      if: success()

    - name: Run Presentation Tests (Layer 1)
      run: pytest tests/test_l1_presentation_*.py -v
      if: success()

---

This architectural test strategy transforms testing from a time-consuming bottleneck into a fast, targeted debugging tool that respects system architecture and optimizes developer productivity.