markitect-main/TEST_ARCHITECTURE.md

# MarkiTect Test Architecture & Execution Strategy

> **Architectural test organization and reverse dependency execution order**

This document defines the test architecture based on the 7-layer system architecture, establishing clear naming conventions and optimal execution order for faster feedback and reduced debugging time.

## 🏗️ Test Architecture Mapping

### Layer-Based Test Organization

Tests are organized by architectural layer with execution order optimized for dependency flow:

```
Test Execution Order (Foundation → Presentation):
Foundation → Infrastructure → Integration → Domain → Service → Application → Presentation
    ↑               ↑              ↑          ↑         ↑           ↑             ↑
  Fastest       Cache Tests   Gitea API   Business   Service    Workflows    CLI Tests
  Feedback        DB Tests      Tests      Logic     Tests      Tests        E2E Tests
```

---

## 📋 Current Test Files → New Architecture Mapping

### **Layer 7: Foundation Tests** (Execute First - Fastest Feedback)
*Core technology and utility validation*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `test_parser.py` | `test_l7_foundation_markdown_parsing.py` | Core markdown parsing engine | **1** |
| `test_issue_1_database_initialization.py` | `test_l7_foundation_database_core.py` | SQLite database initialization | **2** |

**Rationale**: Foundation layer failures break everything downstream. Test these first for immediate feedback.

---

### **Layer 5: Infrastructure Tests** (Execute Second - Core Systems)
*Technical capabilities and system resources*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `test_issue_2_file_ingestion.py` | `test_l5_infrastructure_ast_processing.py` | AST processing and caching | **3** |
| `test_issue_13_cache_commands.py` | `test_l5_infrastructure_cache_management.py` | Cache system operations | **4** |
| `test_issue_13_cache_info_command.py` | `test_l5_infrastructure_cache_monitoring.py` | Cache statistics and monitoring | **5** |
| `test_issue_14_query_commands.py` | `test_l5_infrastructure_database_queries.py` | Database query interface | **6** |
| `test_config_cli_commands.py` | `test_l5_infrastructure_configuration.py` | Configuration management | **7** |
| `unit/infrastructure/test_testing_infrastructure.py` | `test_l5_infrastructure_test_framework.py` | Testing infrastructure | **8** |

**Rationale**: Infrastructure provides technical foundation for all business logic. Failures here cascade widely.

---

### **Layer 6: Integration Tests** (Execute Third - External Dependencies)
*External system integration and APIs*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `test_gitea_facade.py` | `test_l6_integration_gitea_api.py` | Gitea API client functionality | **9** |
| `test_issue_creator.py` | `test_l6_integration_issue_creation.py` | Issue creation via external APIs | **10** |

**Rationale**: Integration failures are often environmental. Test after core systems are validated.

---

### **Layer 3: Domain Tests** (Execute Fourth - Business Logic)
*Pure business logic validation*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `unit/domain/issues/test_issue_models.py` | `test_l3_domain_issue_models.py` | Issue domain models and business rules | **11** |
| `unit/domain/issues/test_issue_services.py` | `test_l3_domain_issue_services.py` | Issue business logic services | **12** |
| `unit/domain/projects/test_project_models.py` | `test_l3_domain_project_models.py` | Project domain models and logic | **13** |
| `e2e/performance/test_domain_performance.py` | `test_l3_domain_performance_validation.py` | Domain logic performance characteristics | **14** |

**Rationale**: Domain logic is independent of technical concerns. Test after infrastructure is stable.

---

### **Layer 4: Service Tests** (Execute Fifth - Application Services)
*Cross-cutting concerns and service coordination*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `test_issue_4_retrieve_all_files.py` | `test_l4_service_document_management.py` | Document service operations | **15** |
| `test_issue_2_get_modify_commands.py` | `test_l4_service_document_modification.py` | Document modification services | **16** |
| `test_issue_15_ast_commands.py` | `test_l4_service_ast_analysis.py` | AST analysis services | **17** |
| `test_issue_14_output_formatting.py` | `test_l4_service_output_formatting.py` | Output formatting services | **18** |

**Rationale**: Services coordinate domain and infrastructure. Test after both layers are stable.

---

### **Layer 2: Application Tests** (Execute Sixth - Use Cases & Workflows)
*Workflow orchestration and use case validation*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `test_issue_11_workspace_creation.py` | `test_l2_application_workspace_workflows.py` | TDD workspace management workflows | **19** |
| `test_issue_11_workspace_creation_validation.py` | `test_l2_application_workspace_validation.py` | Workspace workflow validation | **20** |
| `test_issue_11_workflow_integration.py` | `test_l2_application_tdd_workflows.py` | Complete TDD workflow integration | **21** |
| `test_issue_11_feature.py` | `test_l2_application_feature_workflows.py` | Feature development workflows | **22** |

**Rationale**: Applications orchestrate multiple services. Test after service layer is validated.

---

### **Layer 1: Presentation Tests** (Execute Last - User Interface)
*CLI and user interaction validation*

| Current Test File | New Test File | Test Focus | Execution Priority |
|-------------------|---------------|------------|-------------------|
| `e2e/cli/test_issue_commands_e2e.py` | `test_l1_presentation_cli_interface.py` | End-to-end CLI command testing | **23** |

**Rationale**: Presentation layer depends on all other layers. Test last for comprehensive integration validation.

---

## 🚀 Test Execution Configuration

### Reverse Dependency Order Execution

```bash
# Execute in dependency order for optimal feedback
pytest tests/test_l7_foundation_*.py           # Foundation (fastest)
pytest tests/test_l5_infrastructure_*.py       # Infrastructure
pytest tests/test_l6_integration_*.py          # Integration
pytest tests/test_l3_domain_*.py               # Domain
pytest tests/test_l4_service_*.py              # Service
pytest tests/test_l2_application_*.py          # Application
pytest tests/test_l1_presentation_*.py         # Presentation (slowest)
```

### pytest.ini Configuration

Create optimized test execution configuration:

```ini
[tool:pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

# Test execution order optimization
collect_ignore = ["setup.py"]
addopts =
    --strict-markers
    --strict-config
    --verbose
    --tb=short
    --durations=10

# Custom test execution order
markers =
    foundation: Foundation layer tests (execute first)
    infrastructure: Infrastructure layer tests
    integration: Integration layer tests
    domain: Domain layer tests
    service: Service layer tests
    application: Application layer tests
    presentation: Presentation layer tests (execute last)
    slow: Slow tests requiring longer timeout
    performance: Performance validation tests
```

### Custom Test Runner Script

```python
#!/usr/bin/env python3
"""
Architectural test runner with dependency-optimized execution order.
"""

import subprocess
import sys
from pathlib import Path

def run_layer_tests(layer_name: str, pattern: str) -> bool:
    """Run tests for a specific architectural layer."""
    print(f"\n🧪 Testing {layer_name} Layer...")
    print("=" * 50)

    cmd = [
        "python", "-m", "pytest",
        f"tests/{pattern}",
        "-v", "--tb=short",
        f"--durations=5"
    ]

    result = subprocess.run(cmd)
    success = result.returncode == 0

    if success:
        print(f"✅ {layer_name} layer tests PASSED")
    else:
        print(f"❌ {layer_name} layer tests FAILED")

    return success

def main():
    """Execute tests in reverse dependency order."""
    layers = [
        ("Foundation", "test_l7_foundation_*.py"),
        ("Infrastructure", "test_l5_infrastructure_*.py"),
        ("Integration", "test_l6_integration_*.py"),
        ("Domain", "test_l3_domain_*.py"),
        ("Service", "test_l4_service_*.py"),
        ("Application", "test_l2_application_*.py"),
        ("Presentation", "test_l1_presentation_*.py")
    ]

    print("🏗️  MarkiTect Architectural Test Runner")
    print("Executing tests in reverse dependency order...")

    failed_layers = []

    for layer_name, pattern in layers:
        if not run_layer_tests(layer_name, pattern):
            failed_layers.append(layer_name)
            print(f"\n⚠️  Stopping at {layer_name} layer failure")
            break

    if failed_layers:
        print(f"\n❌ Test execution stopped at layer: {failed_layers[0]}")
        print("Fix foundation issues before testing dependent layers.")
        sys.exit(1)
    else:
        print("\n✅ All architectural layers passed!")
        print("🎉 System is architecturally sound!")

if __name__ == "__main__":
    main()
```

---

## 📊 Test Execution Benefits

### Performance Optimization

| **Strategy** | **Time Savings** | **Debugging Efficiency** |
|-------------|-------------------|---------------------------|
| **Foundation First** | 60-80% faster feedback | Fix root causes immediately |
| **Layer Isolation** | 40-60% reduced test time | Clear failure boundaries |
| **Dependency Order** | 70-90% fewer cascade failures | Targeted fixes |
| **Early Termination** | 80%+ time saved on foundation failures | Stop at first architectural break |

### Development Workflow Benefits

1. **Immediate Feedback**: Foundation failures caught in < 5 seconds
2. **Targeted Debugging**: Clear layer-specific failure isolation
3. **Reduced Context Switching**: Fix architectural layers systematically
4. **Confidence Building**: Green foundation = stable architecture
5. **Parallel Development**: Teams can work on different layers safely

---

## 🔄 Migration Plan

### Phase 1: Test File Renaming (1-2 days)

```bash
# Foundation Layer
mv test_parser.py test_l7_foundation_markdown_parsing.py
mv test_issue_1_database_initialization.py test_l7_foundation_database_core.py

# Infrastructure Layer
mv test_issue_2_file_ingestion.py test_l5_infrastructure_ast_processing.py
mv test_issue_13_cache_commands.py test_l5_infrastructure_cache_management.py
mv test_config_cli_commands.py test_l5_infrastructure_configuration.py

# Integration Layer
mv test_gitea_facade.py test_l6_integration_gitea_api.py
mv test_issue_creator.py test_l6_integration_issue_creation.py

# Domain Layer
mv unit/domain/issues/test_issue_models.py test_l3_domain_issue_models.py
mv unit/domain/issues/test_issue_services.py test_l3_domain_issue_services.py
mv unit/domain/projects/test_project_models.py test_l3_domain_project_models.py

# Service Layer
mv test_issue_4_retrieve_all_files.py test_l4_service_document_management.py
mv test_issue_15_ast_commands.py test_l4_service_ast_analysis.py

# Application Layer
mv test_issue_11_workspace_creation.py test_l2_application_workspace_workflows.py
mv test_issue_11_workflow_integration.py test_l2_application_tdd_workflows.py

# Presentation Layer
mv e2e/cli/test_issue_commands_e2e.py test_l1_presentation_cli_interface.py
```

### Phase 2: Test Configuration Setup (1 day)

1. Update `pytest.ini` with execution order markers
2. Create architectural test runner script
3. Update CI/CD pipeline configuration
4. Create test execution documentation

### Phase 3: Team Training (1 day)

1. Document new test architecture
2. Train team on execution strategy
3. Update development workflow docs
4. Create architectural test guidelines

---

## 🎯 Quality Gates

### Layer-Specific Quality Requirements

| **Layer** | **Coverage Target** | **Performance Target** | **Failure Rate Target** |
|-----------|-------------------|------------------------|-------------------------|
| **Foundation** | 98%+ | < 5s execution | < 0.1% failure rate |
| **Infrastructure** | 95%+ | < 15s execution | < 0.5% failure rate |
| **Integration** | 85%+ | < 30s execution | < 2% failure rate |
| **Domain** | 98%+ | < 10s execution | < 0.2% failure rate |
| **Service** | 90%+ | < 20s execution | < 1% failure rate |
| **Application** | 85%+ | < 45s execution | < 2% failure rate |
| **Presentation** | 80%+ | < 60s execution | < 5% failure rate |

### Architectural Health Metrics

1. **Foundation Stability**: > 99.5% pass rate
2. **Layer Isolation**: Zero upward dependency failures
3. **Execution Time**: < 3 minutes for full architectural test suite
4. **Debugging Efficiency**: Average fix time < 15 minutes per layer

---

## 📝 Architectural Test Patterns

### Foundation Layer Pattern
```python
# test_l7_foundation_markdown_parsing.py
class TestFoundationMarkdownParsing:
    """Foundation layer: Core markdown parsing engine validation."""

    def test_markdown_parser_converts_heading_and_paragraph_to_ast_tokens(self):
        """FOUNDATION: Validate core parsing capability."""
        # Fast, isolated, no dependencies
```

### Infrastructure Layer Pattern
```python
# test_l5_infrastructure_cache_management.py
class TestInfrastructureCacheManagement:
    """Infrastructure layer: Cache system technical capabilities."""

    def test_cache_info_command_works_with_empty_and_populated_cache(self):
        """INFRASTRUCTURE: Validate cache system operations."""
        # Technical capability, depends on foundation
```

### Domain Layer Pattern
```python
# test_l3_domain_issue_models.py
class TestDomainIssueModels:
    """Domain layer: Pure business logic validation."""

    def test_issue_creation_with_valid_data(self):
        """DOMAIN: Validate business rules and logic."""
        # No infrastructure dependencies, pure logic
```

---

## 🚦 Continuous Integration Integration

### GitHub Actions Workflow

```yaml
name: Architectural Test Suite

on: [push, pull_request]

jobs:
  architectural-tests:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.12'

    - name: Install dependencies
      run: |
        pip install -r requirements.txt
        pip install -r requirements-test.txt

    - name: Run Foundation Tests (Layer 7)
      run: pytest tests/test_l7_foundation_*.py -v

    - name: Run Infrastructure Tests (Layer 5)
      run: pytest tests/test_l5_infrastructure_*.py -v
      if: success()

    - name: Run Integration Tests (Layer 6)
      run: pytest tests/test_l6_integration_*.py -v
      if: success()

    - name: Run Domain Tests (Layer 3)
      run: pytest tests/test_l3_domain_*.py -v
      if: success()

    - name: Run Service Tests (Layer 4)
      run: pytest tests/test_l4_service_*.py -v
      if: success()

    - name: Run Application Tests (Layer 2)
      run: pytest tests/test_l2_application_*.py -v
      if: success()

    - name: Run Presentation Tests (Layer 1)
      run: pytest tests/test_l1_presentation_*.py -v
      if: success()
```

---

*This architectural test strategy transforms testing from a time-consuming bottleneck into a fast, targeted debugging tool that respects system architecture and optimizes developer productivity.*