markitect-main/docs/sub_agents/requirements_engineering_agent.md

Requirements Engineering and Incremental Development Planning Agent

Overview

A specialized sub-agent designed to prevent interface compatibility issues and mock object mismatches by ensuring solid foundation planning before implementation. This agent addresses the core problems encountered during Issue #59 development where tests assumed interfaces that didn't match the actual domain models.

Agent Responsibilities

1. Bottom-Up Structure Planning

• Domain Model Discovery: Analyze existing domain models before writing any tests
• Interface Inventory: Map all existing interfaces, abstract classes, and concrete implementations
• Dependency Mapping: Understand the complete dependency graph before adding new components
• Foundation Assessment: Ensure solid architectural foundations before building new features

2. Interface Contract Definition

• Contract Verification: Verify that all interfaces match actual implementations
• Mock Alignment: Ensure mock objects exactly match real domain model attributes and methods
• API Compatibility: Check that new interfaces are compatible with existing infrastructure
• Type Safety: Ensure all type hints and signatures are consistent across layers

3. Incremental Validation Strategy

• Validation Checkpoints: Define specific validation points throughout development
• Integration Testing: Plan integration tests before unit tests
• Compatibility Testing: Verify backward compatibility at each increment
• Interface Evolution: Plan how interfaces will evolve without breaking existing code

4. Test-Driven Architecture

• Domain-First Testing: Ensure tests reflect actual domain model requirements
• Infrastructure Awareness: Write tests that understand existing infrastructure patterns
• Mock Strategy: Create mocks that exactly match real object interfaces
• Test Architecture: Design test architecture that matches application architecture

Core Methodologies

1. Domain Model First (DMF) Approach

Before writing any tests or implementation:

# 1. Analyze existing domain models
grep -r "class.*:" domain/*/models.py
grep -r "def " domain/*/models.py

# 2. Map existing interfaces
find . -name "*.py" -exec grep -l "class.*ABC\|@abstractmethod" {} \;

# 3. Understand data flow
grep -r "Repository\|Service" infrastructure/ domain/

Workflow:

1. Domain Discovery: Map all existing domain models and their attributes
2. Interface Analysis: Understand all abstract base classes and interfaces
3. Dependency Review: Trace dependencies between layers
4. Contract Documentation: Document all interface contracts before modification

2. Interface-Contract-First (ICF) Testing

# WRONG - Assumption-based mocking
mock_issue = Mock()
mock_issue.number = 59
mock_issue.title = "Test"
mock_issue.state = "open"  # String instead of enum!

# RIGHT - Contract-verified mocking
from domain.issues.models import Issue, IssueState, Label
mock_issue = Mock(spec=Issue)
mock_issue.number = 59
mock_issue.title = "Test Issue"
mock_issue.state = IssueState.OPEN  # Proper enum
mock_issue.labels = []
mock_issue.created_at = datetime.now(timezone.utc)
mock_issue.updated_at = datetime.now(timezone.utc)

Workflow:

1. Spec-Based Mocking: Always use spec= parameter with actual classes
2. Attribute Verification: Verify all mock attributes match real object attributes
3. Type Consistency: Ensure mock data types match domain model types
4. Enum Handling: Use actual enums instead of string representations

3. Incremental Architecture Validation (IAV)

Validation Checkpoints:

• Checkpoint 1: Domain model compatibility
• Checkpoint 2: Interface contract verification
• Checkpoint 3: Mock object alignment
• Checkpoint 4: Integration test validation
• Checkpoint 5: End-to-end workflow testing

Implementation:

# Validation script template
validate_domain_compatibility() {
    python -c "
    from domain.issues.models import Issue
    from markitect.issues.base import IssueBackend
    # Verify interface compatibility
    "
}

validate_mock_alignment() {
    # Run tests that verify mocks match real objects
    python -m pytest tests/test_mock_compatibility.py
}

4. Foundation-First Development (FFD)

Principle: Build on solid foundations before adding new layers.

Workflow:

1. Foundation Assessment: Verify existing infrastructure is solid
2. Interface Stability: Ensure base interfaces won't change during development
3. Dependency Injection: Plan dependency injection patterns
4. Layer Separation: Maintain clear separation between architectural layers

Tools and Frameworks

1. Domain Analysis Tools

# Domain Model Inspector
analyze_domain_models() {
    echo "=== Domain Model Analysis ==="
    find domain/ -name "models.py" -exec echo "File: {}" \; -exec grep -n "class\|def " {} \;
}

# Interface Contract Checker
check_interface_contracts() {
    echo "=== Interface Contract Analysis ==="
    grep -r "@abstractmethod\|ABC" . --include="*.py"
}

# Mock Compatibility Validator
validate_mocks() {
    echo "=== Mock Compatibility Check ==="
    python -c "
    import inspect
    from domain.issues.models import Issue
    print('Issue attributes:', [attr for attr in dir(Issue) if not attr.startswith('_')])
    "
}

2. Test Architecture Framework

# Test Base Classes for Interface Compliance
class DomainModelTestBase:
    """Base class ensuring tests match domain models."""

    def setUp(self):
        self.validate_test_setup()

    def validate_test_setup(self):
        """Verify test setup matches actual domain models."""
        pass

    def create_mock_with_spec(self, domain_class):
        """Create spec-compliant mock."""
        return Mock(spec=domain_class)

class IntegrationTestBase:
    """Base class for integration tests."""

    def setUp(self):
        self.verify_infrastructure_availability()

    def verify_infrastructure_availability(self):
        """Ensure required infrastructure is available."""
        pass

3. Interface Evolution Manager

class InterfaceEvolutionManager:
    """Manages interface changes without breaking compatibility."""

    def plan_interface_change(self, interface_name, changes):
        """Plan interface changes with backward compatibility."""
        pass

    def validate_compatibility(self, old_interface, new_interface):
        """Validate that new interface is backward compatible."""
        pass

    def generate_migration_plan(self, changes):
        """Generate step-by-step migration plan."""
        pass

4. Mock Validation Framework

class MockValidator:
    """Validates that mocks match real objects."""

    @staticmethod
    def validate_mock_spec(mock_obj, real_class):
        """Validate mock object matches real class specification."""
        mock_attrs = set(dir(mock_obj))
        real_attrs = set(dir(real_class))

        missing_attrs = real_attrs - mock_attrs
        extra_attrs = mock_attrs - real_attrs

        if missing_attrs:
            raise MockSpecError(f"Mock missing attributes: {missing_attrs}")

        return True

    @staticmethod
    def validate_mock_types(mock_obj, real_instance):
        """Validate mock attribute types match real object types."""
        for attr_name in dir(real_instance):
            if not attr_name.startswith('_'):
                real_value = getattr(real_instance, attr_name)
                mock_value = getattr(mock_obj, attr_name, None)

                if mock_value is not None and type(mock_value) != type(real_value):
                    raise MockTypeError(f"Type mismatch for {attr_name}")

Example Workflows

1. Adding New CLI Command Workflow

Phase 1: Foundation Analysis

# 1. Analyze existing CLI structure
find cli/ -name "*.py" -exec grep -l "click\|@cli" {} \;

# 2. Understand existing domain models
python -c "
from domain.issues.models import Issue
import inspect
print(inspect.signature(Issue.__init__))
"

# 3. Map existing repository interfaces
grep -r "class.*Repository" infrastructure/

Phase 2: Interface Contract Definition

# Define interface contract first
class IssueBackend(ABC):
    @abstractmethod
    def list_issues(self, state: Optional[str] = None) -> List[Issue]:
        """List issues with optional state filter."""
        pass

    @abstractmethod
    def get_issue(self, issue_id: str) -> Issue:
        """Get specific issue by ID."""
        pass

Phase 3: Test Architecture Design

# Design tests that match actual interfaces
class TestIssuesCLIGroup:
    def setup_method(self):
        # Use actual domain model for mock spec
        self.mock_issue = Mock(spec=Issue)
        self.mock_issue.number = 59
        self.mock_issue.title = "Test Issue"
        self.mock_issue.state = IssueState.OPEN  # Use actual enum
        self.mock_issue.labels = []
        self.mock_issue.created_at = datetime.now(timezone.utc)
        self.mock_issue.updated_at = datetime.now(timezone.utc)

Phase 4: Incremental Implementation

• Implement abstract base class
• Create plugin system
• Add CLI commands
• Integrate with existing infrastructure

2. Domain Model Extension Workflow

Phase 1: Impact Analysis

# Find all usages of the domain model
grep -r "Issue" . --include="*.py" | grep -v __pycache__

# Check existing tests
grep -r "Issue" tests/ --include="*.py"

# Analyze database schemas
grep -r "Issue" infrastructure/repositories/

Phase 2: Backward Compatibility Planning

# Plan extension that maintains compatibility
@dataclass
class Issue:
    # Existing attributes (DO NOT CHANGE)
    number: int
    title: str
    state: IssueState
    labels: List[Label]
    created_at: datetime
    updated_at: datetime

    # New attributes (with defaults for compatibility)
    body: str = ""  # Add with default
    assignees: List[str] = field(default_factory=list)
    html_url: str = ""

Phase 3: Migration Strategy

# Create migration tests
class TestIssueModelMigration:
    def test_old_constructor_still_works(self):
        """Ensure old constructor calls still work."""
        issue = Issue(
            number=1,
            title="Test",
            state=IssueState.OPEN,
            labels=[],
            created_at=datetime.now(timezone.utc),
            updated_at=datetime.now(timezone.utc)
        )
        assert issue.body == ""  # Default value

3. Plugin System Development Workflow

Phase 1: Architecture Planning

# Define plugin interface based on existing patterns
class IssueBackend(ABC):
    """Abstract base class matching existing repository patterns."""

    def __init__(self, config: Dict[str, Any]):
        self.config = config

    @abstractmethod
    def list_issues(self, state: Optional[str] = None) -> List[Issue]:
        pass

Phase 2: Integration Strategy

# Plan integration with existing infrastructure
class GiteaPlugin(IssueBackend):
    def __init__(self, config: Dict[str, Any]):
        super().__init__(config)
        # Reuse existing infrastructure
        self.repository = GiteaIssueRepository(
            connection_manager=self._create_connection_manager()
        )

    def list_issues(self, state: Optional[str] = None) -> List[Issue]:
        # Use existing async infrastructure
        return asyncio.run(self.repository.get_issues(state=state))

Integration Points with Existing Development Workflow

1. TDD8 Enhancement

Enhanced TDD8 Workflow:

1. ANALYZE - Analyze existing domain models and interfaces
2. ISSUE - Understand requirements in context of existing architecture
3. TEST - Write tests that match actual interfaces
4. RED - Verify tests fail for right reasons
5. GREEN - Implement with interface compatibility
6. REFACTOR - Maintain interface contracts
7. DOCUMENT - Update interface documentation
8. PUBLISH - Commit with interface change documentation

2. TodoWrite Integration

# Enhanced TodoWrite with validation checkpoints
todos = [
    {
        "content": "Analyze existing Issue domain model",
        "status": "pending",
        "activeForm": "Analyzing existing Issue domain model",
        "checkpoint": "domain_analysis"
    },
    {
        "content": "Define IssueBackend interface contract",
        "status": "pending",
        "activeForm": "Defining IssueBackend interface contract",
        "checkpoint": "interface_definition"
    },
    {
        "content": "Create spec-compliant mocks",
        "status": "pending",
        "activeForm": "Creating spec-compliant mocks",
        "checkpoint": "mock_validation"
    }
]

3. CLI Help Integration

# Enhanced CLI with validation commands
markitect validate-interfaces    # Check interface compatibility
markitect analyze-domain        # Analyze domain models
markitect check-mocks          # Validate mock objects
markitect plan-migration       # Plan interface migrations

Success Metrics

1. Interface Compatibility

• Zero Mock Mismatches: All mocks must match actual object interfaces
• Type Safety: 100% type consistency between tests and implementation
• Backward Compatibility: No breaking changes to existing interfaces

2. Test Quality

• Domain Model Alignment: Tests reflect actual domain model structure
• Integration Coverage: All integration points tested with real interfaces
• Mock Validation: All mocks validated against real object specifications

3. Development Efficiency

• Reduced Debugging: Fewer interface-related bugs
• Faster Development: Less time spent fixing mock mismatches
• Better Architecture: Cleaner interface design and evolution

Prevention of Issue #59 Problems

This agent would have prevented the Issue #59 problems by:

1. Domain Model Analysis: Would have discovered the actual Issue model has IssueState enum, not string
2. Interface Inventory: Would have mapped existing GiteaIssueRepository before designing plugin interface
3. Mock Validation: Would have caught mock attribute mismatches before running tests
4. Integration Planning: Would have planned how new CLI integrates with existing infrastructure
5. Contract Verification: Would have ensured all interfaces match actual implementations

The result would be a solid, well-planned implementation that builds on existing foundations rather than making incorrect assumptions about interfaces and domain models.