markitect-main/docs/development/tdd-workflow.md

# TDD Workflow Guide

MarkiTect uses a sophisticated Test-Driven Development workflow based on the TDD8 methodology. This guide explains how to contribute to the project using our established patterns.

## TDD8 Methodology

MarkiTect implements the complete 8-phase TDD cycle:

1. **ISSUE** - Requirements clearly defined and understood
2. **TEST** - Comprehensive tests created covering all functionality
3. **RED** - Tests initially fail during development process
4. **GREEN** - Implementation completed, all commands working
5. **REFACTOR** - Code quality maintained throughout development
6. **DOCUMENT** - Complete docstrings with usage examples and security notes
7. **REFINE** - Quality checks passed, all tests passing, integration verified
8. **PUBLISH** - TDD8 workflow formally completed, documentation updated

## Workflow Commands

### Starting Work on an Issue

```bash
make tdd-start NUM=X
```

This creates a workspace for issue X with:
- Requirements analysis
- Test plan template
- Isolated test directory
- Workspace status tracking

### Adding Tests

```bash
make tdd-add-test
```

Provides guidance for generating comprehensive tests based on:
- Issue requirements
- Existing test patterns
- TDD best practices

### Checking Status

```bash
make tdd-status
```

Shows current workspace state:
- Active issue number
- Test files created
- Requirements completion
- Current TDD phase

### Finishing Work

```bash
make tdd-finish
```

Completes the TDD cycle by:
- Moving tests to main test directory
- Cleaning up workspace
- Validating completion criteria
- Preparing for integration

## Test Organization

### Test File Naming

```
tests/test_issue_N_description.py
```

Examples:
- `tests/test_issue_13_cache_commands.py`
- `tests/test_issue_14_database_queries.py`
- `tests/test_issue_15_ast_analysis.py`

### Test Structure

```python
"""
Tests for Issue #N: Feature Description.

TDD approach: These tests define exact requirements for the feature.
All tests should initially FAIL (RED) and drive implementation (GREEN).
"""

class TestFeatureName:
    """TDD test suite defining feature requirements."""

    def setup_method(self):
        """Set up test environment."""
        # Common test setup

    def test_feature_exists(self):
        """Feature command/function should exist and be callable."""
        # Test basic existence

    def test_feature_behavior(self):
        """Feature should exhibit specific behavior."""
        # Test specific requirements

    def teardown_method(self):
        """Clean up after tests."""
        # Resource cleanup
```

## Development Best Practices

### Test-First Development

1. **Read the issue requirements thoroughly**
2. **Write failing tests that define the exact behavior needed**
3. **Run tests to see them fail (RED)**
4. **Implement minimal code to make tests pass (GREEN)**
5. **Refactor for quality while keeping tests green**
6. **Document the implementation**
7. **Refine based on integration testing**
8. **Complete the TDD cycle**

### Following Conventions

When implementing features:

1. **Study existing code patterns** in similar components
2. **Follow established naming conventions**
3. **Use existing libraries and utilities** where possible
4. **Maintain consistency** with project architecture
5. **Focus on behavior, not implementation details** in tests

### Example: Cache Management (Issue #13)

The cache management implementation demonstrates proper TDD workflow:

#### Phase 1: ISSUE & TEST
- Created comprehensive test suite defining exact CLI command requirements
- Tests focused on behavior (what commands do) not implementation (where cache is stored)

#### Phase 2: RED & GREEN
- Tests initially failed (no commands existed)
- Implemented minimal CLI commands to make tests pass
- Followed "convention over configuration" for cache directory location

#### Phase 3: REFACTOR & DOCUMENT
- Created `CacheDirectoryService` to separate concerns
- Added comprehensive docstrings and help text
- Organized code following established patterns

#### Phase 4: REFINE & PUBLISH
- Integrated with main CLI framework
- Validated against acceptance criteria
- Moved tests to main test directory

## Common Patterns

### CLI Commands

All CLI commands should follow this pattern:

```python
@cli.command('command-name')
@click.argument('required_arg', type=str)
@click.option('--optional', help='Description')
@pass_config
def command_name(config, required_arg, optional):
    """
    Brief command description.

    Longer description with examples and usage patterns.
    """
    try:
        # Service layer interaction
        service = SomeService()
        result = service.perform_operation(required_arg, optional)

        # User feedback
        click.echo(result['message'])

        # Error handling
        if not result['success']:
            sys.exit(1)

    except Exception as e:
        click.echo(f"Error: {e}", err=True)
        if config and config.get('verbose'):
            import traceback
            click.echo(traceback.format_exc(), err=True)
        sys.exit(1)
```

### Service Layer

Business logic should be implemented in service classes:

```python
class SomeService:
    """Service for handling business logic."""

    def perform_operation(self, input_data) -> dict:
        """
        Perform operation and return structured result.

        Returns:
            Dictionary with 'success', 'message', and result data
        """
        try:
            # Business logic here
            result = self._do_work(input_data)

            return {
                'success': True,
                'message': 'Operation completed successfully',
                'data': result
            }
        except Exception as e:
            return {
                'success': False,
                'message': f'Operation failed: {e}',
                'error': str(e)
            }
```

### Testing Service Layer

```python
def test_service_operation():
    """Service should perform operation correctly."""
    service = SomeService()
    result = service.perform_operation("test_input")

    assert result['success'] is True
    assert 'Operation completed' in result['message']
    assert 'data' in result
```

## Quality Standards

### Test Coverage

Each issue should include comprehensive tests covering:
- **Happy path**: Normal usage scenarios
- **Edge cases**: Boundary conditions and unusual inputs
- **Error handling**: Invalid inputs and failure modes
- **Integration**: Component interaction with existing system

### Code Quality

All code should maintain:
- **Clear naming**: Functions and variables describe their purpose
- **Proper documentation**: Docstrings explain what, why, and how
- **Error handling**: Graceful failure with helpful messages
- **Consistent style**: Following project conventions

### Performance Considerations

When implementing features:
- **Consider caching implications** for document processing
- **Use existing optimizations** like AST cache and database integration
- **Profile performance** for operations on large document sets
- **Document performance characteristics** in code comments

## Integration with Project Workflow

### Milestone Tracking

Issues are organized into strategic milestones:
- **Schema-Driven Architecture** - Core schema and validation features
- **Template & Stub Generation** - Document creation tools
- **Document Relationships** - Cross-reference and hierarchy management
- **Plan-Actual Comparison Engine** - AI-supported analysis tools

### Priority Management

Issues are prioritized as:
- **CRITICAL (P0)** - Foundation features required for other work
- **HIGH (P1)** - Core functionality for primary use cases
- **MEDIUM (P2)** - Important enhancements and supporting features
- **LOW (P3)** - Advanced features and optimizations

### Release Process

Completed issues are integrated through:
1. **TDD completion** using `make tdd-finish`
2. **Integration testing** with full test suite
3. **Documentation updates** including user guides
4. **Milestone progress** tracked in project management
5. **Release preparation** for version deployment

---

This TDD workflow ensures consistent code quality, comprehensive test coverage, and maintainable architecture throughout the project.