markitect-main/ARCHITECTURE.md

# MarkiTect Layered Architecture Blueprint

> **Strategic architectural guidance for evolving the MarkiTect codebase**

This document provides a comprehensive layered architecture blueprint based on the analysis of current capabilities and dependencies in the MarkiTect system. It serves as a roadmap for organizing and improving the codebase going forward.

## Executive Summary

MarkiTect demonstrates a sophisticated multi-domain system that has evolved organically. This blueprint formalizes the architecture into **7 distinct layers** with clear separation of concerns, dependency management, and scalability principles.

### Key Architectural Insights
- **Current State**: Well-separated domains with some cross-cutting concerns
- **Primary Pattern**: Hexagonal Architecture with Clean Architecture principles
- **Growth Areas**: Plugin system, event-driven components, performance optimization
- **Refactoring Priority**: Consolidate cross-cutting concerns, standardize interfaces

---

## 🏗️ Layered Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                    PRESENTATION LAYER                           │
│                     (User Interface)                            │
├─────────────────────────────────────────────────────────────────┤
│                    APPLICATION LAYER                            │
│                  (Use Cases & Workflows)                        │
├─────────────────────────────────────────────────────────────────┤
│                     DOMAIN LAYER                                │
│                   (Business Logic)                              │
├─────────────────────────────────────────────────────────────────┤
│                     SERVICE LAYER                               │
│                  (Application Services)                         │
├─────────────────────────────────────────────────────────────────┤
│                   INFRASTRUCTURE LAYER                          │
│                  (Technical Capabilities)                       │
├─────────────────────────────────────────────────────────────────┤
│                   INTEGRATION LAYER                             │
│                   (External Systems)                            │
├─────────────────────────────────────────────────────────────────┤
│                     FOUNDATION LAYER                            │
│                  (Core Technologies)                            │
└─────────────────────────────────────────────────────────────────┘
```

---

## 📋 Layer-by-Layer Breakdown

### 🎯 Layer 1: Presentation Layer
**Purpose**: User interface and interaction handling

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **CLI Framework** | `cli/core.py` | Command delegation, argument parsing | Application Layer |
| **CLI Commands** | `cli/commands/` | Command implementations | Application Layer |
| **Presenters** | `cli/presenters/` | Output formatting, view logic | None (top layer) |
| **Formatters** | `cli/presenters/formatters.py` | Table/JSON/YAML output | None |

**Architectural Principles:**
- ✅ **Single Responsibility**: Each presenter handles one output concern
- ✅ **No Business Logic**: Pure presentation and formatting
- 🔄 **Improvement Needed**: Standardize presenter interfaces

---

### 🚀 Layer 2: Application Layer
**Purpose**: Use cases, workflows, and orchestration

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **Workspace Workflows** | `tddai/workspace.py` | TDD workflow orchestration | Domain + Service Layers |
| **Issue Workflows** | `tddai/issue_creator.py` | Issue creation workflows | Domain + Integration Layers |
| **Query Workflows** | `markitect/cli.py` | Database query orchestration | Service + Infrastructure |
| **Cache Workflows** | `markitect/cache_service.py` | Caching orchestration | Infrastructure Layer |

**Architectural Principles:**
- ✅ **Orchestration Focus**: Coordinates between layers without business logic
- 🔄 **Improvement Needed**: Extract workflows from CLI commands
- 🔄 **Standardization**: Consistent error handling patterns

---

### 🏛️ Layer 3: Domain Layer
**Purpose**: Core business logic and domain models

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **Issue Models** | `domain/issues/models.py` | Issue lifecycle, label categorization | None (pure domain) |
| **Project Models** | `domain/projects/models.py` | Project tracking, progress calculation | Issue Models |
| **Issue Services** | `domain/issues/services.py` | Business rules, status determination | Issue Models |
| **Project Services** | `domain/projects/services.py` | Project management logic | Project Models |

**Architectural Principles:**
- ✅ **Pure Business Logic**: No infrastructure concerns
- ✅ **Rich Domain Models**: Behavior embedded in entities
- ✅ **Domain Services**: Complex business rules
- 🔄 **Enhancement**: Add domain events for decoupling

---

### ⚙️ Layer 4: Service Layer
**Purpose**: Application services and cross-cutting concerns

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **Document Service** | `markitect/document_manager.py` | Document lifecycle management | Infrastructure Layer |
| **AST Service** | `markitect/ast_service.py` | AST processing coordination | Infrastructure Layer |
| **Export Service** | `services/export_service.py` | Data export coordination | Infrastructure + Domain |
| **Issue Service** | `services/issue_service.py` | Issue management coordination | Domain + Integration |
| **Workspace Service** | `services/workspace_service.py` | Workspace management | Infrastructure Layer |

**Architectural Principles:**
- ✅ **Coordination**: Orchestrates infrastructure and domain
- 🔄 **Improvement Needed**: Extract from mixed locations
- 🔄 **Standardization**: Consistent service interfaces

---

### 🔧 Layer 5: Infrastructure Layer
**Purpose**: Technical capabilities and system resources

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **Database Management** | `markitect/database.py` | SQL database operations | Foundation Layer |
| **AST Processing** | `markitect/parser.py` | Markdown parsing and AST generation | Foundation Layer |
| **Cache Management** | `markitect/ast_cache.py` | File-based caching system | Foundation Layer |
| **Configuration** | `config/` | System configuration management | Foundation Layer |
| **Logging** | `infrastructure/logging/` | Structured logging system | Foundation Layer |
| **Repositories** | `infrastructure/repositories/` | Data access abstractions | Foundation Layer |

**Architectural Principles:**
- ✅ **Technical Focus**: No business logic
- ✅ **Abstraction**: Clean interfaces for technical concerns
- 🔄 **Consolidation**: Merge scattered infrastructure code

---

### 🌐 Layer 6: Integration Layer
**Purpose**: External system integration and APIs

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **Gitea API Client** | `gitea/api_client.py` | HTTP API communication | Foundation Layer |
| **Gitea Models** | `gitea/models.py` | External system data models | Foundation Layer |
| **Git Integration** | `gitea/client.py` | Git platform abstraction | Foundation Layer |
| **HTTP Client** | `gitea/http_client.py` | HTTP communication layer | Foundation Layer |

**Architectural Principles:**
- ✅ **External Focus**: Only external system concerns
- ✅ **Adapter Pattern**: Translate external models to domain
- 🔄 **Enhancement**: Plugin architecture for multiple platforms

---

### 🏢 Layer 7: Foundation Layer
**Purpose**: Core technologies and utilities

| Component | Current Location | Capabilities | Dependencies |
|-----------|------------------|--------------|--------------|
| **SQLite Database** | System dependency | Data persistence | None |
| **Python Runtime** | System dependency | Execution environment | None |
| **File System** | System dependency | File operations | None |
| **Network Stack** | System dependency | HTTP/API communication | None |
| **Markdown-it** | External library | Markdown parsing engine | None |
| **Core Utilities** | Various locations | Common functionality | None |

**Architectural Principles:**
- ✅ **Stable Foundation**: Minimal change frequency
- ✅ **Technology Choices**: Well-established libraries
- 🔄 **Standardization**: Consistent utility patterns

---

## 🔄 Capability Dependencies Map

### Core Dependency Flow
```
CLI Commands → Application Workflows → Domain Services → Infrastructure → Foundation
     ↓               ↓                      ↓               ↓
Presenters    Integration Layer      Repository Layer   System APIs
```

### Cross-Cutting Concerns
```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Logging       │    │  Configuration  │    │     Caching     │
│   (All Layers)  │    │   (All Layers)  │    │ (Service+Infra) │
└─────────────────┘    └─────────────────┘    └─────────────────┘
```

### Capability Interdependencies

| **Foundation Capabilities** | **Dependencies** |
|------------------------------|------------------|
| Database Storage | → Markdown Processing, Configuration |
| AST Processing | → Performance Caching, Query Interface |
| Configuration | → All system components |

| **Infrastructure Capabilities** | **Dependencies** |
|----------------------------------|------------------|
| Cache Management | → Database Storage, AST Processing |
| Repository Abstraction | → Database Storage, External APIs |
| Logging System | → Configuration Management |

| **Service Capabilities** | **Dependencies** |
|---------------------------|------------------|
| Document Management | → AST Processing, Cache Management |
| Issue Management | → Git Integration, Database Storage |
| Workspace Management | → Configuration, File System |

| **Application Capabilities** | **Dependencies** |
|-------------------------------|------------------|
| TDD Workflows | → Workspace Management, Issue Management |
| Query Workflows | → Database Storage, Output Formatting |
| CLI Operations | → All Service Layer capabilities |

---

## 🎯 Architectural Principles

### 1. **Dependency Direction**
- **Rule**: Dependencies flow downward only
- **Implementation**: Higher layers depend on lower layers, never reverse
- **Benefit**: Prevents circular dependencies, enables testing

### 2. **Interface Segregation**
- **Rule**: Small, focused interfaces
- **Implementation**: Repository interfaces, service contracts
- **Benefit**: Easier testing, cleaner implementations

### 3. **Single Responsibility**
- **Rule**: Each component has one reason to change
- **Implementation**: Separate concerns by layer and domain
- **Benefit**: Maintainable, testable code

### 4. **Open/Closed Principle**
- **Rule**: Open for extension, closed for modification
- **Implementation**: Plugin architecture, strategy patterns
- **Benefit**: New features without breaking existing code

### 5. **Configuration Over Convention**
- **Rule**: Explicit configuration rather than implicit behavior
- **Implementation**: Comprehensive configuration system
- **Benefit**: Flexible deployment, clear behavior

---

## 📈 Migration Strategy

### Phase 1: Foundation Consolidation
**Priority**: High | **Timeline**: 2-3 sprints

1. **Consolidate Utilities**
   - Extract common functionality to `foundation/` package
   - Standardize error handling patterns
   - Create consistent logging interfaces

2. **Standardize Infrastructure**
   - Consolidate repository implementations
   - Standardize configuration interfaces
   - Unify caching mechanisms

### Phase 2: Service Layer Extraction
**Priority**: High | **Timeline**: 3-4 sprints

1. **Extract Application Services**
   - Move orchestration logic from CLI to service layer
   - Create consistent service interfaces
   - Implement dependency injection

2. **Standardize Domain Services**
   - Ensure pure business logic in domain layer
   - Add domain events for decoupling
   - Create domain service contracts

### Phase 3: Presentation Enhancement
**Priority**: Medium | **Timeline**: 2-3 sprints

1. **Standardize Presenters**
   - Create presenter interface contracts
   - Implement consistent error handling
   - Add format validation

2. **Enhance CLI Framework**
   - Improve command delegation
   - Add middleware support
   - Implement plugin hooks

### Phase 4: Integration Expansion
**Priority**: Medium | **Timeline**: 3-4 sprints

1. **Plugin Architecture**
   - Design plugin interface
   - Implement plugin discovery
   - Add GitHub/GitLab adapters

2. **Event System**
   - Implement domain events
   - Add event handlers
   - Create async processing

---

## 🚧 Implementation Guidelines

### Directory Structure Recommendation
```
markitect/
├── foundation/           # Layer 7: Core utilities and constants
│   ├── exceptions.py
│   ├── types.py
│   └── utilities.py
├── infrastructure/       # Layer 5: Technical capabilities
│   ├── database/
│   ├── caching/
│   ├── configuration/
│   └── logging/
├── integration/          # Layer 6: External systems
│   ├── gitea/
│   ├── github/          # Future
│   └── plugins/         # Future
├── domain/              # Layer 3: Business logic
│   ├── issues/
│   ├── projects/
│   └── documents/       # Future
├── services/            # Layer 4: Application services
│   ├── document_service.py
│   ├── issue_service.py
│   └── workspace_service.py
├── application/         # Layer 2: Use cases and workflows
│   ├── workflows/
│   └── use_cases/
└── presentation/        # Layer 1: User interfaces
    ├── cli/
    ├── api/             # Future
    └── web/             # Future
```

### Coding Standards

#### Interface Design
```python
# Good: Clear interface with single responsibility
class DocumentRepository(Protocol):
    def save(self, document: Document) -> DocumentId: ...
    def find_by_id(self, id: DocumentId) -> Optional[Document]: ...
    def find_all(self) -> List[Document]: ...

# Avoid: Large interfaces mixing concerns
class MegaRepository(Protocol):
    def save_document(self, doc): ...
    def save_issue(self, issue): ...
    def send_email(self, msg): ...  # Wrong layer!
```

#### Dependency Injection
```python
# Good: Constructor injection with interfaces
class DocumentService:
    def __init__(
        self,
        repository: DocumentRepository,
        cache: CacheService,
        logger: Logger
    ):
        self._repository = repository
        self._cache = cache
        self._logger = logger

# Avoid: Direct instantiation
class DocumentService:
    def __init__(self):
        self._repository = SqliteDocumentRepository()  # Tight coupling
```

#### Error Handling
```python
# Good: Domain-specific exceptions
class DocumentNotFoundError(DomainException):
    def __init__(self, document_id: DocumentId):
        super().__init__(f"Document {document_id} not found")

# Avoid: Generic exceptions
raise Exception("Something went wrong")  # Too generic
```

---

## 📊 Quality Metrics

### Architectural Health Indicators

| **Metric** | **Current State** | **Target State** | **Priority** |
|------------|-------------------|------------------|--------------|
| **Cyclomatic Complexity** | Mixed (2-15) | < 10 per method | High |
| **Dependency Depth** | 3-5 levels | < 4 levels | Medium |
| **Interface Coupling** | Tight in some areas | Loose coupling | High |
| **Test Coverage** | 95%+ | Maintain 95%+ | Medium |
| **Module Cohesion** | Good in domain | High cohesion | Medium |

### Code Quality Targets

1. **Maintainability Index**: > 80
2. **Lines of Code per Method**: < 20
3. **Parameters per Method**: < 5
4. **Nested Depth**: < 3 levels
5. **Documentation Coverage**: > 90%

---

## 🔮 Future Evolution Path

### Short Term (1-2 releases)
- ✅ **Foundation Layer**: Consolidate utilities and infrastructure
- ✅ **Service Layer**: Extract and standardize application services
- ✅ **Interface Standardization**: Create consistent contracts

### Medium Term (3-5 releases)
- 🚀 **Plugin Architecture**: Support multiple Git platforms
- 🚀 **Event System**: Implement domain events and handlers
- 🚀 **API Layer**: Add REST API for external integration

### Long Term (6+ releases)
- 🌟 **Microservices**: Split into focused services if needed
- 🌟 **Event Sourcing**: Consider for audit and replay capabilities
- 🌟 **Multi-tenant**: Support multiple organizations/teams

---

## 📝 Decision Records

### ADR-001: Layered Architecture Adoption
**Status**: Proposed | **Date**: 2025-09-29

**Context**: MarkiTect has grown organically with mixed architectural patterns.

**Decision**: Adopt 7-layer architecture with clear separation of concerns.

**Consequences**:
- ✅ Improved maintainability and testability
- ✅ Clear dependency management
- ⚠️ Requires refactoring effort
- ⚠️ Learning curve for new patterns

### ADR-002: Domain-Driven Design Principles
**Status**: Proposed | **Date**: 2025-09-29

**Context**: Business logic is scattered across layers.

**Decision**: Implement DDD with rich domain models and pure business logic.

**Consequences**:
- ✅ Business logic centralized and testable
- ✅ Domain experts can understand code
- ⚠️ Requires domain modeling effort

### ADR-003: Plugin Architecture for External Systems
**Status**: Proposed | **Date**: 2025-09-29

**Context**: Current tight coupling to Gitea limits platform support.

**Decision**: Implement plugin architecture for Git platform integration.

**Consequences**:
- ✅ Support multiple platforms (GitHub, GitLab, etc.)
- ✅ Community can add new integrations
- ⚠️ Additional complexity in interface design

---

## 🎯 Success Criteria

### Technical Metrics
- [ ] **Dependency Violations**: Zero upward dependencies
- [ ] **Test Coverage**: Maintain > 95% coverage
- [ ] **Build Time**: < 30 seconds for full test suite
- [ ] **Documentation**: All public APIs documented

### Business Metrics
- [ ] **Feature Velocity**: Reduce time-to-market for new features
- [ ] **Bug Rate**: < 1 bug per 1000 lines of code
- [ ] **Developer Onboarding**: New developers productive in < 1 week
- [ ] **Platform Support**: Support 3+ Git platforms

---

*This architecture blueprint is a living document that should evolve with the system. Regular reviews and updates ensure it remains relevant and valuable for development decisions.*