markitect-main/docs/architecture/CAPABILITIES_ARCHITECTURE.md

# Capabilities Architecture

## Overview

MarkiTect uses a **capabilities-based architecture** where functionality is organized into independent, reusable packages called **capabilities**. Each capability is a self-contained git submodule with its own repository, enabling independent development, versioning, and reuse across projects.

## Core Principles

### 1. **Separation of Concerns**

**Critical Rule:** The main repository (`markitect-main`) **MUST NOT** directly modify capability code.

- ✅ **DO**: Use capabilities as dependencies
- ✅ **DO**: Configure capabilities through documented interfaces
- ✅ **DO**: Report issues and feature requests to capability repos
- ❌ **DON'T**: Edit capability code from the main repo
- ❌ **DON'T**: Make commits directly in capability subdirectories
- ❌ **DON'T**: Bypass the submodule boundary

**Why?** Capabilities are independent repositories. Direct modifications create:
- Merge conflicts when syncing with upstream
- Broken dependency management
- Loss of independent versioning
- Confusion about source of truth

### 2. **Git Submodule Architecture**

Capabilities are integrated as **git submodules**, not regular directories:

```
markitect-main/
├── .gitmodules                    # Submodule configuration
├── capabilities/
│   ├── testdrive-jsui/           # Git submodule → separate repo
│   ├── issue-facade/             # Git submodule → separate repo
│   ├── kaizen-agentic/           # Git submodule → separate repo
│   ├── markitect-content/        # Local capability (legacy)
│   └── release-management/       # Local capability (legacy)
```

**Submodule vs Local Capabilities:**
- **Submodules**: Independent git repositories, separate development lifecycle
- **Local**: Part of main repo, shared lifecycle (being phased out)

**Target State:** All capabilities should eventually be submodules.

### 3. **Independent Development Lifecycle**

Each capability has its own:
- ✅ Git repository on gitea
- ✅ Version numbering (semantic versioning)
- ✅ Issue tracking and roadmap
- ✅ Testing and CI/CD pipeline
- ✅ Documentation and README
- ✅ Contributors and maintainers

### 4. **Interface-Based Integration**

Capabilities expose **stable interfaces** to the main project:

```python
# markitect uses capability through documented interface
from testdrive_jsui import TestDriveJSUIEngine

engine = TestDriveJSUIEngine()
engine.render_document(content, mode='edit', config=config)
```

**Integration Points:**
1. **Python Package Interface**: Clean import and API
2. **Configuration**: Via `pyproject.toml` dependencies
3. **Plugin System**: Via plugin registration and metadata
4. **Makefile Targets**: Via capability discovery system

## Development Workflow

### Working on Capabilities

**Rule:** Each capability is developed in **its own Claude Code session**.

#### Main Repository Session
```bash
# In markitect-main/
cd /home/worsch/markitect-main

# Main repo tasks:
# - Integrate capabilities
# - Update dependencies
# - Configure capability usage
# - Test integration points
```

#### Capability Session
```bash
# In capability repository
cd /home/worsch/markitect-main/capabilities/testdrive-jsui

# OR clone separately
git clone http://gitea/coulomb/testdrive-jsui.git
cd testdrive-jsui

# Capability tasks:
# - Implement features
# - Fix bugs
# - Write tests
# - Update documentation
# - Make releases
```

### Recommended Session Pattern

**Scenario: Need to add feature to testdrive-jsui**

1. **Open separate Claude instance** for testdrive-jsui
   ```bash
   # In new terminal/session
   cd /path/to/testdrive-jsui
   # Work on feature
   git commit -m "feat: new feature"
   git push origin main
   ```

2. **Update main project** (different Claude instance)
   ```bash
   cd /home/worsch/markitect-main
   git submodule update --remote capabilities/testdrive-jsui
   git commit -m "chore: update testdrive-jsui submodule"
   ```

**Why separate instances?**
- Clear context boundaries
- Prevents accidental cross-contamination
- Respects repository independence
- Better commit hygiene

### Updating Submodules

When a capability releases a new version:

```bash
# In main repo
cd /home/worsch/markitect-main

# Update specific capability
cd capabilities/testdrive-jsui
git pull origin main
cd ../..
git add capabilities/testdrive-jsui
git commit -m "chore: update testdrive-jsui to v1.2.0"

# Or update all capabilities
git submodule update --remote
git commit -am "chore: update all capabilities"
```

### Adding New Capabilities

```bash
# 1. Create capability repository on gitea
# http://gitea/coulomb/new-capability

# 2. Add as submodule to main repo
cd /home/worsch/markitect-main
git submodule add http://gitea/coulomb/new-capability.git capabilities/new-capability

# 3. Add dependency to pyproject.toml
# dependencies = [
#     "new-capability @ file:./capabilities/new-capability"
# ]

# 4. Commit submodule addition
git commit -m "feat: add new-capability as submodule"
```

## Dependency Management

Capabilities are declared in `pyproject.toml`:

```toml
[project]
dependencies = [
    # Core dependencies
    "markdown-it-py",
    "click>=8.0.0",

    # Capabilities (file-based dependencies)
    "release-management @ file:./capabilities/release-management",
    "testdrive-jsui @ file:./capabilities/testdrive-jsui",
    "issue-facade @ file:./capabilities/issue-facade",
]
```

**Pattern for Capabilities:**
```toml
"capability-name @ file:./capabilities/capability-name"
```

This enables:
- ✅ pip install with editable dependencies
- ✅ Proper package resolution
- ✅ Clean imports in code
- ✅ Development mode support

## Capability Structure

Each capability should follow this structure:

```
capability-name/
├── README.md                      # Comprehensive documentation
├── pyproject.toml                 # Package configuration
├── Makefile                       # Build and test automation
├── .gitignore                     # Git ignore rules
├── src/capability_name/           # Python package source
│   ├── __init__.py
│   ├── core/                      # Core functionality
│   ├── utils/                     # Utilities
│   └── testing/                   # Testing infrastructure
├── js/                            # JavaScript (if applicable)
│   ├── components/
│   ├── controls/
│   └── tests/
├── static/                        # Static assets (if applicable)
│   ├── css/
│   ├── images/
│   └── templates/
├── tests/                         # Python tests
│   └── test_*.py
└── docs/                          # Additional documentation
    ├── API.md
    └── USAGE.md
```

### Required Files

1. **README.md** - Must include:
   - Purpose and description
   - Installation instructions
   - Usage examples
   - API documentation
   - Integration guide

2. **pyproject.toml** - Must define:
   - Package metadata
   - Dependencies
   - Build system
   - Entry points

3. **Makefile** - Should include:
   - help target
   - test targets
   - install/setup targets
   - capability-info target (for discovery)

## Plugin System Integration

For capabilities that provide plugins (like testdrive-jsui):

### Plugin Self-Declaration

```python
class TestDriveJSUIEngine(RenderingEnginePlugin):
    """Plugin declares its own location - no hardcoded paths."""

    def get_plugin_source_dir(self) -> Path:
        """Plugin knows where it lives."""
        return Path(__file__).parent.parent.parent / "capabilities" / "testdrive-jsui"

    def get_asset_paths(self) -> Dict[str, Path]:
        """Plugin declares its asset structure."""
        base = self.get_plugin_source_dir()
        return {
            'js': base / 'js',
            'css': base / 'static' / 'css',
            'templates': base / 'static' / 'templates',
        }
```

### Plugin Discovery

The main repo uses **generic discovery** - no hardcoded capability names:

```python
# ✅ Good: Generic discovery
if hasattr(engine, 'get_plugin_source_dir'):
    source_dir = engine.get_plugin_source_dir()

# ❌ Bad: Hardcoded capability names
if engine_name == 'testdrive-jsui':
    source_dir = Path('capabilities/testdrive-jsui')
```

## Testing Strategy

### Capability Tests
Each capability tests **in isolation**:

```bash
cd capabilities/testdrive-jsui
pytest tests/
npm test
```

### Integration Tests
Main repo tests **integration points**:

```python
# tests/integration/test_capabilities.py
def test_testdrive_jsui_integration():
    """Test that testdrive-jsui integrates correctly."""
    engine = TestDriveJSUIEngine()
    result = engine.render_document(sample_content, 'edit', config)
    assert result is not None
```

### Test Boundaries
- **Capability tests**: Internal functionality, unit tests, component tests
- **Main repo tests**: Integration, configuration, plugin discovery

## Migration Path

### Converting Local Capability to Submodule

1. **Create separate git repo**
   ```bash
   cd /tmp
   cp -r markitect-main/capabilities/capability-name capability-name
   cd capability-name
   git init
   git add .
   git commit -m "Initial commit"
   git remote add origin http://gitea/coulomb/capability-name.git
   git push -u origin main
   ```

2. **Remove from main repo**
   ```bash
   cd markitect-main
   git rm -rf capabilities/capability-name
   git commit -m "chore: remove capability-name for submodule conversion"
   ```

3. **Add as submodule**
   ```bash
   git submodule add http://gitea/coulomb/capability-name.git capabilities/capability-name
   git commit -m "feat: add capability-name as submodule"
   ```

## Best Practices

### DO ✅

1. **Develop capabilities independently**
   - Use separate Claude instances
   - Maintain clear context boundaries
   - Follow semantic versioning

2. **Use documented interfaces**
   - Import via Python packages
   - Use plugin APIs
   - Follow configuration patterns

3. **Test integration points**
   - Verify capability imports
   - Test plugin discovery
   - Validate configurations

4. **Update submodules explicitly**
   - Pull capability changes deliberately
   - Review updates before committing
   - Update main repo pointer

5. **Document integration**
   - How to use the capability
   - Configuration options
   - Example usage

### DON'T ❌

1. **Edit capability code from main repo**
   - Opens separate repo/instance instead
   - Respects submodule boundary
   - Maintains clean separation

2. **Hardcode capability paths**
   - Use self-declaration instead
   - Generic discovery patterns
   - Interface-based access

3. **Create circular dependencies**
   - Main depends on capabilities
   - Capabilities should NOT depend on main
   - Keep dependency graph acyclic

4. **Bypass submodule system**
   - Don't manually copy files
   - Use git submodule commands
   - Respect version control

## Troubleshooting

### Submodule not updated

```bash
# Pull latest from capability repo
cd capabilities/testdrive-jsui
git pull origin main

# Update main repo pointer
cd ../..
git add capabilities/testdrive-jsui
git commit -m "chore: update testdrive-jsui"
```

### Import errors

```bash
# Reinstall capabilities
pip install -e .

# Or specific capability
pip install -e ./capabilities/testdrive-jsui
```

### Merge conflicts in submodules

```bash
# Don't merge in submodule from main repo!
# Instead, go to capability repo directly

cd /path/to/separate/testdrive-jsui
# or
cd capabilities/testdrive-jsui

# Resolve conflicts there
git pull origin main
# fix conflicts
git push origin main

# Then update main repo
cd ../../
git submodule update --remote
```

## See Also

- [Agent: Capability Manager](../../agents/agent-capability-manager.md) - Automated capability management
- [Plugin System](../PLUGIN_SYSTEM.md) - Plugin architecture documentation
- [Git Submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) - Official git submodules guide

---

**Remember:** Capabilities are **independent projects**. Treat them with the same respect you'd give any external dependency - use their interfaces, don't modify their internals.