markitect-main/history/COST_TRACKING_GAMEPLAN.md

# Cost Tracking System Gameplan - Issue #88

**Date**: 2025-10-04
**Status**: DESIGN PHASE
**Priority**: MEDIUM
**Estimated Effort**: 3-4 weeks development

## 🎯 Executive Summary

Implement a comprehensive cost tracking and allocation system that:
- **Tracks operational costs** (monthly subscriptions, one-time expenses)
- **Allocates costs to active issues** in each calculation period
- **Maintains financial audit trail** with transaction history
- **Calculates issue valuations** based on cost allocation
- **Handles historical costs** through Loss Carried Forward mechanism

## 📋 Requirements Analysis

### **Business Model**
```yaml
Cost Sources:
  - Monthly Recurring: Server instances, SaaS subscriptions, domains
  - One-Time Costs: Setup fees, equipment, licenses

Cost Allocation:
  - Target: Active issues (new + modified) in calculation period
  - Method: Equal distribution across active issues
  - Historical: Unallocated costs carry forward to next period

Financial Tracking:
  - Ledger-based accounting with double-entry principles
  - Transaction audit trail for all cost movements
  - Period-based calculations and reporting
```

### **Example Cost Items (from issue description)**
- **Hosteurope server**: €10/month
- **Bubble.io plan**: €32/month
- **Coulomb.social domain**: €5/month
- **Claude Code plan**: €20/month
- **Gemini plan**: €20/month
- **Total monthly**: €87/month

## 🏗️ Technical Architecture

### **Database Schema**
```sql
-- Cost categories and types
CREATE TABLE cost_categories (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL UNIQUE,
    description TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Cost items (recurring and one-time)
CREATE TABLE cost_items (
    id INTEGER PRIMARY KEY,
    category_id INTEGER REFERENCES cost_categories(id),
    name TEXT NOT NULL,
    description TEXT,
    cost_type TEXT CHECK (cost_type IN ('monthly', 'one_time')) NOT NULL,
    amount_eur DECIMAL(10,2) NOT NULL,
    currency TEXT DEFAULT 'EUR',
    starting_from_date DATE NOT NULL,
    ending_date DATE, -- NULL for ongoing monthly costs
    is_active BOOLEAN DEFAULT TRUE,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Calculation periods
CREATE TABLE cost_periods (
    id INTEGER PRIMARY KEY,
    period_start DATE NOT NULL,
    period_end DATE NOT NULL,
    period_type TEXT DEFAULT 'monthly',
    status TEXT CHECK (status IN ('open', 'calculating', 'closed')) DEFAULT 'open',
    total_costs DECIMAL(10,2) DEFAULT 0,
    active_issues_count INTEGER DEFAULT 0,
    cost_per_issue DECIMAL(10,2) DEFAULT 0,
    loss_carried_forward DECIMAL(10,2) DEFAULT 0,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Cost transactions (ledger entries)
CREATE TABLE cost_transactions (
    id INTEGER PRIMARY KEY,
    period_id INTEGER REFERENCES cost_periods(id),
    cost_item_id INTEGER REFERENCES cost_items(id),
    transaction_type TEXT CHECK (transaction_type IN ('cost_incurred', 'cost_allocated', 'loss_forward')) NOT NULL,
    amount_eur DECIMAL(10,2) NOT NULL,
    issue_id INTEGER, -- NULL for non-issue transactions
    transaction_date DATE NOT NULL,
    description TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Issue cost allocations
CREATE TABLE issue_cost_allocations (
    id INTEGER PRIMARY KEY,
    issue_id INTEGER NOT NULL,
    period_id INTEGER REFERENCES cost_periods(id),
    allocated_amount DECIMAL(10,2) NOT NULL,
    allocation_date DATE NOT NULL,
    transaction_id INTEGER REFERENCES cost_transactions(id),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- Issue activity tracking for cost allocation
CREATE TABLE issue_activity_log (
    id INTEGER PRIMARY KEY,
    issue_id INTEGER NOT NULL,
    activity_type TEXT CHECK (activity_type IN ('created', 'modified', 'closed', 'reopened')) NOT NULL,
    activity_date DATE NOT NULL,
    period_id INTEGER REFERENCES cost_periods(id),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```

### **Core Components**

#### **1. Cost Management Module**
```python
# markitect/finance/cost_manager.py
class CostManager:
    - register_cost_item(name, type, amount, start_date)
    - update_cost_item(cost_id, updates)
    - deactivate_cost_item(cost_id, end_date)
    - get_active_costs_for_period(period_start, period_end)
    - calculate_monthly_costs(period)
```

#### **2. Period Management Module**
```python
# markitect/finance/period_manager.py
class PeriodManager:
    - create_period(start_date, end_date)
    - close_period(period_id)
    - get_active_period()
    - calculate_period_costs(period_id)
    - handle_loss_carried_forward(period_id)
```

#### **3. Cost Allocation Engine**
```python
# markitect/finance/allocation_engine.py
class AllocationEngine:
    - identify_active_issues(period_start, period_end)
    - calculate_cost_per_issue(total_costs, active_issue_count)
    - allocate_costs_to_issues(period_id)
    - create_allocation_transactions()
    - update_issue_valuations()
```

#### **4. Financial Reporting Module**
```python
# markitect/finance/reports.py
class FinancialReports:
    - cost_summary_report(period_id)
    - issue_cost_analysis(issue_id)
    - cost_trend_analysis(start_period, end_period)
    - budget_variance_report()
    - export_to_accounting_format()
```

## 🚀 Implementation Plan

### **Phase 1: Core Infrastructure (Week 1-2)**

#### **Issue 1: Cost Ledger Database Schema and Models**
- Create database tables and migrations
- Implement SQLAlchemy/database models
- Add basic CRUD operations
- Database constraints and relationships

#### **Issue 2: Cost Item Management System**
- Cost registration and lifecycle management
- Support for monthly/one-time cost types
- Cost categorization and metadata
- CLI commands for cost management

#### **Issue 3: Period Management Framework**
- Period creation and lifecycle
- Automatic period transitions
- Status management (open/calculating/closed)
- Period-based data aggregation

### **Phase 2: Cost Allocation Logic (Week 2-3)**

#### **Issue 4: Issue Activity Tracking**
- Monitor issue creation/modification events
- Integration with existing issue management
- Activity log for cost allocation purposes
- Historical activity data migration

#### **Issue 5: Cost Allocation Engine**
- Implement cost distribution algorithm
- Handle edge cases (no active issues, partial periods)
- Loss carried forward calculations
- Transaction audit trail generation

#### **Issue 6: Integration with Issue Management**
- Hook into existing issue lifecycle
- Automatic activity tracking
- Cost allocation triggers
- Issue valuation updates

### **Phase 3: Reporting and CLI (Week 3-4)**

#### **Issue 7: Financial Reporting System**
- Cost summary and trend reports
- Issue-specific cost analysis
- Period-over-period comparisons
- Export capabilities (CSV, JSON)

#### **Issue 8: CLI Commands and User Interface**
- Cost management commands
- Period management commands
- Report generation commands
- Financial dashboard views

#### **Issue 9: Automation and Scheduling**
- Automatic period creation
- Scheduled cost calculations
- Alert system for budget overruns
- Integration with external accounting systems

## 💰 **Cost Allocation Algorithm**

### **Monthly Calculation Process**
```python
def calculate_monthly_allocation(period_start, period_end):
    """
    1. Identify all costs for the period
    2. Calculate total cost burden
    3. Identify active issues
    4. Distribute costs equally
    5. Handle loss carried forward
    6. Create audit transactions
    """

    # Step 1: Gather costs
    monthly_costs = get_monthly_costs_for_period(period_start, period_end)
    one_time_costs = get_one_time_costs_for_period(period_start, period_end)
    carried_forward = get_loss_carried_forward(previous_period)

    total_costs = monthly_costs + one_time_costs + carried_forward

    # Step 2: Identify active issues
    active_issues = get_active_issues(period_start, period_end)

    if len(active_issues) == 0:
        # No active issues - carry forward all costs
        carry_forward_loss(total_costs, next_period)
        return

    # Step 3: Calculate allocation
    cost_per_issue = total_costs / len(active_issues)

    # Step 4: Allocate to issues
    for issue in active_issues:
        allocate_cost_to_issue(issue.id, cost_per_issue, period_id)
        create_allocation_transaction(issue.id, cost_per_issue, period_id)
        update_issue_valuation(issue.id, cost_per_issue)
```

### **Active Issue Definition**
```sql
-- Issues are considered "active" if they have any of these activities:
-- 1. Created during the period
-- 2. Modified/updated during the period
-- 3. Had comments added during the period
-- 4. Changed status during the period

SELECT DISTINCT issue_id
FROM issue_activity_log
WHERE activity_date BETWEEN period_start AND period_end
AND activity_type IN ('created', 'modified', 'commented', 'status_changed')
```

## 📊 **Example CLI Commands**

### **Cost Management**
```bash
# Register recurring costs
markitect cost add "Hosteurope Server" --type=monthly --amount=10.00 --start-date=2025-01-01

# Register one-time costs
markitect cost add "SSL Certificate" --type=one_time --amount=50.00 --date=2025-03-15

# List all costs
markitect cost list --active
markitect cost list --period=2025-03

# Update cost
markitect cost update 1 --amount=12.00 --end-date=2025-12-31
```

### **Period Management**
```bash
# Create new period
markitect period create --start=2025-03-01 --end=2025-03-31

# Calculate period costs
markitect period calculate 2025-03

# Close period
markitect period close 2025-03

# List periods
markitect period list --status=open
```

### **Reports and Analysis**
```bash
# Cost summary
markitect reports cost-summary --period=2025-03

# Issue cost analysis
markitect reports issue-costs --issue=123

# Cost trends
markitect reports cost-trends --start=2025-01 --end=2025-03

# Export for accounting
markitect reports export --format=csv --output=costs_2025_q1.csv
```

## 🧪 **Testing Strategy**

### **Unit Tests**
- Cost calculation algorithms
- Period management logic
- Allocation engine correctness
- Database model relationships

### **Integration Tests**
- End-to-end cost allocation workflow
- Issue activity tracking integration
- Report generation accuracy
- CLI command functionality

### **Scenario Tests**
- No active issues in period
- Partial period calculations
- Historical cost migration
- Multiple cost types in single period

## 📈 **Success Metrics**

### **Functional Requirements**
- ✅ Accurate cost tracking and categorization
- ✅ Correct allocation to active issues
- ✅ Audit trail for all financial transactions
- ✅ Historical data preservation and analysis

### **Performance Requirements**
- Process monthly calculations in <5 seconds
- Support 1000+ cost items and 10,000+ issues
- Report generation in <10 seconds
- Database queries optimized for financial data

### **User Experience**
- Intuitive CLI commands for cost management
- Clear financial reports and dashboards
- Easy integration with existing workflows
- Minimal manual intervention required

## ⚠️ **Risk Assessment**

### **Technical Risks**
1. **Data Integrity**: Financial data requires 100% accuracy
   - **Mitigation**: Comprehensive validation, audit trails, backup procedures

2. **Performance**: Large-scale calculations may be slow
   - **Mitigation**: Database optimization, caching, incremental calculations

3. **Complexity**: Financial logic can be complex and error-prone
   - **Mitigation**: Extensive testing, code review, financial validation

### **Business Risks**
1. **Accounting Compliance**: May need integration with formal accounting
   - **Mitigation**: Export capabilities, audit trail, professional review

2. **Currency Handling**: Multi-currency support may be needed
   - **Mitigation**: Design for EUR initially, plan for currency expansion

3. **Tax Implications**: Cost allocation may have tax consequences
   - **Mitigation**: Clear documentation, professional consultation

## 🎯 **Future Enhancements**

### **Phase 2 Features**
- Multi-currency support
- Budget planning and forecasting
- Integration with external accounting systems (QuickBooks, Xero)
- Cost optimization recommendations
- Team/department-based cost allocation

### **Advanced Analytics**
- Cost per feature analysis
- ROI calculations for development efforts
- Predictive cost modeling
- Resource utilization optimization
- Financial KPI dashboards

## 📋 **Implementation Checklist**

### **Prerequisites**
- [ ] Review existing database schema for compatibility
- [ ] Identify integration points with issue management
- [ ] Define chart of accounts and cost categories
- [ ] Plan data migration strategy for historical costs

### **Development**
- [ ] Database schema implementation
- [ ] Core cost management functionality
- [ ] Allocation engine development
- [ ] CLI command implementation
- [ ] Report generation system

### **Testing**
- [ ] Unit test coverage >95%
- [ ] Integration test scenarios
- [ ] Performance benchmarking
- [ ] User acceptance testing
- [ ] Financial accuracy validation

### **Deployment**
- [ ] Database migration scripts
- [ ] Documentation and user guides
- [ ] Training materials
- [ ] Monitoring and alerting setup
- [ ] Backup and recovery procedures

This comprehensive cost tracking system will provide MarkiTect with sophisticated financial management capabilities, enabling data-driven decisions about development priorities and resource allocation.