markitect-main/roadmap/260106-release-management-optimization/WORKPLAN.md

# Release Management Optimization Workplan

**Topic**: 260106-release-management-optimization
**Created**: 2026-01-06
**Status**: Stages 1-2 Complete, v0.10.0 Released
**Priority**: High (blocks v0.10.0 release) ✅ UNBLOCKED

---

## Overview

Enhance release management infrastructure with robust validation and automation, using the newly built schema system. This creates a practical showcase for the schema evolution capabilities while fixing critical release tooling issues.

## Motivation

### Current Issues

1. **setuptools-scm Configuration Bug**
   - Missing tag regex filter → picks up non-version tags
   - Result: `markitect --version` returns "unknown"
   - Impact: Users can't verify installed version

2. **Version History Inconsistency**
   - CHANGELOG shows v0.9.0 (2025-11-14) but git tag never created
   - setuptools-scm sees v0.8.0 as latest
   - 109 commits between v0.8.0 and current

3. **No Validation Infrastructure**
   - No CHANGELOG validation (format, version consistency)
   - No pre-release checks for version/tag alignment
   - No automated version bump validation

4. **Missing CLI Features**
   - No explicit `markitect --version` command
   - Version info only via __version__.py fallback

### Opportunity: Schema System Showcase

Creating a **CHANGELOG schema** demonstrates:
- Real-world use of x-markitect extensions
- Schema validation for structured markdown
- Section classification (Unreleased, versioned releases)
- Content patterns (Keep a Changelog format)
- Integration with release tooling

**Perfect timing**: Release v0.10.0 with the tool that validates its own changelog!

---

## Goals

### Primary
1. ✅ **Fix setuptools-scm configuration** (blocks release)
2. ✅ **Restore version history** (retroactive v0.9.0 tag)
3. ✅ **Create CHANGELOG schema** (Keep a Changelog format)
4. ✅ **Implement CHANGELOG validation** (pre-release check)

### Secondary
5. ⭐ **Add explicit version command** to markitect CLI
6. ⭐ **Extend schema system** (if needed for system calls/agents)
7. ⭐ **Release capability enhancements** (validation hooks)

### Stretch
8. 🎯 **Git tag validation** (ensure CHANGELOG ↔ tags sync)
9. 🎯 **Automated version bumping** suggestions
10. 🎯 **Release notes generation** from CHANGELOG

---

## Staged Workplan

### Stage 1: Critical Fixes (Required for v0.10.0)

**Goal**: Unblock immediate release

#### Task 1.1: Fix setuptools-scm Configuration
**File**: `pyproject.toml`

**Current**:
```toml
[tool.setuptools_scm]
write_to = "markitect/_version.py"
```

**Updated**:
```toml
[tool.setuptools_scm]
write_to = "markitect/_version.py"
version_scheme = "python-simplified-semver"
tag_regex = "^v(?P<version>[0-9]+\\.[0-9]+\\.[0-9]+)$"
local_scheme = "no-local-version"
```

**Validation**:
- Run `python -c "from setuptools_scm import get_version; print(get_version())"`
- Should show `0.8.1.dev109+g5e3646f` (or similar, based on v0.8.0 tag)
- Run `markitect --version` → should show version (after reinstall)

**Estimated**: 10 minutes

#### Task 1.2: Restore Version History
**Action**: Retroactively create v0.9.0 tag

**Find v0.9.0 release commit**:
```bash
# Find commit around 2025-11-14 with plugin/rendering changes
git log --since="2025-11-13" --until="2025-11-15" --oneline
```

**Create tag**:
```bash
# Tag the identified commit
git tag -a v0.9.0 <commit-hash> -m "Release v0.9.0: Plugin infrastructure and TestDrive JSUI

- Plugin Infrastructure Foundation
- RenderingEngineManager
- TestDrive JSUI Plugin
- ChatGPT Document Theme
- CLI Engine Parameter

See CHANGELOG.md for full details."
```

**Validation**:
- `git tag -l 'v*'` → should show v0.9.0
- `git describe --tags --match='v*'` → should show v0.9.0-based description

**Estimated**: 15 minutes

#### Task 1.3: Prepare v0.10.0 Release
**File**: `CHANGELOG.md`

**Actions**:
1. Move Unreleased content to v0.10.0 section
2. Add release date
3. Verify version links at bottom

**Format**:
```markdown
## [Unreleased]

## [0.10.0] - 2026-01-06

### Added
- **ADR Schema**: Architecture Decision Record validation schema
  - 12 section classifications for comprehensive ADR structure
  - Content pattern validation for formatting rules
  - Quality metrics for completeness
- **Markdown Schema Support**: Fixed validate and generate-stub commands
  - load_schema_from_path() supports .json and .md files
  - DocumentWrapper extracts headings from AST
  - All schema commands now work with markdown schemas
- **Schema Evolution Topic Closure**: Complete Phase 1-3 implementation
  - 5 production schemas (manpage, API docs, terminology, schema-schema, ADR)
  - Semantic validation system fully operational
  - Template generation working with schemas

### Fixed
- setuptools-scm configuration with tag_regex for proper version detection
- markitect --version now returns correct version instead of "unknown"
- Semantic validator AST heading extraction via DocumentWrapper

[Unreleased]: https://github.com/worsch/markitect/compare/v0.10.0...HEAD
[0.10.0]: https://github.com/worsch/markitect/compare/v0.9.0...v0.10.0
```

**Estimated**: 20 minutes

**Total Stage 1**: ~45 minutes

---

### Stage 2: CHANGELOG Schema (Showcase Feature)

**Goal**: Create and validate CHANGELOG schema using schema evolution infrastructure

#### Task 2.1: Analyze CHANGELOG Structure
**Method**: Study Keep a Changelog format

**Structure to validate**:
```markdown
# Changelog

[preamble text]

## [Unreleased]
### Added / Changed / Deprecated / Removed / Fixed / Security
- Bullet points

## [X.Y.Z] - YYYY-MM-DD
### Added / Changed / Deprecated / Removed / Fixed / Security
- Bullet points

[version links at bottom]
```

**Schema Requirements**:
- **Sections**: Unreleased (required), Version sections (pattern: `[X.Y.Z]`)
- **Subsections**: Added/Changed/Deprecated/Removed/Fixed/Security (optional)
- **Content Patterns**:
  - Version format: `\[(\d+\.\d+\.\d+)\]`
  - Date format: `YYYY-MM-DD`
  - Links at bottom: `[version]: url`
- **Quality Metrics**:
  - Each version should have at least one subsection
  - Bullet points required in subsections

**Estimated**: 30 minutes

#### Task 2.2: Create changelog-schema-v1.0.md
**File**: `markitect/schemas/changelog-schema-v1.0.md`

**Schema structure**:
```markdown
---
schema-id: "https://markitect.dev/schemas/changelog/v1.0"
version: "1.0.0"
status: "stable"
domain: "changelog"
description: "JSON schema for Keep a Changelog format with version history validation"
---

# Changelog Schema v1.0

## Overview
This schema validates CHANGELOG.md files following the Keep a Changelog format...

## Schema Definition
```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "x-markitect-sections": {
    "Unreleased": {
      "classification": "required",
      "heading_level": 2,
      "error_message": "Unreleased section is mandatory for tracking upcoming changes"
    },
    // Version sections validated via pattern
  },
  "x-markitect-content-control": {
    "unreleased": {
      "content_quality": {
        "min_words": 0  // Can be empty
      }
    },
    // Version sections with date validation
  }
}
```
```

**Considerations for Extension**:
- **System Call Validation**: Check git tags match CHANGELOG versions
  - `x-markitect-validation-hooks`: { "system": "git tag -l 'v*'" }
  - Requires schema system extension

- **Agent Validation**: Use AI to validate version consistency
  - `x-markitect-validation-agents`: { "prompt": "Check versions align" }
  - Requires agent integration

**Decision**: Start with pure schema validation, add hooks/agents if needed

**Estimated**: 90 minutes

#### Task 2.3: Ingest and Test CHANGELOG Schema
**Commands**:
```bash
# Ingest
markitect schema-ingest markitect/schemas/changelog-schema-v1.0.md

# Validate our CHANGELOG
markitect validate CHANGELOG.md --schema changelog-schema-v1.0.md
```

**Expected**:
- Section validation: Unreleased section present ✓
- Version format validation: All versions follow X.Y.Z ✓
- Date validation: ISO format YYYY-MM-DD ✓
- Content structure: Subsections present ✓

**Fix any issues found in CHANGELOG.md**

**Estimated**: 30 minutes

**Total Stage 2**: ~2.5 hours

---

### Stage 3: Release Capability Enhancements

**Goal**: Integrate CHANGELOG validation into release workflow

#### Task 3.1: Add CHANGELOG Validation to Release Manager
**File**: `capabilities/release-management/src/release_management/core/manager.py`

**Add validation method**:
```python
def validate_changelog(self) -> tuple[bool, list[str]]:
    """Validate CHANGELOG.md against changelog schema."""
    from markitect.cli import validate_against_schema

    changelog_path = self.project_root / "CHANGELOG.md"
    schema_path = self.project_root / "markitect/schemas/changelog-schema-v1.0.md"

    if not changelog_path.exists():
        return False, ["CHANGELOG.md not found"]

    is_valid = validate_against_schema(changelog_path, schema_path)
    if not is_valid:
        return False, ["CHANGELOG.md validation failed"]

    return True, []
```

**Integrate into validate_release_state()**:
```python
def validate_release_state(self) -> tuple[bool, list[str]]:
    issues = []

    # Existing validations...

    # Add CHANGELOG validation
    changelog_valid, changelog_issues = self.validate_changelog()
    if not changelog_valid:
        issues.extend(changelog_issues)

    return len(issues) == 0, issues
```

**Estimated**: 45 minutes

#### Task 3.2: Add Version-Tag Consistency Check
**Feature**: Verify CHANGELOG versions match git tags

**Method**:
```python
def check_version_tag_consistency(self) -> tuple[bool, list[str]]:
    """Check CHANGELOG versions have corresponding git tags."""
    import re

    # Parse CHANGELOG for versions
    changelog = (self.project_root / "CHANGELOG.md").read_text()
    versions = re.findall(r'## \[(\d+\.\d+\.\d+)\]', changelog)

    # Get git tags
    result = subprocess.run(['git', 'tag', '-l', 'v*'],
                          capture_output=True, text=True)
    tags = result.stdout.strip().split('\n')
    tag_versions = [t.lstrip('v') for t in tags if t.startswith('v')]

    # Check consistency
    issues = []
    for version in versions:
        if version not in tag_versions:
            issues.append(f"CHANGELOG version {version} has no git tag v{version}")

    return len(issues) == 0, issues
```

**Estimated**: 45 minutes

#### Task 3.3: Add Explicit Version Command
**File**: `markitect/cli.py`

**Add command**:
```python
@cli.command()
def version():
    """Display detailed version information."""
    from markitect.__version__ import get_version_info

    info = get_version_info()

    click.echo(f"MarkiTect {info['full_version']}")
    click.echo(f"Git Commit: {info['git_commit']}")
    click.echo(f"Git Branch: {info['git_branch']}")

    if info['is_dev']:
        click.echo("⚠️  Development version (not released)")
    else:
        click.echo(f"✅ Released version")
```

**Update --version option**:
```python
@click.version_option(version=get_version(), prog_name="markitect")
def cli():
    ...
```

**Estimated**: 30 minutes

**Total Stage 3**: ~2 hours

---

### Stage 4: Schema System Extensions (If Needed)

**Goal**: Add validation hooks for system calls and agents

**Only implement if pure schema validation insufficient**

#### Option A: System Call Hooks
**Extension**: `x-markitect-validation-hooks`

```json
{
  "x-markitect-validation-hooks": {
    "pre-validation": [
      {
        "name": "check-git-tags",
        "command": "git tag -l 'v*'",
        "parser": "lines",
        "validation": {
          "min_count": 5,
          "pattern": "^v\\d+\\.\\d+\\.\\d+$"
        }
      }
    ]
  }
}
```

**Implementation**:
- Add HookValidator to validators package
- Execute system commands securely
- Parse and validate output
- Integrate into SemanticValidator

**Estimated**: 3 hours

#### Option B: Agent Validation
**Extension**: `x-markitect-validation-agents`

```json
{
  "x-markitect-validation-agents": {
    "consistency-check": {
      "prompt": "Verify all CHANGELOG versions have corresponding git tags",
      "context": ["CHANGELOG.md", "git tag -l"],
      "severity": "warning"
    }
  }
}
```

**Implementation**:
- Add AgentValidator to validators package
- Integration with LLM/agent framework
- Structured output parsing
- Integrate into SemanticValidator

**Estimated**: 4 hours

**Decision Point**: Only proceed if needed for CHANGELOG validation

**Total Stage 4**: 0-7 hours (conditional)

---

## Success Criteria

### Stage 1 (Required for Release)
- ✅ `markitect --version` returns actual version (not "unknown")
- ✅ v0.9.0 git tag exists
- ✅ CHANGELOG.md has v0.10.0 section
- ✅ v0.10.0 ready for tagging

### Stage 2 (Showcase Feature)
- ✅ changelog-schema-v1.0.md created and ingested
- ✅ CHANGELOG.md validates against schema
- ✅ Schema demonstrates Keep a Changelog format validation

### Stage 3 (Enhanced Tooling)
- ✅ Release validation includes CHANGELOG check
- ✅ Version-tag consistency checking works
- ✅ `markitect version` command available

### Stage 4 (Optional Extensions)
- ⭐ System call hooks functional (if implemented)
- ⭐ Agent validation working (if implemented)

---

## Timeline Estimate

### Fast Track (Stage 1 Only)
- **Time**: 45 minutes
- **Scope**: Critical fixes for v0.10.0 release
- **Result**: Can release immediately

### Standard Track (Stages 1-3)
- **Time**: 5 hours
- **Scope**: Fixes + CHANGELOG schema + tooling enhancements
- **Result**: Production-ready release management

### Full Track (Stages 1-4)
- **Time**: 5-12 hours (depends on extensions needed)
- **Scope**: Complete system with validation hooks/agents
- **Result**: Advanced validation infrastructure

---

## Decision Points

### 1. Release Strategy
**Options**:
- A. Fast track → Release v0.10.0 now with Stage 1 fixes only
- B. Standard track → Complete Stages 1-3, release as v0.10.1 or v0.11.0
- C. Full track → Include Stage 4, major release v1.0.0

**Recommendation**: **Option B (Standard Track)**
- Showcases schema system with real use case
- Improves release tooling robustly
- Reasonable timeline (5 hours)
- Release as **v0.10.0** with all features

### 2. Schema Extensions
**When to implement**:
- Stage 4 only if pure schema validation can't handle:
  - Git tag checking (may need system call hook)
  - Version consistency (may need agent validation)

**Recommendation**: **Start without extensions**
- Try pure schema validation first
- Add hooks/agents only if needed
- Document extension needs for future enhancement

### 3. release-management Extraction
**Question**: Make it a git submodule?

**Current**: Regular directory (not extracted)

**Recommendation**: **Defer to future**
- Not blocking for release
- Works fine as capability
- Can extract later if needed for reuse

---

## Files to Create/Modify

### Create
- `markitect/schemas/changelog-schema-v1.0.md` - CHANGELOG validation schema
- `roadmap/260106-release-management-optimization/DONE.md` - Completion checklist (when done)

### Modify
- `pyproject.toml` - Fix setuptools-scm configuration
- `CHANGELOG.md` - Add v0.10.0 section, fix format if needed
- `markitect/cli.py` - Add explicit version command
- `capabilities/release-management/src/release_management/core/manager.py` - Add CHANGELOG validation
- (Optional) `markitect/validators/` - Add hook/agent validators if needed

### Git Operations
- Create v0.9.0 tag retroactively
- Create v0.10.0 tag for release

---

## Risks & Mitigations

### Risk 1: Retroactive Tagging
**Issue**: Creating v0.9.0 tag retroactively might confuse users

**Mitigation**:
- Document in CHANGELOG that tag was missing
- Use tag message to explain retroactive creation
- Don't publish v0.9.0 packages (already past that release)

### Risk 2: Schema Extensions Complexity
**Issue**: Implementing hooks/agents might be complex

**Mitigation**:
- Start with pure schema validation
- Only add extensions if necessary
- Document extension API for future

### Risk 3: CHANGELOG Format Variations
**Issue**: Real-world CHANGELOGs may not match Keep a Changelog exactly

**Mitigation**:
- Make schema flexible with optional sections
- Use warnings instead of errors for style issues
- Support alternative section names

---

## Completion Summary

**Completed**: 2026-01-06
**Release**: v0.10.0
**Track**: Standard (Stages 1-2)

### Stage 1: Critical Fixes ✅

**Duration**: ~45 minutes
**Status**: COMPLETE

#### Achievements:
1. ✅ **Fixed setuptools-scm Configuration**
   - Added `git_describe_command = "git describe --tags --long --match 'v*'"`
   - Filters out non-version tags (e.g., "testdrive-jsui-migration-phase4-complete")
   - Version detection now works: `markitect --version` → 0.10.0
   - File: `pyproject.toml`
   - Commit: 061ba88

2. ✅ **Retroactively Created v0.9.0 Git Tag**
   - Tagged commit b9c1b90 from 2025-11-14
   - Maintains version history integrity
   - CHANGELOG documented v0.9.0 but tag was missing
   - Enables proper version progression to v0.10.0
   - Commit: 061ba88

3. ✅ **Prepared CHANGELOG.md for v0.10.0**
   - Created [0.10.0] - 2026-01-06 section
   - Moved Unreleased content to v0.10.0
   - Documented version detection fixes
   - Documented v0.9.0 retroactive tag
   - Commit: 061ba88

### Stage 2: CHANGELOG Schema ✅

**Duration**: ~90 minutes
**Status**: COMPLETE

#### Achievements:
1. ✅ **Created changelog-schema-v1.0.md**
   - Comprehensive schema for Keep a Changelog format
   - 360+ lines of schema definition and documentation
   - File: `markitect/schemas/changelog-schema-v1.0.md`
   - Commit: c4ee5cc

2. ✅ **Implemented x-markitect Extensions**
   - `x-markitect-sections`: 7 section classifications
     - [Unreleased]: required
     - Added/Changed/Deprecated/Removed/Fixed/Security: optional
   - `x-markitect-content-control`: 6 content patterns
     - Title validation, introduction patterns, version format
     - Date format (ISO 8601), change types, reference links
   - `x-markitect-validation-rules`: 4 custom rules
     - Version format, date format, version ordering, unreleased position

3. ✅ **Schema Ingestion and Testing**
   - Ingested into schema catalog (Record ID: 12)
   - Successfully validates project CHANGELOG.md
   - All section requirements met (7 checked, 11 found)
   - All content requirements met
   - All semantic checks passing
   - Command: `markitect validate CHANGELOG.md --schema changelog-schema-v1.0.md --semantic`

4. ✅ **Documentation in CHANGELOG**
   - Documented new schema in v0.10.0 Added section
   - Philosophy: "The release that validates itself"
   - Showcase of schema system practical application

### Version Release ✅

**Tag**: v0.10.0
**Date**: 2026-01-06
**Verification**: `markitect --version` → 0.10.0

### Success Metrics

**Stage 1 Criteria** (Required for Release):
- ✅ `markitect --version` returns actual version (0.10.0, not "unknown")
- ✅ v0.9.0 git tag exists
- ✅ CHANGELOG.md has v0.10.0 section
- ✅ v0.10.0 tagged and ready

**Stage 2 Criteria** (Showcase Feature):
- ✅ changelog-schema-v1.0.md created and ingested
- ✅ CHANGELOG.md validates against schema
- ✅ Schema demonstrates Keep a Changelog format validation
- ✅ All semantic validation checks passing

### Deferred Work

**Stage 3** (Release Capability Enhancements):
- ⭐ CHANGELOG validation in ReleaseManager
- ⭐ Version-tag consistency checking
- ⭐ Explicit `markitect version` command
- **Status**: Deferred to future enhancement
- **Reason**: v0.10.0 release unblocked, showcase feature complete

**Stage 4** (Schema System Extensions):
- 🎯 System call hooks (x-markitect-validation-hooks)
- 🎯 Agent validation (x-markitect-validation-agents)
- **Status**: Not needed for CHANGELOG validation
- **Reason**: Pure schema validation sufficient

### Files Created/Modified

**Created**:
- `markitect/schemas/changelog-schema-v1.0.md` (360 lines)

**Modified**:
- `pyproject.toml` (setuptools-scm configuration)
- `CHANGELOG.md` (v0.10.0 section, changelog schema documentation)
- `roadmap/260106-release-management-optimization/WORKPLAN.md` (this file)

**Tags Created**:
- `v0.9.0` (retroactive, commit b9c1b90)
- `v0.10.0` (release, commit c4ee5cc+)

### Commits

1. `4e9117d` - plan: create release-management-optimization roadmap topic
2. `061ba88` - fix: resolve version detection and prepare v0.10.0 release
3. `c4ee5cc` - feat: add changelog schema for Keep a Changelog validation
4. `v0.10.0` - Release tag created

### Philosophy Achievement

> "Use the tools we build to improve the tools we build."

**Result**: v0.10.0 is "The release that validates itself"
- ✅ Uses its own schema system to validate its CHANGELOG.md
- ✅ Demonstrates schema evolution practical value
- ✅ Real-world showcase of x-markitect extensions
- ✅ Perfect example of dogfooding infrastructure

---

## Notes

- This workplan creates a **perfect showcase** for schema evolution
- Validates its own CHANGELOG with the schema system it just built
- Real-world practical application demonstrating value
- Release v0.10.0 becomes a milestone: "The release that validates itself"

**Philosophy**: Use the tools we build to improve the tools we build.