docs: complete project documentation and task management cleanup
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled

### Documentation Updates
- Added comprehensive WORKSPACE_AND_DATABASES.md documentation explaining:
  - Markitect's workspace-based architecture concept
  - Database separation (markitect.db vs assets.db) and purposes
  - Configuration management and asset integration
  - Best practices for development, collaboration, and production

### Changelog Management
- Updated CHANGELOG.md with complete release history coverage
- Added missing v0.8.0 entry for setuptools-SCM integration and release automation
- Added proper version comparison links for all releases
- Documented all recent work in Unreleased section following Keep a Changelog format

### Task Management
- Cleaned TODO.md file by removing all completed tasks
- Reset to clean state referencing changelog for completed work
- Maintained Keep a Todofile format for future development sessions

This completes the documentation and task management improvements for
the ChatGPT theme implementation, modular theme system, issue-facade
bug fixes, and workspace architecture clarification work.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-11-10 14:34:54 +01:00
parent c699d7d669
commit 7270bc559d
3 changed files with 310 additions and 199 deletions

View File

@@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **ChatGPT Document Theme**: New document theme with Inter font, 580px width, and #10a37f accent color with full CLI support (`markitect md-render --theme chatgpt`)
- **Modular Theme System Architecture**: File-based theme loading with YAML configuration and dynamic theme discovery
- **Theme Directory Structure**: Organized theme components (mode/, ui/, document/, branding/) for better maintainability
- **Database Architecture Documentation**: Comprehensive WORKSPACE_AND_DATABASES.md documenting workspace concepts and database purposes
### Changed
- **Theme Loading System**: Implemented dynamic theme discovery and loading with metadata preservation
- **Test Suite Organization**: Removed obsolete configuration CLI tests (490 lines) for cleaner codebase
### Fixed
- **Issue-facade Click Framework Bug**: Resolved Sentinel bug in list command that was causing CLI failures
- **Issue-facade Version Command**: Fixed installation error preventing version command from working
- **Test Isolation Issues**: Improved test isolation with proper mocking to prevent cross-test interference
- **Theme Color Assertions**: Updated test assertions to work with new modular theme system
## [0.8.0] - 2025-11-08
### Added
- **Setuptools-SCM Integration**: Automatic version management system replacing manual version tracking
- **Gitea Package Publishing**: Complete CI/CD pipeline for automated package publishing to Gitea
- **Enhanced Release Documentation**: Comprehensive documentation for package building and release process
### Changed
- **Release Script Architecture**: Modernized release workflow with setuptools-scm integration
- **Makefile Release Targets**: Updated release targets to support automated version management
- **Package Building Process**: Streamlined package creation with enhanced build targets
### Removed
- **Legacy Release Scripts**: Removed obsolete release_simplified.py in favor of unified release.py
## [0.7.0] - 2025-11-08
### Added
@@ -158,4 +189,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Build System**: Enhanced build targets with venv Python and PYTHONPATH support
- **Target Naming**: Renamed workspace targets to TDD Workspace with tdd- prefix
xxx
[Unreleased]: https://github.com/worsch/markitect/compare/v0.8.0...HEAD
[0.8.0]: https://github.com/worsch/markitect/compare/v0.7.0...v0.8.0
[0.7.0]: https://github.com/worsch/markitect/compare/v0.6.0...v0.7.0
[0.6.0]: https://github.com/worsch/markitect/compare/v0.5.0...v0.6.0
[0.5.0]: https://github.com/worsch/markitect/compare/v0.4.0...v0.5.0
[0.4.0]: https://github.com/worsch/markitect/compare/v0.3.0...v0.4.0
[0.3.0]: https://github.com/worsch/markitect/compare/v0.2.0...v0.3.0
[0.2.0]: https://github.com/worsch/markitect/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/worsch/markitect/releases/tag/v0.1.0

200
TODO.md
View File

@@ -12,206 +12,10 @@ The structure organizes **future tasks** by their impact, just as a changelog or
This section is for tasks currently being discussed with or worked on by the coding assistant. These are the ephemeral, flow-of-thought tasks.
**🔧 FIX BROKEN ISSUE-FACADE CAPABILITY (2025-11-10) - COMPLETED ✅**:
Successfully fixed the issue-facade capability by restructuring the package organization to match pyproject.toml expectations.
**📋 Actions Completed**:
- [x] Diagnosed package structure mismatch (pyproject.toml expected `issue_tracker` package but code was in flat structure)
- [x] Fixed module import issue by creating `issue_tracker/` package and moving `cli/`, `core/`, `backends/` into it
- [x] Verified capability functionality is restored (CLI commands working, package installs correctly)
- [x] Tested with actual issue access (CLI prompts for backend configuration as expected)
**🎨 IMPLEMENT DOCUMENT STYLING (Issue #166) - COMPLETED ✅**:
Successfully implemented Substack-style document theme for enhanced long-form reading experience.
**📋 Features Implemented**:
- [x] Substack document theme in layered theme system
- [x] Spectral serif font for body text (optimized for long-form reading)
- [x] Lora sans-serif font for headings (clean typography hierarchy)
- [x] Warm cream background (#FAF9F1) for reduced eye strain
- [x] Bronze accent color (#b08d57) for visual hierarchy
- [x] 680px max-width for optimal reading line length
- [x] 1.6 line-height for comfortable reading
- [x] Complete TDD test coverage (6/6 tests passing)
- [x] CLI integration: `markitect md-render --theme substack`
- [x] Layered theme compatibility with existing mode/UI themes
**Implementation Details**:
- Added to `LAYERED_THEMES` as document-scope theme
- Extended CSS generation to support `heading_font_family` property
- Maintained backward compatibility with `TEMPLATE_STYLES`
- Full integration with existing theme system architecture
**🧪 TESTDRIVE-JSUI CAPABILITY EXTRACTION (2025-11-09) - COMPLETED ✅**:
Successfully extracted JavaScript UI framework functionality into a dedicated capability with complete automated test integration. The capability now provides 68 JavaScript tests + 11 Python integration tests for comprehensive testing coverage.
**🏗️ MAJOR ARCHITECTURE REFACTORING (2025-11-03) - COMPLETED ✅**: Successfully completed comprehensive JavaScript refactoring using Test-Driven Development methodology.
**PROBLEMS SOLVED**:
1.**Monolithic Architecture**: Extracted 5,188-line `editor.js` into 4 modular components
2.**Server-Side Debug Generation**: Implemented pure client-side DebugPanel component
3.**Architectural Boundary Violations**: Clean separation with no Python code modifications
4.**Tight Coupling**: All components independently testable with event-driven communication
5.**Generic Editor Compromise**: Debug system now purely client-side and component-based
**SOLUTION IMPLEMENTED**: Modular JavaScript Architecture with complete component separation and TDD validation.
**📊 PREVIOUS STATUS (2025-11-02)**: Systematic JavaScript functionality recovery using TDD methodology had made excellent progress. **5 major features** were successfully implemented and tested:
1. **Advanced EditState Management** ✅ - Implemented enum-based state tracking with pending changes preservation
2. **Keyboard Shortcuts** ✅ - Added Ctrl+Enter (accept) and Escape (cancel) functionality
3. **Section Splitting** ✅ - Restored dynamic heading detection with automatic section reorganization
4. **Real-time Status Tracking** ✅ - Implemented periodic updates with visual status panel (2-second intervals)
5. **Intelligent Filename Generation** ✅ - Added 4-method fallback system (options→title→URL→heading→timestamp)
All implementations include comprehensive TDD test suites and are fully integrated into the existing codebase. The recovery approach has proven highly effective for restoring sophisticated lost functionality.
## 🏗️ JAVASCRIPT ARCHITECTURE REFACTORING - COMPLETED ✅
### **Phase 1: Preparation & Backup (CRITICAL) - ✅ COMPLETED**
* ✅ Updated TODO.md with comprehensive refactoring plan
* ✅ Created modular directory structure `markitect/static/js/`
* ✅ Set up component template files with proper exports/imports
* ✅ Implemented TDD test framework for safe refactoring
### **Phase 2: Core System Extraction (HIGH) - ✅ COMPLETED**
* ✅ Extracted SectionManager to `core/section-manager.js` (490 lines)
* ✅ Integrated EventSystem into SectionManager with pub/sub pattern
* ✅ Created comprehensive section state management with EditState enum
### **Phase 3: Component Separation (HIGH) - ✅ COMPLETED**
* ✅ Document Controls → `components/document-controls.js` (200 lines)
* ✅ DOMRenderer (includes status functionality) → `components/dom-renderer.js` (540 lines)
* ✅ Debug Panel → `components/debug-panel.js` (150 lines, pure client-side)
* ✅ Floating Menu → integrated into DOMRenderer component
* ✅ Text/Image Editors → integrated into DOMRenderer component
### **Phase 4: Testing Infrastructure (MEDIUM) - ✅ COMPLETED**
* ✅ Standalone TDD test runner (`RefactorTestRunner`) that doesn't require md-render
* ✅ Component unit tests for all individual functionality
* ✅ Integration tests for component interaction
* ✅ Full system integration tests for complete workflow validation
### **Phase 5: Integration & Cleanup (MEDIUM) - ✅ COMPLETED**
* ✅ All components work together with preserved functionality
* ✅ Monolithic editor.js functionality fully distributed
* ✅ Python code completely unchanged - zero md-render modifications
* ✅ All functionality validated through comprehensive test suite (31 tests passing)
### **Directory Structure Implemented:**
```
markitect/static/js/
├── core/
│ └── section-manager.js # ✅ Section state management with EventSystem (490 lines)
├── components/
│ ├── document-controls.js # ✅ Document controls panel (200 lines)
│ ├── dom-renderer.js # ✅ DOM rendering, FloatingMenu, editors (540 lines)
│ └── debug-panel.js # ✅ Debug panel (150 lines, pure client-side)
└── tests/
├── refactor-test-runner.js # ✅ TDD test framework
├── test-component-integration.js # ✅ Component integration tests
├── test-full-integration.js # ✅ Full system tests
├── test-section-manager-extraction.js # ✅ SectionManager tests
├── test-extracted-section-manager.js # ✅ SectionManager TDD tests
├── test-domrenderer-extraction.js # ✅ DOMRenderer extraction tests
├── test-extracted-domrenderer.js # ✅ DOMRenderer TDD tests
├── test-debugpanel-extraction.js # ✅ DebugPanel extraction tests
├── test-debugpanel-integration.js # ✅ DebugPanel integration tests
└── test-documentcontrols-extraction.js # ✅ DocumentControls tests
```
### **REFACTORING RESULTS SUMMARY:**
- **Lines Extracted**: 1,380 lines from monolithic 5,188-line editor.js
- **Components Created**: 4 modular, independently testable components
- **Tests Created**: 11 comprehensive test files with 31 passing tests
- **Architecture**: Event-driven, pub/sub communication between components
- **Functionality**: 100% preserved with zero regression
- **Performance**: Improved modularity enables better maintainability and testing
- **Python Code**: Zero modifications - clean architectural separation achieved
### **PREVIOUS COMPLETED FEATURES (Now successfully refactored):**
* **Successfully Refactored:**
* ✅ Advanced state management with EditState enum and pending changes (CRITICAL) - REFACTORED INTO SectionManager
* ✅ Keyboard shortcuts (Ctrl+Enter accept, Escape cancel) (CRITICAL) - REFACTORED INTO DOMRenderer
* ✅ Section splitting functionality for dynamic heading detection (HIGH) - REFACTORED INTO SectionManager
* ✅ Real-time status tracking with periodic updates (HIGH) - REFACTORED INTO DocumentControls
* ✅ Intelligent save filename generation with 4-method fallback (MEDIUM) - PRESERVED IN MONOLITH
* ✅ Professional message system with color-coded positioning (MEDIUM) - REFACTORED INTO DebugPanel
* ✅ Multiple concurrent editing sessions support (MEDIUM) - REFACTORED INTO DOMRenderer
* ✅ Enhanced DOM event system with 6 event types (LOW) - REFACTORED INTO DOMRenderer
* ✅ Automatic section type detection (heading, code, list, etc) (LOW) - REFACTORED INTO SectionManager
* ✅ Sophisticated section ID generation with hash-based algorithm (LOW) - REFACTORED INTO SectionManager
* **Successfully Implemented:**
* ✅ Comprehensive status reporting dialog with detailed stats (HIGH) - IMPLEMENTED IN DocumentControls
* ✅ Floating global control panel with professional styling (MEDIUM) - IMPLEMENTED IN DocumentControls
* ✅ Enhanced setupSectionElement with comprehensive styling (LOW) - IMPLEMENTED IN DOMRenderer
* **Core Methods Successfully Refactored:**
* ✅ stopEditing method with state preservation (CRITICAL) - REFACTORED INTO SectionManager
* ✅ getAllSections method for section collection management (MEDIUM) - REFACTORED INTO SectionManager
* ✅ hasChanges detection for unsaved modifications (HIGH) - REFACTORED INTO SectionManager
* ✅ updateGlobalStatus method with 2-second interval updates (MEDIUM) - REFACTORED INTO DocumentControls
* ✅ handleSectionSplit for dynamic section reorganization (LOW) - REFACTORED INTO SectionManager
* ✅ checkForSectionSplits automatic heading detection (LOW) - REFACTORED INTO SectionManager
* **To Remove:**
* None currently identified
*No active tasks at this time.*
***
## Completed Tasks
**JavaScript Architecture Refactoring - COMPLETED ✅ (2025-11-03)**:
- ✅ Successfully extracted monolithic 5,188-line editor.js into 4 modular components using TDD methodology
- ✅ Created SectionManager component (490 lines) handling section state management and event system
- ✅ Created DOMRenderer component (540 lines) handling DOM interactions, rendering, and editing workflows
- ✅ Created DebugPanel component (150 lines) providing pure client-side debug message management
- ✅ Created DocumentControls component (200 lines) managing floating control panel and document actions
- ✅ Implemented comprehensive TDD test framework with 11 test files and 31 passing tests
- ✅ Achieved 100% functionality preservation with zero regression through rigorous testing
- ✅ Established event-driven architecture with pub/sub communication between components
- ✅ Maintained complete separation from Python code - zero md-render modifications required
- ✅ Created modular directory structure enabling independent component development and testing
**Architecture Improvements Achieved**:
- Clean separation of concerns with single-responsibility components
- Event-driven communication reducing tight coupling
- Independent component testing enabling confident refactoring
- Scalable structure supporting future feature development
- Client-side debug system eliminating server-side debug generation issues
- Modular design allowing selective component updates without affecting others
**Asset Shipping for md-render - COMPLETED ✅**:
- ✅ Implemented automatic asset copying when rendering markdown to different output directories
- ✅ Added asset discovery functionality parsing markdown for image/link references
- ✅ Implemented timestamp-based asset copying (only copy if source newer than destination)
- ✅ Added `--ship-assets` and `--no-ship-assets` CLI flags for explicit control
- ✅ Added `MARKITECT_OUTPUT_DIR` environment variable support for default output directory
- ✅ Smart defaults: assets ship automatically when output is directory, disabled for specific files
- ✅ Preserved relative path structure in output directory maintaining markdown link compatibility
- ✅ Graceful handling of missing assets with warning messages
- ✅ Full backward compatibility with existing md-render workflows
- ✅ Comprehensive TDD test suite covering all functionality and edge cases
**Feature Capabilities**:
- Environment variable priority: CLI `--output` > `MARKITECT_OUTPUT_DIR` > input file directory
- Automatic asset discovery from standard markdown syntax: `![alt](path)` and `[text](path)`
- Timestamp-based incremental copying prevents unnecessary file operations
- Directory structure preservation maintains working relative links in output HTML
- Support for images, documents, and other asset types referenced in markdown
**CHANGELOG.md Enhancement - COMPLETED ✅**:
- ✅ Added missing version entries for 0.1.0, 0.2.0, and 0.3.0
- ✅ Added standard Keep a Changelog header with proper format
- ✅ Included Unreleased section
- ✅ Research completed for all historical versions using git log analysis
- ✅ All entries follow Keep a Changelog categories (Added, Changed, Fixed)
- ✅ Chronological order maintained with latest versions first
- ✅ Appropriate release dates included based on git commit timestamps
**Version Details Added**:
- v0.1.0 (2025-10-15): Development infrastructure, TDD workspace, issue management
- v0.2.0 (2025-10-20): Advanced Markdown Engine with GraphQL, search, plugins
- v0.3.0 (2025-10-25): Architectural improvements with kaizen-agentic integration
*Recent completed tasks have been documented in CHANGELOG.md following Keep a Changelog format.*

View File

@@ -0,0 +1,268 @@
# Markitect Workspace and Database Architecture
This document explains Markitect's workspace concept and the two distinct database systems used by the application.
## Workspace Concept
Markitect uses a **workspace-based architecture** where each directory or repository can have its own configuration and local data storage. This allows for flexible, per-project customization while maintaining a global user configuration.
### Workspace Structure
When you initialize Markitect in a directory, it creates the following structure:
```
project-directory/
├── .markitect.yml # Workspace configuration
├── .markitect_workspace/ # Local workspace data
├── .ast_cache/ # AST parsing cache
├── assets/ # Asset storage directory
│ ├── assets.db # Asset management database
│ └── [asset files] # Stored images, files, etc.
└── tests/ # Test files directory
```
### Configuration Files
Markitect searches for configuration in this order:
1. `.markitect.yml` (current directory)
2. `.markitect.yaml` (current directory)
3. `.markitect.json` (current directory)
4. `markitect.config.yml` (current directory)
5. `markitect.config.yaml` (current directory)
6. `markitect.config.json` (current directory)
7. `~/.markitect/config.yml` (user home directory)
8. Environment variables (`MARKITECT_*`)
9. Built-in defaults
## Database Architecture
Markitect uses two distinct SQLite databases for different purposes:
### 1. Main Application Database (`markitect.db`)
**Location**: `~/.markitect/markitect.db` (user home directory)
**Purpose**: Global user-level application data and configuration
**Scope**: User-wide, shared across all workspaces
**Contents**:
- User preferences and settings
- Application state information
- Global configuration data
- Cross-workspace data that needs persistence
**Configuration**: Set via `MARKITECT_DATABASE_PATH` environment variable or `database_path` in configuration
### 2. Asset Management Database (`assets.db`)
**Location**: `assets/assets.db` (within workspace asset storage directory)
**Purpose**: Asset management and tracking for the current workspace
**Scope**: Workspace-specific, local to each directory/repository
**Contents**:
- Asset metadata (filename, size, MIME type, timestamps)
- File content hashes for deduplication
- Asset usage statistics and tracking
- Processing logs and analytics
- Asset relationships and dependencies
**Schema** (key tables):
```sql
-- Basic asset metadata
asset_metadata (
content_hash TEXT PRIMARY KEY,
filename TEXT NOT NULL,
size_bytes INTEGER NOT NULL,
mime_type TEXT,
created_at TIMESTAMP,
updated_at TIMESTAMP
)
-- Usage tracking
asset_usage_stats (
content_hash TEXT,
usage_count INTEGER,
last_used TIMESTAMP,
documents_using TEXT -- JSON array of document paths
)
-- Performance and analytics tables
-- (Additional tables for caching, indexing, and optimization)
```
## Why Two Databases?
This separation serves several important purposes:
### Data Isolation
- **Global data** (user preferences) stays in the user profile
- **Workspace data** (asset files, metadata) stays with the project
### Version Control Considerations
- `markitect.db` is never committed to version control
- `assets.db` is excluded via `.gitignore` (local workspace data)
- Asset files themselves can be optionally committed based on project needs
### Performance Optimization
- Asset database operations are localized to relevant files
- Global database isn't impacted by large asset collections
- Each workspace can optimize its asset database independently
### Portability and Collaboration
- Workspaces can be moved/copied without affecting global configuration
- Teams can share asset storage strategies without sharing personal settings
- Different projects can have different asset management policies
## Configuration Management
### Workspace Initialization
To initialize a new workspace:
```bash
markitect config-init
```
This creates:
1. `.markitect.yml` configuration file
2. Required directories (`.markitect_workspace`, `.ast_cache`, `tests`)
3. Asset storage structure
### Configuration Commands
```bash
# View current configuration
markitect config-show
# Set workspace-specific values
markitect config-set repo_name "my-project"
markitect config-set assets.storage_path "./assets"
# View configuration help
markitect config-help
```
### Environment Variables
Override configuration with environment variables:
```bash
export MARKITECT_GITEA_URL="http://localhost:3000"
export MARKITECT_WORKSPACE_DIR=".custom_workspace"
export MARKITECT_DATABASE_PATH="/custom/path/markitect.db"
```
## Asset Management Integration
The asset management system coordinates between the asset database and file storage:
```python
from markitect.assets import AssetManager
# Initialize with workspace-specific configuration
asset_manager = AssetManager()
# Assets are stored in workspace, tracked in assets.db
asset = asset_manager.add_asset("image.png")
```
### Asset Storage Workflow
1. **File Processing**: Asset files are processed and stored in `assets/` directory
2. **Database Recording**: Metadata recorded in `assets.db`
3. **Deduplication**: Content hashes prevent duplicate storage
4. **Usage Tracking**: Document usage recorded for analytics
5. **Cleanup**: Unused assets can be identified and removed
## Best Practices
### For Development
- Initialize workspace in each project directory
- Commit `.markitect.yml` to version control
- Add `assets.db` and workspace directories to `.gitignore`
- Use relative paths in workspace configuration
### For Collaboration
- Share workspace configuration (`.markitect.yml`)
- Document asset storage strategy for the team
- Establish conventions for asset organization
- Consider asset file version control policies
### for Production
- Backup both databases separately
- Monitor asset database growth in large projects
- Use environment variables for deployment-specific settings
- Implement asset cleanup strategies for long-running projects
## Migration and Compatibility
### Legacy Support
The system maintains backward compatibility:
- Old configuration patterns still work
- Automatic migration of legacy settings
- Graceful fallbacks for missing configuration
### Database Migration
Asset databases support schema migrations:
- Automatic schema updates on version changes
- Backward compatibility preservation
- Safe migration with rollback capability
## Troubleshooting
### Common Issues
**Database Connection Errors**:
- Check file permissions on database directories
- Verify disk space availability
- Ensure SQLite3 is available
**Configuration Not Found**:
- Verify `.markitect.yml` exists and is valid YAML
- Check environment variable names and values
- Run `markitect config-show` to see current configuration
**Asset Storage Issues**:
- Confirm asset directory permissions
- Check available disk space
- Verify `assets.db` is not corrupted
### Recovery Procedures
**Corrupted Asset Database**:
```bash
# Backup and recreate
mv assets/assets.db assets/assets.db.backup
# Restart Markitect to recreate schema
markitect config-show
```
**Missing Configuration**:
```bash
# Reinitialize workspace
markitect config-init
```
## Technical Details
### Database Connections
- Uses SQLite3 with connection pooling for performance
- Automatic connection management and cleanup
- Thread-safe operations for concurrent access
### File System Integration
- Path resolution relative to workspace root
- Cross-platform path handling (Windows, macOS, Linux)
- Symlink and junction support where available
### Security Considerations
- Database files have restricted permissions
- No sensitive data stored in asset database
- Configuration masking for sensitive values in display
---
*This documentation reflects the current architecture as of November 2025. For implementation details, see the source code in `markitect/config_manager.py` and `markitect/assets/`.*