docs: complete project documentation and task management cleanup
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
### Documentation Updates - Added comprehensive WORKSPACE_AND_DATABASES.md documentation explaining: - Markitect's workspace-based architecture concept - Database separation (markitect.db vs assets.db) and purposes - Configuration management and asset integration - Best practices for development, collaboration, and production ### Changelog Management - Updated CHANGELOG.md with complete release history coverage - Added missing v0.8.0 entry for setuptools-SCM integration and release automation - Added proper version comparison links for all releases - Documented all recent work in Unreleased section following Keep a Changelog format ### Task Management - Cleaned TODO.md file by removing all completed tasks - Reset to clean state referencing changelog for completed work - Maintained Keep a Todofile format for future development sessions This completes the documentation and task management improvements for the ChatGPT theme implementation, modular theme system, issue-facade bug fixes, and workspace architecture clarification work. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
41
CHANGELOG.md
41
CHANGELOG.md
@@ -7,6 +7,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **ChatGPT Document Theme**: New document theme with Inter font, 580px width, and #10a37f accent color with full CLI support (`markitect md-render --theme chatgpt`)
|
||||
- **Modular Theme System Architecture**: File-based theme loading with YAML configuration and dynamic theme discovery
|
||||
- **Theme Directory Structure**: Organized theme components (mode/, ui/, document/, branding/) for better maintainability
|
||||
- **Database Architecture Documentation**: Comprehensive WORKSPACE_AND_DATABASES.md documenting workspace concepts and database purposes
|
||||
|
||||
### Changed
|
||||
- **Theme Loading System**: Implemented dynamic theme discovery and loading with metadata preservation
|
||||
- **Test Suite Organization**: Removed obsolete configuration CLI tests (490 lines) for cleaner codebase
|
||||
|
||||
### Fixed
|
||||
- **Issue-facade Click Framework Bug**: Resolved Sentinel bug in list command that was causing CLI failures
|
||||
- **Issue-facade Version Command**: Fixed installation error preventing version command from working
|
||||
- **Test Isolation Issues**: Improved test isolation with proper mocking to prevent cross-test interference
|
||||
- **Theme Color Assertions**: Updated test assertions to work with new modular theme system
|
||||
|
||||
## [0.8.0] - 2025-11-08
|
||||
|
||||
### Added
|
||||
- **Setuptools-SCM Integration**: Automatic version management system replacing manual version tracking
|
||||
- **Gitea Package Publishing**: Complete CI/CD pipeline for automated package publishing to Gitea
|
||||
- **Enhanced Release Documentation**: Comprehensive documentation for package building and release process
|
||||
|
||||
### Changed
|
||||
- **Release Script Architecture**: Modernized release workflow with setuptools-scm integration
|
||||
- **Makefile Release Targets**: Updated release targets to support automated version management
|
||||
- **Package Building Process**: Streamlined package creation with enhanced build targets
|
||||
|
||||
### Removed
|
||||
- **Legacy Release Scripts**: Removed obsolete release_simplified.py in favor of unified release.py
|
||||
|
||||
## [0.7.0] - 2025-11-08
|
||||
|
||||
### Added
|
||||
@@ -158,4 +189,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Build System**: Enhanced build targets with venv Python and PYTHONPATH support
|
||||
- **Target Naming**: Renamed workspace targets to TDD Workspace with tdd- prefix
|
||||
|
||||
xxx
|
||||
[Unreleased]: https://github.com/worsch/markitect/compare/v0.8.0...HEAD
|
||||
[0.8.0]: https://github.com/worsch/markitect/compare/v0.7.0...v0.8.0
|
||||
[0.7.0]: https://github.com/worsch/markitect/compare/v0.6.0...v0.7.0
|
||||
[0.6.0]: https://github.com/worsch/markitect/compare/v0.5.0...v0.6.0
|
||||
[0.5.0]: https://github.com/worsch/markitect/compare/v0.4.0...v0.5.0
|
||||
[0.4.0]: https://github.com/worsch/markitect/compare/v0.3.0...v0.4.0
|
||||
[0.3.0]: https://github.com/worsch/markitect/compare/v0.2.0...v0.3.0
|
||||
[0.2.0]: https://github.com/worsch/markitect/compare/v0.1.0...v0.2.0
|
||||
[0.1.0]: https://github.com/worsch/markitect/releases/tag/v0.1.0
|
||||
|
||||
200
TODO.md
200
TODO.md
@@ -12,206 +12,10 @@ The structure organizes **future tasks** by their impact, just as a changelog or
|
||||
|
||||
This section is for tasks currently being discussed with or worked on by the coding assistant. These are the ephemeral, flow-of-thought tasks.
|
||||
|
||||
**🔧 FIX BROKEN ISSUE-FACADE CAPABILITY (2025-11-10) - COMPLETED ✅**:
|
||||
Successfully fixed the issue-facade capability by restructuring the package organization to match pyproject.toml expectations.
|
||||
|
||||
**📋 Actions Completed**:
|
||||
- [x] Diagnosed package structure mismatch (pyproject.toml expected `issue_tracker` package but code was in flat structure)
|
||||
- [x] Fixed module import issue by creating `issue_tracker/` package and moving `cli/`, `core/`, `backends/` into it
|
||||
- [x] Verified capability functionality is restored (CLI commands working, package installs correctly)
|
||||
- [x] Tested with actual issue access (CLI prompts for backend configuration as expected)
|
||||
|
||||
**🎨 IMPLEMENT DOCUMENT STYLING (Issue #166) - COMPLETED ✅**:
|
||||
Successfully implemented Substack-style document theme for enhanced long-form reading experience.
|
||||
|
||||
**📋 Features Implemented**:
|
||||
- [x] Substack document theme in layered theme system
|
||||
- [x] Spectral serif font for body text (optimized for long-form reading)
|
||||
- [x] Lora sans-serif font for headings (clean typography hierarchy)
|
||||
- [x] Warm cream background (#FAF9F1) for reduced eye strain
|
||||
- [x] Bronze accent color (#b08d57) for visual hierarchy
|
||||
- [x] 680px max-width for optimal reading line length
|
||||
- [x] 1.6 line-height for comfortable reading
|
||||
- [x] Complete TDD test coverage (6/6 tests passing)
|
||||
- [x] CLI integration: `markitect md-render --theme substack`
|
||||
- [x] Layered theme compatibility with existing mode/UI themes
|
||||
|
||||
**Implementation Details**:
|
||||
- Added to `LAYERED_THEMES` as document-scope theme
|
||||
- Extended CSS generation to support `heading_font_family` property
|
||||
- Maintained backward compatibility with `TEMPLATE_STYLES`
|
||||
- Full integration with existing theme system architecture
|
||||
|
||||
**🧪 TESTDRIVE-JSUI CAPABILITY EXTRACTION (2025-11-09) - COMPLETED ✅**:
|
||||
Successfully extracted JavaScript UI framework functionality into a dedicated capability with complete automated test integration. The capability now provides 68 JavaScript tests + 11 Python integration tests for comprehensive testing coverage.
|
||||
|
||||
**🏗️ MAJOR ARCHITECTURE REFACTORING (2025-11-03) - COMPLETED ✅**: Successfully completed comprehensive JavaScript refactoring using Test-Driven Development methodology.
|
||||
|
||||
**PROBLEMS SOLVED**:
|
||||
1. ✅ **Monolithic Architecture**: Extracted 5,188-line `editor.js` into 4 modular components
|
||||
2. ✅ **Server-Side Debug Generation**: Implemented pure client-side DebugPanel component
|
||||
3. ✅ **Architectural Boundary Violations**: Clean separation with no Python code modifications
|
||||
4. ✅ **Tight Coupling**: All components independently testable with event-driven communication
|
||||
5. ✅ **Generic Editor Compromise**: Debug system now purely client-side and component-based
|
||||
|
||||
**SOLUTION IMPLEMENTED**: Modular JavaScript Architecture with complete component separation and TDD validation.
|
||||
|
||||
**📊 PREVIOUS STATUS (2025-11-02)**: Systematic JavaScript functionality recovery using TDD methodology had made excellent progress. **5 major features** were successfully implemented and tested:
|
||||
|
||||
1. **Advanced EditState Management** ✅ - Implemented enum-based state tracking with pending changes preservation
|
||||
2. **Keyboard Shortcuts** ✅ - Added Ctrl+Enter (accept) and Escape (cancel) functionality
|
||||
3. **Section Splitting** ✅ - Restored dynamic heading detection with automatic section reorganization
|
||||
4. **Real-time Status Tracking** ✅ - Implemented periodic updates with visual status panel (2-second intervals)
|
||||
5. **Intelligent Filename Generation** ✅ - Added 4-method fallback system (options→title→URL→heading→timestamp)
|
||||
|
||||
All implementations include comprehensive TDD test suites and are fully integrated into the existing codebase. The recovery approach has proven highly effective for restoring sophisticated lost functionality.
|
||||
|
||||
## 🏗️ JAVASCRIPT ARCHITECTURE REFACTORING - COMPLETED ✅
|
||||
|
||||
### **Phase 1: Preparation & Backup (CRITICAL) - ✅ COMPLETED**
|
||||
* ✅ Updated TODO.md with comprehensive refactoring plan
|
||||
* ✅ Created modular directory structure `markitect/static/js/`
|
||||
* ✅ Set up component template files with proper exports/imports
|
||||
* ✅ Implemented TDD test framework for safe refactoring
|
||||
|
||||
### **Phase 2: Core System Extraction (HIGH) - ✅ COMPLETED**
|
||||
* ✅ Extracted SectionManager to `core/section-manager.js` (490 lines)
|
||||
* ✅ Integrated EventSystem into SectionManager with pub/sub pattern
|
||||
* ✅ Created comprehensive section state management with EditState enum
|
||||
|
||||
### **Phase 3: Component Separation (HIGH) - ✅ COMPLETED**
|
||||
* ✅ Document Controls → `components/document-controls.js` (200 lines)
|
||||
* ✅ DOMRenderer (includes status functionality) → `components/dom-renderer.js` (540 lines)
|
||||
* ✅ Debug Panel → `components/debug-panel.js` (150 lines, pure client-side)
|
||||
* ✅ Floating Menu → integrated into DOMRenderer component
|
||||
* ✅ Text/Image Editors → integrated into DOMRenderer component
|
||||
|
||||
### **Phase 4: Testing Infrastructure (MEDIUM) - ✅ COMPLETED**
|
||||
* ✅ Standalone TDD test runner (`RefactorTestRunner`) that doesn't require md-render
|
||||
* ✅ Component unit tests for all individual functionality
|
||||
* ✅ Integration tests for component interaction
|
||||
* ✅ Full system integration tests for complete workflow validation
|
||||
|
||||
### **Phase 5: Integration & Cleanup (MEDIUM) - ✅ COMPLETED**
|
||||
* ✅ All components work together with preserved functionality
|
||||
* ✅ Monolithic editor.js functionality fully distributed
|
||||
* ✅ Python code completely unchanged - zero md-render modifications
|
||||
* ✅ All functionality validated through comprehensive test suite (31 tests passing)
|
||||
|
||||
### **Directory Structure Implemented:**
|
||||
```
|
||||
markitect/static/js/
|
||||
├── core/
|
||||
│ └── section-manager.js # ✅ Section state management with EventSystem (490 lines)
|
||||
├── components/
|
||||
│ ├── document-controls.js # ✅ Document controls panel (200 lines)
|
||||
│ ├── dom-renderer.js # ✅ DOM rendering, FloatingMenu, editors (540 lines)
|
||||
│ └── debug-panel.js # ✅ Debug panel (150 lines, pure client-side)
|
||||
└── tests/
|
||||
├── refactor-test-runner.js # ✅ TDD test framework
|
||||
├── test-component-integration.js # ✅ Component integration tests
|
||||
├── test-full-integration.js # ✅ Full system tests
|
||||
├── test-section-manager-extraction.js # ✅ SectionManager tests
|
||||
├── test-extracted-section-manager.js # ✅ SectionManager TDD tests
|
||||
├── test-domrenderer-extraction.js # ✅ DOMRenderer extraction tests
|
||||
├── test-extracted-domrenderer.js # ✅ DOMRenderer TDD tests
|
||||
├── test-debugpanel-extraction.js # ✅ DebugPanel extraction tests
|
||||
├── test-debugpanel-integration.js # ✅ DebugPanel integration tests
|
||||
└── test-documentcontrols-extraction.js # ✅ DocumentControls tests
|
||||
```
|
||||
|
||||
### **REFACTORING RESULTS SUMMARY:**
|
||||
- **Lines Extracted**: 1,380 lines from monolithic 5,188-line editor.js
|
||||
- **Components Created**: 4 modular, independently testable components
|
||||
- **Tests Created**: 11 comprehensive test files with 31 passing tests
|
||||
- **Architecture**: Event-driven, pub/sub communication between components
|
||||
- **Functionality**: 100% preserved with zero regression
|
||||
- **Performance**: Improved modularity enables better maintainability and testing
|
||||
- **Python Code**: Zero modifications - clean architectural separation achieved
|
||||
|
||||
### **PREVIOUS COMPLETED FEATURES (Now successfully refactored):**
|
||||
* **Successfully Refactored:**
|
||||
* ✅ Advanced state management with EditState enum and pending changes (CRITICAL) - REFACTORED INTO SectionManager
|
||||
* ✅ Keyboard shortcuts (Ctrl+Enter accept, Escape cancel) (CRITICAL) - REFACTORED INTO DOMRenderer
|
||||
* ✅ Section splitting functionality for dynamic heading detection (HIGH) - REFACTORED INTO SectionManager
|
||||
* ✅ Real-time status tracking with periodic updates (HIGH) - REFACTORED INTO DocumentControls
|
||||
* ✅ Intelligent save filename generation with 4-method fallback (MEDIUM) - PRESERVED IN MONOLITH
|
||||
* ✅ Professional message system with color-coded positioning (MEDIUM) - REFACTORED INTO DebugPanel
|
||||
* ✅ Multiple concurrent editing sessions support (MEDIUM) - REFACTORED INTO DOMRenderer
|
||||
* ✅ Enhanced DOM event system with 6 event types (LOW) - REFACTORED INTO DOMRenderer
|
||||
* ✅ Automatic section type detection (heading, code, list, etc) (LOW) - REFACTORED INTO SectionManager
|
||||
* ✅ Sophisticated section ID generation with hash-based algorithm (LOW) - REFACTORED INTO SectionManager
|
||||
|
||||
* **Successfully Implemented:**
|
||||
* ✅ Comprehensive status reporting dialog with detailed stats (HIGH) - IMPLEMENTED IN DocumentControls
|
||||
* ✅ Floating global control panel with professional styling (MEDIUM) - IMPLEMENTED IN DocumentControls
|
||||
* ✅ Enhanced setupSectionElement with comprehensive styling (LOW) - IMPLEMENTED IN DOMRenderer
|
||||
|
||||
* **Core Methods Successfully Refactored:**
|
||||
* ✅ stopEditing method with state preservation (CRITICAL) - REFACTORED INTO SectionManager
|
||||
* ✅ getAllSections method for section collection management (MEDIUM) - REFACTORED INTO SectionManager
|
||||
* ✅ hasChanges detection for unsaved modifications (HIGH) - REFACTORED INTO SectionManager
|
||||
* ✅ updateGlobalStatus method with 2-second interval updates (MEDIUM) - REFACTORED INTO DocumentControls
|
||||
* ✅ handleSectionSplit for dynamic section reorganization (LOW) - REFACTORED INTO SectionManager
|
||||
* ✅ checkForSectionSplits automatic heading detection (LOW) - REFACTORED INTO SectionManager
|
||||
|
||||
* **To Remove:**
|
||||
* None currently identified
|
||||
|
||||
*No active tasks at this time.*
|
||||
|
||||
***
|
||||
|
||||
## Completed Tasks
|
||||
|
||||
**JavaScript Architecture Refactoring - COMPLETED ✅ (2025-11-03)**:
|
||||
- ✅ Successfully extracted monolithic 5,188-line editor.js into 4 modular components using TDD methodology
|
||||
- ✅ Created SectionManager component (490 lines) handling section state management and event system
|
||||
- ✅ Created DOMRenderer component (540 lines) handling DOM interactions, rendering, and editing workflows
|
||||
- ✅ Created DebugPanel component (150 lines) providing pure client-side debug message management
|
||||
- ✅ Created DocumentControls component (200 lines) managing floating control panel and document actions
|
||||
- ✅ Implemented comprehensive TDD test framework with 11 test files and 31 passing tests
|
||||
- ✅ Achieved 100% functionality preservation with zero regression through rigorous testing
|
||||
- ✅ Established event-driven architecture with pub/sub communication between components
|
||||
- ✅ Maintained complete separation from Python code - zero md-render modifications required
|
||||
- ✅ Created modular directory structure enabling independent component development and testing
|
||||
|
||||
**Architecture Improvements Achieved**:
|
||||
- Clean separation of concerns with single-responsibility components
|
||||
- Event-driven communication reducing tight coupling
|
||||
- Independent component testing enabling confident refactoring
|
||||
- Scalable structure supporting future feature development
|
||||
- Client-side debug system eliminating server-side debug generation issues
|
||||
- Modular design allowing selective component updates without affecting others
|
||||
|
||||
**Asset Shipping for md-render - COMPLETED ✅**:
|
||||
- ✅ Implemented automatic asset copying when rendering markdown to different output directories
|
||||
- ✅ Added asset discovery functionality parsing markdown for image/link references
|
||||
- ✅ Implemented timestamp-based asset copying (only copy if source newer than destination)
|
||||
- ✅ Added `--ship-assets` and `--no-ship-assets` CLI flags for explicit control
|
||||
- ✅ Added `MARKITECT_OUTPUT_DIR` environment variable support for default output directory
|
||||
- ✅ Smart defaults: assets ship automatically when output is directory, disabled for specific files
|
||||
- ✅ Preserved relative path structure in output directory maintaining markdown link compatibility
|
||||
- ✅ Graceful handling of missing assets with warning messages
|
||||
- ✅ Full backward compatibility with existing md-render workflows
|
||||
- ✅ Comprehensive TDD test suite covering all functionality and edge cases
|
||||
|
||||
**Feature Capabilities**:
|
||||
- Environment variable priority: CLI `--output` > `MARKITECT_OUTPUT_DIR` > input file directory
|
||||
- Automatic asset discovery from standard markdown syntax: `` and `[text](path)`
|
||||
- Timestamp-based incremental copying prevents unnecessary file operations
|
||||
- Directory structure preservation maintains working relative links in output HTML
|
||||
- Support for images, documents, and other asset types referenced in markdown
|
||||
|
||||
**CHANGELOG.md Enhancement - COMPLETED ✅**:
|
||||
- ✅ Added missing version entries for 0.1.0, 0.2.0, and 0.3.0
|
||||
- ✅ Added standard Keep a Changelog header with proper format
|
||||
- ✅ Included Unreleased section
|
||||
- ✅ Research completed for all historical versions using git log analysis
|
||||
- ✅ All entries follow Keep a Changelog categories (Added, Changed, Fixed)
|
||||
- ✅ Chronological order maintained with latest versions first
|
||||
- ✅ Appropriate release dates included based on git commit timestamps
|
||||
|
||||
**Version Details Added**:
|
||||
- v0.1.0 (2025-10-15): Development infrastructure, TDD workspace, issue management
|
||||
- v0.2.0 (2025-10-20): Advanced Markdown Engine with GraphQL, search, plugins
|
||||
- v0.3.0 (2025-10-25): Architectural improvements with kaizen-agentic integration
|
||||
*Recent completed tasks have been documented in CHANGELOG.md following Keep a Changelog format.*
|
||||
268
docs/WORKSPACE_AND_DATABASES.md
Normal file
268
docs/WORKSPACE_AND_DATABASES.md
Normal file
@@ -0,0 +1,268 @@
|
||||
# Markitect Workspace and Database Architecture
|
||||
|
||||
This document explains Markitect's workspace concept and the two distinct database systems used by the application.
|
||||
|
||||
## Workspace Concept
|
||||
|
||||
Markitect uses a **workspace-based architecture** where each directory or repository can have its own configuration and local data storage. This allows for flexible, per-project customization while maintaining a global user configuration.
|
||||
|
||||
### Workspace Structure
|
||||
|
||||
When you initialize Markitect in a directory, it creates the following structure:
|
||||
|
||||
```
|
||||
project-directory/
|
||||
├── .markitect.yml # Workspace configuration
|
||||
├── .markitect_workspace/ # Local workspace data
|
||||
├── .ast_cache/ # AST parsing cache
|
||||
├── assets/ # Asset storage directory
|
||||
│ ├── assets.db # Asset management database
|
||||
│ └── [asset files] # Stored images, files, etc.
|
||||
└── tests/ # Test files directory
|
||||
```
|
||||
|
||||
### Configuration Files
|
||||
|
||||
Markitect searches for configuration in this order:
|
||||
1. `.markitect.yml` (current directory)
|
||||
2. `.markitect.yaml` (current directory)
|
||||
3. `.markitect.json` (current directory)
|
||||
4. `markitect.config.yml` (current directory)
|
||||
5. `markitect.config.yaml` (current directory)
|
||||
6. `markitect.config.json` (current directory)
|
||||
7. `~/.markitect/config.yml` (user home directory)
|
||||
8. Environment variables (`MARKITECT_*`)
|
||||
9. Built-in defaults
|
||||
|
||||
## Database Architecture
|
||||
|
||||
Markitect uses two distinct SQLite databases for different purposes:
|
||||
|
||||
### 1. Main Application Database (`markitect.db`)
|
||||
|
||||
**Location**: `~/.markitect/markitect.db` (user home directory)
|
||||
|
||||
**Purpose**: Global user-level application data and configuration
|
||||
|
||||
**Scope**: User-wide, shared across all workspaces
|
||||
|
||||
**Contents**:
|
||||
- User preferences and settings
|
||||
- Application state information
|
||||
- Global configuration data
|
||||
- Cross-workspace data that needs persistence
|
||||
|
||||
**Configuration**: Set via `MARKITECT_DATABASE_PATH` environment variable or `database_path` in configuration
|
||||
|
||||
### 2. Asset Management Database (`assets.db`)
|
||||
|
||||
**Location**: `assets/assets.db` (within workspace asset storage directory)
|
||||
|
||||
**Purpose**: Asset management and tracking for the current workspace
|
||||
|
||||
**Scope**: Workspace-specific, local to each directory/repository
|
||||
|
||||
**Contents**:
|
||||
- Asset metadata (filename, size, MIME type, timestamps)
|
||||
- File content hashes for deduplication
|
||||
- Asset usage statistics and tracking
|
||||
- Processing logs and analytics
|
||||
- Asset relationships and dependencies
|
||||
|
||||
**Schema** (key tables):
|
||||
```sql
|
||||
-- Basic asset metadata
|
||||
asset_metadata (
|
||||
content_hash TEXT PRIMARY KEY,
|
||||
filename TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
mime_type TEXT,
|
||||
created_at TIMESTAMP,
|
||||
updated_at TIMESTAMP
|
||||
)
|
||||
|
||||
-- Usage tracking
|
||||
asset_usage_stats (
|
||||
content_hash TEXT,
|
||||
usage_count INTEGER,
|
||||
last_used TIMESTAMP,
|
||||
documents_using TEXT -- JSON array of document paths
|
||||
)
|
||||
|
||||
-- Performance and analytics tables
|
||||
-- (Additional tables for caching, indexing, and optimization)
|
||||
```
|
||||
|
||||
## Why Two Databases?
|
||||
|
||||
This separation serves several important purposes:
|
||||
|
||||
### Data Isolation
|
||||
- **Global data** (user preferences) stays in the user profile
|
||||
- **Workspace data** (asset files, metadata) stays with the project
|
||||
|
||||
### Version Control Considerations
|
||||
- `markitect.db` is never committed to version control
|
||||
- `assets.db` is excluded via `.gitignore` (local workspace data)
|
||||
- Asset files themselves can be optionally committed based on project needs
|
||||
|
||||
### Performance Optimization
|
||||
- Asset database operations are localized to relevant files
|
||||
- Global database isn't impacted by large asset collections
|
||||
- Each workspace can optimize its asset database independently
|
||||
|
||||
### Portability and Collaboration
|
||||
- Workspaces can be moved/copied without affecting global configuration
|
||||
- Teams can share asset storage strategies without sharing personal settings
|
||||
- Different projects can have different asset management policies
|
||||
|
||||
## Configuration Management
|
||||
|
||||
### Workspace Initialization
|
||||
|
||||
To initialize a new workspace:
|
||||
|
||||
```bash
|
||||
markitect config-init
|
||||
```
|
||||
|
||||
This creates:
|
||||
1. `.markitect.yml` configuration file
|
||||
2. Required directories (`.markitect_workspace`, `.ast_cache`, `tests`)
|
||||
3. Asset storage structure
|
||||
|
||||
### Configuration Commands
|
||||
|
||||
```bash
|
||||
# View current configuration
|
||||
markitect config-show
|
||||
|
||||
# Set workspace-specific values
|
||||
markitect config-set repo_name "my-project"
|
||||
markitect config-set assets.storage_path "./assets"
|
||||
|
||||
# View configuration help
|
||||
markitect config-help
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Override configuration with environment variables:
|
||||
|
||||
```bash
|
||||
export MARKITECT_GITEA_URL="http://localhost:3000"
|
||||
export MARKITECT_WORKSPACE_DIR=".custom_workspace"
|
||||
export MARKITECT_DATABASE_PATH="/custom/path/markitect.db"
|
||||
```
|
||||
|
||||
## Asset Management Integration
|
||||
|
||||
The asset management system coordinates between the asset database and file storage:
|
||||
|
||||
```python
|
||||
from markitect.assets import AssetManager
|
||||
|
||||
# Initialize with workspace-specific configuration
|
||||
asset_manager = AssetManager()
|
||||
|
||||
# Assets are stored in workspace, tracked in assets.db
|
||||
asset = asset_manager.add_asset("image.png")
|
||||
```
|
||||
|
||||
### Asset Storage Workflow
|
||||
|
||||
1. **File Processing**: Asset files are processed and stored in `assets/` directory
|
||||
2. **Database Recording**: Metadata recorded in `assets.db`
|
||||
3. **Deduplication**: Content hashes prevent duplicate storage
|
||||
4. **Usage Tracking**: Document usage recorded for analytics
|
||||
5. **Cleanup**: Unused assets can be identified and removed
|
||||
|
||||
## Best Practices
|
||||
|
||||
### For Development
|
||||
- Initialize workspace in each project directory
|
||||
- Commit `.markitect.yml` to version control
|
||||
- Add `assets.db` and workspace directories to `.gitignore`
|
||||
- Use relative paths in workspace configuration
|
||||
|
||||
### For Collaboration
|
||||
- Share workspace configuration (`.markitect.yml`)
|
||||
- Document asset storage strategy for the team
|
||||
- Establish conventions for asset organization
|
||||
- Consider asset file version control policies
|
||||
|
||||
### for Production
|
||||
- Backup both databases separately
|
||||
- Monitor asset database growth in large projects
|
||||
- Use environment variables for deployment-specific settings
|
||||
- Implement asset cleanup strategies for long-running projects
|
||||
|
||||
## Migration and Compatibility
|
||||
|
||||
### Legacy Support
|
||||
The system maintains backward compatibility:
|
||||
- Old configuration patterns still work
|
||||
- Automatic migration of legacy settings
|
||||
- Graceful fallbacks for missing configuration
|
||||
|
||||
### Database Migration
|
||||
Asset databases support schema migrations:
|
||||
- Automatic schema updates on version changes
|
||||
- Backward compatibility preservation
|
||||
- Safe migration with rollback capability
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Database Connection Errors**:
|
||||
- Check file permissions on database directories
|
||||
- Verify disk space availability
|
||||
- Ensure SQLite3 is available
|
||||
|
||||
**Configuration Not Found**:
|
||||
- Verify `.markitect.yml` exists and is valid YAML
|
||||
- Check environment variable names and values
|
||||
- Run `markitect config-show` to see current configuration
|
||||
|
||||
**Asset Storage Issues**:
|
||||
- Confirm asset directory permissions
|
||||
- Check available disk space
|
||||
- Verify `assets.db` is not corrupted
|
||||
|
||||
### Recovery Procedures
|
||||
|
||||
**Corrupted Asset Database**:
|
||||
```bash
|
||||
# Backup and recreate
|
||||
mv assets/assets.db assets/assets.db.backup
|
||||
# Restart Markitect to recreate schema
|
||||
markitect config-show
|
||||
```
|
||||
|
||||
**Missing Configuration**:
|
||||
```bash
|
||||
# Reinitialize workspace
|
||||
markitect config-init
|
||||
```
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Database Connections
|
||||
- Uses SQLite3 with connection pooling for performance
|
||||
- Automatic connection management and cleanup
|
||||
- Thread-safe operations for concurrent access
|
||||
|
||||
### File System Integration
|
||||
- Path resolution relative to workspace root
|
||||
- Cross-platform path handling (Windows, macOS, Linux)
|
||||
- Symlink and junction support where available
|
||||
|
||||
### Security Considerations
|
||||
- Database files have restricted permissions
|
||||
- No sensitive data stored in asset database
|
||||
- Configuration masking for sensitive values in display
|
||||
|
||||
---
|
||||
|
||||
*This documentation reflects the current architecture as of November 2025. For implementation details, see the source code in `markitect/config_manager.py` and `markitect/assets/`.*
|
||||
Reference in New Issue
Block a user