feat: Issue #38 Phase 1 - Command Restructuring with db-data Implementation

## Command Restructuring Implementation
- Add new db-data command as replacement for metadata command
- Implement complete functionality matching original metadata command
- Support all output formats (table, json, yaml, simple)
- Follow established db- prefix pattern from Issue #39

## Backward Compatibility & Migration
- Maintain existing metadata command with full functionality
- Add deprecation warnings using legacy compatibility system
- Update help documentation with migration guidance
- Provide clear examples showing new db-data usage

## CLI Enhancements
- Consistent error handling across both commands
- Comprehensive help documentation for smooth migration
- Integration with existing legacy compatibility framework
- Support for all established output format options

## Testing & Validation
- Create comprehensive test suite for command restructuring
- Verify backward compatibility with existing scripts
- Test deprecation warning functionality
- Validate format consistency between old and new commands

## GAMEPLAN Documentation
- Create detailed implementation roadmap for all 5 phases
- Document technical architecture for component separation
- Establish testing strategy for comprehensive CLI enhancement
- Plan future phases for content, frontmatter, and tailmatter commands

Phase 1 Complete: ✅ Command restructuring with full backward compatibility
Next: Phase 2 - Content commands (content-stats, content-get)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

NEXT.md

@@ -1,80 +1,45 @@
 # MarkiTect Development Roadmap - Next Steps After Recent Milestone Achievements
 
-## ⚠️ **FAILING TESTS TO RECREATE WITHOUT LEGACY INTERFACES**
-
-**Priority**: These tests currently fail due to test state pollution and should be recreated to use only the new `db-` prefixed commands without any legacy interface dependencies:
-
-```bash
-# Output formatting tests that need new implementation with db- commands:
-tests/test_l4_service_output_formatting.py::TestOutputFormatting::test_json_format_output
-tests/test_l4_service_output_formatting.py::TestOutputFormatting::test_yaml_format_output
-tests/test_l4_service_output_formatting.py::TestOutputFormatting::test_empty_result_formatting
-tests/test_l4_service_output_formatting.py::TestSchemaFormatting::test_schema_json_format
-
-# Database query tests that need new implementation with db- commands:
-tests/test_l5_infrastructure_database_queries.py::TestQueryCommand::test_query_command_supports_output_formats
-```
-
-**Action Required**:
-- Create new test files: `test_db_output_formatting.py` and `test_db_infrastructure_queries.py`
-- Test the new `db-query` and `db-schema` commands exclusively
-- Remove dependencies on legacy `query` and `schema` commands
-- Ensure clean test execution without state pollution
-
----
-
-## 🎯 **CURRENT STATUS: Database CLI Reorganization Complete - Template Generation Ready**
+## 🎯 **CURRENT STATUS: Core Workflow Complete - Enhanced User Experience Ready**
 
 ### 📊 **Recently Completed Achievements**
 - ✅ **Issue #3**: Schema Management with Enhanced Format Control - COMPLETED 🎉
 - ✅ **Issue #5**: Schema Generation Foundation for arc42 Architecture Documentation - COMPLETED
+- ✅ **Issue #6**: Generate Markdown Stub from Schema - COMPLETED 🎉
 - ✅ **Issue #7**: Schema Validation - COMPLETED
 - ✅ **Issue #8**: Detailed Validation Error Reporting and CLI Enhancements - COMPLETED
 - ✅ **Issue #4**: Retrieve All Stored Files - COMPLETED
 - ✅ **Issue #18**: Configuration and Environment Management CLI - COMPLETED
 - ✅ **Issue #39**: Prefix Database Access Commands with 'db' - COMPLETED 🎉
 - ✅ **Issue #40**: Associated Files Management - COMPLETED 🎉
-- ✅ **Revolutionary Test Architecture**: 7-Layer Organization with 466 tests - COMPLETED
+- ✅ **Revolutionary Test Architecture**: 7-Layer Organization with 474 tests - COMPLETED
 - ✅ **Legacy Compatibility System**: Comprehensive versioned interface management - COMPLETED
+- ✅ **Clean Test Suite**: Removed legacy test pollution with enhanced db- command coverage - COMPLETED
 
 ### 🚀 **Current Capabilities Achieved**
 - **Complete Schema-Driven Architecture**: Generate, validate, and get detailed error reports for markdown schemas
+- **Template Generation Workflow**: Create markdown stubs from schemas with intelligent placeholder content
 - **Reorganized CLI Interface**: Clean `db-` prefixed commands with full configuration, cache, and database management
 - **Associated Files Management**: Coordinated handling of markdown-schema file pairs with auto-discovery
 - **Legacy Compatibility System**: Comprehensive versioned interface management with intelligent agent
-- **Production-Ready Foundation**: 466 tests across 7 architectural layers with robust legacy support
+- **Production-Ready Foundation**: 474 tests across 7 architectural layers with clean test execution
 - **High-Performance Processing**: AST caching with 60-85% speedup
 - **Comprehensive Error Handling**: User-friendly validation error reporting with actionable recommendations
 
-## 🎯 **STRATEGIC NEXT STEPS: Template Generation & Document Workflows**
+## 🎯 **STRATEGIC NEXT STEPS: Enhanced User Experience & Advanced Features**
 
-### **Phase 1: Template Generation (IMMEDIATE PRIORITY)**
+### **Phase 1: Enhanced CLI User Experience (IMMEDIATE PRIORITY)**
 
-#### **🎯 Issue #6: Generate Markdown Stub from Schema** ⭐ **HIGHEST PRIORITY**
-**Strategic Value**: Complete the schema-to-document workflow enabling practical arc42 documentation generation
-**Foundation**: Leverage completed schema generation and validation infrastructure
-**Deliverable**: CLI command to create markdown templates from existing schemas
-**Impact**: Unlocks practical template-based document creation workflow
+#### **🎯 Issue #38: Access Metadata, Frontmatter, Content Separately in CLI** ⭐ **HIGHEST PRIORITY**
+**Strategic Value**: Enhance CLI usability and data access granularity for improved developer experience
+**Foundation**: Build on existing CLI and database infrastructure with complete schema workflow
+**Deliverable**: Separate CLI commands for accessing metadata, frontmatter, and content independently
+**Impact**: Enables fine-grained access to document components for automation and tooling
+**Timeline**: 1 week
 
-**Next Command**: `make tdd-start NUM=6` - Begin template generation implementation
+**Next Command**: `make tdd-start NUM=38` - Begin enhanced CLI component access
 
-**Why Issue #6 First:**
-- **Natural Progression**: Completes the bidirectional schema ↔ markdown workflow
-- **High User Value**: Enables practical template-based document creation
-- **Perfect Foundation**: Schema generation and validation are already complete
-- **Arc42 Pathway**: Essential for architecture documentation template generation
-
-#### **Expected Timeline**: 1-2 weeks for complete implementation
-
-### **Phase 2: Enhanced Document Operations (Following Priority)**
-
-#### **🎯 Issue #38: Access Metadata, Frontmatter, Content Separately in CLI**
-**Strategic Value**: Enhance CLI usability and data access granularity
-**Foundation**: Build on existing CLI and database infrastructure
-**Deliverable**: Separate CLI commands for accessing different document components
-**Timeline**: 1 week after Issue #6
-
-### **Phase 3: Advanced Features & User Experience**
+### **Phase 2: Enhanced User Experience Features**
 
 #### **🎯 Issue #37: Emoji Flag and Preferences**
 **Strategic Value**: Enhanced user experience with engaging output options
@@ -84,28 +49,30 @@ tests/test_l5_infrastructure_database_queries.py::TestQueryCommand::test_query_c
 **Strategic Value**: User onboarding and feature discoverability
 **Timeline**: 1 week
 
+### **Phase 3: Advanced Processing Capabilities**
+
 #### **🎯 Issue #17: Batch Processing and Recursive Operations**
 **Strategic Value**: Enable large-scale document processing workflows
 **Timeline**: 2 weeks
 
-## 📋 **Issue Priority Matrix - Updated After Recent Completions**
+## 📋 **Issue Priority Matrix - Updated After Template Generation Completion**
 
 ### **🔥 CRITICAL PATH (Start Immediately)**
-1. **Issue #6**: Generate Markdown Stub from Schema ⭐ **START NOW**
+1. **Issue #38**: Access Metadata, Frontmatter, Content Separately in CLI ⭐ **START NOW**
 
-### **🎯 HIGH PRIORITY (User Experience & Workflow)**
-2. **Issue #38**: Access Metadata, Frontmatter, Content Separately in CLI
+### **🎯 HIGH PRIORITY (Enhanced User Experience)**
+2. **Issue #37**: Emoji Flag and Preferences
+3. **Issue #36**: MarkiTect Tutorial
 
-### **🚀 MEDIUM PRIORITY (Enhanced Features)**
-5. **Issue #37**: Emoji Flag and Preferences
-6. **Issue #36**: MarkiTect Tutorial
-7. **Issue #17**: Batch Processing and Recursive Operations
-8. **Issue #16**: Performance Validation CLI
+### **🚀 MEDIUM PRIORITY (Advanced Features)**
+4. **Issue #17**: Batch Processing and Recursive Operations
+5. **Issue #16**: Performance Validation CLI
+6. **Issue #18**: Configuration and Environment Management CLI (if enhancements needed)
 
 ### **🎯 FUTURE PLANNING (Advanced Architecture)**
-9. **Issue #9**: Expose GraphQL Read Interface
-10. **Issue #10**: Expose GraphQL Write Interface
-11. **Issue #19**: Plugin Architecture and Extensions System
+7. **Issue #9**: Expose GraphQL Read Interface
+8. **Issue #10**: Expose GraphQL Write Interface
+9. **Issue #19**: Plugin Architecture and Extensions System
 
 ### **⏸️ DEFERRED (Specialized Use Cases)**
 - **Issue #35**: Architectural Chaos Testing (advanced robustness testing)
@@ -114,35 +81,37 @@ tests/test_l5_infrastructure_database_queries.py::TestQueryCommand::test_query_c
 
 ## ⚡ **IMMEDIATE ACTION PLAN**
 
-### **NEXT DEVELOPMENT SESSION: Start Issue #6**
+### **NEXT DEVELOPMENT SESSION: Start Issue #38**
 ```bash
-make tdd-start NUM=6  # Begin markdown stub generation from schema
+make tdd-start NUM=38  # Begin enhanced CLI component access implementation
 ```
 
-**Why Issue #6 Is The Perfect Next Step:**
-- **Completes Core Workflow**: Schema generation → validation → template creation
-- **High Success Probability**: Strong foundation already exists
-- **Maximum User Value**: Enables practical template-based document workflows
-- **Arc42 Progress**: Directly supports architecture documentation use cases
+**Why Issue #38 Is The Perfect Next Step:**
+- **Natural Progression**: Builds on completed core workflow to enhance usability
+- **High User Value**: Enables fine-grained access to document components for automation
+- **Strong Foundation**: Complete schema workflow, database CLI, and test infrastructure exists
+- **Developer Experience**: Significantly improves CLI flexibility for advanced use cases
 
 ### **Development Context**
 - **Clean Workspace**: Working tree is clean, no pending changes
-- **Green Test State**: 461/466 tests passing across 7 architectural layers (5 legacy interface tests need recreation)
-- **Strong Foundation**: Schema generation, validation, error reporting, and database CLI reorganization complete
+- **Green Test State**: 474/474 tests passing across 7 architectural layers with clean execution
+- **Complete Core Workflow**: Schema generation → validation → template creation → database CLI reorganization
 - **CLI Maturity**: Comprehensive command-line interface with db- prefixed commands, configuration, and legacy management
 
 ## 🏆 **STRATEGIC ACHIEVEMENTS TO DATE**
 
-### **Schema-Driven Architecture Foundation**
-- ✅ **Schema Generation**: Extract document structure patterns from markdown files
-- ✅ **Schema Validation**: Validate markdown against defined schemas with detailed error reporting
-- ✅ **Error Reporting**: User-friendly validation errors with actionable recommendations
+### **Complete Schema-Driven Architecture Foundation**
+- ✅ **Schema Generation**: Extract document structure patterns from markdown files (Issue #5)
+- ✅ **Schema Validation**: Validate markdown against defined schemas with detailed error reporting (Issue #7)
+- ✅ **Template Generation**: Create markdown stubs from schemas with intelligent placeholders (Issue #6)
+- ✅ **Error Reporting**: User-friendly validation errors with actionable recommendations (Issue #8)
 - ✅ **Format Support**: JSON and YAML schema formats with CLI control
 
 ### **Production-Ready Infrastructure**
-- ✅ **Test Architecture**: 394 tests across 7 layers (Foundation → Infrastructure → Integration → Domain → Service → Application → Presentation)
-- ✅ **CLI Excellence**: Complete command-line interface with configuration, cache, and database management
-- ✅ **Performance**: High-speed AST processing with intelligent caching
+- ✅ **Test Architecture**: 474 tests across 7 layers with clean execution and no legacy pollution
+- ✅ **CLI Excellence**: Complete db- prefixed command interface with configuration, cache, and database management
+- ✅ **Legacy Compatibility**: Comprehensive versioned interface management with intelligent agent
+- ✅ **Performance**: High-speed AST processing with intelligent caching (60-85% speedup)
 - ✅ **Error Handling**: Comprehensive error handling with user-friendly messages
 
 ### **Development Methodology**

markitect/cli.py

@@ -766,16 +766,25 @@ def metadata(config, file_path, format):
     """
     Display file metadata and front matter.
 
+    DEPRECATED: Use 'db-data' instead. This command will be removed in a future version.
+
     Show detailed information about a specific file including its
     front matter, database metadata, and processing information.
 
     FILE_PATH: Name of the file to display metadata for
 
     Examples:
-        markitect metadata README.md
-        markitect metadata docs/guide.md --format json
-        markitect metadata config.md --format yaml
+        markitect metadata README.md  (deprecated - use: markitect db-data README.md)
+        markitect metadata docs/guide.md --format json  (deprecated - use: markitect db-data docs/guide.md --format json)
+        markitect metadata config.md --format yaml  (deprecated - use: markitect db-data config.md --format yaml)
     """
+    # Show deprecation warning (unless in legacy mode)
+    if not LegacyMode.should_suppress_warnings():
+        emit_deprecation_warning(
+            "The 'metadata' command is deprecated. Please use 'db-data' instead. "
+            "This command will be removed in a future version."
+        )
+
     try:
         if config['verbose']:
             click.echo(f"Retrieving metadata for: {file_path}", err=True)
@@ -2135,6 +2144,65 @@ def db_status(config, format, database):
         sys.exit(1)
 
 
+@cli.command('db-data')
+@click.argument('file_path', type=str)
+@click.option('--format', '-f', type=click.Choice(['table', 'json', 'yaml', 'simple']),
+              default=lambda: get_default_format(['table', 'json', 'yaml', 'simple']), help='Output format')
+@pass_config
+def db_data(config, file_path, format):
+    """
+    Display complete file data including metadata, frontmatter, and content.
+
+    Show comprehensive information about a specific file including its
+    front matter, database metadata, and processing information.
+    This is the new name for what was previously called 'metadata'.
+
+    FILE_PATH: Name of the file to display data for
+
+    Examples:
+        markitect db-data README.md
+        markitect db-data docs/guide.md --format json
+        markitect db-data config.md --format yaml
+    """
+    try:
+        if config['verbose']:
+            click.echo(f"Retrieving complete data for: {file_path}", err=True)
+
+        db_manager = config['db_manager']
+
+        # Get file information from database
+        file_info = db_manager.get_markdown_file(file_path)
+
+        if not file_info:
+            click.echo(f"File not found in database: {file_path}", err=True)
+            click.echo("Use 'markitect ingest' to process the file first.", err=True)
+            sys.exit(1)
+
+        # Parse front matter for better display
+        if file_info.get('front_matter'):
+            try:
+                if isinstance(file_info['front_matter'], str):
+                    file_info['front_matter'] = eval(file_info['front_matter'])
+            except (ValueError, TypeError, SyntaxError):
+                if config['verbose']:
+                    click.echo("Warning: Could not parse front matter", err=True)
+
+        # Format and display complete data
+        formatted_output = format_output(file_info, format)
+        click.echo(formatted_output)
+
+        if config['verbose']:
+            content_length = len(file_info.get('content', ''))
+            click.echo(f"Content length: {content_length} characters", err=True)
+
+    except Exception as e:
+        click.echo(f"Error retrieving file data: {e}", err=True)
+        if config['verbose']:
+            import traceback
+            click.echo(traceback.format_exc(), err=True)
+        sys.exit(1)
+
+
 def format_file_size(size_bytes):
     """Format file size in human-readable format."""
     if size_bytes < 1024: