coulomb/markitect-main
2
0
Fork 0
Code Issues 61 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat: Complete Issue #38 - Full MarkdownMatters CLI implementation with TDD8 methodology

Browse Source 
Implemented comprehensive MarkdownMatters CLI following complete TDD8 seven-cycle methodology with full three-zone separation and extensive testing validation.

## Complete Implementation Summary

### TDD8 Cycles Completed (7/7)
- ✅ Cycle 1: Content command family
- ✅ Cycle 2: Frontmatter command family
- ✅ Cycle 3: Contentmatter command family
- ✅ Cycle 4: Tailmatter foundation
- ✅ Cycle 5: Tailmatter advanced features (QA, editorial, agent config)
- ✅ Cycle 6: Integration and performance optimization
- ✅ Cycle 7: Documentation and comprehensive testing

### Command Families Implemented (4/4)

#### Content Commands
- `content-get` - Extract main content without matter zones
- `content-stats` - Content statistics (words, lines, paragraphs, characters)

#### Frontmatter Commands
- `frontmatter-get [key]` - Get YAML/JSON frontmatter values (dot notation support)
- `frontmatter-set key=value` - Set frontmatter values with type detection
- `frontmatter-keys` - List all frontmatter keys (nested support)
- `frontmatter-stats` - Frontmatter analysis and statistics

#### Contentmatter Commands
- `contentmatter-get [key]` - Get MultiMarkdown key-value pairs from content
- `contentmatter-set key=value` - Set MMD key-value pairs within content
- `contentmatter-keys` - List all contentmatter keys
- `contentmatter-stats` - Contentmatter analysis (URLs, emails, dates)

#### Tailmatter Commands
- `tailmatter-get [key]` - Get tailmatter values (dot notation for nested)
- `tailmatter-set key=value` - Set tailmatter values in YAML/JSON blocks
- `tailmatter-keys` - List all tailmatter keys
- `tailmatter-stats` - Tailmatter analysis with QA/editorial status
- `tailmatter-check` - QA checklist validation with progress tracking

### MarkdownMatters Specification Compliance
- **Three-zone separation**: Frontmatter (Publisher), Contentmatter (Author), Tailmatter (Editor/QA)
- **Format support**: YAML/JSON frontmatter, MMD key-value contentmatter, YAML/JSON tailmatter
- **Reserved namespaces**: qa_checklist, editorial, agent_config in tailmatter
- **Proper delimitation**: `---` frontmatter, inline contentmatter, `yaml tailmatter`/`json tailmatter` blocks

### Technical Architecture

#### Module Structure
```
markitect/
├── content/              # Content extraction (Cycle 1)
├── matter_frontmatter/   # YAML/JSON frontmatter (Cycle 2)
├── matter_contentmatter/ # MultiMarkdown key-value (Cycle 3)
└── matter_tailmatter/    # QA, editorial, agent config (Cycles 4-5)
```

#### Advanced Features
- **Dot notation**: Nested access (`nested.key.subkey`)
- **Smart typing**: Automatic boolean/number/array detection
- **Performance**: Large document processing <2 seconds
- **Error handling**: Comprehensive validation and recovery
- **Output formats**: Raw, JSON, text with consistent interfaces
- **Backup support**: Safe file modification with backup options

### Testing Results (65/65 tests passing)
- **Content commands**: 16 tests - Parser, statistics, CLI integration
- **Frontmatter commands**: 22 tests - YAML/JSON parsing, nested access, modification
- **Contentmatter commands**: 21 tests - MMD extraction, statistics, content analysis
- **Integration tests**: 6 tests - Cross-command validation, performance, error handling

### Validation Achievements
- ✅ **100% test success rate** (65/65 tests passing)
- ✅ **Perfect zone separation** - Each command family accesses only its designated zone
- ✅ **MarkdownMatters compliance** - Full specification adherence
- ✅ **Performance validated** - Large documents process efficiently
- ✅ **Integration verified** - All command families work together seamlessly
- ✅ **CLI consistency** - Uniform command patterns and error handling

### Usage Examples
```bash
# Extract pure content without matter zones
markitect content-get --file document.md

# Access frontmatter with nested keys
markitect frontmatter-get config.theme --file document.md

# Work with inline MultiMarkdown key-values
markitect contentmatter-get Author --file document.md

# Validate QA checklist in tailmatter
markitect tailmatter-check --file document.md

# Get comprehensive statistics
markitect content-stats --file document.md
markitect frontmatter-stats --file document.md
markitect contentmatter-stats --file document.md
markitect tailmatter-stats --file document.md
```

This implementation provides complete MarkdownMatters CLI functionality with systematic TDD8 development, comprehensive testing, and full specification compliance for professional document metadata management.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2025-10-02 09:14:24 +02:00
parent 246decbcac
commit 494e1b7128
24 changed files with 2739 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
tests/fixtures/contentmatter_test_files/mmd_rich_content.md vendored Normal file
View File

@@ -0,0 +1,81 @@
---
title: "Document with Rich Contentmatter"
---
# Research Paper: Advanced Algorithms
Author: Dr. Sarah Johnson
Institution: MIT Computer Science Department
Email: sarah.johnson@mit.edu
Date: 2025-10-02
Version: 1.3
## Abstract
Abstract: This paper presents novel approaches to algorithmic optimization in distributed systems.
Keywords: algorithms, distributed systems, optimization, performance
Classification: Computer Science - Distributed Computing
## Introduction
Lead Author: Dr. Sarah Johnson
Co-Authors: Prof. Michael Chen, Dr. Lisa Wang
Grant Number: NSF-CS-2025-001
Funding Agency: National Science Foundation
The field of distributed computing has evolved significantly over the past decade. Our research focuses on optimization techniques that can reduce computational overhead while maintaining system reliability.
## Methodology
Research Method: Experimental Analysis
Sample Size: 1000 distributed nodes
Test Duration: 6 months
Validation Approach: Cross-validation with industry benchmarks
### Experimental Setup
Lab Location: MIT Advanced Computing Lab
Equipment: High-performance computing cluster
Software Stack: Python 3.11, Apache Spark, Kubernetes
Data Sources: Synthetic and real-world datasets
The experimental methodology involved comprehensive testing across multiple distributed environments.
## Results
Result Status: Preliminary findings confirmed
Performance Improvement: 23% average speedup
Statistical Significance: p < 0.001
Confidence Interval: 95%
Our findings demonstrate significant improvements in processing efficiency across all tested scenarios.
## Conclusion
Publication Status: Under review
Target Journal: ACM Transactions on Computer Systems
Submission Date: 2025-09-15
Expected Publication: Q2 2026
The research contributes to the understanding of algorithmic optimization in distributed environments.
---
```yaml tailmatter
qa_checklist:
- requirement: "All citations properly formatted"
complete: true
- requirement: "Statistical analysis verified"
complete: true
- requirement: "Peer review completed"
complete: false
editorial:
status: "In Review"
reviewer: "editorial.board@journal.com"
submission_id: "TOCS-2025-0142"
agent_config:
role: "academic_paper_reviewer"
focus: "methodology and statistical analysis"
```

11
tests/fixtures/contentmatter_test_files/no_contentmatter.md vendored Normal file
View File

@@ -0,0 +1,11 @@
# Document Without Contentmatter
This document contains no MultiMarkdown key-value pairs within the content.
It has regular paragraphs and sections, but no lines in the format "Key: Value".
## Regular Content
Just normal markdown content here. No special metadata embedded within the text.
The contentmatter commands should handle this gracefully by returning empty results.

24
tests/fixtures/contentmatter_test_files/simple_contentmatter.md vendored Normal file
View File

@@ -0,0 +1,24 @@
# Simple Document with Contentmatter
This document demonstrates basic MultiMarkdown key-value usage.
Author: Jane Smith
Project: Contentmatter Testing
Version: 2.0
Status: Active
## Basic Information
The contentmatter above provides metadata within the content flow.
License: MIT
Repository: https://github.com/example/project
Documentation: https://docs.example.com
## Usage Notes
Updated: 2025-10-02
Reviewer: John Doe
Category: Testing
This demonstrates various types of contentmatter values that should be extractable.

14
tests/fixtures/frontmatter_test_files/empty_frontmatter.md vendored Normal file
View File

@@ -0,0 +1,14 @@
---
---
# Document With Empty Frontmatter
This document has frontmatter delimiters but no actual metadata content.
The frontmatter commands should handle this edge case:
- frontmatter-get should return empty/null values
- frontmatter-keys should return empty list
- frontmatter-stats should show zero fields
- frontmatter-set should work to add metadata
This tests the parser's handling of empty frontmatter blocks.

26
tests/fixtures/frontmatter_test_files/json_frontmatter.md vendored Normal file
View File

@@ -0,0 +1,26 @@
---
{
"title": "JSON Frontmatter Test Document",
"author": "Test Author",
"date": "2025-10-02",
"tags": ["json", "frontmatter", "testing"],
"version": 2.1,
"published": false,
"config": {
"theme": "dark",
"language": "en",
"features": ["toc", "search", "navigation"]
}
}
---
# JSON Frontmatter Test Document
This document uses JSON format for frontmatter instead of YAML.
The frontmatter parser should handle JSON format correctly and extract values like:
- title: "JSON Frontmatter Test Document"
- config.theme: "dark"
- config.features: ["toc", "search", "navigation"]
This tests the parser's ability to handle different frontmatter formats.

11
tests/fixtures/frontmatter_test_files/no_frontmatter.md vendored Normal file
View File

@@ -0,0 +1,11 @@
# Document Without Frontmatter
This document has no frontmatter at all. It starts directly with content.
The frontmatter commands should handle this gracefully:
- frontmatter-get should return empty/null values
- frontmatter-keys should return empty list
- frontmatter-stats should show zero fields
- frontmatter-set should be able to add frontmatter to the document
This tests edge case handling for documents without any frontmatter.

28
tests/fixtures/frontmatter_test_files/yaml_frontmatter.md vendored Normal file
View File

@@ -0,0 +1,28 @@
---
title: "YAML Frontmatter Test Document"
author: "Test Author"
date: 2025-10-02
tags: ["yaml", "frontmatter", "testing"]
version: 1.2
published: true
description: "A test document with YAML frontmatter for testing frontmatter commands"
nested:
category: "documentation"
priority: "high"
metadata:
creation_date: "2025-10-02"
last_modified: "2025-10-02"
---
# YAML Frontmatter Test Document
This document contains YAML frontmatter that should be accessible via the frontmatter commands.
The frontmatter includes various data types:
- Strings (title, author)
- Arrays (tags)
- Numbers (version)
- Booleans (published)
- Nested objects (nested.category, nested.priority)
This content should be extracted separately from the frontmatter metadata.

38
tests/fixtures/tailmatter_test_files/complete_tailmatter.md vendored Normal file
View File

@@ -0,0 +1,38 @@
# Document with Complete Tailmatter
This document demonstrates tailmatter usage according to the MarkdownMatters specification.
The main content goes here with various sections and information.
## Section 1
Content for section 1.
## Section 2
Content for section 2.
---
```yaml tailmatter
qa_checklist:
- requirement: "All headers verified"
complete: true
- requirement: "Links checked"
complete: false
- requirement: "Spelling reviewed"
complete: true
editorial:
status: "In Review"
reviewer: "jane.doe@example.com"
version: 2.1
last_updated: "2025-10-02"
approval_required: true
agent_config:
role: "documentation_reviewer"
access_scope: "content"
focus_areas: ["grammar", "technical_accuracy"]
output_format: "detailed_report"
```

429
tests/test_contentmatter_commands.py Normal file
View File

@@ -0,0 +1,429 @@
"""
TDD8 Cycle 3: Contentmatter Commands Tests (RED Phase)
Issue #38 - MarkdownMatters CLI Implementation
This test file implements the RED phase tests for contentmatter command family:
- markitect contentmatter-get [key] [path] - Get MMD key-value from content
- markitect contentmatter-set key=value [path] - Set MMD key-value in content
- markitect contentmatter-keys [path] - List all contentmatter keys
- markitect contentmatter-stats [path] - Contentmatter statistics
Following TDD8 methodology, these tests MUST FAIL initially.
"""
import pytest
import tempfile
import os
from pathlib import Path
from click.testing import CliRunner
from markitect.matter_contentmatter.parser import ContentmatterParser
from markitect.matter_contentmatter.stats import ContentmatterStats
from markitect.matter_contentmatter.commands import contentmatter_get, contentmatter_set, contentmatter_keys, contentmatter_stats
class TestContentmatterExtraction:
"""Test contentmatter extraction and parsing."""
@pytest.fixture
def test_files_dir(self):
"""Path to contentmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
@pytest.fixture
def contentmatter_parser(self):
"""Contentmatter parser instance."""
return ContentmatterParser()
def test_contentmatter_parser_extracts_mmd_keyvalues(self, contentmatter_parser, test_files_dir):
"""Test that parser extracts MultiMarkdown key-value pairs."""
file_path = test_files_dir / "simple_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
contentmatter = contentmatter_parser.extract_contentmatter(text)
# Should extract basic key-value pairs
assert contentmatter["Author"] == "Jane Smith"
assert contentmatter["Project"] == "Contentmatter Testing"
assert contentmatter["Version"] == "2.0"
assert contentmatter["Status"] == "Active"
assert contentmatter["License"] == "MIT"
assert contentmatter["Repository"] == "https://github.com/example/project"
assert contentmatter["Documentation"] == "https://docs.example.com"
assert contentmatter["Updated"] == "2025-10-02"
assert contentmatter["Reviewer"] == "John Doe"
assert contentmatter["Category"] == "Testing"
def test_contentmatter_parser_extracts_complex_content(self, contentmatter_parser, test_files_dir):
"""Test extraction from document with rich contentmatter."""
file_path = test_files_dir / "mmd_rich_content.md"
with open(file_path, 'r') as f:
text = f.read()
contentmatter = contentmatter_parser.extract_contentmatter(text)
# Should extract author information
assert contentmatter["Author"] == "Dr. Sarah Johnson"
assert contentmatter["Institution"] == "MIT Computer Science Department"
assert contentmatter["Email"] == "sarah.johnson@mit.edu"
# Should extract research metadata
assert contentmatter["Keywords"] == "algorithms, distributed systems, optimization, performance"
assert contentmatter["Classification"] == "Computer Science - Distributed Computing"
assert contentmatter["Grant Number"] == "NSF-CS-2025-001"
# Should extract methodology details
assert contentmatter["Research Method"] == "Experimental Analysis"
assert contentmatter["Sample Size"] == "1000 distributed nodes"
assert contentmatter["Performance Improvement"] == "23% average speedup"
def test_contentmatter_parser_handles_no_contentmatter(self, contentmatter_parser, test_files_dir):
"""Test that parser handles documents without contentmatter."""
file_path = test_files_dir / "no_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
contentmatter = contentmatter_parser.extract_contentmatter(text)
# Should return empty dict for no contentmatter
assert contentmatter == {}
def test_contentmatter_parser_ignores_frontmatter_and_tailmatter(self, contentmatter_parser, test_files_dir):
"""Test that parser only extracts from content, not matter zones."""
file_path = test_files_dir / "mmd_rich_content.md"
with open(file_path, 'r') as f:
text = f.read()
contentmatter = contentmatter_parser.extract_contentmatter(text)
# Should not include frontmatter values
assert "title" not in contentmatter # This is in frontmatter
assert "qa_checklist" not in contentmatter # This is in tailmatter
# Should only include content key-values
assert "Author" in contentmatter # This is in content
def test_contentmatter_get_specific_value(self, contentmatter_parser, test_files_dir):
"""Test getting specific contentmatter values."""
file_path = test_files_dir / "simple_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
value = contentmatter_parser.get_contentmatter_value(text, "Author")
assert value == "Jane Smith"
value = contentmatter_parser.get_contentmatter_value(text, "Repository")
assert value == "https://github.com/example/project"
# Should return None for non-existent keys
value = contentmatter_parser.get_contentmatter_value(text, "NonExistent")
assert value is None
class TestContentmatterModification:
"""Test contentmatter modification operations."""
@pytest.fixture
def contentmatter_parser(self):
"""Contentmatter parser instance."""
return ContentmatterParser()
def test_contentmatter_set_new_value(self, contentmatter_parser):
"""Test adding new contentmatter to document."""
text = """# Test Document
Some content here.
## Section
More content."""
new_text = contentmatter_parser.set_contentmatter_value(text, "Author", "New Author")
# Should add the contentmatter
contentmatter = contentmatter_parser.extract_contentmatter(new_text)
assert contentmatter["Author"] == "New Author"
# Should preserve original content
assert "# Test Document" in new_text
assert "Some content here." in new_text
def test_contentmatter_update_existing_value(self, contentmatter_parser):
"""Test updating existing contentmatter value."""
text = """# Test Document
Author: Original Author
Project: Test Project
Some content here."""
new_text = contentmatter_parser.set_contentmatter_value(text, "Author", "Updated Author")
# Should update the existing value
contentmatter = contentmatter_parser.extract_contentmatter(new_text)
assert contentmatter["Author"] == "Updated Author"
assert contentmatter["Project"] == "Test Project" # Should preserve other values
def test_contentmatter_set_multiple_values(self, contentmatter_parser):
"""Test setting multiple contentmatter values."""
text = """# Test Document
Some content here."""
# Add multiple values
text = contentmatter_parser.set_contentmatter_value(text, "Author", "Test Author")
text = contentmatter_parser.set_contentmatter_value(text, "Version", "1.0")
text = contentmatter_parser.set_contentmatter_value(text, "Status", "Active")
contentmatter = contentmatter_parser.extract_contentmatter(text)
assert contentmatter["Author"] == "Test Author"
assert contentmatter["Version"] == "1.0"
assert contentmatter["Status"] == "Active"
class TestContentmatterKeys:
"""Test contentmatter key listing functionality."""
@pytest.fixture
def test_files_dir(self):
"""Path to contentmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
@pytest.fixture
def contentmatter_parser(self):
"""Contentmatter parser instance."""
return ContentmatterParser()
def test_contentmatter_keys_simple_document(self, contentmatter_parser, test_files_dir):
"""Test listing keys from simple contentmatter document."""
file_path = test_files_dir / "simple_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
keys = contentmatter_parser.get_contentmatter_keys(text)
# Should return all contentmatter keys
expected_keys = ["Author", "Project", "Version", "Status", "License", "Repository", "Documentation", "Updated", "Reviewer", "Category"]
assert set(keys) == set(expected_keys)
def test_contentmatter_keys_complex_document(self, contentmatter_parser, test_files_dir):
"""Test listing keys from complex contentmatter document."""
file_path = test_files_dir / "mmd_rich_content.md"
with open(file_path, 'r') as f:
text = f.read()
keys = contentmatter_parser.get_contentmatter_keys(text)
# Should include research paper metadata keys
assert "Author" in keys
assert "Institution" in keys
assert "Keywords" in keys
assert "Research Method" in keys
assert "Performance Improvement" in keys
# Should not include frontmatter or tailmatter keys
assert "title" not in keys # frontmatter
assert "qa_checklist" not in keys # tailmatter
def test_contentmatter_keys_empty_document(self, contentmatter_parser, test_files_dir):
"""Test listing keys from document without contentmatter."""
file_path = test_files_dir / "no_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
keys = contentmatter_parser.get_contentmatter_keys(text)
# Should return empty list
assert keys == []
class TestContentmatterStatistics:
"""Test contentmatter statistics calculation."""
@pytest.fixture
def test_files_dir(self):
"""Path to contentmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
@pytest.fixture
def contentmatter_parser(self):
"""Contentmatter parser instance."""
return ContentmatterParser()
def test_contentmatter_stats_simple_document(self, contentmatter_parser, test_files_dir):
"""Test statistics calculation for simple contentmatter."""
file_path = test_files_dir / "simple_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
stats = contentmatter_parser.calculate_contentmatter_stats(text)
# Should count contentmatter correctly
assert stats.total_pairs == 10 # Number of key-value pairs
assert stats.has_contentmatter is True
assert stats.average_key_length > 0
assert stats.average_value_length > 0
# Should categorize value types
assert stats.url_values > 0 # Repository and Documentation URLs
assert stats.date_values > 0 # Updated field
assert stats.email_values == 0 # No email in simple document
def test_contentmatter_stats_complex_document(self, contentmatter_parser, test_files_dir):
"""Test statistics calculation for complex contentmatter."""
file_path = test_files_dir / "mmd_rich_content.md"
with open(file_path, 'r') as f:
text = f.read()
stats = contentmatter_parser.calculate_contentmatter_stats(text)
# Should count rich contentmatter
assert stats.total_pairs > 15 # Many key-value pairs in research paper
assert stats.has_contentmatter is True
# Should detect email values
assert stats.email_values > 0 # Email field in author info
def test_contentmatter_stats_no_contentmatter(self, contentmatter_parser, test_files_dir):
"""Test statistics for document without contentmatter."""
file_path = test_files_dir / "no_contentmatter.md"
with open(file_path, 'r') as f:
text = f.read()
stats = contentmatter_parser.calculate_contentmatter_stats(text)
# Should indicate no contentmatter
assert stats.has_contentmatter is False
assert stats.total_pairs == 0
assert stats.url_values == 0
assert stats.email_values == 0
class TestContentmatterCLICommands:
"""Test CLI command integration."""
@pytest.fixture
def runner(self):
"""CLI test runner."""
return CliRunner()
@pytest.fixture
def test_files_dir(self):
"""Path to contentmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
def test_contentmatter_get_command(self, runner, test_files_dir):
"""Test contentmatter-get CLI command."""
file_path = test_files_dir / "simple_contentmatter.md"
# Test getting simple value
result = runner.invoke(contentmatter_get, ['Author', '--file', str(file_path)])
assert result.exit_code == 0
assert "Jane Smith" in result.output
# Test getting URL value
result = runner.invoke(contentmatter_get, ['Repository', '--file', str(file_path)])
assert result.exit_code == 0
assert "https://github.com/example/project" in result.output
def test_contentmatter_keys_command(self, runner, test_files_dir):
"""Test contentmatter-keys CLI command."""
file_path = test_files_dir / "simple_contentmatter.md"
result = runner.invoke(contentmatter_keys, ['--file', str(file_path)])
assert result.exit_code == 0
assert "Author" in result.output
assert "Project" in result.output
assert "Repository" in result.output
def test_contentmatter_stats_command(self, runner, test_files_dir):
"""Test contentmatter-stats CLI command."""
file_path = test_files_dir / "simple_contentmatter.md"
result = runner.invoke(contentmatter_stats, ['--file', str(file_path)])
assert result.exit_code == 0
assert "total_pairs" in result.output
assert "has_contentmatter" in result.output
def test_contentmatter_set_command(self, runner, test_files_dir):
"""Test contentmatter-set CLI command."""
# Create temporary file for testing
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write("""# Test Document
Author: Original Author
Some content here.""")
temp_file = f.name
try:
result = runner.invoke(contentmatter_set, ['Author=New Author', '--file', temp_file])
assert result.exit_code == 0
# Verify the change was made
with open(temp_file, 'r') as f:
content = f.read()
assert "Author: New Author" in content
finally:
os.unlink(temp_file)
def test_contentmatter_commands_help_text(self, runner):
"""Test that help text is available for all contentmatter commands."""
commands = [contentmatter_get, contentmatter_keys, contentmatter_stats, contentmatter_set]
for command in commands:
result = runner.invoke(command, ['--help'])
assert result.exit_code == 0
assert "contentmatter" in result.output.lower()
class TestContentmatterStats:
"""Test ContentmatterStats data class."""
def test_contentmatter_stats_creation(self):
"""Test ContentmatterStats object creation."""
stats = ContentmatterStats(
has_contentmatter=True,
total_pairs=10,
average_key_length=8.5,
average_value_length=15.2,
url_values=2,
email_values=1,
date_values=1
)
assert stats.has_contentmatter is True
assert stats.total_pairs == 10
assert stats.average_key_length == 8.5
assert stats.url_values == 2
def test_contentmatter_stats_to_dict(self):
"""Test ContentmatterStats conversion to dictionary."""
stats = ContentmatterStats(
has_contentmatter=True,
total_pairs=5,
average_key_length=8.0,
average_value_length=12.0,
url_values=1,
email_values=0,
date_values=1
)
stats_dict = stats.to_dict()
assert stats_dict["has_contentmatter"] is True
assert stats_dict["total_pairs"] == 5
assert stats_dict["url_values"] == 1

428
tests/test_frontmatter_commands.py Normal file
View File

@@ -0,0 +1,428 @@
"""
TDD8 Cycle 2: Frontmatter Commands Tests (RED Phase)
Issue #38 - MarkdownMatters CLI Implementation
This test file implements the RED phase tests for frontmatter command family:
- markitect frontmatter-get [key] [path] - Get specific frontmatter value
- markitect frontmatter-set key=value [path] - Set frontmatter value
- markitect frontmatter-keys [path] - List all frontmatter keys
- markitect frontmatter-stats [path] - Frontmatter statistics
Following TDD8 methodology, these tests MUST FAIL initially.
"""
import pytest
import tempfile
import os
from pathlib import Path
from click.testing import CliRunner
from markitect.matter_frontmatter.parser import FrontmatterParser
from markitect.matter_frontmatter.stats import FrontmatterStats
from markitect.matter_frontmatter.commands import frontmatter_get, frontmatter_set, frontmatter_keys, frontmatter_stats
class TestFrontmatterExtraction:
"""Test frontmatter extraction and parsing."""
@pytest.fixture
def test_files_dir(self):
"""Path to frontmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
@pytest.fixture
def frontmatter_parser(self):
"""Frontmatter parser instance."""
return FrontmatterParser()
def test_frontmatter_parser_extracts_yaml_frontmatter(self, frontmatter_parser, test_files_dir):
"""Test that parser extracts YAML frontmatter correctly."""
file_path = test_files_dir / "yaml_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
frontmatter = frontmatter_parser.extract_frontmatter(text)
# Should extract all YAML frontmatter fields
assert frontmatter["title"] == "YAML Frontmatter Test Document"
assert frontmatter["author"] == "Test Author"
assert str(frontmatter["date"]) == "2025-10-02"
assert frontmatter["tags"] == ["yaml", "frontmatter", "testing"]
assert frontmatter["version"] == 1.2
assert frontmatter["published"] is True
# Should handle nested objects
assert frontmatter["nested"]["category"] == "documentation"
assert frontmatter["nested"]["priority"] == "high"
assert frontmatter["nested"]["metadata"]["creation_date"] == "2025-10-02"
def test_frontmatter_parser_extracts_json_frontmatter(self, frontmatter_parser, test_files_dir):
"""Test that parser extracts JSON frontmatter correctly."""
file_path = test_files_dir / "json_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
frontmatter = frontmatter_parser.extract_frontmatter(text)
# Should extract all JSON frontmatter fields
assert frontmatter["title"] == "JSON Frontmatter Test Document"
assert frontmatter["author"] == "Test Author"
assert frontmatter["tags"] == ["json", "frontmatter", "testing"]
assert frontmatter["version"] == 2.1
assert frontmatter["published"] is False
# Should handle nested objects
assert frontmatter["config"]["theme"] == "dark"
assert frontmatter["config"]["language"] == "en"
assert frontmatter["config"]["features"] == ["toc", "search", "navigation"]
def test_frontmatter_parser_handles_no_frontmatter(self, frontmatter_parser, test_files_dir):
"""Test that parser handles documents without frontmatter."""
file_path = test_files_dir / "no_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
frontmatter = frontmatter_parser.extract_frontmatter(text)
# Should return empty dict for no frontmatter
assert frontmatter == {}
def test_frontmatter_parser_handles_empty_frontmatter(self, frontmatter_parser, test_files_dir):
"""Test that parser handles empty frontmatter blocks."""
file_path = test_files_dir / "empty_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
frontmatter = frontmatter_parser.extract_frontmatter(text)
# Should return empty dict for empty frontmatter
assert frontmatter == {}
def test_frontmatter_parser_get_nested_value(self, frontmatter_parser, test_files_dir):
"""Test getting nested values using dot notation."""
file_path = test_files_dir / "yaml_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
frontmatter = frontmatter_parser.extract_frontmatter(text)
# Should support dot notation for nested access
value = frontmatter_parser.get_nested_value(frontmatter, "nested.category")
assert value == "documentation"
value = frontmatter_parser.get_nested_value(frontmatter, "nested.metadata.creation_date")
assert value == "2025-10-02"
# Should return None for non-existent keys
value = frontmatter_parser.get_nested_value(frontmatter, "non.existent.key")
assert value is None
class TestFrontmatterModification:
"""Test frontmatter modification operations."""
@pytest.fixture
def frontmatter_parser(self):
"""Frontmatter parser instance."""
return FrontmatterParser()
def test_frontmatter_set_simple_value(self, frontmatter_parser):
"""Test setting simple frontmatter values."""
text = """---
title: "Original Title"
author: "Original Author"
---
# Content here"""
new_text = frontmatter_parser.set_frontmatter_value(text, "title", "New Title")
# Should update the title value
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
assert frontmatter["title"] == "New Title"
assert frontmatter["author"] == "Original Author"
def test_frontmatter_set_new_value(self, frontmatter_parser):
"""Test adding new frontmatter values."""
text = """---
title: "Original Title"
---
# Content here"""
new_text = frontmatter_parser.set_frontmatter_value(text, "author", "New Author")
# Should add the new field
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
assert frontmatter["title"] == "Original Title"
assert frontmatter["author"] == "New Author"
def test_frontmatter_set_nested_value(self, frontmatter_parser):
"""Test setting nested frontmatter values using dot notation."""
text = """---
title: "Test"
config:
theme: "light"
---
# Content here"""
new_text = frontmatter_parser.set_frontmatter_value(text, "config.theme", "dark")
# Should update nested value
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
assert frontmatter["config"]["theme"] == "dark"
def test_frontmatter_add_to_empty_document(self, frontmatter_parser):
"""Test adding frontmatter to document without any."""
text = """# Content Without Frontmatter
Just some content here."""
new_text = frontmatter_parser.set_frontmatter_value(text, "title", "New Title")
# Should add frontmatter block
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
assert frontmatter["title"] == "New Title"
# Should preserve content
assert "# Content Without Frontmatter" in new_text
class TestFrontmatterKeys:
"""Test frontmatter key listing functionality."""
@pytest.fixture
def test_files_dir(self):
"""Path to frontmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
@pytest.fixture
def frontmatter_parser(self):
"""Frontmatter parser instance."""
return FrontmatterParser()
def test_frontmatter_keys_yaml_document(self, frontmatter_parser, test_files_dir):
"""Test listing keys from YAML frontmatter."""
file_path = test_files_dir / "yaml_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
keys = frontmatter_parser.get_frontmatter_keys(text)
# Should return all top-level keys
expected_keys = ["title", "author", "date", "tags", "version", "published", "description", "nested"]
assert set(keys) == set(expected_keys)
def test_frontmatter_keys_with_nested_option(self, frontmatter_parser, test_files_dir):
"""Test listing keys including nested keys with dot notation."""
file_path = test_files_dir / "yaml_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
keys = frontmatter_parser.get_frontmatter_keys(text, include_nested=True)
# Should include nested keys with dot notation
assert "nested.category" in keys
assert "nested.priority" in keys
assert "nested.metadata.creation_date" in keys
assert "nested.metadata.last_modified" in keys
def test_frontmatter_keys_empty_document(self, frontmatter_parser, test_files_dir):
"""Test listing keys from document without frontmatter."""
file_path = test_files_dir / "no_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
keys = frontmatter_parser.get_frontmatter_keys(text)
# Should return empty list
assert keys == []
class TestFrontmatterStatistics:
"""Test frontmatter statistics calculation."""
@pytest.fixture
def test_files_dir(self):
"""Path to frontmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
@pytest.fixture
def frontmatter_parser(self):
"""Frontmatter parser instance."""
return FrontmatterParser()
def test_frontmatter_stats_yaml_document(self, frontmatter_parser, test_files_dir):
"""Test statistics calculation for YAML frontmatter."""
file_path = test_files_dir / "yaml_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
stats = frontmatter_parser.calculate_frontmatter_stats(text)
# Should count fields correctly
assert stats.total_fields == 8 # Top-level fields
assert stats.nested_fields == 5 # Nested fields (category, priority, creation_date, last_modified, metadata object)
assert stats.format == "yaml"
assert stats.has_frontmatter is True
# Should categorize field types
assert "string" in stats.field_types
assert "array" in stats.field_types
assert "number" in stats.field_types
assert "boolean" in stats.field_types
assert "object" in stats.field_types
def test_frontmatter_stats_json_document(self, frontmatter_parser, test_files_dir):
"""Test statistics calculation for JSON frontmatter."""
file_path = test_files_dir / "json_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
stats = frontmatter_parser.calculate_frontmatter_stats(text)
# Should identify JSON format
assert stats.format == "json"
assert stats.has_frontmatter is True
assert stats.total_fields > 0
def test_frontmatter_stats_no_frontmatter(self, frontmatter_parser, test_files_dir):
"""Test statistics for document without frontmatter."""
file_path = test_files_dir / "no_frontmatter.md"
with open(file_path, 'r') as f:
text = f.read()
stats = frontmatter_parser.calculate_frontmatter_stats(text)
# Should indicate no frontmatter
assert stats.has_frontmatter is False
assert stats.total_fields == 0
assert stats.nested_fields == 0
assert stats.format is None
class TestFrontmatterCLICommands:
"""Test CLI command integration."""
@pytest.fixture
def runner(self):
"""CLI test runner."""
return CliRunner()
@pytest.fixture
def test_files_dir(self):
"""Path to frontmatter test fixture files."""
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
def test_frontmatter_get_command(self, runner, test_files_dir):
"""Test frontmatter-get CLI command."""
file_path = test_files_dir / "yaml_frontmatter.md"
# Test getting simple value
result = runner.invoke(frontmatter_get, ['title', '--file', str(file_path)])
assert result.exit_code == 0
assert "YAML Frontmatter Test Document" in result.output
# Test getting nested value
result = runner.invoke(frontmatter_get, ['nested.category', '--file', str(file_path)])
assert result.exit_code == 0
assert "documentation" in result.output
def test_frontmatter_keys_command(self, runner, test_files_dir):
"""Test frontmatter-keys CLI command."""
file_path = test_files_dir / "yaml_frontmatter.md"
result = runner.invoke(frontmatter_keys, ['--file', str(file_path)])
assert result.exit_code == 0
assert "title" in result.output
assert "author" in result.output
assert "tags" in result.output
def test_frontmatter_stats_command(self, runner, test_files_dir):
"""Test frontmatter-stats CLI command."""
file_path = test_files_dir / "yaml_frontmatter.md"
result = runner.invoke(frontmatter_stats, ['--file', str(file_path)])
assert result.exit_code == 0
assert "total_fields" in result.output
assert "format" in result.output
def test_frontmatter_set_command(self, runner, test_files_dir):
"""Test frontmatter-set CLI command."""
# Create temporary file for testing
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write("""---
title: "Original Title"
---
# Test Content""")
temp_file = f.name
try:
result = runner.invoke(frontmatter_set, ['title=New Title', '--file', temp_file])
assert result.exit_code == 0
# Verify the change was made
with open(temp_file, 'r') as f:
content = f.read()
assert "New Title" in content
finally:
os.unlink(temp_file)
def test_frontmatter_commands_help_text(self, runner):
"""Test that help text is available for all frontmatter commands."""
commands = [frontmatter_get, frontmatter_keys, frontmatter_stats, frontmatter_set]
for command in commands:
result = runner.invoke(command, ['--help'])
assert result.exit_code == 0
assert "frontmatter" in result.output.lower()
class TestFrontmatterStats:
"""Test FrontmatterStats data class."""
def test_frontmatter_stats_creation(self):
"""Test FrontmatterStats object creation."""
stats = FrontmatterStats(
has_frontmatter=True,
total_fields=5,
nested_fields=2,
format="yaml",
field_types={"string": 3, "number": 1, "boolean": 1}
)
assert stats.has_frontmatter is True
assert stats.total_fields == 5
assert stats.nested_fields == 2
assert stats.format == "yaml"
assert stats.field_types["string"] == 3
def test_frontmatter_stats_to_dict(self):
"""Test FrontmatterStats conversion to dictionary."""
stats = FrontmatterStats(
has_frontmatter=True,
total_fields=5,
nested_fields=2,
format="yaml",
field_types={"string": 3}
)
stats_dict = stats.to_dict()
assert stats_dict["has_frontmatter"] is True
assert stats_dict["total_fields"] == 5
assert stats_dict["format"] == "yaml"

295
tests/test_markdownmatters_integration.py Normal file
View File

@@ -0,0 +1,295 @@
"""
Integration tests for complete MarkdownMatters CLI implementation.
Tests all four command families working together.
"""
import pytest
import tempfile
import os
from pathlib import Path
from click.testing import CliRunner
from markitect.content.commands import content_get, content_stats
from markitect.matter_frontmatter.commands import frontmatter_get, frontmatter_keys
from markitect.matter_contentmatter.commands import contentmatter_get, contentmatter_keys
from markitect.matter_tailmatter.commands import tailmatter_get, tailmatter_check
class TestMarkdownMattersIntegration:
"""Test complete MarkdownMatters functionality integration."""
@pytest.fixture
def complete_document(self):
"""A complete MarkdownMatters document with all three zones."""
return """---
title: "Complete MarkdownMatters Document"
author: "Integration Test"
version: 1.0
status: "testing"
---
# Complete MarkdownMatters Document
This document demonstrates all three matter zones working together.
Author: Dr. Test Researcher
Institution: MarkdownMatters University
Email: test@markdownmatters.edu
Project: Integration Testing
Version: 2.0
Status: Active
## Research Content
Research Method: Integration Testing
Sample Size: Complete document
Test Framework: MarkdownMatters CLI
The content includes various MultiMarkdown key-value pairs that provide contextual metadata.
## Results
Result Status: All systems operational
Performance: Excellent
Coverage: 100%
All matter zones are properly separated and accessible through their respective CLI commands.
---
```yaml tailmatter
qa_checklist:
- requirement: "All three matter zones tested"
complete: true
- requirement: "CLI commands validated"
complete: true
- requirement: "Integration verified"
complete: false
editorial:
status: "Integration Testing"
reviewer: "integration.tester@markdownmatters.edu"
version: 3.0
agent_config:
role: "integration_validator"
access_scope: "all_zones"
validation_mode: "comprehensive"
```"""
@pytest.fixture
def runner(self):
"""CLI test runner."""
return CliRunner()
def test_all_command_families_work_on_same_document(self, runner, complete_document):
"""Test that all four command families can process the same document."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write(complete_document)
temp_file = f.name
try:
# Test content commands
result = runner.invoke(content_get, ['--file', temp_file])
assert result.exit_code == 0
assert "Complete MarkdownMatters Document" in result.output
assert "---" not in result.output # No frontmatter
assert "qa_checklist" not in result.output # No tailmatter
result = runner.invoke(content_stats, ['--file', temp_file])
assert result.exit_code == 0
assert "word_count" in result.output
# Test frontmatter commands
result = runner.invoke(frontmatter_get, ['title', '--file', temp_file])
assert result.exit_code == 0
assert "Complete MarkdownMatters Document" in result.output
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
assert result.exit_code == 0
assert "title" in result.output
assert "author" in result.output
# Test contentmatter commands
result = runner.invoke(contentmatter_get, ['Author', '--file', temp_file])
assert result.exit_code == 0
assert "Dr. Test Researcher" in result.output
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
assert result.exit_code == 0
assert "Author" in result.output
assert "Institution" in result.output
# Test tailmatter commands
result = runner.invoke(tailmatter_get, ['editorial.status', '--file', temp_file])
assert result.exit_code == 0
assert "Integration Testing" in result.output
result = runner.invoke(tailmatter_check, ['--file', temp_file])
assert result.exit_code == 0
assert "QA Checklist Status" in result.output
assert "✅" in result.output
assert "❌" in result.output
finally:
os.unlink(temp_file)
def test_matter_zone_separation(self, runner, complete_document):
"""Test that each command family only accesses its designated zone."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write(complete_document)
temp_file = f.name
try:
# Frontmatter should not include contentmatter or tailmatter
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
assert "Author" not in result.output # This is contentmatter
assert "qa_checklist" not in result.output # This is tailmatter
# Contentmatter should not include frontmatter or tailmatter
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
assert "title" not in result.output # This is frontmatter
assert "qa_checklist" not in result.output # This is tailmatter
# Content should not include any matter zones in the actual content
result = runner.invoke(content_get, ['--file', temp_file])
assert "title:" not in result.output # No frontmatter YAML
assert "qa_checklist:" not in result.output # No tailmatter YAML
finally:
os.unlink(temp_file)
def test_performance_with_large_document(self, runner):
"""Test performance with a large document containing all matter zones."""
# Create a large document
large_content = []
large_content.append("---")
large_content.append("title: 'Large Document Performance Test'")
for i in range(50):
large_content.append(f"field_{i}: 'value_{i}'")
large_content.append("---")
large_content.append("")
large_content.append("# Large Document Performance Test")
large_content.append("")
# Add many contentmatter pairs
for i in range(100):
large_content.append(f"Data Field {i}: Value for field {i}")
large_content.append("")
# Add substantial content
for i in range(50):
large_content.append(f"## Section {i}")
large_content.append("")
large_content.append(f"Content for section {i} with detailed information and multiple paragraphs.")
large_content.append("")
large_content.append("More content here to make the document substantial in size.")
large_content.append("")
large_content.append("---")
large_content.append("")
large_content.append("```yaml tailmatter")
large_content.append("qa_checklist:")
for i in range(20):
complete = "true" if i % 3 == 0 else "false"
large_content.append(f" - requirement: 'Test requirement {i}'")
large_content.append(f" complete: {complete}")
large_content.append("editorial:")
large_content.append(" status: 'Performance Testing'")
large_content.append("```")
large_document = "\n".join(large_content)
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write(large_document)
temp_file = f.name
try:
# Test that all commands complete in reasonable time
import time
start_time = time.time()
result = runner.invoke(content_stats, ['--file', temp_file])
content_time = time.time() - start_time
assert result.exit_code == 0
assert content_time < 2.0 # Should complete in under 2 seconds
start_time = time.time()
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
frontmatter_time = time.time() - start_time
assert result.exit_code == 0
assert frontmatter_time < 1.0 # Should complete in under 1 second
start_time = time.time()
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
contentmatter_time = time.time() - start_time
assert result.exit_code == 0
assert contentmatter_time < 2.0 # Should complete in under 2 seconds
start_time = time.time()
result = runner.invoke(tailmatter_check, ['--file', temp_file])
tailmatter_time = time.time() - start_time
assert result.exit_code == 0
assert tailmatter_time < 1.0 # Should complete in under 1 second
finally:
os.unlink(temp_file)
def test_error_handling_consistency(self, runner):
"""Test that all command families handle errors consistently."""
non_existent_file = "/tmp/non_existent_file.md"
# All commands should handle missing files gracefully
commands_and_args = [
(content_get, ['--file', non_existent_file]),
(content_stats, ['--file', non_existent_file]),
(frontmatter_get, ['title', '--file', non_existent_file]),
(frontmatter_keys, ['--file', non_existent_file]),
(contentmatter_get, ['Author', '--file', non_existent_file]),
(contentmatter_keys, ['--file', non_existent_file]),
(tailmatter_get, ['editorial.status', '--file', non_existent_file]),
(tailmatter_check, ['--file', non_existent_file]),
]
for command, args in commands_and_args:
result = runner.invoke(command, args)
assert result.exit_code != 0 # Should fail for non-existent file
def test_help_commands_consistency(self, runner):
"""Test that all commands provide consistent help."""
commands = [
content_get, content_stats,
frontmatter_get, frontmatter_keys,
contentmatter_get, contentmatter_keys,
tailmatter_get, tailmatter_check
]
for command in commands:
result = runner.invoke(command, ['--help'])
assert result.exit_code == 0
assert "Usage:" in result.output
assert "--help" in result.output
def test_output_format_consistency(self, runner, complete_document):
"""Test that commands with format options work consistently."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
f.write(complete_document)
temp_file = f.name
try:
# Test JSON format consistency
result = runner.invoke(content_stats, ['--file', temp_file, '--format', 'json'])
assert result.exit_code == 0
assert result.output.startswith('{')
result = runner.invoke(frontmatter_keys, ['--file', temp_file, '--format', 'json'])
assert result.exit_code == 0
assert result.output.startswith('[')
result = runner.invoke(contentmatter_keys, ['--file', temp_file, '--format', 'json'])
assert result.exit_code == 0
assert result.output.startswith('[')
finally:
os.unlink(temp_file)