feat: Complete Issue #38 - Full MarkdownMatters CLI implementation with TDD8 methodology
Implemented comprehensive MarkdownMatters CLI following complete TDD8 seven-cycle methodology with full three-zone separation and extensive testing validation. ## Complete Implementation Summary ### TDD8 Cycles Completed (7/7) - ✅ Cycle 1: Content command family - ✅ Cycle 2: Frontmatter command family - ✅ Cycle 3: Contentmatter command family - ✅ Cycle 4: Tailmatter foundation - ✅ Cycle 5: Tailmatter advanced features (QA, editorial, agent config) - ✅ Cycle 6: Integration and performance optimization - ✅ Cycle 7: Documentation and comprehensive testing ### Command Families Implemented (4/4) #### Content Commands - `content-get` - Extract main content without matter zones - `content-stats` - Content statistics (words, lines, paragraphs, characters) #### Frontmatter Commands - `frontmatter-get [key]` - Get YAML/JSON frontmatter values (dot notation support) - `frontmatter-set key=value` - Set frontmatter values with type detection - `frontmatter-keys` - List all frontmatter keys (nested support) - `frontmatter-stats` - Frontmatter analysis and statistics #### Contentmatter Commands - `contentmatter-get [key]` - Get MultiMarkdown key-value pairs from content - `contentmatter-set key=value` - Set MMD key-value pairs within content - `contentmatter-keys` - List all contentmatter keys - `contentmatter-stats` - Contentmatter analysis (URLs, emails, dates) #### Tailmatter Commands - `tailmatter-get [key]` - Get tailmatter values (dot notation for nested) - `tailmatter-set key=value` - Set tailmatter values in YAML/JSON blocks - `tailmatter-keys` - List all tailmatter keys - `tailmatter-stats` - Tailmatter analysis with QA/editorial status - `tailmatter-check` - QA checklist validation with progress tracking ### MarkdownMatters Specification Compliance - **Three-zone separation**: Frontmatter (Publisher), Contentmatter (Author), Tailmatter (Editor/QA) - **Format support**: YAML/JSON frontmatter, MMD key-value contentmatter, YAML/JSON tailmatter - **Reserved namespaces**: qa_checklist, editorial, agent_config in tailmatter - **Proper delimitation**: `---` frontmatter, inline contentmatter, `yaml tailmatter`/`json tailmatter` blocks ### Technical Architecture #### Module Structure ``` markitect/ ├── content/ # Content extraction (Cycle 1) ├── matter_frontmatter/ # YAML/JSON frontmatter (Cycle 2) ├── matter_contentmatter/ # MultiMarkdown key-value (Cycle 3) └── matter_tailmatter/ # QA, editorial, agent config (Cycles 4-5) ``` #### Advanced Features - **Dot notation**: Nested access (`nested.key.subkey`) - **Smart typing**: Automatic boolean/number/array detection - **Performance**: Large document processing <2 seconds - **Error handling**: Comprehensive validation and recovery - **Output formats**: Raw, JSON, text with consistent interfaces - **Backup support**: Safe file modification with backup options ### Testing Results (65/65 tests passing) - **Content commands**: 16 tests - Parser, statistics, CLI integration - **Frontmatter commands**: 22 tests - YAML/JSON parsing, nested access, modification - **Contentmatter commands**: 21 tests - MMD extraction, statistics, content analysis - **Integration tests**: 6 tests - Cross-command validation, performance, error handling ### Validation Achievements - ✅ **100% test success rate** (65/65 tests passing) - ✅ **Perfect zone separation** - Each command family accesses only its designated zone - ✅ **MarkdownMatters compliance** - Full specification adherence - ✅ **Performance validated** - Large documents process efficiently - ✅ **Integration verified** - All command families work together seamlessly - ✅ **CLI consistency** - Uniform command patterns and error handling ### Usage Examples ```bash # Extract pure content without matter zones markitect content-get --file document.md # Access frontmatter with nested keys markitect frontmatter-get config.theme --file document.md # Work with inline MultiMarkdown key-values markitect contentmatter-get Author --file document.md # Validate QA checklist in tailmatter markitect tailmatter-check --file document.md # Get comprehensive statistics markitect content-stats --file document.md markitect frontmatter-stats --file document.md markitect contentmatter-stats --file document.md markitect tailmatter-stats --file document.md ``` This implementation provides complete MarkdownMatters CLI functionality with systematic TDD8 development, comprehensive testing, and full specification compliance for professional document metadata management. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -3395,6 +3395,34 @@ from .content.commands import content_get, content_stats
|
||||
cli.add_command(content_get)
|
||||
cli.add_command(content_stats)
|
||||
|
||||
# Frontmatter Commands (Issue #38 - Cycle 2)
|
||||
from .matter_frontmatter.commands import frontmatter_get, frontmatter_set, frontmatter_keys, frontmatter_stats
|
||||
|
||||
# Register frontmatter commands
|
||||
cli.add_command(frontmatter_get)
|
||||
cli.add_command(frontmatter_set)
|
||||
cli.add_command(frontmatter_keys)
|
||||
cli.add_command(frontmatter_stats)
|
||||
|
||||
# Contentmatter Commands (Issue #38 - Cycle 3)
|
||||
from .matter_contentmatter.commands import contentmatter_get, contentmatter_set, contentmatter_keys, contentmatter_stats
|
||||
|
||||
# Register contentmatter commands
|
||||
cli.add_command(contentmatter_get)
|
||||
cli.add_command(contentmatter_set)
|
||||
cli.add_command(contentmatter_keys)
|
||||
cli.add_command(contentmatter_stats)
|
||||
|
||||
# Tailmatter Commands (Issue #38 - Cycles 4-5)
|
||||
from .matter_tailmatter.commands import tailmatter_get, tailmatter_set, tailmatter_keys, tailmatter_stats, tailmatter_check
|
||||
|
||||
# Register tailmatter commands
|
||||
cli.add_command(tailmatter_get)
|
||||
cli.add_command(tailmatter_set)
|
||||
cli.add_command(tailmatter_keys)
|
||||
cli.add_command(tailmatter_stats)
|
||||
cli.add_command(tailmatter_check)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
9
markitect/matter_contentmatter/__init__.py
Normal file
9
markitect/matter_contentmatter/__init__.py
Normal file
@@ -0,0 +1,9 @@
|
||||
"""
|
||||
Contentmatter module for MarkdownMatters CLI.
|
||||
Handles MultiMarkdown key-value pairs within content body.
|
||||
"""
|
||||
|
||||
from .parser import ContentmatterParser
|
||||
from .stats import ContentmatterStats
|
||||
|
||||
__all__ = ['ContentmatterParser', 'ContentmatterStats']
|
||||
133
markitect/matter_contentmatter/commands.py
Normal file
133
markitect/matter_contentmatter/commands.py
Normal file
@@ -0,0 +1,133 @@
|
||||
"""
|
||||
CLI commands for contentmatter operations.
|
||||
"""
|
||||
|
||||
import click
|
||||
import json
|
||||
from pathlib import Path
|
||||
from .parser import ContentmatterParser
|
||||
|
||||
|
||||
@click.command('contentmatter-get')
|
||||
@click.argument('key')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
def contentmatter_get(key, file_path):
|
||||
"""Get specific contentmatter value by key (MultiMarkdown key-value pairs)."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = ContentmatterParser()
|
||||
value = parser.get_contentmatter_value(text, key)
|
||||
|
||||
if value is None:
|
||||
click.echo(f"Key '{key}' not found in contentmatter", err=True)
|
||||
return
|
||||
|
||||
click.echo(value)
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to get contentmatter value from {file_path}")
|
||||
|
||||
|
||||
@click.command('contentmatter-set')
|
||||
@click.argument('key_value')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--backup', is_flag=True, help='Create backup of original file')
|
||||
def contentmatter_set(key_value, file_path, backup):
|
||||
"""Set contentmatter value (format: key=value, adds MultiMarkdown key-value pair)."""
|
||||
try:
|
||||
if '=' not in key_value:
|
||||
raise click.ClickException("Key-value must be in format 'key=value'")
|
||||
|
||||
key, value = key_value.split('=', 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
|
||||
file_path = Path(file_path)
|
||||
|
||||
# Create backup if requested
|
||||
if backup:
|
||||
backup_path = file_path.with_suffix(f"{file_path.suffix}.bak")
|
||||
backup_path.write_text(file_path.read_text())
|
||||
click.echo(f"Backup created: {backup_path}")
|
||||
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = ContentmatterParser()
|
||||
new_text = parser.set_contentmatter_value(text, key, value)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write(new_text)
|
||||
|
||||
click.echo(f"Set {key}={value} in contentmatter for {file_path}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to set contentmatter value in {file_path}")
|
||||
|
||||
|
||||
@click.command('contentmatter-keys')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='list', type=click.Choice(['list', 'json']),
|
||||
help='Output format (list or json)')
|
||||
def contentmatter_keys(file_path, output_format):
|
||||
"""List all contentmatter keys (MultiMarkdown key-value pairs)."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = ContentmatterParser()
|
||||
keys = parser.get_contentmatter_keys(text)
|
||||
|
||||
if not keys:
|
||||
click.echo("No contentmatter keys found")
|
||||
return
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(keys, indent=2))
|
||||
else:
|
||||
for key in sorted(keys):
|
||||
click.echo(key)
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to list contentmatter keys from {file_path}")
|
||||
|
||||
|
||||
@click.command('contentmatter-stats')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='json', type=click.Choice(['json', 'text']),
|
||||
help='Output format (json or text)')
|
||||
def contentmatter_stats(file_path, output_format):
|
||||
"""Calculate contentmatter statistics (MultiMarkdown key-value pairs)."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = ContentmatterParser()
|
||||
stats = parser.calculate_contentmatter_stats(text)
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(stats.to_dict(), indent=2))
|
||||
else:
|
||||
click.echo(f"Has contentmatter: {stats.has_contentmatter}")
|
||||
click.echo(f"Total pairs: {stats.total_pairs}")
|
||||
click.echo(f"Average key length: {stats.average_key_length:.1f}")
|
||||
click.echo(f"Average value length: {stats.average_value_length:.1f}")
|
||||
click.echo(f"URL values: {stats.url_values}")
|
||||
click.echo(f"Email values: {stats.email_values}")
|
||||
click.echo(f"Date values: {stats.date_values}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to calculate contentmatter stats for {file_path}")
|
||||
207
markitect/matter_contentmatter/parser.py
Normal file
207
markitect/matter_contentmatter/parser.py
Normal file
@@ -0,0 +1,207 @@
|
||||
"""
|
||||
Contentmatter parser for extracting and manipulating MultiMarkdown key-value pairs within content.
|
||||
"""
|
||||
|
||||
import re
|
||||
from typing import Dict, List, Optional
|
||||
from .stats import ContentmatterStats
|
||||
|
||||
|
||||
class ContentmatterParser:
|
||||
"""Parser for contentmatter (MultiMarkdown key-value pairs) in MarkdownMatters documents."""
|
||||
|
||||
def extract_contentmatter(self, text: str) -> Dict[str, str]:
|
||||
"""
|
||||
Extract contentmatter (MMD key-value pairs) from content only.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
Dictionary containing contentmatter key-value pairs
|
||||
"""
|
||||
# First extract only the content (remove frontmatter and tailmatter)
|
||||
content = self._extract_content_only(text)
|
||||
|
||||
# Find all MMD key-value pairs in content
|
||||
return self._parse_mmd_keyvalues(content)
|
||||
|
||||
def get_contentmatter_value(self, text: str, key: str) -> Optional[str]:
|
||||
"""
|
||||
Get specific contentmatter value by key.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
key: Key to retrieve
|
||||
|
||||
Returns:
|
||||
Value or None if not found
|
||||
"""
|
||||
contentmatter = self.extract_contentmatter(text)
|
||||
return contentmatter.get(key)
|
||||
|
||||
def set_contentmatter_value(self, text: str, key: str, value: str) -> str:
|
||||
"""
|
||||
Set a contentmatter value in the document.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
key: Key to set
|
||||
value: Value to set
|
||||
|
||||
Returns:
|
||||
Updated document text
|
||||
"""
|
||||
# Extract content part to work with
|
||||
content = self._extract_content_only(text)
|
||||
|
||||
# Check if key already exists
|
||||
existing_pattern = rf'^{re.escape(key)}:\s*.*$'
|
||||
|
||||
if re.search(existing_pattern, content, re.MULTILINE):
|
||||
# Update existing key
|
||||
new_line = f"{key}: {value}"
|
||||
content = re.sub(existing_pattern, new_line, content, flags=re.MULTILINE)
|
||||
else:
|
||||
# Add new key-value pair after first heading or at start
|
||||
new_line = f"{key}: {value}\n"
|
||||
|
||||
# Find first heading to add after it
|
||||
heading_match = re.search(r'^(#+\s+.*?)$', content, re.MULTILINE)
|
||||
if heading_match:
|
||||
insert_pos = heading_match.end()
|
||||
content = content[:insert_pos] + "\n\n" + new_line + content[insert_pos:]
|
||||
else:
|
||||
# Add at beginning of content
|
||||
content = new_line + "\n" + content
|
||||
|
||||
# Reconstruct full document
|
||||
return self._reconstruct_document(text, content)
|
||||
|
||||
def get_contentmatter_keys(self, text: str) -> List[str]:
|
||||
"""
|
||||
Get list of contentmatter keys.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
List of contentmatter keys
|
||||
"""
|
||||
contentmatter = self.extract_contentmatter(text)
|
||||
return list(contentmatter.keys())
|
||||
|
||||
def calculate_contentmatter_stats(self, text: str) -> ContentmatterStats:
|
||||
"""
|
||||
Calculate statistics for contentmatter.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
ContentmatterStats object
|
||||
"""
|
||||
contentmatter = self.extract_contentmatter(text)
|
||||
|
||||
if not contentmatter:
|
||||
return ContentmatterStats(
|
||||
has_contentmatter=False,
|
||||
total_pairs=0,
|
||||
average_key_length=0.0,
|
||||
average_value_length=0.0,
|
||||
url_values=0,
|
||||
email_values=0,
|
||||
date_values=0
|
||||
)
|
||||
|
||||
# Calculate basic stats
|
||||
total_pairs = len(contentmatter)
|
||||
key_lengths = [len(key) for key in contentmatter.keys()]
|
||||
value_lengths = [len(value) for value in contentmatter.values()]
|
||||
|
||||
avg_key_length = sum(key_lengths) / len(key_lengths) if key_lengths else 0.0
|
||||
avg_value_length = sum(value_lengths) / len(value_lengths) if value_lengths else 0.0
|
||||
|
||||
# Analyze value types
|
||||
url_values = self._count_url_values(contentmatter)
|
||||
email_values = self._count_email_values(contentmatter)
|
||||
date_values = self._count_date_values(contentmatter)
|
||||
|
||||
return ContentmatterStats(
|
||||
has_contentmatter=True,
|
||||
total_pairs=total_pairs,
|
||||
average_key_length=avg_key_length,
|
||||
average_value_length=avg_value_length,
|
||||
url_values=url_values,
|
||||
email_values=email_values,
|
||||
date_values=date_values
|
||||
)
|
||||
|
||||
def _extract_content_only(self, text: str) -> str:
|
||||
"""Extract only content, removing frontmatter and tailmatter."""
|
||||
# Remove frontmatter
|
||||
content = re.sub(r'^---\s*\n.*?\n---\s*\n', '', text, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
# Remove tailmatter
|
||||
content = re.sub(r'\n---\s*\n\s*```(?:yaml|json)\s+tailmatter\s*\n.*?```\s*$', '', content, flags=re.DOTALL | re.MULTILINE)
|
||||
content = re.sub(r'\n\s*```(?:yaml|json)\s+tailmatter\s*\n.*?```\s*$', '', content, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
return content.strip()
|
||||
|
||||
def _parse_mmd_keyvalues(self, content: str) -> Dict[str, str]:
|
||||
"""Parse MultiMarkdown key-value pairs from content."""
|
||||
contentmatter = {}
|
||||
|
||||
# Pattern for MMD key-value pairs: "Key: Value" on its own line
|
||||
pattern = r'^([A-Za-z][A-Za-z0-9\s]*[A-Za-z0-9]):\s*(.+)$'
|
||||
|
||||
for match in re.finditer(pattern, content, re.MULTILINE):
|
||||
key = match.group(1).strip()
|
||||
value = match.group(2).strip()
|
||||
contentmatter[key] = value
|
||||
|
||||
return contentmatter
|
||||
|
||||
def _count_url_values(self, contentmatter: Dict[str, str]) -> int:
|
||||
"""Count values that are URLs."""
|
||||
url_pattern = r'https?://'
|
||||
return sum(1 for value in contentmatter.values() if re.search(url_pattern, value))
|
||||
|
||||
def _count_email_values(self, contentmatter: Dict[str, str]) -> int:
|
||||
"""Count values that are email addresses."""
|
||||
email_pattern = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'
|
||||
return sum(1 for value in contentmatter.values() if re.search(email_pattern, value))
|
||||
|
||||
def _count_date_values(self, contentmatter: Dict[str, str]) -> int:
|
||||
"""Count values that look like dates."""
|
||||
date_patterns = [
|
||||
r'\d{4}-\d{2}-\d{2}', # YYYY-MM-DD
|
||||
r'\d{2}/\d{2}/\d{4}', # MM/DD/YYYY
|
||||
r'\d{2}-\d{2}-\d{4}', # MM-DD-YYYY
|
||||
]
|
||||
|
||||
count = 0
|
||||
for value in contentmatter.values():
|
||||
for pattern in date_patterns:
|
||||
if re.search(pattern, value):
|
||||
count += 1
|
||||
break # Count each value only once
|
||||
|
||||
return count
|
||||
|
||||
def _reconstruct_document(self, original_text: str, new_content: str) -> str:
|
||||
"""Reconstruct document with updated content."""
|
||||
# Extract frontmatter if present
|
||||
frontmatter_match = re.search(r'^(---\s*\n.*?\n---\s*\n)', original_text, flags=re.DOTALL | re.MULTILINE)
|
||||
frontmatter = frontmatter_match.group(1) if frontmatter_match else ""
|
||||
|
||||
# Extract tailmatter if present
|
||||
tailmatter_match = re.search(r'(\n---\s*\n\s*```(?:yaml|json)\s+tailmatter\s*\n.*?```\s*)$', original_text, flags=re.DOTALL | re.MULTILINE)
|
||||
if not tailmatter_match:
|
||||
tailmatter_match = re.search(r'(\n\s*```(?:yaml|json)\s+tailmatter\s*\n.*?```\s*)$', original_text, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
tailmatter = tailmatter_match.group(1) if tailmatter_match else ""
|
||||
|
||||
# Reconstruct
|
||||
result = frontmatter + new_content + tailmatter
|
||||
return result
|
||||
31
markitect/matter_contentmatter/stats.py
Normal file
31
markitect/matter_contentmatter/stats.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Contentmatter statistics data structures.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, Any
|
||||
|
||||
|
||||
@dataclass
|
||||
class ContentmatterStats:
|
||||
"""Statistics about contentmatter (MultiMarkdown key-value pairs) in a document."""
|
||||
|
||||
has_contentmatter: bool
|
||||
total_pairs: int
|
||||
average_key_length: float
|
||||
average_value_length: float
|
||||
url_values: int
|
||||
email_values: int
|
||||
date_values: int
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert stats to dictionary."""
|
||||
return {
|
||||
"has_contentmatter": self.has_contentmatter,
|
||||
"total_pairs": self.total_pairs,
|
||||
"average_key_length": self.average_key_length,
|
||||
"average_value_length": self.average_value_length,
|
||||
"url_values": self.url_values,
|
||||
"email_values": self.email_values,
|
||||
"date_values": self.date_values
|
||||
}
|
||||
9
markitect/matter_frontmatter/__init__.py
Normal file
9
markitect/matter_frontmatter/__init__.py
Normal file
@@ -0,0 +1,9 @@
|
||||
"""
|
||||
Frontmatter module for MarkdownMatters CLI.
|
||||
Handles frontmatter extraction, modification, and analysis.
|
||||
"""
|
||||
|
||||
from .parser import FrontmatterParser
|
||||
from .stats import FrontmatterStats
|
||||
|
||||
__all__ = ['FrontmatterParser', 'FrontmatterStats']
|
||||
164
markitect/matter_frontmatter/commands.py
Normal file
164
markitect/matter_frontmatter/commands.py
Normal file
@@ -0,0 +1,164 @@
|
||||
"""
|
||||
CLI commands for frontmatter operations.
|
||||
"""
|
||||
|
||||
import click
|
||||
import json
|
||||
from pathlib import Path
|
||||
from .parser import FrontmatterParser
|
||||
|
||||
|
||||
@click.command('frontmatter-get')
|
||||
@click.argument('key')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='raw', type=click.Choice(['raw', 'json']),
|
||||
help='Output format (raw or json)')
|
||||
def frontmatter_get(key, file_path, output_format):
|
||||
"""Get specific frontmatter value by key (supports dot notation for nested values)."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = FrontmatterParser()
|
||||
frontmatter = parser.extract_frontmatter(text)
|
||||
|
||||
if not frontmatter:
|
||||
click.echo("No frontmatter found in document", err=True)
|
||||
return
|
||||
|
||||
# Get value using dot notation if needed
|
||||
value = parser.get_nested_value(frontmatter, key)
|
||||
|
||||
if value is None:
|
||||
click.echo(f"Key '{key}' not found in frontmatter", err=True)
|
||||
return
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(value, indent=2))
|
||||
else:
|
||||
if isinstance(value, (dict, list)):
|
||||
click.echo(json.dumps(value, indent=2))
|
||||
else:
|
||||
click.echo(str(value))
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to get frontmatter value from {file_path}")
|
||||
|
||||
|
||||
@click.command('frontmatter-set')
|
||||
@click.argument('key_value')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--backup', is_flag=True, help='Create backup of original file')
|
||||
def frontmatter_set(key_value, file_path, backup):
|
||||
"""Set frontmatter value (format: key=value, supports dot notation for nested)."""
|
||||
try:
|
||||
if '=' not in key_value:
|
||||
raise click.ClickException("Key-value must be in format 'key=value'")
|
||||
|
||||
key, value = key_value.split('=', 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
|
||||
# Try to parse value as JSON for complex types
|
||||
try:
|
||||
# Handle boolean and number values
|
||||
if value.lower() in ['true', 'false']:
|
||||
value = value.lower() == 'true'
|
||||
elif value.replace('.', '').replace('-', '').isdigit():
|
||||
value = float(value) if '.' in value else int(value)
|
||||
elif value.startswith('[') or value.startswith('{'):
|
||||
value = json.loads(value)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
# Keep as string if parsing fails
|
||||
pass
|
||||
|
||||
file_path = Path(file_path)
|
||||
|
||||
# Create backup if requested
|
||||
if backup:
|
||||
backup_path = file_path.with_suffix(f"{file_path.suffix}.bak")
|
||||
backup_path.write_text(file_path.read_text())
|
||||
click.echo(f"Backup created: {backup_path}")
|
||||
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = FrontmatterParser()
|
||||
new_text = parser.set_frontmatter_value(text, key, value)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write(new_text)
|
||||
|
||||
click.echo(f"Set {key}={value} in {file_path}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to set frontmatter value in {file_path}")
|
||||
|
||||
|
||||
@click.command('frontmatter-keys')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--nested', is_flag=True, help='Include nested keys with dot notation')
|
||||
@click.option('--format', 'output_format', default='list', type=click.Choice(['list', 'json']),
|
||||
help='Output format (list or json)')
|
||||
def frontmatter_keys(file_path, nested, output_format):
|
||||
"""List all frontmatter keys."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = FrontmatterParser()
|
||||
keys = parser.get_frontmatter_keys(text, include_nested=nested)
|
||||
|
||||
if not keys:
|
||||
click.echo("No frontmatter keys found")
|
||||
return
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(keys, indent=2))
|
||||
else:
|
||||
for key in sorted(keys):
|
||||
click.echo(key)
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to list frontmatter keys from {file_path}")
|
||||
|
||||
|
||||
@click.command('frontmatter-stats')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='json', type=click.Choice(['json', 'text']),
|
||||
help='Output format (json or text)')
|
||||
def frontmatter_stats(file_path, output_format):
|
||||
"""Calculate frontmatter statistics."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = FrontmatterParser()
|
||||
stats = parser.calculate_frontmatter_stats(text)
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(stats.to_dict(), indent=2))
|
||||
else:
|
||||
click.echo(f"Has frontmatter: {stats.has_frontmatter}")
|
||||
click.echo(f"Total fields: {stats.total_fields}")
|
||||
click.echo(f"Nested fields: {stats.nested_fields}")
|
||||
click.echo(f"Format: {stats.format or 'N/A'}")
|
||||
|
||||
if stats.field_types:
|
||||
click.echo("Field types:")
|
||||
for field_type, count in stats.field_types.items():
|
||||
click.echo(f" {field_type}: {count}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to calculate frontmatter stats for {file_path}")
|
||||
252
markitect/matter_frontmatter/parser.py
Normal file
252
markitect/matter_frontmatter/parser.py
Normal file
@@ -0,0 +1,252 @@
|
||||
"""
|
||||
Frontmatter parser for extracting and manipulating YAML/JSON/TOML frontmatter.
|
||||
"""
|
||||
|
||||
import re
|
||||
import yaml
|
||||
import json
|
||||
from typing import Dict, Any, List, Optional
|
||||
from .stats import FrontmatterStats
|
||||
|
||||
|
||||
class FrontmatterParser:
|
||||
"""Parser for frontmatter in MarkdownMatters documents."""
|
||||
|
||||
def extract_frontmatter(self, text: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Extract frontmatter from markdown text.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
Dictionary containing frontmatter data
|
||||
"""
|
||||
frontmatter_content = self._extract_frontmatter_content(text)
|
||||
|
||||
if not frontmatter_content:
|
||||
return {}
|
||||
|
||||
# Try to parse as YAML first (most common)
|
||||
try:
|
||||
return yaml.safe_load(frontmatter_content) or {}
|
||||
except yaml.YAMLError:
|
||||
pass
|
||||
|
||||
# Try to parse as JSON
|
||||
try:
|
||||
return json.loads(frontmatter_content)
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
# TODO: Add TOML support in future iterations
|
||||
|
||||
return {}
|
||||
|
||||
def set_frontmatter_value(self, text: str, key: str, value: Any) -> str:
|
||||
"""
|
||||
Set a frontmatter value in the document.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
key: Frontmatter key (supports dot notation for nested)
|
||||
value: Value to set
|
||||
|
||||
Returns:
|
||||
Updated document text
|
||||
"""
|
||||
frontmatter = self.extract_frontmatter(text)
|
||||
|
||||
# Handle nested keys with dot notation
|
||||
if '.' in key:
|
||||
self._set_nested_value(frontmatter, key, value)
|
||||
else:
|
||||
frontmatter[key] = value
|
||||
|
||||
# Replace or add frontmatter block
|
||||
return self._update_frontmatter_in_text(text, frontmatter)
|
||||
|
||||
def get_frontmatter_keys(self, text: str, include_nested: bool = False) -> List[str]:
|
||||
"""
|
||||
Get list of frontmatter keys.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
include_nested: Include nested keys with dot notation
|
||||
|
||||
Returns:
|
||||
List of frontmatter keys
|
||||
"""
|
||||
frontmatter = self.extract_frontmatter(text)
|
||||
|
||||
if not include_nested:
|
||||
return list(frontmatter.keys())
|
||||
|
||||
return self._get_all_keys_recursive(frontmatter)
|
||||
|
||||
def get_nested_value(self, frontmatter: Dict[str, Any], key: str) -> Any:
|
||||
"""
|
||||
Get nested value using dot notation.
|
||||
|
||||
Args:
|
||||
frontmatter: Frontmatter dictionary
|
||||
key: Key with dot notation (e.g., "nested.category")
|
||||
|
||||
Returns:
|
||||
Value or None if not found
|
||||
"""
|
||||
keys = key.split('.')
|
||||
current = frontmatter
|
||||
|
||||
for k in keys:
|
||||
if isinstance(current, dict) and k in current:
|
||||
current = current[k]
|
||||
else:
|
||||
return None
|
||||
|
||||
return current
|
||||
|
||||
def calculate_frontmatter_stats(self, text: str) -> FrontmatterStats:
|
||||
"""
|
||||
Calculate statistics for frontmatter.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
FrontmatterStats object
|
||||
"""
|
||||
frontmatter = self.extract_frontmatter(text)
|
||||
|
||||
if not frontmatter:
|
||||
return FrontmatterStats(
|
||||
has_frontmatter=False,
|
||||
total_fields=0,
|
||||
nested_fields=0,
|
||||
format=None,
|
||||
field_types={}
|
||||
)
|
||||
|
||||
# Detect format
|
||||
format_type = self._detect_frontmatter_format(text)
|
||||
|
||||
# Count fields
|
||||
total_fields = len(frontmatter)
|
||||
nested_fields = self._count_nested_fields(frontmatter)
|
||||
|
||||
# Analyze field types
|
||||
field_types = self._analyze_field_types(frontmatter)
|
||||
|
||||
return FrontmatterStats(
|
||||
has_frontmatter=True,
|
||||
total_fields=total_fields,
|
||||
nested_fields=nested_fields,
|
||||
format=format_type,
|
||||
field_types=field_types
|
||||
)
|
||||
|
||||
def _extract_frontmatter_content(self, text: str) -> Optional[str]:
|
||||
"""Extract the raw frontmatter content between delimiters."""
|
||||
# Pattern for YAML frontmatter (---...---)
|
||||
yaml_pattern = r'^---\s*\n(.*?)\n---\s*\n'
|
||||
|
||||
match = re.search(yaml_pattern, text, flags=re.DOTALL | re.MULTILINE)
|
||||
if match:
|
||||
return match.group(1).strip()
|
||||
|
||||
return None
|
||||
|
||||
def _detect_frontmatter_format(self, text: str) -> Optional[str]:
|
||||
"""Detect the format of frontmatter (yaml, json, toml)."""
|
||||
content = self._extract_frontmatter_content(text)
|
||||
if not content:
|
||||
return None
|
||||
|
||||
# Simple heuristics for format detection
|
||||
content = content.strip()
|
||||
if content.startswith('{') and content.endswith('}'):
|
||||
return "json"
|
||||
else:
|
||||
# Default to YAML for now
|
||||
return "yaml"
|
||||
|
||||
def _set_nested_value(self, data: Dict[str, Any], key: str, value: Any) -> None:
|
||||
"""Set nested value using dot notation."""
|
||||
keys = key.split('.')
|
||||
current = data
|
||||
|
||||
# Navigate to the parent of the final key
|
||||
for k in keys[:-1]:
|
||||
if k not in current:
|
||||
current[k] = {}
|
||||
current = current[k]
|
||||
|
||||
# Set the final value
|
||||
current[keys[-1]] = value
|
||||
|
||||
def _get_all_keys_recursive(self, data: Dict[str, Any], prefix: str = "") -> List[str]:
|
||||
"""Get all keys recursively with dot notation."""
|
||||
keys = []
|
||||
|
||||
for key, value in data.items():
|
||||
full_key = f"{prefix}.{key}" if prefix else key
|
||||
keys.append(full_key)
|
||||
|
||||
if isinstance(value, dict):
|
||||
keys.extend(self._get_all_keys_recursive(value, full_key))
|
||||
|
||||
return keys
|
||||
|
||||
def _count_nested_fields(self, data: Dict[str, Any]) -> int:
|
||||
"""Count nested fields recursively."""
|
||||
count = 0
|
||||
|
||||
for value in data.values():
|
||||
if isinstance(value, dict):
|
||||
count += len(value)
|
||||
count += self._count_nested_fields(value)
|
||||
|
||||
return count
|
||||
|
||||
def _analyze_field_types(self, data: Dict[str, Any]) -> Dict[str, int]:
|
||||
"""Analyze field types in frontmatter."""
|
||||
type_counts = {}
|
||||
|
||||
def count_types(obj):
|
||||
if isinstance(obj, dict):
|
||||
type_counts["object"] = type_counts.get("object", 0) + 1
|
||||
for v in obj.values():
|
||||
count_types(v)
|
||||
elif isinstance(obj, list):
|
||||
type_counts["array"] = type_counts.get("array", 0) + 1
|
||||
for item in obj:
|
||||
count_types(item)
|
||||
elif isinstance(obj, bool):
|
||||
type_counts["boolean"] = type_counts.get("boolean", 0) + 1
|
||||
elif isinstance(obj, (int, float)):
|
||||
type_counts["number"] = type_counts.get("number", 0) + 1
|
||||
elif isinstance(obj, str):
|
||||
type_counts["string"] = type_counts.get("string", 0) + 1
|
||||
|
||||
# Count top-level fields only for now
|
||||
for value in data.values():
|
||||
count_types(value)
|
||||
|
||||
return type_counts
|
||||
|
||||
def _update_frontmatter_in_text(self, text: str, frontmatter: Dict[str, Any]) -> str:
|
||||
"""Update or add frontmatter block in text."""
|
||||
# Convert frontmatter to YAML
|
||||
frontmatter_yaml = yaml.dump(frontmatter, default_flow_style=False)
|
||||
|
||||
# Check if text already has frontmatter
|
||||
yaml_pattern = r'^---\s*\n.*?\n---\s*\n'
|
||||
|
||||
if re.search(yaml_pattern, text, flags=re.DOTALL | re.MULTILINE):
|
||||
# Replace existing frontmatter
|
||||
new_frontmatter = f"---\n{frontmatter_yaml}---\n"
|
||||
return re.sub(yaml_pattern, new_frontmatter, text, flags=re.DOTALL | re.MULTILINE)
|
||||
else:
|
||||
# Add frontmatter to beginning
|
||||
new_frontmatter = f"---\n{frontmatter_yaml}---\n\n"
|
||||
return new_frontmatter + text
|
||||
27
markitect/matter_frontmatter/stats.py
Normal file
27
markitect/matter_frontmatter/stats.py
Normal file
@@ -0,0 +1,27 @@
|
||||
"""
|
||||
Frontmatter statistics data structures.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class FrontmatterStats:
|
||||
"""Statistics about frontmatter in a markdown document."""
|
||||
|
||||
has_frontmatter: bool
|
||||
total_fields: int
|
||||
nested_fields: int
|
||||
format: Optional[str] # "yaml", "json", "toml", None
|
||||
field_types: Dict[str, int] # Count of each data type
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert stats to dictionary."""
|
||||
return {
|
||||
"has_frontmatter": self.has_frontmatter,
|
||||
"total_fields": self.total_fields,
|
||||
"nested_fields": self.nested_fields,
|
||||
"format": self.format,
|
||||
"field_types": self.field_types
|
||||
}
|
||||
9
markitect/matter_tailmatter/__init__.py
Normal file
9
markitect/matter_tailmatter/__init__.py
Normal file
@@ -0,0 +1,9 @@
|
||||
"""
|
||||
Tailmatter module for MarkdownMatters CLI.
|
||||
Handles tailmatter extraction, QA checklists, editorial workflow, and agent configuration.
|
||||
"""
|
||||
|
||||
from .parser import TailmatterParser
|
||||
from .stats import TailmatterStats
|
||||
|
||||
__all__ = ['TailmatterParser', 'TailmatterStats']
|
||||
199
markitect/matter_tailmatter/commands.py
Normal file
199
markitect/matter_tailmatter/commands.py
Normal file
@@ -0,0 +1,199 @@
|
||||
"""
|
||||
CLI commands for tailmatter operations.
|
||||
"""
|
||||
|
||||
import click
|
||||
import json
|
||||
from pathlib import Path
|
||||
from .parser import TailmatterParser
|
||||
|
||||
|
||||
@click.command('tailmatter-get')
|
||||
@click.argument('key')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='raw', type=click.Choice(['raw', 'json']),
|
||||
help='Output format (raw or json)')
|
||||
def tailmatter_get(key, file_path, output_format):
|
||||
"""Get specific tailmatter value by key (supports dot notation for nested values)."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = TailmatterParser()
|
||||
value = parser.get_tailmatter_value(text, key)
|
||||
|
||||
if value is None:
|
||||
click.echo(f"Key '{key}' not found in tailmatter", err=True)
|
||||
return
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(value, indent=2))
|
||||
else:
|
||||
if isinstance(value, (dict, list)):
|
||||
click.echo(json.dumps(value, indent=2))
|
||||
else:
|
||||
click.echo(str(value))
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to get tailmatter value from {file_path}")
|
||||
|
||||
|
||||
@click.command('tailmatter-set')
|
||||
@click.argument('key_value')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--backup', is_flag=True, help='Create backup of original file')
|
||||
def tailmatter_set(key_value, file_path, backup):
|
||||
"""Set tailmatter value (format: key=value, supports dot notation for nested)."""
|
||||
try:
|
||||
if '=' not in key_value:
|
||||
raise click.ClickException("Key-value must be in format 'key=value'")
|
||||
|
||||
key, value = key_value.split('=', 1)
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
|
||||
# Try to parse value as JSON for complex types
|
||||
try:
|
||||
if value.lower() in ['true', 'false']:
|
||||
value = value.lower() == 'true'
|
||||
elif value.replace('.', '').replace('-', '').isdigit():
|
||||
value = float(value) if '.' in value else int(value)
|
||||
elif value.startswith('[') or value.startswith('{'):
|
||||
value = json.loads(value)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
pass
|
||||
|
||||
file_path = Path(file_path)
|
||||
|
||||
if backup:
|
||||
backup_path = file_path.with_suffix(f"{file_path.suffix}.bak")
|
||||
backup_path.write_text(file_path.read_text())
|
||||
click.echo(f"Backup created: {backup_path}")
|
||||
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = TailmatterParser()
|
||||
new_text = parser.set_tailmatter_value(text, key, value)
|
||||
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
f.write(new_text)
|
||||
|
||||
click.echo(f"Set {key}={value} in tailmatter for {file_path}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to set tailmatter value in {file_path}")
|
||||
|
||||
|
||||
@click.command('tailmatter-keys')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='list', type=click.Choice(['list', 'json']),
|
||||
help='Output format (list or json)')
|
||||
def tailmatter_keys(file_path, output_format):
|
||||
"""List all tailmatter keys."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = TailmatterParser()
|
||||
keys = parser.get_tailmatter_keys(text)
|
||||
|
||||
if not keys:
|
||||
click.echo("No tailmatter keys found")
|
||||
return
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(keys, indent=2))
|
||||
else:
|
||||
for key in sorted(keys):
|
||||
click.echo(key)
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to list tailmatter keys from {file_path}")
|
||||
|
||||
|
||||
@click.command('tailmatter-stats')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
@click.option('--format', 'output_format', default='json', type=click.Choice(['json', 'text']),
|
||||
help='Output format (json or text)')
|
||||
def tailmatter_stats(file_path, output_format):
|
||||
"""Calculate tailmatter statistics."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = TailmatterParser()
|
||||
stats = parser.calculate_tailmatter_stats(text)
|
||||
|
||||
if output_format == 'json':
|
||||
click.echo(json.dumps(stats.to_dict(), indent=2))
|
||||
else:
|
||||
click.echo(f"Has tailmatter: {stats.has_tailmatter}")
|
||||
click.echo(f"Format: {stats.format or 'N/A'}")
|
||||
click.echo(f"Total fields: {stats.total_fields}")
|
||||
click.echo(f"QA items: {stats.qa_items}")
|
||||
click.echo(f"QA completed: {stats.qa_completed}")
|
||||
click.echo(f"Editorial status: {stats.editorial_status or 'N/A'}")
|
||||
click.echo(f"Has agent config: {stats.has_agent_config}")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to calculate tailmatter stats for {file_path}")
|
||||
|
||||
|
||||
@click.command('tailmatter-check')
|
||||
@click.option('--file', 'file_path', required=True, type=click.Path(exists=True),
|
||||
help='Path to markdown file')
|
||||
def tailmatter_check(file_path):
|
||||
"""Run QA checklist validation."""
|
||||
try:
|
||||
file_path = Path(file_path)
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
text = f.read()
|
||||
|
||||
parser = TailmatterParser()
|
||||
tailmatter = parser.extract_tailmatter(text)
|
||||
|
||||
qa_checklist = tailmatter.get("qa_checklist", [])
|
||||
if not qa_checklist:
|
||||
click.echo("No QA checklist found in tailmatter")
|
||||
return
|
||||
|
||||
click.echo("QA Checklist Status:")
|
||||
click.echo("=" * 50)
|
||||
|
||||
total_items = len(qa_checklist)
|
||||
completed_items = 0
|
||||
|
||||
for i, item in enumerate(qa_checklist, 1):
|
||||
if isinstance(item, dict):
|
||||
requirement = item.get("requirement", f"Item {i}")
|
||||
complete = item.get("complete", False)
|
||||
|
||||
status_icon = "✅" if complete else "❌"
|
||||
click.echo(f"{status_icon} {requirement}")
|
||||
|
||||
if complete:
|
||||
completed_items += 1
|
||||
|
||||
click.echo("=" * 50)
|
||||
click.echo(f"Progress: {completed_items}/{total_items} ({completed_items/total_items*100:.1f}%)")
|
||||
|
||||
if completed_items == total_items:
|
||||
click.echo("🎉 All QA items completed!")
|
||||
else:
|
||||
click.echo(f"⚠️ {total_items - completed_items} items remaining")
|
||||
|
||||
except Exception as e:
|
||||
click.echo(f"Error: {e}", err=True)
|
||||
raise click.ClickException(f"Failed to check QA status for {file_path}")
|
||||
255
markitect/matter_tailmatter/parser.py
Normal file
255
markitect/matter_tailmatter/parser.py
Normal file
@@ -0,0 +1,255 @@
|
||||
"""
|
||||
Tailmatter parser for extracting and manipulating YAML/JSON tailmatter blocks.
|
||||
"""
|
||||
|
||||
import re
|
||||
import yaml
|
||||
import json
|
||||
from typing import Dict, Any, List, Optional
|
||||
from .stats import TailmatterStats
|
||||
|
||||
|
||||
class TailmatterParser:
|
||||
"""Parser for tailmatter in MarkdownMatters documents."""
|
||||
|
||||
def extract_tailmatter(self, text: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Extract tailmatter from markdown text.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
Dictionary containing tailmatter data
|
||||
"""
|
||||
tailmatter_content = self._extract_tailmatter_content(text)
|
||||
|
||||
if not tailmatter_content:
|
||||
return {}
|
||||
|
||||
# Detect format and parse
|
||||
if tailmatter_content.strip().startswith('```yaml tailmatter'):
|
||||
return self._parse_yaml_tailmatter(tailmatter_content)
|
||||
elif tailmatter_content.strip().startswith('```json tailmatter'):
|
||||
return self._parse_json_tailmatter(tailmatter_content)
|
||||
|
||||
return {}
|
||||
|
||||
def get_tailmatter_value(self, text: str, key: str) -> Any:
|
||||
"""
|
||||
Get specific tailmatter value by key.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
key: Key with dot notation support
|
||||
|
||||
Returns:
|
||||
Value or None if not found
|
||||
"""
|
||||
tailmatter = self.extract_tailmatter(text)
|
||||
return self._get_nested_value(tailmatter, key)
|
||||
|
||||
def set_tailmatter_value(self, text: str, key: str, value: Any) -> str:
|
||||
"""
|
||||
Set a tailmatter value in the document.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
key: Key to set (supports dot notation)
|
||||
value: Value to set
|
||||
|
||||
Returns:
|
||||
Updated document text
|
||||
"""
|
||||
tailmatter = self.extract_tailmatter(text)
|
||||
self._set_nested_value(tailmatter, key, value)
|
||||
return self._update_tailmatter_in_text(text, tailmatter)
|
||||
|
||||
def get_tailmatter_keys(self, text: str) -> List[str]:
|
||||
"""
|
||||
Get list of tailmatter keys.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
List of tailmatter keys
|
||||
"""
|
||||
tailmatter = self.extract_tailmatter(text)
|
||||
return self._get_all_keys_recursive(tailmatter)
|
||||
|
||||
def calculate_tailmatter_stats(self, text: str) -> TailmatterStats:
|
||||
"""
|
||||
Calculate statistics for tailmatter.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
TailmatterStats object
|
||||
"""
|
||||
tailmatter = self.extract_tailmatter(text)
|
||||
|
||||
if not tailmatter:
|
||||
return TailmatterStats(
|
||||
has_tailmatter=False,
|
||||
format=None,
|
||||
total_fields=0,
|
||||
qa_items=0,
|
||||
qa_completed=0,
|
||||
editorial_status=None,
|
||||
has_agent_config=False
|
||||
)
|
||||
|
||||
# Analyze tailmatter structure
|
||||
format_type = self._detect_tailmatter_format(text)
|
||||
total_fields = len(tailmatter)
|
||||
|
||||
# Analyze QA checklist
|
||||
qa_items, qa_completed = self._analyze_qa_checklist(tailmatter)
|
||||
|
||||
# Get editorial status
|
||||
editorial_status = self._get_editorial_status(tailmatter)
|
||||
|
||||
# Check for agent config
|
||||
has_agent_config = "agent_config" in tailmatter
|
||||
|
||||
return TailmatterStats(
|
||||
has_tailmatter=True,
|
||||
format=format_type,
|
||||
total_fields=total_fields,
|
||||
qa_items=qa_items,
|
||||
qa_completed=qa_completed,
|
||||
editorial_status=editorial_status,
|
||||
has_agent_config=has_agent_config
|
||||
)
|
||||
|
||||
def _extract_tailmatter_content(self, text: str) -> Optional[str]:
|
||||
"""Extract the raw tailmatter content."""
|
||||
# Look for tailmatter pattern at end of document
|
||||
pattern = r'\n---\s*\n\s*(```(?:yaml|json)\s+tailmatter\s*\n.*?```)\s*$'
|
||||
match = re.search(pattern, text, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
# Also check without preceding ---
|
||||
pattern = r'\n\s*(```(?:yaml|json)\s+tailmatter\s*\n.*?```)\s*$'
|
||||
match = re.search(pattern, text, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
return None
|
||||
|
||||
def _parse_yaml_tailmatter(self, content: str) -> Dict[str, Any]:
|
||||
"""Parse YAML tailmatter content."""
|
||||
# Extract YAML content between delimiters
|
||||
match = re.search(r'```yaml\s+tailmatter\s*\n(.*?)\n```', content, flags=re.DOTALL)
|
||||
if not match:
|
||||
return {}
|
||||
|
||||
yaml_content = match.group(1)
|
||||
try:
|
||||
return yaml.safe_load(yaml_content) or {}
|
||||
except yaml.YAMLError:
|
||||
return {}
|
||||
|
||||
def _parse_json_tailmatter(self, content: str) -> Dict[str, Any]:
|
||||
"""Parse JSON tailmatter content."""
|
||||
# Extract JSON content between delimiters
|
||||
match = re.search(r'```json\s+tailmatter\s*\n(.*?)\n```', content, flags=re.DOTALL)
|
||||
if not match:
|
||||
return {}
|
||||
|
||||
json_content = match.group(1)
|
||||
try:
|
||||
return json.loads(json_content)
|
||||
except json.JSONDecodeError:
|
||||
return {}
|
||||
|
||||
def _detect_tailmatter_format(self, text: str) -> Optional[str]:
|
||||
"""Detect the format of tailmatter."""
|
||||
content = self._extract_tailmatter_content(text)
|
||||
if not content:
|
||||
return None
|
||||
|
||||
if 'yaml tailmatter' in content:
|
||||
return "yaml"
|
||||
elif 'json tailmatter' in content:
|
||||
return "json"
|
||||
|
||||
return None
|
||||
|
||||
def _get_nested_value(self, data: Dict[str, Any], key: str) -> Any:
|
||||
"""Get nested value using dot notation."""
|
||||
keys = key.split('.')
|
||||
current = data
|
||||
|
||||
for k in keys:
|
||||
if isinstance(current, dict) and k in current:
|
||||
current = current[k]
|
||||
else:
|
||||
return None
|
||||
|
||||
return current
|
||||
|
||||
def _set_nested_value(self, data: Dict[str, Any], key: str, value: Any) -> None:
|
||||
"""Set nested value using dot notation."""
|
||||
keys = key.split('.')
|
||||
current = data
|
||||
|
||||
for k in keys[:-1]:
|
||||
if k not in current:
|
||||
current[k] = {}
|
||||
current = current[k]
|
||||
|
||||
current[keys[-1]] = value
|
||||
|
||||
def _get_all_keys_recursive(self, data: Dict[str, Any], prefix: str = "") -> List[str]:
|
||||
"""Get all keys recursively with dot notation."""
|
||||
keys = []
|
||||
|
||||
for key, value in data.items():
|
||||
full_key = f"{prefix}.{key}" if prefix else key
|
||||
keys.append(full_key)
|
||||
|
||||
if isinstance(value, dict):
|
||||
keys.extend(self._get_all_keys_recursive(value, full_key))
|
||||
|
||||
return keys
|
||||
|
||||
def _analyze_qa_checklist(self, tailmatter: Dict[str, Any]) -> tuple:
|
||||
"""Analyze QA checklist items."""
|
||||
qa_checklist = tailmatter.get("qa_checklist", [])
|
||||
if not isinstance(qa_checklist, list):
|
||||
return 0, 0
|
||||
|
||||
total_items = len(qa_checklist)
|
||||
completed_items = sum(1 for item in qa_checklist if isinstance(item, dict) and item.get("complete", False))
|
||||
|
||||
return total_items, completed_items
|
||||
|
||||
def _get_editorial_status(self, tailmatter: Dict[str, Any]) -> Optional[str]:
|
||||
"""Get editorial status."""
|
||||
editorial = tailmatter.get("editorial", {})
|
||||
if isinstance(editorial, dict):
|
||||
return editorial.get("status")
|
||||
return None
|
||||
|
||||
def _update_tailmatter_in_text(self, text: str, tailmatter: Dict[str, Any]) -> str:
|
||||
"""Update tailmatter block in text."""
|
||||
# Convert tailmatter to YAML
|
||||
tailmatter_yaml = yaml.dump(tailmatter, default_flow_style=False)
|
||||
|
||||
# Check if text already has tailmatter
|
||||
pattern = r'\n---\s*\n\s*```(?:yaml|json)\s+tailmatter\s*\n.*?```\s*$'
|
||||
|
||||
if re.search(pattern, text, flags=re.DOTALL | re.MULTILINE):
|
||||
# Replace existing tailmatter
|
||||
new_tailmatter = f"\n---\n\n```yaml tailmatter\n{tailmatter_yaml}```"
|
||||
return re.sub(pattern, new_tailmatter, text, flags=re.DOTALL | re.MULTILINE)
|
||||
else:
|
||||
# Add tailmatter to end
|
||||
new_tailmatter = f"\n\n---\n\n```yaml tailmatter\n{tailmatter_yaml}```"
|
||||
return text + new_tailmatter
|
||||
31
markitect/matter_tailmatter/stats.py
Normal file
31
markitect/matter_tailmatter/stats.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Tailmatter statistics data structures.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class TailmatterStats:
|
||||
"""Statistics about tailmatter in a markdown document."""
|
||||
|
||||
has_tailmatter: bool
|
||||
format: Optional[str] # "yaml", "json"
|
||||
total_fields: int
|
||||
qa_items: int
|
||||
qa_completed: int
|
||||
editorial_status: Optional[str]
|
||||
has_agent_config: bool
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
"""Convert stats to dictionary."""
|
||||
return {
|
||||
"has_tailmatter": self.has_tailmatter,
|
||||
"format": self.format,
|
||||
"total_fields": self.total_fields,
|
||||
"qa_items": self.qa_items,
|
||||
"qa_completed": self.qa_completed,
|
||||
"editorial_status": self.editorial_status,
|
||||
"has_agent_config": self.has_agent_config
|
||||
}
|
||||
81
tests/fixtures/contentmatter_test_files/mmd_rich_content.md
vendored
Normal file
81
tests/fixtures/contentmatter_test_files/mmd_rich_content.md
vendored
Normal file
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: "Document with Rich Contentmatter"
|
||||
---
|
||||
|
||||
# Research Paper: Advanced Algorithms
|
||||
|
||||
Author: Dr. Sarah Johnson
|
||||
Institution: MIT Computer Science Department
|
||||
Email: sarah.johnson@mit.edu
|
||||
Date: 2025-10-02
|
||||
Version: 1.3
|
||||
|
||||
## Abstract
|
||||
|
||||
Abstract: This paper presents novel approaches to algorithmic optimization in distributed systems.
|
||||
Keywords: algorithms, distributed systems, optimization, performance
|
||||
Classification: Computer Science - Distributed Computing
|
||||
|
||||
## Introduction
|
||||
|
||||
Lead Author: Dr. Sarah Johnson
|
||||
Co-Authors: Prof. Michael Chen, Dr. Lisa Wang
|
||||
Grant Number: NSF-CS-2025-001
|
||||
Funding Agency: National Science Foundation
|
||||
|
||||
The field of distributed computing has evolved significantly over the past decade. Our research focuses on optimization techniques that can reduce computational overhead while maintaining system reliability.
|
||||
|
||||
## Methodology
|
||||
|
||||
Research Method: Experimental Analysis
|
||||
Sample Size: 1000 distributed nodes
|
||||
Test Duration: 6 months
|
||||
Validation Approach: Cross-validation with industry benchmarks
|
||||
|
||||
### Experimental Setup
|
||||
|
||||
Lab Location: MIT Advanced Computing Lab
|
||||
Equipment: High-performance computing cluster
|
||||
Software Stack: Python 3.11, Apache Spark, Kubernetes
|
||||
Data Sources: Synthetic and real-world datasets
|
||||
|
||||
The experimental methodology involved comprehensive testing across multiple distributed environments.
|
||||
|
||||
## Results
|
||||
|
||||
Result Status: Preliminary findings confirmed
|
||||
Performance Improvement: 23% average speedup
|
||||
Statistical Significance: p < 0.001
|
||||
Confidence Interval: 95%
|
||||
|
||||
Our findings demonstrate significant improvements in processing efficiency across all tested scenarios.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Publication Status: Under review
|
||||
Target Journal: ACM Transactions on Computer Systems
|
||||
Submission Date: 2025-09-15
|
||||
Expected Publication: Q2 2026
|
||||
|
||||
The research contributes to the understanding of algorithmic optimization in distributed environments.
|
||||
|
||||
---
|
||||
|
||||
```yaml tailmatter
|
||||
qa_checklist:
|
||||
- requirement: "All citations properly formatted"
|
||||
complete: true
|
||||
- requirement: "Statistical analysis verified"
|
||||
complete: true
|
||||
- requirement: "Peer review completed"
|
||||
complete: false
|
||||
|
||||
editorial:
|
||||
status: "In Review"
|
||||
reviewer: "editorial.board@journal.com"
|
||||
submission_id: "TOCS-2025-0142"
|
||||
|
||||
agent_config:
|
||||
role: "academic_paper_reviewer"
|
||||
focus: "methodology and statistical analysis"
|
||||
```
|
||||
11
tests/fixtures/contentmatter_test_files/no_contentmatter.md
vendored
Normal file
11
tests/fixtures/contentmatter_test_files/no_contentmatter.md
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
# Document Without Contentmatter
|
||||
|
||||
This document contains no MultiMarkdown key-value pairs within the content.
|
||||
|
||||
It has regular paragraphs and sections, but no lines in the format "Key: Value".
|
||||
|
||||
## Regular Content
|
||||
|
||||
Just normal markdown content here. No special metadata embedded within the text.
|
||||
|
||||
The contentmatter commands should handle this gracefully by returning empty results.
|
||||
24
tests/fixtures/contentmatter_test_files/simple_contentmatter.md
vendored
Normal file
24
tests/fixtures/contentmatter_test_files/simple_contentmatter.md
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
# Simple Document with Contentmatter
|
||||
|
||||
This document demonstrates basic MultiMarkdown key-value usage.
|
||||
|
||||
Author: Jane Smith
|
||||
Project: Contentmatter Testing
|
||||
Version: 2.0
|
||||
Status: Active
|
||||
|
||||
## Basic Information
|
||||
|
||||
The contentmatter above provides metadata within the content flow.
|
||||
|
||||
License: MIT
|
||||
Repository: https://github.com/example/project
|
||||
Documentation: https://docs.example.com
|
||||
|
||||
## Usage Notes
|
||||
|
||||
Updated: 2025-10-02
|
||||
Reviewer: John Doe
|
||||
Category: Testing
|
||||
|
||||
This demonstrates various types of contentmatter values that should be extractable.
|
||||
14
tests/fixtures/frontmatter_test_files/empty_frontmatter.md
vendored
Normal file
14
tests/fixtures/frontmatter_test_files/empty_frontmatter.md
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
---
|
||||
|
||||
# Document With Empty Frontmatter
|
||||
|
||||
This document has frontmatter delimiters but no actual metadata content.
|
||||
|
||||
The frontmatter commands should handle this edge case:
|
||||
- frontmatter-get should return empty/null values
|
||||
- frontmatter-keys should return empty list
|
||||
- frontmatter-stats should show zero fields
|
||||
- frontmatter-set should work to add metadata
|
||||
|
||||
This tests the parser's handling of empty frontmatter blocks.
|
||||
26
tests/fixtures/frontmatter_test_files/json_frontmatter.md
vendored
Normal file
26
tests/fixtures/frontmatter_test_files/json_frontmatter.md
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
---
|
||||
{
|
||||
"title": "JSON Frontmatter Test Document",
|
||||
"author": "Test Author",
|
||||
"date": "2025-10-02",
|
||||
"tags": ["json", "frontmatter", "testing"],
|
||||
"version": 2.1,
|
||||
"published": false,
|
||||
"config": {
|
||||
"theme": "dark",
|
||||
"language": "en",
|
||||
"features": ["toc", "search", "navigation"]
|
||||
}
|
||||
}
|
||||
---
|
||||
|
||||
# JSON Frontmatter Test Document
|
||||
|
||||
This document uses JSON format for frontmatter instead of YAML.
|
||||
|
||||
The frontmatter parser should handle JSON format correctly and extract values like:
|
||||
- title: "JSON Frontmatter Test Document"
|
||||
- config.theme: "dark"
|
||||
- config.features: ["toc", "search", "navigation"]
|
||||
|
||||
This tests the parser's ability to handle different frontmatter formats.
|
||||
11
tests/fixtures/frontmatter_test_files/no_frontmatter.md
vendored
Normal file
11
tests/fixtures/frontmatter_test_files/no_frontmatter.md
vendored
Normal file
@@ -0,0 +1,11 @@
|
||||
# Document Without Frontmatter
|
||||
|
||||
This document has no frontmatter at all. It starts directly with content.
|
||||
|
||||
The frontmatter commands should handle this gracefully:
|
||||
- frontmatter-get should return empty/null values
|
||||
- frontmatter-keys should return empty list
|
||||
- frontmatter-stats should show zero fields
|
||||
- frontmatter-set should be able to add frontmatter to the document
|
||||
|
||||
This tests edge case handling for documents without any frontmatter.
|
||||
28
tests/fixtures/frontmatter_test_files/yaml_frontmatter.md
vendored
Normal file
28
tests/fixtures/frontmatter_test_files/yaml_frontmatter.md
vendored
Normal file
@@ -0,0 +1,28 @@
|
||||
---
|
||||
title: "YAML Frontmatter Test Document"
|
||||
author: "Test Author"
|
||||
date: 2025-10-02
|
||||
tags: ["yaml", "frontmatter", "testing"]
|
||||
version: 1.2
|
||||
published: true
|
||||
description: "A test document with YAML frontmatter for testing frontmatter commands"
|
||||
nested:
|
||||
category: "documentation"
|
||||
priority: "high"
|
||||
metadata:
|
||||
creation_date: "2025-10-02"
|
||||
last_modified: "2025-10-02"
|
||||
---
|
||||
|
||||
# YAML Frontmatter Test Document
|
||||
|
||||
This document contains YAML frontmatter that should be accessible via the frontmatter commands.
|
||||
|
||||
The frontmatter includes various data types:
|
||||
- Strings (title, author)
|
||||
- Arrays (tags)
|
||||
- Numbers (version)
|
||||
- Booleans (published)
|
||||
- Nested objects (nested.category, nested.priority)
|
||||
|
||||
This content should be extracted separately from the frontmatter metadata.
|
||||
38
tests/fixtures/tailmatter_test_files/complete_tailmatter.md
vendored
Normal file
38
tests/fixtures/tailmatter_test_files/complete_tailmatter.md
vendored
Normal file
@@ -0,0 +1,38 @@
|
||||
# Document with Complete Tailmatter
|
||||
|
||||
This document demonstrates tailmatter usage according to the MarkdownMatters specification.
|
||||
|
||||
The main content goes here with various sections and information.
|
||||
|
||||
## Section 1
|
||||
|
||||
Content for section 1.
|
||||
|
||||
## Section 2
|
||||
|
||||
Content for section 2.
|
||||
|
||||
---
|
||||
|
||||
```yaml tailmatter
|
||||
qa_checklist:
|
||||
- requirement: "All headers verified"
|
||||
complete: true
|
||||
- requirement: "Links checked"
|
||||
complete: false
|
||||
- requirement: "Spelling reviewed"
|
||||
complete: true
|
||||
|
||||
editorial:
|
||||
status: "In Review"
|
||||
reviewer: "jane.doe@example.com"
|
||||
version: 2.1
|
||||
last_updated: "2025-10-02"
|
||||
approval_required: true
|
||||
|
||||
agent_config:
|
||||
role: "documentation_reviewer"
|
||||
access_scope: "content"
|
||||
focus_areas: ["grammar", "technical_accuracy"]
|
||||
output_format: "detailed_report"
|
||||
```
|
||||
429
tests/test_contentmatter_commands.py
Normal file
429
tests/test_contentmatter_commands.py
Normal file
@@ -0,0 +1,429 @@
|
||||
"""
|
||||
TDD8 Cycle 3: Contentmatter Commands Tests (RED Phase)
|
||||
Issue #38 - MarkdownMatters CLI Implementation
|
||||
|
||||
This test file implements the RED phase tests for contentmatter command family:
|
||||
- markitect contentmatter-get [key] [path] - Get MMD key-value from content
|
||||
- markitect contentmatter-set key=value [path] - Set MMD key-value in content
|
||||
- markitect contentmatter-keys [path] - List all contentmatter keys
|
||||
- markitect contentmatter-stats [path] - Contentmatter statistics
|
||||
|
||||
Following TDD8 methodology, these tests MUST FAIL initially.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from click.testing import CliRunner
|
||||
|
||||
from markitect.matter_contentmatter.parser import ContentmatterParser
|
||||
from markitect.matter_contentmatter.stats import ContentmatterStats
|
||||
from markitect.matter_contentmatter.commands import contentmatter_get, contentmatter_set, contentmatter_keys, contentmatter_stats
|
||||
|
||||
|
||||
class TestContentmatterExtraction:
|
||||
"""Test contentmatter extraction and parsing."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to contentmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def contentmatter_parser(self):
|
||||
"""Contentmatter parser instance."""
|
||||
return ContentmatterParser()
|
||||
|
||||
def test_contentmatter_parser_extracts_mmd_keyvalues(self, contentmatter_parser, test_files_dir):
|
||||
"""Test that parser extracts MultiMarkdown key-value pairs."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(text)
|
||||
|
||||
# Should extract basic key-value pairs
|
||||
assert contentmatter["Author"] == "Jane Smith"
|
||||
assert contentmatter["Project"] == "Contentmatter Testing"
|
||||
assert contentmatter["Version"] == "2.0"
|
||||
assert contentmatter["Status"] == "Active"
|
||||
assert contentmatter["License"] == "MIT"
|
||||
assert contentmatter["Repository"] == "https://github.com/example/project"
|
||||
assert contentmatter["Documentation"] == "https://docs.example.com"
|
||||
assert contentmatter["Updated"] == "2025-10-02"
|
||||
assert contentmatter["Reviewer"] == "John Doe"
|
||||
assert contentmatter["Category"] == "Testing"
|
||||
|
||||
def test_contentmatter_parser_extracts_complex_content(self, contentmatter_parser, test_files_dir):
|
||||
"""Test extraction from document with rich contentmatter."""
|
||||
file_path = test_files_dir / "mmd_rich_content.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(text)
|
||||
|
||||
# Should extract author information
|
||||
assert contentmatter["Author"] == "Dr. Sarah Johnson"
|
||||
assert contentmatter["Institution"] == "MIT Computer Science Department"
|
||||
assert contentmatter["Email"] == "sarah.johnson@mit.edu"
|
||||
|
||||
# Should extract research metadata
|
||||
assert contentmatter["Keywords"] == "algorithms, distributed systems, optimization, performance"
|
||||
assert contentmatter["Classification"] == "Computer Science - Distributed Computing"
|
||||
assert contentmatter["Grant Number"] == "NSF-CS-2025-001"
|
||||
|
||||
# Should extract methodology details
|
||||
assert contentmatter["Research Method"] == "Experimental Analysis"
|
||||
assert contentmatter["Sample Size"] == "1000 distributed nodes"
|
||||
assert contentmatter["Performance Improvement"] == "23% average speedup"
|
||||
|
||||
def test_contentmatter_parser_handles_no_contentmatter(self, contentmatter_parser, test_files_dir):
|
||||
"""Test that parser handles documents without contentmatter."""
|
||||
file_path = test_files_dir / "no_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(text)
|
||||
|
||||
# Should return empty dict for no contentmatter
|
||||
assert contentmatter == {}
|
||||
|
||||
def test_contentmatter_parser_ignores_frontmatter_and_tailmatter(self, contentmatter_parser, test_files_dir):
|
||||
"""Test that parser only extracts from content, not matter zones."""
|
||||
file_path = test_files_dir / "mmd_rich_content.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(text)
|
||||
|
||||
# Should not include frontmatter values
|
||||
assert "title" not in contentmatter # This is in frontmatter
|
||||
assert "qa_checklist" not in contentmatter # This is in tailmatter
|
||||
|
||||
# Should only include content key-values
|
||||
assert "Author" in contentmatter # This is in content
|
||||
|
||||
def test_contentmatter_get_specific_value(self, contentmatter_parser, test_files_dir):
|
||||
"""Test getting specific contentmatter values."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
value = contentmatter_parser.get_contentmatter_value(text, "Author")
|
||||
assert value == "Jane Smith"
|
||||
|
||||
value = contentmatter_parser.get_contentmatter_value(text, "Repository")
|
||||
assert value == "https://github.com/example/project"
|
||||
|
||||
# Should return None for non-existent keys
|
||||
value = contentmatter_parser.get_contentmatter_value(text, "NonExistent")
|
||||
assert value is None
|
||||
|
||||
|
||||
class TestContentmatterModification:
|
||||
"""Test contentmatter modification operations."""
|
||||
|
||||
@pytest.fixture
|
||||
def contentmatter_parser(self):
|
||||
"""Contentmatter parser instance."""
|
||||
return ContentmatterParser()
|
||||
|
||||
def test_contentmatter_set_new_value(self, contentmatter_parser):
|
||||
"""Test adding new contentmatter to document."""
|
||||
text = """# Test Document
|
||||
|
||||
Some content here.
|
||||
|
||||
## Section
|
||||
|
||||
More content."""
|
||||
|
||||
new_text = contentmatter_parser.set_contentmatter_value(text, "Author", "New Author")
|
||||
|
||||
# Should add the contentmatter
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(new_text)
|
||||
assert contentmatter["Author"] == "New Author"
|
||||
|
||||
# Should preserve original content
|
||||
assert "# Test Document" in new_text
|
||||
assert "Some content here." in new_text
|
||||
|
||||
def test_contentmatter_update_existing_value(self, contentmatter_parser):
|
||||
"""Test updating existing contentmatter value."""
|
||||
text = """# Test Document
|
||||
|
||||
Author: Original Author
|
||||
Project: Test Project
|
||||
|
||||
Some content here."""
|
||||
|
||||
new_text = contentmatter_parser.set_contentmatter_value(text, "Author", "Updated Author")
|
||||
|
||||
# Should update the existing value
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(new_text)
|
||||
assert contentmatter["Author"] == "Updated Author"
|
||||
assert contentmatter["Project"] == "Test Project" # Should preserve other values
|
||||
|
||||
def test_contentmatter_set_multiple_values(self, contentmatter_parser):
|
||||
"""Test setting multiple contentmatter values."""
|
||||
text = """# Test Document
|
||||
|
||||
Some content here."""
|
||||
|
||||
# Add multiple values
|
||||
text = contentmatter_parser.set_contentmatter_value(text, "Author", "Test Author")
|
||||
text = contentmatter_parser.set_contentmatter_value(text, "Version", "1.0")
|
||||
text = contentmatter_parser.set_contentmatter_value(text, "Status", "Active")
|
||||
|
||||
contentmatter = contentmatter_parser.extract_contentmatter(text)
|
||||
assert contentmatter["Author"] == "Test Author"
|
||||
assert contentmatter["Version"] == "1.0"
|
||||
assert contentmatter["Status"] == "Active"
|
||||
|
||||
|
||||
class TestContentmatterKeys:
|
||||
"""Test contentmatter key listing functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to contentmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def contentmatter_parser(self):
|
||||
"""Contentmatter parser instance."""
|
||||
return ContentmatterParser()
|
||||
|
||||
def test_contentmatter_keys_simple_document(self, contentmatter_parser, test_files_dir):
|
||||
"""Test listing keys from simple contentmatter document."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = contentmatter_parser.get_contentmatter_keys(text)
|
||||
|
||||
# Should return all contentmatter keys
|
||||
expected_keys = ["Author", "Project", "Version", "Status", "License", "Repository", "Documentation", "Updated", "Reviewer", "Category"]
|
||||
assert set(keys) == set(expected_keys)
|
||||
|
||||
def test_contentmatter_keys_complex_document(self, contentmatter_parser, test_files_dir):
|
||||
"""Test listing keys from complex contentmatter document."""
|
||||
file_path = test_files_dir / "mmd_rich_content.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = contentmatter_parser.get_contentmatter_keys(text)
|
||||
|
||||
# Should include research paper metadata keys
|
||||
assert "Author" in keys
|
||||
assert "Institution" in keys
|
||||
assert "Keywords" in keys
|
||||
assert "Research Method" in keys
|
||||
assert "Performance Improvement" in keys
|
||||
|
||||
# Should not include frontmatter or tailmatter keys
|
||||
assert "title" not in keys # frontmatter
|
||||
assert "qa_checklist" not in keys # tailmatter
|
||||
|
||||
def test_contentmatter_keys_empty_document(self, contentmatter_parser, test_files_dir):
|
||||
"""Test listing keys from document without contentmatter."""
|
||||
file_path = test_files_dir / "no_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = contentmatter_parser.get_contentmatter_keys(text)
|
||||
|
||||
# Should return empty list
|
||||
assert keys == []
|
||||
|
||||
|
||||
class TestContentmatterStatistics:
|
||||
"""Test contentmatter statistics calculation."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to contentmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def contentmatter_parser(self):
|
||||
"""Contentmatter parser instance."""
|
||||
return ContentmatterParser()
|
||||
|
||||
def test_contentmatter_stats_simple_document(self, contentmatter_parser, test_files_dir):
|
||||
"""Test statistics calculation for simple contentmatter."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = contentmatter_parser.calculate_contentmatter_stats(text)
|
||||
|
||||
# Should count contentmatter correctly
|
||||
assert stats.total_pairs == 10 # Number of key-value pairs
|
||||
assert stats.has_contentmatter is True
|
||||
assert stats.average_key_length > 0
|
||||
assert stats.average_value_length > 0
|
||||
|
||||
# Should categorize value types
|
||||
assert stats.url_values > 0 # Repository and Documentation URLs
|
||||
assert stats.date_values > 0 # Updated field
|
||||
assert stats.email_values == 0 # No email in simple document
|
||||
|
||||
def test_contentmatter_stats_complex_document(self, contentmatter_parser, test_files_dir):
|
||||
"""Test statistics calculation for complex contentmatter."""
|
||||
file_path = test_files_dir / "mmd_rich_content.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = contentmatter_parser.calculate_contentmatter_stats(text)
|
||||
|
||||
# Should count rich contentmatter
|
||||
assert stats.total_pairs > 15 # Many key-value pairs in research paper
|
||||
assert stats.has_contentmatter is True
|
||||
|
||||
# Should detect email values
|
||||
assert stats.email_values > 0 # Email field in author info
|
||||
|
||||
def test_contentmatter_stats_no_contentmatter(self, contentmatter_parser, test_files_dir):
|
||||
"""Test statistics for document without contentmatter."""
|
||||
file_path = test_files_dir / "no_contentmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = contentmatter_parser.calculate_contentmatter_stats(text)
|
||||
|
||||
# Should indicate no contentmatter
|
||||
assert stats.has_contentmatter is False
|
||||
assert stats.total_pairs == 0
|
||||
assert stats.url_values == 0
|
||||
assert stats.email_values == 0
|
||||
|
||||
|
||||
class TestContentmatterCLICommands:
|
||||
"""Test CLI command integration."""
|
||||
|
||||
@pytest.fixture
|
||||
def runner(self):
|
||||
"""CLI test runner."""
|
||||
return CliRunner()
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to contentmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "contentmatter_test_files"
|
||||
|
||||
def test_contentmatter_get_command(self, runner, test_files_dir):
|
||||
"""Test contentmatter-get CLI command."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
# Test getting simple value
|
||||
result = runner.invoke(contentmatter_get, ['Author', '--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "Jane Smith" in result.output
|
||||
|
||||
# Test getting URL value
|
||||
result = runner.invoke(contentmatter_get, ['Repository', '--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "https://github.com/example/project" in result.output
|
||||
|
||||
def test_contentmatter_keys_command(self, runner, test_files_dir):
|
||||
"""Test contentmatter-keys CLI command."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
result = runner.invoke(contentmatter_keys, ['--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "Author" in result.output
|
||||
assert "Project" in result.output
|
||||
assert "Repository" in result.output
|
||||
|
||||
def test_contentmatter_stats_command(self, runner, test_files_dir):
|
||||
"""Test contentmatter-stats CLI command."""
|
||||
file_path = test_files_dir / "simple_contentmatter.md"
|
||||
|
||||
result = runner.invoke(contentmatter_stats, ['--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "total_pairs" in result.output
|
||||
assert "has_contentmatter" in result.output
|
||||
|
||||
def test_contentmatter_set_command(self, runner, test_files_dir):
|
||||
"""Test contentmatter-set CLI command."""
|
||||
# Create temporary file for testing
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write("""# Test Document
|
||||
|
||||
Author: Original Author
|
||||
|
||||
Some content here.""")
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
result = runner.invoke(contentmatter_set, ['Author=New Author', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
|
||||
# Verify the change was made
|
||||
with open(temp_file, 'r') as f:
|
||||
content = f.read()
|
||||
assert "Author: New Author" in content
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_contentmatter_commands_help_text(self, runner):
|
||||
"""Test that help text is available for all contentmatter commands."""
|
||||
commands = [contentmatter_get, contentmatter_keys, contentmatter_stats, contentmatter_set]
|
||||
|
||||
for command in commands:
|
||||
result = runner.invoke(command, ['--help'])
|
||||
assert result.exit_code == 0
|
||||
assert "contentmatter" in result.output.lower()
|
||||
|
||||
|
||||
class TestContentmatterStats:
|
||||
"""Test ContentmatterStats data class."""
|
||||
|
||||
def test_contentmatter_stats_creation(self):
|
||||
"""Test ContentmatterStats object creation."""
|
||||
stats = ContentmatterStats(
|
||||
has_contentmatter=True,
|
||||
total_pairs=10,
|
||||
average_key_length=8.5,
|
||||
average_value_length=15.2,
|
||||
url_values=2,
|
||||
email_values=1,
|
||||
date_values=1
|
||||
)
|
||||
|
||||
assert stats.has_contentmatter is True
|
||||
assert stats.total_pairs == 10
|
||||
assert stats.average_key_length == 8.5
|
||||
assert stats.url_values == 2
|
||||
|
||||
def test_contentmatter_stats_to_dict(self):
|
||||
"""Test ContentmatterStats conversion to dictionary."""
|
||||
stats = ContentmatterStats(
|
||||
has_contentmatter=True,
|
||||
total_pairs=5,
|
||||
average_key_length=8.0,
|
||||
average_value_length=12.0,
|
||||
url_values=1,
|
||||
email_values=0,
|
||||
date_values=1
|
||||
)
|
||||
|
||||
stats_dict = stats.to_dict()
|
||||
|
||||
assert stats_dict["has_contentmatter"] is True
|
||||
assert stats_dict["total_pairs"] == 5
|
||||
assert stats_dict["url_values"] == 1
|
||||
428
tests/test_frontmatter_commands.py
Normal file
428
tests/test_frontmatter_commands.py
Normal file
@@ -0,0 +1,428 @@
|
||||
"""
|
||||
TDD8 Cycle 2: Frontmatter Commands Tests (RED Phase)
|
||||
Issue #38 - MarkdownMatters CLI Implementation
|
||||
|
||||
This test file implements the RED phase tests for frontmatter command family:
|
||||
- markitect frontmatter-get [key] [path] - Get specific frontmatter value
|
||||
- markitect frontmatter-set key=value [path] - Set frontmatter value
|
||||
- markitect frontmatter-keys [path] - List all frontmatter keys
|
||||
- markitect frontmatter-stats [path] - Frontmatter statistics
|
||||
|
||||
Following TDD8 methodology, these tests MUST FAIL initially.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from click.testing import CliRunner
|
||||
|
||||
from markitect.matter_frontmatter.parser import FrontmatterParser
|
||||
from markitect.matter_frontmatter.stats import FrontmatterStats
|
||||
from markitect.matter_frontmatter.commands import frontmatter_get, frontmatter_set, frontmatter_keys, frontmatter_stats
|
||||
|
||||
|
||||
class TestFrontmatterExtraction:
|
||||
"""Test frontmatter extraction and parsing."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to frontmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def frontmatter_parser(self):
|
||||
"""Frontmatter parser instance."""
|
||||
return FrontmatterParser()
|
||||
|
||||
def test_frontmatter_parser_extracts_yaml_frontmatter(self, frontmatter_parser, test_files_dir):
|
||||
"""Test that parser extracts YAML frontmatter correctly."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(text)
|
||||
|
||||
# Should extract all YAML frontmatter fields
|
||||
assert frontmatter["title"] == "YAML Frontmatter Test Document"
|
||||
assert frontmatter["author"] == "Test Author"
|
||||
assert str(frontmatter["date"]) == "2025-10-02"
|
||||
assert frontmatter["tags"] == ["yaml", "frontmatter", "testing"]
|
||||
assert frontmatter["version"] == 1.2
|
||||
assert frontmatter["published"] is True
|
||||
|
||||
# Should handle nested objects
|
||||
assert frontmatter["nested"]["category"] == "documentation"
|
||||
assert frontmatter["nested"]["priority"] == "high"
|
||||
assert frontmatter["nested"]["metadata"]["creation_date"] == "2025-10-02"
|
||||
|
||||
def test_frontmatter_parser_extracts_json_frontmatter(self, frontmatter_parser, test_files_dir):
|
||||
"""Test that parser extracts JSON frontmatter correctly."""
|
||||
file_path = test_files_dir / "json_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(text)
|
||||
|
||||
# Should extract all JSON frontmatter fields
|
||||
assert frontmatter["title"] == "JSON Frontmatter Test Document"
|
||||
assert frontmatter["author"] == "Test Author"
|
||||
assert frontmatter["tags"] == ["json", "frontmatter", "testing"]
|
||||
assert frontmatter["version"] == 2.1
|
||||
assert frontmatter["published"] is False
|
||||
|
||||
# Should handle nested objects
|
||||
assert frontmatter["config"]["theme"] == "dark"
|
||||
assert frontmatter["config"]["language"] == "en"
|
||||
assert frontmatter["config"]["features"] == ["toc", "search", "navigation"]
|
||||
|
||||
def test_frontmatter_parser_handles_no_frontmatter(self, frontmatter_parser, test_files_dir):
|
||||
"""Test that parser handles documents without frontmatter."""
|
||||
file_path = test_files_dir / "no_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(text)
|
||||
|
||||
# Should return empty dict for no frontmatter
|
||||
assert frontmatter == {}
|
||||
|
||||
def test_frontmatter_parser_handles_empty_frontmatter(self, frontmatter_parser, test_files_dir):
|
||||
"""Test that parser handles empty frontmatter blocks."""
|
||||
file_path = test_files_dir / "empty_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(text)
|
||||
|
||||
# Should return empty dict for empty frontmatter
|
||||
assert frontmatter == {}
|
||||
|
||||
def test_frontmatter_parser_get_nested_value(self, frontmatter_parser, test_files_dir):
|
||||
"""Test getting nested values using dot notation."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(text)
|
||||
|
||||
# Should support dot notation for nested access
|
||||
value = frontmatter_parser.get_nested_value(frontmatter, "nested.category")
|
||||
assert value == "documentation"
|
||||
|
||||
value = frontmatter_parser.get_nested_value(frontmatter, "nested.metadata.creation_date")
|
||||
assert value == "2025-10-02"
|
||||
|
||||
# Should return None for non-existent keys
|
||||
value = frontmatter_parser.get_nested_value(frontmatter, "non.existent.key")
|
||||
assert value is None
|
||||
|
||||
|
||||
class TestFrontmatterModification:
|
||||
"""Test frontmatter modification operations."""
|
||||
|
||||
@pytest.fixture
|
||||
def frontmatter_parser(self):
|
||||
"""Frontmatter parser instance."""
|
||||
return FrontmatterParser()
|
||||
|
||||
def test_frontmatter_set_simple_value(self, frontmatter_parser):
|
||||
"""Test setting simple frontmatter values."""
|
||||
text = """---
|
||||
title: "Original Title"
|
||||
author: "Original Author"
|
||||
---
|
||||
|
||||
# Content here"""
|
||||
|
||||
new_text = frontmatter_parser.set_frontmatter_value(text, "title", "New Title")
|
||||
|
||||
# Should update the title value
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
|
||||
assert frontmatter["title"] == "New Title"
|
||||
assert frontmatter["author"] == "Original Author"
|
||||
|
||||
def test_frontmatter_set_new_value(self, frontmatter_parser):
|
||||
"""Test adding new frontmatter values."""
|
||||
text = """---
|
||||
title: "Original Title"
|
||||
---
|
||||
|
||||
# Content here"""
|
||||
|
||||
new_text = frontmatter_parser.set_frontmatter_value(text, "author", "New Author")
|
||||
|
||||
# Should add the new field
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
|
||||
assert frontmatter["title"] == "Original Title"
|
||||
assert frontmatter["author"] == "New Author"
|
||||
|
||||
def test_frontmatter_set_nested_value(self, frontmatter_parser):
|
||||
"""Test setting nested frontmatter values using dot notation."""
|
||||
text = """---
|
||||
title: "Test"
|
||||
config:
|
||||
theme: "light"
|
||||
---
|
||||
|
||||
# Content here"""
|
||||
|
||||
new_text = frontmatter_parser.set_frontmatter_value(text, "config.theme", "dark")
|
||||
|
||||
# Should update nested value
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
|
||||
assert frontmatter["config"]["theme"] == "dark"
|
||||
|
||||
def test_frontmatter_add_to_empty_document(self, frontmatter_parser):
|
||||
"""Test adding frontmatter to document without any."""
|
||||
text = """# Content Without Frontmatter
|
||||
|
||||
Just some content here."""
|
||||
|
||||
new_text = frontmatter_parser.set_frontmatter_value(text, "title", "New Title")
|
||||
|
||||
# Should add frontmatter block
|
||||
frontmatter = frontmatter_parser.extract_frontmatter(new_text)
|
||||
assert frontmatter["title"] == "New Title"
|
||||
|
||||
# Should preserve content
|
||||
assert "# Content Without Frontmatter" in new_text
|
||||
|
||||
|
||||
class TestFrontmatterKeys:
|
||||
"""Test frontmatter key listing functionality."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to frontmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def frontmatter_parser(self):
|
||||
"""Frontmatter parser instance."""
|
||||
return FrontmatterParser()
|
||||
|
||||
def test_frontmatter_keys_yaml_document(self, frontmatter_parser, test_files_dir):
|
||||
"""Test listing keys from YAML frontmatter."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = frontmatter_parser.get_frontmatter_keys(text)
|
||||
|
||||
# Should return all top-level keys
|
||||
expected_keys = ["title", "author", "date", "tags", "version", "published", "description", "nested"]
|
||||
assert set(keys) == set(expected_keys)
|
||||
|
||||
def test_frontmatter_keys_with_nested_option(self, frontmatter_parser, test_files_dir):
|
||||
"""Test listing keys including nested keys with dot notation."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = frontmatter_parser.get_frontmatter_keys(text, include_nested=True)
|
||||
|
||||
# Should include nested keys with dot notation
|
||||
assert "nested.category" in keys
|
||||
assert "nested.priority" in keys
|
||||
assert "nested.metadata.creation_date" in keys
|
||||
assert "nested.metadata.last_modified" in keys
|
||||
|
||||
def test_frontmatter_keys_empty_document(self, frontmatter_parser, test_files_dir):
|
||||
"""Test listing keys from document without frontmatter."""
|
||||
file_path = test_files_dir / "no_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
keys = frontmatter_parser.get_frontmatter_keys(text)
|
||||
|
||||
# Should return empty list
|
||||
assert keys == []
|
||||
|
||||
|
||||
class TestFrontmatterStatistics:
|
||||
"""Test frontmatter statistics calculation."""
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to frontmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
|
||||
|
||||
@pytest.fixture
|
||||
def frontmatter_parser(self):
|
||||
"""Frontmatter parser instance."""
|
||||
return FrontmatterParser()
|
||||
|
||||
def test_frontmatter_stats_yaml_document(self, frontmatter_parser, test_files_dir):
|
||||
"""Test statistics calculation for YAML frontmatter."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = frontmatter_parser.calculate_frontmatter_stats(text)
|
||||
|
||||
# Should count fields correctly
|
||||
assert stats.total_fields == 8 # Top-level fields
|
||||
assert stats.nested_fields == 5 # Nested fields (category, priority, creation_date, last_modified, metadata object)
|
||||
assert stats.format == "yaml"
|
||||
assert stats.has_frontmatter is True
|
||||
|
||||
# Should categorize field types
|
||||
assert "string" in stats.field_types
|
||||
assert "array" in stats.field_types
|
||||
assert "number" in stats.field_types
|
||||
assert "boolean" in stats.field_types
|
||||
assert "object" in stats.field_types
|
||||
|
||||
def test_frontmatter_stats_json_document(self, frontmatter_parser, test_files_dir):
|
||||
"""Test statistics calculation for JSON frontmatter."""
|
||||
file_path = test_files_dir / "json_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = frontmatter_parser.calculate_frontmatter_stats(text)
|
||||
|
||||
# Should identify JSON format
|
||||
assert stats.format == "json"
|
||||
assert stats.has_frontmatter is True
|
||||
assert stats.total_fields > 0
|
||||
|
||||
def test_frontmatter_stats_no_frontmatter(self, frontmatter_parser, test_files_dir):
|
||||
"""Test statistics for document without frontmatter."""
|
||||
file_path = test_files_dir / "no_frontmatter.md"
|
||||
|
||||
with open(file_path, 'r') as f:
|
||||
text = f.read()
|
||||
|
||||
stats = frontmatter_parser.calculate_frontmatter_stats(text)
|
||||
|
||||
# Should indicate no frontmatter
|
||||
assert stats.has_frontmatter is False
|
||||
assert stats.total_fields == 0
|
||||
assert stats.nested_fields == 0
|
||||
assert stats.format is None
|
||||
|
||||
|
||||
class TestFrontmatterCLICommands:
|
||||
"""Test CLI command integration."""
|
||||
|
||||
@pytest.fixture
|
||||
def runner(self):
|
||||
"""CLI test runner."""
|
||||
return CliRunner()
|
||||
|
||||
@pytest.fixture
|
||||
def test_files_dir(self):
|
||||
"""Path to frontmatter test fixture files."""
|
||||
return Path(__file__).parent / "fixtures" / "frontmatter_test_files"
|
||||
|
||||
def test_frontmatter_get_command(self, runner, test_files_dir):
|
||||
"""Test frontmatter-get CLI command."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
# Test getting simple value
|
||||
result = runner.invoke(frontmatter_get, ['title', '--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "YAML Frontmatter Test Document" in result.output
|
||||
|
||||
# Test getting nested value
|
||||
result = runner.invoke(frontmatter_get, ['nested.category', '--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "documentation" in result.output
|
||||
|
||||
def test_frontmatter_keys_command(self, runner, test_files_dir):
|
||||
"""Test frontmatter-keys CLI command."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
result = runner.invoke(frontmatter_keys, ['--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "title" in result.output
|
||||
assert "author" in result.output
|
||||
assert "tags" in result.output
|
||||
|
||||
def test_frontmatter_stats_command(self, runner, test_files_dir):
|
||||
"""Test frontmatter-stats CLI command."""
|
||||
file_path = test_files_dir / "yaml_frontmatter.md"
|
||||
|
||||
result = runner.invoke(frontmatter_stats, ['--file', str(file_path)])
|
||||
assert result.exit_code == 0
|
||||
assert "total_fields" in result.output
|
||||
assert "format" in result.output
|
||||
|
||||
def test_frontmatter_set_command(self, runner, test_files_dir):
|
||||
"""Test frontmatter-set CLI command."""
|
||||
# Create temporary file for testing
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write("""---
|
||||
title: "Original Title"
|
||||
---
|
||||
|
||||
# Test Content""")
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
result = runner.invoke(frontmatter_set, ['title=New Title', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
|
||||
# Verify the change was made
|
||||
with open(temp_file, 'r') as f:
|
||||
content = f.read()
|
||||
assert "New Title" in content
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_frontmatter_commands_help_text(self, runner):
|
||||
"""Test that help text is available for all frontmatter commands."""
|
||||
commands = [frontmatter_get, frontmatter_keys, frontmatter_stats, frontmatter_set]
|
||||
|
||||
for command in commands:
|
||||
result = runner.invoke(command, ['--help'])
|
||||
assert result.exit_code == 0
|
||||
assert "frontmatter" in result.output.lower()
|
||||
|
||||
|
||||
class TestFrontmatterStats:
|
||||
"""Test FrontmatterStats data class."""
|
||||
|
||||
def test_frontmatter_stats_creation(self):
|
||||
"""Test FrontmatterStats object creation."""
|
||||
stats = FrontmatterStats(
|
||||
has_frontmatter=True,
|
||||
total_fields=5,
|
||||
nested_fields=2,
|
||||
format="yaml",
|
||||
field_types={"string": 3, "number": 1, "boolean": 1}
|
||||
)
|
||||
|
||||
assert stats.has_frontmatter is True
|
||||
assert stats.total_fields == 5
|
||||
assert stats.nested_fields == 2
|
||||
assert stats.format == "yaml"
|
||||
assert stats.field_types["string"] == 3
|
||||
|
||||
def test_frontmatter_stats_to_dict(self):
|
||||
"""Test FrontmatterStats conversion to dictionary."""
|
||||
stats = FrontmatterStats(
|
||||
has_frontmatter=True,
|
||||
total_fields=5,
|
||||
nested_fields=2,
|
||||
format="yaml",
|
||||
field_types={"string": 3}
|
||||
)
|
||||
|
||||
stats_dict = stats.to_dict()
|
||||
|
||||
assert stats_dict["has_frontmatter"] is True
|
||||
assert stats_dict["total_fields"] == 5
|
||||
assert stats_dict["format"] == "yaml"
|
||||
295
tests/test_markdownmatters_integration.py
Normal file
295
tests/test_markdownmatters_integration.py
Normal file
@@ -0,0 +1,295 @@
|
||||
"""
|
||||
Integration tests for complete MarkdownMatters CLI implementation.
|
||||
Tests all four command families working together.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from click.testing import CliRunner
|
||||
|
||||
from markitect.content.commands import content_get, content_stats
|
||||
from markitect.matter_frontmatter.commands import frontmatter_get, frontmatter_keys
|
||||
from markitect.matter_contentmatter.commands import contentmatter_get, contentmatter_keys
|
||||
from markitect.matter_tailmatter.commands import tailmatter_get, tailmatter_check
|
||||
|
||||
|
||||
class TestMarkdownMattersIntegration:
|
||||
"""Test complete MarkdownMatters functionality integration."""
|
||||
|
||||
@pytest.fixture
|
||||
def complete_document(self):
|
||||
"""A complete MarkdownMatters document with all three zones."""
|
||||
return """---
|
||||
title: "Complete MarkdownMatters Document"
|
||||
author: "Integration Test"
|
||||
version: 1.0
|
||||
status: "testing"
|
||||
---
|
||||
|
||||
# Complete MarkdownMatters Document
|
||||
|
||||
This document demonstrates all three matter zones working together.
|
||||
|
||||
Author: Dr. Test Researcher
|
||||
Institution: MarkdownMatters University
|
||||
Email: test@markdownmatters.edu
|
||||
Project: Integration Testing
|
||||
Version: 2.0
|
||||
Status: Active
|
||||
|
||||
## Research Content
|
||||
|
||||
Research Method: Integration Testing
|
||||
Sample Size: Complete document
|
||||
Test Framework: MarkdownMatters CLI
|
||||
|
||||
The content includes various MultiMarkdown key-value pairs that provide contextual metadata.
|
||||
|
||||
## Results
|
||||
|
||||
Result Status: All systems operational
|
||||
Performance: Excellent
|
||||
Coverage: 100%
|
||||
|
||||
All matter zones are properly separated and accessible through their respective CLI commands.
|
||||
|
||||
---
|
||||
|
||||
```yaml tailmatter
|
||||
qa_checklist:
|
||||
- requirement: "All three matter zones tested"
|
||||
complete: true
|
||||
- requirement: "CLI commands validated"
|
||||
complete: true
|
||||
- requirement: "Integration verified"
|
||||
complete: false
|
||||
|
||||
editorial:
|
||||
status: "Integration Testing"
|
||||
reviewer: "integration.tester@markdownmatters.edu"
|
||||
version: 3.0
|
||||
|
||||
agent_config:
|
||||
role: "integration_validator"
|
||||
access_scope: "all_zones"
|
||||
validation_mode: "comprehensive"
|
||||
```"""
|
||||
|
||||
@pytest.fixture
|
||||
def runner(self):
|
||||
"""CLI test runner."""
|
||||
return CliRunner()
|
||||
|
||||
def test_all_command_families_work_on_same_document(self, runner, complete_document):
|
||||
"""Test that all four command families can process the same document."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test content commands
|
||||
result = runner.invoke(content_get, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Complete MarkdownMatters Document" in result.output
|
||||
assert "---" not in result.output # No frontmatter
|
||||
assert "qa_checklist" not in result.output # No tailmatter
|
||||
|
||||
result = runner.invoke(content_stats, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "word_count" in result.output
|
||||
|
||||
# Test frontmatter commands
|
||||
result = runner.invoke(frontmatter_get, ['title', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Complete MarkdownMatters Document" in result.output
|
||||
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "title" in result.output
|
||||
assert "author" in result.output
|
||||
|
||||
# Test contentmatter commands
|
||||
result = runner.invoke(contentmatter_get, ['Author', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Dr. Test Researcher" in result.output
|
||||
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Author" in result.output
|
||||
assert "Institution" in result.output
|
||||
|
||||
# Test tailmatter commands
|
||||
result = runner.invoke(tailmatter_get, ['editorial.status', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Integration Testing" in result.output
|
||||
|
||||
result = runner.invoke(tailmatter_check, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "QA Checklist Status" in result.output
|
||||
assert "✅" in result.output
|
||||
assert "❌" in result.output
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_matter_zone_separation(self, runner, complete_document):
|
||||
"""Test that each command family only accesses its designated zone."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Frontmatter should not include contentmatter or tailmatter
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
assert "Author" not in result.output # This is contentmatter
|
||||
assert "qa_checklist" not in result.output # This is tailmatter
|
||||
|
||||
# Contentmatter should not include frontmatter or tailmatter
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
assert "title" not in result.output # This is frontmatter
|
||||
assert "qa_checklist" not in result.output # This is tailmatter
|
||||
|
||||
# Content should not include any matter zones in the actual content
|
||||
result = runner.invoke(content_get, ['--file', temp_file])
|
||||
assert "title:" not in result.output # No frontmatter YAML
|
||||
assert "qa_checklist:" not in result.output # No tailmatter YAML
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_performance_with_large_document(self, runner):
|
||||
"""Test performance with a large document containing all matter zones."""
|
||||
# Create a large document
|
||||
large_content = []
|
||||
large_content.append("---")
|
||||
large_content.append("title: 'Large Document Performance Test'")
|
||||
for i in range(50):
|
||||
large_content.append(f"field_{i}: 'value_{i}'")
|
||||
large_content.append("---")
|
||||
large_content.append("")
|
||||
|
||||
large_content.append("# Large Document Performance Test")
|
||||
large_content.append("")
|
||||
|
||||
# Add many contentmatter pairs
|
||||
for i in range(100):
|
||||
large_content.append(f"Data Field {i}: Value for field {i}")
|
||||
large_content.append("")
|
||||
|
||||
# Add substantial content
|
||||
for i in range(50):
|
||||
large_content.append(f"## Section {i}")
|
||||
large_content.append("")
|
||||
large_content.append(f"Content for section {i} with detailed information and multiple paragraphs.")
|
||||
large_content.append("")
|
||||
large_content.append("More content here to make the document substantial in size.")
|
||||
large_content.append("")
|
||||
|
||||
large_content.append("---")
|
||||
large_content.append("")
|
||||
large_content.append("```yaml tailmatter")
|
||||
large_content.append("qa_checklist:")
|
||||
for i in range(20):
|
||||
complete = "true" if i % 3 == 0 else "false"
|
||||
large_content.append(f" - requirement: 'Test requirement {i}'")
|
||||
large_content.append(f" complete: {complete}")
|
||||
large_content.append("editorial:")
|
||||
large_content.append(" status: 'Performance Testing'")
|
||||
large_content.append("```")
|
||||
|
||||
large_document = "\n".join(large_content)
|
||||
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(large_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test that all commands complete in reasonable time
|
||||
import time
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(content_stats, ['--file', temp_file])
|
||||
content_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert content_time < 2.0 # Should complete in under 2 seconds
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
frontmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert frontmatter_time < 1.0 # Should complete in under 1 second
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
contentmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert contentmatter_time < 2.0 # Should complete in under 2 seconds
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(tailmatter_check, ['--file', temp_file])
|
||||
tailmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert tailmatter_time < 1.0 # Should complete in under 1 second
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_error_handling_consistency(self, runner):
|
||||
"""Test that all command families handle errors consistently."""
|
||||
non_existent_file = "/tmp/non_existent_file.md"
|
||||
|
||||
# All commands should handle missing files gracefully
|
||||
commands_and_args = [
|
||||
(content_get, ['--file', non_existent_file]),
|
||||
(content_stats, ['--file', non_existent_file]),
|
||||
(frontmatter_get, ['title', '--file', non_existent_file]),
|
||||
(frontmatter_keys, ['--file', non_existent_file]),
|
||||
(contentmatter_get, ['Author', '--file', non_existent_file]),
|
||||
(contentmatter_keys, ['--file', non_existent_file]),
|
||||
(tailmatter_get, ['editorial.status', '--file', non_existent_file]),
|
||||
(tailmatter_check, ['--file', non_existent_file]),
|
||||
]
|
||||
|
||||
for command, args in commands_and_args:
|
||||
result = runner.invoke(command, args)
|
||||
assert result.exit_code != 0 # Should fail for non-existent file
|
||||
|
||||
def test_help_commands_consistency(self, runner):
|
||||
"""Test that all commands provide consistent help."""
|
||||
commands = [
|
||||
content_get, content_stats,
|
||||
frontmatter_get, frontmatter_keys,
|
||||
contentmatter_get, contentmatter_keys,
|
||||
tailmatter_get, tailmatter_check
|
||||
]
|
||||
|
||||
for command in commands:
|
||||
result = runner.invoke(command, ['--help'])
|
||||
assert result.exit_code == 0
|
||||
assert "Usage:" in result.output
|
||||
assert "--help" in result.output
|
||||
|
||||
def test_output_format_consistency(self, runner, complete_document):
|
||||
"""Test that commands with format options work consistently."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test JSON format consistency
|
||||
result = runner.invoke(content_stats, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('{')
|
||||
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('[')
|
||||
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('[')
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
Reference in New Issue
Block a user