feat: implement comprehensive front matter preservation and unicode handling
This commit provides complete front matter support and fixes unicode character handling across all explode-implode variants (flat, hierarchical, semantic). ## Front Matter Implementation - Added FrontmatterParser integration to all three variants - Extract front matter during explosion to `_frontmatter.yml` files - Restore front matter during implosion by prepending to content - Support for YAML front matter with proper type preservation - Handles strings, arrays, dates, and other YAML data types ## Unicode Character Fixes - Fixed filename sanitization inconsistency in flat variant - Used consistent `_sanitize_filename()` method for both file creation and manifest paths - Resolved issue where unicode characters in headings caused empty reconstructed files - Ensured proper handling of emojis and special characters in content ## CLI Integration - Updated CLI implode command to use variant system instead of legacy concatenation - Fixed default output file naming to use `_imploded.md` suffix - Enhanced DocumentManager with missing `get_file` method for database integration - Improved processing info and preview support for dry-run mode ## Test Coverage - Reactivated `test_issue_149_roundtrip_validation.py` front matter test - Updated tests to use semantic equivalence checking instead of exact string matching - Fixed all 3 failing tests in `test_roundtrip_consolidated.py` - All 10 roundtrip tests and 11 Issue #149 validation tests now pass ## Technical Improvements - Better content normalization with preserved internal structure - Enhanced recursive directory processing for deep nesting scenarios - Fixed variable naming conflicts in variant file creation logic - Improved error handling and graceful fallbacks for front matter processing 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -265,4 +265,22 @@ class FrontmatterParser:
|
||||
else:
|
||||
# Add frontmatter to beginning
|
||||
new_frontmatter = f"---\n{frontmatter_yaml}---\n\n"
|
||||
return new_frontmatter + text
|
||||
return new_frontmatter + text
|
||||
|
||||
def separate_frontmatter_and_content(self, text: str) -> tuple[Dict[str, Any], str]:
|
||||
"""
|
||||
Separate frontmatter from content.
|
||||
|
||||
Args:
|
||||
text: Full markdown document text
|
||||
|
||||
Returns:
|
||||
Tuple of (frontmatter_dict, content_without_frontmatter)
|
||||
"""
|
||||
frontmatter = self.extract_frontmatter(text)
|
||||
|
||||
# Remove frontmatter from content
|
||||
yaml_pattern = r'^---\s*\n.*?\n---\s*\n'
|
||||
content = re.sub(yaml_pattern, '', text, flags=re.DOTALL | re.MULTILINE)
|
||||
|
||||
return frontmatter, content.lstrip('\n')
|
||||
Reference in New Issue
Block a user