a969c5de47
feat: add semantic document validator for x-markitect extensions
...
Implements semantic validation to complement existing structural validation:
Phase 1 & 2 Complete:
- SemanticValidator: Main validator orchestrating sub-validators
- SectionValidator: Enforces section classifications (required, recommended,
optional, discouraged, improper) from x-markitect-sections
- ContentValidator: Validates content patterns, forbidden patterns, and
quality metrics (word counts, sentence counts) from x-markitect-content-control
Features:
- Pattern matching with regex for required/forbidden/discouraged patterns
- Word count and sentence count validation
- Detailed error reporting with severity levels (ERROR, WARNING)
- Support for section alternatives (e.g., FLAGS vs OPTIONS)
- Comprehensive test coverage (16 tests, 100% passing)
Architecture:
- Complements existing SchemaValidator (structural AST validation)
- Clean separation: validators/ package for modular validators
- Semantic validation focuses on x-markitect-* extensions
- LinkValidator planned for Phase 3 (optional --check-links)
Next: Phase 4 - CLI integration to enhance 'markitect validate' command
Workplan: roadmap/20260106-semantic-document-validation/WORKPLAN.md
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-06 03:24:32 +01:00
3003b9b8da
chore: archive completed schema-of-schemas implementation
...
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Moved schema-of-schemas planning artifacts from roadmap to history
with datestamp prefix, marking completion of all 6 implementation phases.
**Changes:**
- Moved roadmap/schema-of-schemas/ → history/2026-01-05-schema-of-schemas/
- Updated all documentation references to new location
- Marked implementation as completed in TODO.md
- Updated CHANGELOG.md to reflect archived status
**Implementation Summary:**
All 6 phases completed successfully:
- Phase 1: Filename validation (50 tests)
- Phase 2: Markdown schema loader (35 tests)
- Phase 3: Schema-for-schemas metaschema (12 tests)
- Phase 4: Schema migration (2 migrated, 3 deleted)
- Phase 5: CLI enhancements (multi-schema validation)
- Phase 6: Integration testing and documentation
**Deliverables:**
- 97 unit tests (100% passing)
- 4 production schemas in registry
- Comprehensive user documentation
- Updated examples (manpages, terminology)
- Complete schema management system
The schema-of-schemas topic is now complete and archived for
historical reference.
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-05 14:13:48 +01:00
b81ce5631d
feat: implement Phase 2 - Markdown Schema Loader
...
Completed Phase 2 of the schema-of-schemas implementation with full
markdown schema support. This enables schemas to be authored as
markdown files with rich documentation and embedded JSON schemas.
Core Implementation (markitect/schema_loader.py):
- MarkdownSchemaLoader class with comprehensive parsing capabilities
- YAML frontmatter extraction with error handling
- JSON code block extraction with section preference (## Schema Definition)
- Metadata merging with x-markitect-source tracking
- Schema saving with template support and round-trip capability
- Helper methods: list_json_blocks(), validate_schema_structure()
Test Coverage (tests/test_schema_loader.py):
- 35 comprehensive unit tests (100% passing)
- Tests for loading, parsing, saving, round-trip conversion
- Edge case handling (empty files, binary files, malformed blocks)
- Fixed binary file test to use invalid UTF-8 sequences
Example Schema (markitect/schemas/manpage-schema-v1.0.md):
- First markdown schema following naming convention
- Complete manpage schema with frontmatter + documentation + JSON
- Demonstrates section classification and content control
- Shows proper structure for future schema authors
Documentation (roadmap/schema-of-schemas/SCHEMA_LOADER_GUIDE.md):
- Comprehensive user guide (600+ lines)
- API reference with examples
- Best practices and troubleshooting
- Integration patterns for CLI and validator
Progress Tracking:
- Updated TODO.md with Phase 2 completion
- Updated CHANGELOG.md with implementation details
- Next: Phase 3 - Schema-for-Schemas Metaschema
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-05 00:02:15 +01:00
14108533fb
feat: implement schema filename validation (Phase 1 complete)
...
Implements filename convention enforcement for schema files as part of
the schema-of-schemas implementation. All schemas must now follow the
naming pattern: {domain}-schema-v{major}.{minor}.md
## Phase 1 Deliverables
### Schema Naming Module
**File:** `markitect/schema_naming.py` (380 lines)
**Functions:**
- `validate_schema_filename()` - Validate filename against pattern
- `suggest_schema_filename()` - Generate valid filename from domain/version
- `extract_schema_metadata()` - Extract domain and version from filename
- `get_validation_errors()` - Detailed error messages for invalid filenames
- `is_valid_schema_filename()` - Simple boolean validation
- `format_validation_message()` - User-friendly error formatting
**Features:**
- Regex-based pattern matching
- Automatic normalization (spaces → hyphens, lowercase)
- Detailed error reporting
- Domain validation (must start with letter)
- Version validation (major.minor format)
### Comprehensive Test Suite
**File:** `tests/test_schema_naming.py` (500+ lines, 50 tests)
**Test Coverage:**
- ✅ Valid filename variations (simple, hyphenated, with numbers)
- ✅ Invalid filenames (wrong extension, missing components, wrong case)
- ✅ Filename suggestion with normalization
- ✅ Metadata extraction
- ✅ Error message generation
- ✅ Edge cases (long names, many hyphens, large versions)
- ✅ Pattern regex validation
**Results:** 50/50 tests passing (100%)
### Specification Document
**File:** `roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md`
**Contents:**
- Formal specification of naming convention
- Regular expression pattern with explanation
- Valid and invalid examples
- Version numbering guidelines
- Domain naming best practices
- Normalization rules
- Migration strategy from legacy naming
- Implementation guide
## Naming Convention
### Format
```
{domain}-schema-v{major}.{minor}.md
```
### Examples
```
✓ manpage-schema-v1.0.md
✓ api-documentation-schema-v1.0.md
✓ terminology-schema-v1.0.md
✓ arc42-schema-v2.1.md
✗ manpage.json (wrong extension)
✗ ManPage-schema-v1.0.md (uppercase)
✗ manpage-v1.0.md (missing 'schema')
✗ manpage-schema-v1.md (missing minor version)
```
### Components
- **domain**: Lowercase, hyphen-separated, starts with letter
- **schema**: Literal keyword
- **version**: v{major}.{minor} (SemVer simplified)
- **extension**: .md (markdown)
## Implementation Highlights
### Automatic Normalization
```python
suggest_schema_filename("API Documentation", "2.1")
# → "api-documentation-schema-v2.1.md"
suggest_schema_filename("My_Custom Type", "1.0")
# → "my-custom-type-schema-v1.0.md"
```
### Detailed Error Reporting
```python
format_validation_message("invalid.json")
# → Detailed error list + suggested fix
```
### Metadata Extraction
```python
extract_schema_metadata("manpage-schema-v1.0.md")
# → {'domain': 'manpage', 'version': '1.0', 'major': 1, 'minor': 0}
```
## Migration Plan
Current schemas will be renamed:
```
Old → New
────────────────────────────────────────────────────────
terminology-schema.json → terminology-schema-v1.0.md
api-documentation → api-documentation-schema-v1.0.md
enhanced-manpage → manpage-schema-v2.0.md
markdown-manpage → DELETE (duplicate)
markdown-manpage-schema.json → DELETE (duplicate)
```
## Phase 1 Status: ✅ COMPLETE
### Completed
- [x] Schema naming module implementation
- [x] Comprehensive test suite (50 tests, 100% passing)
- [x] Specification document
- [x] TODO.md updated
### Next: Phase 2
- [ ] Update CLI schema-ingest with validation
- [ ] Implement markdown schema loader
- [ ] Parse frontmatter and JSON code blocks
- [ ] Update SchemaValidator for .md support
## Testing
```bash
# Run tests
pytest tests/test_schema_naming.py -v
# → 50 passed in 0.48s
# Test interactively
python -c "
from markitect.schema_naming import validate_schema_filename
print(validate_schema_filename('manpage-schema-v1.0.md'))
"
# → (True, {'domain': 'manpage', 'version': '1.0', ...})
```
## Files Changed
- markitect/schema_naming.py (NEW, 380 lines)
- tests/test_schema_naming.py (NEW, 500+ lines)
- roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md (NEW)
- TODO.md (updated progress tracking)
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-04 23:51:29 +01:00
b6f95066a3
chore: establish schema-of-schemas workplan and reorganize roadmap
...
This commit sets up the comprehensive workplan for implementing a
markdown-first schema management system with naming conventions,
versioning, and self-validation capabilities.
## Directory Reorganization
- Renamed `todo/` → `roadmap/` for better organization
- Created `roadmap/schema-of-schemas/` subdirectory
- Moved schema management planning artifacts to dedicated directory
## Planning Artifacts Created
### Workplan & Documentation
- **WORKPLAN.md** (19KB) - Comprehensive 6-phase implementation plan
- **SCHEMA_MANAGEMENT_PROPOSAL.md** - Full analysis with 4 options
- **SCHEMA_MANAGEMENT_SUMMARY.md** - Executive summary
- **README.md** - Quick reference guide
### Example Schema
- **examples/schemas/manpage-schema-v1.md** - Demonstrates markdown format
## Schema Management System Design
### Naming Convention
**Format:** `{domain}-schema-v{major}.{minor}.md`
**Examples:**
- `manpage-schema-v1.0.md`
- `terminology-schema-v1.0.md`
- `api-documentation-schema-v1.0.md`
### Markdown-First Format
Schemas will be markdown files with:
- YAML frontmatter for metadata
- Rich documentation sections
- Embedded JSON schema in code block
- Version history and examples
### Implementation Phases (8-10 days)
**Phase 0:** Planning & Setup ✅ (0.5 days) - COMPLETE
**Phase 1:** Filename Convention (1 day) - NEXT
**Phase 2:** Markdown Loader (2-3 days)
**Phase 3:** Schema-for-Schemas (2 days)
**Phase 4:** Schema Migration (1-2 days)
**Phase 5:** CLI & Documentation (1 day)
**Phase 6:** Testing & Validation (1 day)
### Goals
1. ✅ Establish naming convention
2. ⏳ Implement filename validation
3. ⏳ Create markdown schema loader
4. ⏳ Build schema-for-schemas metaschema
5. ⏳ Migrate 5 existing schemas (remove 2 duplicates)
6. ⏳ Update CLI and documentation
## Updated Tracking
### TODO.md
- Added Schema-of-Schemas as active work item
- Documented Phase 1 tasks and timeline
- Paused capability extraction work
### CHANGELOG.md
- Added schema management system to [Unreleased]
- Documented directory reorganization
- Added "In Progress" section for current work
## Next Steps
Begin Phase 1:
1. Implement schema_naming.py with validation
2. Add unit tests
3. Update CLI schema-ingest command
4. Create naming specification document
## Files Changed
- CHANGELOG.md - Added unreleased schema management features
- TODO.md - Updated active work tracking
- roadmap/ - Reorganized from todo/
- roadmap/schema-of-schemas/ - New planning directory
- examples/schemas/ - Example markdown schema
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com >
2026-01-04 23:47:02 +01:00
