markitect-main/docs/SCHEMA_MANAGEMENT_GUIDE.md

# Schema Management Guide

Complete guide to managing schemas in MarkiTect using the Schema-of-Schemas system.

## Overview

MarkiTect provides a comprehensive schema management system with:
- Markdown-first schema format with embedded JSON
- Strict naming conventions for consistency
- Metaschema validation for all schemas
- Multi-schema batch validation
- Schema registry with version tracking

## Quick Start

### 1. Create a New Schema

Create a markdown file following the naming convention: `{domain}-schema-v{major}.{minor}.md`

```bash
# Example: blog-post-schema-v1.0.md
```

**Template:**

```markdown
---
schema-id: https://markitect.dev/schemas/blog-post/v1.0
version: 1.0.0
status: stable
domain: blog-post
description: Schema for blog post documents
---

# Blog Post Schema v1.0.0

## Overview
This schema validates blog post documents with frontmatter and content sections.

## Schema Definition

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://markitect.dev/schemas/blog-post/v1.0",
  "title": "Blog Post Schema",
  "description": "Schema for blog post documents",
  "version": "1.0.0",
  "type": "object",
  "properties": {
    "title": {
      "type": "string",
      "minLength": 1
    },
    "author": {
      "type": "string"
    },
    "date": {
      "type": "string",
      "format": "date"
    }
  },
  "required": ["title", "author"]
}
```
\`\`\`

### 2. Validate Your Schema

Validate against the metaschema to ensure it follows MarkiTect conventions:

```bash
# Validate a single schema file
markitect schema-validate ./blog-post-schema-v1.0.md

# See detailed errors
markitect schema-validate ./blog-post-schema-v1.0.md --detailed-errors
```

### 3. Ingest into Registry

Add your schema to the registry:

```bash
markitect schema-ingest blog-post-schema-v1.0.md
```

### 4. List Registered Schemas

View all schemas with numbered references:

```bash
# Simple format (default)
markitect schema-list

# Table format
markitect schema-list --format table

# JSON format
markitect schema-list --format json
```

**Output:**
```
Found 4 schema(s):

[1] 🔧 blog-post-schema-v1.0.md         (added: 2026-01-05T10:30:00)
[2] 🔧 schema-schema-v1.0.md            (added: 2026-01-05T03:33:42)
[3] 🔧 manpage-schema-v1.0.md           (added: 2026-01-05T03:33:42)
[4] 🔧 api-documentation-schema-v1.0.md (added: 2026-01-05T03:33:35)
```

## Schema Validation

### Single Schema Validation

**By number:**
```bash
markitect schema-validate 1
```

**By filename (from registry):**
```bash
markitect schema-validate blog-post-schema-v1.0.md
```

**By filesystem path:**
```bash
markitect schema-validate ./my-schema.md
```

### Batch Validation

**Validate a range:**
```bash
markitect schema-validate 1-3
```

**Validate specific schemas:**
```bash
markitect schema-validate 1,3,5
```

**Validate all schemas:**
```bash
markitect schema-validate --all
```

**Output:**
```
Validating 4 schema(s)...

Results:

  #  Schema                            Status    Details
---  --------------------------------  --------  ---------
  1  blog-post-schema-v1.0.md          ✅ Valid   v1.0.0
  2  schema-schema-v1.0.md             ✅ Valid   v1.0.0
  3  manpage-schema-v1.0.md            ✅ Valid   v1.0.0
  4  api-documentation-schema-v1.0.md  ✅ Valid   v1.0.0

Summary: 4 valid, 0 failed
```

## Schema Naming Conventions

All schema filenames must follow this pattern:

```
{domain}-schema-v{major}.{minor}.md
```

### Rules

- **Domain**: Lowercase letters, numbers, and hyphens only
- **Version**: Major.minor format (e.g., `v1.0`, `v2.3`)
- **Extension**: Must be `.md`
- **No spaces**: Use hyphens for separation

### Valid Examples

- `blog-post-schema-v1.0.md`
- `api-documentation-schema-v2.1.md`
- `user-profile-schema-v1.0.md`

### Invalid Examples

- `BlogPost-schema-v1.0.md` (uppercase)
- `blog_post-schema-v1.0.md` (underscore)
- `blog-post-v1.0.md` (missing "schema")
- `blog-post-schema-v1.md` (missing minor version)

## Required Schema Fields

All schemas must include these fields:

### Frontmatter (YAML)
```yaml
---
schema-id: https://markitect.dev/schemas/{domain}/v{major}.{minor}
version: {major}.{minor}.{patch}
status: draft|stable|deprecated
domain: {domain}
description: Brief description
---
```

### JSON Schema
```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://markitect.dev/schemas/{domain}/v{major}.{minor}",
  "title": "Schema Title",
  "description": "Schema description",
  "version": "{major}.{minor}.{patch}"
}
```

## Common Workflows

### Revalidate All Schemas After Metaschema Changes

When you update the metaschema, revalidate all registered schemas:

```bash
markitect schema-validate --all
```

### Check Schema Rigidity

Analyze a schema for overly rigid constraints:

```bash
markitect schema-analyze my-schema.md
```

### Refine a Rigid Schema

Automatically loosen overly specific constraints:

```bash
# Dry run (preview changes)
markitect schema-refine my-schema.md --dry-run

# Apply changes
markitect schema-refine my-schema.md

# Interactive mode
markitect schema-refine my-schema.md --interactive
```

### Get Schema Details

View schema metadata:

```bash
markitect schema-get blog-post-schema-v1.0.md
```

### Delete a Schema

Remove a schema from the registry:

```bash
markitect schema-delete blog-post-schema-v1.0.md --confirm
```

## Resolution Precedence

When validating schemas, MarkiTect uses this resolution order:

1. **Registry (by filename)**: Exact match in the database
2. **Filesystem (fallback)**: If not found in registry or looks like a path

### Examples

```bash
# Looks up in registry first
markitect schema-validate blog-post-schema-v1.0.md

# Forces filesystem lookup (contains /)
markitect schema-validate ./blog-post-schema-v1.0.md

# Also forces filesystem
markitect schema-validate ../schemas/blog-post-schema-v1.0.md
```

## Best Practices

### Schema Development

1. **Start with a template**: Use an existing schema as a starting point
2. **Validate early**: Validate against the metaschema before ingesting
3. **Use semantic versioning**: Major.minor.patch for all versions
4. **Document thoroughly**: Include overview, usage, and examples
5. **Test with real documents**: Validate actual documents against your schema

### Version Management

- **Increment major version**: Breaking changes to schema structure
- **Increment minor version**: Backward-compatible additions
- **Increment patch version**: Bug fixes and clarifications

### Schema Organization

```
markitect/schemas/
├── schema-schema-v1.0.md        # Metaschema
├── manpage-schema-v1.0.md       # Man page documents
├── api-documentation-schema-v1.0.md
├── terminology-schema-v1.0.md
└── blog-post-schema-v1.0.md     # Your schemas
```

## Troubleshooting

### Schema Not Found

```
❌ Schema 'my-schema.md' not found in registry or filesystem
```

**Solution:** Use `markitect schema-list` to see available schemas, or provide a path: `./my-schema.md`

### Validation Fails

```
❌ Schema validation failed: my-schema.md
Found 2 validation error(s):
```

**Solution:** Check error messages and compare with metaschema requirements. Use `--detailed-errors` for more context.

### Invalid Selector

```
❌ Invalid selector: Range 1-10 is out of bounds. Valid range: 1-4
```

**Solution:** Use `markitect schema-list` to see valid numbers, or check your range syntax.

## Advanced Usage

### Scripting with Schema Commands

Validate schemas in CI/CD:

```bash
#!/bin/bash
# Validate all schemas and exit with error if any fail
if ! markitect schema-validate --all; then
    echo "Schema validation failed!"
    exit 1
fi
echo "All schemas valid"
```

### Batch Operations

```bash
# Validate recently added schemas
markitect schema-validate 1-3

# Validate specific critical schemas
markitect schema-validate 1,5,8

# Check just the metaschema
markitect schema-validate 2
```

## Schema Extensions

MarkiTect supports custom extensions in schemas:

- `x-markitect-sections`: Section classification (required, recommended, optional, discouraged, improper)
- `x-markitect-content-control`: Content validation rules and patterns
- `x-markitect-metadata`: Additional metadata for MarkiTect processing

See existing schemas for examples of these extensions.

## Future Enhancements

Planned features:
- Wildcard/globbing support: `markitect schema-validate */manpage*`
- Schema diff tool: Compare schema versions
- Schema migration assistant: Help upgrade documents to new schema versions

## Related Documentation

- [Schema Naming Specification](../history/2026-01-05-schema-of-schemas/SCHEMA_NAMING_SPEC.md)
- [Schema Loader Guide](../history/2026-01-05-schema-of-schemas/SCHEMA_LOADER_GUIDE.md)
- [Metaschema Reference](../markitect/schemas/schema-schema-v1.0.md)
- [Implementation Workplan](../history/2026-01-05-schema-of-schemas/WORKPLAN.md) (archived)

## Support

For issues or questions:
- Check existing schemas as examples
- Review metaschema validation errors carefully
- Use `--detailed-errors` for more context
- Consult the metaschema for requirements