Commit Graph

227 Commits

Author SHA1 Message Date
60f33443ae feat(schema): add semantic schema generation as default mode
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
schema-generate now builds content-aware schemas from the document's
section hierarchy instead of counting markdown syntax elements. Detects
key-value tables, data tables, link lists, and mixed content patterns
to produce schemas that reflect the actual document outline.

Old behavior preserved via --mode syntactic. Validator and visualization
tools pinned to syntactic mode for compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 18:49:50 +01:00
120ed89780 fix(proxy): catch markitdown missing-dependency errors with clean hint
When markitdown is installed but a format-specific sub-dependency is
missing (e.g. pdfminer-six for PDF), translate the raw traceback into
a DependencyMissingError with the correct install command.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 21:00:51 +01:00
9fa239c140 fix(proxy): register markitdown extractor unconditionally
Always register MarkitdownExtractor so it overrides specialized extractors
for all its extensions. When markitdown-no-magika is not installed, users
now see the correct install hint instead of the old pymupdf4llm message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 20:52:07 +01:00
e4fbba8a57 feat(proxy): add markitdown as default proxy backend
Uses markitdown-no-magika (lighter fork without magika/onnxruntime) to
handle PDF, HTML, DOCX, PPTX, XLSX, XLS, CSV, JSON, and XML files.
Specialized extractors (pymupdf4llm, markdownify) remain as fallbacks
when markitdown is not installed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 20:48:47 +01:00
ac334c679d feat(proxy): add proxy file system for non-markdown source conversion
Introduces a new `markitect/proxy/` module with pluggable extractors that
convert non-markdown sources (PDF, HTML) into tracked markdown proxy files.
Proxy files preserve origin metadata (path, checksum, timestamp) so they
can be kept in sync when the original changes.

CLI commands: `proxy create`, `proxy update`, `proxy status`, `proxy extractors`.
Built-in extractors: PDF (pymupdf4llm), HTML (markdownify), Markdown (built-in).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 19:06:09 +01:00
69aea1ada7 refactor(version): separate version and release commands
`markitect version` now prints a clean version string (Unix style),
with -v for commit/branch/dirty. `markitect release` shows detailed
development status: commits since tag, local changes, upstream
divergence. No overlap between the two commands.

Replaces get_version_info()/get_release_info() with get_version()
and get_release_status(). Drops yaml output format from release
(json + text sufficient).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 17:49:14 +01:00
be3b4e3aae fix(version): resolve version dynamically from git in dev checkouts
When running from a git repo, use setuptools-scm at runtime to derive
the version from tags. Falls back to the static _version.py only when
not in a git repo (e.g. installed from wheel). This ensures
`markitect version` stays correct without requiring `pip install -e .`
after every tag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 17:22:38 +01:00
ad23bb0b86 fix(version): normalize release info for CLI release command
Add _normalize_release_info() to ensure get_release_info() returns
keys expected by the CLI release command regardless of whether the
release-management capability is available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:37:40 +01:00
5085c44de3 feat(llm): add llm-default and llm-preference commands, switch hardcoded default to gemini
Add TOML-based config resolution with 7-level priority chain:
CLI flags > env var > user preference > directory preference >
directory default > user default > hardcoded fallback.

New commands: llm-default (view/set/clear defaults), llm-preference
(view/set/clear preferences). Each shows only its own scope. llm-check
now displays source attribution for resolved provider/model.

Existing commands (llm-helper, llm-check) refactored to use
resolve_llm() instead of manual resolution. Hardcoded fallback
changed from openrouter/aurora-alpha to gemini/gemini-2.5-flash
due to persistent OpenRouter 502 errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:35:44 +01:00
4631a9f794 feat(llm): add qwen3-coder-next to catalog and Known Models column
Register qwen/qwen3-coder-next under the openrouter provider and extend
llm-catalog with a "Known Models" column so all cataloged models are
discoverable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 00:17:57 +01:00
269184f7a1 feat(llm): add llm-catalog and llm-check commands, rename helper → llm-helper
Consistent llm-* naming scheme for all LLM CLI commands. llm-catalog shows
provider metadata and key status; llm-check sends a minimal prompt to verify
connectivity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 00:12:50 +01:00
69e2ec25ff feat(helper): add interactive Q&A helper command
Add `markitect helper <QUESTION>` CLI command that answers questions
about markitect using its own documentation as LLM context. Uses
OpenRouter with openrouter/aurora-alpha by default; model is
configurable via --model flag or MARKITECT_HELPER_MODEL env var.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 23:28:20 +01:00
41773f1320 feat(llm): add OpenAI adapter, entity archive policy, process chapters 5-7
Add OpenAIAdapter for the OpenAI chat completions API (apikey-chatgpt.txt
or OPENAI_API_KEY). Set default model to arcee-ai/trinity-large-preview:free
for the infospace pipeline and increase max_tokens from 4096 to 8192.

Reprocess chapter 05 with Trinity Large (was Gemini: 1 truncated entity,
now 19 complete entities). Process chapters 06 (Aurora Alpha, 10 entities)
and 07 (Trinity Large, 15 entities including regenerated violent-policy.md).
Canonical set now at 85 unique entities.

Add entity archive policy: entities are never silently deleted. Retired
entities move to output/entities/archive/ with a dated reason header.
New CLI option: --archive-entity <slug> --reason "...". The --list
output shows the archive count alongside the canonical set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 23:39:44 +01:00
880c1d1374 feat(llm): add Gemini adapter and process book-1-chapter-05
Add GeminiAdapter calling Google's Generative Language REST API
(default model: gemini-2.5-flash). Register "gemini" as third
provider in the factory and CLI. Add rate-limit retry with
exponential backoff to the pipeline's _call_llm helper. Increase
default max_tokens from 2000 to 4096.

Process book-1-chapter-05 via Gemini free tier — 1 new entity
extracted (necessaries-conveniencies-and-amusements-of-life),
41 existing entities correctly skipped by dedup. Canonical set
now at 42 unique entities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 22:54:37 +01:00
706981c39f fix(prompts): fix three infrastructure bugs in prompt dependency resolution
- ContentMacro: add __post_init__ to auto-derive raw_text when built
  programmatically, preventing str.replace("", X) corruption
- MacroParser: add @{target} shorthand syntax support mapped to REQUIRED kind,
  updating parse, has_macros, count_macros, and find_macro_positions
- Artifact: store content in model and SQLite DB, replace resolver placeholder
  with actual artifact content, add migration for existing databases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 20:53:02 +01:00
fecc2fd4fa feat(llm): add LLM integration module with OpenRouter and Claude Code adapters
Implements markitect/llm/ package with concrete LLMAdapter implementations:
- OpenRouterAdapter: HTTP via urllib with retry/backoff on 429/5xx
- ClaudeCodeAdapter: subprocess-based Claude CLI with stdin piping
- Factory pattern: create_adapter("openrouter") or create_adapter("claude-code")
- API key resolution chain: constructor > env var > project-root key file
- 42 unit tests, 2 integration tests (gated on API key / CLI availability)

Also adds the infospace-with-history example with Wealth of Nations VSM
analysis pipeline, templates, schemas, source chapters, and processed
output for chapters 1-2. process_chapters.py now supports --provider
and --model flags for automatic LLM-driven processing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 01:17:58 +01:00
7b4bd461c9 feat(prompts): implement Phase 8 - Observability & Traceability (FR-11)
Complete implementation of Phase 8, the final phase of prompt dependency
resolution infrastructure, adding full observability and traceability.

## Features (FR-11)

### FR-11.1: Complete Artifact Provenance Tracing
- TraceabilityService: composition layer for full artifact lineage
- Trace any artifact to producing PromptTemplate, input artifacts,
  generator runs, and quality validation results
- ProvenanceTrace model with complete dependency chain reconstruction
- RunSummary and ArtifactLineage models for structured trace output

### FR-11.2: Recomputation Query Infrastructure
- PromptQueryService: cross-service complex queries
- Run history queries with template and status filters
- Stale artifact detection via impact debt analysis
- Dependency graph statistics (nodes, edges, cycles, roots, leaves)
- Content-based artifact lookups by digest

### Visualization Support
- GraphExporter: DOT (Graphviz) and Mermaid format export
- Supports all edge types (requires, generates, includes)
- Handles isolated nodes, linear chains, diamonds, and complex graphs

### CLI Commands (prompt group)
- `prompt trace <artifact_id>` - Full provenance trace as JSON
- `prompt graph <artifact_id>` - Dependency graph (DOT/Mermaid)
- `prompt runs` - List execution runs with filters
- `prompt debt` - Show impact debt and stale artifacts
- `prompt stats` - Dependency graph statistics

## Implementation

Source files (8):
- markitect/prompts/traceability/models.py - Trace data models
- markitect/prompts/traceability/service.py - TraceabilityService
- markitect/prompts/visualization/graph.py - Graph export
- markitect/prompts/queries/operations.py - PromptQueryService
- markitect/prompts/cli.py - Click CLI commands
- Package __init__.py files (3)

Tests (64 total, all passing):
- tests/unit/prompts/test_traceability_service.py (21 tests)
- tests/unit/prompts/test_visualization.py (14 tests)
- tests/unit/prompts/test_query_operations.py (12 tests)
- tests/integration/prompts/test_traceability_workflow.py (7 tests)
- tests/integration/prompts/test_prompt_cli.py (10 tests)

## Architecture

TraceabilityService is a composition layer that delegates to:
- DependencyQueryService (transitive dependency lookups)
- QualityValidator (validation history)
- IncrementalExecutionEngine (impact debt queries)
- Direct repository access (artifacts, edges)

No duplicate data storage - all data comes from existing Phase 1-7
infrastructure (artifact repo, dependency repo, validation DB, debt DB).

## Verification

All 2250 tests pass with 0 regressions.
Phase 8 completes the full 8-phase implementation roadmap.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-09 20:32:18 +01:00
704272644c feat(prompts): implement Phase 7 - Quality & Validation (FR-9, FR-10)
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Add quality gate framework with schema validation (JSON Schema via
jsonschema library), pattern validation (regex-based), multi-gate
QualityValidator with SQLite persistence, HaltingPolicyEngine with
budget/iteration/improvement checks, and RefinementLoop for iterative
execute-validate-halt cycles.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:31:37 +01:00
bd1d05ba79 feat(prompts): implement Phase 6 - Incremental Execution (FR-7, FR-8)
Add change detection, structural diff-based impact analysis,
configurable-depth incremental recomputation with circular suppression,
and impact debt tracking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:18:27 +01:00
9ce157400e feat(prompts): implement Phase 5 - Dependency Tracking (FR-6)
Add directed dependency graph with cycle detection, topological sort,
and query service for finding dependents/dependencies transitively.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:18:18 +01:00
c56c92c815 feat(prompts): implement Phase 4 - Execution Engine (FR-4, FR-5)
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Implement three-stage execution lifecycle with idempotent runs and complete
provenance tracking via RunManifest.

Core Features:
- PromptRun model with execution lifecycle stages:
  1. Analysis: Template analysis and macro extraction
  2. Compilation: Macro resolution and context compilation
  3. Processing: LLM execution and output generation
- InputBundleHash for deterministic idempotency (FR-4.3)
- RunManifest for complete execution provenance (FR-5)
- LLMAdapter interface for pluggable model providers
- MockLLMAdapter for testing without API calls
- PromptExecutionEngine orchestrating full lifecycle

Idempotent Execution (FR-4.4):
- Calculate SHA-256 hash of complete input context
- Skip execution if identical hash exists
- Cache successful runs by hash
- Support force re-execution via config flag

RunManifest Tracking (FR-5.2):
- Template metadata (id, name, digest)
- Resolved input artifacts and digests
- Compiled prompt digest
- Model configuration
- Output artifacts
- Dependency edges for graph construction
- Timing metadata for performance analysis

Tests (27 passing):
- 17 execution model tests (config, bundle, runs, stages)
- 10 engine tests (execution, idempotency, errors, caching)

Implements:
- FR-4.1: Three-stage execution lifecycle
- FR-4.2: CompiledPrompt during compilation
- FR-4.3: InputBundleHash calculation
- FR-4.4: Skip execution for identical hashes
- FR-5.1: RunManifest persistence
- FR-5.2: Complete manifest contents
- FR-5.3: Nested run linking (foundation)

Files Created:
- markitect/prompts/execution/models.py
- markitect/prompts/execution/manifest.py
- markitect/prompts/execution/llm_adapter.py
- markitect/prompts/execution/engine.py
- migrations/prompts/003_create_runs_and_manifests.sql
- tests/unit/prompts/test_execution_models.py
- tests/unit/prompts/test_execution_engine.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:15:33 +01:00
5f463e5b20 feat(prompts): implement Phase 3 - Resolver Engine (FR-3)
Implement deterministic multi-space resolution with configurable search order.

Core Features:
- ResolutionContext and ResolutionResult for tracking resolution state
- MultiSpaceResolutionStrategy implementing FR-3.1 search order:
  1. Local InformationSpace
  2. Explicitly included InformationSpaces
  3. Default InformationSpace
  4. Team/Shared InformationSpace
- PromptResolver with macro resolution logic
- ContextCompiler for assembling resolved prompts
- ResolutionConfig for configurable resolution behavior

Resolution Behavior:
- Required macros fail if not found (FR-3.2)
- Optional macros resolve to empty (FR-3.3)
- Generate macros detected for deferred execution (FR-3.4)
- Deterministic search order with duplicate removal
- Partial compilation support for debugging

Tests (31 passing):
- 14 strategy tests (search order, duplicates, priority)
- 9 resolver tests (required, optional, generate, multi-space)
- 8 compiler tests (substitution, dependencies, digests)

Implements:
- FR-3.1: Deterministic resolution order
- FR-3.2: Required macro validation
- FR-3.3: Optional macro fallback
- FR-3.4: Generate macro detection
- FR-3.5: Max generation depth configuration

Files Created:
- markitect/prompts/resolver/models.py
- markitect/prompts/resolver/strategy.py
- markitect/prompts/resolver/resolver.py
- markitect/prompts/resolver/compiler.py
- migrations/prompts/002_create_resolution_config.sql
- tests/unit/prompts/test_resolution_strategy.py
- tests/unit/prompts/test_prompt_resolver.py
- tests/unit/prompts/test_context_compiler.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:45:46 +01:00
e6840fe696 feat(prompts): implement Phase 2 - Templates & Macros (FR-2)
Implement PromptTemplate models with ContentMacro parsing and analysis.

Core Features:
- PromptTemplate extending Artifact for template-specific operations
- ContentMacro model supporting REQUIRED, OPTIONAL, GENERATE kinds
- MacroParser for extracting macros from template content
- TemplateAnalyzer for dependency extraction and validation
- TemplateService for high-level template operations
- Template metadata for model hints and expected inputs

Macro Syntax:
- {{require:artifact-name}} - Required dependency
- {{optional:artifact-name}} - Optional dependency
- {{generate:template-name|param=value}} - Nested generation

Tests (38 passing):
- 18 template model tests (macros, templates, metadata)
- 20 parser tests (parsing, validation, parameters, aliases)

Implements:
- FR-2.1: PromptTemplate as content artifact with macros
- FR-2.2: ContentMacro detection and extraction
- FR-2.3: Required/Optional/Generate macro kinds

Files Created:
- markitect/prompts/templates/models.py
- markitect/prompts/templates/parser.py
- markitect/prompts/templates/analyzer.py
- markitect/prompts/services/template_service.py
- tests/unit/prompts/test_template_models.py
- tests/unit/prompts/test_macro_parser.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:34:22 +01:00
945544880d feat(prompts): implement Phase 1 - Foundation (FR-1)
Implement addressable artifacts with content-based identity and change detection.

Core Features:
- Artifact model with SHA-256 content digests
- ArtifactReference for cross-space addressing
- IArtifactRepository interface for pluggable storage
- SQLiteArtifactRepository implementation
- ArtifactService for high-level operations
- Content digest calculation utilities

Database:
- prompt_artifacts table with indexes
- Support for artifact metadata and types
- UNIQUE constraint on space_id+name

Tests (41 passing):
- 26 model tests (metadata, artifacts, references, digests)
- 15 repository tests (CRUD, queries, constraints)

Implements:
- FR-1.1: Unique addressability by name and ID
- FR-1.2: Content digest computation and storage
- FR-1.3: Cross-space artifact references

Files Created:
- markitect/prompts/models.py
- markitect/prompts/repositories/interfaces.py
- markitect/prompts/repositories/sqlite.py
- markitect/prompts/services/artifact_service.py
- migrations/prompts/001_create_artifacts_table.sql
- tests/unit/prompts/test_artifact_models.py
- tests/unit/prompts/test_artifact_repository.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:30:26 +01:00
4588cbeee8 feat(spaces): implement Phase 8 Git History Tracking
Implements optional git-based version control for information spaces:
- HistoryConfig model for configuring history tracking
- Commit, Branch, HistoryEntry, DiffResult models
- IHistoryBackend and IHistoryQuery interfaces
- GitHistoryBackend using git CLI for version control
- GitHistoryEventHandler for event-driven auto-commits
- HistoryEventCoordinator for managing space history
- HistoryQueryService for high-level history queries
- Automatic commits on DOCUMENT_ADDED/REMOVED/CONTENT_CHANGED events
- Support for:
  * Commit log with pagination and filtering
  * Diff between versions
  * File content at specific versions
  * Branch creation and switching
  * Version restoration
  * Uncommitted changes detection
- 43 comprehensive unit tests with git availability checks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 18:03:35 +01:00
727ce4d3c5 feat(spaces): implement Phase 7 Composability
Implements space composition and inheritance features:
- SpaceReference model for space-to-space references (includes, extends, links_to, composed_of)
- Variable inheritance through parent chain with local override
- Config inheritance with source tracking
- Access control models (SpacePermission, SpaceRole, AccessLevel)
- InheritanceResolver for walking parent chains
- AccessControlService for permission management
- ComposableSpaceService integrating all composability features
- Circular reference detection for EXTENDS references
- SQLite repositories for references and permissions
- 57 comprehensive unit tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 17:41:40 +01:00
7de57a389d feat(spaces): implement Phase 6 API Layer
Implements API layer for Information Spaces:
- GraphQL schema types for spaces, documents, variables
- GraphQL queries and mutations for space operations
- CLI command group with all space management commands
- Resolver functions connecting GraphQL to SpaceService
- 38 unit tests for API components

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 12:29:11 +01:00
535b83996b feat(spaces): implement Phase 5 Directory Sync Mode
Implements directory synchronization for Information Spaces:

- SpaceDirectoryExporter: Export space to directory structure
  - Multiple variants: flat, hierarchical, by_path
  - Manifest generation for reimport
  - Incremental export (skip unchanged files)
  - Metadata file export
  - IncrementalExporter for change detection

- DirectorySpaceImporter: Import directory content into space
  - Recursive directory scanning
  - Multiple file pattern support
  - Conflict detection with strategies (skip/overwrite/rename)
  - Manifest-based import for intelligent reimport
  - Structure preservation in space paths

- BidirectionalSyncCoordinator: Two-way sync with conflict detection
  - Sync directions: space-to-directory, directory-to-space, bidirectional
  - Conflict resolution strategies: space_wins, directory_wins, newer_wins, manual, skip
  - Dry-run mode for preview
  - Orphan cleanup option
  - Event emission for progress tracking

45 unit tests covering all sync components.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 12:11:37 +01:00
2a5c265458 feat(spaces): implement Phase 4 HTML Rendering Mode
Implements HTML rendering system for Information Spaces:

- SpaceRenderer: Abstract base class for renderers
- RenderConfig: Configuration for format, theme, TOC, etc.
- RenderResult: Immutable result with content hash and metadata
- ThemeConfig: Layered theme system with customization
- CompositeRenderer: Multi-format renderer delegation

- MarkdownToHTMLRenderer: Full markdown-to-HTML conversion
  - Theme support (github, dark, minimal, academic)
  - Code block handling
  - Link target="_blank" for external links
  - Table of contents generation
  - Heading ID generation for navigation
- HTMLRendererFactory: Factory for common renderer configurations

- SpaceRenderingService: Orchestration layer
  - Transclusion variable substitution
  - Render caching with automatic invalidation
  - Event emission (RENDER_STARTED, RENDER_COMPLETED, RENDER_FAILED)
  - Batch rendering support
  - Statistics tracking
- SpaceRenderingServiceBuilder: Fluent builder pattern

60 unit tests covering all components.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 08:42:27 +01:00
7da77396a9 feat(spaces): implement Phase 3 Persistent Transclusion Context
Implements persistent transclusion context for Information Spaces:

- ScopedVariables: Variable scope layers (request > document > space)
- SpaceTransclusionContext: Extends TransclusionContext with DB persistence
- CrossSpaceResolver: Resolve references across space boundaries
- ReferenceGraph: Track document dependencies for cache invalidation
- PersistentReferenceGraph: Repository-backed reference tracking
- RenderCache: Cache rendered output with invalidation support
- CacheInvalidator: Event-driven cache invalidation using reference graph

Key features:
- Variable precedence: request overrides document overrides space
- Reference tracking during transclusion processing
- Transitive dependent calculation for cache invalidation
- Event bus integration for automatic invalidation on content changes

47 unit tests covering all components.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 08:36:50 +01:00
0a494b2011 feat(spaces): implement Phase 2 Event System
Week 4 - Event Infrastructure:
- Create SpaceEventType enum with 18 event types covering space lifecycle,
  document operations, variables, references, rendering, sync, and cache
- Create SpaceEvent dataclass with serialization/deserialization
- Create EventBus with sync/async handler support, priority ordering,
  global handlers, and optional event history
- Add event factory functions for common events

Week 5 - Event Integration:
- Wire EventBus into SpaceService as optional dependency
- Emit events for all space operations:
  - SPACE_CREATED, SPACE_UPDATED, SPACE_DELETED, SPACE_ACTIVATED, SPACE_ARCHIVED
  - DOCUMENT_ADDED, DOCUMENT_REMOVED, DOCUMENT_MOVED, DOCUMENT_CONTENT_CHANGED
  - VARIABLE_SET, VARIABLE_DELETED
- Create integration tests for event propagation patterns

Test coverage: 187 tests total
- 43 unit tests for event system
- 20 integration tests for event propagation
- 124 existing tests continue to pass

Capabilities delivered:
- CAP-010: SpaceEvent base with type, payload, timestamp
- CAP-011: EventBus with in-process publish/subscribe
- CAP-012: Event handlers registry with priority support
- CAP-013: Change detection via content hash comparison

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 07:41:47 +01:00
9b12875681 feat(spaces): implement Phase 0-1 of Information Space Service
Phase 0 - Project Organization:
- Create docs/PROJECT_STRUCTURE.md documenting codebase layout
- Create markitect/core/ with parser, serializer, document_manager, workspace
- Create markitect/schema/ consolidating 6 schema_*.py modules
- Create markitect/storage/ with database module
- Maintain backward compatibility via re-exports from original locations
- Add docs/roadmap/information-space-service/ with README and WORKPLAN

Phase 1 - Foundation (Weeks 1-3):
- Week 1: Core domain models (InformationSpace, SpaceDocument, SpaceConfig,
  SpaceMetadata, SpaceVariable, TransclusionReference, SpaceStatus)
- Week 2: Repository layer with interfaces (ISpaceRepository,
  IDocumentAssociationRepository, IVariableRepository, IReferenceRepository)
  and SQLite implementations with foreign key cascade deletes
- Week 3: SpaceService orchestration layer with full CRUD, document,
  variable, and reference tracking operations

Test coverage: 124 tests (25 model + 63 repository + 36 integration)

Capabilities delivered:
- CAP-001: InformationSpace entity with lifecycle management
- CAP-002: SpaceRepository CRUD with SQLite backing
- CAP-003: Document-Space associations with path-based organization
- CAP-004: Space metadata and configuration schemas
- CAP-005: Database schema with migrations

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-08 02:02:46 +01:00
7515b9c0e5 feat: implement optimization #8 - schema auto-ingestion
Add automated schema ingestion from markitect/schemas/ directory:

- Create auto_ingest_schemas() function in schema_loader module
- Automatically detect and ingest .md schema files from schemas/
- Skip schemas that are already ingested in database
- Return detailed results with ingested/skipped/failed lists
- Add 'markitect schema-auto-ingest' CLI command
- Support verbose mode for detailed progress reporting
- Useful for post-install setup and development workflows

This eliminates the manual step of running schema-ingest for each
bundled schema file, streamlining schema management.

Usage:
  markitect schema-auto-ingest           # Ingest all new schemas
  markitect schema-auto-ingest --verbose # Show detailed progress

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-06 21:34:46 +01:00
c4ee5cc645 feat: add changelog schema for Keep a Changelog validation
Created comprehensive changelog-schema-v1.0.md to validate CHANGELOG.md
files following the Keep a Changelog format. This schema demonstrates
the practical application of the schema evolution system.

**Schema Features**:
- Section validation: Enforces [Unreleased] section presence
- Version format validation: [X.Y.Z] - YYYY-MM-DD pattern
- Semantic versioning compliance
- ISO 8601 date format checking
- Change type subsections: Added, Changed, Deprecated, Removed, Fixed, Security
- Content pattern matching via x-markitect-content-control extensions
- Structural validation via JSON Schema properties

**Validation Results**:
 Successfully validates project CHANGELOG.md
 All section requirements met (7 sections checked, 11 found)
 All content requirements met
 All semantic checks passing

**Implementation Notes**:
- H1 "Changelog" title validated via JSON Schema structural checks
- H2 sections validated via x-markitect-sections classifications
- SectionValidator limitation: Only checks H2+ headings, not H1
- Workaround: Structural validation covers H1 title requirement

**Philosophy**: "The release that validates itself"
- v0.10.0 uses its own schema system to validate its CHANGELOG
- Perfect showcase of schema evolution practical value
- Demonstrates x-markitect extensions in real-world use case

**Stage 2 Complete** per release-management-optimization workplan.

Files:
- markitect/schemas/changelog-schema-v1.0.md (new)
- CHANGELOG.md (documented new schema)
2026-01-06 13:31:02 +01:00
5e3646fdff feat: complete schema-evolution topic with ADR schema and markdown support
This commit closes the schema-evolution topic (260105) by adding the final
deliverable (ADR schema) and fixing markdown schema support across commands.

**ADR Schema Created**:
- Comprehensive Architecture Decision Record validation schema
- 12 section classifications (7 required, 2 recommended, 2 optional, 3 improper/discouraged)
- Content pattern validation for ADR formatting rules (status dates, decision statements, rationale structure)
- Quality metrics for completeness (word counts, sentence counts)
- Follows title case naming convention (Status, Context, Decision, etc.)

**Markdown Schema Support Fixed**:
- Fixed `markitect validate` command to support .md schemas
  - Added load_schema_from_path() for both .json and .md files
  - Updated structural and semantic validation to use schema dict
- Fixed `markitect generate-stub` command to support .md schemas
  - Uses load_schema_from_path() instead of direct JSON loading
- Created DocumentWrapper class in semantic_validator.py
  - Extracts headings from AST tokens (heading_open, inline)
  - Provides get_headings_by_level() interface expected by validators
  - Enables section validation to work with real documents

**Topic Closure**:
- Updated SCHEMA_EVOLUTION_WORKPLAN.md with completion summary
  - Phases 1-3: 100% complete (via Schema-of-Schemas and Semantic Validation)
  - Phase 4: Deferred as future enhancement (15-20 sessions)
  - Phase 5: 70% complete (docs done, CI/CD templates deferred)
- Created DONE.md with comprehensive task checklist
- Generated ADR template stub (examples/templates/adr-template.md)
- Moved topic from roadmap/ to history/260105-schema-evolution/

**Files Changed**:
- markitect/cli.py: Added markdown schema support to validate and generate-stub
- markitect/semantic_validator.py: Added DocumentWrapper class for AST parsing
- markitect/schemas/adr-schema-v1.0.md: New ADR validation schema (560 lines)
- examples/templates/adr-template.md: Generated ADR template stub
- history/260105-schema-evolution/: Moved completed topic to history

**Status**: Schema evolution topic successfully closed with ADR schema as final deliverable.
All schema commands now support markdown schemas. Section validation working correctly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-06 12:32:38 +01:00
20c0cfece7 feat: add LinkValidator for semantic link validation (Phase 3)
Implement comprehensive link validation as part of semantic validation:

Core Features:
- Link classification: internal, external, fragment, email
- Internal link validation: fragment anchors and file paths
- External link validation: HTTP/HTTPS with configurable timeout
- Email validation: mailto: link format checking
- Fragment policy enforcement: allow/disallow fragment identifiers

Link Validator:
- markitect/validators/link_validator.py - Full link validation implementation
- Supports x-markitect-content-control.link_validation configuration
- Default: check internal links, skip external (fast)
- Opt-in external checking with --check-links flag

Integration:
- Updated SemanticValidator to include link_result in reports
- CLI already supports --check-links flag (line 1629 in cli.py)
- Link validation runs by default for internal links (fast)
- External link checking requires explicit --check-links flag

Test Coverage:
- Added 9 comprehensive tests for LinkValidator
- Tests cover: classification, broken links, fragments, email, statistics
- All 25 semantic validator tests passing (100%)

Documentation:
- Updated SCHEMA_MANAGEMENT_GUIDE.md with link validation section
- Added examples for broken links and external link checking
- Documented link types, validation rules, and configuration

Statistics Tracking:
- Links checked, internal/external/fragment/email counts
- Detailed error/warning reporting with line numbers
- Integration with existing semantic validation reporting

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-06 03:41:03 +01:00
5264a6083c feat: enhance validate command with semantic validation
Integrates SemanticValidator into CLI validate command:

New Options:
- --semantic/--no-semantic (default: True) - Enable/disable semantic validation
- --check-links - Enable link validation (requires semantic validation)
- --strict - Treat warnings as errors (fail on WARNING-level issues)

Features:
- Automatically detects x-markitect extensions in schema
- Runs semantic validation alongside structural validation
- Combines results with clear separation in output
- Maintains full backward compatibility (--no-semantic for classic mode)
- Supports .md schema files with embedded JSON
- Graceful degradation: semantic validation errors don't crash command

Example Usage:
  # Full validation (structural + semantic)
  markitect validate doc.md --schema manpage-schema-v1.0.md

  # Strict mode (warnings = errors)
  markitect validate doc.md --schema schema.md --strict

  # Classic mode (structural only)
  markitect validate doc.md --schema schema.json --no-semantic

Output Format:
- Shows structural validation results first
- Then semantic validation results (sections, content)
- Clear summary with error/warning counts
- Exit codes: 0=pass, 1=fail (respects --strict flag)

Integration: cli.py:1493-1668

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-06 03:27:39 +01:00
a969c5de47 feat: add semantic document validator for x-markitect extensions
Implements semantic validation to complement existing structural validation:

Phase 1 & 2 Complete:
- SemanticValidator: Main validator orchestrating sub-validators
- SectionValidator: Enforces section classifications (required, recommended,
  optional, discouraged, improper) from x-markitect-sections
- ContentValidator: Validates content patterns, forbidden patterns, and
  quality metrics (word counts, sentence counts) from x-markitect-content-control

Features:
- Pattern matching with regex for required/forbidden/discouraged patterns
- Word count and sentence count validation
- Detailed error reporting with severity levels (ERROR, WARNING)
- Support for section alternatives (e.g., FLAGS vs OPTIONS)
- Comprehensive test coverage (16 tests, 100% passing)

Architecture:
- Complements existing SchemaValidator (structural AST validation)
- Clean separation: validators/ package for modular validators
- Semantic validation focuses on x-markitect-* extensions
- LinkValidator planned for Phase 3 (optional --check-links)

Next: Phase 4 - CLI integration to enhance 'markitect validate' command

Workplan: roadmap/20260106-semantic-document-validation/WORKPLAN.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-06 03:24:32 +01:00
3003b9b8da chore: archive completed schema-of-schemas implementation
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Moved schema-of-schemas planning artifacts from roadmap to history
with datestamp prefix, marking completion of all 6 implementation phases.

**Changes:**
- Moved roadmap/schema-of-schemas/ → history/2026-01-05-schema-of-schemas/
- Updated all documentation references to new location
- Marked implementation as completed in TODO.md
- Updated CHANGELOG.md to reflect archived status

**Implementation Summary:**
All 6 phases completed successfully:
- Phase 1: Filename validation (50 tests)
- Phase 2: Markdown schema loader (35 tests)
- Phase 3: Schema-for-schemas metaschema (12 tests)
- Phase 4: Schema migration (2 migrated, 3 deleted)
- Phase 5: CLI enhancements (multi-schema validation)
- Phase 6: Integration testing and documentation

**Deliverables:**
- 97 unit tests (100% passing)
- 4 production schemas in registry
- Comprehensive user documentation
- Updated examples (manpages, terminology)
- Complete schema management system

The schema-of-schemas topic is now complete and archived for
historical reference.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 14:13:48 +01:00
7d115b6325 feat: add multi-schema validation with numbered selection
Enhanced schema-list and schema-validate commands to support efficient
batch validation of multiple schemas, especially useful when the
metaschema changes.

**schema-list enhancements:**
- Added numbered references (#1, #2, etc.) to all output formats
- Simple format: [1] prefix for each schema
- Table format: # column as first column
- JSON/YAML: number field added to each schema

**schema-validate enhancements:**
- Number selection: `markitect schema-validate 1`
- Range selection: `markitect schema-validate 1-3`
- List selection: `markitect schema-validate 1,3,5`
- Batch validation: `markitect schema-validate --all`
- Filename selection: `markitect schema-validate schema.md`
- Filesystem path: `markitect schema-validate ./schema.md`
- Batch results displayed as clear summary table
- Registry schemas take precedence with filesystem fallback
- Full backward compatibility maintained

**Implementation details:**
- Added ValidationResult dataclass for structured results
- Added helper functions: parse_schema_selector, resolve_schema_source,
  is_filesystem_path, format_validation_summary
- Changed schema_selector from Path to str for flexible input
- Added --all flag for validating all registered schemas
- Comprehensive error handling and helpful usage messages

**Testing:**
- All selection methods tested and working
- Backward compatibility verified
- Parsing utilities tested with unit tests

Completes Phase 5 of Schema-of-Schemas implementation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 10:55:48 +01:00
60d9f7a2c3 feat: implement Phase 4 - Schema Migration
Completed Phase 4 of the schema-of-schemas implementation with successful
migration of all legacy schemas to the new markdown format following the
naming convention.

Migration Script (scripts/migrate_schemas.py - 240 lines):
- Automated schema migration from JSON to markdown format
- Updates version and $id fields to follow conventions
- Generates proper frontmatter metadata
- Dry-run mode for safe testing
- Database cleanup functionality
- Comprehensive progress reporting

Schemas Migrated (2):
- terminology-schema.json → terminology-schema-v1.0.md
  - Fixed missing version field
  - Updated $id from /terminology-v1.json to /terminology/v1.0
  - Validates successfully against metaschema

- api-documentation → api-documentation-schema-v1.0.md
  - Added version: 1.0.0
  - Updated $id to follow /api-documentation/v1.0 format
  - Validates successfully against metaschema

Schemas Deleted (3):
- markdown-manpage (duplicate of manpage-schema-v1.0.md)
- markdown-manpage-schema.json (duplicate of manpage-schema-v1.0.md)
- enhanced-manpage (replaced by manpage-schema-v1.0.md)

CLI Enhancement (markitect/cli.py):
- Updated schema-ingest to support markdown (.md) files
- Auto-detects file type and uses MarkdownSchemaLoader for .md files
- Extracts JSON schema from markdown for database storage
- Maintains backward compatibility with JSON files

Final Schema Registry (4 schemas):
 terminology-schema-v1.0.md - Terminology validation
 api-documentation-schema-v1.0.md - API documentation structure
 manpage-schema-v1.0.md - Unix manual pages
 schema-schema-v1.0.md - Metaschema for validating schemas

All schemas:
- Follow naming convention: {domain}-schema-v{major}.{minor}.md
- Include proper frontmatter with schema-id, version, status
- Validate successfully against schema-schema-v1.0.md metaschema
- Stored in database and ready for use

Progress Tracking:
- Updated TODO.md with Phase 4 completion
- Updated CHANGELOG.md with migration details
- Next: Phase 5 - CLI & Documentation Updates

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 09:38:43 +01:00
f3aaec99bb feat: implement Phase 3 - Schema-for-Schemas Metaschema
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Completed Phase 3 of the schema-of-schemas implementation with a
comprehensive metaschema that validates all MarkiTect schema files
against conventions and standards.

Metaschema Implementation (schema-schema-v1.0.md - 650+ lines):
- Validates core JSON Schema fields ($schema, $id, title, description)
- Validates MarkiTect version field (SemVer: major.minor.patch)
- Validates $id URL format (HTTPS with version path)
- Validates MarkiTect extensions:
  - x-markitect-sections: section classifications and content rules
  - x-markitect-content-control: pattern and quality validation
  - x-markitect-metadata: status, authors, tags
  - x-markitect-source: loader metadata (auto-added)
- Section classification validation (required, recommended, optional,
  discouraged, improper)
- Content control pattern validation
- Comprehensive documentation with examples and usage guides

CLI Command (markitect schema-validate):
- Validates schema files against metaschema
- Supports both markdown and JSON schema files
- Detailed error reporting with schema paths
- Structure validation recommendations
- Exit codes for CI/CD integration

Test Coverage (tests/test_schema_metaschema.py - 12 tests, 100% passing):
- Metaschema self-validation
- Manpage schema validation
- Required fields enforcement
- Version format validation (valid and invalid cases)
- $id format validation (valid and invalid cases)
- Section classification validation
- Complete schema with all extensions

Validation Results:
-  Metaschema validates itself successfully
-  Manpage schema (v1.0.md) validates successfully
- ⚠️  Terminology schema needs migration (missing version, incorrect $id)

Progress Tracking:
- Updated TODO.md with Phase 3 completion
- Updated CHANGELOG.md with implementation details
- Next: Phase 4 - Schema Migration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 03:10:49 +01:00
b81ce5631d feat: implement Phase 2 - Markdown Schema Loader
Completed Phase 2 of the schema-of-schemas implementation with full
markdown schema support. This enables schemas to be authored as
markdown files with rich documentation and embedded JSON schemas.

Core Implementation (markitect/schema_loader.py):
- MarkdownSchemaLoader class with comprehensive parsing capabilities
- YAML frontmatter extraction with error handling
- JSON code block extraction with section preference (## Schema Definition)
- Metadata merging with x-markitect-source tracking
- Schema saving with template support and round-trip capability
- Helper methods: list_json_blocks(), validate_schema_structure()

Test Coverage (tests/test_schema_loader.py):
- 35 comprehensive unit tests (100% passing)
- Tests for loading, parsing, saving, round-trip conversion
- Edge case handling (empty files, binary files, malformed blocks)
- Fixed binary file test to use invalid UTF-8 sequences

Example Schema (markitect/schemas/manpage-schema-v1.0.md):
- First markdown schema following naming convention
- Complete manpage schema with frontmatter + documentation + JSON
- Demonstrates section classification and content control
- Shows proper structure for future schema authors

Documentation (roadmap/schema-of-schemas/SCHEMA_LOADER_GUIDE.md):
- Comprehensive user guide (600+ lines)
- API reference with examples
- Best practices and troubleshooting
- Integration patterns for CLI and validator

Progress Tracking:
- Updated TODO.md with Phase 2 completion
- Updated CHANGELOG.md with implementation details
- Next: Phase 3 - Schema-for-Schemas Metaschema

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-05 00:02:15 +01:00
14108533fb feat: implement schema filename validation (Phase 1 complete)
Implements filename convention enforcement for schema files as part of
the schema-of-schemas implementation. All schemas must now follow the
naming pattern: {domain}-schema-v{major}.{minor}.md

## Phase 1 Deliverables

### Schema Naming Module
**File:** `markitect/schema_naming.py` (380 lines)

**Functions:**
- `validate_schema_filename()` - Validate filename against pattern
- `suggest_schema_filename()` - Generate valid filename from domain/version
- `extract_schema_metadata()` - Extract domain and version from filename
- `get_validation_errors()` - Detailed error messages for invalid filenames
- `is_valid_schema_filename()` - Simple boolean validation
- `format_validation_message()` - User-friendly error formatting

**Features:**
- Regex-based pattern matching
- Automatic normalization (spaces → hyphens, lowercase)
- Detailed error reporting
- Domain validation (must start with letter)
- Version validation (major.minor format)

### Comprehensive Test Suite
**File:** `tests/test_schema_naming.py` (500+ lines, 50 tests)

**Test Coverage:**
-  Valid filename variations (simple, hyphenated, with numbers)
-  Invalid filenames (wrong extension, missing components, wrong case)
-  Filename suggestion with normalization
-  Metadata extraction
-  Error message generation
-  Edge cases (long names, many hyphens, large versions)
-  Pattern regex validation

**Results:** 50/50 tests passing (100%)

### Specification Document
**File:** `roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md`

**Contents:**
- Formal specification of naming convention
- Regular expression pattern with explanation
- Valid and invalid examples
- Version numbering guidelines
- Domain naming best practices
- Normalization rules
- Migration strategy from legacy naming
- Implementation guide

## Naming Convention

### Format
```
{domain}-schema-v{major}.{minor}.md
```

### Examples
```
✓ manpage-schema-v1.0.md
✓ api-documentation-schema-v1.0.md
✓ terminology-schema-v1.0.md
✓ arc42-schema-v2.1.md

✗ manpage.json (wrong extension)
✗ ManPage-schema-v1.0.md (uppercase)
✗ manpage-v1.0.md (missing 'schema')
✗ manpage-schema-v1.md (missing minor version)
```

### Components
- **domain**: Lowercase, hyphen-separated, starts with letter
- **schema**: Literal keyword
- **version**: v{major}.{minor} (SemVer simplified)
- **extension**: .md (markdown)

## Implementation Highlights

### Automatic Normalization
```python
suggest_schema_filename("API Documentation", "2.1")
# → "api-documentation-schema-v2.1.md"

suggest_schema_filename("My_Custom Type", "1.0")
# → "my-custom-type-schema-v1.0.md"
```

### Detailed Error Reporting
```python
format_validation_message("invalid.json")
# → Detailed error list + suggested fix
```

### Metadata Extraction
```python
extract_schema_metadata("manpage-schema-v1.0.md")
# → {'domain': 'manpage', 'version': '1.0', 'major': 1, 'minor': 0}
```

## Migration Plan

Current schemas will be renamed:
```
Old                           → New
────────────────────────────────────────────────────────
terminology-schema.json       → terminology-schema-v1.0.md
api-documentation             → api-documentation-schema-v1.0.md
enhanced-manpage              → manpage-schema-v2.0.md
markdown-manpage              → DELETE (duplicate)
markdown-manpage-schema.json  → DELETE (duplicate)
```

## Phase 1 Status:  COMPLETE

### Completed
- [x] Schema naming module implementation
- [x] Comprehensive test suite (50 tests, 100% passing)
- [x] Specification document
- [x] TODO.md updated

### Next: Phase 2
- [ ] Update CLI schema-ingest with validation
- [ ] Implement markdown schema loader
- [ ] Parse frontmatter and JSON code blocks
- [ ] Update SchemaValidator for .md support

## Testing

```bash
# Run tests
pytest tests/test_schema_naming.py -v
# → 50 passed in 0.48s

# Test interactively
python -c "
from markitect.schema_naming import validate_schema_filename
print(validate_schema_filename('manpage-schema-v1.0.md'))
"
# → (True, {'domain': 'manpage', 'version': '1.0', ...})
```

## Files Changed

- markitect/schema_naming.py (NEW, 380 lines)
- tests/test_schema_naming.py (NEW, 500+ lines)
- roadmap/schema-of-schemas/SCHEMA_NAMING_SPEC.md (NEW)
- TODO.md (updated progress tracking)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 23:51:29 +01:00
6df9b5df05 feat: add terminology schema example and improve schema-list command
This commit completes Phase 2 of schema evolution work and establishes
a new example demonstrating schema usage for terminology documents.

## New Features

### Terminology Validation Example (examples/terminology/)
- Complete example terminology document with proper structure
- JSON schema with MarkiTect extensions for validation
- Demonstrates schema usage beyond manpages (glossaries, lexicons)
- Validates term structure: Definition, Synonyms, Related Terms, Examples
- Includes content control and quality validation rules
- Full documentation with usage examples and best practices

### Schema Registration System
- Registered terminology schema in markitect database
- Created schema catalog (markitect/schemas/schema-catalog.yaml)
- Copied schema to official location (markitect/schemas/)
- Provides metadata, features, and usage info for all schemas

### Improved schema-list Command
- Now displays creation timestamps in default output
- Table format includes Created/Updated columns
- Cleaner timestamp formatting (removed microseconds)
- Better visibility into when schemas were added

## Files Changed

Added:
- examples/terminology/README.md - Complete documentation
- examples/terminology/terminology-example.md - Example glossary
- examples/terminology/terminology-schema.json - Validation schema
- markitect/schemas/terminology-schema.json - Registered schema
- markitect/schemas/schema-catalog.yaml - Schema registry

Modified:
- markitect/cli.py - Enhanced schema-list with timestamps
- TODO.md - Documented Phase 2 completion and new example

Moved:
- SCHEMA_EVOLUTION_WORKPLAN.md → todo/ directory

## Schema Features Demonstrated

- Heading hierarchy validation (H1 → H2 → H3)
- Term structure validation with required/optional fields
- Content quality metrics (word counts, readability targets)
- MarkiTect extensions (x-markitect-sections, x-markitect-content-control)
- Classification system (required/recommended/optional/discouraged/improper)

## Usage

```bash
# List schemas with timestamps
markitect schema-list

# Validate terminology document
markitect validate glossary.md --schema terminology-schema.json

# View in table format
markitect schema-list --format table
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 23:07:36 +01:00
48e0b60be5 feat: add interactive mode to schema-refine command
Added --interactive/-i flag to schema-refine command that allows users to
review and approve each refinement individually:

- Displays each detected issue with details
- Shows current and suggested values
- Prompts for confirmation (y/N/q)
- Applies only approved fixes
- Shows summary at completion

This gives users fine-grained control over which refinements to apply.

Example usage:
  markitect schema-refine schema.json --interactive

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 21:30:55 +01:00
2b35fcde62 feat: add Phase 2 schema refinement tools (schema-analyze and schema-refine)
Implemented two new CLI commands for schema analysis and refinement:

1. schema-analyze: Analyzes schemas for rigidity issues
   - Detects exact counts that should be ranges
   - Identifies missing classification system
   - Flags deprecated extensions
   - Calculates rigidity score (0-100)
   - Provides detailed or summary reports

2. schema-refine: Automatically refines rigid schemas
   - Converts exact counts to flexible ranges
   - Rounds overly specific numbers
   - Widens narrow integer constraints
   - Supports dry-run mode
   - Can save to new file or overwrite in place

Key improvements:
- Created SchemaAnalyzer class with issue detection
- Created SchemaRefiner class with automatic fixes
- Improved schema navigation to handle nested properties
- Tested on example schemas (reduced rigidity from 60/100 to 24/100)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 21:29:08 +01:00
d68e762612 feat: implement Phase 1 - Enhanced Schema Format with Classifications
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Complete Phase 1 of Schema Evolution Workplan implementing flexible content
control and section classification system.

## New Features

### 1. x-markitect-sections Extension
- Five classification levels: required, recommended, optional, discouraged, improper
- Per-section content constraints (paragraphs, code blocks, lists)
- Position hints for section ordering
- Custom error/warning messages
- Alternative section names support
- Content instructions for authors

### 2. x-markitect-content-control Extension
- Required/discouraged/forbidden pattern matching
- Content quality metrics (word count, readability target, sentence count)
- Content instruction arrays
- Link validation configuration

### 3. Metaschema Validation
- Updated markitect-metaschema.json with complete validation rules
- Enhanced metaschema.py with validation methods for both extensions
- Comprehensive validation of all extension properties
- Clear error messages for invalid schemas

### 4. Documentation & Examples
- Complete specification in docs/specifications/schema-extensions-spec.md
- Enhanced manpage schema demonstrating all 5 classification levels
- API documentation schema showing alternative patterns
- Detailed usage examples and validation behavior

## Implementation Details

**Files Modified:**
- markitect/schemas/markitect-metaschema.json: Added extension definitions
- markitect/metaschema.py: Added _validate_sections() and _validate_content_control()

**Files Created:**
- docs/specifications/schema-extensions-spec.md: Complete specification (v1.0)
- examples/manpages/enhanced-manpage-schema.json: Demonstrates all classifications
- examples/manpages/api-documentation-schema.json: Shows API doc patterns

## Validation Behavior

**Classification Levels:**
- required: Missing = ERROR (validation fails)
- recommended: Missing = WARNING (validation succeeds with warnings)
- optional: No validation impact
- discouraged: Present = WARNING (validation succeeds with warnings)
- improper: Present = ERROR (validation fails)

## Next Steps

Phase 2: Schema Refinement Tools (schema-analyze, schema-refine, schema-compose)
Phase 3: Enhanced Validation Engine (classification-aware validation, quality metrics)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-04 21:02:51 +01:00
21189f7664 fix: CSS injection and theme application bugs
This commit fixes two related bugs and removes obsolete tests from the old architecture.

Bug Fixes:
1. CSS Injection Bug: --css option now properly reads and injects custom CSS files
   - Added {css_content} placeholder to document.html template
   - Implemented CSS file reading logic in both view and edit modes
   - Custom CSS is now correctly embedded in generated HTML

2. Theme Application Bug: ChatGPT and Substack themes now render correctly
   - Theme CSS generation was working but wasn't being injected
   - Fixed by adding CSS placeholder replacement logic
   - All theme tests now passing

Test Suite Cleanup (46 obsolete tests removed):
- test_clean_architecture.py (5 tests) - tested old embedded JS approach
- test_issue_132_basic_rendering.py (5 tests) - tested old HTML generation
- test_issue_132_template_system.py (8 tests) - tested old template system
- test_issue_133_cli_integration.py (10 tests) - tested old edit mode
- test_issue_144_edit_mode_regression.py (11 tests) - tested old JS bugs
- test_js_sanity.py (7 tests) - tested old JS validation

These tests were validating the old architecture before the testdrive-jsui v1.0.0 migration.
The new architecture uses standalone JavaScript library, making these tests obsolete.

Test Results:
- Before: 1,256 tests, 1,166 passed, 52 failed (92.8% pass rate)
- After: 1,210 tests, 1,160 passed, 0 failed (100% pass rate)

Modified Files:
- markitect/templates/document.html: Added {css_content} placeholder
- markitect/clean_document_manager.py: Added CSS file reading and injection logic

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-17 12:02:42 +01:00
d592c5b8b3 feat: Complete Phase 4 - Remove legacy JavaScript files
Phase 4 Complete: Cleanup legacy files after successful migration

Removed Files (29 total):
- /markitect/static/js/ directory (entire directory deleted)
  * Core modules: debug-system.js, section-manager.js
  * Components: debug-panel.js, dom-renderer.js, document-controls.js
  * Configuration: config-loader.js
  * Main files: main.js, main-updated.js
  * Plugins: document-navigator-plugin.js
  * Widgets: UIWidget.js, Widget.js, DocumentNavigator.js
  * Test files: All test JS files and test HTML/MD files
- /markitect/static/editor.js (unused legacy file)

Preserved:
- /markitect/static/css/ (still referenced in templates)

Migration Impact:
-  Single source of truth: All JavaScript now in /capabilities/testdrive-jsui/js/
-  No duplicate files in codebase
-  Clean separation: Capability is authoritative location
-  All tests still passing (84 automated tests)
-  Main app rendering verified (view & edit modes)

Migration Status:
- Phase 1:  Complete (files copied to capability)
- Phase 2: ⏭️ Skipped (comprehensive testing in Phase 1)
- Phase 3:  Complete (templates updated)
- Phase 4:  Complete (legacy files removed)

🎉 MIGRATION FULLY COMPLETE - All phases done

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-16 10:27:05 +01:00