feat: reorganize tests by capability with separate test targets
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Some checks failed
Test Suite / unit-tests (3.11) (push) Has been cancelled
Test Suite / unit-tests (3.12) (push) Has been cancelled
Test Suite / integration-tests (push) Has been cancelled
Test Suite / e2e-tests (push) Has been cancelled
Test Suite / performance-tests (push) Has been cancelled
Test Suite / code-quality (push) Has been cancelled
Test Suite / security-scan (push) Has been cancelled
Test Suite / test-summary (push) Has been cancelled
Separate capability-specific tests from core system tests to establish clear test organization and separation of concerns. ## Test Reorganization: - **markitect-content tests**: Moved 6 tests to capabilities/markitect-content/tests/ - **markitect-finance tests**: Moved 7 tests to markitect/finance/tests/ - **markitect-query tests**: Moved 1 test to markitect/query_paradigms/tests/ - **markitect-graphql tests**: Moved 2 tests to markitect/graphql/tests/ - **markitect-plugins tests**: Moved 2 tests to markitect/plugins/tests/ ## Makefile Updates: - **make test**: Excludes capability tests, runs only core system tests - **make test-capabilities**: Runs all capability tests - **make test-capability-***: Individual capability test targets - Updated all test targets (test-red, test-green, test-ultra-fast, test-perf) - Added capability test targets to help documentation ## Benefits: - Clear separation between core system tests and capability-specific tests - Faster core test execution (capability tests not run by default) - Individual capability testing for focused development - Supports future capability extraction workflow - Maintains capability test independence Test verification: - Core tests: 1291 tests (capability tests excluded) - Finance capability: 143 tests working independently - Content capability: 79 tests working independently 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,295 @@
|
||||
"""
|
||||
Integration tests for complete MarkdownMatters CLI implementation.
|
||||
Tests all four command families working together.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from click.testing import CliRunner
|
||||
|
||||
from markitect_content.commands import content_get, content_stats
|
||||
from markitect.matter_frontmatter.commands import frontmatter_get, frontmatter_keys
|
||||
from markitect.matter_contentmatter.commands import contentmatter_get, contentmatter_keys
|
||||
from markitect.matter_tailmatter.commands import tailmatter_get, tailmatter_check
|
||||
|
||||
|
||||
class TestMarkdownMattersIntegration:
|
||||
"""Test complete MarkdownMatters functionality integration."""
|
||||
|
||||
@pytest.fixture
|
||||
def complete_document(self):
|
||||
"""A complete MarkdownMatters document with all three zones."""
|
||||
return """---
|
||||
title: "Complete MarkdownMatters Document"
|
||||
author: "Integration Test"
|
||||
version: 1.0
|
||||
status: "testing"
|
||||
---
|
||||
|
||||
# Complete MarkdownMatters Document
|
||||
|
||||
This document demonstrates all three matter zones working together.
|
||||
|
||||
Author: Dr. Test Researcher
|
||||
Institution: MarkdownMatters University
|
||||
Email: test@markdownmatters.edu
|
||||
Project: Integration Testing
|
||||
Version: 2.0
|
||||
Status: Active
|
||||
|
||||
## Research Content
|
||||
|
||||
Research Method: Integration Testing
|
||||
Sample Size: Complete document
|
||||
Test Framework: MarkdownMatters CLI
|
||||
|
||||
The content includes various MultiMarkdown key-value pairs that provide contextual metadata.
|
||||
|
||||
## Results
|
||||
|
||||
Result Status: All systems operational
|
||||
Performance: Excellent
|
||||
Coverage: 100%
|
||||
|
||||
All matter zones are properly separated and accessible through their respective CLI commands.
|
||||
|
||||
---
|
||||
|
||||
```yaml tailmatter
|
||||
qa_checklist:
|
||||
- requirement: "All three matter zones tested"
|
||||
complete: true
|
||||
- requirement: "CLI commands validated"
|
||||
complete: true
|
||||
- requirement: "Integration verified"
|
||||
complete: false
|
||||
|
||||
editorial:
|
||||
status: "Integration Testing"
|
||||
reviewer: "integration.tester@markdownmatters.edu"
|
||||
version: 3.0
|
||||
|
||||
agent_config:
|
||||
role: "integration_validator"
|
||||
access_scope: "all_zones"
|
||||
validation_mode: "comprehensive"
|
||||
```"""
|
||||
|
||||
@pytest.fixture
|
||||
def runner(self):
|
||||
"""CLI test runner."""
|
||||
return CliRunner()
|
||||
|
||||
def test_all_command_families_work_on_same_document(self, runner, complete_document):
|
||||
"""Test that all four command families can process the same document."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test content commands
|
||||
result = runner.invoke(content_get, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Complete MarkdownMatters Document" in result.output
|
||||
assert "---" not in result.output # No frontmatter
|
||||
assert "qa_checklist" not in result.output # No tailmatter
|
||||
|
||||
result = runner.invoke(content_stats, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "word_count" in result.output
|
||||
|
||||
# Test frontmatter commands
|
||||
result = runner.invoke(frontmatter_get, ['title', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Complete MarkdownMatters Document" in result.output
|
||||
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "title" in result.output
|
||||
assert "author" in result.output
|
||||
|
||||
# Test contentmatter commands
|
||||
result = runner.invoke(contentmatter_get, ['Author', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Dr. Test Researcher" in result.output
|
||||
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Author" in result.output
|
||||
assert "Institution" in result.output
|
||||
|
||||
# Test tailmatter commands
|
||||
result = runner.invoke(tailmatter_get, ['editorial.status', '--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "Integration Testing" in result.output
|
||||
|
||||
result = runner.invoke(tailmatter_check, ['--file', temp_file])
|
||||
assert result.exit_code == 0
|
||||
assert "QA Checklist Status" in result.output
|
||||
assert "✅" in result.output
|
||||
assert "❌" in result.output
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_matter_zone_separation(self, runner, complete_document):
|
||||
"""Test that each command family only accesses its designated zone."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Frontmatter should not include contentmatter or tailmatter
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
assert "Author" not in result.output # This is contentmatter
|
||||
assert "qa_checklist" not in result.output # This is tailmatter
|
||||
|
||||
# Contentmatter should not include frontmatter or tailmatter
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
assert "title" not in result.output # This is frontmatter
|
||||
assert "qa_checklist" not in result.output # This is tailmatter
|
||||
|
||||
# Content should not include any matter zones in the actual content
|
||||
result = runner.invoke(content_get, ['--file', temp_file])
|
||||
assert "title:" not in result.output # No frontmatter YAML
|
||||
assert "qa_checklist:" not in result.output # No tailmatter YAML
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_performance_with_large_document(self, runner):
|
||||
"""Test performance with a large document containing all matter zones."""
|
||||
# Create a large document
|
||||
large_content = []
|
||||
large_content.append("---")
|
||||
large_content.append("title: 'Large Document Performance Test'")
|
||||
for i in range(50):
|
||||
large_content.append(f"field_{i}: 'value_{i}'")
|
||||
large_content.append("---")
|
||||
large_content.append("")
|
||||
|
||||
large_content.append("# Large Document Performance Test")
|
||||
large_content.append("")
|
||||
|
||||
# Add many contentmatter pairs
|
||||
for i in range(100):
|
||||
large_content.append(f"Data Field {i}: Value for field {i}")
|
||||
large_content.append("")
|
||||
|
||||
# Add substantial content
|
||||
for i in range(50):
|
||||
large_content.append(f"## Section {i}")
|
||||
large_content.append("")
|
||||
large_content.append(f"Content for section {i} with detailed information and multiple paragraphs.")
|
||||
large_content.append("")
|
||||
large_content.append("More content here to make the document substantial in size.")
|
||||
large_content.append("")
|
||||
|
||||
large_content.append("---")
|
||||
large_content.append("")
|
||||
large_content.append("```yaml tailmatter")
|
||||
large_content.append("qa_checklist:")
|
||||
for i in range(20):
|
||||
complete = "true" if i % 3 == 0 else "false"
|
||||
large_content.append(f" - requirement: 'Test requirement {i}'")
|
||||
large_content.append(f" complete: {complete}")
|
||||
large_content.append("editorial:")
|
||||
large_content.append(" status: 'Performance Testing'")
|
||||
large_content.append("```")
|
||||
|
||||
large_document = "\n".join(large_content)
|
||||
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(large_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test that all commands complete in reasonable time
|
||||
import time
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(content_stats, ['--file', temp_file])
|
||||
content_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert content_time < 2.0 # Should complete in under 2 seconds
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file])
|
||||
frontmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert frontmatter_time < 1.0 # Should complete in under 1 second
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file])
|
||||
contentmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert contentmatter_time < 2.0 # Should complete in under 2 seconds
|
||||
|
||||
start_time = time.time()
|
||||
result = runner.invoke(tailmatter_check, ['--file', temp_file])
|
||||
tailmatter_time = time.time() - start_time
|
||||
assert result.exit_code == 0
|
||||
assert tailmatter_time < 1.0 # Should complete in under 1 second
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
|
||||
def test_error_handling_consistency(self, runner):
|
||||
"""Test that all command families handle errors consistently."""
|
||||
non_existent_file = "/tmp/non_existent_file.md"
|
||||
|
||||
# All commands should handle missing files gracefully
|
||||
commands_and_args = [
|
||||
(content_get, ['--file', non_existent_file]),
|
||||
(content_stats, ['--file', non_existent_file]),
|
||||
(frontmatter_get, ['title', '--file', non_existent_file]),
|
||||
(frontmatter_keys, ['--file', non_existent_file]),
|
||||
(contentmatter_get, ['Author', '--file', non_existent_file]),
|
||||
(contentmatter_keys, ['--file', non_existent_file]),
|
||||
(tailmatter_get, ['editorial.status', '--file', non_existent_file]),
|
||||
(tailmatter_check, ['--file', non_existent_file]),
|
||||
]
|
||||
|
||||
for command, args in commands_and_args:
|
||||
result = runner.invoke(command, args)
|
||||
assert result.exit_code != 0 # Should fail for non-existent file
|
||||
|
||||
def test_help_commands_consistency(self, runner):
|
||||
"""Test that all commands provide consistent help."""
|
||||
commands = [
|
||||
content_get, content_stats,
|
||||
frontmatter_get, frontmatter_keys,
|
||||
contentmatter_get, contentmatter_keys,
|
||||
tailmatter_get, tailmatter_check
|
||||
]
|
||||
|
||||
for command in commands:
|
||||
result = runner.invoke(command, ['--help'])
|
||||
assert result.exit_code == 0
|
||||
assert "Usage:" in result.output
|
||||
assert "--help" in result.output
|
||||
|
||||
def test_output_format_consistency(self, runner, complete_document):
|
||||
"""Test that commands with format options work consistently."""
|
||||
with tempfile.NamedTemporaryFile(mode='w', suffix='.md', delete=False) as f:
|
||||
f.write(complete_document)
|
||||
temp_file = f.name
|
||||
|
||||
try:
|
||||
# Test JSON format consistency
|
||||
result = runner.invoke(content_stats, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('{')
|
||||
|
||||
result = runner.invoke(frontmatter_keys, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('[')
|
||||
|
||||
result = runner.invoke(contentmatter_keys, ['--file', temp_file, '--format', 'json'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.startswith('[')
|
||||
|
||||
finally:
|
||||
os.unlink(temp_file)
|
||||
Reference in New Issue
Block a user