coulomb/markitect-main
2
0
Fork 0
Code Issues 61 Pull Requests Actions Projects 12 Releases 1 Wiki Activity

feat: implement comprehensive plugin architecture and extensions system (issue #19)

Browse Source 
Complete plugin system implementation providing extensible architecture for MarkiTect:

🏗️ **Core Plugin Architecture**:
- BasePlugin abstract class with lifecycle management (initialize/cleanup)
- Specialized plugin types: ProcessorPlugin, FormatterPlugin, ValidatorPlugin, ExporterPlugin, CommandPlugin
- PluginMetadata system with version, dependencies, and type information
- Plugin initialization and configuration validation

🔍 **Plugin Discovery & Management**:
- PluginManager with automatic discovery from built-in modules and directories
- PluginRegistry for centralized plugin registration and lifecycle management
- Support for plugin loading, unloading, and reloading with configuration
- Plugin discovery from multiple sources (built-in, directories, packages)

🛠️ **CLI Integration**:
- markitect plugin-list: List all available plugins with metadata
- markitect plugin-load: Load plugins with optional configuration
- markitect plugin-unload: Unload plugins and cleanup resources
- markitect plugin-info: Show detailed plugin information
- markitect plugin-discover: Discover and refresh plugin catalog

📦 **Built-in Plugins**:
- JSON/YAML/Table formatters for output formatting
- Markdown/Text processors for content processing
- Auto-registered via @register_plugin decorator
- Comprehensive configuration options

🔧 **Developer Experience**:
- @register_plugin decorator for easy plugin registration
- Plugin configuration validation and error handling
- Comprehensive API documentation with examples
- Plugin development guide and best practices

📋 **Example Plugins**:
- Advanced text processor with case conversion and pattern replacement
- XML/CSV formatters demonstrating custom output formats
- Complete examples showing plugin development patterns

🧪 **Test Coverage**:
- 59 comprehensive tests covering all plugin functionality
- Tests for plugin lifecycle, registration, discovery, and CLI integration
- Error handling and edge case coverage
- Built-in plugin validation

Technical Implementation:
- Plugin types: processor, formatter, validator, exporter, generator, importer, transformer, extension, backend, command
- Configuration-driven plugin management with YAML/JSON support
- Graceful error handling and plugin isolation
- Plugin dependency validation and compatibility checking

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2025-10-03 11:23:32 +02:00
parent e6adb3e6db
commit b0de32d083
13 changed files with 3160 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

424
markitect/plugins/README.md Normal file
View File

@@ -0,0 +1,424 @@
# MarkiTect Plugin System
The MarkiTect Plugin System provides a flexible and extensible architecture for adding custom functionality to MarkiTect. Plugins can extend processors, formatters, validators, exporters, and more.
## Plugin Types
### Supported Plugin Types
- **Processor**: Content processors (markdown, text transformation, etc.)
- **Formatter**: Output formatters (JSON, YAML, tables, etc.)
- **Validator**: Content validators (schema validation, lint checking, etc.)
- **Exporter**: Export handlers (PDF, HTML, etc.)
- **Generator**: Content generators (templates, stubs, etc.)
- **Importer**: Import handlers (various formats)
- **Transformer**: Content transformers (data manipulation, etc.)
- **Extension**: General extensions (any functionality)
- **Backend**: Storage/API backends (databases, APIs, etc.)
- **Command**: CLI command extensions
## Quick Start
### Creating a Simple Plugin
```python
from markitect.plugins import BasePlugin, PluginMetadata, PluginType, register_plugin
@register_plugin("my_processor")
class MyProcessor(ProcessorPlugin):
@property
def metadata(self) -> PluginMetadata:
return PluginMetadata(
name="my_processor",
version="1.0.0",
description="My custom processor",
author="Your Name",
plugin_type=PluginType.PROCESSOR
)
def process(self, content: str, **kwargs) -> str:
# Your processing logic here
return content.upper()
```
### Using Plugins via CLI
```bash
# List all available plugins
markitect plugin-list
# Load a specific plugin
markitect plugin-load my_processor
# Get plugin information
markitect plugin-info my_processor
# Discover new plugins
markitect plugin-discover --refresh
```
### Using Plugins Programmatically
```python
from markitect.plugins import PluginManager
# Initialize plugin manager
manager = PluginManager()
# Discover and load plugins
manager.discover_plugins()
processor = manager.load_plugin("my_processor")
# Use the plugin
if processor:
result = processor.process("hello world")
print(result) # HELLO WORLD
```
## Plugin Development Guide
### 1. Choose Base Class
Select the appropriate base class for your plugin:
```python
from markitect.plugins.base import (
ProcessorPlugin, # For content processing
FormatterPlugin, # For output formatting
ValidatorPlugin, # For content validation
ExporterPlugin, # For export functionality
CommandPlugin, # For CLI commands
BasePlugin # For general extensions
)
```
### 2. Implement Required Methods
Each plugin type has specific methods you must implement:
#### ProcessorPlugin
```python
class MyProcessor(ProcessorPlugin):
def process(self, content: str, **kwargs) -> str:
"""Process content and return result."""
return processed_content
def can_process(self, content: str, **kwargs) -> bool:
"""Check if this processor can handle the content."""
return True # or your logic
```
#### FormatterPlugin
```python
class MyFormatter(FormatterPlugin):
def format(self, data: Any, **kwargs) -> str:
"""Format data to string representation."""
return formatted_string
def get_file_extension(self) -> str:
"""Get file extension for this format."""
return '.txt'
```
#### ValidatorPlugin
```python
class MyValidator(ValidatorPlugin):
def validate(self, content: str, **kwargs) -> List[str]:
"""Validate content and return list of errors."""
errors = []
# Your validation logic
return errors
```
### 3. Add Metadata
```python
@property
def metadata(self) -> PluginMetadata:
return PluginMetadata(
name="plugin_name",
version="1.0.0",
description="Plugin description",
author="Your Name",
plugin_type=PluginType.PROCESSOR, # Choose appropriate type
dependencies=["optional", "list", "of", "dependencies"],
markitect_version=">=0.1.0"
)
```
### 4. Register Plugin
Use the decorator for automatic registration:
```python
@register_plugin("plugin_name")
class MyPlugin(BasePlugin):
# Implementation
```
Or register manually:
```python
from markitect.plugins import plugin_registry
plugin_registry.register(MyPlugin, "plugin_name")
```
## Configuration
### Plugin Configuration File
Create a configuration file (`.markitect/plugins.yml`) to manage plugins:
```yaml
plugin_directories:
- "plugins"
- ".markitect/plugins"
- "~/.markitect/plugins"
auto_discover: true
auto_load_enabled: true
plugins:
my_processor:
enabled: true
config:
option1: value1
option2: value2
json_formatter:
enabled: true
config:
indent: 4
```
### Plugin Configuration in Code
```python
# Load plugin with configuration
config = {"option1": "value1", "option2": "value2"}
plugin = manager.load_plugin("my_processor", config)
# Access configuration in plugin
class MyProcessor(ProcessorPlugin):
def process(self, content: str, **kwargs) -> str:
option1 = self.config.get('option1', 'default')
# Use configuration
return processed_content
```
## Built-in Plugins
MarkiTect includes several built-in plugins:
### Formatters
- `json_formatter`: Format output as JSON
- `yaml_formatter`: Format output as YAML
- `table_formatter`: Format output as ASCII tables
### Processors
- `markdown_processor`: Process markdown content
- `text_processor`: Process generic text content
## Plugin Discovery
Plugins are discovered from:
1. **Built-in plugins**: `markitect.plugins.builtin.*`
2. **Local directories**: Configured plugin directories
3. **Installed packages**: Python packages with entry points (future)
### Plugin Directory Structure
```
plugins/
├── my_plugin.py # Single file plugin
├── complex_plugin/ # Multi-file plugin
│ ├── __init__.py
│ ├── main.py
│ └── utils.py
└── another_plugin.py
```
## Advanced Features
### Plugin Dependencies
Specify dependencies in metadata:
```python
PluginMetadata(
# ...
dependencies=["requests", "pyyaml"],
markitect_version=">=0.2.0"
)
```
### Plugin Initialization and Cleanup
```python
class MyPlugin(BasePlugin):
def _initialize(self) -> None:
"""Called when plugin is loaded."""
self.connection = setup_connection()
def cleanup(self) -> None:
"""Called when plugin is unloaded."""
if hasattr(self, 'connection'):
self.connection.close()
```
### Configuration Validation
```python
def validate_config(self) -> List[str]:
"""Validate plugin configuration."""
errors = []
if 'required_option' not in self.config:
errors.append("Missing required_option")
return errors
```
### Command Plugins
Extend the CLI with custom commands:
```python
import click
class MyCommandPlugin(CommandPlugin):
def get_commands(self) -> Dict[str, Any]:
return {
'my-command': self.my_command
}
@click.command()
@click.argument('input_file')
def my_command(self, input_file):
"""My custom command."""
click.echo(f"Processing {input_file}")
```
## Best Practices
### 1. Error Handling
```python
def process(self, content: str, **kwargs) -> str:
try:
return self._do_processing(content)
except Exception as e:
# Log error but don't crash
self.logger.error(f"Processing failed: {e}")
return content # Return original content
```
### 2. Configuration Defaults
```python
def _initialize(self) -> None:
# Set defaults for configuration
defaults = {
'timeout': 30,
'retries': 3,
'format': 'json'
}
for key, value in defaults.items():
if key not in self.config:
self.config[key] = value
```
### 3. Graceful Degradation
```python
def can_process(self, content: str, **kwargs) -> bool:
"""Only claim we can process if we actually can."""
try:
# Test if content is processable
return self._test_content(content)
except Exception:
return False
```
### 4. Documentation
Always provide clear documentation for your plugins:
```python
class MyProcessor(ProcessorPlugin):
"""
Advanced text processor for MarkiTect.
This processor provides advanced text transformation capabilities
including normalization, cleanup, and format conversion.
Configuration:
normalize_whitespace (bool): Normalize whitespace (default: True)
remove_comments (bool): Remove comment lines (default: False)
encoding (str): Text encoding (default: 'utf-8')
Example:
processor = MyProcessor({
'normalize_whitespace': True,
'remove_comments': True
})
result = processor.process(content)
"""
```
## Testing Plugins
Create comprehensive tests for your plugins:
```python
import pytest
from markitect.plugins import plugin_registry
def test_my_processor():
# Load plugin
plugin = plugin_registry.get_plugin("my_processor")
assert plugin is not None
# Test processing
result = plugin.process("test content")
assert result == "expected result"
# Test error handling
result = plugin.process(None)
assert result is not None
```
## Troubleshooting
### Common Issues
1. **Plugin not discovered**: Check plugin directory configuration
2. **Import errors**: Ensure all dependencies are installed
3. **Registration fails**: Check plugin class inheritance
4. **Plugin not loading**: Check `_initialize()` method
### Debug Mode
Enable verbose logging to debug plugin issues:
```bash
markitect --verbose plugin-list
markitect --verbose plugin-load my_plugin
```
### Plugin Validation
```bash
# Validate specific plugin
markitect plugin-info my_plugin
# Discover and validate all plugins
markitect plugin-discover --refresh
```