Add complete Gitea package publishing support

 Features:
- GiteaPackageRegistry client for PyPI-compatible uploads
- Enhanced release.py with upload/registry commands
- New Makefile targets for Gitea publishing workflow
- Comprehensive documentation with examples

📦 New Commands:
- `release.py registry` - Show registry info & authentication
- `release.py upload` - Upload packages to Gitea
- `release.py publish --to-gitea` - Complete release + upload
- `make release-publish-gitea VERSION=x.y.z` - One-command release

🔧 Infrastructure:
- Automatic package detection (wheel + sdist)
- Dry-run support for safe testing
- Error handling and detailed feedback
- Authentication validation

📚 Documentation:
- PACKAGE_PUBLISHING.md with complete setup guide
- Usage examples and troubleshooting

🚀 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2025-11-08 21:06:03 +01:00
parent ab67997324
commit d8d823b101
5 changed files with 575 additions and 4 deletions

View File

@@ -41,6 +41,9 @@ help:
@echo " release-build - Build release packages (version auto-detected)" @echo " release-build - Build release packages (version auto-detected)"
@echo " release-tag VERSION=x.y.z - Create release git tag" @echo " release-tag VERSION=x.y.z - Create release git tag"
@echo " release-publish VERSION=x.y.z - Complete release workflow (tag + build)" @echo " release-publish VERSION=x.y.z - Complete release workflow (tag + build)"
@echo " release-publish-gitea VERSION=x.y.z - Release + upload to Gitea registry"
@echo " release-upload-gitea - Upload existing packages to Gitea registry"
@echo " release-registry - Show Gitea package registry information"
@echo " release-dry-run VERSION=x.y.z - Test release workflow" @echo " release-dry-run VERSION=x.y.z - Test release workflow"
@echo "" @echo ""
@echo "Chaos Engineering:" @echo "Chaos Engineering:"
@@ -522,6 +525,23 @@ release-dry-run:
fi fi
$(VENV_PYTHON) release.py publish --version $(VERSION) --dry-run $(VENV_PYTHON) release.py publish --version $(VERSION) --dry-run
release-publish-gitea:
@echo "🚀 Publishing complete release with Gitea upload..."
@if [ -z "$(VERSION)" ]; then \
echo "❌ Usage: make release-publish-gitea VERSION=1.0.0"; \
echo " This creates git tag + builds packages + uploads to Gitea"; \
exit 1; \
fi
$(VENV_PYTHON) release.py publish --version $(VERSION) --to-gitea
release-upload-gitea:
@echo "📡 Uploading packages to Gitea registry..."
$(VENV_PYTHON) release.py upload
release-registry:
@echo "📦 Gitea package registry information..."
$(VENV_PYTHON) release.py registry
# Chaos Engineering targets # Chaos Engineering targets
chaos-validate: chaos-validate:
@echo "🔥 Running architectural independence validation..." @echo "🔥 Running architectural independence validation..."

159
PACKAGE_PUBLISHING.md Normal file
View File

@@ -0,0 +1,159 @@
# Package Publishing to Gitea
This document explains how to publish MarkiTect packages to the Gitea package registry.
## Prerequisites
1. **Gitea API Token**: Set the `GITEA_API_TOKEN` environment variable with your Gitea API token
2. **Repository Access**: The token must have write access to the repository's package registry
## Quick Setup
```bash
# Set your Gitea API token
export GITEA_API_TOKEN="your_gitea_api_token_here"
# Or add it to your shell profile
echo "export GITEA_API_TOKEN=your_token" >> ~/.bashrc
```
## Usage
### Check Registry Status
```bash
# Check registry configuration and authentication
make release-registry
# or
python release.py registry
```
### Build and Upload Packages
```bash
# Complete release workflow with Gitea upload
make release-publish-gitea VERSION=0.8.0
# or
python release.py publish --version 0.8.0 --to-gitea
# Upload existing packages
make release-upload-gitea
# or
python release.py upload
# Dry run (test without uploading)
python release.py upload --dry-run
```
### Traditional Release (without Gitea)
```bash
# Standard release without Gitea upload
make release-publish VERSION=0.8.0
python release.py publish --version 0.8.0
```
## Available Commands
### Makefile Targets
- `make release-registry` - Show Gitea package registry information
- `make release-upload-gitea` - Upload existing packages to Gitea
- `make release-publish-gitea VERSION=x.y.z` - Complete release + Gitea upload
### Python Script Commands
- `python release.py registry` - Show registry information
- `python release.py upload` - Upload packages to Gitea
- `python release.py upload --dry-run` - Test upload without uploading
- `python release.py publish --version x.y.z --to-gitea` - Release with Gitea upload
## Registry Information
- **Gitea URL**: http://92.205.130.254:32166
- **Repository**: coulomb/markitect_project
- **PyPI Registry URL**: http://92.205.130.254:32166/api/packages/coulomb/pypi
- **Package List URL**: http://92.205.130.254:32166/api/v1/packages/coulomb
## Installing from Gitea Registry
Once packages are published, users can install them using:
```bash
# Install from Gitea registry
pip install markitect --extra-index-url http://92.205.130.254:32166/api/packages/coulomb/pypi/simple/
# Or configure pip permanently
mkdir -p ~/.pip
cat >> ~/.pip/pip.conf << EOF
[global]
extra-index-url = http://92.205.130.254:32166/api/packages/coulomb/pypi/simple/
EOF
```
## Features
### Automatic Package Detection
The system automatically detects and uploads:
- **Wheel files** (`.whl`) - Binary distributions
- **Source distributions** (`.tar.gz`) - Source code packages
### Version Management with setuptools-scm
Versions are automatically determined by git tags:
- `v0.8.0` tag → `0.8.0` package version
- Development commits → `0.8.1.dev3+gcommithash` versions
### Error Handling
The system provides detailed error messages for:
- Missing authentication tokens
- Network connectivity issues
- Package upload failures
- Invalid package formats
## Troubleshooting
### Authentication Issues
```bash
# Check if token is set
echo $GITEA_API_TOKEN
# Test authentication
python release.py registry
```
### Upload Failures
```bash
# Test with dry run first
python release.py upload --dry-run
# Check package files exist
ls -la dist/
# Rebuild packages if needed
make release-build
```
### Network Issues
- Ensure Gitea server is accessible: `ping 92.205.130.254`
- Check firewall and proxy settings
- Verify Gitea is running on port 32166
## Development
The package registry functionality is implemented in:
- `gitea/package_registry.py` - Main package registry client
- `release.py` - Release script with Gitea integration
- `Makefile` - Convenient targets for package management
## Security Notes
- Never commit API tokens to version control
- Use environment variables or secure credential storage
- Tokens should have minimal required permissions
- Rotate tokens regularly for security

View File

@@ -24,10 +24,12 @@ from .client import GiteaClient
from .models import Issue, Milestone, Label, ProjectState, Priority from .models import Issue, Milestone, Label, ProjectState, Priority
from .config import GiteaConfig from .config import GiteaConfig
from .exceptions import GiteaError, GiteaAuthError, GiteaNotFoundError from .exceptions import GiteaError, GiteaAuthError, GiteaNotFoundError
from .package_registry import GiteaPackageRegistry
__all__ = [ __all__ = [
'GiteaClient', 'GiteaClient',
'Issue', 'Milestone', 'Label', 'ProjectState', 'Priority', 'Issue', 'Milestone', 'Label', 'ProjectState', 'Priority',
'GiteaConfig', 'GiteaConfig',
'GiteaError', 'GiteaAuthError', 'GiteaNotFoundError' 'GiteaError', 'GiteaAuthError', 'GiteaNotFoundError',
'GiteaPackageRegistry'
] ]

271
gitea/package_registry.py Normal file
View File

@@ -0,0 +1,271 @@
"""
Gitea Package Registry Client
This module provides functionality to publish Python packages to Gitea's package registry.
Gitea supports multiple package registries including PyPI-compatible registries.
"""
import os
import subprocess
import tempfile
from pathlib import Path
from typing import Optional, List, Dict
from .config import GiteaConfig
from .exceptions import GiteaError
class GiteaPackageRegistry:
"""Client for publishing packages to Gitea package registry."""
def __init__(self, config: Optional[GiteaConfig] = None):
"""Initialize the package registry client.
Args:
config: Gitea configuration. If None, auto-detects from git repository.
"""
self.config = config or GiteaConfig.from_git_repository()
self.config.validate()
@property
def pypi_registry_url(self) -> str:
"""Get the PyPI-compatible registry URL for this repository."""
return f"{self.config.gitea_url}/api/packages/{self.config.repo_owner}/pypi"
@property
def package_list_url(self) -> str:
"""Get the package listing URL for this repository."""
return f"{self.config.gitea_url}/api/v1/packages/{self.config.repo_owner}"
def check_auth(self) -> bool:
"""Check if authentication token is available and valid."""
if not self.config.auth_token:
return False
try:
# Test auth by trying to access packages API
import requests
headers = {"Authorization": f"token {self.config.auth_token}"}
response = requests.get(self.package_list_url, headers=headers, timeout=10)
return response.status_code in [200, 404] # 404 is okay if no packages exist yet
except Exception:
return False
def list_packages(self) -> List[Dict]:
"""List all packages for this repository owner.
Returns:
List of package information dictionaries
"""
try:
import requests
headers = {}
if self.config.auth_token:
headers["Authorization"] = f"token {self.config.auth_token}"
response = requests.get(self.package_list_url, headers=headers, timeout=10)
response.raise_for_status()
return response.json()
except Exception as e:
raise GiteaError(f"Failed to list packages: {e}")
def get_package_info(self, package_name: str) -> Optional[Dict]:
"""Get information about a specific package.
Args:
package_name: Name of the package
Returns:
Package information dictionary or None if not found
"""
try:
packages = self.list_packages()
for package in packages:
if package.get("name") == package_name:
return package
return None
except Exception:
return None
def upload_package(self,
wheel_path: Path,
sdist_path: Optional[Path] = None,
dry_run: bool = False) -> bool:
"""Upload package files to Gitea registry.
Args:
wheel_path: Path to wheel (.whl) file
sdist_path: Optional path to source distribution (.tar.gz) file
dry_run: If True, show what would be done without uploading
Returns:
True if upload successful, False otherwise
"""
if not self.config.auth_token:
raise GiteaError("Authentication token required for package upload. Set GITEA_API_TOKEN environment variable.")
if not wheel_path.exists():
raise GiteaError(f"Wheel file not found: {wheel_path}")
if sdist_path and not sdist_path.exists():
raise GiteaError(f"Source distribution file not found: {sdist_path}")
files_to_upload = [wheel_path]
if sdist_path:
files_to_upload.append(sdist_path)
if dry_run:
print(f"[DRY RUN] Would upload to: {self.pypi_registry_url}")
for file_path in files_to_upload:
print(f"[DRY RUN] Would upload: {file_path}")
return True
success = True
for file_path in files_to_upload:
if not self._upload_file(file_path):
success = False
return success
def _upload_file(self, file_path: Path) -> bool:
"""Upload a single file to the registry.
Args:
file_path: Path to file to upload
Returns:
True if upload successful, False otherwise
"""
try:
import requests
# Upload using multipart form data (PyPI-compatible)
with open(file_path, 'rb') as f:
files = {
'content': (file_path.name, f, 'application/octet-stream')
}
headers = {
'Authorization': f'token {self.config.auth_token}'
}
upload_url = f"{self.pypi_registry_url}/simple/"
response = requests.post(
upload_url,
headers=headers,
files=files,
timeout=60
)
if response.status_code in [200, 201, 409]: # 409 = already exists
print(f"✅ Uploaded: {file_path.name}")
if response.status_code == 409:
print(f" (already exists)")
return True
else:
print(f"❌ Upload failed for {file_path.name}: {response.status_code} {response.text}")
return False
except Exception as e:
print(f"❌ Upload failed for {file_path.name}: {e}")
return False
def delete_package_version(self, package_name: str, version: str, dry_run: bool = False) -> bool:
"""Delete a specific version of a package.
Args:
package_name: Name of the package
version: Version to delete
dry_run: If True, show what would be done without deleting
Returns:
True if deletion successful, False otherwise
"""
if not self.config.auth_token:
raise GiteaError("Authentication token required for package deletion.")
delete_url = f"{self.config.gitea_url}/api/v1/packages/{self.config.repo_owner}/pypi/{package_name}/{version}"
if dry_run:
print(f"[DRY RUN] Would delete: {package_name} v{version}")
print(f"[DRY RUN] DELETE {delete_url}")
return True
try:
import requests
headers = {"Authorization": f"token {self.config.auth_token}"}
response = requests.delete(delete_url, headers=headers, timeout=10)
if response.status_code in [200, 204, 404]: # 404 = already deleted
print(f"✅ Deleted: {package_name} v{version}")
return True
else:
print(f"❌ Delete failed: {response.status_code} {response.text}")
return False
except Exception as e:
print(f"❌ Delete failed: {e}")
return False
def get_registry_info(self) -> Dict:
"""Get information about the package registry configuration.
Returns:
Dictionary with registry information
"""
return {
"gitea_url": self.config.gitea_url,
"repo_owner": self.config.repo_owner,
"repo_name": self.config.repo_name,
"pypi_registry_url": self.pypi_registry_url,
"package_list_url": self.package_list_url,
"auth_configured": bool(self.config.auth_token),
"auth_valid": self.check_auth() if self.config.auth_token else False
}
def configure_pip_for_gitea(config: Optional[GiteaConfig] = None,
pip_conf_path: Optional[Path] = None) -> Path:
"""Configure pip to use Gitea package registry as additional index.
Args:
config: Gitea configuration
pip_conf_path: Custom path for pip.conf file
Returns:
Path to created/updated pip.conf file
"""
config = config or GiteaConfig.from_git_repository()
if pip_conf_path is None:
# Default pip config location
pip_conf_path = Path.home() / ".pip" / "pip.conf"
pip_conf_path.parent.mkdir(parents=True, exist_ok=True)
registry = GiteaPackageRegistry(config)
gitea_index = f"{registry.pypi_registry_url}/simple/"
# Read existing config or create new
config_content = ""
if pip_conf_path.exists():
config_content = pip_conf_path.read_text()
# Add Gitea index if not already present
if "extra-index-url" not in config_content:
if "[global]" not in config_content:
config_content = "[global]\n" + config_content
lines = config_content.split('\n')
global_section_idx = next(i for i, line in enumerate(lines) if line.strip() == "[global]")
lines.insert(global_section_idx + 1, f"extra-index-url = {gitea_index}")
config_content = '\n'.join(lines)
elif gitea_index not in config_content:
# Add to existing extra-index-url
lines = config_content.split('\n')
for i, line in enumerate(lines):
if line.startswith("extra-index-url"):
lines[i] = f"{line} {gitea_index}"
break
config_content = '\n'.join(lines)
pip_conf_path.write_text(config_content)
return pip_conf_path

View File

@@ -6,7 +6,7 @@ This simplified script works with setuptools-scm for automatic version managemen
Versions are automatically derived from git tags - no manual version bumping needed. Versions are automatically derived from git tags - no manual version bumping needed.
Usage: Usage:
python release_simplified.py [command] [options] python release.py [command] [options]
Commands: Commands:
status Show current release status status Show current release status
@@ -14,11 +14,14 @@ Commands:
tag Create git tag for version (e.g., v0.8.0) tag Create git tag for version (e.g., v0.8.0)
build Build release packages build Build release packages
publish Complete release workflow (tag + build + distribute) publish Complete release workflow (tag + build + distribute)
upload Upload packages to Gitea registry
registry Show Gitea package registry information
Options: Options:
--version VERSION Git tag version (e.g., 0.8.0, 1.0.0-rc1) --version VERSION Git tag version (e.g., 0.8.0, 1.0.0-rc1)
--dry-run Show what would be done without making changes --dry-run Show what would be done without making changes
--force Force operation even with warnings --force Force operation even with warnings
--to-gitea Upload to Gitea package registry
""" """
import subprocess import subprocess
@@ -28,6 +31,12 @@ from pathlib import Path
from datetime import datetime from datetime import datetime
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
try:
from gitea.package_registry import GiteaPackageRegistry
GITEA_AVAILABLE = True
except ImportError:
GITEA_AVAILABLE = False
class SimpleReleaseManager: class SimpleReleaseManager:
"""Simplified release manager using setuptools-scm.""" """Simplified release manager using setuptools-scm."""
@@ -203,6 +212,105 @@ class SimpleReleaseManager:
print(f"🏷️ Git tag v{version} created") print(f"🏷️ Git tag v{version} created")
return True return True
def upload_to_gitea(self, dry_run: bool = False) -> bool:
"""Upload packages to Gitea package registry."""
if not GITEA_AVAILABLE:
print("❌ Gitea package registry not available (missing gitea module)")
return False
try:
registry = GiteaPackageRegistry()
print(f"📡 Uploading to Gitea registry: {registry.pypi_registry_url}")
# Find built packages
dist_dir = self.project_root / "dist"
if not dist_dir.exists():
print("❌ No dist/ directory found. Run 'build' command first.")
return False
wheel_files = list(dist_dir.glob("*.whl"))
sdist_files = list(dist_dir.glob("*.tar.gz"))
if not wheel_files and not sdist_files:
print("❌ No package files found in dist/")
return False
# Upload each package
success = True
for wheel_file in wheel_files:
# Find matching sdist
sdist_file = None
for sdist in sdist_files:
if wheel_file.stem.split('-')[0] == sdist.stem.split('-')[0]:
sdist_file = sdist
break
if not registry.upload_package(wheel_file, sdist_file, dry_run=dry_run):
success = False
# Upload any remaining sdists
uploaded_sdists = []
for wheel_file in wheel_files:
for sdist in sdist_files:
if wheel_file.stem.split('-')[0] == sdist.stem.split('-')[0]:
uploaded_sdists.append(sdist)
for sdist_file in sdist_files:
if sdist_file not in uploaded_sdists:
if not registry.upload_package(sdist_file, dry_run=dry_run):
success = False
return success
except Exception as e:
print(f"❌ Upload to Gitea failed: {e}")
return False
def show_gitea_registry_info(self):
"""Show Gitea package registry information."""
if not GITEA_AVAILABLE:
print("❌ Gitea package registry not available (missing gitea module)")
return
try:
registry = GiteaPackageRegistry()
info = registry.get_registry_info()
print("📦 Gitea Package Registry Information")
print("=" * 50)
print(f"Gitea URL: {info['gitea_url']}")
print(f"Repository: {info['repo_owner']}/{info['repo_name']}")
print(f"PyPI Registry URL: {info['pypi_registry_url']}")
print(f"Package List URL: {info['package_list_url']}")
print(f"Authentication Configured: {'' if info['auth_configured'] else ''}")
print(f"Authentication Valid: {'' if info['auth_valid'] else '' if info['auth_configured'] else 'N/A'}")
if info['auth_configured']:
try:
packages = registry.list_packages()
print(f"\nExisting Packages: {len(packages)}")
for package in packages[:5]: # Show first 5
print(f" - {package.get('name', 'unknown')} (type: {package.get('type', 'unknown')})")
except Exception as e:
print(f"\nError listing packages: {e}")
else:
print("\n Set GITEA_API_TOKEN environment variable for package management")
except Exception as e:
print(f"❌ Error getting registry info: {e}")
def publish_with_gitea(self, version: str, dry_run: bool = False) -> bool:
"""Complete release workflow including Gitea upload."""
if not self.publish_release(version):
return False
if not self.upload_to_gitea(dry_run=dry_run):
print("⚠️ Release completed but Gitea upload failed")
return False
print("🎉 Complete release with Gitea upload successful!")
return True
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
@@ -211,11 +319,12 @@ def main():
epilog=__doc__.split('\n\n')[1] epilog=__doc__.split('\n\n')[1]
) )
parser.add_argument('command', choices=['status', 'validate', 'tag', 'build', 'publish'], parser.add_argument('command', choices=['status', 'validate', 'tag', 'build', 'publish', 'upload', 'registry'],
help='Release command to execute') help='Release command to execute')
parser.add_argument('--version', type=str, help='Target version for git tag (e.g., 0.8.0)') parser.add_argument('--version', type=str, help='Target version for git tag (e.g., 0.8.0)')
parser.add_argument('--dry-run', action='store_true', help='Show what would be done') parser.add_argument('--dry-run', action='store_true', help='Show what would be done')
parser.add_argument('--force', action='store_true', help='Force operation despite warnings') parser.add_argument('--force', action='store_true', help='Force operation despite warnings')
parser.add_argument('--to-gitea', action='store_true', help='Include Gitea package registry upload')
args = parser.parse_args() args = parser.parse_args()
manager = SimpleReleaseManager(dry_run=args.dry_run, force=args.force) manager = SimpleReleaseManager(dry_run=args.dry_run, force=args.force)
@@ -247,7 +356,17 @@ def main():
if not args.version: if not args.version:
print("❌ --version is required for publish command") print("❌ --version is required for publish command")
sys.exit(1) sys.exit(1)
manager.publish_release(args.version)
if args.to_gitea:
manager.publish_with_gitea(args.version, args.dry_run)
else:
manager.publish_release(args.version)
elif args.command == 'upload':
manager.upload_to_gitea(args.dry_run)
elif args.command == 'registry':
manager.show_gitea_registry_info()
except Exception as e: except Exception as e:
print(f"❌ Error: {e}") print(f"❌ Error: {e}")