generated from coulomb/repo-seed
Implement SAND-WP-0003: validation meta-framework extraction
Port e2e-framework schema, runner, and reporter into wise-validator with sand-boxer CLI integration, validate run CLI, unit tests, registry capability, and operator docs.
This commit is contained in:
45
docs/integrations/sand-boxer.md
Normal file
45
docs/integrations/sand-boxer.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# sand-boxer integration
|
||||
|
||||
wise-validator **consumes** sand-boxer for isolated execution environments.
|
||||
sand-boxer is self-sustained and does not depend on wise-validator.
|
||||
|
||||
## Provision
|
||||
|
||||
```bash
|
||||
# Invoked internally by `validate run`; equivalent manual call:
|
||||
sandboxer create \
|
||||
--profile profile.compose-e2e \
|
||||
--input repo=/path/to/repo \
|
||||
--actor atm \
|
||||
--project wise-validator \
|
||||
--host "${SANDBOXER_HOST:-coulombcore}"
|
||||
```
|
||||
|
||||
Environment variables:
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `SANDBOXER_HOST` | Default sandbox host |
|
||||
| `SANDBOXER_SSH_USER` | SSH user when not in reachability |
|
||||
| `SANDBOXER_SSH_KEY` | SSH private key path |
|
||||
| `SANDBOXER_COMPOSE_CMD` | `podman-compose` on CoulombCore |
|
||||
| `SANDBOXER_BIN` | Override `sandboxer` binary path |
|
||||
|
||||
## wise-validator steps after `ready`
|
||||
|
||||
1. Poll `health_checks` from repo `e2e/e2e.yml` via SSH `curl` on remote host
|
||||
2. Run `test_command` in `reachability.remote_dir`
|
||||
3. `destroy` per `cleanup` policy (`always` / `on_success` / `never`, or `--keep`)
|
||||
|
||||
## Ownership
|
||||
|
||||
| Concern | Owner |
|
||||
|---------|-------|
|
||||
| rsync + compose up + reachability | sand-boxer |
|
||||
| `e2e/e2e.yml` parsing | wise-validator |
|
||||
| HTTP health polling | wise-validator |
|
||||
| Test command + pass/fail | wise-validator |
|
||||
| State Hub `e2e_result` events | wise-validator |
|
||||
| Sandbox lifecycle events | sand-boxer |
|
||||
|
||||
Canon in sand-boxer: `sand-boxer/docs/integrations/wise-validator.md`
|
||||
87
docs/runbooks/validate-compose-e2e.md
Normal file
87
docs/runbooks/validate-compose-e2e.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# validate compose-e2e — Runbook
|
||||
|
||||
Cross-repo e2e validation using wise-validator + sand-boxer.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
**Workstation:**
|
||||
|
||||
- `sandboxer` on PATH (`cd ~/sand-boxer && make install`)
|
||||
- `validate` on PATH (`cd ~/wise-validator && make install`)
|
||||
- `ssh` available (BatchMode; respects `~/.ssh/config`)
|
||||
- State Hub on `:8000` for result reporting (optional)
|
||||
|
||||
**Sandbox host (CoulombCore / sandboxer01):**
|
||||
|
||||
- SSH access
|
||||
- `podman-compose` or `docker compose`
|
||||
- Sufficient disk for images
|
||||
|
||||
## First run
|
||||
|
||||
```bash
|
||||
export SANDBOXER_HOST=92.205.130.254 # or coulombcore alias
|
||||
export SANDBOXER_COMPOSE_CMD=podman-compose
|
||||
|
||||
validate run ~/activity-core
|
||||
```
|
||||
|
||||
Output: sandbox create → health wait → test → destroy. Exit 0 = pass, 1 = fail.
|
||||
|
||||
## Options
|
||||
|
||||
```bash
|
||||
# Keep sandbox for debugging
|
||||
validate run ~/activity-core --keep
|
||||
|
||||
# Attach State Hub workstream
|
||||
validate run ~/activity-core --workstream-id <uuid>
|
||||
|
||||
# Skip State Hub reporting
|
||||
validate run ~/activity-core --no-report
|
||||
|
||||
# Override host
|
||||
validate run ~/activity-core --host 92.205.130.254
|
||||
```
|
||||
|
||||
## Adding a repo contract
|
||||
|
||||
Create `<repo>/e2e/e2e.yml`:
|
||||
|
||||
```yaml
|
||||
name: my-repo
|
||||
compose_file: docker-compose.dev.yml
|
||||
health_checks:
|
||||
- name: api
|
||||
url: http://localhost:8080
|
||||
timeout: 120
|
||||
test_command: uv run python -m pytest e2e/tests/ -v
|
||||
timeout: 300
|
||||
cleanup: always
|
||||
```
|
||||
|
||||
Run: `validate run ~/my-repo`
|
||||
|
||||
## Self-smoke (sand-boxer repo)
|
||||
|
||||
```bash
|
||||
validate run ~/sand-boxer
|
||||
```
|
||||
|
||||
Uses `e2e/docker-compose.smoke.yml` (nginx on `127.0.0.1:18080`).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**`sandboxer not found`:** Install sand-boxer CLI.
|
||||
|
||||
**Health check timeout:** SSH to host and `curl` the URL from inside the sandbox dir.
|
||||
|
||||
**Stale sandboxes:** `sandboxer inspect stale` / `sandboxer reap-stale --apply`
|
||||
|
||||
## Migration from the-custodian
|
||||
|
||||
| Legacy | Replacement |
|
||||
|--------|-------------|
|
||||
| `make e2e REPO=` | `validate run ~/REPO` (SAND-WP-0004 shim pending) |
|
||||
| `python -m e2e_framework` | `validate run` |
|
||||
| Embedded SSH provision | `sandboxer create` (sand-boxer) |
|
||||
Reference in New Issue
Block a user