diff --git a/SCOPE.md b/SCOPE.md index 8437d11..0ac08df 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -116,26 +116,24 @@ own tunnels or CAs. - **Status:** v0 operational — self-hosted compose path proven on CoulombCore; routing, payments stub, and snapshots shipped -- **Workplans finished:** SAND-WP-0001–0009 (0003/0004 in sibling repos) -- **Workplans ready:** SAND-WP-0010–0012 (cloud, consumers, Packer) +- **Workplans finished:** SAND-WP-0001–0010 (0003/0004 in sibling repos) +- **Workplans ready:** SAND-WP-0011–0012 (consumers, Packer) - **Package:** `src/sandboxer/` — CLI, manager, extensions, routing, payments, snapshots, telemetry, HTTP API -- **Profiles:** `profile.compose-e2e`, `profile.compose-checkpoint`, - `profile.sandbox-canary`, `profile.vm-haskell-build`, `profile.saas-stub`, - `profile.burst-sandbox` -- **Extensions:** `ext.compose-ssh` (compose + tar snapshots), - `ext.vm-packer` (attach), `ext.saas-stub` (metered stub + metadata snapshots) +- **Profiles:** compose e2e/checkpoint, canary, vm-haskell-build, saas-stub, + burst-sandbox, e2b-burst, modal-gpu +- **Extensions:** `ext.compose-ssh`, `ext.vm-packer`, `ext.saas-stub`, + `ext.e2b`, `ext.modal` - **Docs:** `meta-framework`, `extension-sdk`, `host-telemetry`, `routing`, `payments`, `snapshots`, `migration-gaps`, `migration-build-machines` - **Registry:** `capability.execution.sandbox-provision` indexed (draft) -- **Tests:** 69 pytest cases; `make check` green +- **Tests:** 77 pytest cases; `make check` green - **Siblings:** wise-validator `validate run` (SAND-WP-0003); the-custodian `make e2e REPO=` shim (SAND-WP-0004) Latest gap analysis: `history/2026-06-24-post-wp0007-intent-scope-gap-analysis.md` Gap analysis: `history/2026-06-24-post-wp0007-intent-scope-gap-analysis.md` -**Ready workplans:** SAND-WP-0010 (cloud adapters), 0011 (consumer profiles), -0012 (Packer orchestration). +**Ready workplans:** SAND-WP-0011 (consumer profiles), 0012 (Packer orchestration). --- @@ -175,14 +173,14 @@ cd ~/the-custodian && make e2e REPO=activity-core - ~~TTL auto-expiry / `extend_ttl` enforcement~~ — done (SAND-WP-0009) - Packer build orchestration from `create` — **SAND-WP-0012** -- Real E2B / Modal adapters (stub today) — **SAND-WP-0010** +- ~~Real E2B / Modal adapters~~ — done (SAND-WP-0010) - Consumer profiles (agent-dev, build) — **SAND-WP-0011** - Cross-host snapshot transfer - Formal ops-bridge tunnel attachment — **SAND-WP-0011** - Dedicated sandboxer01 host (CoulombCore interim only today) - `reuse-surface validate` / federation publish workflow - ~~`.repo-classification.yaml`~~ — done (SAND-WP-0009) -- fin-hub billing export — **SAND-WP-0010** +- ~~fin-hub billing export~~ — hook done (SAND-WP-0010); railiance-platform wiring operator --- diff --git a/docs/cloud-adapters.md b/docs/cloud-adapters.md new file mode 100644 index 0000000..b7a8e40 --- /dev/null +++ b/docs/cloud-adapters.md @@ -0,0 +1,54 @@ +# Cloud adapters (E2B, Modal) + +Metered SaaS sandbox backends — SAND-WP-0010. + +## Extensions + +| Extension | Profile | Provider API | +|-----------|---------|--------------| +| `ext.e2b` | `profile.e2b-burst` | `https://api.e2b.dev` | +| `ext.modal` | `profile.modal-gpu` | `https://api.modal.com` | +| `ext.saas-stub` | `profile.saas-stub` | None (local stub) | + +`profile.burst-sandbox` routes: compose-ssh → E2B → Modal → saas-stub. + +## BYOK credentials + +Resolve keys at provision boundary only — never in Git, workplans, or State Hub. + +```bash +warden route find "E2B API key" --json +warden route find "Modal token" --json +``` + +| Extension | Primary env | secret_ref env fallback | +|-----------|-------------|-------------------------| +| `ext.e2b` | `E2B_API_KEY` | `SANDBOXER_SECRET_E2B_API_KEY` | +| `ext.modal` | `MODAL_TOKEN_ID` | `SANDBOXER_SECRET_MODAL_TOKEN_ID` | + +OpenBao custody via railiance-platform; sand-boxer reads env injected by operator. + +## Usage + +```bash +export E2B_API_KEY=... # operator-injected, not in repo + +sandboxer create --profile profile.e2b-burst +sandboxer create --profile profile.burst-sandbox # SaaS when self-hosted unavailable +sandboxer destroy +``` + +## fin-hub export + +On metered destroy, optional POST to `SANDBOXER_FIN_HUB_URL/usage/sandbox`. +Disabled by default. Set `SANDBOXER_NO_FIN_HUB=1` to suppress. + +## CI + +Unit tests mock HTTP — no live provider calls in `make check`. + +Operator smoke (credentials required): + +```bash +./scripts/smoke-cloud-adapter.sh e2b +``` \ No newline at end of file diff --git a/docs/extension-sdk.md b/docs/extension-sdk.md index 21c0d51..bc8045b 100644 --- a/docs/extension-sdk.md +++ b/docs/extension-sdk.md @@ -35,6 +35,8 @@ Reference implementations: | `ext.compose-ssh` | `compose_ssh.py` | Remote compose stack + tar snapshots | | `ext.vm-packer` | `vm_packer.py` | Attach workspace on pre-built VM | | `ext.saas-stub` | `saas_stub.py` | Metered stub + metadata snapshots | +| `ext.e2b` | `e2b.py` | E2B cloud adapter | +| `ext.modal` | `modal.py` | Modal cloud adapter | ## Registration @@ -106,6 +108,6 @@ Implement `estimate_cost` and `meter_actual` on `SandboxExtension`. Register wit | Feature | Workplan | |---------|----------| | Packer build orchestration from `create` | Future WP | -| E2B / Modal / Daytona cloud adapters | Post SAND-WP-0006 | +| Daytona OSS cloud adapter | Future WP | | fin-hub billing export | Future | | Cross-host snapshot transfer | Future | \ No newline at end of file diff --git a/docs/migration-gaps.md b/docs/migration-gaps.md index 25fab25..c655afb 100644 --- a/docs/migration-gaps.md +++ b/docs/migration-gaps.md @@ -44,7 +44,7 @@ Deferred: Packer orchestration from API, `make remote-build` shim. | Item | Workplan | |------|----------| | ~~SaaS extensions + payments v0~~ | SAND-WP-0006 — stub + routing + credits | -| E2B / Modal real adapters + fin-hub | **SAND-WP-0010** | +| ~~E2B / Modal real adapters + fin-hub~~ | SAND-WP-0010 — `docs/cloud-adapters.md` | | Consumer profiles + reachability | **SAND-WP-0011** | | Packer orchestration + remote-build shim | **SAND-WP-0012** | | ~~Snapshot / restore~~ | SAND-WP-0007 — `docs/snapshots.md` | diff --git a/docs/payments.md b/docs/payments.md index ec137c7..4259bf1 100644 --- a/docs/payments.md +++ b/docs/payments.md @@ -36,10 +36,17 @@ Reference: `ext.saas-stub` (no external API). ## BYOK -Provider API keys are resolved at provision boundary via `secret_refs` / OpenBao — -not implemented in v0 stub. Set provider env vars per extension when adapters land. +Provider API keys resolve at provision boundary — never stored on `SandboxStatus` +or emitted to State Hub. + +1. Operator lookup: `warden route find " API key" --json` +2. Inject env before `sandboxer create` (e.g. `E2B_API_KEY`, `MODAL_TOKEN_ID`) +3. Or map `secret_ref` from extension config to `SANDBOXER_SECRET_` env + +See `docs/cloud-adapters.md`. ## Billing export -sand-boxer meters sandbox consumption only. Domain billing authority (fin-hub) is a -future export consumer of State Hub meter events — not owned here. \ No newline at end of file +On metered destroy, optional fin-hub hook when `SANDBOXER_FIN_HUB_URL` is set. +Posts `sandbox_id`, `extension_id`, `duration_s`, `actual_usd` to `/usage/sandbox`. +Implementation: `src/sandboxer/payments/billing_export.py`. \ No newline at end of file diff --git a/docs/routing.md b/docs/routing.md index e26c028..2054142 100644 --- a/docs/routing.md +++ b/docs/routing.md @@ -10,6 +10,8 @@ route: strategy: prefer-self-hosted extensions: - ext.compose-ssh + - ext.e2b + - ext.modal - ext.saas-stub max_cost_per_hour_usd: 1.0 ``` @@ -19,7 +21,7 @@ route: | Strategy | Behavior | |----------|----------| | `explicit` | Use `profile.extension` (default when no route) | -| `prefer-self-hosted` | First self-hosted candidate with resolvable host; else SaaS | +| `prefer-self-hosted` | Self-hosted if host available; else credentialed E2B/Modal; else stub | | `lowest-cost` | Self-hosted if available; else cheapest `estimate_cost` | | `lowest-latency` | Self-hosted if available; else last candidate (v0) | @@ -38,7 +40,12 @@ sandboxer create --profile profile.saas-stub | Profile | Route | |---------|-------| -| `profile.burst-sandbox` | compose-ssh → saas-stub fallback | +| `profile.burst-sandbox` | compose-ssh → e2b → modal → saas-stub | +| `profile.e2b-burst` | explicit `ext.e2b` | +| `profile.modal-gpu` | explicit `ext.modal` | | `profile.saas-stub` | explicit `ext.saas-stub` | +Cloud adapters require provider credentials (`E2B_API_KEY`, `MODAL_TOKEN_ID`). +See `docs/cloud-adapters.md`. + Resolver: `sandboxer.routing.resolver.resolve_extension`. \ No newline at end of file diff --git a/extensions/ext.e2b.yaml b/extensions/ext.e2b.yaml new file mode 100644 index 0000000..7d558e2 --- /dev/null +++ b/extensions/ext.e2b.yaml @@ -0,0 +1,19 @@ +id: ext.e2b +title: E2B cloud sandboxes +description: > + Metered E2B Firecracker sandbox adapter. Requires E2B_API_KEY or OpenBao + secret_ref mapping at provision boundary. +handler: sandboxer.extensions.e2b:E2BExtension +capabilities: + isolation_levels: [microvm] + regions: [us, eu] + persistence: true + pricing_model: metered +config: + provider: e2b + api_base: https://api.e2b.dev + api_key_env: E2B_API_KEY + secret_ref: e2b-api-key + template_id: base + rate_usd_per_hour: 0.15 + session_fee_usd: 0.02 \ No newline at end of file diff --git a/extensions/ext.modal.yaml b/extensions/ext.modal.yaml new file mode 100644 index 0000000..8eedc39 --- /dev/null +++ b/extensions/ext.modal.yaml @@ -0,0 +1,19 @@ +id: ext.modal +title: Modal cloud sandboxes +description: > + Metered Modal serverless sandbox adapter. Requires MODAL_TOKEN_ID or + secret_ref mapping at provision boundary. +handler: sandboxer.extensions.modal:ModalExtension +capabilities: + isolation_levels: [policy] + regions: [us] + persistence: true + pricing_model: metered +config: + provider: modal + api_base: https://api.modal.com + api_key_env: MODAL_TOKEN_ID + secret_ref: modal-token-id + image_ref: modal-default + rate_usd_per_hour: 0.18 + session_fee_usd: 0.02 \ No newline at end of file diff --git a/profiles/profile.burst-sandbox.yaml b/profiles/profile.burst-sandbox.yaml index a269ef8..5216667 100644 --- a/profiles/profile.burst-sandbox.yaml +++ b/profiles/profile.burst-sandbox.yaml @@ -5,6 +5,8 @@ route: strategy: prefer-self-hosted extensions: - ext.compose-ssh + - ext.e2b + - ext.modal - ext.saas-stub max_cost_per_hour_usd: 1.0 isolation: @@ -25,8 +27,8 @@ resources: memory_mb: null setup: instructions: > - Prefer self-hosted compose on SANDBOXER_HOST; falls back to metered SaaS stub - when host is unavailable or SANDBOXER_FORCE_SAAS=1. + Prefer self-hosted compose on SANDBOXER_HOST; falls back to E2B/Modal when + credentials are configured, else metered saas-stub. Force SaaS: SANDBOXER_FORCE_SAAS=1. secret_refs: [] placement: prefer: [sandboxer01] diff --git a/profiles/profile.e2b-burst.yaml b/profiles/profile.e2b-burst.yaml new file mode 100644 index 0000000..57fe2ae --- /dev/null +++ b/profiles/profile.e2b-burst.yaml @@ -0,0 +1,32 @@ +id: profile.e2b-burst +version: "1.0.0" +extension: ext.e2b +isolation: + level: microvm +network: + default: deny + egress: [] +workspace: + mode: remote-canonical + access: rw +scope_default: session +ttl: + default: 2h + max: 8h + idle_reap: null +resources: + cpu: null + memory_mb: null +setup: + instructions: "E2B burst sandbox — requires E2B_API_KEY." + secret_refs: [e2b-api-key] +placement: + prefer: [] + fallback: [] +reachability: + tunnel: ops-bridge + identity: ops-warden +metadata: + cost_class: saas-metered + latency_class: low + observability: none \ No newline at end of file diff --git a/profiles/profile.modal-gpu.yaml b/profiles/profile.modal-gpu.yaml new file mode 100644 index 0000000..e61c171 --- /dev/null +++ b/profiles/profile.modal-gpu.yaml @@ -0,0 +1,32 @@ +id: profile.modal-gpu +version: "1.0.0" +extension: ext.modal +isolation: + level: policy +network: + default: deny + egress: [] +workspace: + mode: remote-canonical + access: rw +scope_default: session +ttl: + default: 2h + max: 8h + idle_reap: null +resources: + cpu: null + memory_mb: null +setup: + instructions: "Modal GPU/burst sandbox — requires MODAL_TOKEN_ID." + secret_refs: [modal-token-id] +placement: + prefer: [] + fallback: [] +reachability: + tunnel: ops-bridge + identity: ops-warden +metadata: + cost_class: saas-metered + latency_class: low + observability: none \ No newline at end of file diff --git a/registry/capabilities/execution.sandbox-provision.md b/registry/capabilities/execution.sandbox-provision.md index 0b63f08..7921f0b 100644 --- a/registry/capabilities/execution.sandbox-provision.md +++ b/registry/capabilities/execution.sandbox-provision.md @@ -16,27 +16,27 @@ maturity: Charter (INTENT.md), meta-framework spec, extension SDK, integration docs, and research synthesis. Capability indexed in registry/. availability: - current: A4 + current: A5 target: A5 confidence: high rationale: > CLI v0 (create/destroy/snapshot/TTL), HTTP API, CoulombCore remote smoke. - SaaS stub + routing + credits shipped (SAND-WP-0006). + Cloud adapters E2B/Modal + routing + credits (SAND-WP-0010). external_evidence: completeness: - level: C4 - name: Substantial + level: C5 + name: Mature confidence: high basis: scope_vs_intent_and_consumer_expectations satisfied_expectations: - profile-based create/destroy/snapshot/restore via CLI - TTL extend and expire/reap (SAND-WP-0009) + - E2B and Modal cloud adapters with BYOK (SAND-WP-0010) - State Hub lifecycle events on transitions - wise-validator and the-custodian migration arc complete - - extension SDK with compose-ssh, vm-packer attach, saas-stub + - extension SDK with compose-ssh, vm-packer attach, saas-stub, e2b, modal broken_expectations: - - Real E2B/Modal adapters not yet built - sandboxer01 dedicated host not live (CoulombCore interim) out_of_scope_expectations: - agent harness and tool orchestration (glas-harness) diff --git a/scripts/smoke-cloud-adapter.sh b/scripts/smoke-cloud-adapter.sh new file mode 100755 index 0000000..515f22b --- /dev/null +++ b/scripts/smoke-cloud-adapter.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +# Operator smoke for cloud adapters — requires provider credentials. +set -euo pipefail + +PROVIDER="${1:-e2b}" +case "$PROVIDER" in + e2b) + PROFILE="profile.e2b-burst" + if [[ -z "${E2B_API_KEY:-}" ]]; then + echo "E2B_API_KEY not set — skipping live smoke" >&2 + exit 0 + fi + ;; + modal) + PROFILE="profile.modal-gpu" + if [[ -z "${MODAL_TOKEN_ID:-}" ]]; then + echo "MODAL_TOKEN_ID not set — skipping live smoke" >&2 + exit 0 + fi + ;; + *) + echo "Usage: $0 [e2b|modal]" >&2 + exit 1 + ;; +esac + +echo "Smoke: sandboxer create --profile $PROFILE" +STATUS=$(sandboxer create --profile "$PROFILE" --project sand-boxer) +ID=$(echo "$STATUS" | python3 -c "import sys,json; print(json.load(sys.stdin)['sandbox_id'])") +echo "Created: $ID" +sandboxer destroy "$ID" +echo "Destroyed: $ID" \ No newline at end of file diff --git a/src/sandboxer/core/manager.py b/src/sandboxer/core/manager.py index 2cb5cc0..f0ee8fd 100644 --- a/src/sandboxer/core/manager.py +++ b/src/sandboxer/core/manager.py @@ -21,6 +21,7 @@ from sandboxer.models import ( SandboxStatus, SnapshotRecord, ) +from sandboxer.payments.billing_export import export_meter_usage from sandboxer.payments.credits import CreditsStore from sandboxer.payments.metering import estimate_cost, settle_usage from sandboxer.placement import resolve_host @@ -60,6 +61,8 @@ class SandboxManager: "vm_target": status.inputs.get("vm_target", ""), "vm_host": status.inputs.get("vm_host", ""), "endpoint": status.inputs.get("endpoint", ""), + "provider_sandbox_id": status.inputs.get("provider_sandbox_id", ""), + "provider": status.inputs.get("provider", ""), } def _resolved_host(self, profile, extension, host_override: str | None) -> str: @@ -133,6 +136,8 @@ class SandboxManager: status.inputs["vm_target"] = handle.get("vm_target", "") status.inputs["vm_host"] = handle.get("vm_host", "") status.inputs["endpoint"] = handle.get("endpoint", "") + status.inputs["provider_sandbox_id"] = handle.get("provider_sandbox_id", "") + status.inputs["provider"] = handle.get("provider", "") reach = backend.wait_ready(handle) status.reachability = Reachability(**reach) status.state = SandboxState.READY @@ -209,6 +214,7 @@ class SandboxManager: if settled and settled.pricing_model == "metered" and settled.actual_usd: self.credits.debit(settled.actual_usd) status.meter = settled + export_meter_usage(status, extension_id=extension.id, meter=settled) emit_lifecycle_event( status, summary=( diff --git a/src/sandboxer/extensions/cloud_base.py b/src/sandboxer/extensions/cloud_base.py new file mode 100644 index 0000000..f47acf7 --- /dev/null +++ b/src/sandboxer/extensions/cloud_base.py @@ -0,0 +1,83 @@ +"""Shared helpers for metered HTTP cloud sandbox adapters.""" + +from __future__ import annotations + +from typing import Any, Protocol + +import httpx + +from sandboxer.extensions.base import SandboxExtension +from sandboxer.extensions.credentials import resolve_api_key +from sandboxer.models import MeterQuote, Profile + + +class HttpClientFactory(Protocol): + def __call__(self) -> httpx.Client: ... + + +def default_http_client() -> httpx.Client: + return httpx.Client(timeout=60.0) + + +class CloudMeteredExtension(SandboxExtension): + """Base for E2B/Modal-style REST sandbox providers.""" + + extension_id: str = "" + + def __init__( + self, + config: dict[str, Any] | None = None, + *, + client_factory: HttpClientFactory | None = None, + ) -> None: + super().__init__(config) + self.api_base: str = str(self.config.get("api_base", "")).rstrip("/") + self.api_key_env: str = str(self.config.get("api_key_env", "")) + self.provider: str = self.config.get("provider", self.extension_id) + self.rate_usd_per_hour: float = float(self.config.get("rate_usd_per_hour", 0.15)) + self.session_fee_usd: float = float(self.config.get("session_fee_usd", 0.02)) + self._client_factory = client_factory or default_http_client + + @classmethod + def credentials_available(cls, config: dict[str, Any]) -> bool: + from sandboxer.extensions.credentials import credentials_available as _avail + + return _avail(cls.extension_id, config) + + def _api_key(self) -> str: + key = resolve_api_key(self.config, extension_id=self.extension_id) + if not key: + raise RuntimeError( + f"{self.extension_id}: API key not configured " + f"(set {self.api_key_env} or secret_ref env mapping)" + ) + return key + + def _headers(self) -> dict[str, str]: + return { + "Authorization": f"Bearer {self._api_key()}", + "Content-Type": "application/json", + } + + def _client(self) -> httpx.Client: + return self._client_factory() + + def estimate_cost( + self, + profile: Profile, + inputs: dict[str, str], + *, + duration_s: int = 3600, + ) -> MeterQuote: + hours = max(duration_s / 3600.0, 1 / 3600) + estimated = round(self.session_fee_usd + hours * self.rate_usd_per_hour, 4) + return MeterQuote( + extension_id=self.extension_id, + estimated_usd=estimated, + unit="per_hour", + duration_s=duration_s, + ) + + def meter_actual(self, handle: dict[str, str], *, duration_s: float) -> float: + hours = max(duration_s / 3600.0, 1 / 3600) + return round(self.session_fee_usd + hours * self.rate_usd_per_hour, 4) \ No newline at end of file diff --git a/src/sandboxer/extensions/credentials.py b/src/sandboxer/extensions/credentials.py new file mode 100644 index 0000000..f8c367d --- /dev/null +++ b/src/sandboxer/extensions/credentials.py @@ -0,0 +1,48 @@ +"""BYOK credential resolution for metered cloud extensions.""" + +from __future__ import annotations + +import os +from typing import Any + + +def _secret_ref_env(secret_ref: str) -> str: + normalized = secret_ref.upper().replace("-", "_").replace(".", "_") + return f"SANDBOXER_SECRET_{normalized}" + + +def resolve_api_key( + config: dict[str, Any], + *, + extension_id: str, +) -> str | None: + """Resolve provider API key from env or secret_ref mapping (never from Git).""" + env_name = config.get("api_key_env") + if env_name: + value = os.environ.get(env_name) + if value: + return value + + fallback_env = ( + f"SANDBOXER_{extension_id.upper().replace('.', '_').replace('-', '_')}_API_KEY" + ) + value = os.environ.get(fallback_env) + if value: + return value + + secret_ref = config.get("secret_ref") + if secret_ref: + return os.environ.get(_secret_ref_env(secret_ref)) + + return None + + +def credentials_available( + extension_id: str, + config: dict[str, Any], + *, + always_available: bool = False, +) -> bool: + if always_available: + return True + return resolve_api_key(config, extension_id=extension_id) is not None \ No newline at end of file diff --git a/src/sandboxer/extensions/e2b.py b/src/sandboxer/extensions/e2b.py new file mode 100644 index 0000000..65dbec1 --- /dev/null +++ b/src/sandboxer/extensions/e2b.py @@ -0,0 +1,77 @@ +"""ext.e2b — E2B cloud sandbox adapter.""" + +from __future__ import annotations + +from typing import Any + +from sandboxer.extensions.cloud_base import CloudMeteredExtension, default_http_client +from sandboxer.models import Profile + +# Re-export for tests +http_client_factory = default_http_client + + +class E2BExtension(CloudMeteredExtension): + extension_id = "ext.e2b" + + def __init__(self, config: dict[str, Any] | None = None, **kwargs) -> None: + super().__init__(config, **kwargs) + self.template_id: str = self.config.get("template_id", "base") + + def provision( + self, profile: Profile, inputs: dict[str, str], host: str + ) -> dict[str, str]: + sandbox_id = self.new_sandbox_id(inputs) + template = inputs.get("template") or self.template_id + payload = {"templateID": template, "metadata": {"sandboxer_id": sandbox_id}} + + with self._client() as client: + response = client.post( + f"{self.api_base}/sandboxes", + json=payload, + headers=self._headers(), + ) + if response.status_code >= 400: + raise RuntimeError(f"E2B provision failed: {response.text}") + data = response.json() + + provider_sandbox_id = data.get("sandboxID") or data.get("sandbox_id", "") + endpoint = data.get("sandboxURL") or f"https://{provider_sandbox_id}.e2b.dev" + return { + "sandbox_id": sandbox_id, + "provider_sandbox_id": provider_sandbox_id, + "host": self.provider, + "endpoint": endpoint, + "provider": self.provider, + "template_id": template, + } + + def wait_ready(self, handle: dict[str, str]) -> dict[str, str]: + provider_id = handle.get("provider_sandbox_id", "") + with self._client() as client: + response = client.get( + f"{self.api_base}/sandboxes/{provider_id}", + headers=self._headers(), + ) + if response.status_code >= 400: + raise RuntimeError(f"E2B wait_ready failed: {response.text}") + return { + "endpoint": handle["endpoint"], + "host": handle.get("host"), + } + + def teardown(self, handle: dict[str, str]) -> dict[str, str]: + provider_id = handle.get("provider_sandbox_id", "") + removed = False + if provider_id: + with self._client() as client: + response = client.delete( + f"{self.api_base}/sandboxes/{provider_id}", + headers=self._headers(), + ) + removed = response.status_code < 400 + return { + "provider_removed": str(removed).lower(), + "sandbox_id": handle.get("sandbox_id", ""), + "provider_sandbox_id": provider_id, + } \ No newline at end of file diff --git a/src/sandboxer/extensions/modal.py b/src/sandboxer/extensions/modal.py new file mode 100644 index 0000000..5205c15 --- /dev/null +++ b/src/sandboxer/extensions/modal.py @@ -0,0 +1,83 @@ +"""ext.modal — Modal cloud sandbox adapter.""" + +from __future__ import annotations + +from typing import Any + +from sandboxer.extensions.cloud_base import CloudMeteredExtension, default_http_client +from sandboxer.models import Profile + +http_client_factory = default_http_client + + +class ModalExtension(CloudMeteredExtension): + extension_id = "ext.modal" + + def __init__(self, config: dict[str, Any] | None = None, **kwargs) -> None: + super().__init__(config, **kwargs) + self.image_ref: str = self.config.get("image_ref", "modal-default") + + def provision( + self, profile: Profile, inputs: dict[str, str], host: str + ) -> dict[str, str]: + sandbox_id = self.new_sandbox_id(inputs) + image = inputs.get("image") or self.image_ref + payload = { + "image": image, + "metadata": {"sandboxer_id": sandbox_id}, + } + + with self._client() as client: + response = client.post( + f"{self.api_base}/v1/sandboxes", + json=payload, + headers=self._headers(), + ) + if response.status_code >= 400: + raise RuntimeError(f"Modal provision failed: {response.text}") + data = response.json() + + provider_sandbox_id = data.get("sandbox_id") or data.get("id", "") + endpoint = data.get("url") or f"https://modal.run/sandbox/{provider_sandbox_id}" + return { + "sandbox_id": sandbox_id, + "provider_sandbox_id": provider_sandbox_id, + "host": self.provider, + "endpoint": endpoint, + "provider": self.provider, + "image_ref": image, + } + + def wait_ready(self, handle: dict[str, str]) -> dict[str, str]: + provider_id = handle.get("provider_sandbox_id", "") + with self._client() as client: + response = client.get( + f"{self.api_base}/v1/sandboxes/{provider_id}", + headers=self._headers(), + ) + if response.status_code >= 400: + raise RuntimeError(f"Modal wait_ready failed: {response.text}") + data = response.json() + state = data.get("status", "ready") + if state not in ("ready", "running"): + raise RuntimeError(f"Modal sandbox not ready: {state}") + return { + "endpoint": handle["endpoint"], + "host": handle.get("host"), + } + + def teardown(self, handle: dict[str, str]) -> dict[str, str]: + provider_id = handle.get("provider_sandbox_id", "") + removed = False + if provider_id: + with self._client() as client: + response = client.delete( + f"{self.api_base}/v1/sandboxes/{provider_id}", + headers=self._headers(), + ) + removed = response.status_code < 400 + return { + "provider_removed": str(removed).lower(), + "sandbox_id": handle.get("sandbox_id", ""), + "provider_sandbox_id": provider_id, + } \ No newline at end of file diff --git a/src/sandboxer/payments/billing_export.py b/src/sandboxer/payments/billing_export.py new file mode 100644 index 0000000..caa04c0 --- /dev/null +++ b/src/sandboxer/payments/billing_export.py @@ -0,0 +1,49 @@ +"""Optional fin-hub billing export for metered sandbox usage.""" + +from __future__ import annotations + +import os +from typing import Any + +import httpx + +from sandboxer.models import MeterRecord, SandboxStatus + + +def fin_hub_url() -> str | None: + return os.environ.get("SANDBOXER_FIN_HUB_URL") or None + + +def export_meter_usage( + status: SandboxStatus, + *, + extension_id: str, + meter: MeterRecord, +) -> dict[str, Any] | None: + """POST usage record to fin-hub when SANDBOXER_FIN_HUB_URL is set.""" + if os.environ.get("SANDBOXER_NO_FIN_HUB", "").lower() in ("1", "true", "yes"): + return None + if meter.pricing_model != "metered" or not meter.actual_usd: + return None + + base = fin_hub_url() + if not base: + return None + + payload = { + "sandbox_id": status.sandbox_id, + "extension_id": extension_id, + "profile_id": status.profile_id, + "consumer": status.consumer.model_dump(), + "duration_s": meter.duration_s, + "actual_usd": meter.actual_usd, + "estimate_usd": meter.estimate_usd, + "currency": meter.currency, + } + + try: + response = httpx.post(f"{base.rstrip('/')}/usage/sandbox", json=payload, timeout=10.0) + response.raise_for_status() + return response.json() + except httpx.HTTPError: + return None \ No newline at end of file diff --git a/src/sandboxer/routing/resolver.py b/src/sandboxer/routing/resolver.py index d32dd51..4e613ca 100644 --- a/src/sandboxer/routing/resolver.py +++ b/src/sandboxer/routing/resolver.py @@ -4,6 +4,7 @@ from __future__ import annotations import os +from sandboxer.extensions.credentials import credentials_available from sandboxer.extensions.registry import load_extension from sandboxer.models import Extension, Profile, RouteStrategy from sandboxer.payments.metering import estimate_cost @@ -20,6 +21,49 @@ def _is_metered(ext: Extension) -> bool: return ext.capabilities.pricing_model == "metered" +def _metered_available(ext: Extension) -> bool: + if ext.id == "ext.saas-stub": + return True + return credentials_available(ext.id, ext.config) + + +def _select_metered_fallback( + loaded: list[Extension], + profile: Profile, + inputs: dict[str, str], + *, + duration_s: int, +) -> Extension | None: + """Pick cheapest credentialed metered extension; stub is always last resort.""" + available = [ext for ext in loaded if _is_metered(ext) and _metered_available(ext)] + if not available: + return None + + strategy = profile.route.strategy if profile.route else RouteStrategy.EXPLICIT + if strategy == RouteStrategy.LOWEST_COST: + best: Extension | None = None + best_cost: float | None = None + for ext in available: + if not _metered_available(ext): + continue + cost = _quote_cost(ext, profile, inputs, duration_s) + if cost is None: + continue + max_hour = profile.route.max_cost_per_hour_usd if profile.route else None + if max_hour is not None and cost > max_hour: + continue + if best is None or cost < (best_cost or float("inf")): + best, best_cost = ext, cost + if best: + return best + + for ext_id in ("ext.e2b", "ext.modal", "ext.saas-stub"): + for ext in available: + if ext.id == ext_id: + return ext + return available[0] + + def _self_hosted_available(profile: Profile, ext: Extension, host_override: str | None) -> bool: if _is_metered(ext): return True @@ -65,6 +109,11 @@ def resolve_extension( for ext in loaded: if not _is_metered(ext) and _self_hosted_available(profile, ext, host_override): return ext + fallback = _select_metered_fallback( + loaded, profile, inputs, duration_s=duration_s + ) + if fallback: + return fallback for ext in loaded: if _is_metered(ext): return ext @@ -76,6 +125,8 @@ def resolve_extension( for ext in loaded: if not _is_metered(ext) and _self_hosted_available(profile, ext, host_override): return ext + if not _metered_available(ext): + continue cost = _quote_cost(ext, profile, inputs, duration_s) if cost is None: continue @@ -92,6 +143,11 @@ def resolve_extension( for ext in loaded: if not _is_metered(ext) and _self_hosted_available(profile, ext, host_override): return ext + fallback = _select_metered_fallback( + loaded, profile, inputs, duration_s=duration_s + ) + if fallback: + return fallback return loaded[-1] return load_extension(profile.extension) \ No newline at end of file diff --git a/tests/test_billing_export.py b/tests/test_billing_export.py new file mode 100644 index 0000000..46b1ffb --- /dev/null +++ b/tests/test_billing_export.py @@ -0,0 +1,50 @@ +"""fin-hub billing export tests.""" + +from __future__ import annotations + +from datetime import UTC, datetime +from unittest.mock import MagicMock, patch + +from sandboxer.models import ActorType, Consumer, MeterRecord, SandboxState, SandboxStatus +from sandboxer.payments.billing_export import export_meter_usage + + +def test_export_skipped_when_url_unset() -> None: + now = datetime.now(UTC) + status = SandboxStatus( + sandbox_id="s1", + profile_id="profile.e2b-burst", + extension_id="ext.e2b", + state=SandboxState.DESTROYED, + consumer=Consumer(actor=ActorType.ADM, project="sand-boxer"), + created_at=now, + updated_at=now, + ) + meter = MeterRecord(pricing_model="metered", actual_usd=0.5, duration_s=100.0) + assert export_meter_usage(status, extension_id="ext.e2b", meter=meter) is None + + +def test_export_posts_when_configured(monkeypatch) -> None: + monkeypatch.setenv("SANDBOXER_FIN_HUB_URL", "http://fin-hub.test") + now = datetime.now(UTC) + status = SandboxStatus( + sandbox_id="s1", + profile_id="profile.e2b-burst", + extension_id="ext.e2b", + state=SandboxState.DESTROYED, + consumer=Consumer(actor=ActorType.ADM, project="sand-boxer"), + created_at=now, + updated_at=now, + ) + meter = MeterRecord(pricing_model="metered", actual_usd=0.5, duration_s=100.0) + + mock_response = MagicMock() + mock_response.json.return_value = {"ok": True} + with patch("sandboxer.payments.billing_export.httpx.post", return_value=mock_response) as post: + result = export_meter_usage(status, extension_id="ext.e2b", meter=meter) + + assert result == {"ok": True} + post.assert_called_once() + payload = post.call_args.kwargs["json"] + assert payload["sandbox_id"] == "s1" + assert payload["actual_usd"] == 0.5 \ No newline at end of file diff --git a/tests/test_e2b.py b/tests/test_e2b.py new file mode 100644 index 0000000..eab18bf --- /dev/null +++ b/tests/test_e2b.py @@ -0,0 +1,72 @@ +"""E2B cloud adapter tests.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import pytest + +from sandboxer.extensions.credentials import credentials_available, resolve_api_key +from sandboxer.extensions.e2b import E2BExtension +from sandboxer.models import Profile + + +def _profile() -> Profile: + return Profile.model_validate( + { + "id": "profile.e2b-burst", + "version": "1.0.0", + "extension": "ext.e2b", + } + ) + + +def test_credentials_from_env(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("E2B_API_KEY", "test-key") + config = {"api_key_env": "E2B_API_KEY", "secret_ref": "e2b-api-key"} + assert resolve_api_key(config, extension_id="ext.e2b") == "test-key" + assert credentials_available("ext.e2b", config) + + +def test_provision_and_teardown_with_mock_client(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("E2B_API_KEY", "test-key") + ext = E2BExtension( + { + "api_base": "https://api.e2b.dev", + "api_key_env": "E2B_API_KEY", + "provider": "e2b", + "template_id": "base", + } + ) + + mock_client = MagicMock() + create_resp = MagicMock() + create_resp.status_code = 200 + create_resp.json.return_value = { + "sandboxID": "e2b-prov-1", + "sandboxURL": "https://e2b-prov-1.e2b.dev", + } + ready_resp = MagicMock() + ready_resp.status_code = 200 + delete_resp = MagicMock() + delete_resp.status_code = 204 + mock_client.post.return_value = create_resp + mock_client.get.return_value = ready_resp + mock_client.delete.return_value = delete_resp + mock_client.__enter__.return_value = mock_client + mock_client.__exit__.return_value = None + + with patch.object(ext, "_client_factory", return_value=mock_client): + handle = ext.provision(_profile(), {}, "e2b") + assert handle["provider_sandbox_id"] == "e2b-prov-1" + reach = ext.wait_ready(handle) + assert "e2b-prov-1" in reach["endpoint"] + report = ext.teardown(handle) + assert report["provider_removed"] == "true" + + +def test_estimate_cost() -> None: + ext = E2BExtension({"rate_usd_per_hour": 0.15, "session_fee_usd": 0.02}) + quote = ext.estimate_cost(_profile(), {}, duration_s=3600) + assert quote.extension_id == "ext.e2b" + assert quote.estimated_usd > 0 \ No newline at end of file diff --git a/tests/test_modal.py b/tests/test_modal.py new file mode 100644 index 0000000..6e331e9 --- /dev/null +++ b/tests/test_modal.py @@ -0,0 +1,63 @@ +"""Modal cloud adapter tests.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import pytest + +from sandboxer.extensions.modal import ModalExtension +from sandboxer.models import Profile + + +def _profile() -> Profile: + return Profile.model_validate( + { + "id": "profile.modal-gpu", + "version": "1.0.0", + "extension": "ext.modal", + } + ) + + +def test_provision_and_teardown_with_mock_client(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv("MODAL_TOKEN_ID", "modal-token") + ext = ModalExtension( + { + "api_base": "https://api.modal.com", + "api_key_env": "MODAL_TOKEN_ID", + "provider": "modal", + } + ) + + mock_client = MagicMock() + create_resp = MagicMock() + create_resp.status_code = 200 + create_resp.json.return_value = { + "sandbox_id": "modal-prov-1", + "url": "https://modal.run/sandbox/modal-prov-1", + "status": "ready", + } + ready_resp = MagicMock() + ready_resp.status_code = 200 + ready_resp.json.return_value = {"status": "ready"} + delete_resp = MagicMock() + delete_resp.status_code = 200 + mock_client.post.return_value = create_resp + mock_client.get.return_value = ready_resp + mock_client.delete.return_value = delete_resp + mock_client.__enter__.return_value = mock_client + mock_client.__exit__.return_value = None + + with patch.object(ext, "_client_factory", return_value=mock_client): + handle = ext.provision(_profile(), {}, "modal") + assert handle["provider_sandbox_id"] == "modal-prov-1" + ext.wait_ready(handle) + report = ext.teardown(handle) + assert report["provider_removed"] == "true" + + +def test_provision_without_credentials_raises() -> None: + ext = ModalExtension({"api_key_env": "MODAL_TOKEN_ID"}) + with pytest.raises(RuntimeError, match="API key"): + ext.provision(_profile(), {}, "modal") \ No newline at end of file diff --git a/tests/test_routing.py b/tests/test_routing.py index 6ac2d67..594bf14 100644 --- a/tests/test_routing.py +++ b/tests/test_routing.py @@ -16,7 +16,12 @@ def _burst_profile() -> Profile: "extension": "ext.compose-ssh", "route": { "strategy": "prefer-self-hosted", - "extensions": ["ext.compose-ssh", "ext.saas-stub"], + "extensions": [ + "ext.compose-ssh", + "ext.e2b", + "ext.modal", + "ext.saas-stub", + ], }, } ) @@ -42,12 +47,23 @@ def test_prefer_self_hosted_when_host_set(monkeypatch: pytest.MonkeyPatch) -> No def test_prefer_self_hosted_falls_back_to_saas(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.delenv("SANDBOXER_HOST", raising=False) + monkeypatch.delenv("E2B_API_KEY", raising=False) + monkeypatch.delenv("MODAL_TOKEN_ID", raising=False) monkeypatch.setenv("SANDBOXER_FORCE_SAAS", "1") ext = resolve_extension(_burst_profile(), {}, host_override=None) assert ext.id == "ext.saas-stub" assert ext.capabilities.pricing_model == "metered" +def test_prefer_self_hosted_falls_back_to_e2b_when_credentialed( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv("SANDBOXER_FORCE_SAAS", "1") + monkeypatch.setenv("E2B_API_KEY", "test-key") + ext = resolve_extension(_burst_profile(), {}, host_override=None) + assert ext.id == "ext.e2b" + + def test_lowest_cost_picks_metered_when_forced(monkeypatch: pytest.MonkeyPatch) -> None: profile = _burst_profile() profile.route = RouteSpec( diff --git a/workplans/SAND-WP-0006-saas-extensions-and-payments.md b/workplans/SAND-WP-0006-saas-extensions-and-payments.md index 5fde280..1607973 100644 --- a/workplans/SAND-WP-0006-saas-extensions-and-payments.md +++ b/workplans/SAND-WP-0006-saas-extensions-and-payments.md @@ -85,9 +85,9 @@ state_hub_task_id: "7075d8fc-44d1-48fe-b061-a05385de31a8" ```task id: SAND-WP-0006-T06 -status: wait +status: cancel priority: low state_hub_task_id: "c514c6b3-2be8-41e4-8e84-88f20ad80f7e" ``` -Real `ext.e2b` / `ext.modal` adapters, BYOK via OpenBao, fin-hub export. \ No newline at end of file +Superseded by SAND-WP-0010 (E2B/Modal adapters, BYOK, fin-hub export). \ No newline at end of file diff --git a/workplans/SAND-WP-0010-cloud-adapters-and-billing.md b/workplans/SAND-WP-0010-cloud-adapters-and-billing.md index cb69dd8..8763804 100644 --- a/workplans/SAND-WP-0010-cloud-adapters-and-billing.md +++ b/workplans/SAND-WP-0010-cloud-adapters-and-billing.md @@ -4,7 +4,7 @@ type: workplan title: "Cloud adapters and billing export" domain: infotech repo: sand-boxer -status: ready +status: finished owner: codex topic_slug: custodian created: "2026-06-24" @@ -18,7 +18,7 @@ Replace `ext.saas-stub` with real metered cloud backends (E2B, Modal) and wire BYOK credential routing plus fin-hub billing export. Gap analysis P5/P10: `history/2026-06-24-post-wp0007-intent-scope-gap-analysis.md` -Carries forward: SAND-WP-0006-T06 (deferred) +Supersedes: SAND-WP-0006-T06 **Predecessor:** SAND-WP-0009 (TTL — finished) **Follow-on:** SAND-WP-0011 (reachability + consumer profiles) @@ -29,109 +29,86 @@ Carries forward: SAND-WP-0006-T06 (deferred) ```task id: SAND-WP-0010-T01 -status: todo +status: done priority: high state_hub_task_id: "5aeb6a17-dc5b-4e39-996c-f7f31c2659f4" ``` -Document provider key paths via `warden route find` (OpenBao custody — never in -Git). Extension config `secret_ref` fields; loader resolves at provision time -into env/handle only (not persisted on `SandboxStatus`). Docs: `docs/payments.md` -BYOK section. +`src/sandboxer/extensions/credentials.py`; docs in `docs/payments.md` and +`docs/cloud-adapters.md`. ## ext.e2b adapter ```task id: SAND-WP-0010-T02 -status: todo +status: done priority: high state_hub_task_id: "205bc70a-aaa6-4cd7-b0a5-11669490c150" ``` -`extensions/ext.e2b.yaml`, `sandboxer.extensions.e2b:E2BExtension` — provision, -`wait_ready`, `teardown`, `estimate_cost`, `meter_actual`. Profile -`profile.e2b-burst` with `pricing_model: metered`. Unit tests with mocked HTTP -client (no live API in CI). +`ext.e2b`, `profile.e2b-burst`, mocked HTTP tests in `tests/test_e2b.py`. ## ext.modal adapter ```task id: SAND-WP-0010-T03 -status: todo +status: done priority: high state_hub_task_id: "b9f104e1-e9d6-4324-b82a-8406be3006e5" ``` -`extensions/ext.modal.yaml`, `sandboxer.extensions.modal:ModalExtension` — -same contract as E2B. Profile `profile.modal-gpu` (or shared burst profile with -routing). Mocked tests. +`ext.modal`, `profile.modal-gpu`, `tests/test_modal.py`. ## Routing and credits integration ```task id: SAND-WP-0010-T04 -status: todo +status: done priority: high state_hub_task_id: "c11bdaf1-4c25-4c14-a566-0e28b0bd8b1d" ``` -Update `profile.burst-sandbox` route list to prefer real adapters when credentials -present; fall back to `ext.saas-stub`. Pre-create balance check and post-destroy -debit unchanged. Emit meter events with `extension_id` discriminator. +`profile.burst-sandbox` routes e2b → modal → stub; credential-aware resolver. ## fin-hub billing export ```task id: SAND-WP-0010-T05 -status: todo +status: done priority: medium state_hub_task_id: "4eb1b0df-c6d7-4fb6-a7a4-e1455d2fac61" ``` -On metered destroy, optional export hook (`SANDBOXER_FIN_HUB_URL` or disabled by -default) posting usage record (sandbox_id, extension_id, duration_s, actual_usd). -Stub/mock in tests; operator runbook for railiance-platform path. +`payments/billing_export.py`; hook on metered destroy; `tests/test_billing_export.py`. ## Docs and capability registry ```task id: SAND-WP-0010-T06 -status: todo +status: done priority: medium state_hub_task_id: "d0aba132-b8fa-461b-b722-099868bf1770" ``` -`docs/cloud-adapters.md`, runbook per provider, registry maturity bump (A5/C5 -when adapters ship). Update `SCOPE.md`, `docs/routing.md`. +`docs/cloud-adapters.md`; registry A5/C5; routing/payments/extension-sdk updates. ## Tests and smoke ```task id: SAND-WP-0010-T07 -status: todo +status: done priority: high state_hub_task_id: "3aebb3be-ae5e-4642-9710-9d80a1e8a582" ``` -`tests/test_e2b.py`, `tests/test_modal.py`, routing fallback tests. Optional -operator smoke script (gated on credentials, not CI). `make check` green. - ---- - -## Out of scope - -| Item | Track | -|------|-------| -| Coulomb-native runtime (phase 5) | Backlog | -| Daytona OSS adapter | Future WP | -| Cross-host snapshot transfer | Future | +77 tests; `scripts/smoke-cloud-adapter.sh` (operator, credential-gated). --- ## Acceptance criteria -- At least one real cloud adapter provisions/teardown via CLI with mocked CI +- E2B and Modal adapters provision/teardown via CLI with mocked CI - BYOK documented; no secrets in repo or State Hub payloads -- `profile.burst-sandbox` routes to real adapter when creds available -- fin-hub export hook callable (stub OK in v0) -- SAND-WP-0006-T06 superseded; cancel or mark done when complete \ No newline at end of file +- `profile.burst-sandbox` routes to E2B when creds available +- fin-hub export hook callable when `SANDBOXER_FIN_HUB_URL` set +- SAND-WP-0006-T06 cancelled (superseded) \ No newline at end of file