tegwick
9bc761c2b5
Packer build definition, cloud-init autoinstall, GHCup toolchain script, boot-time registration agent (state-hub + autossh dual tunnel), systemd unit, key injection, remote-build Makefile, smoke test, and deployment README. All 15 tasks complete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
146 lines
3.9 KiB
Markdown
146 lines
3.9 KiB
Markdown
# Build Machines
|
|
|
|
Reproducible VirtualBox images for offloading compilation to dedicated hardware.
|
|
Each VM self-registers with the Custodian State Hub on boot and connects back to
|
|
the development workstation via SSH reverse tunnel.
|
|
|
|
## Prerequisites
|
|
|
|
- **Packer** >= 1.10 (`packer version`)
|
|
- **VirtualBox** >= 7.0 (`VBoxManage --version`)
|
|
- **autossh** on both workstation and VM (installed automatically in VM image)
|
|
- **State Hub** running on workstation (`cd ~/the-custodian/state-hub && make api`)
|
|
|
|
## Quick Start
|
|
|
|
### 1. Generate SSH keypair (one-time)
|
|
|
|
```bash
|
|
ssh-keygen -t ed25519 -f ~/.ssh/id_build -N "" -C "build-agent"
|
|
```
|
|
|
|
### 2. Build the OVA
|
|
|
|
```bash
|
|
cd infra/build-machines/haskell
|
|
packer init .
|
|
packer build .
|
|
```
|
|
|
|
This produces `haskell-build-YYYYMMDD.ova` (~4-6 GB, depending on GHC versions).
|
|
|
|
### 3. Import and configure
|
|
|
|
```bash
|
|
# Import the OVA
|
|
VBoxManage import haskell-build-20260420.ova
|
|
|
|
# Switch from NAT (build-time) to bridged networking
|
|
scripts/setup-vm.sh haskell-build
|
|
|
|
# Start the VM
|
|
VBoxManage startvm haskell-build --type headless
|
|
```
|
|
|
|
### 4. Inject credentials
|
|
|
|
```bash
|
|
# Prepare a directory with keys and config
|
|
mkdir -p ~/vm-keys/haskell-build
|
|
cp ~/.ssh/id_build ~/vm-keys/haskell-build/
|
|
cp ~/.ssh/id_build.pub ~/vm-keys/haskell-build/
|
|
|
|
# Edit build-agent.env from template
|
|
cp haskell/files/build-agent.env.template ~/vm-keys/haskell-build/build-agent.env
|
|
# Edit SSH_RELAY_HOST to your workstation's LAN IP
|
|
|
|
# Inject (VM must be running; uses temporary password auth)
|
|
scripts/inject-keys.sh <vm-ip> ~/vm-keys/haskell-build/
|
|
```
|
|
|
|
### 5. Install SSH config
|
|
|
|
```bash
|
|
make install-ssh-config
|
|
```
|
|
|
|
### 6. Verify
|
|
|
|
```bash
|
|
make bridge-status # check tunnel is up
|
|
ssh haskell-build # should connect via tunnel
|
|
./smoke-test.sh # full stack validation
|
|
```
|
|
|
|
## Using the VM
|
|
|
|
```bash
|
|
# Build a Haskell project remotely
|
|
make remote-build PROJECT=~/projects/my-app
|
|
|
|
# Run tests
|
|
make remote-test PROJECT=~/projects/my-app
|
|
|
|
# Interactive GHCi
|
|
make remote-ghci PROJECT=~/projects/my-app
|
|
|
|
# Fetch build artefacts back to workstation
|
|
make fetch-artifacts PROJECT=~/projects/my-app
|
|
|
|
# Check VM info
|
|
make vm-info
|
|
```
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Workstation (WSL2)
|
|
├── state-hub (:8000) — sees capability entries, knows tunnel ports
|
|
└── SSH listener — accepts reverse tunnel from VM
|
|
|
|
Laptop (VirtualBox host)
|
|
└── haskell-build VM (Ubuntu 24.04, bridged)
|
|
├── GHC 9.8.4 + 9.6.6 via GHCup
|
|
├── build-agent (systemd) — registers with state-hub on boot
|
|
└── autossh: -R 12222→local:22, -L 18000→state-hub:8000
|
|
```
|
|
|
|
The VM connects OUT to the workstation. Two tunnels in one SSH connection:
|
|
- **Reverse** (`-R 12222:localhost:22`): workstation can SSH into VM
|
|
- **Forward** (`-L 18000:localhost:8000`): VM can reach state-hub
|
|
|
|
## Port Registry
|
|
|
|
See `port-registry.yml`. Range 12221-12230 supports up to 10 concurrent VMs.
|
|
Each VM must use a unique port.
|
|
|
|
## Adding a GHC Version Post-Deployment
|
|
|
|
```bash
|
|
ssh haskell-build "source ~/.ghcup/env && ghcup install ghc 9.10.1"
|
|
```
|
|
|
|
No image rebuild required.
|
|
|
|
## Troubleshooting
|
|
|
|
**Tunnel not up:**
|
|
- Check `journalctl -u build-agent` on the VM
|
|
- Verify `SSH_RELAY_HOST` in `/etc/build-agent.env` is reachable from the VM
|
|
- Ensure the workstation's SSH server accepts the build key
|
|
|
|
**Capability not in state-hub:**
|
|
- Check `curl http://127.0.0.1:8000/capability-catalog/?capability_type=haskell-build-agent`
|
|
- The agent retries 20 times on boot; check logs for registration errors
|
|
- The forward tunnel (`-L 18000:localhost:8000`) must be up before registration works
|
|
|
|
**Build fails with missing libraries:**
|
|
- The VM includes common Haskell build deps. For additional system libraries:
|
|
`ssh haskell-build "sudo apt-get install -y libXXX-dev"`
|
|
|
|
## Updating the Image
|
|
|
|
Re-run Packer to build a new OVA. Import alongside the existing VM or replace it.
|
|
Build artefacts and keys live on the workstation (via rsync), not in the VM — the
|
|
image is disposable.
|