# Haskell Vibe Primer
## Hard-won lessons for coding agents working on inter-hub and IHP projects

---

## Quick orientation

Inter-hub is an IHP (Integrated Haskell Platform) v1.5 application on GHC 9.10.3, managed via Nix/devenv. It has two distinct compilation modes with fundamentally different behaviour:

| Mode | Command | Compiler | Output | Crashes? |
|------|---------|----------|--------|----------|
| Dev | `devenv up` → ghcid | GHCi byte-code | In-memory | No (byte-code avoids static linker) |
| Production | `nix build .#docker` | Native (cabal) | OCI image | Yes, if archive bug present |

A build that works in `devenv up` does **not** guarantee `nix build .#docker` will succeed. The production build hits code paths that ghcid never reaches.

The build host for production is **haskelseed** (192.168.178.135, root/hcs26!x), not CoulombCore. Do not try to run `nix build` on CoulombCore.

---

## Part 1 — Known GHC 9.10.3 bugs in this project

### Bug 1: `module M` re-export causes `.hi` overflow (FIXED in flake.nix)

**Symptom:**
```
Data.Binary.Get.runGet at position ~287000000: not enough bytes
```
Crash occurs while GHC reads a `.hi` (interface) file.

**Cause:** IHP generates a hub module `Generated/ActualTypes.hs` that originally used:
```haskell
module Generated.ActualTypes
  ( module Generated.Foo
  , module Generated.Bar
  ...  -- 61 sub-modules
  ) where
import Generated.Foo
import Generated.Bar
```

The `module M` re-export syntax causes GHC to **embed the full exported interface of every sub-module verbatim** into `ActualTypes.hi`. With 61 sub-modules and thousands of typeclass instances, the resulting `.hi` reaches ~287 MB — exceeding GHC 9.10.3's `Data.Binary.Get` 274 MB deserialization limit.

**Fix (active in `flake.nix`):** The `inter-hub-models` postUnpack overlay rewrites the export list to explicit `T(..)` per-type exports:
```haskell
module Generated.ActualTypes
  ( Foo(..)
  , FooId(..)
  , Bar(..)
  , ...
  ) where
```
Explicit re-exports store only compact name-references in `.hi` (not embedded sub-interfaces). `ActualTypes.hi` drops from ~287 MB to ~51 KB.

**Rule: NEVER use `module M` re-export syntax for generated hub modules with many sub-modules.** Always use explicit `T(..)` exports.

```haskell
-- BAD: embeds sub-interfaces into hub .hi
module MyHub (module Sub1, module Sub2) where

-- GOOD: stores compact references
module MyHub (Foo(..), Bar, baz) where
```

---

### Bug 2: Truncated `libHSghc-9.10.3-5702.a` in Nix store (FIXED on haskelseed)

**Symptom:**
```
panic! (the 'impossible' happened)
  GHC version 9.10.3:
  Data.Binary.Get.runGet at position 287686318: not enough bytes
```
Crash occurs **after** `[477 of 477] Compiling Generated.WidgetVersionInclude` — all modules compiled, crash in post-compilation step.

**Cause:** The Nix-provisioned static archive for the GHC compiler-as-library is truncated:
- Truncated: `ffg3yf2.../lib/.../ghc-9.10.3-5702/libHSghc-9.10.3-5702.a` — 287,768,576 bytes
- Full archive: `ffg3yf2.../ghc-9.10.3-partial/lib/.../ghc-9.10.3-5702/libHSghc-9.10.3-5702.a` — 289,295,782 bytes

The last AR entry (`Expr.o`) claims 517,544 bytes but only 82,258 are present. GHC's `readAr` (via `Data.Binary.Get.runGet`) panics when it tries to read the full entry.

Why GHC reads this archive at all: The global `ghc-with-packages` package db is searched during the build (cabal configure does not pass `--package-db=clear`). That db registers the `ghc` compiler-as-library package with `library-dirs` pointing to the directory containing the truncated archive symlink. GHC loads this archive during internal post-compilation processing — even when the package has **no Template Haskell**.

**What does NOT fix it:**
- `-fomit-interface-pragmas` — reduces `.hi` size, unrelated to archive loading
- `--disable-shared` — affects output type, not input archive reading
- `-fexternal-interpreter -pgmi ghc-iserv-dyn` — routes TH to external process, but inter-hub-models has **zero TH**; the archive read is from a different code path

**Fix applied on haskelseed (2026-05-02):**
```bash
TRUNCATED="/nix/store/ffg3yf2ypnbz3hc31y7nglrkihz0if01-ghc-9.10.3/lib/ghc-9.10.3/lib/x86_64-linux-ghc-9.10.3/ghc-9.10.3-5702/libHSghc-9.10.3-5702.a"
FULL="/nix/store/ffg3yf2ypnbz3hc31y7nglrkihz0if01-ghc-9.10.3/ghc-9.10.3-partial/lib/ghc-9.10.3/lib/x86_64-linux-ghc-9.10.3/ghc-9.10.3-5702/libHSghc-9.10.3-5702.a"
chmod u+w "$(dirname "$TRUNCATED")"
cp "$FULL" "$TRUNCATED"
chmod a-w "$TRUNCATED" "$(dirname "$TRUNCATED")"
```

**If the fix is lost** (flake lock update changing GHC version):
```bash
# Check archive size before building — should be ~289 MB
wc -c /nix/store/HASH-ghc-9.10.3/lib/ghc-9.10.3/lib/x86_64-linux-ghc-9.10.3/ghc-9.10.3-5702/libHSghc-9.10.3-5702.a
# If ~287 MB, look for ghc-9.10.3-partial/ in same store path and apply patch
```

---

### Bug 3: `--disable-shared` causes missing `.dyn_hi` files (FIXED in flake.nix)

If `inter-hub-models` is built with `--disable-shared`, it produces only `.hi` files but no `.dyn_hi` (dynamic interface files). The dependent `inter-hub-lib` package (built with `--enable-shared` by default in NixPkgs) requires `.dyn_hi` from its dependencies and fails with:
```
Failed to load dynamic interface file for Generated.Foo:
  .../Generated/Foo.dyn_hi: does not exist
```

Do not add `--disable-shared` to the inter-hub-models configureFlags unless you also suppress `--enable-shared` in inter-hub-lib (and its transitive dependents). The archive fix (Bug 2) is the correct solution; `--disable-shared` was a failed hypothesis.

---

## Part 2 — IHP compilation architecture

### Compilation layers (from CLAUDE.md)

```
Layer 1 — stable core (compile once, expensive to change):
  build/Generated/Types.hs      IHP auto-generated from Schema.sql
  Web/Types.hs                  controller/action type definitions

Layer 2 — helpers (only touch if Layer 1 is clean):
  Application/Helper/*.hs       12 helper modules

Layer 3 — working surface (most errors live here):
  Web/Controller/*.hs           59 controllers
  Web/View/**/*.hs              120 views

Layer 4 — wiring (fix last):
  Web/FrontController.hs
  Web/Routes.hs
```

**Fix errors bottom-up, one module at a time.** Never touch Layer 1 during an error-fix loop. A single change to `Web/Types.hs` invalidates all 59 controllers and 120 views simultaneously — equivalent to a full rebuild.

### Package split

The project compiles as two separate Cabal packages:

| Package | Source | Contains |
|---------|--------|----------|
| `inter-hub-models` | `inter-hub-models-src/build/Generated/` | All IHP-generated types, ~477 modules |
| `inter-hub-lib` | `inter-hub-lib-src/` | Application code: `Application/`, `Config/`, `Web/` |

Critical: **`inter-hub-lib-src` has no `build/` directory.** Generated modules come from the `inter-hub-models` package, not from the lib source tree. Any script that assumes lib-src contains `build/Generated/` will fail. To access generated file content from within inter-hub-lib's build scripts, find inter-hub-models-src dynamically:
```bash
_models_src=$(find /nix/store -maxdepth 1 -name "*-inter-hub-models-src" ! -name "*.drv" | head -1)
```

### Generated code — what changes when schema changes

When `Application/Schema.sql` changes and you regenerate via the IHP IDE:
- `build/Generated/Types.hs` — one-liner re-export hub (regenerated, do not edit)
- `build/Generated/ActualTypes.hs` — type hub (regenerated, patched by postUnpack overlay)
- `build/Generated/Foo.hs` — one per table, data type + instances
- `build/Generated/FooInclude.hs` — `type instance Include` for eager loading
- `build/Generated/ActualTypes/Foo.hs` — sub-module used by ActualTypes hub

After any schema change, check whether the postUnpack in `flake.nix` still correctly rewrites `Generated.ActualTypes` — run `nix build --keep-failed` and inspect the rewritten file.

---

## Part 3 — Nix/devenv specifics

### How the Nix overlay works

The `flake.nix` has an `overlays = lib.mkAfter [...]` block in `devenv.shells.default` that overrides the Haskell package set. It intercepts derivations by `pname`:

```nix
mkDerivation = args:
    let drv = hprev.mkDerivation args;
    in if (args.pname or "") == "inter-hub-models"
       then drv.overrideAttrs (old: { ... })
       else if (args.pname or "") == "inter-hub-lib"
       then drv.overrideAttrs (old: { ... })
       else drv;
```

`postUnpack` runs after the source is unpacked but before configure. `postConfigure` runs after `setup configure`. Both run inside the Nix sandbox.

### Environment variables in Nix builds

- `$sourceRoot` — relative path to unpacked source (e.g., `inter-hub-models-src`)
- `$TMPDIR` — build-specific temp directory (e.g., `/nix/var/nix/builds/nix-PID-RND/`)
- `$NIX_BUILD_CORES` — parallelism (8 on haskelseed); NixPkgs Haskell builder passes `--ghc-option=-j$NIX_BUILD_CORES` by default
- `$NIX_BUILD_TOP` — may not be set; prefer `$TMPDIR`

### Configure flags ordering matters

NixPkgs Haskell builder prepends its own configureFlags before yours. Your overlay's flags are **appended** via `(old.configureFlags or []) ++ [ your-flags ]`. For boolean flags (`--enable-X` / `--disable-X`), the last one wins. For `--ghc-option=-jN`, all are passed and GHC uses the last one.

Example of NixPkgs flags you must know about:
```
--enable-shared
--enable-split-sections
--ghc-option=-j8       (from NIX_BUILD_CORES)
```

To override: append `--disable-shared`, `--disable-split-sections`, `--ghc-option=-j1` in your configureFlags. But beware: disabling shared breaks `.dyn_hi` for downstream packages.

### The `ghc-with-packages` symlink chain

The GHC wrapper used in builds (`ghc-with-packages`) exposes packages from both the wrapped GHC derivation and registered Haskell packages. Its `lib/` directory contains symlinks back into the unwrapped GHC derivation. When patching Nix store archives, verify the symlink chain resolves correctly:
```bash
readlink -f /nix/store/WRAP-ghc-9.10.3-with-packages/lib/.../libHSghc.a
wc -c /nix/store/WRAP-ghc-9.10.3-with-packages/lib/.../libHSghc.a
```

### Diagnosing `nix build` failures efficiently

```bash
# Get full build log for a specific derivation
nix log /nix/store/HASH-inter-hub-models-0.1.0.drv

# What derivations would be built (dry run)
nix build .#docker --dry-run

# Keep failed build artifacts for inspection
nix build .#docker --keep-failed

# Check if a store path exists (was the drv already built?)
ls /nix/store/EXPECTED-OUTPUT-HASH 2>/dev/null && echo "cached" || echo "not built"
```

---

## Part 4 — Haskell type system patterns

### Interface files (`.hi` and `.dyn_hi`)

GHC produces two kinds of interface files:
- `.hi` — static interface, for non-shared compilations
- `.dyn_hi` — dynamic interface, required when compiling with `--enable-shared` / `-dynamic-too`

A package compiled with `--disable-shared` produces `.hi` only. Any package depending on it that was compiled with `--enable-shared` will fail to load the `.dyn_hi`. Always maintain consistent shared/static settings across a package dependency chain.

### Type families vs Template Haskell

`type instance` declarations (like `Include "widgetId" ...`) are **type-level computations processed entirely by GHC during type-checking**. They are NOT Template Haskell, do NOT require the TH runtime, and do NOT trigger archive loading via the internal linker.

Template Haskell requires the TH evaluator (internal or external interpreter) only for modules with:
- `{-# LANGUAGE TemplateHaskell #-}` pragma
- Splice expressions `$(...)` or quasi-quotes `[| ... |]`
- `deriveGeneric`, `makeLenses`, etc. (lens/generics libraries)

If a module has none of these, `-fexternal-interpreter` has no meaningful effect on it.

### `module M` re-export vs explicit exports

```haskell
-- EMBEDS sub-interface into .hi (dangerous for large hubs):
module Hub (module Sub) where
import Sub

-- Stores compact name-reference in .hi (correct for large hubs):
module Hub (Foo(..), Bar, baz) where
import Sub (Foo(..), Bar, baz)
```

The `module M` form is fine for small modules (< 10 re-exported sub-modules with modest interfaces). It becomes a GHC 9.10.3 bomb for generated hubs with 50+ sub-modules full of typeclass instances.

### AR archive format and `readAr`

GHC's internal static linker uses `Data.Binary.Get.runGet` to read AR archives (`.a` files). Unlike `ar t` (which only reads headers), `readAr` reads **all entry content**. A truncated archive that passes `ar t` will still panic `readAr`. If you see:

```
panic! (the 'impossible' happened)
  Data.Binary.Get.runGet at position N: not enough bytes
```

Run `ar tv /path/to/suspect.a | tail -5` — if the last entry shows a size, check whether `wc -c /path/to/suspect.a` is substantially smaller than expected.

---

## Part 5 — Build compile tools and discipline

### Compile check scripts

Inside `devenv shell`:
```bash
scripts/compile-check          # full build via ghcid (all layers), errors → /tmp/ihub-compile-errors.txt
scripts/compile-check --bg     # background-friendly (no colour/title escape codes)
scripts/compile-check-core     # Layer 1+2 only — verify clean base
```

### Error-fix discipline

1. Fix **bottom-up**: Layer 1 → 2 → 3 → 4
2. Fix **one module at a time** — wait for ghcid to reload before touching the next module
3. **Never change Layer 1 (Web/Types.hs, Generated/Types.hs) while debugging Layer 3 errors** — it restarts the entire recompile
4. `Generated/Types.hs` is auto-generated from `Application/Schema.sql`. Edit the schema in IHP IDE, not the generated file

### Reading GHC error messages

GHC errors reference **source locations**, not compiled output. When you see:
```
Web/Controller/Foo.hs:42:5: error:
    • Couldn't match expected type 'UUID' with actual type 'Text'
```

The fix is in `Web/Controller/Foo.hs:42`, not in any generated file.

When you see:
```
Failed to load interface for 'Generated.Foo'
```
Check that `Generated.Foo` is in the `inter-hub-models` package and that models compiled successfully first.

### Parallel compilation flag (`-j`)

GHC's `-j` flag controls **parallel module compilation** within a single package. On haskelseed (2 CPU, ~3.8 GiB RAM), NixPkgs sets `-j8` (from `NIX_BUILD_CORES=8`). The overlay overrides to `-j1` for the models package to prevent OOM on the constrained host. Do not increase this without verifying available memory:
```bash
free -m  # on haskelseed
```

The env var `GHCRTS = "-A32m -M2g"` caps the GHC heap at 2 GiB and sets minor GC allocation to 32 MB. These are set in `devenv.shells.default.env`.

---

## Part 6 — Guardrails: expensive mistakes to avoid

### NEVER do these

| Action | Why it's expensive |
|--------|-------------------|
| Edit `build/Generated/Types.hs` directly | Regenerated on next schema sync; change is lost. Also, editing it changes Layer 1 and invalidates all 477 modules |
| Add `module M` re-export syntax to a new generated hub module | Embeds sub-interfaces → `.hi` overflow → GHC crash |
| Change `Web/Types.hs` or `Web/Types.hs` during a Layer 3 error loop | Restarts compile of all 59 controllers + 120 views |
| Add `--disable-shared` to models without removing it from lib | Missing `.dyn_hi` crash in every downstream package |
| Hardcode Nix store hashes in postUnpack scripts | Hash changes with every schema change; use `find /nix/store` instead |
| Run `nix build .#docker` on CoulombCore | Insufficient RAM (< 4 GiB), will OOM during GHC compilation |
| Trust `ar t` to validate an archive | `ar t` reads only headers; GHC reads content. Use `wc -c` and compare to expected size |
| Assume `devenv up` success = `nix build` success | Different compilation modes; static linker issues only surface in `nix build` |

### Before modifying `flake.nix` overlays

1. **Check what derivation hash the current flake produces**: `nix build .#docker --dry-run`
2. **Understand layer dependencies**: changing inter-hub-models configureFlags invalidates models AND all downstream (lib, binaries, docker image)
3. **Test postUnpack scripts locally first**: simulate with `bash -n yourscript.sh` and run `awk` commands against sample files
4. **Verify `.dyn_hi` will be produced** if `--enable-shared` is set (which is the NixPkgs default)

### Diagnosing a new crash before trying fixes

1. Get the full log: `nix log /nix/store/HASH-inter-hub-models-0.1.0.drv`
2. Find the crash position: `Data.Binary.Get.runGet at position N`
3. Determine which file is being read at that position (check file sizes of `.hi`, `.a`, `.conf` etc.)
4. Check `ar tv suspect.a | tail -5` to see if the last AR entry header is valid
5. Only then propose a fix — flag-based fixes almost never work for archive truncation issues

---

## Part 7 — IHP-specific Haskell patterns

### IHP record access

IHP generates `HasField` instances. Access fields with:
```haskell
record.fieldName          -- accessor (uses OverloadedRecordDot or get)
record |> set #fieldName value   -- functional update
```

### IHP type family: `Include`

```haskell
-- Generated/WidgetVersionInclude.hs (typical):
type instance Include "widgetId" (WidgetVersion' widgetId) =
    WidgetVersion' (GetModelById widgetId)
```

This is pure type-level. `Include` is a closed type family that fills in the concrete type for an association when you do eager loading with `fetchRelated`. It is NOT Template Haskell.

### IHP controller pattern

```haskell
-- Each action is a function returning an IO Response (via implicit context)
action ShowWidgetAction { widgetId } = do
    widget <- fetch widgetId
    render ShowView { .. }
```

Controller type errors almost always mean a mismatch in `Web/Types.hs` — the action type declared there must match the implementation.

### Schema → Generated code cycle

```
Edit Application/Schema.sql
  → IHP IDE at localhost:8001 generates build/Generated/
  → Regenerated: Types.hs, ActualTypes.hs, Foo.hs, FooInclude.hs
  → Also regenerates Web/Types.hs (action types) and Web/Routes.hs
  → Run compile-check to verify
```

After schema changes, always verify the postUnpack overlay in `flake.nix` still produces valid Haskell. The `_types` / `_exports` awk scripts in postUnpack depend on the generated file structure remaining stable.

---

## Quick reference card

```
# Build status checks (on haskelseed)
nix build .#docker --dry-run                     # what would be built?
nix log /nix/store/HASH-inter-hub-models-0.1.0.drv  # full build log
wc -c /nix/store/ffg3yf2.../libHSghc-9.10.3-5702.a  # verify archive size (must be ~289 MB)

# If archive is truncated (287 MB), apply patch:
# Full archive is at same derivation's ghc-9.10.3-partial/ subdirectory
# See project_ghc_build_fix.md in Claude memory for exact commands

# Compilation layers (in devenv shell)
scripts/compile-check-core    # Layer 1+2 only
scripts/compile-check         # Full build, errors → /tmp/ihub-compile-errors.txt

# Schema regeneration
# Use IHP IDE at localhost:8001, then run compile-check

# Production build → push → deploy
nix build .#docker
skopeo copy docker-archive:result docker://92.205.130.254:32166/coulomb/inter-hub:$(git rev-parse --short HEAD)
```