---
id: ADHOC-2026-06-15
type: workplan
title: "Ad hoc Inter-Hub production fixes"
domain: custodian
repo: inter-hub
status: blocked
owner: codex
created: "2026-06-15"
updated: "2026-06-16"
state_hub_workstream_id: "9e7a50b4-da7f-4df9-9154-7b89a071f520"
---

# Ad hoc Inter-Hub production fixes

## Fix COUNT decode failures in v2 bootstrap endpoints

```task
id: ADHOC-2026-06-15-T01
status: blocked
priority: high
state_hub_task_id: "cceee9f1-56af-44bc-898d-21c4508df07c"
```

Production Ops Hub bootstrap exposed a PostgreSQL/Haskell type mismatch in
the v2 API helpers. `COUNT(*)` returns `bigint`, while the helper code decoded
the result as `Int`, causing `UnexpectedColumnTypeStatementError` in widget
type validation and API request log rate-limit checks.

Fix the count queries so widget creation and authenticated hub-registry reads
work through the documented v2 bootstrap API.

Source fix on 2026-06-15:

- `Application/Helper/TypeRegistry.hs` now casts registry validation
  `COUNT(*)` queries to `int`.
- `Application/Helper/ApiRateLimit.hs` now casts API request log
  `COUNT(*)` queries to `int`.
- Commit `5101eb5 Fix API count decoding` was pushed to `origin/main`.

Blocked before live completion:

- The Gitea deploy workflow did not update production during the session.
- Production still reports image `gitea.coulomb.social/coulomb/inter-hub:5c13de1`.
- Local `nix develop ... scripts/compile-check` is blocked by local devenv
  setup, and the local `nix build .#docker` remained in dependency compilation
  after more than 20 minutes. The build was stopped cleanly.

Deploy trigger attempt on 2026-06-15:

- Confirmed current `main` contains the COUNT decode fix and is at commit
  `f8fde35`.
- Confirmed the deploy workflow is the normal path and is pinned to
  `runs-on: [self-hosted, haskelseed]`.
- Confirmed image tag `gitea.coulomb.social/coulomb/inter-hub:f8fde35`
  returns `manifest unknown`.
- Gitea Actions API inspection/dispatch was attempted using the locally
  configured `tea` token, but the public HTTPS API returned `401 Unauthorized`
  for Actions endpoints; the raw configured HTTP endpoint was not reachable
  from this session.
- Pushed empty commit `68c66b9` (`chore: trigger inter-hub deploy`) because
  the previous contract/docs commit was ignored by the deploy workflow's
  `paths-ignore` rules.
- Polled the registry for
  `gitea.coulomb.social/coulomb/inter-hub:68c66b9` for about five minutes
  after push; it continued to return `manifest unknown`.

Current wait reason: the source fix is pushed, but image publication/deploy now
requires authenticated Gitea Actions workflow dispatch or inspection of the
self-hosted `haskelseed` runner path. The normal workflow needs haskelseed as
build runner; an equivalent operator-controlled build host with Nix, registry
push credentials, and Railiance deploy credentials could substitute.

Recheck on 2026-06-16:

- The local source fix is still present:
  `Application/Helper/TypeRegistry.hs` casts registry validation counts with
  `COUNT(*)::int`, and `Application/Helper/ApiRateLimit.hs` casts API request
  log counts with `COUNT(*)::int`.
- A source-wide `COUNT` search found the targeted v2 bootstrap helpers fixed.
  Other raw aggregate counts remain in non-bootstrap dashboard/marketplace/API
  surfaces and are outside this ad hoc task's acceptance path unless they are
  separately reproduced as decode failures.
- Live public `GET https://hub.coulomb.social/api/v2/hubs` returns `200` and
  lists `ops-hub`, confirming the public API and ops-hub route surface are
  present.
- Live unauthenticated `GET /api/v2/widgets` and `GET /api/v2/hub-registry`
  return `401`, confirming the protected routes exist and authentication is
  enforced before the code path that previously failed.
- Unauthenticated registry manifest checks for tags `68c66b9` and `5101eb5`
  now return `401`, not the earlier unauthenticated `manifest unknown`; this
  session cannot prove image publication from the public registry endpoint.
- The previously documented local temp key
  `/tmp/ops-hub-runtime-key-gb5nxg92` is absent. No approved runtime key or
  operator key is available in this session, so the protected widget-create and
  hub-registry smoke checks could not be run without a secret handoff.

Current blocked reason: source-side work appears complete, but production
closure still requires one of:

1. an attended operator/runtime key handoff so Codex can run the protected
   smoke without printing the key;
2. operator-provided non-secret evidence that production is running an image
   containing commit `5101eb5` or an equivalent COUNT decode fix; or
3. operator-run smoke evidence showing authenticated `POST /api/v2/widgets`
   and authenticated `GET /api/v2/hub-registry` succeed against production.

Until one of those exists, this ad hoc workplan should remain `blocked`, not
`done`.