feat(WP-0004): railiance deployment & service ops

- Dockerfile (multi-stage, uv-based, slim runtime) - .dockerignore - docker-compose.railiance.yml (Temporal + NATS + PG, no Elasticsearch) - GET /health endpoint (db + temporal probes, 200/503) - .env.example (complete env var reference) - Makefile: migrate, sync-all, dev-up/down, railiance-up/down, start-worker, start-api, start-event-router, help targets; extracted sync-event-types Python to scripts/sync_event_types.py - SIGTERM graceful shutdown in worker.py and event_router.py - docs/runbook.md: Railiance deployment section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-15 00:04:39 +02:00
parent 987cf5a75c
commit 2a8e6cfe7f
11 changed files with 830 additions and 25 deletions
--- a/workplans/custodian-WP-0004-railiance-ops.md
+++ b/workplans/custodian-WP-0004-railiance-ops.md
@@ -0,0 +1,393 @@
+---
+id: custodian-WP-0004
+type: workplan
+domain: custodian
+repo: activity-core
+status: active
+state_hub_workstream_id: 759b1255-aa78-42b5-8fab-c5fcf168f5f4
+tasks:
+  - id: T58
+    title: Dockerfile (multi-stage, uv-based)
+    status: todo
+    priority: high
+    state_hub_task_id: 2a81f9ba-47cb-480f-a5d3-af74fe238485
+  - id: T59
+    title: docker-compose.railiance.yml (full stack, no Elasticsearch)
+    status: todo
+    priority: high
+    state_hub_task_id: 67981de4-b766-4c47-9985-9573297ac464
+  - id: T60
+    title: "GET /health endpoint"
+    status: todo
+    priority: high
+    state_hub_task_id: 7c7a1617-29e9-4ea5-a675-fd75e325c451
+  - id: T61
+    title: .env.example — complete env var reference
+    status: todo
+    priority: medium
+    state_hub_task_id: 2a2a8d02-19dc-4b09-8792-f855ed48388a
+  - id: T62
+    title: Makefile ops targets
+    status: todo
+    priority: medium
+    state_hub_task_id: 55ac37b5-5606-42e5-bf11-e8dae3a188b7
+  - id: T63
+    title: SIGTERM graceful shutdown (worker + event_router)
+    status: todo
+    priority: medium
+    state_hub_task_id: 65b3229b-5f03-450d-a263-f2c205be3d28
+  - id: T64
+    title: docs/runbook.md — railiance deployment section
+    status: todo
+    priority: medium
+    state_hub_task_id: 83dcd765-5715-499f-be23-f728d0261cfb
+created: "2026-05-14"
+---
+
+# activity-core WP-0004 — Railiance Deployment & Service Ops
+
+**Hub workstream:** `759b1255-aa78-42b5-8fab-c5fcf168f5f4`
+**Goal:** Package activity-core as a fully standalone deployable service, runnable
+on railiance with no imports from or requirements against other custodian repos.
+
+## Context
+
+WP-0001 through WP-0003 built the complete event-bridge implementation. The
+service is functionally complete. This workplan makes it operationally deployable:
+containerised, healthchecked, gracefully shutting down, and documented for railiance.
+
+Runtime dependencies are infrastructure only (Temporal, PostgreSQL, NATS). The
+optional soft dependencies (state-hub, repo-scoping, issue-core) are already
+gracefully degraded in the code — they bind `{}` on failure and never abort a run.
+
+## Existing assets (no rework needed)
+
+- `docker-compose.dev.yml` — full dev stack (Temporal + ES + PG + NATS); keep as-is
+- `worker.py`, `api.py`, `event_router.py` — entry points exist and are functional
+- `migrations/`, `alembic.ini` — schema management is in place
+- `docs/runbook.md` — dev quick-start section exists; extend for railiance
+
+## Build Order
+
+```
+T58 (Dockerfile) → T59 (railiance compose — depends on image)
+T60, T61, T62 — parallel, no deps
+T63 — independent, improves T59 (clean shutdown in compose)
+T64 — last, documents T58-T63
+```
+
+---
+
+## T58: Dockerfile (multi-stage, uv-based)
+
+**File:** `Dockerfile`
+
+Two-stage build to keep the runtime image lean.
+
+```dockerfile
+# Stage 1 — install Python deps
+FROM python:3.12-slim AS builder
+RUN pip install uv --no-cache-dir
+WORKDIR /app
+COPY pyproject.toml uv.lock ./
+COPY src/ ./src/
+RUN uv sync --no-dev --frozen
+
+# Stage 2 — runtime image
+FROM python:3.12-slim AS runtime
+WORKDIR /app
+COPY --from=builder /app/.venv /app/.venv
+COPY --from=builder /app/src /app/src
+# Include definition files that ship with the repo
+COPY activity-definitions/ ./activity-definitions/
+COPY event-types/ ./event-types/
+COPY tasks/ ./tasks/
+ENV PATH="/app/.venv/bin:$PATH"
+ENV PYTHONPATH="/app/src"
+CMD ["python", "-m", "activity_core.worker"]
+```
+
+Also add `.dockerignore`:
+```
+.venv/
+__pycache__/
+*.pyc
+.git/
+tests/
+*.egg-info/
+.env
+```
+
+The three processes share the same image; docker-compose overrides `command:`:
+- worker: default (`python -m activity_core.worker`)
+- api: `uvicorn activity_core.api:app --host 0.0.0.0 --port 8010`
+- event-router: `python -m activity_core.event_router`
+
+---
+
+## T59: docker-compose.railiance.yml (full stack, no Elasticsearch)
+
+**File:** `docker-compose.railiance.yml`
+
+Self-contained production stack. Uses PostgreSQL-based Temporal visibility (no
+Elasticsearch required). All activity-core services read env from `env_file: .env`.
+
+Services:
+- `temporal-db` — postgres:16 (Temporal schema + visibility)
+- `temporal` — temporalio/auto-setup:1.29.1 with `ENABLE_ES: "false"`,
+  `DB: postgres12`, `VISIBILITY_DBNAME: temporal_visibility`
+- `temporal-ui` — temporalio/ui on port 8080
+- `nats` — nats:2.10-alpine with `-js` and persistent volume for JetStream state
+- `app-db` — postgres:16 (activity-core application data)
+- `actcore-migrate` — one-shot service: `build: .`, `command: alembic upgrade head`,
+  `restart: no`, runs migrations then exits; other actcore services depend on it
+- `actcore-worker` — worker process, metrics port 9090
+- `actcore-api` — API server, port 8010, with healthcheck against `/health`
+- `actcore-event-router` — event router process
+
+All three actcore processes share one image (`build: .`) and depend on:
+- `temporal` (condition: service_healthy)
+- `app-db` (condition: service_healthy)
+- `nats` (condition: service_healthy)
+- `actcore-migrate` (condition: service_completed_successfully)
+
+Persistent volumes: `temporal-db-data`, `app-db-data`, `nats-data`.
+
+Network: `actcore-net` (bridge).
+
+---
+
+## T60: GET /health endpoint
+
+**File:** `src/activity_core/api.py` (add route)
+
+```python
+from sqlalchemy import text
+
+@app.get("/health")
+async def health() -> JSONResponse:
+    db_ok = False
+    temporal_ok = False
+
+    try:
+        async with _get_db()() as session:
+            await session.execute(text("SELECT 1"))
+        db_ok = True
+    except Exception:
+        pass
+
+    try:
+        await _get_temporal().describe_namespace(TEMPORAL_NAMESPACE)
+        temporal_ok = True
+    except Exception:
+        pass
+
+    status = "ok" if db_ok and temporal_ok else "degraded"
+    code = 200 if status == "ok" else 503
+    return JSONResponse(
+        {"status": status, "db": db_ok, "temporal": temporal_ok},
+        status_code=code,
+    )
+```
+
+No authentication. Used by:
+- `docker-compose healthcheck: test: ["CMD-SHELL", "curl -sf http://localhost:8010/health"]`
+- Railiance monitoring (external HTTP probe)
+
+---
+
+## T61: .env.example — complete env var reference
+
+**File:** `.env.example`
+
+```bash
+# ── Required ──────────────────────────────────────────────────────────────────
+# PostgreSQL connection string for activity-core application data.
+ACTCORE_DB_URL=postgresql+asyncpg://actcore:actcore@app-db:5432/actcore
+
+# ── Temporal ──────────────────────────────────────────────────────────────────
+# Temporal frontend gRPC address.
+TEMPORAL_HOST=temporal:7233
+# Temporal namespace (must exist before workers start).
+TEMPORAL_NAMESPACE=default
+
+# ── NATS ──────────────────────────────────────────────────────────────────────
+# NATS server URL. JetStream must be enabled (-js flag).
+NATS_URL=nats://nats:4222
+
+# ── Service integrations (gracefully degraded if unavailable) ─────────────────
+# State Hub — used by the state-hub context adapter. Binds {} on failure.
+STATE_HUB_URL=http://127.0.0.1:8000
+# Repo scoping — used by the repo-scoping context adapter. Binds {} on failure.
+REPO_SCOPING_URL=http://127.0.0.1:8020
+# Issue Core — task emission backend.
+ISSUE_CORE_URL=http://127.0.0.1:8010
+# Sink type: 'rest' (POST to issue-core) or 'null' (discard, for dry-run).
+ISSUE_SINK_TYPE=rest
+
+# ── Activity definitions ───────────────────────────────────────────────────────
+# Colon-separated paths to additional activity-definitions/ directories.
+# The local activity-definitions/ directory is always scanned.
+ACTIVITY_DEFINITION_DIRS=
+
+# ── Observability ─────────────────────────────────────────────────────────────
+# Prometheus metrics bind address (Temporal SDK metrics).
+PROMETHEUS_BIND_ADDR=0.0.0.0:9090
+
+# ── Security (webhook receiver) ───────────────────────────────────────────────
+# HMAC-SHA256 secret for Gitea webhook signature validation.
+WEBHOOK_SECRET_GITEA=
+# HMAC-SHA256 secret for GitHub webhook signature validation.
+WEBHOOK_SECRET_GITHUB=
+
+# ── Curator gate ──────────────────────────────────────────────────────────────
+# 'disabled': accepts active + pending event types (pending logged as warning).
+# 'required': only active event types accepted; pending events are discarded.
+ACTIVITY_CURATOR_GATE=disabled
+```
+
+---
+
+## T62: Makefile ops targets
+
+**File:** `Makefile` (extend)
+
+New targets to add:
+
+```makefile
+# ── Infrastructure ────────────────────────────────────────────────────────────
+dev-up:         ## Start full dev stack (Temporal + PG + NATS)
+dev-down:       ## Stop and remove dev stack containers
+railiance-up:   ## Start full railiance stack (builds image first)
+railiance-down: ## Stop and remove railiance stack containers
+
+# ── Database ──────────────────────────────────────────────────────────────────
+migrate:        ## Apply all pending Alembic migrations
+sync-all:       ## Sync event types and activity definitions (runs both)
+
+# ── Local dev processes ───────────────────────────────────────────────────────
+start-worker:        ## Start Temporal worker (uses ACTCORE_DB_URL from env)
+start-api:           ## Start FastAPI server on :8010 (hot reload)
+start-event-router:  ## Start NATS event router
+
+# ── Help ──────────────────────────────────────────────────────────────────────
+help:           ## Show this help message
+```
+
+The `help` target uses `grep -E '^[a-zA-Z_-]+:.*?##' Makefile` to extract the
+`## description` comments and format them as a table. Makes `make help` the entry
+point for operators.
+
+`start-*` targets load `.env` if it exists (`-include .env; export`) so developers
+can run locally without setting env vars manually.
+
+---
+
+## T63: SIGTERM graceful shutdown (worker + event_router)
+
+**Files:** `src/activity_core/worker.py`, `src/activity_core/event_router.py`
+
+Replace the `await asyncio.Future()` pattern (which blocks forever, ignoring
+SIGTERM) with a signal-aware stop event:
+
+```python
+import signal
+
+async def run() -> None:
+    ...
+    loop = asyncio.get_running_loop()
+    stop = asyncio.Event()
+    loop.add_signal_handler(signal.SIGTERM, stop.set)
+    loop.add_signal_handler(signal.SIGINT, stop.set)
+
+    async with orchestrator_worker, task_worker:  # (worker.py)
+        logger.info("Workers running — waiting for shutdown signal")
+        await stop.wait()
+        logger.info("Shutdown signal received — draining workers")
+    logger.info("Workers stopped cleanly")
+```
+
+For `event_router.py`, the same pattern applies: `stop.wait()` replaces the
+infinite future; on signal, the NATS consumer is unsubscribed before exiting.
+
+This ensures `docker stop` (SIGTERM → 10-second grace period → SIGKILL) completes
+within the grace window by draining in-flight Temporal tasks and NATS messages
+before the process exits.
+
+---
+
+## T64: docs/runbook.md — Railiance Deployment section
+
+**File:** `docs/runbook.md` (extend with new section)
+
+Add after the existing Dev environment section:
+
+```markdown
+## Railiance Deployment
+
+### Pre-requisites
+- Docker ≥ 24 with Compose v2 (`docker compose` not `docker-compose`)
+- ≥ 4 GB RAM available (Temporal server takes ~1 GB)
+- Ports available: 7233 (Temporal gRPC), 8010 (API), 8080 (Temporal UI),
+  9090 (Prometheus metrics)
+
+### First-time setup
+1. `cp .env.example .env` — edit all values, especially secrets
+2. `make railiance-up` — builds image and starts all services
+3. Wait for health: `curl -sf http://localhost:8010/health` → `{"status":"ok",...}`
+4. Register Temporal search attributes (one-time per namespace):
+   `docker exec actcore-temporal temporal operator search-attribute create \`
+   `  --name ActivityId --type Keyword --name ActivityName --type Keyword \`
+   `  --address temporal:7233`
+5. `make sync-all` — load event types and activity definitions
+
+### Upgrade procedure
+1. `git pull`
+2. `make railiance-up` — rebuilds image and restarts changed services
+3. `make migrate` — apply any new migrations (safe to run even if none pending)
+4. `curl -sf http://localhost:8010/health` — verify health
+
+### Health verification
+- API health: `curl -s http://localhost:8010/health | python3 -m json.tool`
+- Temporal UI: http://localhost:8080
+- Prometheus metrics: http://localhost:9090/metrics
+
+### Common ops
+- View logs: `docker compose -f docker-compose.railiance.yml logs -f actcore-worker`
+- Restart one service: `docker compose -f docker-compose.railiance.yml restart actcore-api`
+- Wipe and reset (destructive): `make railiance-down && docker volume rm ...`
+```
+
+---
+
+## Completion Criteria
+
+1. `docker build -t activity-core .` succeeds and image is < 400 MB
+2. `make railiance-up` starts all 8 services; all reach healthy state
+3. `curl http://localhost:8010/health` returns `{"status":"ok",...}` with HTTP 200
+4. `docker stop actcore-worker` causes graceful drain (no SIGKILL within 10s)
+5. `make help` prints a clean table of all targets with descriptions
+6. `.env.example` covers every env var used anywhere in the codebase
+
+## New Files Produced
+
+| Path | Task |
+|---|---|
+| `Dockerfile` | T58 |
+| `.dockerignore` | T58 |
+| `docker-compose.railiance.yml` | T59 |
+| `.env.example` | T61 |
+
+## Modified Files
+
+| Path | Task | Change |
+|---|---|---|
+| `src/activity_core/api.py` | T60 | Add `/health` route |
+| `Makefile` | T62 | Add ops targets |
+| `src/activity_core/worker.py` | T63 | SIGTERM handler |
+| `src/activity_core/event_router.py` | T63 | SIGTERM handler |
+| `docs/runbook.md` | T64 | Railiance deployment section |
+
+## Change History
+
+- v1.0 (2026-05-14): Initial workplan.