coulomb/ops-warden
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(WARDEN-WP-0021): T1+T2 — scheduled worker tick enabled (systemd --user timer)

Browse Source 
T1: systemd --user units (ops-warden-worker.{service,timer}) + scripts/install-worker-timer.sh
(--enable opt-in, cron fallback documented) + examples/worker.env.example. Kill switch:
`systemctl --user disable --now ops-warden-worker.timer` or WORKER_ENABLED=0. Installed and
ENABLED — verified a real systemd run (Result=success, used the llm brain) and the timer is
active (next run +15min).

T2: hardened worker-tick.sh — State Hub /state/health precheck → graceful skip (exit 0) when
unreachable; worker-run failure logged but never fails the unit (retry next tick). Verified
hub-down skip and a live tick.

Conservative tier only; nothing auto-sent. Kill switch is one command.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-06-30 15:19:23 +02:00
parent 97504aa444
commit 9dc1db0162
6 changed files with 105 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
scripts/worker-tick.sh
View File

@@ -16,16 +16,29 @@ ROOT="$(cd "$(dirname "$0")/.." && pwd)"
STATE="${WARDEN_STATE_DIR:-$HOME/.local/state/warden}"
mkdir -p "$STATE"
# Master off-switch (env file / WORKER_ENABLED=0) — skip without touching the timer.
if [[ "${WORKER_ENABLED:-1}" == "0" ]]; then
echo "$(date -Is) tick: WORKER_ENABLED=0; skip"
exit 0
fi
# Concurrency guard — never let two ticks overlap.
exec 9>"$STATE/worker-tick.lock"
flock -n 9 || { echo "$(date -Is) tick: another run holds the lock; skip"; exit 0; }
BRAIN="${WORKER_BRAIN:-llm}"
HUB_URL="${WARDEN_HUB_URL:-http://127.0.0.1:8000}"
LLM_URL="${LLM_CONNECT_URL:-}"
PF_PID=""
cleanup() { [[ -n "$PF_PID" ]] && kill "$PF_PID" 2>/dev/null || true; }
trap cleanup EXIT
# Graceful skip if the State Hub is unreachable — a transient outage is not a fault.
if ! curl -fsS -m 6 "$HUB_URL/state/health" >/dev/null 2>&1; then
echo "$(date -Is) tick: State Hub unreachable at $HUB_URL; skip"
exit 0
fi
if [[ "$BRAIN" == "llm" && -z "$LLM_URL" ]]; then
if command -v kubectl >/dev/null 2>&1; then
kubectl -n activity-core port-forward deploy/llm-connect 18080:8080 >/dev/null 2>&1 &
@@ -38,5 +51,11 @@ if [[ "$BRAIN" == "llm" && -z "$LLM_URL" ]]; then
fi
fi
echo "$(date -Is) tick: brain=$BRAIN"
LLM_CONNECT_URL="$LLM_URL" uv run --directory "$ROOT" warden worker run --execute --brain "$BRAIN"
echo "$(date -Is) tick: brain=$BRAIN hub=$HUB_URL"
# A worker-run failure (transient hub/llm hiccup) is logged but never fails the unit —
# the next tick retries. Real bugs still surface in the log.
if ! LLM_CONNECT_URL="$LLM_URL" WARDEN_HUB_URL="$HUB_URL" \
uv run --directory "$ROOT" warden worker run --execute --brain "$BRAIN"; then
echo "$(date -Is) tick: worker run returned non-zero; will retry next tick"
fi
exit 0