generated from coulomb/repo-seed
175 lines
6.1 KiB
Markdown
175 lines
6.1 KiB
Markdown
---
|
|
id: ACTIVITY-WP-0012
|
|
type: workplan
|
|
title: "Definition And Schedule Hot Reload"
|
|
domain: custodian
|
|
repo: activity-core
|
|
status: active
|
|
owner: codex
|
|
topic_slug: custodian
|
|
created: "2026-06-18"
|
|
updated: "2026-06-19"
|
|
state_hub_workstream_id: "8887075e-21ec-451b-b82b-cd81035c9ca5"
|
|
---
|
|
|
|
# ACTIVITY-WP-0012 - Definition And Schedule Hot Reload
|
|
|
|
## Context
|
|
|
|
State Hub message `f4876517-f738-4571-a2d6-76f2965e9a13` from
|
|
`coulomb-loop` reports an operational gap from the Coulomb cadence ramp: after
|
|
renaming customer definitions from hourly to daily, operators had to run
|
|
definition/schedule sync and restart the worker before new Temporal schedule
|
|
state was reliable.
|
|
|
|
Current behavior:
|
|
|
|
- `worker.py` runs `sync_activity_definitions` and `sync_schedules` once at
|
|
startup.
|
|
- `RunActivityWorkflow` loads ActivityDefinitions from the DB at activity time.
|
|
- The event router reloads enabled event definitions per NATS message.
|
|
- Cron schedule changes only take effect when `sync_schedules` runs.
|
|
|
|
This belongs in activity-core because the repo owns ActivityDefinition sync,
|
|
Temporal schedule projection, and the admin API. The first implementation
|
|
should expose an operator-triggered sync path without turning activity-core into
|
|
a repo checkout manager or CI system.
|
|
|
|
## Extract Reusable Sync Service
|
|
|
|
```task
|
|
id: ACTIVITY-WP-0012-T01
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "53a7970b-7eec-47f5-ad30-bbd7c6271952"
|
|
```
|
|
|
|
Refactor the worker-startup sync sequence into a reusable async service that can
|
|
be called by startup and the API.
|
|
|
|
Done when:
|
|
|
|
- the service can run ActivityDefinition sync, event type sync, and Temporal
|
|
schedule sync independently based on booleans;
|
|
- it accepts the existing DB session factory / Temporal client dependencies
|
|
without creating hidden global state;
|
|
- startup behavior remains unchanged except for calling the shared service;
|
|
- failures are collected into a bounded `errors[]` result while preserving the
|
|
current startup best-effort behavior.
|
|
|
|
2026-06-19: Completed. Added `activity_core.sync_service.run_sync`, which
|
|
orchestrates ActivityDefinition, event type, and schedule sync independently
|
|
from explicit DB session factory and Temporal client dependencies. Worker
|
|
startup now calls the shared service for definitions+schedules and logs bounded
|
|
stage errors while continuing startup.
|
|
|
|
## Add Admin Sync Endpoint
|
|
|
|
```task
|
|
id: ACTIVITY-WP-0012-T02
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "8697c761-15d1-4da0-b66b-d838218a2495"
|
|
```
|
|
|
|
Add an operator-only API endpoint:
|
|
|
|
`POST /admin/sync?definitions=true&schedules=true&event_types=true`
|
|
|
|
Done when:
|
|
|
|
- the endpoint runs the shared sync service without requiring worker restart;
|
|
- response JSON reports counts for definitions, event types, schedules upserted,
|
|
schedules paused/deleted, and errors;
|
|
- default parameters sync definitions and schedules, with event types opt-in or
|
|
clearly documented;
|
|
- endpoint tests cover definitions-only, schedules-only, all-sync, and failure
|
|
result behavior.
|
|
|
|
2026-06-19: Completed. Added `POST /admin/sync` with defaults
|
|
`definitions=true`, `schedules=true`, and `event_types=false`. The response
|
|
reports definition/event counts, schedule upsert/pause/orphan-delete counts, and
|
|
bounded `errors[]`. Tests cover definitions-only, schedules-only, all-sync, and
|
|
failure-result behavior.
|
|
|
|
## Preserve Schedule Drift Semantics
|
|
|
|
```task
|
|
id: ACTIVITY-WP-0012-T03
|
|
status: done
|
|
priority: high
|
|
state_hub_task_id: "efeac412-632c-4c90-9428-bb575ac7a624"
|
|
```
|
|
|
|
Make the sync result explicit enough for cadence changes and renames.
|
|
|
|
Done when:
|
|
|
|
- disabled cron definitions pause their Temporal schedules on sync;
|
|
- renamed definitions create the new schedule and pause/delete orphaned old
|
|
schedules according to the existing `sync_schedules` semantics;
|
|
- event-triggered definitions remain hot through the existing router DB reload
|
|
path;
|
|
- regression tests demonstrate the Coulomb hourly-to-daily rename shape without
|
|
needing a worker restart.
|
|
|
|
2026-06-19: Completed. `sync_schedules` now returns explicit counts for enabled
|
|
schedule upserts, disabled schedule pauses, and orphan deletes. Regression tests
|
|
cover the hourly-to-daily rename shape: a new enabled cron schedule is upserted,
|
|
the old disabled cron schedule is preserved as paused, unrelated orphan
|
|
schedules are deleted, event-triggered definitions do not create schedules, and
|
|
one-shot scheduled definitions are no longer mistaken for orphans.
|
|
|
|
## Optional Background Sync Loop
|
|
|
|
```task
|
|
id: ACTIVITY-WP-0012-T04
|
|
status: done
|
|
priority: medium
|
|
state_hub_task_id: "d774087b-c51d-4444-8e90-bfef43765456"
|
|
```
|
|
|
|
Decide whether to add a periodic sync loop after the admin endpoint exists.
|
|
|
|
Done when:
|
|
|
|
- either `ACTIVITY_SYNC_INTERVAL_SECONDS` is implemented with a default disabled
|
|
or conservative interval, or the workplan records why manual/admin-triggered
|
|
sync is the safer v1 posture;
|
|
- if implemented, logs and metrics expose the last successful sync timestamp and
|
|
last error summary;
|
|
- the loop does not block worker startup or workflow task processing.
|
|
|
|
2026-06-19: Completed by decision. v1 stays manual/operator-triggered through
|
|
`POST /admin/sync`; no background loop was added. The runbook records this
|
|
posture so customer definition changes stay explicit and the worker does not
|
|
start background repo scanning. A periodic loop remains a future option if live
|
|
operator use proves it is needed.
|
|
|
|
## Live No-Restart Smoke
|
|
|
|
```task
|
|
id: ACTIVITY-WP-0012-T05
|
|
status: wait
|
|
priority: high
|
|
state_hub_task_id: "68a0e22a-106a-4d21-9f39-c6279850cb5e"
|
|
```
|
|
|
|
Validate the hot-reload path in the cluster/operator environment.
|
|
|
|
Done when non-secret State Hub evidence shows:
|
|
|
|
- a customer repo definition rename or `enabled` flip is synced through
|
|
`/admin/sync`;
|
|
- new Temporal schedules are active and retired schedules are paused/deleted
|
|
without worker SIGTERM or pod restart;
|
|
- event-triggered definitions still fire normally;
|
|
- rollback or repeat sync is idempotent.
|
|
|
|
Current wait reason: this gate depends on the implementation tasks and a
|
|
cluster-owned smoke path.
|
|
|
|
2026-06-19: Implementation tasks T01-T04 are complete and the repository test
|
|
suite passed (`192 passed, 1 skipped`). This task is now ready for a
|
|
Railiance/operator no-restart smoke using `POST /admin/sync`.
|