Add project scaffold: contracts, schemas, docker-compose, workplans

Phase 0 contracts (event envelope, ActivityDefinition, idempotency doc, naming conventions) and Phase 1 Temporal cluster setup (docker-compose.dev.yml, Temporal dynamic config) are complete. Includes Pydantic models, JSON schemas, wiki architecture docs, and ADR-001 workplan files for both workstreams. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 22:45:40 +01:00
parent 098c13febd
commit 6f9132314f
16 changed files with 2652 additions and 0 deletions
--- a/workplans/custodian-WP-0001-temporal-backbone.md
+++ b/workplans/custodian-WP-0001-temporal-backbone.md
@@ -0,0 +1,179 @@
+---
+id: custodian-WP-0001
+type: workplan
+domain: custodian
+repo: activity-core
+status: active
+state_hub_workstream_id: e22516cc-4fdf-4dad-a928-a9e04a9fa7c7
+tasks:
+  - id: T01
+    title: Define event envelope schema
+    status: done
+    state_hub_task_id: 3dc749eb-7a73-4199-9c8d-e8f31bac5372
+  - id: T02
+    title: Write ActivityDefinition schema (Pydantic + JSON Schema)
+    status: done
+    state_hub_task_id: 107115b0-4476-424d-84ff-03992a85d40b
+  - id: T03
+    title: Document idempotency contract
+    status: done
+    state_hub_task_id: aa638595-0bca-41ab-b140-b077d05994e1
+  - id: T04
+    title: Define Temporal namespace and task queue naming conventions
+    status: done
+    state_hub_task_id: a03c823e-5873-4586-912a-6d1370383c7b
+  - id: T05
+    title: Stand up Temporal via docker-compose
+    status: done
+    state_hub_task_id: 70e7f742-378f-4129-a0f4-f25ccaccffd0
+  - id: T06
+    title: Add docker-compose.dev.yml to repo
+    status: done
+    state_hub_task_id: 61cb7426-e0ec-4a52-870d-18a26b3c4f3e
+  - id: T07
+    title: Smoke test Temporal cluster and UI
+    status: blocked
+    blocking_reason: Docker image pulls failing with tls bad-record-MAC errors (network/MTU issue). Images must be pulled on a stable network connection.
+    state_hub_task_id: f28a405d-ddcc-44d8-b443-f26f3ab34eaa
+  - id: T08
+    title: Choose and configure DB migration tool
+    status: todo
+    state_hub_task_id: d9fe3e54-ec1a-4e23-aa76-d8869f4e024d
+  - id: T09
+    title: Write activity_definitions migration
+    status: todo
+    state_hub_task_id: 47774e01-1026-478e-9a46-7d676bfed45c
+  - id: T10
+    title: Write activity_runs migration
+    status: todo
+    state_hub_task_id: 0a74f29f-c07d-4338-be90-e5cf4087261b
+  - id: T11
+    title: Write task_instances migration
+    status: todo
+    state_hub_task_id: 491a6903-8189-43bb-958f-4d16abc84f8e
+  - id: T12
+    title: Seed one example ActivityDefinition
+    status: todo
+    state_hub_task_id: f24662ff-4a26-48bd-b997-57e7586c7f11
+  - id: T13
+    title: Scaffold Python worker project
+    status: todo
+    state_hub_task_id: e0205c56-1d40-4142-952b-e27ff6a44e1d
+  - id: T14
+    title: Implement load_activity_definition activity
+    status: todo
+    state_hub_task_id: b05f046f-a6ba-4d96-a298-a0bbea067427
+  - id: T15
+    title: Implement resolve_context activity (stub)
+    status: todo
+    state_hub_task_id: 2417912f-845d-489a-ace4-fb9280d3b679
+  - id: T16
+    title: Implement evaluate_templates (pure function)
+    status: todo
+    state_hub_task_id: b7decbb6-ad2b-4fa5-8efc-05a7eb435d76
+  - id: T17
+    title: Implement log_run activity
+    status: todo
+    state_hub_task_id: e019cb5a-adf0-4a5d-9410-c41810128190
+  - id: T18
+    title: Implement RunActivityWorkflow
+    status: todo
+    state_hub_task_id: fb6b3440-47d2-4b0a-97c1-6e780cc497c4
+  - id: T19
+    title: Implement TaskExecutorWorkflow (stub)
+    status: todo
+    state_hub_task_id: 70a7365f-3042-4770-b3cd-3c6724b0790d
+  - id: T20
+    title: Wire up worker entrypoint
+    status: todo
+    state_hub_task_id: 1da921f5-86a8-488f-a015-402079194e10
+  - id: T21
+    title: Manual end-to-end test
+    status: todo
+    state_hub_task_id: f72bba1a-eb24-496e-9498-f4676facc5c9
+---
+
+# activity-core Foundation — Temporal Backbone
+
+**Workstream:** activity-core Foundation — Temporal Backbone
+**Hub ID:** `e22516cc-4fdf-4dad-a928-a9e04a9fa7c7`
+
+## Purpose
+
+Build the first working slice of `activity-core`: contracts, a self-hosted Temporal cluster,
+PostgreSQL schema, and the `RunActivityWorkflow` end-to-end. Deliverable: a manually-triggered
+workflow that loads an `ActivityDefinition`, resolves context, evaluates task templates, spawns
+child workflows, and writes a run log — all durable and observable via Temporal Web UI.
+
+## Architecture reference
+
+`wiki/ActivityCorePlan_chtgpt.md` and `wiki/ActivityCorePlan_grok.md` — both converge on
+Temporal self-hosted · Python SDK · PostgreSQL.
+
+| Concept | Implementation |
+|---|---|
+| `ActivityDefinition` | Versioned Postgres record: trigger config, context sources, task templates |
+| Activity Orchestrator | `RunActivityWorkflow` — durable Temporal workflow |
+| `TaskInstance` | Child `TaskExecutorWorkflow` (stub in this workstream) |
+| Idempotency | Workflow ID = `activity_id + ":" + scheduled_for` |
+
+## Open decisions (resolve before Phase 1)
+
+- **Dev deployment target** (hub: `9672adcc-da86-4038-9263-b556789c109e`): Docker Compose vs K8s
+  — proto-plans recommend Docker Compose for the initial dev slice.
+
+---
+
+## Phase 0 — Contracts & Conventions
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T01: Define event envelope schema | high | `3dc749eb-7a73-4199-9c8d-e8f31bac5372` |
+| T02: Write ActivityDefinition schema (Pydantic + JSON Schema) | high | `107115b0-4476-424d-84ff-03992a85d40b` |
+| T03: Document idempotency contract | high | `aa638595-0bca-41ab-b140-b077d05994e1` |
+| T04: Define Temporal namespace + task queue naming conventions | high | `a03c823e-5873-4586-912a-6d1370383c7b` |
+
+---
+
+## Phase 1 — Temporal Cluster (dev)
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T05: Stand up Temporal via docker-compose | high | `70e7f742-378f-4129-a0f4-f25ccaccffd0` |
+| T06: Add docker-compose.dev.yml to repo | high | `61cb7426-e0ec-4a52-870d-18a26b3c4f3e` |
+| T07: Smoke test Temporal cluster and UI | high | `f28a405d-ddcc-44d8-b443-f26f3ab34eaa` |
+
+---
+
+## Phase 2 — App DB Schema
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T08: Choose and configure DB migration tool | high | `d9fe3e54-ec1a-4e23-aa76-d8869f4e024d` |
+| T09: Write activity_definitions migration | high | `47774e01-1026-478e-9a46-7d676bfed45c` |
+| T10: Write activity_runs migration | high | `0a74f29f-c07d-4338-be90-e5cf4087261b` |
+| T11: Write task_instances migration | high | `491a6903-8189-43bb-958f-4d16abc84f8e` |
+| T12: Seed one example ActivityDefinition | medium | `f24662ff-4a26-48bd-b997-57e7586c7f11` |
+
+---
+
+## Phase 3 — RunActivityWorkflow
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T13: Scaffold Python worker project | medium | `e0205c56-1d40-4142-952b-e27ff6a44e1d` |
+| T14: Implement load_activity_definition activity | medium | `b05f046f-a6ba-4d96-a298-a0bbea067427` |
+| T15: Implement resolve_context activity (stub) | medium | `2417912f-845d-489a-ace4-fb9280d3b679` |
+| T16: Implement evaluate_templates (pure function) | medium | `b7decbb6-ad2b-4fa5-8efc-05a7eb435d76` |
+| T17: Implement log_run activity | medium | `e019cb5a-adf0-4a5d-9410-c41810128190` |
+| T18: Implement RunActivityWorkflow | medium | `fb6b3440-47d2-4b0a-97c1-6e780cc497c4` |
+| T19: Implement TaskExecutorWorkflow (stub) | medium | `70a7365f-3042-4770-b3cd-3c6724b0790d` |
+| T20: Wire up worker entrypoint | medium | `1da921f5-86a8-488f-a015-402079194e10` |
+| T21: Manual end-to-end test | medium | `f72bba1a-eb24-496e-9498-f4676facc5c9` |
+
+---
+
+## Completion criteria
+
+Trigger `RunActivityWorkflow` manually → workflow completes in Temporal UI → `activity_runs`
+row written → `task_instances` rows written. Zero scheduler or queue infrastructure required.
--- a/workplans/custodian-WP-0002-triggers-ops.md
+++ b/workplans/custodian-WP-0002-triggers-ops.md
@@ -0,0 +1,117 @@
+---
+id: custodian-WP-0002
+type: workplan
+domain: custodian
+repo: activity-core
+status: active
+state_hub_workstream_id: 3a4f47d9-8bc1-434e-acb4-bed5d4dacda0
+tasks:
+  - id: T22
+    title: Write schedule_manager.py
+    status: todo
+    state_hub_task_id: e50550d1-9904-41d7-afd8-492a1f1e91b8
+  - id: T23
+    title: Bootstrap script to sync schedules on startup
+    status: todo
+    state_hub_task_id: 5a1f7fa3-acb9-4f60-9892-c9eaa120272e
+  - id: T24
+    title: Handle misfire policy in schedule config
+    status: todo
+    state_hub_task_id: 00231668-95c5-447f-b3d0-1fb8c20b487f
+  - id: T25
+    title: Test schedule pause/resume lifecycle
+    status: todo
+    state_hub_task_id: 7abfd375-ea9d-4209-8371-e5664dc2c6c4
+  - id: T26
+    title: Implement Event Router service
+    status: todo
+    state_hub_task_id: 68b6610b-159c-4f1c-92a9-7128efea0961
+  - id: T27
+    title: Implement routing rules (event.type + filters → activity_ids)
+    status: todo
+    state_hub_task_id: 9348efea-a7e9-4f92-b866-8fc82cf28fee
+  - id: T28
+    title: Start/signal workflow from Event Router
+    status: todo
+    state_hub_task_id: cac1f45a-7391-471a-9566-97cdbd96eb2d
+  - id: T29
+    title: Integration test — publish event → observe workflow run
+    status: todo
+    state_hub_task_id: 7f10b5a3-7cad-4914-b603-d57508c85629
+  - id: T30
+    title: REST API (FastAPI) — CRUD for ActivityDefinition
+    status: todo
+    state_hub_task_id: b27e54a1-5dcc-476d-8f4a-c995aea6a8c2
+  - id: T31
+    title: Wire Temporal SDK metrics to Prometheus
+    status: todo
+    state_hub_task_id: 0eafb60c-f00e-4fd7-a921-7de75fcfe81e
+  - id: T32
+    title: Tag workflows with activity_id for Temporal visibility search
+    status: todo
+    state_hub_task_id: 7bdfc5c2-1f06-4fce-aac3-fae036dcb47e
+  - id: T33
+    title: Write operational runbook
+    status: todo
+    state_hub_task_id: 766d602d-1b23-4247-a46d-03c0d3b8e498
+---
+
+# activity-core Triggers & Ops
+
+**Workstream:** activity-core Triggers & Ops
+**Hub ID:** `3a4f47d9-8bc1-434e-acb4-bed5d4dacda0`
+**Depends on:** custodian-WP-0001 (Foundation — Temporal Backbone)
+
+## Purpose
+
+Add automated triggering (time-based via Temporal Schedules and event-driven via Event Router),
+a REST admin API, Prometheus metrics, and an operational runbook. Transforms the manually-triggered
+backbone from WP-0001 into a self-operating service.
+
+## Open decisions (resolve before Phase 5)
+
+- **Event broker choice** (hub: `bc47c9c2-5643-4a88-8114-601738a2f64e`): Kafka vs NATS vs RabbitMQ.
+  T26–T29 are blocked until this is resolved.
+
+---
+
+## Phase 4 — Time-Based Triggers (Temporal Schedules)
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T22: Write schedule_manager.py | medium | `e50550d1-9904-41d7-afd8-492a1f1e91b8` |
+| T23: Bootstrap script to sync schedules on startup | medium | `5a1f7fa3-acb9-4f60-9892-c9eaa120272e` |
+| T24: Handle misfire policy in schedule config | medium | `00231668-95c5-447f-b3d0-1fb8c20b487f` |
+| T25: Test schedule pause/resume lifecycle | medium | `7abfd375-ea9d-4209-8371-e5664dc2c6c4` |
+
+---
+
+## Phase 5 — Event-Driven Triggers
+
+*Blocked by broker decision (`bc47c9c2-5643-4a88-8114-601738a2f64e`).*
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T26: Implement Event Router service | medium | `68b6610b-159c-4f1c-92a9-7128efea0961` |
+| T27: Implement routing rules (event.type + filters → activity_ids) | medium | `9348efea-a7e9-4f92-b866-8fc82cf28fee` |
+| T28: Start/signal workflow from Event Router | medium | `cac1f45a-7391-471a-9566-97cdbd96eb2d` |
+| T29: Integration test — publish event → observe workflow run | medium | `7f10b5a3-7cad-4914-b603-d57508c85629` |
+
+---
+
+## Phase 6 — Observability & Admin
+
+| Task | Priority | Hub task ID |
+|---|---|---|
+| T30: REST API (FastAPI) — CRUD for ActivityDefinition | low | `b27e54a1-5dcc-476d-8f4a-c995aea6a8c2` |
+| T31: Wire Temporal SDK metrics to Prometheus | low | `0eafb60c-f00e-4fd7-a921-7de75fcfe81e` |
+| T32: Tag workflows with activity_id for Temporal visibility search | low | `7bdfc5c2-1f06-4fce-aac3-fae036dcb47e` |
+| T33: Write operational runbook | low | `766d602d-1b23-4247-a46d-03c0d3b8e498` |
+
+---
+
+## Completion criteria
+
+Schedules fire `RunActivityWorkflow` automatically on cron cadence. An external event published
+to the broker reaches the correct ActivityDefinition end-to-end. ActivityDefinitions are
+manageable via REST API. Prometheus metrics are scraped. Runbook is written.