generated from coulomb/repo-seed
Add project scaffold: contracts, schemas, docker-compose, workplans
Phase 0 contracts (event envelope, ActivityDefinition, idempotency doc, naming conventions) and Phase 1 Temporal cluster setup (docker-compose.dev.yml, Temporal dynamic config) are complete. Includes Pydantic models, JSON schemas, wiki architecture docs, and ADR-001 workplan files for both workstreams. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
93
docs/conventions.md
Normal file
93
docs/conventions.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# Temporal Conventions
|
||||
|
||||
## Namespace
|
||||
|
||||
| Environment | Namespace |
|
||||
|---|---|
|
||||
| Dev (Docker Compose) | `default` |
|
||||
| Production | `activity-core-prod` |
|
||||
|
||||
Use the `default` namespace for all dev and test work. The production namespace is
|
||||
created via the Temporal admin API as part of the Kubernetes deployment (EP-custodian,
|
||||
extension point `af654abb`).
|
||||
|
||||
---
|
||||
|
||||
## Task Queues
|
||||
|
||||
| Queue name | Registered workers |
|
||||
|---|---|
|
||||
| `orchestrator-tq` | `RunActivityWorkflow` and all its activities (`load_activity_definition`, `resolve_context`, `log_run`) |
|
||||
| `task-execution-tq` | `TaskExecutorWorkflow` and all concrete task type workflows |
|
||||
|
||||
**Rule:** a workflow and its activities must be registered on the same task queue.
|
||||
Cross-queue activity calls require an explicit `task_queue` argument on
|
||||
`workflow.execute_activity()`.
|
||||
|
||||
---
|
||||
|
||||
## Workflow ID conventions
|
||||
|
||||
See `docs/idempotency.md` for the full workflow ID strategy.
|
||||
|
||||
Summary:
|
||||
- `RunActivityWorkflow`: `activity-{activity_id}:{trigger_key}`
|
||||
- `TaskExecutorWorkflow`: `task-{run_id}:{task_type}:{index}`
|
||||
- Temporal Schedules: `activity-schedule-{activity_id}`
|
||||
|
||||
---
|
||||
|
||||
## Schedule ID conventions
|
||||
|
||||
Temporal Schedules are identified by `schedule_id`. The convention:
|
||||
|
||||
```
|
||||
activity-schedule-{activity_definition.id}
|
||||
```
|
||||
|
||||
This makes it trivial to look up the schedule for a given ActivityDefinition
|
||||
without a separate mapping table.
|
||||
|
||||
---
|
||||
|
||||
## Worker registration
|
||||
|
||||
Each worker process registers:
|
||||
- **Workflows**: `worker.register_workflow(WorkflowClass)`
|
||||
- **Activities**: `worker.register_activity(activity_function)`
|
||||
|
||||
A single process may run workers for multiple task queues, but each `Worker`
|
||||
instance is bound to one task queue. Use separate `Worker` instances for
|
||||
`orchestrator-tq` and `task-execution-tq`.
|
||||
|
||||
---
|
||||
|
||||
## Search attributes
|
||||
|
||||
`RunActivityWorkflow` sets the following search attributes (requires Elasticsearch
|
||||
visibility, enabled in the dev docker-compose):
|
||||
|
||||
| Attribute | Type | Value |
|
||||
|---|---|---|
|
||||
| `ActivityId` | `Keyword` | The `ActivityDefinition.id` UUID |
|
||||
| `ActivityName` | `Keyword` | The `ActivityDefinition.name` |
|
||||
|
||||
These allow filtering workflow runs by activity in the Temporal UI.
|
||||
|
||||
---
|
||||
|
||||
## Retry policy defaults
|
||||
|
||||
Unless overridden, all activities use:
|
||||
|
||||
```python
|
||||
RetryPolicy(
|
||||
initial_interval=timedelta(seconds=1),
|
||||
backoff_coefficient=2.0,
|
||||
maximum_interval=timedelta(minutes=5),
|
||||
maximum_attempts=10,
|
||||
)
|
||||
```
|
||||
|
||||
Long-running context-resolution activities that call external services should set
|
||||
`heartbeat_timeout` to detect stalls.
|
||||
72
docs/idempotency.md
Normal file
72
docs/idempotency.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Idempotency Contract
|
||||
|
||||
## Workflow ID strategy
|
||||
|
||||
Every `RunActivityWorkflow` execution has a deterministic, stable workflow ID.
|
||||
This is the primary idempotency mechanism — Temporal rejects duplicate starts
|
||||
with the same workflow ID in the same namespace.
|
||||
|
||||
| Trigger type | Workflow ID format |
|
||||
|---|---|
|
||||
| Cron / scheduled | `activity-{activity_id}:{scheduled_for.isoformat()}` |
|
||||
| External event | `activity-{activity_id}:{event.event_id}` |
|
||||
|
||||
**Example (cron):** `activity-550e8400-e29b-41d4-a716-446655440000:2026-03-01T09:00:00+00:00`
|
||||
**Example (event):** `activity-550e8400-e29b-41d4-a716-446655440000:evt_abc123`
|
||||
|
||||
Both formats guarantee that re-delivering a trigger (broker retry, duplicate schedule fire)
|
||||
produces at most one workflow execution.
|
||||
|
||||
---
|
||||
|
||||
## Misfire policy
|
||||
|
||||
A *misfire* occurs when a scheduled trigger fires after its nominal time because the
|
||||
worker was down, overloaded, or the schedule was paused.
|
||||
|
||||
| Policy | `dedupe_key_strategy` value | Behaviour |
|
||||
|---|---|---|
|
||||
| Skip | `"skip"` | Missed runs are discarded. Temporal `ScheduleOverlapPolicy.SKIP`. |
|
||||
| Catch up | `"catchup"` | Missed runs are replayed up to 10 times (configurable). Uses `schedule.handle.backfill()`. |
|
||||
| Compress | `"compress"` | One run is executed with a widened context window covering all missed intervals. |
|
||||
|
||||
The `dedupe_key_strategy` on `ActivityDefinition` must match the `misfire_policy` on
|
||||
`CronTriggerConfig`. They are separate fields to allow the schedule configuration
|
||||
(how Temporal fires) to be decoupled from the workflow logic (how missed runs are handled).
|
||||
|
||||
---
|
||||
|
||||
## Event deduplication
|
||||
|
||||
For event-driven activities:
|
||||
|
||||
1. The Event Router computes the workflow ID as `activity-{activity_id}:{event.event_id}`.
|
||||
2. It calls `client.start_workflow(..., id=workflow_id, id_reuse_policy=REJECT_DUPLICATE)`.
|
||||
3. If the workflow ID already exists (event was already processed), Temporal returns
|
||||
`WorkflowAlreadyStartedError` — the router logs it and moves on.
|
||||
|
||||
**Prerequisite:** every inbound event must have a stable `event_id`. Events without a
|
||||
stable ID must be assigned one by the ingress boundary before entering the system.
|
||||
|
||||
---
|
||||
|
||||
## Task instance idempotency
|
||||
|
||||
Each `TaskInstance` spawned by `RunActivityWorkflow` gets its own unique workflow ID:
|
||||
|
||||
```
|
||||
task-{run_id}:{task_type}:{index}
|
||||
```
|
||||
|
||||
This ensures that if `RunActivityWorkflow` is replayed by Temporal (e.g. after a worker
|
||||
restart), it does not re-spawn task instances that were already started.
|
||||
|
||||
---
|
||||
|
||||
## Database idempotency
|
||||
|
||||
`activity_runs` uses `run_id` as the primary key (UUID). The `log_run` activity
|
||||
uses an upsert (`INSERT ... ON CONFLICT DO NOTHING`) so that Temporal activity retries
|
||||
do not produce duplicate run records.
|
||||
|
||||
`task_instances` similarly uses an upsert on `id`.
|
||||
Reference in New Issue
Block a user