--- id: ACTIVITY-WP-0018 type: workplan title: "Own-infrastructure automation status surface" domain: infotech repo: activity-core status: finished owner: codex topic_slug: automation-observability created: "2026-06-29" updated: "2026-06-29" state_hub_workstream_id: "0220b38b-7c73-4601-9601-5f2c1a5b29e8" --- # Own-infrastructure automation status surface ## Goal Make activity-core's own scheduling and evidence infrastructure the explicit operating preference for durable automations, independent of any coding assistant-provided scheduler or reminder system. An operator should be able to answer a question like "How did our automations go since Friday?" with a repo-native command that does not require an LLM. Coding assistants may inspect or summarize that command's output, but they must not be the source of truth for scheduled execution, run history, or operational evidence. ## Review notes The repo already owns the correct infrastructure direction: - `SCOPE.md` defines activity-core as the org-wide event bridge for cron, one-off scheduled datetime, and event-triggered automation. - `Makefile` exposes sync and service targets, but no operator status target for recent automation outcomes. - `docs/runbook.md` documents daily-triage verification through `scripts/verify_daily_triage.py`, but that helper is activity-specific and still reads like a checklist rather than the baseline answer surface for all automations. - Existing workplan evidence shows the status question is operationally common: 2026-06-24 and 2026-06-25 daily triage runs were clean, while 2026-06-26 and 2026-06-27 fired on schedule but failed output validation. That distinction is exactly what the baseline command must make obvious. ## Task: Codify the own-infra scheduling preference ```task id: ACTIVITY-WP-0018-T01 status: done priority: high state_hub_task_id: "00127678-5ce4-4cb3-b81c-f42e04407c73" ``` Record the repository preference that durable automation scheduling, execution history, and run evidence belong to activity-core's own infrastructure: Temporal Schedules, NATS JetStream, activity-core run records, State Hub progress, and working-memory/report sinks. Acceptance: - `AGENTS.md` repo-specific instructions say not to use coding assistant-provided automation tooling as the execution or evidence source for activity-core automations. - `SCOPE.md` and `docs/runbook.md` describe coding assistants as callers or summarizers of repo-native automation commands, not as schedulers. - The preference distinguishes durable automation from harmless local session reminders: production/operational recurrence belongs to activity-core. - The text names the authoritative evidence sources and avoids tying the policy to any one assistant product. 2026-06-29 progress: Added the immediate repo-agent instruction in AGENTS.md that durable activity-core automations must use repo-owned infrastructure, not coding assistant automation/reminder/heartbeat tooling, as the execution or evidence source. Remaining T01 work is to carry the same preference into SCOPE.md and docs/runbook.md. ## Task: Define the automation status evidence contract ```task id: ACTIVITY-WP-0018-T02 status: done priority: high state_hub_task_id: "17e6bb87-d4bf-4ef3-b91c-4bdfe2fe3492" ``` Define a small, deterministic report contract for answering recent automation status questions across all ActivityDefinitions. Acceptance: - The contract covers schedule state, expected fires in the requested window, observed workflow runs, `activity_runs` rows, State Hub progress events, working-memory/report sink evidence, and known validation or sink failures. - It defines normalized statuses such as `completed`, `running`, `retrying`, `validation_failed`, `sink_failed`, `missed`, `disabled`, and `unknown`. - Partial data is explicit: if Temporal, Postgres, State Hub, or a sink path is unavailable, the report includes warnings rather than silently passing or failing the whole check. - The contract is safe for operator logs: no secrets, prompts, raw model output, or credential-bearing URLs. - The contract can be emitted as JSON for scripts and rendered as concise text for humans. ## Task: Implement the non-LLM automation status CLI ```task id: ACTIVITY-WP-0018-T03 status: done priority: high state_hub_task_id: "7831f2fc-8b76-48fe-aa34-9dcc11ee84db" ``` Add a deterministic CLI, likely under `scripts/automation_status.py` or an `activity_core` module, that answers recent automation status questions without calling an LLM. Acceptance: - Supports `--since`, `--until`, activity name/id filters, JSON output, and a concise human summary. - Accepts simple operator dates, including absolute dates and a documented `friday`/`last-friday` style shortcut, resolving them to concrete dates in the configured timezone. - Inspects all enabled scheduled ActivityDefinitions by default, not just daily triage. - Uses live sources when configured: Postgres `activity_definitions` / `activity_runs`, Temporal schedule and workflow visibility, State Hub progress, and configured local report sink paths. - Degrades usefully when a source is unavailable and exits non-zero only for real status failures or invalid input, not for optional evidence gaps that are clearly reported. - Includes focused unit tests with fixture data for clean runs, validation failures, missed runs, disabled schedules, and partial-source availability. ## Task: Add the Make target baseline ```task id: ACTIVITY-WP-0018-T04 status: done priority: high state_hub_task_id: "451bdf62-b619-4ace-9262-46d20b912781" ``` Expose the CLI through a Make target that is easy for an operator or any coding assistant to run before attempting a prose summary. Acceptance: - `make automation-status SINCE=2026-06-26` prints the human-readable baseline. - `make automation-status SINCE=friday` is supported or documented with the exact accepted shortcut. - A JSON form is available, either through `FORMAT=json` or a separate target such as `make automation-status-json`. - The target does not require LLM credentials, coding assistant automation tooling, or interactive prompts. - `make help` lists the target with a clear one-line description. ## Task: Update operator docs and examples ```task id: ACTIVITY-WP-0018-T05 status: done priority: medium state_hub_task_id: "233659aa-e14a-4b3d-b156-d04f0fa16db6" ``` Update the runbook so "How did automations go since Friday?" has an obvious operator recipe. Acceptance: - `docs/runbook.md` has a short "Automation status" section near the scheduling operations. - The docs include example output or a compact sample for the known daily triage distinction: fired on time versus completed successfully versus output validation failure. - The docs clarify that LLM summaries are optional convenience only; the Make target output is the baseline evidence. - The daily-triage-specific helper is either kept as a lower-level diagnostic or folded into the generalized status command. ## Task: Verify against recent scheduled-run evidence ```task id: ACTIVITY-WP-0018-T06 status: done priority: medium state_hub_task_id: "24efbe9f-dfff-482f-9edc-456379c9a2aa" ``` Prove the new surface against the recent evidence that motivated this workplan. Acceptance: - Running the status command over the window starting Friday, 2026-06-26 shows that the daily triage schedule fired on 2026-06-26 and 2026-06-27 but did not produce clean validated reports. - The command distinguishes scheduling health from output/schema validation failure. - Disabled or waiting schedules, such as the weekly coding retro gate when its upstream read model is not available, are reported without being counted as missed runs. - Verification results are recorded in this workplan and as a State Hub progress note once the implementation lands. ## Implementation Result Completed 2026-06-29: implemented the own-infrastructure automation status surface and codified the scheduling preference. Delivered: - `AGENTS.md` now states that durable activity-core automations use repo-owned infrastructure, not coding assistant automation/reminder/heartbeat tooling, as execution or evidence authority. - `SCOPE.md` and `docs/runbook.md` describe the deterministic status surface and assistant boundary. - `src/activity_core/automation_status.py` and `scripts/automation_status.py` provide the non-LLM CLI. - `make automation-status SINCE=...` and `make automation-status-json` expose the baseline operator commands. - `tests/test_automation_status.py` covers date shortcuts, cron fire estimation, completed runs, validation failures, missed runs, disabled schedules, partial source availability, and working-memory evidence parsing. Verification: ```bash python3 -m py_compile src/activity_core/automation_status.py scripts/automation_status.py tests/test_automation_status.py /home/worsch/.local/bin/uv run pytest tests/test_automation_status.py tests/test_daily_triage_verifier.py -q /home/worsch/.local/bin/uv run python scripts/automation_status.py \ --since 2026-06-26 --until 2026-06-27 --db-url '' \ --progress-event-type daily_triage --timeout-seconds 10 \ --working-memory-dir /tmp --format json ``` Results: - focused tests: `11 passed`; - `make help` lists `automation-status` and `automation-status-json`; - the 2026-06-26 through 2026-06-27 status run exited `1` as expected because State Hub evidence classified daily triage activity `6fca51fa-387a-4fd0-bc4e-d62c29eb859a` as `validation_failed` with two non-secret evidence records: 2026-06-26 `Expecting ',' delimiter` and 2026-06-27 `Unterminated string`; - the same report classified the gated weekly coding retro as `disabled`, not `missed`.