Load limiting safeguards

This commit is contained in:
2026-05-06 04:04:53 +02:00
parent 47f6971c56
commit 2484ed2815
22 changed files with 374 additions and 144 deletions

View File

@@ -17,9 +17,19 @@ The compose file is `infra/docker-compose.yml`. Copy `.env.example` to `.env` an
## Periodic Repo Sync — systemd user timer
The custodian sync timer runs `fix-consistency-all` every 15 minutes, keeping
workplan file state in sync with the state-hub DB automatically (belt-and-suspenders
alongside the per-repo git post-commit hooks).
The custodian sync timer runs `consistency_check.py --remote --all` every 15
minutes, keeping workplan file state in sync with the state-hub DB automatically
(belt-and-suspenders alongside the per-repo git post-commit hooks).
The all-repo remote sweep has two built-in load guards:
- A nonblocking process lock at `/tmp/custodian-consistency-remote-all.lock`;
if a prior sweep is still active, the next timer run exits cleanly.
- A wall-clock budget, defaulting to 300 seconds. Remaining repos are skipped
once the budget is exhausted. Override with `--max-seconds N` or set
`CONSISTENCY_REMOTE_ALL_MAX_SECONDS`.
- Warn-only sweeps exit 0 in `--remote --all` mode so the systemd unit only
goes failed for hard consistency failures.
### Installed unit files
@@ -64,7 +74,7 @@ If systemd is not available, fall back to crontab:
```bash
# Crontab fallback (run crontab -e and add):
*/15 * * * * curl -sf http://127.0.0.1:8000/state/health && cd ~/the-custodian/state-hub && .venv/bin/python scripts/consistency_check.py --all --fix >> /tmp/custodian-sync.log 2>&1
*/15 * * * * curl -sf http://127.0.0.1:8000/state/health && cd ~/the-custodian/state-hub && .venv/bin/python scripts/consistency_check.py --remote --all >> /tmp/custodian-sync.log 2>&1
```
---