generated from coulomb/repo-seed
REUSE-WP-0019-T02: hub recompose staleness tracking + Forgejo webhook
Some checks failed
ci / validate-registry (push) Has been cancelled
Some checks failed
ci / validate-registry (push) Has been cancelled
reuse_surface/hub/store.py: compose_state table tracking composed_at/stale, updated only on a *forced* recompose (refresh=true, webhook, future scheduled fallback) -- a plain GET still serves current best-effort data but never silently reports itself as freshly composed. reuse_surface/hub/webhooks.py: constant-time HMAC-SHA256 signature verification (fails closed on an empty/unconfigured secret) and path-only push-payload inspection (never parses file content, per design principle 2 -- webhook only decides whether to trigger a pull-based recompose). New endpoint POST /v1/webhooks/forgejo, accepting both X-Forgejo-Signature and X-Gitea-Signature headers since sibling repos migrate independently. GET /v1/federated and POST /v1/federated/compose now share an asyncio.Lock with the webhook so concurrent recompose triggers coalesce instead of overlapping. No separate /v1/recompose route was added -- POST /v1/federated/compose already did that job from earlier hub work; specs/FederationHubAPI.md now documents this explicitly instead of duplicating it. 28 new pytest cases (128 total pass). Live-verified: ran the actual hub service locally and sent a real HMAC-signed webhook over HTTP, confirming the full webhook-to-recompose path, signature rejection, and the irrelevant-path no-op. Does NOT deploy to the live reuse.coulomb.social hub -- that needs a container rebuild/push/k8s rollout, a separate production-deployment action out of scope here without explicit sign-off. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -195,18 +195,30 @@ capabilities:
|
||||
source_repo: reuse-surface
|
||||
source_url: https://...
|
||||
# ... index fields ...
|
||||
composed_at: "2026-07-07T16:22:09+00:00" # REUSE-WP-0019-T02
|
||||
stale: false # REUSE-WP-0019-T02
|
||||
```
|
||||
|
||||
`composed_at`/`stale` (added REUSE-WP-0019-T02) track *forced* recomposes
|
||||
only (`refresh=true`, the webhook, or the scheduled fallback) — a plain
|
||||
`GET` still serves current best-effort data (per-source `cache_ttl_seconds`
|
||||
still applies) but never silently clears a staleness signal nothing
|
||||
actually refreshed. `composed_at` is `null` until the first forced
|
||||
recompose since the hub process's SQLite DB was created. `stale: true`
|
||||
means a `registry/indexes/` change was pushed (via webhook) since the last
|
||||
forced recompose completed.
|
||||
|
||||
Query parameters:
|
||||
|
||||
| Param | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `format` | `json` | `json` or `yaml` |
|
||||
| `refresh` | `false` | Bypass remote cache when `true` |
|
||||
| `refresh` | `false` | Bypass remote cache when `true`; also updates `composed_at` and clears `stale` |
|
||||
|
||||
Warnings from compose (duplicate IDs, fetch fallbacks) are returned in response
|
||||
header `X-Federation-Warnings` (semicolon-separated) for MVP; JSON envelope
|
||||
extension is a future option.
|
||||
extension is a future option. `composed_at` is also echoed as response header
|
||||
`X-Composed-At` when set.
|
||||
|
||||
**Response `200`:** Federated index document.
|
||||
**Response `502`:** Required source unavailable with no cache.
|
||||
@@ -215,8 +227,50 @@ extension is a future option.
|
||||
|
||||
Trigger federated index refresh (same as `GET /v1/federated?refresh=true`).
|
||||
**Auth required.** Useful for operators after bulk registration changes.
|
||||
This is the hub's recompose endpoint referenced elsewhere as "trigger a
|
||||
recompose" — there is no separate `/v1/recompose` route; this one already
|
||||
does that job.
|
||||
|
||||
**Response `200`:** Federated index document.
|
||||
**Response `200`:** Federated index document (with `composed_at` updated,
|
||||
`stale` cleared).
|
||||
|
||||
### 5.9 `POST /v1/webhooks/forgejo`
|
||||
|
||||
Forgejo (or Gitea, during the transition) push-event webhook receiver.
|
||||
Added REUSE-WP-0019-T02. Design: **webhook triggers, compose stays
|
||||
pull-based** — the webhook never parses pushed file content into registry
|
||||
state; it only decides whether to trigger a pull-based recompose from the
|
||||
already-registered raw URLs.
|
||||
|
||||
**Signature verification (required, not optional):** HMAC-SHA256 over the
|
||||
raw request body, hex-encoded, in `X-Forgejo-Signature` (or
|
||||
`X-Gitea-Signature` — both accepted, since Forgejo is Gitea-compatible and
|
||||
repos migrate independently). Secret from `REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET`
|
||||
(never in code or committed config — route via the standard credential
|
||||
mechanism for this deployment). A missing/wrong signature is `401`; an
|
||||
unconfigured secret is `503` (fails closed, not open).
|
||||
|
||||
**Behavior:**
|
||||
|
||||
1. Verify signature (raw body, constant-time compare).
|
||||
2. Parse the push-event payload; check every commit's `added`/`modified`/`removed`
|
||||
lists for any path under `registry/indexes/`.
|
||||
3. If none match: `200 {"accepted": false, "reason": "no registry/indexes/ change"}` — no-op.
|
||||
4. If a match: mark the compose state stale, run a real recompose
|
||||
(`refresh=true` equivalent) under the same lock used by
|
||||
`POST /v1/federated/compose` (concurrent triggers coalesce rather than
|
||||
overlap), record `composed_at`, clear `stale`.
|
||||
|
||||
**Response `200`:** `{"accepted": true|false, ...}`.
|
||||
**Response `401`:** Missing or invalid signature.
|
||||
**Response `503`:** Webhook secret not configured.
|
||||
**Response `502`:** Recompose failed (stale is deliberately left set so the
|
||||
next check reports the failure honestly, not silently).
|
||||
|
||||
Org-level webhook rollout (single config covering all repos, T03) and the
|
||||
Forgejo Actions scheduled fallback for when webhooks are unavailable are
|
||||
separate, deployment-side follow-ups — this section covers the receiver
|
||||
contract only.
|
||||
|
||||
---
|
||||
|
||||
@@ -238,6 +292,7 @@ Non-2xx responses use:
|
||||
| `unauthorized` | 401 |
|
||||
| `not_found` | 404 |
|
||||
| `conflict` | 409 |
|
||||
| `misconfigured` | 503 |
|
||||
| `compose_error` | 502 |
|
||||
|
||||
---
|
||||
@@ -250,6 +305,8 @@ Non-2xx responses use:
|
||||
| `REUSE_SURFACE_DB` | no | SQLite path (default `/data/reuse.db`) |
|
||||
| `REUSE_SURFACE_CACHE_DIR` | no | Remote index cache (default `/data/cache`) |
|
||||
| `REUSE_SURFACE_DOMAIN` | no | Default federated `domain` (default `helix_forge`) |
|
||||
| `REUSE_SURFACE_FORGEJO_WEBHOOK_SECRET` | for webhook | HMAC secret for `POST /v1/webhooks/forgejo` (REUSE-WP-0019-T02) |
|
||||
| `REUSE_SURFACE_FORGE_BASE_URL` | no | Default target host for `reuse-surface federation migrate-host` (REUSE-WP-0019-T01), e.g. `https://forgejo.coulomb.social` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user