generated from coulomb/repo-seed
tegwick
747afc27a6
Aligns the v1 architecture with the longer-horizon platform thesis so we can start implementation without the schema-level inconsistencies the prior review surfaced. ADRs (docs/adr/0001..0006): content-addressed dual-digest storage, append-only event log as source of truth, canonical CBOR manifests, control/data-plane contract, v1 tech stack (Python 3.12 / uv / FastAPI / SQLAlchemy Core + asyncpg / Alembic / cbor2 / blake3 / ruff / mypy / pytest / typer), OCI compatibility kept reachable. Architecture blueprint rewritten to v2: library-first (ffmpeg-shaped) module layout, materialised-view data model over the event log, upload-session and event-stream endpoints pinned, retrieval tiering promoted into the schema. Roadmap added (docs/ROADMAP.md) with three phases. WP-0001 rewritten as the Foundation plan (scaffold + kernels + local FS + minimal app). WP-0002..0005 created carrying the existing state_hub_task_ids forward semantically: ingestion API (T004), retention lifecycle (T005), S3-compatible backend (T006), guide-board pilot (T007). T001/T002/T003/T008 remain in WP-0001 with refined acceptance. README and AGENTS.md refreshed to reflect the new repo shape. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
151 lines
4.1 KiB
Markdown
151 lines
4.1 KiB
Markdown
---
|
|
id: ARTIFACT-STORE-WP-0002
|
|
type: workplan
|
|
title: "Ingestion API And Manifest Surface"
|
|
repo: artifact-store
|
|
domain: stack
|
|
status: planned
|
|
owner: codex
|
|
topic_slug: stack
|
|
planning_priority: high
|
|
planning_order: 2
|
|
created: "2026-05-15"
|
|
updated: "2026-05-15"
|
|
---
|
|
|
|
# ARTIFACT-STORE-WP-0002: Ingestion API And Manifest Surface
|
|
|
|
## Purpose
|
|
|
|
Expose the WP-0001 library as a complete HTTP API. Producers can create
|
|
packages, ingest files (single-shot or via the upload-session resource
|
|
shape), finalise to produce a manifest, list and search packages,
|
|
download files, and tail the event stream.
|
|
|
|
## Constraints
|
|
|
|
- ADR-0001, ADR-0002, ADR-0003, ADR-0004, ADR-0005, ADR-0006.
|
|
- `docs/ARCHITECTURE-BLUEPRINT.md` API shape section.
|
|
- All handlers must be thin: translate transport → `registry.*` calls.
|
|
|
|
## Prerequisites
|
|
|
|
- WP-0001 done (library is functional against local backend).
|
|
|
|
## D2.1 - Package CRUD Endpoints
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T001
|
|
status: todo
|
|
priority: high
|
|
state_hub_task_id: "e3879111-4be9-4731-8aea-15abb874f960"
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- `POST /packages`, `GET /packages` (filterable by producer / subject /
|
|
retention_class / metadata key), `GET /packages/{id}`,
|
|
`POST /packages/{id}/files` (single-shot multipart),
|
|
`POST /packages/{id}/finalize`.
|
|
- `GET /packages/{id}/manifest` (`Accept: application/cbor`) and
|
|
`GET /packages/{id}/manifest.json` (JCS projection).
|
|
- Validation errors return RFC 7807 problem documents.
|
|
- OpenAPI is generated automatically (FastAPI default) and served at
|
|
`/openapi.json` + `/docs`.
|
|
|
|
## D2.2 - File Download And Range Reads
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T002
|
|
status: todo
|
|
priority: high
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- `GET /files/{file_id}` returns metadata.
|
|
- `GET /files/{file_id}/download` streams bytes; supports `Range`
|
|
request headers (single contiguous range; multi-range is out of
|
|
scope for v1).
|
|
- ETag is the file's primary content address; `If-None-Match` returns
|
|
`304`.
|
|
- Streaming uses `AsyncIterator[bytes]` end-to-end; no full-file
|
|
buffering.
|
|
|
|
## D2.3 - Upload Session Resource (Wire Shape Pinned)
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T003
|
|
status: todo
|
|
priority: medium
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- `POST /uploads` opens a session, returns an upload id and content
|
|
upload URL.
|
|
- `PATCH /uploads/{upload_id}` accepts a body with `Content-Range`;
|
|
v1 implementation may accept the whole body in one call.
|
|
- `POST /uploads/{upload_id}/complete` promotes the upload into a
|
|
file under a given package id and relative path.
|
|
- Implementation is allowed to be single-shot internally; the wire
|
|
shape and resource lifecycle must be the final one (per
|
|
PLATFORM-AMBITION A6).
|
|
|
|
## D2.4 - Event Stream Long-Poll
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T004
|
|
status: todo
|
|
priority: medium
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- `GET /events?since=<sequence>&limit=N` returns events in order with
|
|
a long-poll wait when the tail is reached.
|
|
- Events are CBOR by default; `Accept: application/json` returns the
|
|
JCS projection of each event payload.
|
|
- Test: a consumer that tails from sequence 1 never misses an event
|
|
produced during the test.
|
|
|
|
## D2.5 - Auth Scaffolding (Shared-Secret Bearer)
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T005
|
|
status: todo
|
|
priority: medium
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- Bearer token auth on all mutating endpoints; configurable per-tenant
|
|
token list via env / config file.
|
|
- Read endpoints are also gated by default; an explicit
|
|
`ARTIFACTSTORE_ANON_READ=true` opt-in for dev.
|
|
- Health endpoint remains anonymous.
|
|
|
|
## D2.6 - Integration Tests Through The Full HTTP Surface
|
|
|
|
```task
|
|
id: ARTIFACT-STORE-WP-0002-T006
|
|
status: todo
|
|
priority: high
|
|
```
|
|
|
|
Acceptance:
|
|
|
|
- httpx-based test suite exercises every endpoint.
|
|
- A scripted test ingests a 50-file package, finalises it, downloads
|
|
every file, verifies digests, and tails events.
|
|
- A property-based test fuzzes the upload session lifecycle.
|
|
|
|
## Success criteria
|
|
|
|
- A producer can run the full ingest-and-retrieve flow against
|
|
`make dev` with curl.
|
|
- All blueprint endpoints in the v1 native surface are implemented.
|
|
- The CLI gains `artifactstore push <dir>` and
|
|
`artifactstore manifest <package_id>` subcommands as thin clients
|
|
over the HTTP API.
|