Commit Graph

6 Commits

Author SHA1 Message Date
91bb08c8e5 Add guide-board pilot ingestion 2026-05-17 00:09:11 +02:00
864f7f203c Add S3 backend and storage verification 2026-05-16 23:26:03 +02:00
c33baa3635 Implement HTTP ingestion and retention lifecycle 2026-05-16 23:10:21 +02:00
fe47058e1f WP-0001-T014: minimal HTTP app and CLI
src/artifactstore/app.py (new): composition root. build_registry(settings)
wires AsyncEngine + LocalBackend + InProcessDataPlane + RegistryViewWriter
into a Registry. Used by both the HTTP app and the CLI.

src/artifactstore/registry/__init__.py: adds db_health() (SELECT 1 probe),
backend_health() (pass-through to dataplane), and dispose() (engine
shutdown) helpers so the HTTP /health endpoint and CLI commands can talk
to the registry without reaching for private state.

src/artifactstore/api/http/__init__.py:
- create_app(settings=None) factory; lifespan owns the registry instance
  and disposes it on shutdown.
- GET / returns the scaffold banner.
- GET /health reports overall status + db {healthy, detail} + backend
  {backend_id, healthy, detail, free_bytes, total_bytes}. Uses
  FastAPI Depends() with a request->state.registry helper rather than
  reaching app.state directly.
- Module-level `app = create_app()` so `uvicorn artifactstore.api.http:app`
  keeps working.

src/artifactstore/cli/__init__.py:
- migrate: `alembic upgrade head` via the alembic command API.
- replay: drops + rebuilds materialised views from the event log; prints
  the highest applied sequence.
- health: prints the same payload as the HTTP /health endpoint, as JSON.
- version unchanged.

Tests:
- tests/integration/test_http_health.py (TestClient-based): /
  scaffold banner; /health reports ok with db.healthy + backend.healthy
  + free_bytes populated.
- tests/integration/test_cli_commands.py (typer CliRunner): version
  prints; migrate creates the schema (events + retention_classes +
  alembic_version); replay against an empty log exits ok with
  "replayed up to sequence 0"; health prints a status=ok JSON payload.

Gates: ruff clean, mypy --strict clean on 48 files, 83 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 08:56:13 +02:00
847b146475 WP-0001-T013: registry orchestrator (library surface)
src/artifactstore/registry/__init__.py implements the Registry class with
six operations the HTTP API and CLI both consume:

* create_package(name, producer, subject, retention_class, actor, metadata?)
  -> UUID. Validates retention_class against the seed table; emits
  v1.package.created with CBOR payload; applies view in same transaction.
* ingest_file(package_id, relative_path, media_type, stream, actor) -> UUID.
  Validates the package is in 'created' status and rejects duplicate
  relative_path. Calls dataplane.ingest_stream (which dual-hashes and writes
  to the backend). Emits v1.file.ingested whose payload carries the file
  metadata + storage receipt + deterministic storage_location_id so replay
  reproduces UUIDs. View handler in events/views.py inserts artifact_files
  + storage_locations and bumps last_event_sequence on the package.
* finalize_package(package_id, actor) -> ContentAddress. Queries the views
  to build a Manifest dataclass, encodes it as canonical CBOR, computes the
  BLAKE3 content address, and writes v1.package.finalized whose payload IS
  the canonical CBOR manifest. The view handler now records
  manifest_digest = event.payload_digest (BLAKE3 of the manifest), not a
  separate field parsed from the payload.
* get_manifest_bytes(package_id, format='cbor'|'json') -> bytes. Reads the
  finalize event payload (CBOR) and optionally projects to JCS.
* get_file(file_id) -> AsyncIterator[bytes]. Looks up the storage location
  and serves bytes via the data plane.
* tail_events(since_sequence, poll_interval_seconds) -> AsyncIterator[Event].
  Pass-through to events.tail.

src/artifactstore/events/views.py:
- New v1.file.ingested handler.
- v1.package.finalized handler updated: manifest_digest now derived from
  event.payload_digest (= BLAKE3 of the canonical CBOR manifest payload).
- All inserts now pass created_at=event.created_at explicitly so replay
  produces byte-identical materialised state (server_default=now() was
  firing fresh on each replay insert).

tests/integration/test_registry.py (7 cases):
- Rejects unknown retention class.
- create_package writes the event and the package row.
- ingest_file writes file + storage_location, populates content_address
  with blake3 prefix.
- Duplicate relative_path raises DuplicateRelativePathError.
- ingest into unknown package raises PackageNotFoundError.
- Finalising twice raises IllegalPackageStateError.
- End-to-end: create + ingest 3 files + finalize + read manifest in CBOR
  and JSON + download each file with byte equality + tail 5 events + replay
  + assert byte-identical materialised state across pre and post snapshots.

tests/integration/test_event_log.py updated: the v1.package.finalized
replay test now uses the new payload semantics (payload is the canonical
CBOR manifest; manifest_digest = BLAKE3 of payload).

Gates: ruff clean, mypy --strict clean on 45 files, 77 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 08:17:40 +02:00
a6b6746f91 WP-0001-T001: service scaffold (Python, FastAPI, uv, ruff, mypy, pytest)
Lands the smallest credible foundation per ADR-0005:

- pyproject.toml: hatchling build, runtime deps (FastAPI, uvicorn, SQLAlchemy 2.0,
  asyncpg, aiosqlite, alembic, blake3, cbor2, typer, structlog, pydantic,
  pydantic-settings); dev deps (pytest, pytest-asyncio, httpx, hypothesis, ruff,
  mypy); ruff + mypy --strict + pytest configured.
- uv.lock committed.
- Makefile thin shims: install / dev / test / lint / format / type / migrate / clean.
- src/artifactstore/ package skeleton with placeholder __init__.py per concern:
  identity, manifest, events, retention, audit, storage, dataplane, registry,
  api/http (minimal FastAPI app, GET / scaffold banner), cli (typer app with
  version subcommand), config (pydantic-settings).
- tests/{unit,integration}/conftest.py present; unit smoke tests assert package
  imports, HTTP root route, CLI version round-trip, settings defaults.
- .env.example documents ARTIFACTSTORE_DATABASE_URL,
  ARTIFACTSTORE_STORAGE_LOCAL_ROOT, ARTIFACTSTORE_LOG_LEVEL.
- README updated with install / dev / test instructions.
- .gitignore: claude local state, local runtime data (var/, sqlite db).

make lint && make type && make test pass on a clean checkout (4 tests, 20
source files type-clean under mypy --strict).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 01:30:22 +02:00