generated from coulomb/repo-seed
Compare commits
2 Commits
f59213f512
...
f90c761ef6
| Author | SHA1 | Date | |
|---|---|---|---|
| f90c761ef6 | |||
| fe47058e1f |
23
AGENTS.md
23
AGENTS.md
@@ -162,22 +162,33 @@ To create a new workplan:
|
|||||||
|
|
||||||
## Current Repo Shape
|
## Current Repo Shape
|
||||||
|
|
||||||
This repository is in service-baseline planning. The current sources of truth are:
|
v0.1 baseline (WP-0001) is live: library, CLI, minimal HTTP app, local FS
|
||||||
|
backend, end-to-end ingest + finalize + replay. The pinned tech stack is in
|
||||||
|
[ADR-0005](docs/adr/0005-v1-tech-stack.md).
|
||||||
|
|
||||||
|
Sources of truth:
|
||||||
|
|
||||||
- `INTENT.md` — purpose, product thesis, scope, service boundary.
|
- `INTENT.md` — purpose, product thesis, scope, service boundary.
|
||||||
- `SCOPE.md` — lightweight orientation.
|
- `SCOPE.md` — lightweight orientation.
|
||||||
|
- `docs/OPERATOR.md` — runbook: env vars, DB backends, CLI / HTTP reference, smoke test, replay procedure.
|
||||||
- `docs/ARCHITECTURE-BLUEPRINT.md` — architecture v2: modules, data model, API shape.
|
- `docs/ARCHITECTURE-BLUEPRINT.md` — architecture v2: modules, data model, API shape.
|
||||||
- `docs/PLATFORM-AMBITION.md` — longer-horizon thesis and the v1 schema commitments (A1–A9).
|
- `docs/PLATFORM-AMBITION.md` — longer-horizon thesis and the v1 schema commitments (A1–A9).
|
||||||
- `docs/adr/` — architecture decision records ADR-0001 … ADR-0006 (content-addressed storage, event log as source of truth, canonical CBOR manifests, control/data plane contract, v1 tech stack, OCI reachability).
|
- `docs/adr/` — architecture decision records ADR-0001 … ADR-0006 (content-addressed storage, event log as source of truth, canonical CBOR manifests, control/data plane contract, v1 tech stack, OCI reachability).
|
||||||
- `docs/ROADMAP.md` — workplan sequencing across phases.
|
- `docs/ROADMAP.md` — workplan sequencing across phases.
|
||||||
- `docs/ASSEMBLY-EXPERIMENT.md` — opt-in research line on hand-tuned asm for hot kernels.
|
- `docs/ASSEMBLY-EXPERIMENT.md` — opt-in research line on hand-tuned asm for hot kernels.
|
||||||
- `workplans/ARTIFACT-STORE-WP-0001-service-baseline.md` — Foundation workplan; first to start.
|
- `workplans/ARTIFACT-STORE-WP-0001-service-baseline.md` — Foundation workplan (done).
|
||||||
- `workplans/ARTIFACT-STORE-WP-{0002..0005}-*.md` — planned next workplans.
|
- `workplans/ARTIFACT-STORE-WP-{0002..0005}-*.md` — planned next workplans.
|
||||||
|
|
||||||
No runnable service scaffold exists yet. The pinned tech stack is in ADR-0005
|
Local commands:
|
||||||
(Python 3.12, uv, FastAPI, SQLAlchemy Core + asyncpg/aiosqlite, Alembic,
|
|
||||||
cbor2, blake3, ruff, mypy, pytest, typer). Add install, dev-server, and test
|
```sh
|
||||||
commands here when `ARTIFACT-STORE-WP-0001-T001` lands.
|
make install # uv sync --all-extras
|
||||||
|
make migrate-fresh # drop + re-create the dev SQLite DB
|
||||||
|
make dev # uvicorn on 127.0.0.1:8000
|
||||||
|
make test # pytest
|
||||||
|
make lint type # ruff + mypy --strict
|
||||||
|
artifactstore health
|
||||||
|
```
|
||||||
|
|
||||||
## Repo Boundary
|
## Repo Boundary
|
||||||
|
|
||||||
|
|||||||
65
README.md
65
README.md
@@ -6,53 +6,78 @@ artifacts.
|
|||||||
|
|
||||||
The registry owns artifact identity, metadata, provenance, retention
|
The registry owns artifact identity, metadata, provenance, retention
|
||||||
policy, and retrieval records. Bytes are delegated to configured storage
|
policy, and retrieval records. Bytes are delegated to configured storage
|
||||||
backends (local filesystem in v1, S3-compatible / Ceph RGW next).
|
backends (local filesystem in v1, S3-compatible / Ceph RGW in WP-0004).
|
||||||
|
|
||||||
The shape is library-first (`artifactstore` Python package); the HTTP
|
The shape is library-first (`artifactstore` Python package); the HTTP
|
||||||
server and the CLI are thin consumers. Content is addressed by digest;
|
server and the CLI are thin consumers. Content is addressed by digest;
|
||||||
state is authoritative in an append-only event log; materialised views
|
state is authoritative in an append-only event log
|
||||||
are rebuildable.
|
([ADR-0002](docs/adr/0002-event-log-source-of-truth.md)); materialised
|
||||||
|
views are rebuildable.
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
|
|
||||||
Scaffold landed. The core kernels and local FS backend follow in the
|
**v0.1 (WP-0001 baseline)** — library, CLI, minimal HTTP app, local FS
|
||||||
remaining tasks of [WP-0001](workplans/ARTIFACT-STORE-WP-0001-service-baseline.md).
|
backend, end-to-end ingest + finalize + replay all working. The full
|
||||||
|
package CRUD HTTP API lands in WP-0002.
|
||||||
|
|
||||||
## Develop
|
## Quick start
|
||||||
|
|
||||||
Requires Python ≥ 3.12 and [`uv`](https://docs.astral.sh/uv/) on the path.
|
Requires Python ≥ 3.12 and [`uv`](https://docs.astral.sh/uv/).
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
make install # uv sync --all-extras
|
uv sync --all-extras # install dependencies (creates .venv + uv.lock)
|
||||||
cp .env.example .env
|
cp .env.example .env # optional; defaults work out of the box
|
||||||
|
|
||||||
make dev # uvicorn artifactstore.api.http:app --reload
|
make migrate-fresh # initialise ./var/artifactstore.db
|
||||||
make test # pytest
|
make dev # uvicorn on 127.0.0.1:8000
|
||||||
make lint # ruff check + ruff format --check
|
|
||||||
make type # mypy --strict
|
# in another terminal
|
||||||
make migrate # alembic upgrade head (configured in WP-0001-T002)
|
curl -s http://127.0.0.1:8000/health | python3 -m json.tool
|
||||||
|
artifactstore health
|
||||||
```
|
```
|
||||||
|
|
||||||
The dev server listens on `127.0.0.1:8000`. The scaffold root route
|
## Make targets
|
||||||
returns `{"service": "artifact-store", "status": "scaffold"}`; the real
|
|
||||||
`/health` endpoint lands in `WP-0001-T014`.
|
| Target | Purpose |
|
||||||
|
|----------------------|---------|
|
||||||
|
| `make install` | `uv sync --all-extras` |
|
||||||
|
| `make dev` | run the FastAPI app with reload |
|
||||||
|
| `make test` | run the pytest suite |
|
||||||
|
| `make lint` | ruff check + ruff format --check |
|
||||||
|
| `make format` | apply ruff format + ruff check --fix |
|
||||||
|
| `make type` | mypy --strict |
|
||||||
|
| `make migrate` | `alembic upgrade head` |
|
||||||
|
| `make migrate-fresh` | drop the local SQLite DB and re-run migrations |
|
||||||
|
| `make clean` | remove caches and build artefacts |
|
||||||
|
|
||||||
|
## CLI
|
||||||
|
|
||||||
|
```sh
|
||||||
|
artifactstore version # print the package version
|
||||||
|
artifactstore migrate # alembic upgrade head
|
||||||
|
artifactstore replay # rebuild materialised views from the event log
|
||||||
|
artifactstore health # JSON liveness summary (same payload as /health)
|
||||||
|
```
|
||||||
|
|
||||||
## Documentation
|
## Documentation
|
||||||
|
|
||||||
- [INTENT.md](INTENT.md) — purpose, product thesis, scope, boundary.
|
- [INTENT.md](INTENT.md) — purpose, product thesis, scope, boundary.
|
||||||
- [SCOPE.md](SCOPE.md) — lightweight orientation.
|
- [SCOPE.md](SCOPE.md) — lightweight orientation.
|
||||||
|
- [docs/OPERATOR.md](docs/OPERATOR.md) — runbook: env vars, DB backends,
|
||||||
|
CLI / HTTP reference, end-to-end smoke test, replay procedure.
|
||||||
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md) — v2
|
- [docs/ARCHITECTURE-BLUEPRINT.md](docs/ARCHITECTURE-BLUEPRINT.md) — v2
|
||||||
architecture: modules, data model, API shape.
|
architecture: modules, data model, API shape.
|
||||||
- [docs/PLATFORM-AMBITION.md](docs/PLATFORM-AMBITION.md) — longer-horizon
|
- [docs/PLATFORM-AMBITION.md](docs/PLATFORM-AMBITION.md) — longer-horizon
|
||||||
thesis and v1 schema commitments.
|
thesis and v1 schema commitments.
|
||||||
- [docs/ROADMAP.md](docs/ROADMAP.md) — workplan sequencing across phases.
|
- [docs/ROADMAP.md](docs/ROADMAP.md) — workplan sequencing across phases.
|
||||||
- [docs/adr/](docs/adr/) — architecture decision records.
|
- [docs/adr/](docs/adr/) — architecture decision records (ADR-0001 …
|
||||||
|
ADR-0006).
|
||||||
- [docs/ASSEMBLY-EXPERIMENT.md](docs/ASSEMBLY-EXPERIMENT.md) — opt-in
|
- [docs/ASSEMBLY-EXPERIMENT.md](docs/ASSEMBLY-EXPERIMENT.md) — opt-in
|
||||||
research line on hand-tuned asm for hot kernels.
|
research line on hand-tuned asm for hot kernels.
|
||||||
|
|
||||||
## Active workplans
|
## Workplans
|
||||||
|
|
||||||
- [WP-0001 — Foundation: scaffold, core kernels, local FS backend](workplans/ARTIFACT-STORE-WP-0001-service-baseline.md)
|
- [WP-0001 — Foundation: scaffold, core kernels, local FS backend](workplans/ARTIFACT-STORE-WP-0001-service-baseline.md) — **done**
|
||||||
- [WP-0002 — Ingestion API and manifest surface](workplans/ARTIFACT-STORE-WP-0002-ingestion-api.md) (planned)
|
- [WP-0002 — Ingestion API and manifest surface](workplans/ARTIFACT-STORE-WP-0002-ingestion-api.md) (planned)
|
||||||
- [WP-0003 — Retention lifecycle](workplans/ARTIFACT-STORE-WP-0003-retention-lifecycle.md) (planned)
|
- [WP-0003 — Retention lifecycle](workplans/ARTIFACT-STORE-WP-0003-retention-lifecycle.md) (planned)
|
||||||
- [WP-0004 — S3-compatible backend](workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md) (planned)
|
- [WP-0004 — S3-compatible backend](workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md) (planned)
|
||||||
|
|||||||
223
docs/OPERATOR.md
Normal file
223
docs/OPERATOR.md
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
# Operator Guide
|
||||||
|
|
||||||
|
Status: v0.1 (WP-0001 baseline)
|
||||||
|
Updated: 2026-05-16
|
||||||
|
|
||||||
|
This guide is the user manual for running `artifact-store` v0.1 — the
|
||||||
|
library + CLI + minimal HTTP app that landed in WP-0001. Ingest, finalize,
|
||||||
|
and retrieve workflows go through the Python library today; the HTTP
|
||||||
|
upload API arrives in WP-0002.
|
||||||
|
|
||||||
|
For architectural background see
|
||||||
|
[ARCHITECTURE-BLUEPRINT.md](ARCHITECTURE-BLUEPRINT.md), the ADRs under
|
||||||
|
[adr/](adr/), and the [ROADMAP](ROADMAP.md).
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Python 3.12 or 3.13
|
||||||
|
- [`uv`](https://docs.astral.sh/uv/) on the PATH (one static binary)
|
||||||
|
- A POSIX-ish shell (Linux, macOS, WSL2)
|
||||||
|
|
||||||
|
The pinned tech stack is documented in
|
||||||
|
[ADR-0005](adr/0005-v1-tech-stack.md).
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uv sync --all-extras # install deps; produces .venv/ and uv.lock
|
||||||
|
cp .env.example .env # optional — the defaults work out of the box
|
||||||
|
|
||||||
|
make migrate-fresh # creates ./var/artifactstore.db and applies migrations
|
||||||
|
make dev # uvicorn on 127.0.0.1:8000
|
||||||
|
```
|
||||||
|
|
||||||
|
In another terminal:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -s http://127.0.0.1:8000/health | python3 -m json.tool
|
||||||
|
artifactstore health
|
||||||
|
```
|
||||||
|
|
||||||
|
Both should report ``status: ok``.
|
||||||
|
|
||||||
|
## Environment variables
|
||||||
|
|
||||||
|
All settings are prefixed with ``ARTIFACTSTORE_`` and read by
|
||||||
|
`pydantic-settings` from the environment and (optionally) `./.env`.
|
||||||
|
|
||||||
|
| Variable | Default | Purpose |
|
||||||
|
|-----------------------------------|-----------------------------------------------|---------|
|
||||||
|
| `ARTIFACTSTORE_DATABASE_URL` | `sqlite+aiosqlite:///./var/artifactstore.db` | SQLAlchemy async URL. Alembic translates `+aiosqlite` and `+asyncpg` to their sync drivers at migrate-time. |
|
||||||
|
| `ARTIFACTSTORE_STORAGE_LOCAL_ROOT`| `./var/storage` | Root directory for the local filesystem storage backend. Created on first use. |
|
||||||
|
| `ARTIFACTSTORE_LOG_LEVEL` | `INFO` | Python logging level (`DEBUG` / `INFO` / `WARNING` / `ERROR`). |
|
||||||
|
|
||||||
|
See [`.env.example`](../.env.example) for the canonical template.
|
||||||
|
|
||||||
|
## Database backends
|
||||||
|
|
||||||
|
### SQLite (development default)
|
||||||
|
|
||||||
|
Zero-config. The database file lives at `./var/artifactstore.db` by default
|
||||||
|
and is gitignored.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
make migrate-fresh # drop and re-create
|
||||||
|
make migrate # idempotent: apply pending migrations
|
||||||
|
```
|
||||||
|
|
||||||
|
### PostgreSQL 16+ (shared deployments)
|
||||||
|
|
||||||
|
Install the optional `postgres` extra (pulls in `psycopg[binary]` for
|
||||||
|
Alembic's sync driver):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uv sync --all-extras --extra postgres
|
||||||
|
```
|
||||||
|
|
||||||
|
Set the URL with the async driver; Alembic switches to `+psycopg` for
|
||||||
|
migrations automatically:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export ARTIFACTSTORE_DATABASE_URL=postgresql+asyncpg://artifactstore:secret@db.internal:5432/artifactstore
|
||||||
|
make migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
The schema is identical to SQLite (per
|
||||||
|
[ADR-0002](adr/0002-event-log-source-of-truth.md) the events table drives
|
||||||
|
all materialised views).
|
||||||
|
|
||||||
|
## Storage backends
|
||||||
|
|
||||||
|
The storage adapter SPI is documented in
|
||||||
|
[ADR-0001](adr/0001-content-addressed-storage.md) and
|
||||||
|
[ADR-0004](adr/0004-control-plane-data-plane-contract.md).
|
||||||
|
|
||||||
|
### Local filesystem (default)
|
||||||
|
|
||||||
|
Objects are addressed by content (`blake3:<hex>`) and laid out as
|
||||||
|
|
||||||
|
```
|
||||||
|
<root>/<algorithm>/<hex[0:2]>/<hex[2:4]>/<hex>
|
||||||
|
```
|
||||||
|
|
||||||
|
with atomic writes (tmpfile + fsync + rename). The S3-compatible backend
|
||||||
|
lands in WP-0004.
|
||||||
|
|
||||||
|
## CLI reference
|
||||||
|
|
||||||
|
`artifactstore --help` lists every subcommand. The v0.1 set:
|
||||||
|
|
||||||
|
| Command | Purpose |
|
||||||
|
|--------------------------|---------|
|
||||||
|
| `artifactstore version` | Print the package version and exit. |
|
||||||
|
| `artifactstore migrate` | Run `alembic upgrade head` against the configured database. |
|
||||||
|
| `artifactstore replay` | Truncate every materialised view and rebuild it from the event log; prints the highest sequence applied. |
|
||||||
|
| `artifactstore health` | JSON liveness summary (db, backend, status). Same payload as the HTTP `/health` endpoint. |
|
||||||
|
|
||||||
|
The CLI is a thin client over `artifactstore.registry.Registry`
|
||||||
|
(see [ADR-0005](adr/0005-v1-tech-stack.md)).
|
||||||
|
|
||||||
|
## HTTP reference (v0.1)
|
||||||
|
|
||||||
|
| Route | Purpose |
|
||||||
|
|----------------|---------|
|
||||||
|
| `GET /` | Service banner (scaffold marker). |
|
||||||
|
| `GET /health` | Liveness summary. Returns ``{status, db, backend, version}``. `status` is `ok` only when both the DB probe (`SELECT 1`) and the backend `health()` succeed. |
|
||||||
|
| `GET /docs` | FastAPI's interactive OpenAPI docs (`/openapi.json` underneath). |
|
||||||
|
|
||||||
|
Package CRUD, file upload/download, manifest retrieval, retention controls,
|
||||||
|
and the event stream all land in WP-0002–WP-0003. Today they are reachable
|
||||||
|
via the Python library.
|
||||||
|
|
||||||
|
## End-to-end smoke test (Python library)
|
||||||
|
|
||||||
|
This exercises every layer (identity, manifest, events, dataplane, storage,
|
||||||
|
registry, replay) end-to-end against the default SQLite + local FS configuration.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
from collections.abc import AsyncIterator
|
||||||
|
from artifactstore.app import build_registry
|
||||||
|
from artifactstore.manifest import decode as manifest_decode
|
||||||
|
|
||||||
|
|
||||||
|
async def chunks(data: bytes) -> AsyncIterator[bytes]:
|
||||||
|
yield data
|
||||||
|
|
||||||
|
|
||||||
|
async def main() -> None:
|
||||||
|
registry = build_registry()
|
||||||
|
try:
|
||||||
|
pkg = await registry.create_package(
|
||||||
|
name="smoke-test",
|
||||||
|
producer="ops",
|
||||||
|
subject="example.org",
|
||||||
|
retention_class="raw-evidence",
|
||||||
|
actor="ops",
|
||||||
|
metadata={"smoke": True},
|
||||||
|
)
|
||||||
|
await registry.ingest_file(
|
||||||
|
pkg, relative_path="hello.txt", media_type="text/plain",
|
||||||
|
stream=chunks(b"hello world"), actor="ops",
|
||||||
|
)
|
||||||
|
manifest_addr = await registry.finalize_package(pkg, actor="ops")
|
||||||
|
cbor = await registry.get_manifest_bytes(pkg, format="cbor")
|
||||||
|
manifest = manifest_decode(cbor)
|
||||||
|
print("package:", pkg)
|
||||||
|
print("manifest digest:", manifest_addr)
|
||||||
|
print("files in manifest:", [f.relative_path for f in manifest.files])
|
||||||
|
finally:
|
||||||
|
await registry.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
Prerequisites: `make migrate-fresh` has been run so the schema and the
|
||||||
|
retention class seeds exist.
|
||||||
|
|
||||||
|
## Replay / disaster recovery
|
||||||
|
|
||||||
|
Every state-changing operation writes one row to `events` and updates the
|
||||||
|
materialised views in the same transaction
|
||||||
|
([ADR-0002](adr/0002-event-log-source-of-truth.md)). If the materialised
|
||||||
|
views are lost or corrupted, rebuild them from the event log:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
artifactstore replay
|
||||||
|
```
|
||||||
|
|
||||||
|
The command drops every row from `artifact_packages`, `artifact_files`,
|
||||||
|
`storage_locations`, and `retention_state`, then replays the events in
|
||||||
|
sequence order through the canonical view writer. The result is
|
||||||
|
**byte-identical** to the materialised state before the replay
|
||||||
|
(verified by the WP-0001-T013 integration test).
|
||||||
|
|
||||||
|
## Failure modes operators should expect
|
||||||
|
|
||||||
|
| Symptom | Likely cause | Fix |
|
||||||
|
|--------------------------------------------------|----------------------------------------------|-----|
|
||||||
|
| `/health` returns `status: degraded`, `db.healthy: false` | DB unreachable or migrations not applied | Check `ARTIFACTSTORE_DATABASE_URL`; run `make migrate`. |
|
||||||
|
| `/health` returns `status: degraded`, `backend.healthy: false` | Storage root missing or unreadable | Recreate `ARTIFACTSTORE_STORAGE_LOCAL_ROOT` or fix permissions. |
|
||||||
|
| `ObjectNotFoundError` from `get_file` | Underlying bytes deleted but the file row remains | Investigate; v1 does not garbage-collect orphaned rows (WP-0006). |
|
||||||
|
| `DuplicateRelativePathError` from `ingest_file` | Same package + path ingested twice | Use a distinct `relative_path` per file within one package. |
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- [INTENT.md](../INTENT.md) — purpose and scope.
|
||||||
|
- [SCOPE.md](../SCOPE.md) — what this repo does and does not own.
|
||||||
|
- [ARCHITECTURE-BLUEPRINT.md](ARCHITECTURE-BLUEPRINT.md) — module layout,
|
||||||
|
data model, API shape.
|
||||||
|
- [PLATFORM-AMBITION.md](PLATFORM-AMBITION.md) — longer-horizon thesis
|
||||||
|
and the v1 schema commitments.
|
||||||
|
- [ROADMAP.md](ROADMAP.md) — workplan sequencing.
|
||||||
|
- [ASSEMBLY-EXPERIMENT.md](ASSEMBLY-EXPERIMENT.md) — opt-in asm research line.
|
||||||
|
|
||||||
|
### Architecture Decision Records
|
||||||
|
|
||||||
|
- [ADR-0001 — Content-Addressed Storage with Dual Digest](adr/0001-content-addressed-storage.md)
|
||||||
|
- [ADR-0002 — Append-Only Event Log as Source of Truth](adr/0002-event-log-source-of-truth.md)
|
||||||
|
- [ADR-0003 — Manifest Canonicalisation = Canonical CBOR](adr/0003-manifest-canonical-cbor.md)
|
||||||
|
- [ADR-0004 — Control Plane / Data Plane Contract](adr/0004-control-plane-data-plane-contract.md)
|
||||||
|
- [ADR-0005 — V1 Technology Stack](adr/0005-v1-tech-stack.md)
|
||||||
|
- [ADR-0006 — OCI Artifact Compatibility Kept Reachable](adr/0006-oci-compatibility-reachable.md)
|
||||||
@@ -1,24 +1,80 @@
|
|||||||
"""FastAPI application entry point.
|
"""FastAPI application — HTTP surface for the registry.
|
||||||
|
|
||||||
The full registry-aware ``/health`` endpoint lands in
|
T014 ships a minimal app with two routes:
|
||||||
ARTIFACT-STORE-WP-0001-T014. This scaffold exposes a minimal root route so
|
|
||||||
``make dev`` has something to serve while the registry layer is being built.
|
* ``GET /`` — service banner.
|
||||||
|
* ``GET /health`` — registry liveness + DB connectivity + storage backend.
|
||||||
|
|
||||||
|
Richer endpoints (package CRUD, file upload, manifest retrieval, event
|
||||||
|
stream) land in workplan WP-0002. The app is built through
|
||||||
|
:func:`create_app` so tests can inject their own settings.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from fastapi import FastAPI
|
from contextlib import asynccontextmanager
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import Depends, FastAPI, Request
|
||||||
|
|
||||||
from artifactstore import __version__
|
from artifactstore import __version__
|
||||||
|
from artifactstore.app import build_registry
|
||||||
|
from artifactstore.config import Settings
|
||||||
|
from artifactstore.registry import Registry
|
||||||
|
|
||||||
app = FastAPI(title="artifact-store", version=__version__)
|
__all__ = ["app", "create_app"]
|
||||||
|
|
||||||
|
|
||||||
@app.get("/")
|
def get_registry(request: Request) -> Registry:
|
||||||
def root() -> dict[str, str]:
|
return request.app.state.registry # type: ignore[no-any-return]
|
||||||
"""Return a service banner indicating the scaffold is up."""
|
|
||||||
|
|
||||||
|
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||||
|
"""Build the FastAPI app. Lifespan owns the registry instance."""
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(application: FastAPI) -> Any:
|
||||||
|
registry = build_registry(settings)
|
||||||
|
application.state.registry = registry
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
await registry.dispose()
|
||||||
|
|
||||||
|
application = FastAPI(
|
||||||
|
title="artifact-store",
|
||||||
|
version=__version__,
|
||||||
|
lifespan=lifespan,
|
||||||
|
)
|
||||||
|
|
||||||
|
@application.get("/")
|
||||||
|
def root() -> dict[str, str]:
|
||||||
return {
|
return {
|
||||||
"service": "artifact-store",
|
"service": "artifact-store",
|
||||||
"version": __version__,
|
"version": __version__,
|
||||||
"status": "scaffold",
|
"status": "scaffold",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@application.get("/health")
|
||||||
|
async def health(registry: Registry = Depends(get_registry)) -> dict[str, Any]:
|
||||||
|
db_ok, db_detail = await registry.db_health()
|
||||||
|
backend_status = await registry.backend_health()
|
||||||
|
overall = "ok" if db_ok and backend_status.healthy else "degraded"
|
||||||
|
return {
|
||||||
|
"service": "artifact-store",
|
||||||
|
"version": __version__,
|
||||||
|
"status": overall,
|
||||||
|
"db": {"healthy": db_ok, "detail": db_detail},
|
||||||
|
"backend": {
|
||||||
|
"backend_id": backend_status.backend_id,
|
||||||
|
"healthy": backend_status.healthy,
|
||||||
|
"detail": backend_status.detail,
|
||||||
|
"free_bytes": backend_status.free_bytes,
|
||||||
|
"total_bytes": backend_status.total_bytes,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
return application
|
||||||
|
|
||||||
|
|
||||||
|
app = create_app()
|
||||||
|
|||||||
27
src/artifactstore/app.py
Normal file
27
src/artifactstore/app.py
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
"""Composition root: build the runtime registry from settings.
|
||||||
|
|
||||||
|
The HTTP server, the CLI, and any future host all instantiate the
|
||||||
|
:class:`Registry` through :func:`build_registry`. Wiring lives here so the
|
||||||
|
control-plane consumers stay thin (per ADR-0004).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from artifactstore.config import Settings, get_settings
|
||||||
|
from artifactstore.dataplane import InProcessDataPlane
|
||||||
|
from artifactstore.db.engine import create_engine
|
||||||
|
from artifactstore.events import RegistryViewWriter
|
||||||
|
from artifactstore.registry import Registry
|
||||||
|
from artifactstore.storage import LocalBackend
|
||||||
|
|
||||||
|
__all__ = ["build_registry"]
|
||||||
|
|
||||||
|
|
||||||
|
def build_registry(settings: Settings | None = None) -> Registry:
|
||||||
|
"""Wire engine, local FS backend, in-process data plane, and registry."""
|
||||||
|
effective = settings or get_settings()
|
||||||
|
engine = create_engine(effective)
|
||||||
|
backend = LocalBackend(effective.storage_local_root, backend_id="local")
|
||||||
|
dataplane = InProcessDataPlane(backend)
|
||||||
|
view_writer = RegistryViewWriter()
|
||||||
|
return Registry(engine, dataplane, view_writer)
|
||||||
@@ -1,15 +1,28 @@
|
|||||||
"""artifact-store command-line interface.
|
"""artifact-store command-line interface.
|
||||||
|
|
||||||
The CLI is a thin consumer of :mod:`artifactstore.registry` (per ADR-0005).
|
The CLI is a thin consumer of :mod:`artifactstore.registry` (per ADR-0005).
|
||||||
The scaffold exposes only ``version``; richer subcommands land in later
|
T014 ships ``version``, ``migrate``, ``replay``, and ``health`` subcommands;
|
||||||
tasks of ARTIFACT-STORE-WP-0001 and follow-on workplans.
|
richer subcommands (e.g. ``push``, ``manifest``) land alongside the HTTP API
|
||||||
|
in WP-0002.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
import typer
|
import typer
|
||||||
|
|
||||||
from artifactstore import __version__
|
from artifactstore import __version__
|
||||||
|
from artifactstore.config import Settings, get_settings
|
||||||
|
from artifactstore.db.engine import create_engine
|
||||||
|
from artifactstore.events import RegistryViewWriter
|
||||||
|
from artifactstore.events import replay as events_replay
|
||||||
|
from artifactstore.registry import Registry
|
||||||
|
|
||||||
|
__all__ = ["app"]
|
||||||
|
|
||||||
app = typer.Typer(
|
app = typer.Typer(
|
||||||
help="artifact-store: artifact registry and storage gateway",
|
help="artifact-store: artifact registry and storage gateway",
|
||||||
@@ -19,11 +32,7 @@ app = typer.Typer(
|
|||||||
|
|
||||||
@app.callback()
|
@app.callback()
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
"""Top-level CLI entry point.
|
"""Top-level CLI entry point."""
|
||||||
|
|
||||||
Forces typer into multi-command mode so subcommands behave consistently
|
|
||||||
even while the scaffold only ships ``version``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
@app.command()
|
@app.command()
|
||||||
@@ -32,5 +41,74 @@ def version() -> None:
|
|||||||
typer.echo(__version__)
|
typer.echo(__version__)
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def migrate(
|
||||||
|
alembic_ini: Path = typer.Option(
|
||||||
|
Path("alembic.ini"),
|
||||||
|
"--alembic-ini",
|
||||||
|
help="Path to alembic.ini (defaults to the repo root).",
|
||||||
|
),
|
||||||
|
) -> None:
|
||||||
|
"""Run ``alembic upgrade head`` against the configured database."""
|
||||||
|
from alembic import command
|
||||||
|
from alembic.config import Config
|
||||||
|
|
||||||
|
cfg = Config(str(alembic_ini))
|
||||||
|
command.upgrade(cfg, "head")
|
||||||
|
typer.echo("alembic upgrade head: ok")
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def replay() -> None:
|
||||||
|
"""Truncate materialised views and replay every event from sequence 1."""
|
||||||
|
settings = get_settings()
|
||||||
|
last_seq = asyncio.run(_replay_async(settings))
|
||||||
|
typer.echo(f"replayed up to sequence {last_seq}")
|
||||||
|
|
||||||
|
|
||||||
|
@app.command()
|
||||||
|
def health() -> None:
|
||||||
|
"""Print a JSON liveness summary (db, backend)."""
|
||||||
|
settings = get_settings()
|
||||||
|
payload = asyncio.run(_health_async(settings))
|
||||||
|
typer.echo(json.dumps(payload, indent=2))
|
||||||
|
|
||||||
|
|
||||||
|
# ---- internals -------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def _replay_async(settings: Settings) -> int:
|
||||||
|
engine = create_engine(settings)
|
||||||
|
try:
|
||||||
|
return await events_replay(engine, RegistryViewWriter(), reset=True)
|
||||||
|
finally:
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
async def _health_async(settings: Settings) -> dict[str, Any]:
|
||||||
|
from artifactstore.app import build_registry
|
||||||
|
|
||||||
|
registry: Registry = build_registry(settings)
|
||||||
|
try:
|
||||||
|
db_ok, db_detail = await registry.db_health()
|
||||||
|
backend_status = await registry.backend_health()
|
||||||
|
finally:
|
||||||
|
await registry.dispose()
|
||||||
|
overall = "ok" if db_ok and backend_status.healthy else "degraded"
|
||||||
|
return {
|
||||||
|
"service": "artifact-store",
|
||||||
|
"version": __version__,
|
||||||
|
"status": overall,
|
||||||
|
"db": {"healthy": db_ok, "detail": db_detail},
|
||||||
|
"backend": {
|
||||||
|
"backend_id": backend_status.backend_id,
|
||||||
|
"healthy": backend_status.healthy,
|
||||||
|
"detail": backend_status.detail,
|
||||||
|
"free_bytes": backend_status.free_bytes,
|
||||||
|
"total_bytes": backend_status.total_bytes,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__": # pragma: no cover
|
if __name__ == "__main__": # pragma: no cover
|
||||||
app()
|
app()
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ from typing import Any
|
|||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
import cbor2
|
import cbor2
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select, text
|
||||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
from artifactstore.dataplane.spi import DataPlane
|
from artifactstore.dataplane.spi import DataPlane
|
||||||
@@ -61,6 +61,7 @@ from artifactstore.manifest import (
|
|||||||
)
|
)
|
||||||
from artifactstore.manifest.codec import decode as manifest_decode
|
from artifactstore.manifest.codec import decode as manifest_decode
|
||||||
from artifactstore.manifest.projection import jcs_projection
|
from artifactstore.manifest.projection import jcs_projection
|
||||||
|
from artifactstore.storage.spi import BackendStatus
|
||||||
|
|
||||||
__all__ = ["Registry"]
|
__all__ = ["Registry"]
|
||||||
|
|
||||||
@@ -316,6 +317,29 @@ class Registry:
|
|||||||
poll_interval_seconds=poll_interval_seconds,
|
poll_interval_seconds=poll_interval_seconds,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# ---- health / lifecycle -------------------------------------------------
|
||||||
|
|
||||||
|
async def db_health(self) -> tuple[bool, str]:
|
||||||
|
"""Probe the database with ``SELECT 1``.
|
||||||
|
|
||||||
|
Returns ``(healthy, detail)``; ``detail`` is "ok" on success and the
|
||||||
|
exception message on failure.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
async with self._engine.connect() as conn:
|
||||||
|
await conn.execute(text("SELECT 1"))
|
||||||
|
except Exception as exc:
|
||||||
|
return False, f"{type(exc).__name__}: {exc}"
|
||||||
|
return True, "ok"
|
||||||
|
|
||||||
|
async def backend_health(self) -> BackendStatus:
|
||||||
|
"""Probe the configured storage backend through the data plane."""
|
||||||
|
return await self._dataplane.backend_health()
|
||||||
|
|
||||||
|
async def dispose(self) -> None:
|
||||||
|
"""Release the engine's connection pool. Idempotent."""
|
||||||
|
await self._engine.dispose()
|
||||||
|
|
||||||
# ---- internals ----------------------------------------------------------
|
# ---- internals ----------------------------------------------------------
|
||||||
|
|
||||||
async def _validate_retention_class(self, retention_class: str) -> None:
|
async def _validate_retention_class(self, retention_class: str) -> None:
|
||||||
|
|||||||
86
tests/integration/test_cli_commands.py
Normal file
86
tests/integration/test_cli_commands.py
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
"""CLI command tests (ARTIFACT-STORE-WP-0001-T014)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine, insert, inspect
|
||||||
|
from typer.testing import CliRunner
|
||||||
|
|
||||||
|
from artifactstore.cli import app as cli_app
|
||||||
|
from artifactstore.db.schema import metadata, retention_classes
|
||||||
|
from artifactstore.db.seed import RETENTION_CLASS_SEEDS
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def runner() -> CliRunner:
|
||||||
|
return CliRunner()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def env_db(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
||||||
|
db_path = tmp_path / "cli-test.db"
|
||||||
|
storage_root = tmp_path / "storage"
|
||||||
|
storage_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
monkeypatch.setenv("ARTIFACTSTORE_DATABASE_URL", f"sqlite+aiosqlite:///{db_path}")
|
||||||
|
monkeypatch.setenv("ARTIFACTSTORE_STORAGE_LOCAL_ROOT", str(storage_root))
|
||||||
|
return db_path
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_version_prints_version(runner: CliRunner) -> None:
|
||||||
|
result = runner.invoke(cli_app, ["version"])
|
||||||
|
assert result.exit_code == 0
|
||||||
|
assert result.output.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_migrate_creates_schema(
|
||||||
|
runner: CliRunner,
|
||||||
|
env_db: Path,
|
||||||
|
monkeypatch: pytest.MonkeyPatch,
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.chdir(REPO_ROOT)
|
||||||
|
result = runner.invoke(cli_app, ["migrate"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "ok" in result.output
|
||||||
|
|
||||||
|
sync_engine = create_engine(f"sqlite:///{env_db}", future=True)
|
||||||
|
names = set(inspect(sync_engine).get_table_names())
|
||||||
|
sync_engine.dispose()
|
||||||
|
assert {"events", "retention_classes", "alembic_version"}.issubset(names)
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_replay_against_empty_log(
|
||||||
|
runner: CliRunner,
|
||||||
|
env_db: Path,
|
||||||
|
) -> None:
|
||||||
|
# Pre-create schema (without using migrate command to keep the test
|
||||||
|
# focused on the replay surface).
|
||||||
|
sync_engine = create_engine(f"sqlite:///{env_db}", future=True)
|
||||||
|
metadata.create_all(sync_engine)
|
||||||
|
with sync_engine.begin() as conn:
|
||||||
|
conn.execute(insert(retention_classes), [dict(s) for s in RETENTION_CLASS_SEEDS])
|
||||||
|
sync_engine.dispose()
|
||||||
|
|
||||||
|
result = runner.invoke(cli_app, ["replay"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
assert "replayed up to sequence 0" in result.output
|
||||||
|
|
||||||
|
|
||||||
|
def test_cli_health_reports_ok(
|
||||||
|
runner: CliRunner,
|
||||||
|
env_db: Path,
|
||||||
|
) -> None:
|
||||||
|
sync_engine = create_engine(f"sqlite:///{env_db}", future=True)
|
||||||
|
metadata.create_all(sync_engine)
|
||||||
|
sync_engine.dispose()
|
||||||
|
|
||||||
|
result = runner.invoke(cli_app, ["health"])
|
||||||
|
assert result.exit_code == 0, result.output
|
||||||
|
payload = json.loads(result.output)
|
||||||
|
assert payload["status"] == "ok"
|
||||||
|
assert payload["db"]["healthy"] is True
|
||||||
|
assert payload["backend"]["healthy"] is True
|
||||||
63
tests/integration/test_http_health.py
Normal file
63
tests/integration/test_http_health.py
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
"""HTTP /health endpoint tests (ARTIFACT-STORE-WP-0001-T014)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlalchemy import create_engine, insert
|
||||||
|
|
||||||
|
from artifactstore.api.http import create_app
|
||||||
|
from artifactstore.config import Settings
|
||||||
|
from artifactstore.db.schema import metadata, retention_classes
|
||||||
|
from artifactstore.db.seed import RETENTION_CLASS_SEEDS
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def settings(tmp_path: Path) -> Settings:
|
||||||
|
db_path = tmp_path / "http-test.db"
|
||||||
|
storage_root = tmp_path / "storage"
|
||||||
|
storage_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
# Build a fresh schema synchronously so the app starts against a
|
||||||
|
# ready-to-use database.
|
||||||
|
sync_engine = create_engine(f"sqlite:///{db_path}", future=True)
|
||||||
|
metadata.create_all(sync_engine)
|
||||||
|
with sync_engine.begin() as conn:
|
||||||
|
conn.execute(insert(retention_classes), [dict(s) for s in RETENTION_CLASS_SEEDS])
|
||||||
|
sync_engine.dispose()
|
||||||
|
return Settings(
|
||||||
|
database_url=f"sqlite+aiosqlite:///{db_path}",
|
||||||
|
storage_local_root=str(storage_root),
|
||||||
|
log_level="INFO",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_root_route_returns_banner(settings: Settings) -> None:
|
||||||
|
app = create_app(settings)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
resp = client.get("/")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["service"] == "artifact-store"
|
||||||
|
assert body["status"] == "scaffold"
|
||||||
|
|
||||||
|
|
||||||
|
def test_health_endpoint_reports_db_and_backend(settings: Settings) -> None:
|
||||||
|
app = create_app(settings)
|
||||||
|
with TestClient(app) as client:
|
||||||
|
resp = client.get("/health")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
body = resp.json()
|
||||||
|
assert body["status"] == "ok"
|
||||||
|
assert body["db"]["healthy"] is True
|
||||||
|
assert body["db"]["detail"] == "ok"
|
||||||
|
assert body["backend"]["healthy"] is True
|
||||||
|
assert body["backend"]["backend_id"] == "local"
|
||||||
|
assert body["backend"]["free_bytes"] is not None
|
||||||
|
|
||||||
|
|
||||||
|
# NB: a "degraded backend" path test was considered but the LocalBackend
|
||||||
|
# constructor recreates its root in mkdir(parents=True, exist_ok=True) when
|
||||||
|
# the lifespan instantiates it. The unhealthy-root branch is covered
|
||||||
|
# directly in tests/unit/test_storage_local.py without that re-creation.
|
||||||
@@ -4,13 +4,13 @@ type: workplan
|
|||||||
title: "Foundation: Scaffold, Core Kernels, Local FS Backend"
|
title: "Foundation: Scaffold, Core Kernels, Local FS Backend"
|
||||||
repo: artifact-store
|
repo: artifact-store
|
||||||
domain: stack
|
domain: stack
|
||||||
status: active
|
status: done
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: stack
|
topic_slug: stack
|
||||||
planning_priority: high
|
planning_priority: high
|
||||||
planning_order: 1
|
planning_order: 1
|
||||||
created: "2026-05-15"
|
created: "2026-05-15"
|
||||||
updated: "2026-05-15"
|
updated: "2026-05-16"
|
||||||
state_hub_workstream_id: "aebf996c-8721-4e8c-9e56-61d5e4bf8dcb"
|
state_hub_workstream_id: "aebf996c-8721-4e8c-9e56-61d5e4bf8dcb"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -262,7 +262,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: ARTIFACT-STORE-WP-0001-T014
|
id: ARTIFACT-STORE-WP-0001-T014
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "a43628ab-8b53-45fa-852a-ff0118dd12e7"
|
state_hub_task_id: "a43628ab-8b53-45fa-852a-ff0118dd12e7"
|
||||||
```
|
```
|
||||||
@@ -280,7 +280,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: ARTIFACT-STORE-WP-0001-T008
|
id: ARTIFACT-STORE-WP-0001-T008
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "9b60036c-61f2-4c22-ad31-7213473d42d0"
|
state_hub_task_id: "9b60036c-61f2-4c22-ad31-7213473d42d0"
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in New Issue
Block a user