generated from coulomb/repo-seed
tegwick
f90c761ef6
docs/OPERATOR.md (new): runbook with prerequisites, quick start, environment variables, SQLite + PostgreSQL setup, storage layout, CLI reference, HTTP /health reference, an end-to-end Python smoke test (create_package -> ingest_file -> finalize -> manifest), the replay / disaster-recovery procedure, common failure modes, and a References section that links every ADR (0001..0006), the blueprint, platform ambition, roadmap, and assembly experiment. README.md: refreshed to v0.1 baseline status. Quick-start uses the real flow (uv sync, migrate-fresh, dev, /health, artifactstore health). Make targets and CLI commands tabulated. Links docs/OPERATOR.md. AGENTS.md: Current Repo Shape now reflects the landed scaffold + library + CLI + HTTP app rather than "no runnable scaffold yet"; links OPERATOR.md and lists the canonical local commands. workplans/ARTIFACT-STORE-WP-0001-service-baseline.md: - T008 marked done. - frontmatter status: active -> done; updated: 2026-05-16. All ten WP-0001 tasks are now done (T001/T002/T003/T008/T009/T010/ T011/T012/T013/T014). Foundation workplan retires. Gates: ruff clean, mypy --strict clean, 83 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
224 lines
8.5 KiB
Markdown
224 lines
8.5 KiB
Markdown
# Operator Guide
|
||
|
||
Status: v0.1 (WP-0001 baseline)
|
||
Updated: 2026-05-16
|
||
|
||
This guide is the user manual for running `artifact-store` v0.1 — the
|
||
library + CLI + minimal HTTP app that landed in WP-0001. Ingest, finalize,
|
||
and retrieve workflows go through the Python library today; the HTTP
|
||
upload API arrives in WP-0002.
|
||
|
||
For architectural background see
|
||
[ARCHITECTURE-BLUEPRINT.md](ARCHITECTURE-BLUEPRINT.md), the ADRs under
|
||
[adr/](adr/), and the [ROADMAP](ROADMAP.md).
|
||
|
||
## Prerequisites
|
||
|
||
- Python 3.12 or 3.13
|
||
- [`uv`](https://docs.astral.sh/uv/) on the PATH (one static binary)
|
||
- A POSIX-ish shell (Linux, macOS, WSL2)
|
||
|
||
The pinned tech stack is documented in
|
||
[ADR-0005](adr/0005-v1-tech-stack.md).
|
||
|
||
## Quick start
|
||
|
||
```sh
|
||
uv sync --all-extras # install deps; produces .venv/ and uv.lock
|
||
cp .env.example .env # optional — the defaults work out of the box
|
||
|
||
make migrate-fresh # creates ./var/artifactstore.db and applies migrations
|
||
make dev # uvicorn on 127.0.0.1:8000
|
||
```
|
||
|
||
In another terminal:
|
||
|
||
```sh
|
||
curl -s http://127.0.0.1:8000/health | python3 -m json.tool
|
||
artifactstore health
|
||
```
|
||
|
||
Both should report ``status: ok``.
|
||
|
||
## Environment variables
|
||
|
||
All settings are prefixed with ``ARTIFACTSTORE_`` and read by
|
||
`pydantic-settings` from the environment and (optionally) `./.env`.
|
||
|
||
| Variable | Default | Purpose |
|
||
|-----------------------------------|-----------------------------------------------|---------|
|
||
| `ARTIFACTSTORE_DATABASE_URL` | `sqlite+aiosqlite:///./var/artifactstore.db` | SQLAlchemy async URL. Alembic translates `+aiosqlite` and `+asyncpg` to their sync drivers at migrate-time. |
|
||
| `ARTIFACTSTORE_STORAGE_LOCAL_ROOT`| `./var/storage` | Root directory for the local filesystem storage backend. Created on first use. |
|
||
| `ARTIFACTSTORE_LOG_LEVEL` | `INFO` | Python logging level (`DEBUG` / `INFO` / `WARNING` / `ERROR`). |
|
||
|
||
See [`.env.example`](../.env.example) for the canonical template.
|
||
|
||
## Database backends
|
||
|
||
### SQLite (development default)
|
||
|
||
Zero-config. The database file lives at `./var/artifactstore.db` by default
|
||
and is gitignored.
|
||
|
||
```sh
|
||
make migrate-fresh # drop and re-create
|
||
make migrate # idempotent: apply pending migrations
|
||
```
|
||
|
||
### PostgreSQL 16+ (shared deployments)
|
||
|
||
Install the optional `postgres` extra (pulls in `psycopg[binary]` for
|
||
Alembic's sync driver):
|
||
|
||
```sh
|
||
uv sync --all-extras --extra postgres
|
||
```
|
||
|
||
Set the URL with the async driver; Alembic switches to `+psycopg` for
|
||
migrations automatically:
|
||
|
||
```sh
|
||
export ARTIFACTSTORE_DATABASE_URL=postgresql+asyncpg://artifactstore:secret@db.internal:5432/artifactstore
|
||
make migrate
|
||
```
|
||
|
||
The schema is identical to SQLite (per
|
||
[ADR-0002](adr/0002-event-log-source-of-truth.md) the events table drives
|
||
all materialised views).
|
||
|
||
## Storage backends
|
||
|
||
The storage adapter SPI is documented in
|
||
[ADR-0001](adr/0001-content-addressed-storage.md) and
|
||
[ADR-0004](adr/0004-control-plane-data-plane-contract.md).
|
||
|
||
### Local filesystem (default)
|
||
|
||
Objects are addressed by content (`blake3:<hex>`) and laid out as
|
||
|
||
```
|
||
<root>/<algorithm>/<hex[0:2]>/<hex[2:4]>/<hex>
|
||
```
|
||
|
||
with atomic writes (tmpfile + fsync + rename). The S3-compatible backend
|
||
lands in WP-0004.
|
||
|
||
## CLI reference
|
||
|
||
`artifactstore --help` lists every subcommand. The v0.1 set:
|
||
|
||
| Command | Purpose |
|
||
|--------------------------|---------|
|
||
| `artifactstore version` | Print the package version and exit. |
|
||
| `artifactstore migrate` | Run `alembic upgrade head` against the configured database. |
|
||
| `artifactstore replay` | Truncate every materialised view and rebuild it from the event log; prints the highest sequence applied. |
|
||
| `artifactstore health` | JSON liveness summary (db, backend, status). Same payload as the HTTP `/health` endpoint. |
|
||
|
||
The CLI is a thin client over `artifactstore.registry.Registry`
|
||
(see [ADR-0005](adr/0005-v1-tech-stack.md)).
|
||
|
||
## HTTP reference (v0.1)
|
||
|
||
| Route | Purpose |
|
||
|----------------|---------|
|
||
| `GET /` | Service banner (scaffold marker). |
|
||
| `GET /health` | Liveness summary. Returns ``{status, db, backend, version}``. `status` is `ok` only when both the DB probe (`SELECT 1`) and the backend `health()` succeed. |
|
||
| `GET /docs` | FastAPI's interactive OpenAPI docs (`/openapi.json` underneath). |
|
||
|
||
Package CRUD, file upload/download, manifest retrieval, retention controls,
|
||
and the event stream all land in WP-0002–WP-0003. Today they are reachable
|
||
via the Python library.
|
||
|
||
## End-to-end smoke test (Python library)
|
||
|
||
This exercises every layer (identity, manifest, events, dataplane, storage,
|
||
registry, replay) end-to-end against the default SQLite + local FS configuration.
|
||
|
||
```python
|
||
import asyncio
|
||
from collections.abc import AsyncIterator
|
||
from artifactstore.app import build_registry
|
||
from artifactstore.manifest import decode as manifest_decode
|
||
|
||
|
||
async def chunks(data: bytes) -> AsyncIterator[bytes]:
|
||
yield data
|
||
|
||
|
||
async def main() -> None:
|
||
registry = build_registry()
|
||
try:
|
||
pkg = await registry.create_package(
|
||
name="smoke-test",
|
||
producer="ops",
|
||
subject="example.org",
|
||
retention_class="raw-evidence",
|
||
actor="ops",
|
||
metadata={"smoke": True},
|
||
)
|
||
await registry.ingest_file(
|
||
pkg, relative_path="hello.txt", media_type="text/plain",
|
||
stream=chunks(b"hello world"), actor="ops",
|
||
)
|
||
manifest_addr = await registry.finalize_package(pkg, actor="ops")
|
||
cbor = await registry.get_manifest_bytes(pkg, format="cbor")
|
||
manifest = manifest_decode(cbor)
|
||
print("package:", pkg)
|
||
print("manifest digest:", manifest_addr)
|
||
print("files in manifest:", [f.relative_path for f in manifest.files])
|
||
finally:
|
||
await registry.dispose()
|
||
|
||
|
||
asyncio.run(main())
|
||
```
|
||
|
||
Prerequisites: `make migrate-fresh` has been run so the schema and the
|
||
retention class seeds exist.
|
||
|
||
## Replay / disaster recovery
|
||
|
||
Every state-changing operation writes one row to `events` and updates the
|
||
materialised views in the same transaction
|
||
([ADR-0002](adr/0002-event-log-source-of-truth.md)). If the materialised
|
||
views are lost or corrupted, rebuild them from the event log:
|
||
|
||
```sh
|
||
artifactstore replay
|
||
```
|
||
|
||
The command drops every row from `artifact_packages`, `artifact_files`,
|
||
`storage_locations`, and `retention_state`, then replays the events in
|
||
sequence order through the canonical view writer. The result is
|
||
**byte-identical** to the materialised state before the replay
|
||
(verified by the WP-0001-T013 integration test).
|
||
|
||
## Failure modes operators should expect
|
||
|
||
| Symptom | Likely cause | Fix |
|
||
|--------------------------------------------------|----------------------------------------------|-----|
|
||
| `/health` returns `status: degraded`, `db.healthy: false` | DB unreachable or migrations not applied | Check `ARTIFACTSTORE_DATABASE_URL`; run `make migrate`. |
|
||
| `/health` returns `status: degraded`, `backend.healthy: false` | Storage root missing or unreadable | Recreate `ARTIFACTSTORE_STORAGE_LOCAL_ROOT` or fix permissions. |
|
||
| `ObjectNotFoundError` from `get_file` | Underlying bytes deleted but the file row remains | Investigate; v1 does not garbage-collect orphaned rows (WP-0006). |
|
||
| `DuplicateRelativePathError` from `ingest_file` | Same package + path ingested twice | Use a distinct `relative_path` per file within one package. |
|
||
|
||
## References
|
||
|
||
- [INTENT.md](../INTENT.md) — purpose and scope.
|
||
- [SCOPE.md](../SCOPE.md) — what this repo does and does not own.
|
||
- [ARCHITECTURE-BLUEPRINT.md](ARCHITECTURE-BLUEPRINT.md) — module layout,
|
||
data model, API shape.
|
||
- [PLATFORM-AMBITION.md](PLATFORM-AMBITION.md) — longer-horizon thesis
|
||
and the v1 schema commitments.
|
||
- [ROADMAP.md](ROADMAP.md) — workplan sequencing.
|
||
- [ASSEMBLY-EXPERIMENT.md](ASSEMBLY-EXPERIMENT.md) — opt-in asm research line.
|
||
|
||
### Architecture Decision Records
|
||
|
||
- [ADR-0001 — Content-Addressed Storage with Dual Digest](adr/0001-content-addressed-storage.md)
|
||
- [ADR-0002 — Append-Only Event Log as Source of Truth](adr/0002-event-log-source-of-truth.md)
|
||
- [ADR-0003 — Manifest Canonicalisation = Canonical CBOR](adr/0003-manifest-canonical-cbor.md)
|
||
- [ADR-0004 — Control Plane / Data Plane Contract](adr/0004-control-plane-data-plane-contract.md)
|
||
- [ADR-0005 — V1 Technology Stack](adr/0005-v1-tech-stack.md)
|
||
- [ADR-0006 — OCI Artifact Compatibility Kept Reachable](adr/0006-oci-compatibility-reachable.md)
|