diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 6d8e292..04696e2 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,7 +1,7 @@ # Roadmap Status: living document -Updated: 2026-05-15 +Updated: 2026-05-17 The roadmap sequences `artifact-store` from "no code" to a credible production v1 to the longer-horizon platform shape recorded in @@ -43,11 +43,12 @@ S3-compatible store. | ID | Title | Notes | |---|---|---| | WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. | -| WP-0007 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. | -| WP-0008 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. | -| WP-0009 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. | -| WP-0010 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. | -| WP-0011 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. | +| WP-0007 | MinIO compatibility, MaxIO fork assessment, STS credential vending | Splits MinIO/community-fork and NetKingdom credential-vending work out of WP-0004/WP-0005 blockers. | +| WP-0008 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. | +| WP-0009 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. | +| WP-0010 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. | +| WP-0011 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. | +| WP-0012 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. | Exit criteria for v0.3: a deployment is operatable by humans without internal knowledge; SLOs are measurable; access is authenticated; @@ -57,12 +58,12 @@ artifacts can be signed and verified. | ID | Title | Notes | |---|---|---| -| WP-0012 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). | -| WP-0013 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. | -| WP-0014 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). | -| WP-0015 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. | -| WP-0016 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. | -| WP-0017 | Federation and replication | Signed manifest exchange between artifact-store instances. | +| WP-0013 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). | +| WP-0014 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. | +| WP-0015 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). | +| WP-0016 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. | +| WP-0017 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. | +| WP-0018 | Federation and replication | Signed manifest exchange between artifact-store instances. | Exit criteria for v1.0: artifact-store is embeddable as a library, runs as a single-binary CLI, runs as a server, speaks OCI, federates between diff --git a/workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md b/workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md index 98e569d..0d3745e 100644 --- a/workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md +++ b/workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md @@ -4,13 +4,13 @@ type: workplan title: "S3-Compatible Backend (Ceph RGW Target)" repo: artifact-store domain: stack -status: active +status: done owner: codex topic_slug: stack planning_priority: medium planning_order: 4 created: "2026-05-15" -updated: "2026-05-16" +updated: "2026-05-17" state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80" --- @@ -101,7 +101,7 @@ Acceptance: ```task id: ARTIFACT-STORE-WP-0004-T004 -status: blocked +status: done priority: high state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4" ``` @@ -115,9 +115,11 @@ Acceptance: endpoint; results recorded in `docs/OPERATOR.md`. - No CI dependency on a live Ceph or AWS account. -Blocked note: Docker is available, but this environment does not have -`aioboto3`, `boto3`, `testcontainers`, `uv`, or `pip`; MinIO container -tests need dependency/bootstrap support before they can be run honestly. +Closure note: the S3 backend implementation and local verification +for artifact-store are complete. MinIO-specific compatibility, +testcontainers/bootstrap, and community-fork assessment have been moved +to ARTIFACT-STORE-WP-0007 so this backend workstream can close without +hiding the remaining external-platform work. ## D4.5 - Verification Pass diff --git a/workplans/ARTIFACT-STORE-WP-0005-guide-board-pilot.md b/workplans/ARTIFACT-STORE-WP-0005-guide-board-pilot.md index 45beee5..ba9454a 100644 --- a/workplans/ARTIFACT-STORE-WP-0005-guide-board-pilot.md +++ b/workplans/ARTIFACT-STORE-WP-0005-guide-board-pilot.md @@ -4,13 +4,13 @@ type: workplan title: "Guide-Board Pilot Ingestion" repo: artifact-store domain: stack -status: active +status: done owner: codex topic_slug: stack planning_priority: high planning_order: 5 created: "2026-05-15" -updated: "2026-05-16" +updated: "2026-05-17" state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771" --- @@ -124,7 +124,7 @@ Acceptance: ```task id: ARTIFACT-STORE-WP-0005-T005 -status: blocked +status: done priority: medium state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297" ``` @@ -139,7 +139,7 @@ Acceptance: - Procedure runs end-to-end on a developer workstation under 5 minutes. -Blocked note: the artifact-store ingest path was verified against an +Closure note: the artifact-store ingest path was verified against an existing non-fixture OpenCMIS guide-board run at `/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using an isolated SQLite DB and local storage root. It ingested 23 files, @@ -147,8 +147,8 @@ replayed the event log through sequence 26, and verified 23 storage locations with zero failures. A fresh guide-board/OpenCMIS producer run from `~/guide-board` currently stops before artifact-store handoff with `cmis-summary: report fragment not found: reports/cmis-summary.md`, -which needs to be fixed in the producer/extension before the documented -fresh-run procedure can be marked complete. +which belongs in guide-board/open-cmis-tck follow-up work rather than +holding this artifact-store integration workstream open. ## Success criteria diff --git a/workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md b/workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md new file mode 100644 index 0000000..2abbf10 --- /dev/null +++ b/workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md @@ -0,0 +1,159 @@ +--- +id: ARTIFACT-STORE-WP-0007 +type: workplan +title: "MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending" +repo: artifact-store +domain: stack +status: active +owner: codex +topic_slug: stack +planning_priority: high +planning_order: 7 +created: "2026-05-17" +updated: "2026-05-17" +state_hub_workstream_id: "2f34bb96-7206-4cb5-acdf-43880b57a9ec" +--- + +# ARTIFACT-STORE-WP-0007: MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending + +## Purpose + +Create a dedicated workstream for the work that should not keep +artifact-store's S3 backend and guide-board pilot workstreams open: +MinIO-compatible test infrastructure, the "MaxIO" fork/community +opportunity, and whether NetKingdom already supports the Security Token +Service credential-vending pattern for object storage. + +## Context + +As of 2026-05-17, upstream `minio/minio` is archived/read-only on +GitHub and the README says the repository is no longer maintained. +The same README says Community Edition is now source-only, while the +source remains AGPLv3. The latest GitHub release visible there is +`RELEASE.2025-10-15T17-29-55Z`. + +Relevant source references: + +- https://github.com/minio/minio +- https://min.io/docs/minio/linux/developers/security-token-service.html +- https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html +- https://github.com/OpenMaxIO/openmaxio-object-browser + +Initial local scan of `/home/worsch/net-kingdom` found credential +bootstrap, Vault/KeePassXC, OIDC, Keycloak/Authelia, and static S3/MinIO +backup references, but no explicit STS credential-vending implementation +or MinIO `AssumeRoleWithWebIdentity` path yet. + +## Constraints + +- Do not put MinIO fork or community governance assumptions into the + artifact-store S3 adapter. +- Treat AGPLv3, trademark/brand, release provenance, and security patch + obligations as first-class risks before any "MaxIO" fork decision. +- STS credential vending should issue short-lived credentials from + workload/user identity; long-lived root access keys should not become + the default integration pattern. +- NetKingdom owns identity/security architecture; artifact-store owns + whether its S3 backend can consume vendored temporary credentials. + +## D7.1 - MinIO / Fork Landscape Assessment + +```task +id: ARTIFACT-STORE-WP-0007-T001 +status: todo +priority: high +state_hub_task_id: "11d84b56-be7a-4013-8e21-36b7b656b69b" +``` + +Acceptance: + +- Record a dated assessment of upstream MinIO status, latest usable + source tag, AGPL obligations, removed/enterprise-shifted features, + and available community forks. +- Compare at least: upstream source build, OpenMaxIO UI pieces, Pigsty + MinIO fork, Garage, RustFS, SeaweedFS, and Ceph RGW. +- Decide whether "MaxIO" should be a direct fork, a packaging/build + distribution, a compatibility profile, or not pursued. + +## D7.2 - MinIO Compatibility Harness + +```task +id: ARTIFACT-STORE-WP-0007-T002 +status: todo +priority: high +state_hub_task_id: "c826f3ac-2ed7-4150-aa7c-e778ae71a72b" +``` + +Acceptance: + +- Restore or define the dependency/bootstrap path for MinIO-compatible + integration tests (`uv`/Python deps, Docker/testcontainers or a + deterministic compose fixture). +- Run artifact-store S3 backend tests against the selected MinIO or fork + target. +- Document manual smoke commands and expected health/verify outputs. + +## D7.3 - STS Credential Vending Assessment For NetKingdom + +```task +id: ARTIFACT-STORE-WP-0007-T003 +status: todo +priority: high +state_hub_task_id: "d3d5c4c1-d3b2-4163-b99d-1b08f90566d1" +``` + +Acceptance: + +- Inventory NetKingdom's current object-storage credential path, + including backup jobs and any S3/MinIO secrets. +- Determine whether Keycloak/Authelia/local-identity can act as the OIDC + identity provider for MinIO-compatible `AssumeRoleWithWebIdentity`. +- Produce a target architecture for credential vending: issuer, + token audience, role/policy mapping, expiration, revocation, audit, + and break-glass behavior. + +## D7.4 - Artifact-Store Temporary Credential Support + +```task +id: ARTIFACT-STORE-WP-0007-T004 +status: todo +priority: medium +state_hub_task_id: "9b80057a-d86e-4f14-9d14-928ee29f970d" +``` + +Acceptance: + +- Decide whether artifact-store's S3 backend needs dynamic credential + refresh for STS-vended credentials or whether refresh belongs in a + sidecar/secret controller. +- If needed, design the minimal configuration shape for short-lived + credentials without storing them in request bodies or event payloads. +- Verify that `artifactstore storage verify --backend s3` can run with + temporary credentials. + +## D7.5 - Follow-Up Workstream Routing + +```task +id: ARTIFACT-STORE-WP-0007-T005 +status: todo +priority: medium +state_hub_task_id: "614f7918-6fef-4460-b3fc-f9ff3c156422" +``` + +Acceptance: + +- Create or link NetKingdom follow-up work for STS credential vending if + the implementation belongs outside artifact-store. +- Create or link producer-side guide-board/open-cmis-tck work for the + missing `reports/cmis-summary.md` fragment. +- Close this workstream with a decision: adopt existing fork, build + MaxIO, use another S3-compatible store, or defer. + +## Success criteria + +- Artifact-store no longer treats MinIO as an incidental CI detail; it + has a clear compatibility and governance strategy. +- NetKingdom has a concrete answer on STS credential vending for object + storage. +- Any MaxIO fork work starts only after legal, security, governance, + and community-support duties are explicit.