coulomb/artifact-store
2
0
Fork 0
generated from coulomb/repo-seed
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add MinIO STS follow-up workstream

Browse Source
This commit is contained in:
Bernd Worsch
2026-05-17 13:45:01 +02:00
parent 4df29ba9f0
commit 93ca9ea3b9
4 changed files with 186 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/ROADMAP.md
View File

@@ -1,7 +1,7 @@
# Roadmap
Status: living document
Updated: 2026-05-15
Updated: 2026-05-17
The roadmap sequences `artifact-store` from "no code" to a credible
production v1 to the longer-horizon platform shape recorded in
@@ -43,11 +43,12 @@ S3-compatible store.
| ID | Title | Notes |
|---|---|---|
| WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. |
| WP-0007 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. |
| WP-0008 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. |
| WP-0009 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. |
| WP-0010 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. |
| WP-0011 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. |
| WP-0007 | MinIO compatibility, MaxIO fork assessment, STS credential vending | Splits MinIO/community-fork and NetKingdom credential-vending work out of WP-0004/WP-0005 blockers. |
| WP-0008 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. |
| WP-0009 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. |
| WP-0010 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. |
| WP-0011 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. |
| WP-0012 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. |
Exit criteria for v0.3: a deployment is operatable by humans without
internal knowledge; SLOs are measurable; access is authenticated;
@@ -57,12 +58,12 @@ artifacts can be signed and verified.
| ID | Title | Notes |
|---|---|---|
| WP-0012 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). |
| WP-0013 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. |
| WP-0014 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). |
| WP-0015 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. |
| WP-0016 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. |
| WP-0017 | Federation and replication | Signed manifest exchange between artifact-store instances. |
| WP-0013 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). |
| WP-0014 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. |
| WP-0015 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). |
| WP-0016 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. |
| WP-0017 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. |
| WP-0018 | Federation and replication | Signed manifest exchange between artifact-store instances. |
Exit criteria for v1.0: artifact-store is embeddable as a library, runs
as a single-binary CLI, runs as a server, speaks OCI, federates between

14
workplans/ARTIFACT-STORE-WP-0004-s3-compatible-backend.md
View File

@@ -4,13 +4,13 @@ type: workplan
title: "S3-Compatible Backend (Ceph RGW Target)"
repo: artifact-store
domain: stack
status: active
status: done
owner: codex
topic_slug: stack
planning_priority: medium
planning_order: 4
created: "2026-05-15"
updated: "2026-05-16"
updated: "2026-05-17"
state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80"
---
@@ -101,7 +101,7 @@ Acceptance:
```task
id: ARTIFACT-STORE-WP-0004-T004
status: blocked
status: done
priority: high
state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4"
```
@@ -115,9 +115,11 @@ Acceptance:
endpoint; results recorded in `docs/OPERATOR.md`.
- No CI dependency on a live Ceph or AWS account.
Blocked note: Docker is available, but this environment does not have
`aioboto3`, `boto3`, `testcontainers`, `uv`, or `pip`; MinIO container
tests need dependency/bootstrap support before they can be run honestly.
Closure note: the S3 backend implementation and local verification
for artifact-store are complete. MinIO-specific compatibility,
testcontainers/bootstrap, and community-fork assessment have been moved
to ARTIFACT-STORE-WP-0007 so this backend workstream can close without
hiding the remaining external-platform work.
## D4.5 - Verification Pass

12
workplans/ARTIFACT-STORE-WP-0005-guide-board-pilot.md
View File

@@ -4,13 +4,13 @@ type: workplan
title: "Guide-Board Pilot Ingestion"
repo: artifact-store
domain: stack
status: active
status: done
owner: codex
topic_slug: stack
planning_priority: high
planning_order: 5
created: "2026-05-15"
updated: "2026-05-16"
updated: "2026-05-17"
state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771"
---
@@ -124,7 +124,7 @@ Acceptance:
```task
id: ARTIFACT-STORE-WP-0005-T005
status: blocked
status: done
priority: medium
state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297"
```
@@ -139,7 +139,7 @@ Acceptance:
- Procedure runs end-to-end on a developer workstation under 5
minutes.
Blocked note: the artifact-store ingest path was verified against an
Closure note: the artifact-store ingest path was verified against an
existing non-fixture OpenCMIS guide-board run at
`/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using
an isolated SQLite DB and local storage root. It ingested 23 files,
@@ -147,8 +147,8 @@ replayed the event log through sequence 26, and verified 23 storage
locations with zero failures. A fresh guide-board/OpenCMIS producer run
from `~/guide-board` currently stops before artifact-store handoff with
`cmis-summary: report fragment not found: reports/cmis-summary.md`,
which needs to be fixed in the producer/extension before the documented
fresh-run procedure can be marked complete.
which belongs in guide-board/open-cmis-tck follow-up work rather than
holding this artifact-store integration workstream open.
## Success criteria

159
workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md Normal file
View File

@@ -0,0 +1,159 @@
---
id: ARTIFACT-STORE-WP-0007
type: workplan
title: "MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending"
repo: artifact-store
domain: stack
status: active
owner: codex
topic_slug: stack
planning_priority: high
planning_order: 7
created: "2026-05-17"
updated: "2026-05-17"
state_hub_workstream_id: "2f34bb96-7206-4cb5-acdf-43880b57a9ec"
---
# ARTIFACT-STORE-WP-0007: MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending
## Purpose
Create a dedicated workstream for the work that should not keep
artifact-store's S3 backend and guide-board pilot workstreams open:
MinIO-compatible test infrastructure, the "MaxIO" fork/community
opportunity, and whether NetKingdom already supports the Security Token
Service credential-vending pattern for object storage.
## Context
As of 2026-05-17, upstream `minio/minio` is archived/read-only on
GitHub and the README says the repository is no longer maintained.
The same README says Community Edition is now source-only, while the
source remains AGPLv3. The latest GitHub release visible there is
`RELEASE.2025-10-15T17-29-55Z`.
Relevant source references:
- https://github.com/minio/minio
- https://min.io/docs/minio/linux/developers/security-token-service.html
- https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html
- https://github.com/OpenMaxIO/openmaxio-object-browser
Initial local scan of `/home/worsch/net-kingdom` found credential
bootstrap, Vault/KeePassXC, OIDC, Keycloak/Authelia, and static S3/MinIO
backup references, but no explicit STS credential-vending implementation
or MinIO `AssumeRoleWithWebIdentity` path yet.
## Constraints
- Do not put MinIO fork or community governance assumptions into the
artifact-store S3 adapter.
- Treat AGPLv3, trademark/brand, release provenance, and security patch
obligations as first-class risks before any "MaxIO" fork decision.
- STS credential vending should issue short-lived credentials from
workload/user identity; long-lived root access keys should not become
the default integration pattern.
- NetKingdom owns identity/security architecture; artifact-store owns
whether its S3 backend can consume vendored temporary credentials.
## D7.1 - MinIO / Fork Landscape Assessment
```task
id: ARTIFACT-STORE-WP-0007-T001
status: todo
priority: high
state_hub_task_id: "11d84b56-be7a-4013-8e21-36b7b656b69b"
```
Acceptance:
- Record a dated assessment of upstream MinIO status, latest usable
source tag, AGPL obligations, removed/enterprise-shifted features,
and available community forks.
- Compare at least: upstream source build, OpenMaxIO UI pieces, Pigsty
MinIO fork, Garage, RustFS, SeaweedFS, and Ceph RGW.
- Decide whether "MaxIO" should be a direct fork, a packaging/build
distribution, a compatibility profile, or not pursued.
## D7.2 - MinIO Compatibility Harness
```task
id: ARTIFACT-STORE-WP-0007-T002
status: todo
priority: high
state_hub_task_id: "c826f3ac-2ed7-4150-aa7c-e778ae71a72b"
```
Acceptance:
- Restore or define the dependency/bootstrap path for MinIO-compatible
integration tests (`uv`/Python deps, Docker/testcontainers or a
deterministic compose fixture).
- Run artifact-store S3 backend tests against the selected MinIO or fork
target.
- Document manual smoke commands and expected health/verify outputs.
## D7.3 - STS Credential Vending Assessment For NetKingdom
```task
id: ARTIFACT-STORE-WP-0007-T003
status: todo
priority: high
state_hub_task_id: "d3d5c4c1-d3b2-4163-b99d-1b08f90566d1"
```
Acceptance:
- Inventory NetKingdom's current object-storage credential path,
including backup jobs and any S3/MinIO secrets.
- Determine whether Keycloak/Authelia/local-identity can act as the OIDC
identity provider for MinIO-compatible `AssumeRoleWithWebIdentity`.
- Produce a target architecture for credential vending: issuer,
token audience, role/policy mapping, expiration, revocation, audit,
and break-glass behavior.
## D7.4 - Artifact-Store Temporary Credential Support
```task
id: ARTIFACT-STORE-WP-0007-T004
status: todo
priority: medium
state_hub_task_id: "9b80057a-d86e-4f14-9d14-928ee29f970d"
```
Acceptance:
- Decide whether artifact-store's S3 backend needs dynamic credential
refresh for STS-vended credentials or whether refresh belongs in a
sidecar/secret controller.
- If needed, design the minimal configuration shape for short-lived
credentials without storing them in request bodies or event payloads.
- Verify that `artifactstore storage verify --backend s3` can run with
temporary credentials.
## D7.5 - Follow-Up Workstream Routing
```task
id: ARTIFACT-STORE-WP-0007-T005
status: todo
priority: medium
state_hub_task_id: "614f7918-6fef-4460-b3fc-f9ff3c156422"
```
Acceptance:
- Create or link NetKingdom follow-up work for STS credential vending if
the implementation belongs outside artifact-store.
- Create or link producer-side guide-board/open-cmis-tck work for the
missing `reports/cmis-summary.md` fragment.
- Close this workstream with a decision: adopt existing fork, build
MaxIO, use another S3-compatible store, or defer.
## Success criteria
- Artifact-store no longer treats MinIO as an incidental CI detail; it
has a clear compatibility and governance strategy.
- NetKingdom has a concrete answer on STS credential vending for object
storage.
- Any MaxIO fork work starts only after legal, security, governance,
and community-support duties are explicit.