Add MinIO STS follow-up workstream

This commit is contained in:
2026-05-17 13:45:01 +02:00
parent 4df29ba9f0
commit 93ca9ea3b9
4 changed files with 186 additions and 24 deletions

View File

@@ -1,7 +1,7 @@
# Roadmap # Roadmap
Status: living document Status: living document
Updated: 2026-05-15 Updated: 2026-05-17
The roadmap sequences `artifact-store` from "no code" to a credible The roadmap sequences `artifact-store` from "no code" to a credible
production v1 to the longer-horizon platform shape recorded in production v1 to the longer-horizon platform shape recorded in
@@ -43,11 +43,12 @@ S3-compatible store.
| ID | Title | Notes | | ID | Title | Notes |
|---|---|---| |---|---|---|
| WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. | | WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. |
| WP-0007 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. | | WP-0007 | MinIO compatibility, MaxIO fork assessment, STS credential vending | Splits MinIO/community-fork and NetKingdom credential-vending work out of WP-0004/WP-0005 blockers. |
| WP-0008 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. | | WP-0008 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. |
| WP-0009 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. | | WP-0009 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. |
| WP-0010 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. | | WP-0010 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. |
| WP-0011 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. | | WP-0011 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. |
| WP-0012 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. |
Exit criteria for v0.3: a deployment is operatable by humans without Exit criteria for v0.3: a deployment is operatable by humans without
internal knowledge; SLOs are measurable; access is authenticated; internal knowledge; SLOs are measurable; access is authenticated;
@@ -57,12 +58,12 @@ artifacts can be signed and verified.
| ID | Title | Notes | | ID | Title | Notes |
|---|---|---| |---|---|---|
| WP-0012 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). | | WP-0013 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). |
| WP-0013 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. | | WP-0014 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. |
| WP-0014 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). | | WP-0015 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). |
| WP-0015 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. | | WP-0016 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. |
| WP-0016 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. | | WP-0017 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. |
| WP-0017 | Federation and replication | Signed manifest exchange between artifact-store instances. | | WP-0018 | Federation and replication | Signed manifest exchange between artifact-store instances. |
Exit criteria for v1.0: artifact-store is embeddable as a library, runs Exit criteria for v1.0: artifact-store is embeddable as a library, runs
as a single-binary CLI, runs as a server, speaks OCI, federates between as a single-binary CLI, runs as a server, speaks OCI, federates between

View File

@@ -4,13 +4,13 @@ type: workplan
title: "S3-Compatible Backend (Ceph RGW Target)" title: "S3-Compatible Backend (Ceph RGW Target)"
repo: artifact-store repo: artifact-store
domain: stack domain: stack
status: active status: done
owner: codex owner: codex
topic_slug: stack topic_slug: stack
planning_priority: medium planning_priority: medium
planning_order: 4 planning_order: 4
created: "2026-05-15" created: "2026-05-15"
updated: "2026-05-16" updated: "2026-05-17"
state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80" state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80"
--- ---
@@ -101,7 +101,7 @@ Acceptance:
```task ```task
id: ARTIFACT-STORE-WP-0004-T004 id: ARTIFACT-STORE-WP-0004-T004
status: blocked status: done
priority: high priority: high
state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4" state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4"
``` ```
@@ -115,9 +115,11 @@ Acceptance:
endpoint; results recorded in `docs/OPERATOR.md`. endpoint; results recorded in `docs/OPERATOR.md`.
- No CI dependency on a live Ceph or AWS account. - No CI dependency on a live Ceph or AWS account.
Blocked note: Docker is available, but this environment does not have Closure note: the S3 backend implementation and local verification
`aioboto3`, `boto3`, `testcontainers`, `uv`, or `pip`; MinIO container for artifact-store are complete. MinIO-specific compatibility,
tests need dependency/bootstrap support before they can be run honestly. testcontainers/bootstrap, and community-fork assessment have been moved
to ARTIFACT-STORE-WP-0007 so this backend workstream can close without
hiding the remaining external-platform work.
## D4.5 - Verification Pass ## D4.5 - Verification Pass

View File

@@ -4,13 +4,13 @@ type: workplan
title: "Guide-Board Pilot Ingestion" title: "Guide-Board Pilot Ingestion"
repo: artifact-store repo: artifact-store
domain: stack domain: stack
status: active status: done
owner: codex owner: codex
topic_slug: stack topic_slug: stack
planning_priority: high planning_priority: high
planning_order: 5 planning_order: 5
created: "2026-05-15" created: "2026-05-15"
updated: "2026-05-16" updated: "2026-05-17"
state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771" state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771"
--- ---
@@ -124,7 +124,7 @@ Acceptance:
```task ```task
id: ARTIFACT-STORE-WP-0005-T005 id: ARTIFACT-STORE-WP-0005-T005
status: blocked status: done
priority: medium priority: medium
state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297" state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297"
``` ```
@@ -139,7 +139,7 @@ Acceptance:
- Procedure runs end-to-end on a developer workstation under 5 - Procedure runs end-to-end on a developer workstation under 5
minutes. minutes.
Blocked note: the artifact-store ingest path was verified against an Closure note: the artifact-store ingest path was verified against an
existing non-fixture OpenCMIS guide-board run at existing non-fixture OpenCMIS guide-board run at
`/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using `/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using
an isolated SQLite DB and local storage root. It ingested 23 files, an isolated SQLite DB and local storage root. It ingested 23 files,
@@ -147,8 +147,8 @@ replayed the event log through sequence 26, and verified 23 storage
locations with zero failures. A fresh guide-board/OpenCMIS producer run locations with zero failures. A fresh guide-board/OpenCMIS producer run
from `~/guide-board` currently stops before artifact-store handoff with from `~/guide-board` currently stops before artifact-store handoff with
`cmis-summary: report fragment not found: reports/cmis-summary.md`, `cmis-summary: report fragment not found: reports/cmis-summary.md`,
which needs to be fixed in the producer/extension before the documented which belongs in guide-board/open-cmis-tck follow-up work rather than
fresh-run procedure can be marked complete. holding this artifact-store integration workstream open.
## Success criteria ## Success criteria

View File

@@ -0,0 +1,159 @@
---
id: ARTIFACT-STORE-WP-0007
type: workplan
title: "MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending"
repo: artifact-store
domain: stack
status: active
owner: codex
topic_slug: stack
planning_priority: high
planning_order: 7
created: "2026-05-17"
updated: "2026-05-17"
state_hub_workstream_id: "2f34bb96-7206-4cb5-acdf-43880b57a9ec"
---
# ARTIFACT-STORE-WP-0007: MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending
## Purpose
Create a dedicated workstream for the work that should not keep
artifact-store's S3 backend and guide-board pilot workstreams open:
MinIO-compatible test infrastructure, the "MaxIO" fork/community
opportunity, and whether NetKingdom already supports the Security Token
Service credential-vending pattern for object storage.
## Context
As of 2026-05-17, upstream `minio/minio` is archived/read-only on
GitHub and the README says the repository is no longer maintained.
The same README says Community Edition is now source-only, while the
source remains AGPLv3. The latest GitHub release visible there is
`RELEASE.2025-10-15T17-29-55Z`.
Relevant source references:
- https://github.com/minio/minio
- https://min.io/docs/minio/linux/developers/security-token-service.html
- https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html
- https://github.com/OpenMaxIO/openmaxio-object-browser
Initial local scan of `/home/worsch/net-kingdom` found credential
bootstrap, Vault/KeePassXC, OIDC, Keycloak/Authelia, and static S3/MinIO
backup references, but no explicit STS credential-vending implementation
or MinIO `AssumeRoleWithWebIdentity` path yet.
## Constraints
- Do not put MinIO fork or community governance assumptions into the
artifact-store S3 adapter.
- Treat AGPLv3, trademark/brand, release provenance, and security patch
obligations as first-class risks before any "MaxIO" fork decision.
- STS credential vending should issue short-lived credentials from
workload/user identity; long-lived root access keys should not become
the default integration pattern.
- NetKingdom owns identity/security architecture; artifact-store owns
whether its S3 backend can consume vendored temporary credentials.
## D7.1 - MinIO / Fork Landscape Assessment
```task
id: ARTIFACT-STORE-WP-0007-T001
status: todo
priority: high
state_hub_task_id: "11d84b56-be7a-4013-8e21-36b7b656b69b"
```
Acceptance:
- Record a dated assessment of upstream MinIO status, latest usable
source tag, AGPL obligations, removed/enterprise-shifted features,
and available community forks.
- Compare at least: upstream source build, OpenMaxIO UI pieces, Pigsty
MinIO fork, Garage, RustFS, SeaweedFS, and Ceph RGW.
- Decide whether "MaxIO" should be a direct fork, a packaging/build
distribution, a compatibility profile, or not pursued.
## D7.2 - MinIO Compatibility Harness
```task
id: ARTIFACT-STORE-WP-0007-T002
status: todo
priority: high
state_hub_task_id: "c826f3ac-2ed7-4150-aa7c-e778ae71a72b"
```
Acceptance:
- Restore or define the dependency/bootstrap path for MinIO-compatible
integration tests (`uv`/Python deps, Docker/testcontainers or a
deterministic compose fixture).
- Run artifact-store S3 backend tests against the selected MinIO or fork
target.
- Document manual smoke commands and expected health/verify outputs.
## D7.3 - STS Credential Vending Assessment For NetKingdom
```task
id: ARTIFACT-STORE-WP-0007-T003
status: todo
priority: high
state_hub_task_id: "d3d5c4c1-d3b2-4163-b99d-1b08f90566d1"
```
Acceptance:
- Inventory NetKingdom's current object-storage credential path,
including backup jobs and any S3/MinIO secrets.
- Determine whether Keycloak/Authelia/local-identity can act as the OIDC
identity provider for MinIO-compatible `AssumeRoleWithWebIdentity`.
- Produce a target architecture for credential vending: issuer,
token audience, role/policy mapping, expiration, revocation, audit,
and break-glass behavior.
## D7.4 - Artifact-Store Temporary Credential Support
```task
id: ARTIFACT-STORE-WP-0007-T004
status: todo
priority: medium
state_hub_task_id: "9b80057a-d86e-4f14-9d14-928ee29f970d"
```
Acceptance:
- Decide whether artifact-store's S3 backend needs dynamic credential
refresh for STS-vended credentials or whether refresh belongs in a
sidecar/secret controller.
- If needed, design the minimal configuration shape for short-lived
credentials without storing them in request bodies or event payloads.
- Verify that `artifactstore storage verify --backend s3` can run with
temporary credentials.
## D7.5 - Follow-Up Workstream Routing
```task
id: ARTIFACT-STORE-WP-0007-T005
status: todo
priority: medium
state_hub_task_id: "614f7918-6fef-4460-b3fc-f9ff3c156422"
```
Acceptance:
- Create or link NetKingdom follow-up work for STS credential vending if
the implementation belongs outside artifact-store.
- Create or link producer-side guide-board/open-cmis-tck work for the
missing `reports/cmis-summary.md` fragment.
- Close this workstream with a decision: adopt existing fork, build
MaxIO, use another S3-compatible store, or defer.
## Success criteria
- Artifact-store no longer treats MinIO as an incidental CI detail; it
has a clear compatibility and governance strategy.
- NetKingdom has a concrete answer on STS credential vending for object
storage.
- Any MaxIO fork work starts only after legal, security, governance,
and community-support duties are explicit.