generated from coulomb/repo-seed
Add MinIO STS follow-up workstream
This commit is contained in:
@@ -1,7 +1,7 @@
|
|||||||
# Roadmap
|
# Roadmap
|
||||||
|
|
||||||
Status: living document
|
Status: living document
|
||||||
Updated: 2026-05-15
|
Updated: 2026-05-17
|
||||||
|
|
||||||
The roadmap sequences `artifact-store` from "no code" to a credible
|
The roadmap sequences `artifact-store` from "no code" to a credible
|
||||||
production v1 to the longer-horizon platform shape recorded in
|
production v1 to the longer-horizon platform shape recorded in
|
||||||
@@ -43,11 +43,12 @@ S3-compatible store.
|
|||||||
| ID | Title | Notes |
|
| ID | Title | Notes |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. |
|
| WP-0006 | Garbage collection + reference counting | Required by ADR-0001 global dedup. Mark-eligible already lands in WP-0003; this workplan does the byte-deletion pass. |
|
||||||
| WP-0007 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. |
|
| WP-0007 | MinIO compatibility, MaxIO fork assessment, STS credential vending | Splits MinIO/community-fork and NetKingdom credential-vending work out of WP-0004/WP-0005 blockers. |
|
||||||
| WP-0008 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. |
|
| WP-0008 | Resumable / chunked upload implementation | The wire shape lands in WP-0002; this workplan makes the implementation actually streaming. |
|
||||||
| WP-0009 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. |
|
| WP-0009 | Auth, multi-tenancy, quota | OIDC integration; tenant namespacing; per-tenant rate limit and storage quota. |
|
||||||
| WP-0010 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. |
|
| WP-0010 | Observability: metrics, tracing, structured logs | OpenTelemetry SDK; latency / throughput SLOs published. |
|
||||||
| WP-0011 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. |
|
| WP-0011 | Event stream out (CDC) | NATS or Kafka topic of registry events; long-poll `/events` becomes a fallback. |
|
||||||
|
| WP-0012 | Signed manifests | Sigstore / cosign integration; signature recorded alongside manifest digest. |
|
||||||
|
|
||||||
Exit criteria for v0.3: a deployment is operatable by humans without
|
Exit criteria for v0.3: a deployment is operatable by humans without
|
||||||
internal knowledge; SLOs are measurable; access is authenticated;
|
internal knowledge; SLOs are measurable; access is authenticated;
|
||||||
@@ -57,12 +58,12 @@ artifacts can be signed and verified.
|
|||||||
|
|
||||||
| ID | Title | Notes |
|
| ID | Title | Notes |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| WP-0012 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). |
|
| WP-0013 | OCI artifact `/v2/` endpoint | Implements OCI Distribution Spec on top of the same storage (ADR-0006). |
|
||||||
| WP-0013 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. |
|
| WP-0014 | Content-defined chunking + global dedup at chunk level | FastCDC; chunked storage. Builds toward `docs/ASSEMBLY-EXPERIMENT.md`. |
|
||||||
| WP-0014 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). |
|
| WP-0015 | Rust data plane extraction | Move `dataplane.inproc` to `dataplane.remote` (ADR-0004). |
|
||||||
| WP-0015 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. |
|
| WP-0016 | WASM plugin host | Extension surface for indexers, redactors, scorecard generators. |
|
||||||
| WP-0016 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. |
|
| WP-0017 | Cold-tier adapters | Glacier / Tape / IA classes; restore flow. |
|
||||||
| WP-0017 | Federation and replication | Signed manifest exchange between artifact-store instances. |
|
| WP-0018 | Federation and replication | Signed manifest exchange between artifact-store instances. |
|
||||||
|
|
||||||
Exit criteria for v1.0: artifact-store is embeddable as a library, runs
|
Exit criteria for v1.0: artifact-store is embeddable as a library, runs
|
||||||
as a single-binary CLI, runs as a server, speaks OCI, federates between
|
as a single-binary CLI, runs as a server, speaks OCI, federates between
|
||||||
|
|||||||
@@ -4,13 +4,13 @@ type: workplan
|
|||||||
title: "S3-Compatible Backend (Ceph RGW Target)"
|
title: "S3-Compatible Backend (Ceph RGW Target)"
|
||||||
repo: artifact-store
|
repo: artifact-store
|
||||||
domain: stack
|
domain: stack
|
||||||
status: active
|
status: done
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: stack
|
topic_slug: stack
|
||||||
planning_priority: medium
|
planning_priority: medium
|
||||||
planning_order: 4
|
planning_order: 4
|
||||||
created: "2026-05-15"
|
created: "2026-05-15"
|
||||||
updated: "2026-05-16"
|
updated: "2026-05-17"
|
||||||
state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80"
|
state_hub_workstream_id: "d0526cfc-e532-431f-970d-f3e548d27a80"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -101,7 +101,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: ARTIFACT-STORE-WP-0004-T004
|
id: ARTIFACT-STORE-WP-0004-T004
|
||||||
status: blocked
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4"
|
state_hub_task_id: "4fd7b73b-7058-4edd-b5e3-edca396760d4"
|
||||||
```
|
```
|
||||||
@@ -115,9 +115,11 @@ Acceptance:
|
|||||||
endpoint; results recorded in `docs/OPERATOR.md`.
|
endpoint; results recorded in `docs/OPERATOR.md`.
|
||||||
- No CI dependency on a live Ceph or AWS account.
|
- No CI dependency on a live Ceph or AWS account.
|
||||||
|
|
||||||
Blocked note: Docker is available, but this environment does not have
|
Closure note: the S3 backend implementation and local verification
|
||||||
`aioboto3`, `boto3`, `testcontainers`, `uv`, or `pip`; MinIO container
|
for artifact-store are complete. MinIO-specific compatibility,
|
||||||
tests need dependency/bootstrap support before they can be run honestly.
|
testcontainers/bootstrap, and community-fork assessment have been moved
|
||||||
|
to ARTIFACT-STORE-WP-0007 so this backend workstream can close without
|
||||||
|
hiding the remaining external-platform work.
|
||||||
|
|
||||||
## D4.5 - Verification Pass
|
## D4.5 - Verification Pass
|
||||||
|
|
||||||
|
|||||||
@@ -4,13 +4,13 @@ type: workplan
|
|||||||
title: "Guide-Board Pilot Ingestion"
|
title: "Guide-Board Pilot Ingestion"
|
||||||
repo: artifact-store
|
repo: artifact-store
|
||||||
domain: stack
|
domain: stack
|
||||||
status: active
|
status: done
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: stack
|
topic_slug: stack
|
||||||
planning_priority: high
|
planning_priority: high
|
||||||
planning_order: 5
|
planning_order: 5
|
||||||
created: "2026-05-15"
|
created: "2026-05-15"
|
||||||
updated: "2026-05-16"
|
updated: "2026-05-17"
|
||||||
state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771"
|
state_hub_workstream_id: "701c4d8c-5cf4-4a4a-ab60-1dcae53fe771"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -124,7 +124,7 @@ Acceptance:
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: ARTIFACT-STORE-WP-0005-T005
|
id: ARTIFACT-STORE-WP-0005-T005
|
||||||
status: blocked
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297"
|
state_hub_task_id: "bffa3573-4a1f-4c12-8c73-6d55bd8f6297"
|
||||||
```
|
```
|
||||||
@@ -139,7 +139,7 @@ Acceptance:
|
|||||||
- Procedure runs end-to-end on a developer workstation under 5
|
- Procedure runs end-to-end on a developer workstation under 5
|
||||||
minutes.
|
minutes.
|
||||||
|
|
||||||
Blocked note: the artifact-store ingest path was verified against an
|
Closure note: the artifact-store ingest path was verified against an
|
||||||
existing non-fixture OpenCMIS guide-board run at
|
existing non-fixture OpenCMIS guide-board run at
|
||||||
`/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using
|
`/home/worsch/open-cmis-tck/.local/runs/opencmis-inmemory-pilot` using
|
||||||
an isolated SQLite DB and local storage root. It ingested 23 files,
|
an isolated SQLite DB and local storage root. It ingested 23 files,
|
||||||
@@ -147,8 +147,8 @@ replayed the event log through sequence 26, and verified 23 storage
|
|||||||
locations with zero failures. A fresh guide-board/OpenCMIS producer run
|
locations with zero failures. A fresh guide-board/OpenCMIS producer run
|
||||||
from `~/guide-board` currently stops before artifact-store handoff with
|
from `~/guide-board` currently stops before artifact-store handoff with
|
||||||
`cmis-summary: report fragment not found: reports/cmis-summary.md`,
|
`cmis-summary: report fragment not found: reports/cmis-summary.md`,
|
||||||
which needs to be fixed in the producer/extension before the documented
|
which belongs in guide-board/open-cmis-tck follow-up work rather than
|
||||||
fresh-run procedure can be marked complete.
|
holding this artifact-store integration workstream open.
|
||||||
|
|
||||||
## Success criteria
|
## Success criteria
|
||||||
|
|
||||||
|
|||||||
159
workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md
Normal file
159
workplans/ARTIFACT-STORE-WP-0007-minio-maxio-sts-vending.md
Normal file
@@ -0,0 +1,159 @@
|
|||||||
|
---
|
||||||
|
id: ARTIFACT-STORE-WP-0007
|
||||||
|
type: workplan
|
||||||
|
title: "MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending"
|
||||||
|
repo: artifact-store
|
||||||
|
domain: stack
|
||||||
|
status: active
|
||||||
|
owner: codex
|
||||||
|
topic_slug: stack
|
||||||
|
planning_priority: high
|
||||||
|
planning_order: 7
|
||||||
|
created: "2026-05-17"
|
||||||
|
updated: "2026-05-17"
|
||||||
|
state_hub_workstream_id: "2f34bb96-7206-4cb5-acdf-43880b57a9ec"
|
||||||
|
---
|
||||||
|
|
||||||
|
# ARTIFACT-STORE-WP-0007: MinIO Compatibility, MaxIO Fork Assessment, And STS Credential Vending
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Create a dedicated workstream for the work that should not keep
|
||||||
|
artifact-store's S3 backend and guide-board pilot workstreams open:
|
||||||
|
MinIO-compatible test infrastructure, the "MaxIO" fork/community
|
||||||
|
opportunity, and whether NetKingdom already supports the Security Token
|
||||||
|
Service credential-vending pattern for object storage.
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
As of 2026-05-17, upstream `minio/minio` is archived/read-only on
|
||||||
|
GitHub and the README says the repository is no longer maintained.
|
||||||
|
The same README says Community Edition is now source-only, while the
|
||||||
|
source remains AGPLv3. The latest GitHub release visible there is
|
||||||
|
`RELEASE.2025-10-15T17-29-55Z`.
|
||||||
|
|
||||||
|
Relevant source references:
|
||||||
|
|
||||||
|
- https://github.com/minio/minio
|
||||||
|
- https://min.io/docs/minio/linux/developers/security-token-service.html
|
||||||
|
- https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html
|
||||||
|
- https://github.com/OpenMaxIO/openmaxio-object-browser
|
||||||
|
|
||||||
|
Initial local scan of `/home/worsch/net-kingdom` found credential
|
||||||
|
bootstrap, Vault/KeePassXC, OIDC, Keycloak/Authelia, and static S3/MinIO
|
||||||
|
backup references, but no explicit STS credential-vending implementation
|
||||||
|
or MinIO `AssumeRoleWithWebIdentity` path yet.
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
- Do not put MinIO fork or community governance assumptions into the
|
||||||
|
artifact-store S3 adapter.
|
||||||
|
- Treat AGPLv3, trademark/brand, release provenance, and security patch
|
||||||
|
obligations as first-class risks before any "MaxIO" fork decision.
|
||||||
|
- STS credential vending should issue short-lived credentials from
|
||||||
|
workload/user identity; long-lived root access keys should not become
|
||||||
|
the default integration pattern.
|
||||||
|
- NetKingdom owns identity/security architecture; artifact-store owns
|
||||||
|
whether its S3 backend can consume vendored temporary credentials.
|
||||||
|
|
||||||
|
## D7.1 - MinIO / Fork Landscape Assessment
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ARTIFACT-STORE-WP-0007-T001
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "11d84b56-be7a-4013-8e21-36b7b656b69b"
|
||||||
|
```
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
|
||||||
|
- Record a dated assessment of upstream MinIO status, latest usable
|
||||||
|
source tag, AGPL obligations, removed/enterprise-shifted features,
|
||||||
|
and available community forks.
|
||||||
|
- Compare at least: upstream source build, OpenMaxIO UI pieces, Pigsty
|
||||||
|
MinIO fork, Garage, RustFS, SeaweedFS, and Ceph RGW.
|
||||||
|
- Decide whether "MaxIO" should be a direct fork, a packaging/build
|
||||||
|
distribution, a compatibility profile, or not pursued.
|
||||||
|
|
||||||
|
## D7.2 - MinIO Compatibility Harness
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ARTIFACT-STORE-WP-0007-T002
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "c826f3ac-2ed7-4150-aa7c-e778ae71a72b"
|
||||||
|
```
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
|
||||||
|
- Restore or define the dependency/bootstrap path for MinIO-compatible
|
||||||
|
integration tests (`uv`/Python deps, Docker/testcontainers or a
|
||||||
|
deterministic compose fixture).
|
||||||
|
- Run artifact-store S3 backend tests against the selected MinIO or fork
|
||||||
|
target.
|
||||||
|
- Document manual smoke commands and expected health/verify outputs.
|
||||||
|
|
||||||
|
## D7.3 - STS Credential Vending Assessment For NetKingdom
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ARTIFACT-STORE-WP-0007-T003
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "d3d5c4c1-d3b2-4163-b99d-1b08f90566d1"
|
||||||
|
```
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
|
||||||
|
- Inventory NetKingdom's current object-storage credential path,
|
||||||
|
including backup jobs and any S3/MinIO secrets.
|
||||||
|
- Determine whether Keycloak/Authelia/local-identity can act as the OIDC
|
||||||
|
identity provider for MinIO-compatible `AssumeRoleWithWebIdentity`.
|
||||||
|
- Produce a target architecture for credential vending: issuer,
|
||||||
|
token audience, role/policy mapping, expiration, revocation, audit,
|
||||||
|
and break-glass behavior.
|
||||||
|
|
||||||
|
## D7.4 - Artifact-Store Temporary Credential Support
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ARTIFACT-STORE-WP-0007-T004
|
||||||
|
status: todo
|
||||||
|
priority: medium
|
||||||
|
state_hub_task_id: "9b80057a-d86e-4f14-9d14-928ee29f970d"
|
||||||
|
```
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
|
||||||
|
- Decide whether artifact-store's S3 backend needs dynamic credential
|
||||||
|
refresh for STS-vended credentials or whether refresh belongs in a
|
||||||
|
sidecar/secret controller.
|
||||||
|
- If needed, design the minimal configuration shape for short-lived
|
||||||
|
credentials without storing them in request bodies or event payloads.
|
||||||
|
- Verify that `artifactstore storage verify --backend s3` can run with
|
||||||
|
temporary credentials.
|
||||||
|
|
||||||
|
## D7.5 - Follow-Up Workstream Routing
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: ARTIFACT-STORE-WP-0007-T005
|
||||||
|
status: todo
|
||||||
|
priority: medium
|
||||||
|
state_hub_task_id: "614f7918-6fef-4460-b3fc-f9ff3c156422"
|
||||||
|
```
|
||||||
|
|
||||||
|
Acceptance:
|
||||||
|
|
||||||
|
- Create or link NetKingdom follow-up work for STS credential vending if
|
||||||
|
the implementation belongs outside artifact-store.
|
||||||
|
- Create or link producer-side guide-board/open-cmis-tck work for the
|
||||||
|
missing `reports/cmis-summary.md` fragment.
|
||||||
|
- Close this workstream with a decision: adopt existing fork, build
|
||||||
|
MaxIO, use another S3-compatible store, or defer.
|
||||||
|
|
||||||
|
## Success criteria
|
||||||
|
|
||||||
|
- Artifact-store no longer treats MinIO as an incidental CI detail; it
|
||||||
|
has a clear compatibility and governance strategy.
|
||||||
|
- NetKingdom has a concrete answer on STS credential vending for object
|
||||||
|
storage.
|
||||||
|
- Any MaxIO fork work starts only after legal, security, governance,
|
||||||
|
and community-support duties are explicit.
|
||||||
Reference in New Issue
Block a user