net-kingdom/docs/object-storage-sts-credential-vending.md

# Object Storage STS Credential Vending

Status: architecture baseline for NK-WP-0007
Date: 2026-05-18

## Purpose

This document defines the NetKingdom pattern for vending short-lived
object-storage credentials from verified identity and policy decisions.
It is provider-neutral at the NetKingdom boundary and provider-aware at
the backend exchange boundary.

The goal is to let consumers such as `artifact-store` use S3-compatible
temporary credentials without owning identity, authorization, secret
custody, or object-storage root credentials.

## Ownership Boundary

| Capability | Owner |
| --- | --- |
| IAM Profile, issuer and claim requirements | NetKingdom |
| Resource/action vocabulary and policy decision envelope | flex-auth, governed by NetKingdom architecture |
| Delegated PDP runtime | Topaz first, behind flex-auth |
| Runtime secret custody, broker configuration, audit, leases | OpenBao, deployed by Railiance platform |
| Object-storage backend configuration | Railiance platform |
| Artifact package behavior and S3 client refresh behavior | artifact-store |
| Application deployment | Railiance apps or the owning application repo |

OpenBao may store parent credentials, broker configuration, or issued
credential metadata where appropriate. It does not replace flex-auth as
the authorization decision point and must not become the object-storage
policy model.

## Core Flow

```text
Human, service, or agent principal
        |
        v
NetKingdom IAM Profile token
  key-cape lightweight mode or Keycloak expanded mode
        |
        v
credential-vending service
  verifies issuer, audience, subject, assurance, tenant
        |
        v
flex-auth decision
  tenant, protected-system, bucket, prefix, actions, TTL, obligations
        |
        v
backend exchange
  AWS STS, Ceph RGW STS, MinIO/AIStor STS, Cloudflare R2 temp API,
  or OpenBao-assisted broker path
        |
        v
temporary S3 credentials
  access key id, secret access key, session token, expiration
        |
        v
consumer
  artifact-store, SDK, CLI, sidecar, controller, or batch job
```

## Trust Boundaries

### Platform Control Plane

`tenant:platform` administers the credential-vending service, approved
issuer list, flex-auth policy import pipeline, OpenBao mounts/auth
methods, backend parent credentials, audit retention, and emergency
recovery.

### Tenant Plane

`tenant:coulomb` and later tenants may request scoped credentials for
registered tenant resources. Tenant administrators must not receive
OpenBao root tokens, object-storage root credentials, global backend STS
configuration, or platform policy import authority.

### Backend Boundary

The credential-vending service is the only component that exchanges an
approved decision for provider-native credentials. Consumers receive only
short-lived credentials scoped to the approved bucket, prefix, actions,
and TTL.

## Token And Decision Flow

1. The caller authenticates through a NetKingdom IAM Profile
   implementation.
2. The caller sends a request to the credential-vending service with a
   bearer token or a workload identity binding.
3. The service validates issuer, audience, signature, expiration,
   subject, tenant claim, and assurance evidence.
4. The service builds a flex-auth request with the protected-system id,
   resource, action set, requested TTL, tenant, actor, and context.
5. flex-auth evaluates policy through its standalone evaluator or a
   delegated PDP such as Topaz.
6. If denied, the service returns a deny envelope with a stable reason
   code and audit correlation id.
7. If allowed, the service exchanges the approved request with the
   backend or OpenBao-assisted broker path.
8. The service returns normalized temporary credentials and records
   identity, policy, backend, lease, and audit metadata.

## Resource Model

Every object-storage resource belongs to a protected system and tenant.

Suggested identifiers:

```text
protected_system:object-storage:artifact-store-prod
tenant:platform
tenant:coulomb

bucket:artifact-store-prod
prefix:tenant/coulomb/packages/
object:tenant/coulomb/packages/<digest>
```

The protected-system id names the storage integration boundary, not just
the backend product. For example, a MinIO tenant and an AWS bucket used
by the same application should still be distinct protected systems if
their trust, audit, or policy lifecycle differs.

## flex-auth Vocabulary

| Resource | Example | Notes |
| --- | --- | --- |
| protected system | `object-storage:artifact-store-prod` | Required in every decision |
| bucket | `bucket:artifact-store-prod` | Coarse storage boundary |
| prefix | `prefix:tenant/coulomb/packages/` | Preferred grant boundary for workloads |
| object | `object:tenant/coulomb/packages/a.tar.zst` | Use for exceptional single-object decisions |

Canonical action names:

| Action | Meaning |
| --- | --- |
| `s3:GetObject` | Read object data |
| `s3:PutObject` | Create or replace object data |
| `s3:DeleteObject` | Delete object data |
| `s3:ListBucket` | List bucket or prefix contents |
| `s3:GetObjectAttributes` | Read metadata, checksums, or object attributes |
| `s3:AbortMultipartUpload` | Abort multipart state |
| `s3:CreateMultipartUpload` | Start multipart upload |
| `s3:UploadPart` | Upload multipart chunk |
| `s3:CompleteMultipartUpload` | Complete multipart upload |

Required decision inputs:

- subject id, subject type, issuer, audience, and tenant;
- protected-system id;
- bucket and prefix or object;
- requested action set;
- requested TTL;
- assurance level and MFA evidence where privileged or destructive
  actions are requested;
- workload identity evidence for service or agent callers;
- request purpose and audit correlation id when available.

Required decision outputs:

- allow or deny;
- maximum TTL;
- permitted actions;
- permitted bucket and prefix/object scope;
- obligations such as read-only, checksum-required, write-once, or
  audit-detail-required;
- deny reason code;
- explanation/audit correlation id;
- backend exchange hint where policy deliberately restricts backend use.

TTL policy:

- default interactive TTL: 15 minutes;
- default workload TTL: 30 minutes;
- maximum normal TTL: 1 hour;
- longer TTLs require explicit policy and should not exceed backend
  limits;
- destructive or platform-scoped credentials should use shorter TTLs and
  MFA or dual-control obligations.

## IAM Profile Requirements

Accepted issuers:

- key-cape lightweight mode for local, sandbox, and small deployments;
- Keycloak expanded mode for production and enterprise federation;
- local-identity only for development or bootstrap contexts explicitly
  marked non-production.

Required token properties:

- `iss` matches an approved NetKingdom issuer;
- `aud` targets the credential-vending service or an approved backend
  exchange audience;
- `sub` is stable for the principal;
- `exp`, `nbf`, and `iat` are present and within skew tolerance;
- `tenant` or equivalent tenant mapping is present for tenant-scoped
  requests;
- service accounts and agents are distinguishable from humans;
- assurance/MFA claims are present when policy needs them;
- groups or roles are mapped through IAM Profile semantics, not
  provider-specific bucket policy.

Local-dev restrictions:

- local issuers must only be accepted by explicitly configured dev
  vending instances;
- local issuer tokens must not be trusted by production backends;
- credentials minted from local issuers must be restricted to local or
  sandbox object stores.

Emergency principals:

- break-glass use is platform-control-plane access, not tenant access;
- emergency credentials must be short-lived where possible;
- every emergency vending event requires a post-event review record.

## Backend Assessment

| Backend | Temporary credential path | NetKingdom stance |
| --- | --- | --- |
| AWS S3 | AWS STS `AssumeRoleWithWebIdentity` returns access key id, secret access key, session token, and expiration | Best fit for AWS-native deployments. Use IAM OIDC provider and role trust policies, with flex-auth deciding before exchange. |
| Ceph RGW | RGW implements a subset of STS, including `AssumeRoleWithWebIdentity` for OIDC-backed temporary credentials | Good fit for self-hosted S3-compatible storage when RGW IAM/STS maturity is acceptable for the deployment. |
| MinIO/AIStor | MinIO STS supports `AssumeRoleWithWebIdentity` with OIDC JWTs and AWS-like response semantics | Strong fit for lightweight/self-hosted deployments if session-token support is wired through consumers. |
| Cloudflare R2 | R2 temporary credentials are created through the R2 Temporary Credentials API or local signing with parent access material | Use a backend-specific broker. Store parent material in OpenBao; do not expose parent credentials to workloads. |
| OpenBao | Can store parent credentials, broker dynamic material, record leases, and audit secret access | Runtime secret infrastructure and audit point, not the canonical object-storage authorization engine. |

Decision summary: prefer provider-native temporary credentials when the
backend has a mature STS or temporary-credentials API. Keep the
NetKingdom interface stable and normalize backend differences in the
credential-vending service.

## OpenBao Role

OpenBao participates in credential vending only after flex-auth approval.
Allowed OpenBao responsibilities:

- store backend parent credentials for Cloudflare R2 or other APIs that
  need privileged signing material;
- store broker configuration and backend endpoint metadata;
- issue or lease dynamic credentials where a supported backend plugin or
  controlled broker path exists;
- provide audit records for parent credential access and broker
  operations;
- deliver credential-vending service configuration through Kubernetes
  auth, CSI, or External Secrets Operator.

Prohibited OpenBao responsibilities:

- deciding whether a tenant may access a bucket or prefix;
- storing tenant policy as the canonical object-storage authorization
  model;
- exposing platform mounts, root tokens, unseal/recovery material, or
  parent credentials to tenants;
- bypassing flex-auth because a backend secret path is readable.

## Interface Prototype

HTTP request:

```http
POST /v1/object-storage/credentials
Authorization: Bearer <iam-profile-token>
Content-Type: application/json
```

```json
{
  "protected_system_id": "object-storage:artifact-store-prod",
  "tenant_id": "tenant:coulomb",
  "bucket": "artifact-store-prod",
  "prefix": "tenant/coulomb/packages/",
  "actions": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"],
  "ttl_seconds": 1800,
  "purpose": "artifact-store package upload",
  "correlation_id": "01JYNETKINGDOMSTS000000000001"
}
```

Normalized response:

```json
{
  "credentials": {
    "access_key_id": "AKIA...",
    "secret_access_key": "redacted-by-client-logging",
    "session_token": "token...",
    "expiration": "2026-05-18T16:45:00Z"
  },
  "scope": {
    "protected_system_id": "object-storage:artifact-store-prod",
    "tenant_id": "tenant:coulomb",
    "bucket": "artifact-store-prod",
    "prefix": "tenant/coulomb/packages/",
    "actions": ["s3:GetObject", "s3:PutObject", "s3:ListBucket"]
  },
  "lease": {
    "ttl_seconds": 1800,
    "renewable": false,
    "backend": "minio-assume-role-with-web-identity",
    "openbao_lease_id": null
  },
  "decision": {
    "decision_id": "dec_01JYNETKINGDOMSTS000000000001",
    "policy_package": "object-storage-artifact-store-prod@2026-05-18",
    "obligations": ["checksum-required"],
    "audit_correlation_id": "01JYNETKINGDOMSTS000000000001"
  }
}
```

Deny response:

```json
{
  "error": "credential_denied",
  "reason_code": "prefix_not_registered_for_tenant",
  "decision_id": "dec_01JYNETKINGDOMSTS000000000002",
  "audit_correlation_id": "01JYNETKINGDOMSTS000000000002"
}
```

`credential_process` output for SDK consumers:

```json
{
  "Version": 1,
  "AccessKeyId": "AKIA...",
  "SecretAccessKey": "...",
  "SessionToken": "...",
  "Expiration": "2026-05-18T16:45:00Z"
}
```

CLI shape:

```bash
netkingdom-object-creds vend \
  --protected-system object-storage:artifact-store-prod \
  --tenant tenant:coulomb \
  --bucket artifact-store-prod \
  --prefix tenant/coulomb/packages/ \
  --action s3:GetObject \
  --action s3:PutObject \
  --ttl 1800 \
  --credential-process
```

## Audit Event

Each successful or denied request should emit one canonical audit event:

```json
{
  "event_type": "object_storage_credential_vending",
  "outcome": "allowed",
  "actor": {
    "subject": "service:artifact-store",
    "issuer": "https://kc.coulomb.social",
    "tenant": "tenant:coulomb",
    "assurance": "workload"
  },
  "request": {
    "protected_system_id": "object-storage:artifact-store-prod",
    "bucket": "artifact-store-prod",
    "prefix": "tenant/coulomb/packages/",
    "actions": ["s3:GetObject", "s3:PutObject"],
    "ttl_seconds": 1800
  },
  "decision": {
    "decision_id": "dec_01JYNETKINGDOMSTS000000000001",
    "policy_package": "object-storage-artifact-store-prod@2026-05-18"
  },
  "backend": {
    "type": "minio-assume-role-with-web-identity",
    "credential_expiration": "2026-05-18T16:45:00Z",
    "openbao_lease_id": null
  }
}
```

OpenBao audit events should be correlated when OpenBao parent material,
broker config, dynamic secret engines, or delivery paths are used.

## Consumer Guidance

### artifact-store

`artifact-store` should consume temporary credentials without owning the
vending authority.

Required consumer support:

- `AWS_ACCESS_KEY_ID`;
- `AWS_SECRET_ACCESS_KEY`;
- `AWS_SESSION_TOKEN`;
- credential expiration awareness;
- refresh before expiration, preferably with jitter;
- env, file, sidecar, controller, or `credential_process` delivery.

The existing static bridge can remain transitional:

```bash
export ARTIFACTSTORE_S3_ACCESS_KEY_REF=file:/run/secrets/artifactstore/s3-access-key
export ARTIFACTSTORE_S3_SECRET_KEY_REF=file:/run/secrets/artifactstore/s3-secret-key
```

Temporary credentials require either a session-token ref or a refresh
pattern that updates all three credential values atomically:

```bash
export ARTIFACTSTORE_S3_ACCESS_KEY_REF=file:/run/secrets/artifactstore/aws-access-key-id
export ARTIFACTSTORE_S3_SECRET_KEY_REF=file:/run/secrets/artifactstore/aws-secret-access-key
export ARTIFACTSTORE_S3_SESSION_TOKEN_REF=file:/run/secrets/artifactstore/aws-session-token
export ARTIFACTSTORE_S3_CREDENTIAL_EXPIRATION_REF=file:/run/secrets/artifactstore/expiration
```

Recommended deployment patterns:

- CLI or SDK `credential_process` for developer and batch use;
- sidecar refresh process for pods that cannot call the vending API
  directly;
- controller plus mounted files when platform operators need centralized
  refresh and audit;
- direct vending API call only when the workload can protect its IAM
  token and handle refresh safely.

### Other S3 Consumers

Consumers must support the session token. Access-key/secret-key-only
clients are limited to transitional static credentials and should not be
used for production tenant workloads.

Prohibited patterns:

- object-store root credentials in application pods;
- long-lived tenant access keys for normal workload traffic;
- bucket policy managed by application repos as the source of truth;
- storing parent R2/API credentials in tenant namespaces;
- ignoring credential expiration and retrying indefinitely with expired
  credentials;
- accepting local-identity tokens in production.

## Failure Modes

| Failure | Expected behavior |
| --- | --- |
| IAM token invalid or wrong audience | Deny before policy evaluation; emit audit event |
| Tenant missing or mismatched | Deny with `tenant_scope_missing` or `tenant_mismatch` |
| Prefix not registered | Deny with `prefix_not_registered_for_tenant` |
| TTL too long | Reduce to policy maximum or deny, depending on policy |
| flex-auth or Topaz unavailable | Fail closed except for explicitly documented emergency platform workflows |
| Backend STS unavailable | Do not mint credentials; return retryable backend error |
| OpenBao unavailable | Fail if parent material or broker config requires OpenBao; otherwise continue only for backend paths that do not depend on it |
| Audit sink unavailable | Deny privileged/platform-scoped requests; allow low-risk tenant requests only if policy permits buffered audit |
| Consumer refresh fails | Stop writes before expiration; retry vending with backoff; never fall back to root credentials |

## Readiness Checks

- IAM Profile token validation test passes for key-cape or Keycloak.
- flex-auth has policy packages for platform and tenant scopes.
- Topaz policy load and health are verified where delegated PDP is used.
- Backend-specific STS or temporary credential path returns credentials
  with session token and expiration.
- OpenBao parent credential access, lease metadata, and audit correlation
  work where OpenBao is in the path.
- artifact-store or the consumer can refresh all credential fields before
  expiration.
- Deny paths produce stable reason codes and audit records.
- Break-glass operation is documented and post-event review is required.

## References

- [AWS STS AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html)
- [Ceph RGW STS](https://docs.ceph.com/en/latest/radosgw/STS/)
- [MinIO AssumeRoleWithWebIdentity](https://min.io/docs/minio/linux/developers/security-token-service/AssumeRoleWithWebIdentity.html)
- [Cloudflare R2 Temporary Credentials API](https://developers.cloudflare.com/api/resources/r2/subresources/temporary_credentials/)
- [Cloudflare R2 temporary credential example](https://developers.cloudflare.com/r2/examples/authenticate-r2-temp-credentials/)