generated from coulomb/repo-seed
Add Postgres durable store requirements
This commit is contained in:
262
docs/postgres-durable-store-consumer-requirements.md
Normal file
262
docs/postgres-durable-store-consumer-requirements.md
Normal file
@@ -0,0 +1,262 @@
|
|||||||
|
# Postgres Durable Store Consumer Requirements
|
||||||
|
|
||||||
|
Status: requirements
|
||||||
|
Date: 2026-06-05
|
||||||
|
Related workplan: USER-WP-0009
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This document defines what `user-engine` needs from a durable Postgres-backed
|
||||||
|
store, from the consumer side. It intentionally does not design or implement
|
||||||
|
the Postgres provider. The expected direction is that an independent
|
||||||
|
NetKingdom infrastructure repository provides a tenant-aware, security
|
||||||
|
integrated Postgres capability, and `user-engine` consumes that capability
|
||||||
|
through a durable store adapter.
|
||||||
|
|
||||||
|
## Consumer Story
|
||||||
|
|
||||||
|
As a `user-engine` consumer, I want the service to persist identity-domain
|
||||||
|
facts durably while keeping NetKingdom security, IAM, secrets, network, tenant
|
||||||
|
isolation, backup, and operational controls outside the user-engine domain
|
||||||
|
implementation.
|
||||||
|
|
||||||
|
The desired experience is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
NetKingdom gives user-engine a scoped Postgres capability.
|
||||||
|
user-engine applies or verifies its own schema for its own tables.
|
||||||
|
service operations keep the same behavior as the isolated MVP.
|
||||||
|
mutations, audit records, and outbox events commit atomically.
|
||||||
|
tenant boundaries and security controls are enforced by both adapter logic and
|
||||||
|
the provided database capability.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Ownership Boundary
|
||||||
|
|
||||||
|
### NetKingdom Postgres Provider Owns
|
||||||
|
|
||||||
|
- Database or cluster provisioning.
|
||||||
|
- Tenant isolation primitive, such as database-per-tenant, schema-per-tenant,
|
||||||
|
row-level security, or another accepted model.
|
||||||
|
- Roles, credentials, certificate material, TLS requirements, secret rotation,
|
||||||
|
and credential lease policy.
|
||||||
|
- Network reachability, firewall rules, service identity admission, and
|
||||||
|
runtime policy integration.
|
||||||
|
- Backup, restore, PITR, replication, retention, and disaster recovery
|
||||||
|
controls.
|
||||||
|
- Platform-level metrics, logs, traces, alert routing, and operational
|
||||||
|
runbooks for the database capability.
|
||||||
|
- Base security posture, hardening, encryption at rest, and administrative
|
||||||
|
access controls.
|
||||||
|
|
||||||
|
### user-engine Owns
|
||||||
|
|
||||||
|
- Domain table definitions for its own data.
|
||||||
|
- Schema version expectations and forward migrations for user-engine tables.
|
||||||
|
- Store adapter behavior that satisfies the public service contract.
|
||||||
|
- Transaction boundaries for user-engine mutations.
|
||||||
|
- Domain constraints, validation, and deterministic query behavior.
|
||||||
|
- Local audit and outbox table semantics when those records are persisted in
|
||||||
|
the user-engine store.
|
||||||
|
- Store conformance tests.
|
||||||
|
|
||||||
|
### External Systems Continue To Own
|
||||||
|
|
||||||
|
- Identity provider configuration, token issuance, credentials, MFA, and
|
||||||
|
sessions.
|
||||||
|
- Authorization policy decisions.
|
||||||
|
- Platform audit custody and long-term audit archive.
|
||||||
|
- Secrets authority and secret distribution.
|
||||||
|
- Organization or tenant authority beyond user-engine references.
|
||||||
|
|
||||||
|
## Functional Requirements
|
||||||
|
|
||||||
|
### Store Parity
|
||||||
|
|
||||||
|
The durable store must satisfy the same behavior currently exercised against
|
||||||
|
the isolated store:
|
||||||
|
|
||||||
|
- Persist users, accounts, tenant accounts, external identities, applications,
|
||||||
|
application bindings, catalogs, profile values, memberships, audit records,
|
||||||
|
and outbox events.
|
||||||
|
- Return stable records by the same logical keys used by
|
||||||
|
`UserEngineService`.
|
||||||
|
- Preserve `schema_version`, `ready`, and migration readiness semantics.
|
||||||
|
- Support the same service-level exceptions for not found, conflict,
|
||||||
|
validation, and authorization-denied flows.
|
||||||
|
|
||||||
|
### Identity And Account Constraints
|
||||||
|
|
||||||
|
- `(issuer, subject)` must uniquely identify one external identity link.
|
||||||
|
- An external identity must not be linked to two different users.
|
||||||
|
- A user must have at most one primary account record in the current model.
|
||||||
|
- Tenant account records must be unique by `(tenant, user_id)`.
|
||||||
|
- Deleted or disabled account states must remain inspectable for audit and
|
||||||
|
lifecycle decisions.
|
||||||
|
|
||||||
|
### Tenant Boundary Requirements
|
||||||
|
|
||||||
|
- Every tenant-scoped row must carry an explicit tenant identifier or be
|
||||||
|
reachable only through an explicit tenant-scoped relationship.
|
||||||
|
- Queries that resolve tenant-scoped data must require tenant context from the
|
||||||
|
service layer.
|
||||||
|
- The adapter must fail closed when tenant context is missing for tenant-bound
|
||||||
|
operations.
|
||||||
|
- The provider should make tenant isolation enforceable below application code,
|
||||||
|
for example through separate databases, schemas, RLS policies, or scoped
|
||||||
|
database roles.
|
||||||
|
- Platform-level access must be represented as an explicit NetKingdom security
|
||||||
|
capability, not as a default database superuser path.
|
||||||
|
|
||||||
|
### Application And Catalog Requirements
|
||||||
|
|
||||||
|
- Application ids must be unique.
|
||||||
|
- Application bindings must be retrievable by application id.
|
||||||
|
- Active catalog namespace ownership must not move silently between
|
||||||
|
applications.
|
||||||
|
- Catalog versioning must preserve the existing rule that active definitions
|
||||||
|
cannot downgrade sensitivity or move versions backwards.
|
||||||
|
- Attribute lookup by key must remain deterministic and efficient enough for
|
||||||
|
projection generation.
|
||||||
|
|
||||||
|
### Profile And Projection Requirements
|
||||||
|
|
||||||
|
- Profile values must be unique by user, attribute key, scope, and scope id.
|
||||||
|
- Effective profile resolution must remain deterministic across global,
|
||||||
|
tenant, application, and membership scopes.
|
||||||
|
- Sensitive and secret values must not leak through diagnostics or logs.
|
||||||
|
- Projection reads should avoid N+1 query patterns for common application
|
||||||
|
runtime and claims-enrichment use cases.
|
||||||
|
|
||||||
|
### Membership Requirements
|
||||||
|
|
||||||
|
- Memberships must be queryable by user and tenant.
|
||||||
|
- Membership facts must carry scope type, scope id, kind, source system,
|
||||||
|
owning system, and freshness version.
|
||||||
|
- Privileged memberships should remain traceable to audit records, evidence
|
||||||
|
references, or explicit evidence gaps.
|
||||||
|
- The store must support future revoke/update behavior without losing the
|
||||||
|
ability to inspect historical role changes.
|
||||||
|
|
||||||
|
### Audit And Outbox Requirements
|
||||||
|
|
||||||
|
- Mutating service operations must commit domain changes, local audit records,
|
||||||
|
and outbox events atomically.
|
||||||
|
- Authorization denials must be auditable without emitting outbox events.
|
||||||
|
- Audit records should be append-only from the service perspective.
|
||||||
|
- Outbox records must support pending reads and future claim/ack/retry
|
||||||
|
semantics.
|
||||||
|
- Outbox event ids must be stable delivery ids, and correlation ids must remain
|
||||||
|
queryable for cross-system tracing.
|
||||||
|
|
||||||
|
### Transaction Requirements
|
||||||
|
|
||||||
|
- Each public mutating service operation must run in one transaction.
|
||||||
|
- Failed validation or authorization must not partially write domain state.
|
||||||
|
- Store implementation must handle uniqueness races deterministically and map
|
||||||
|
them to `ConflictError` where appropriate.
|
||||||
|
- Migration and outbox claiming should use explicit locking strategies that do
|
||||||
|
not require consumers to understand Postgres internals.
|
||||||
|
|
||||||
|
### Migration Requirements
|
||||||
|
|
||||||
|
- user-engine owns migrations for its own tables.
|
||||||
|
- Migrations must be forward-only unless an explicit rollback strategy is
|
||||||
|
accepted for a release.
|
||||||
|
- Readiness must report the expected schema version and actual store version.
|
||||||
|
- Startup behavior must distinguish "store unreachable", "migration required",
|
||||||
|
"migration in progress", and "ready".
|
||||||
|
- Destructive migrations require an explicit operator-controlled process.
|
||||||
|
- The provider may supply the database and schema container, but should not
|
||||||
|
need to know user-engine domain tables.
|
||||||
|
|
||||||
|
## Security Requirements
|
||||||
|
|
||||||
|
- Database credentials must come through a NetKingdom secret or identity
|
||||||
|
mechanism, not literal config files.
|
||||||
|
- Connections must require TLS when crossing process or host boundaries.
|
||||||
|
- Credentials should be scoped to the minimum database, schema, tenant, and
|
||||||
|
operations needed by user-engine.
|
||||||
|
- Logs, errors, readiness output, and diagnostics must not expose credentials,
|
||||||
|
connection strings, secret values, sensitive profile data, or full personal
|
||||||
|
records.
|
||||||
|
- The adapter must make tenant context explicit and auditable.
|
||||||
|
- The provider should expose enough security metadata for `identity_context`
|
||||||
|
evidence or gap references when privileged access or lifecycle work depends
|
||||||
|
on database-side controls.
|
||||||
|
|
||||||
|
## Operability Requirements
|
||||||
|
|
||||||
|
- Health checks should report whether the adapter can reach the store.
|
||||||
|
- Readiness checks should report schema compatibility and migration state.
|
||||||
|
- Diagnostics should include redacted connection target, schema version, last
|
||||||
|
migration, pending outbox count, and recent store error class.
|
||||||
|
- Metrics should cover connection failures, transaction failures, conflicts,
|
||||||
|
migration duration, query latency, and outbox backlog.
|
||||||
|
- Backup/restore expectations must be testable from the consumer side through
|
||||||
|
restore validation or equivalent provider evidence.
|
||||||
|
- Store failures should produce actionable errors without leaking sensitive
|
||||||
|
details.
|
||||||
|
|
||||||
|
## Provider Interface Expectations
|
||||||
|
|
||||||
|
The future provider repository should be able to give user-engine:
|
||||||
|
|
||||||
|
- A logical store reference for the NetKingdom environment and tenant scope.
|
||||||
|
- A secret handle or service identity mechanism for credentials.
|
||||||
|
- TLS or certificate requirements.
|
||||||
|
- Tenant isolation metadata that the adapter can record in diagnostics.
|
||||||
|
- Migration permission policy for user-engine-owned tables.
|
||||||
|
- Backup and restore evidence or status references.
|
||||||
|
- Operational contact/runbook references.
|
||||||
|
|
||||||
|
`user-engine` should not require:
|
||||||
|
|
||||||
|
- Cluster administrator credentials.
|
||||||
|
- Knowledge of physical cluster topology.
|
||||||
|
- Direct control over backups, replication, firewall rules, or secret
|
||||||
|
rotation.
|
||||||
|
- Provider-specific SQL outside an adapter layer.
|
||||||
|
|
||||||
|
## Acceptance Tests For A Future Adapter
|
||||||
|
|
||||||
|
A future Postgres adapter should pass conformance tests for:
|
||||||
|
|
||||||
|
- Creating a user from verified identity claims and reading it through `me`.
|
||||||
|
- Preventing duplicate `(issuer, subject)` links across users.
|
||||||
|
- Creating tenant accounts and denying cross-tenant reads through the service
|
||||||
|
layer.
|
||||||
|
- Adding memberships and returning them in `identity_context`.
|
||||||
|
- Registering an application, publishing a catalog, writing profile values,
|
||||||
|
and producing application runtime and claims-enrichment projections.
|
||||||
|
- Redacting sensitive values in non-eligible projections.
|
||||||
|
- Rolling back all writes when a mutation fails after validation or
|
||||||
|
authorization.
|
||||||
|
- Persisting audit records and outbox events atomically with mutations.
|
||||||
|
- Reporting not-ready state when schema version is missing or incompatible.
|
||||||
|
- Recovering from restart without losing users, memberships, profiles, audit,
|
||||||
|
or outbox records.
|
||||||
|
|
||||||
|
## Open Questions For The Provider Repository
|
||||||
|
|
||||||
|
- Should NetKingdom use database-per-tenant, schema-per-tenant, RLS, or a
|
||||||
|
hybrid model for user-engine data?
|
||||||
|
- Who runs user-engine migrations in production: user-engine startup, a
|
||||||
|
deployment job, or a provider-controlled migration runner?
|
||||||
|
- How are credential leases issued, renewed, revoked, and audited?
|
||||||
|
- What backup unit maps to a family or organization dataspace: cluster,
|
||||||
|
database, schema, tenant row set, or application scope?
|
||||||
|
- What evidence references can the provider expose for backup status, restore
|
||||||
|
tests, encryption posture, and access reviews?
|
||||||
|
- How should local development emulate the provider without weakening the
|
||||||
|
production contract?
|
||||||
|
|
||||||
|
## First Implementation Follow-Ups
|
||||||
|
|
||||||
|
After this requirements work is accepted, likely follow-up work should be:
|
||||||
|
|
||||||
|
- Define the durable store protocol changes, if any.
|
||||||
|
- Add a Postgres adapter behind the existing store boundary.
|
||||||
|
- Add migration files for user-engine tables.
|
||||||
|
- Add conformance tests that run against both in-memory and Postgres stores.
|
||||||
|
- Integrate the adapter with the future NetKingdom Postgres provider repo.
|
||||||
135
workplans/USER-WP-0009-postgres-durable-store-requirements.md
Normal file
135
workplans/USER-WP-0009-postgres-durable-store-requirements.md
Normal file
@@ -0,0 +1,135 @@
|
|||||||
|
---
|
||||||
|
id: USER-WP-0009
|
||||||
|
type: workplan
|
||||||
|
title: "Postgres Durable Store Consumer Requirements"
|
||||||
|
domain: netkingdom
|
||||||
|
repo: user-engine
|
||||||
|
status: proposed
|
||||||
|
owner: codex
|
||||||
|
topic_slug: netkingdom
|
||||||
|
planning_priority: high
|
||||||
|
planning_order: 9
|
||||||
|
created: "2026-06-05"
|
||||||
|
updated: "2026-06-05"
|
||||||
|
depends_on:
|
||||||
|
- USER-WP-0007
|
||||||
|
---
|
||||||
|
|
||||||
|
# USER-WP-0009 - Postgres Durable Store Consumer Requirements
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Define, from the `user-engine` consumer perspective, what a durable
|
||||||
|
Postgres-backed store must provide before user-engine depends on it in
|
||||||
|
NetKingdom. This workplan is requirements-only: it should not implement the
|
||||||
|
Postgres adapter, provision databases, create tenant infrastructure, or choose
|
||||||
|
the final provider repository design.
|
||||||
|
|
||||||
|
## Scope Direction
|
||||||
|
|
||||||
|
`user-engine` should be able to consume a NetKingdom-provided, tenant-aware,
|
||||||
|
security-integrated Postgres capability through an adapter boundary. The
|
||||||
|
future Postgres/provider repository should own provisioning, credentials,
|
||||||
|
network policy, tenant isolation primitives, backup/restore, platform
|
||||||
|
observability, and operational security. `user-engine` should own its domain
|
||||||
|
schema, migrations for its own tables, store semantics, and conformance tests.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
- Do not implement a Postgres store adapter in this workplan.
|
||||||
|
- Do not add database dependencies to the package in this workplan.
|
||||||
|
- Do not provision Postgres, schemas, roles, credentials, certificates, or
|
||||||
|
network access from this repository.
|
||||||
|
- Do not decide the final independent infrastructure repository layout.
|
||||||
|
- Do not move audit-platform, IAM, secrets, or authorization ownership into
|
||||||
|
user-engine.
|
||||||
|
- Do not change the public service surface unless the requirements reveal a
|
||||||
|
missing durable-store contract.
|
||||||
|
|
||||||
|
## Tasks
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T1
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Inventory the current in-memory store behavior and document the durable
|
||||||
|
persistence semantics user-engine consumers already rely on: users, accounts,
|
||||||
|
tenant accounts, external identities, applications, bindings, catalogs,
|
||||||
|
profile values, memberships, audit records, outbox events, readiness, and
|
||||||
|
schema version reporting.
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T2
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Create a consumer-facing requirements document for a Postgres durable store.
|
||||||
|
Cover connection handoff, tenant context, schema ownership, migrations,
|
||||||
|
transactions, isolation, constraints, query behavior, audit/outbox durability,
|
||||||
|
security, observability, backup/restore expectations, and acceptance tests.
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T3
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
```
|
||||||
|
|
||||||
|
Define the boundary between user-engine and the future NetKingdom Postgres
|
||||||
|
provider repository. Specify which responsibilities belong to the provider,
|
||||||
|
which belong to the user-engine adapter, and which must remain external IAM,
|
||||||
|
secrets, authorization, or audit-platform concerns.
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T4
|
||||||
|
status: todo
|
||||||
|
priority: medium
|
||||||
|
```
|
||||||
|
|
||||||
|
Identify required changes, if any, to the existing store protocol or migration
|
||||||
|
contract so durable implementations can satisfy the same service behavior as
|
||||||
|
the isolated MVP without leaking Postgres concepts into domain code.
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T5
|
||||||
|
status: todo
|
||||||
|
priority: medium
|
||||||
|
```
|
||||||
|
|
||||||
|
Define conformance scenarios and failure-mode tests the future Postgres store
|
||||||
|
must pass. Include transaction rollback, duplicate identity prevention,
|
||||||
|
tenant-boundary enforcement, outbox exactly-once handoff semantics, migration
|
||||||
|
readiness, and redacted diagnostics.
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: USER-WP-0009-T6
|
||||||
|
status: todo
|
||||||
|
priority: medium
|
||||||
|
```
|
||||||
|
|
||||||
|
Record open questions for the independent provider repository, including
|
||||||
|
tenant isolation model, credential lease model, schema-per-service or
|
||||||
|
database-per-tenant strategy, migration runner ownership, backup unit, PITR
|
||||||
|
expectations, encryption, and operational runbooks.
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
- `docs/postgres-durable-store-consumer-requirements.md` exists and is clear
|
||||||
|
enough for an independent NetKingdom Postgres provider repo to implement
|
||||||
|
against.
|
||||||
|
- The document describes user-engine as a consumer of a secure Postgres
|
||||||
|
capability, not as the owner of Postgres provisioning or platform security.
|
||||||
|
- Requirements cover domain persistence, transactions, migrations, tenant
|
||||||
|
isolation, security, audit/outbox durability, operability, and acceptance
|
||||||
|
tests.
|
||||||
|
- The provider-repo boundary is explicit and avoids duplicating IAM, secrets,
|
||||||
|
authorization, audit-platform, or infrastructure ownership.
|
||||||
|
- No Postgres implementation code is added as part of this workplan.
|
||||||
|
|
||||||
|
## Expected Outputs
|
||||||
|
|
||||||
|
- `docs/postgres-durable-store-consumer-requirements.md`
|
||||||
|
- Store-boundary notes suitable for a future provider repo.
|
||||||
|
- Follow-up implementation workplan inputs for a Postgres adapter.
|
||||||
Reference in New Issue
Block a user