generated from coulomb/repo-seed
Add Postgres durable store requirements
This commit is contained in:
262
docs/postgres-durable-store-consumer-requirements.md
Normal file
262
docs/postgres-durable-store-consumer-requirements.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# Postgres Durable Store Consumer Requirements
|
||||
|
||||
Status: requirements
|
||||
Date: 2026-06-05
|
||||
Related workplan: USER-WP-0009
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines what `user-engine` needs from a durable Postgres-backed
|
||||
store, from the consumer side. It intentionally does not design or implement
|
||||
the Postgres provider. The expected direction is that an independent
|
||||
NetKingdom infrastructure repository provides a tenant-aware, security
|
||||
integrated Postgres capability, and `user-engine` consumes that capability
|
||||
through a durable store adapter.
|
||||
|
||||
## Consumer Story
|
||||
|
||||
As a `user-engine` consumer, I want the service to persist identity-domain
|
||||
facts durably while keeping NetKingdom security, IAM, secrets, network, tenant
|
||||
isolation, backup, and operational controls outside the user-engine domain
|
||||
implementation.
|
||||
|
||||
The desired experience is:
|
||||
|
||||
```text
|
||||
NetKingdom gives user-engine a scoped Postgres capability.
|
||||
user-engine applies or verifies its own schema for its own tables.
|
||||
service operations keep the same behavior as the isolated MVP.
|
||||
mutations, audit records, and outbox events commit atomically.
|
||||
tenant boundaries and security controls are enforced by both adapter logic and
|
||||
the provided database capability.
|
||||
```
|
||||
|
||||
## Ownership Boundary
|
||||
|
||||
### NetKingdom Postgres Provider Owns
|
||||
|
||||
- Database or cluster provisioning.
|
||||
- Tenant isolation primitive, such as database-per-tenant, schema-per-tenant,
|
||||
row-level security, or another accepted model.
|
||||
- Roles, credentials, certificate material, TLS requirements, secret rotation,
|
||||
and credential lease policy.
|
||||
- Network reachability, firewall rules, service identity admission, and
|
||||
runtime policy integration.
|
||||
- Backup, restore, PITR, replication, retention, and disaster recovery
|
||||
controls.
|
||||
- Platform-level metrics, logs, traces, alert routing, and operational
|
||||
runbooks for the database capability.
|
||||
- Base security posture, hardening, encryption at rest, and administrative
|
||||
access controls.
|
||||
|
||||
### user-engine Owns
|
||||
|
||||
- Domain table definitions for its own data.
|
||||
- Schema version expectations and forward migrations for user-engine tables.
|
||||
- Store adapter behavior that satisfies the public service contract.
|
||||
- Transaction boundaries for user-engine mutations.
|
||||
- Domain constraints, validation, and deterministic query behavior.
|
||||
- Local audit and outbox table semantics when those records are persisted in
|
||||
the user-engine store.
|
||||
- Store conformance tests.
|
||||
|
||||
### External Systems Continue To Own
|
||||
|
||||
- Identity provider configuration, token issuance, credentials, MFA, and
|
||||
sessions.
|
||||
- Authorization policy decisions.
|
||||
- Platform audit custody and long-term audit archive.
|
||||
- Secrets authority and secret distribution.
|
||||
- Organization or tenant authority beyond user-engine references.
|
||||
|
||||
## Functional Requirements
|
||||
|
||||
### Store Parity
|
||||
|
||||
The durable store must satisfy the same behavior currently exercised against
|
||||
the isolated store:
|
||||
|
||||
- Persist users, accounts, tenant accounts, external identities, applications,
|
||||
application bindings, catalogs, profile values, memberships, audit records,
|
||||
and outbox events.
|
||||
- Return stable records by the same logical keys used by
|
||||
`UserEngineService`.
|
||||
- Preserve `schema_version`, `ready`, and migration readiness semantics.
|
||||
- Support the same service-level exceptions for not found, conflict,
|
||||
validation, and authorization-denied flows.
|
||||
|
||||
### Identity And Account Constraints
|
||||
|
||||
- `(issuer, subject)` must uniquely identify one external identity link.
|
||||
- An external identity must not be linked to two different users.
|
||||
- A user must have at most one primary account record in the current model.
|
||||
- Tenant account records must be unique by `(tenant, user_id)`.
|
||||
- Deleted or disabled account states must remain inspectable for audit and
|
||||
lifecycle decisions.
|
||||
|
||||
### Tenant Boundary Requirements
|
||||
|
||||
- Every tenant-scoped row must carry an explicit tenant identifier or be
|
||||
reachable only through an explicit tenant-scoped relationship.
|
||||
- Queries that resolve tenant-scoped data must require tenant context from the
|
||||
service layer.
|
||||
- The adapter must fail closed when tenant context is missing for tenant-bound
|
||||
operations.
|
||||
- The provider should make tenant isolation enforceable below application code,
|
||||
for example through separate databases, schemas, RLS policies, or scoped
|
||||
database roles.
|
||||
- Platform-level access must be represented as an explicit NetKingdom security
|
||||
capability, not as a default database superuser path.
|
||||
|
||||
### Application And Catalog Requirements
|
||||
|
||||
- Application ids must be unique.
|
||||
- Application bindings must be retrievable by application id.
|
||||
- Active catalog namespace ownership must not move silently between
|
||||
applications.
|
||||
- Catalog versioning must preserve the existing rule that active definitions
|
||||
cannot downgrade sensitivity or move versions backwards.
|
||||
- Attribute lookup by key must remain deterministic and efficient enough for
|
||||
projection generation.
|
||||
|
||||
### Profile And Projection Requirements
|
||||
|
||||
- Profile values must be unique by user, attribute key, scope, and scope id.
|
||||
- Effective profile resolution must remain deterministic across global,
|
||||
tenant, application, and membership scopes.
|
||||
- Sensitive and secret values must not leak through diagnostics or logs.
|
||||
- Projection reads should avoid N+1 query patterns for common application
|
||||
runtime and claims-enrichment use cases.
|
||||
|
||||
### Membership Requirements
|
||||
|
||||
- Memberships must be queryable by user and tenant.
|
||||
- Membership facts must carry scope type, scope id, kind, source system,
|
||||
owning system, and freshness version.
|
||||
- Privileged memberships should remain traceable to audit records, evidence
|
||||
references, or explicit evidence gaps.
|
||||
- The store must support future revoke/update behavior without losing the
|
||||
ability to inspect historical role changes.
|
||||
|
||||
### Audit And Outbox Requirements
|
||||
|
||||
- Mutating service operations must commit domain changes, local audit records,
|
||||
and outbox events atomically.
|
||||
- Authorization denials must be auditable without emitting outbox events.
|
||||
- Audit records should be append-only from the service perspective.
|
||||
- Outbox records must support pending reads and future claim/ack/retry
|
||||
semantics.
|
||||
- Outbox event ids must be stable delivery ids, and correlation ids must remain
|
||||
queryable for cross-system tracing.
|
||||
|
||||
### Transaction Requirements
|
||||
|
||||
- Each public mutating service operation must run in one transaction.
|
||||
- Failed validation or authorization must not partially write domain state.
|
||||
- Store implementation must handle uniqueness races deterministically and map
|
||||
them to `ConflictError` where appropriate.
|
||||
- Migration and outbox claiming should use explicit locking strategies that do
|
||||
not require consumers to understand Postgres internals.
|
||||
|
||||
### Migration Requirements
|
||||
|
||||
- user-engine owns migrations for its own tables.
|
||||
- Migrations must be forward-only unless an explicit rollback strategy is
|
||||
accepted for a release.
|
||||
- Readiness must report the expected schema version and actual store version.
|
||||
- Startup behavior must distinguish "store unreachable", "migration required",
|
||||
"migration in progress", and "ready".
|
||||
- Destructive migrations require an explicit operator-controlled process.
|
||||
- The provider may supply the database and schema container, but should not
|
||||
need to know user-engine domain tables.
|
||||
|
||||
## Security Requirements
|
||||
|
||||
- Database credentials must come through a NetKingdom secret or identity
|
||||
mechanism, not literal config files.
|
||||
- Connections must require TLS when crossing process or host boundaries.
|
||||
- Credentials should be scoped to the minimum database, schema, tenant, and
|
||||
operations needed by user-engine.
|
||||
- Logs, errors, readiness output, and diagnostics must not expose credentials,
|
||||
connection strings, secret values, sensitive profile data, or full personal
|
||||
records.
|
||||
- The adapter must make tenant context explicit and auditable.
|
||||
- The provider should expose enough security metadata for `identity_context`
|
||||
evidence or gap references when privileged access or lifecycle work depends
|
||||
on database-side controls.
|
||||
|
||||
## Operability Requirements
|
||||
|
||||
- Health checks should report whether the adapter can reach the store.
|
||||
- Readiness checks should report schema compatibility and migration state.
|
||||
- Diagnostics should include redacted connection target, schema version, last
|
||||
migration, pending outbox count, and recent store error class.
|
||||
- Metrics should cover connection failures, transaction failures, conflicts,
|
||||
migration duration, query latency, and outbox backlog.
|
||||
- Backup/restore expectations must be testable from the consumer side through
|
||||
restore validation or equivalent provider evidence.
|
||||
- Store failures should produce actionable errors without leaking sensitive
|
||||
details.
|
||||
|
||||
## Provider Interface Expectations
|
||||
|
||||
The future provider repository should be able to give user-engine:
|
||||
|
||||
- A logical store reference for the NetKingdom environment and tenant scope.
|
||||
- A secret handle or service identity mechanism for credentials.
|
||||
- TLS or certificate requirements.
|
||||
- Tenant isolation metadata that the adapter can record in diagnostics.
|
||||
- Migration permission policy for user-engine-owned tables.
|
||||
- Backup and restore evidence or status references.
|
||||
- Operational contact/runbook references.
|
||||
|
||||
`user-engine` should not require:
|
||||
|
||||
- Cluster administrator credentials.
|
||||
- Knowledge of physical cluster topology.
|
||||
- Direct control over backups, replication, firewall rules, or secret
|
||||
rotation.
|
||||
- Provider-specific SQL outside an adapter layer.
|
||||
|
||||
## Acceptance Tests For A Future Adapter
|
||||
|
||||
A future Postgres adapter should pass conformance tests for:
|
||||
|
||||
- Creating a user from verified identity claims and reading it through `me`.
|
||||
- Preventing duplicate `(issuer, subject)` links across users.
|
||||
- Creating tenant accounts and denying cross-tenant reads through the service
|
||||
layer.
|
||||
- Adding memberships and returning them in `identity_context`.
|
||||
- Registering an application, publishing a catalog, writing profile values,
|
||||
and producing application runtime and claims-enrichment projections.
|
||||
- Redacting sensitive values in non-eligible projections.
|
||||
- Rolling back all writes when a mutation fails after validation or
|
||||
authorization.
|
||||
- Persisting audit records and outbox events atomically with mutations.
|
||||
- Reporting not-ready state when schema version is missing or incompatible.
|
||||
- Recovering from restart without losing users, memberships, profiles, audit,
|
||||
or outbox records.
|
||||
|
||||
## Open Questions For The Provider Repository
|
||||
|
||||
- Should NetKingdom use database-per-tenant, schema-per-tenant, RLS, or a
|
||||
hybrid model for user-engine data?
|
||||
- Who runs user-engine migrations in production: user-engine startup, a
|
||||
deployment job, or a provider-controlled migration runner?
|
||||
- How are credential leases issued, renewed, revoked, and audited?
|
||||
- What backup unit maps to a family or organization dataspace: cluster,
|
||||
database, schema, tenant row set, or application scope?
|
||||
- What evidence references can the provider expose for backup status, restore
|
||||
tests, encryption posture, and access reviews?
|
||||
- How should local development emulate the provider without weakening the
|
||||
production contract?
|
||||
|
||||
## First Implementation Follow-Ups
|
||||
|
||||
After this requirements work is accepted, likely follow-up work should be:
|
||||
|
||||
- Define the durable store protocol changes, if any.
|
||||
- Add a Postgres adapter behind the existing store boundary.
|
||||
- Add migration files for user-engine tables.
|
||||
- Add conformance tests that run against both in-memory and Postgres stores.
|
||||
- Integrate the adapter with the future NetKingdom Postgres provider repo.
|
||||
135
workplans/USER-WP-0009-postgres-durable-store-requirements.md
Normal file
135
workplans/USER-WP-0009-postgres-durable-store-requirements.md
Normal file
@@ -0,0 +1,135 @@
|
||||
---
|
||||
id: USER-WP-0009
|
||||
type: workplan
|
||||
title: "Postgres Durable Store Consumer Requirements"
|
||||
domain: netkingdom
|
||||
repo: user-engine
|
||||
status: proposed
|
||||
owner: codex
|
||||
topic_slug: netkingdom
|
||||
planning_priority: high
|
||||
planning_order: 9
|
||||
created: "2026-06-05"
|
||||
updated: "2026-06-05"
|
||||
depends_on:
|
||||
- USER-WP-0007
|
||||
---
|
||||
|
||||
# USER-WP-0009 - Postgres Durable Store Consumer Requirements
|
||||
|
||||
## Goal
|
||||
|
||||
Define, from the `user-engine` consumer perspective, what a durable
|
||||
Postgres-backed store must provide before user-engine depends on it in
|
||||
NetKingdom. This workplan is requirements-only: it should not implement the
|
||||
Postgres adapter, provision databases, create tenant infrastructure, or choose
|
||||
the final provider repository design.
|
||||
|
||||
## Scope Direction
|
||||
|
||||
`user-engine` should be able to consume a NetKingdom-provided, tenant-aware,
|
||||
security-integrated Postgres capability through an adapter boundary. The
|
||||
future Postgres/provider repository should own provisioning, credentials,
|
||||
network policy, tenant isolation primitives, backup/restore, platform
|
||||
observability, and operational security. `user-engine` should own its domain
|
||||
schema, migrations for its own tables, store semantics, and conformance tests.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not implement a Postgres store adapter in this workplan.
|
||||
- Do not add database dependencies to the package in this workplan.
|
||||
- Do not provision Postgres, schemas, roles, credentials, certificates, or
|
||||
network access from this repository.
|
||||
- Do not decide the final independent infrastructure repository layout.
|
||||
- Do not move audit-platform, IAM, secrets, or authorization ownership into
|
||||
user-engine.
|
||||
- Do not change the public service surface unless the requirements reveal a
|
||||
missing durable-store contract.
|
||||
|
||||
## Tasks
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T1
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Inventory the current in-memory store behavior and document the durable
|
||||
persistence semantics user-engine consumers already rely on: users, accounts,
|
||||
tenant accounts, external identities, applications, bindings, catalogs,
|
||||
profile values, memberships, audit records, outbox events, readiness, and
|
||||
schema version reporting.
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T2
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Create a consumer-facing requirements document for a Postgres durable store.
|
||||
Cover connection handoff, tenant context, schema ownership, migrations,
|
||||
transactions, isolation, constraints, query behavior, audit/outbox durability,
|
||||
security, observability, backup/restore expectations, and acceptance tests.
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T3
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Define the boundary between user-engine and the future NetKingdom Postgres
|
||||
provider repository. Specify which responsibilities belong to the provider,
|
||||
which belong to the user-engine adapter, and which must remain external IAM,
|
||||
secrets, authorization, or audit-platform concerns.
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T4
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Identify required changes, if any, to the existing store protocol or migration
|
||||
contract so durable implementations can satisfy the same service behavior as
|
||||
the isolated MVP without leaking Postgres concepts into domain code.
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T5
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Define conformance scenarios and failure-mode tests the future Postgres store
|
||||
must pass. Include transaction rollback, duplicate identity prevention,
|
||||
tenant-boundary enforcement, outbox exactly-once handoff semantics, migration
|
||||
readiness, and redacted diagnostics.
|
||||
|
||||
```task
|
||||
id: USER-WP-0009-T6
|
||||
status: todo
|
||||
priority: medium
|
||||
```
|
||||
|
||||
Record open questions for the independent provider repository, including
|
||||
tenant isolation model, credential lease model, schema-per-service or
|
||||
database-per-tenant strategy, migration runner ownership, backup unit, PITR
|
||||
expectations, encryption, and operational runbooks.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- `docs/postgres-durable-store-consumer-requirements.md` exists and is clear
|
||||
enough for an independent NetKingdom Postgres provider repo to implement
|
||||
against.
|
||||
- The document describes user-engine as a consumer of a secure Postgres
|
||||
capability, not as the owner of Postgres provisioning or platform security.
|
||||
- Requirements cover domain persistence, transactions, migrations, tenant
|
||||
isolation, security, audit/outbox durability, operability, and acceptance
|
||||
tests.
|
||||
- The provider-repo boundary is explicit and avoids duplicating IAM, secrets,
|
||||
authorization, audit-platform, or infrastructure ownership.
|
||||
- No Postgres implementation code is added as part of this workplan.
|
||||
|
||||
## Expected Outputs
|
||||
|
||||
- `docs/postgres-durable-store-consumer-requirements.md`
|
||||
- Store-boundary notes suitable for a future provider repo.
|
||||
- Follow-up implementation workplan inputs for a Postgres adapter.
|
||||
Reference in New Issue
Block a user