diff --git a/docs/postgres-durable-store-consumer-requirements.md b/docs/postgres-durable-store-consumer-requirements.md new file mode 100644 index 0000000..1202a91 --- /dev/null +++ b/docs/postgres-durable-store-consumer-requirements.md @@ -0,0 +1,262 @@ +# Postgres Durable Store Consumer Requirements + +Status: requirements +Date: 2026-06-05 +Related workplan: USER-WP-0009 + +## Purpose + +This document defines what `user-engine` needs from a durable Postgres-backed +store, from the consumer side. It intentionally does not design or implement +the Postgres provider. The expected direction is that an independent +NetKingdom infrastructure repository provides a tenant-aware, security +integrated Postgres capability, and `user-engine` consumes that capability +through a durable store adapter. + +## Consumer Story + +As a `user-engine` consumer, I want the service to persist identity-domain +facts durably while keeping NetKingdom security, IAM, secrets, network, tenant +isolation, backup, and operational controls outside the user-engine domain +implementation. + +The desired experience is: + +```text +NetKingdom gives user-engine a scoped Postgres capability. +user-engine applies or verifies its own schema for its own tables. +service operations keep the same behavior as the isolated MVP. +mutations, audit records, and outbox events commit atomically. +tenant boundaries and security controls are enforced by both adapter logic and +the provided database capability. +``` + +## Ownership Boundary + +### NetKingdom Postgres Provider Owns + +- Database or cluster provisioning. +- Tenant isolation primitive, such as database-per-tenant, schema-per-tenant, + row-level security, or another accepted model. +- Roles, credentials, certificate material, TLS requirements, secret rotation, + and credential lease policy. +- Network reachability, firewall rules, service identity admission, and + runtime policy integration. +- Backup, restore, PITR, replication, retention, and disaster recovery + controls. +- Platform-level metrics, logs, traces, alert routing, and operational + runbooks for the database capability. +- Base security posture, hardening, encryption at rest, and administrative + access controls. + +### user-engine Owns + +- Domain table definitions for its own data. +- Schema version expectations and forward migrations for user-engine tables. +- Store adapter behavior that satisfies the public service contract. +- Transaction boundaries for user-engine mutations. +- Domain constraints, validation, and deterministic query behavior. +- Local audit and outbox table semantics when those records are persisted in + the user-engine store. +- Store conformance tests. + +### External Systems Continue To Own + +- Identity provider configuration, token issuance, credentials, MFA, and + sessions. +- Authorization policy decisions. +- Platform audit custody and long-term audit archive. +- Secrets authority and secret distribution. +- Organization or tenant authority beyond user-engine references. + +## Functional Requirements + +### Store Parity + +The durable store must satisfy the same behavior currently exercised against +the isolated store: + +- Persist users, accounts, tenant accounts, external identities, applications, + application bindings, catalogs, profile values, memberships, audit records, + and outbox events. +- Return stable records by the same logical keys used by + `UserEngineService`. +- Preserve `schema_version`, `ready`, and migration readiness semantics. +- Support the same service-level exceptions for not found, conflict, + validation, and authorization-denied flows. + +### Identity And Account Constraints + +- `(issuer, subject)` must uniquely identify one external identity link. +- An external identity must not be linked to two different users. +- A user must have at most one primary account record in the current model. +- Tenant account records must be unique by `(tenant, user_id)`. +- Deleted or disabled account states must remain inspectable for audit and + lifecycle decisions. + +### Tenant Boundary Requirements + +- Every tenant-scoped row must carry an explicit tenant identifier or be + reachable only through an explicit tenant-scoped relationship. +- Queries that resolve tenant-scoped data must require tenant context from the + service layer. +- The adapter must fail closed when tenant context is missing for tenant-bound + operations. +- The provider should make tenant isolation enforceable below application code, + for example through separate databases, schemas, RLS policies, or scoped + database roles. +- Platform-level access must be represented as an explicit NetKingdom security + capability, not as a default database superuser path. + +### Application And Catalog Requirements + +- Application ids must be unique. +- Application bindings must be retrievable by application id. +- Active catalog namespace ownership must not move silently between + applications. +- Catalog versioning must preserve the existing rule that active definitions + cannot downgrade sensitivity or move versions backwards. +- Attribute lookup by key must remain deterministic and efficient enough for + projection generation. + +### Profile And Projection Requirements + +- Profile values must be unique by user, attribute key, scope, and scope id. +- Effective profile resolution must remain deterministic across global, + tenant, application, and membership scopes. +- Sensitive and secret values must not leak through diagnostics or logs. +- Projection reads should avoid N+1 query patterns for common application + runtime and claims-enrichment use cases. + +### Membership Requirements + +- Memberships must be queryable by user and tenant. +- Membership facts must carry scope type, scope id, kind, source system, + owning system, and freshness version. +- Privileged memberships should remain traceable to audit records, evidence + references, or explicit evidence gaps. +- The store must support future revoke/update behavior without losing the + ability to inspect historical role changes. + +### Audit And Outbox Requirements + +- Mutating service operations must commit domain changes, local audit records, + and outbox events atomically. +- Authorization denials must be auditable without emitting outbox events. +- Audit records should be append-only from the service perspective. +- Outbox records must support pending reads and future claim/ack/retry + semantics. +- Outbox event ids must be stable delivery ids, and correlation ids must remain + queryable for cross-system tracing. + +### Transaction Requirements + +- Each public mutating service operation must run in one transaction. +- Failed validation or authorization must not partially write domain state. +- Store implementation must handle uniqueness races deterministically and map + them to `ConflictError` where appropriate. +- Migration and outbox claiming should use explicit locking strategies that do + not require consumers to understand Postgres internals. + +### Migration Requirements + +- user-engine owns migrations for its own tables. +- Migrations must be forward-only unless an explicit rollback strategy is + accepted for a release. +- Readiness must report the expected schema version and actual store version. +- Startup behavior must distinguish "store unreachable", "migration required", + "migration in progress", and "ready". +- Destructive migrations require an explicit operator-controlled process. +- The provider may supply the database and schema container, but should not + need to know user-engine domain tables. + +## Security Requirements + +- Database credentials must come through a NetKingdom secret or identity + mechanism, not literal config files. +- Connections must require TLS when crossing process or host boundaries. +- Credentials should be scoped to the minimum database, schema, tenant, and + operations needed by user-engine. +- Logs, errors, readiness output, and diagnostics must not expose credentials, + connection strings, secret values, sensitive profile data, or full personal + records. +- The adapter must make tenant context explicit and auditable. +- The provider should expose enough security metadata for `identity_context` + evidence or gap references when privileged access or lifecycle work depends + on database-side controls. + +## Operability Requirements + +- Health checks should report whether the adapter can reach the store. +- Readiness checks should report schema compatibility and migration state. +- Diagnostics should include redacted connection target, schema version, last + migration, pending outbox count, and recent store error class. +- Metrics should cover connection failures, transaction failures, conflicts, + migration duration, query latency, and outbox backlog. +- Backup/restore expectations must be testable from the consumer side through + restore validation or equivalent provider evidence. +- Store failures should produce actionable errors without leaking sensitive + details. + +## Provider Interface Expectations + +The future provider repository should be able to give user-engine: + +- A logical store reference for the NetKingdom environment and tenant scope. +- A secret handle or service identity mechanism for credentials. +- TLS or certificate requirements. +- Tenant isolation metadata that the adapter can record in diagnostics. +- Migration permission policy for user-engine-owned tables. +- Backup and restore evidence or status references. +- Operational contact/runbook references. + +`user-engine` should not require: + +- Cluster administrator credentials. +- Knowledge of physical cluster topology. +- Direct control over backups, replication, firewall rules, or secret + rotation. +- Provider-specific SQL outside an adapter layer. + +## Acceptance Tests For A Future Adapter + +A future Postgres adapter should pass conformance tests for: + +- Creating a user from verified identity claims and reading it through `me`. +- Preventing duplicate `(issuer, subject)` links across users. +- Creating tenant accounts and denying cross-tenant reads through the service + layer. +- Adding memberships and returning them in `identity_context`. +- Registering an application, publishing a catalog, writing profile values, + and producing application runtime and claims-enrichment projections. +- Redacting sensitive values in non-eligible projections. +- Rolling back all writes when a mutation fails after validation or + authorization. +- Persisting audit records and outbox events atomically with mutations. +- Reporting not-ready state when schema version is missing or incompatible. +- Recovering from restart without losing users, memberships, profiles, audit, + or outbox records. + +## Open Questions For The Provider Repository + +- Should NetKingdom use database-per-tenant, schema-per-tenant, RLS, or a + hybrid model for user-engine data? +- Who runs user-engine migrations in production: user-engine startup, a + deployment job, or a provider-controlled migration runner? +- How are credential leases issued, renewed, revoked, and audited? +- What backup unit maps to a family or organization dataspace: cluster, + database, schema, tenant row set, or application scope? +- What evidence references can the provider expose for backup status, restore + tests, encryption posture, and access reviews? +- How should local development emulate the provider without weakening the + production contract? + +## First Implementation Follow-Ups + +After this requirements work is accepted, likely follow-up work should be: + +- Define the durable store protocol changes, if any. +- Add a Postgres adapter behind the existing store boundary. +- Add migration files for user-engine tables. +- Add conformance tests that run against both in-memory and Postgres stores. +- Integrate the adapter with the future NetKingdom Postgres provider repo. diff --git a/workplans/USER-WP-0009-postgres-durable-store-requirements.md b/workplans/USER-WP-0009-postgres-durable-store-requirements.md new file mode 100644 index 0000000..8211063 --- /dev/null +++ b/workplans/USER-WP-0009-postgres-durable-store-requirements.md @@ -0,0 +1,135 @@ +--- +id: USER-WP-0009 +type: workplan +title: "Postgres Durable Store Consumer Requirements" +domain: netkingdom +repo: user-engine +status: proposed +owner: codex +topic_slug: netkingdom +planning_priority: high +planning_order: 9 +created: "2026-06-05" +updated: "2026-06-05" +depends_on: + - USER-WP-0007 +--- + +# USER-WP-0009 - Postgres Durable Store Consumer Requirements + +## Goal + +Define, from the `user-engine` consumer perspective, what a durable +Postgres-backed store must provide before user-engine depends on it in +NetKingdom. This workplan is requirements-only: it should not implement the +Postgres adapter, provision databases, create tenant infrastructure, or choose +the final provider repository design. + +## Scope Direction + +`user-engine` should be able to consume a NetKingdom-provided, tenant-aware, +security-integrated Postgres capability through an adapter boundary. The +future Postgres/provider repository should own provisioning, credentials, +network policy, tenant isolation primitives, backup/restore, platform +observability, and operational security. `user-engine` should own its domain +schema, migrations for its own tables, store semantics, and conformance tests. + +## Non-Goals + +- Do not implement a Postgres store adapter in this workplan. +- Do not add database dependencies to the package in this workplan. +- Do not provision Postgres, schemas, roles, credentials, certificates, or + network access from this repository. +- Do not decide the final independent infrastructure repository layout. +- Do not move audit-platform, IAM, secrets, or authorization ownership into + user-engine. +- Do not change the public service surface unless the requirements reveal a + missing durable-store contract. + +## Tasks + +```task +id: USER-WP-0009-T1 +status: todo +priority: high +``` + +Inventory the current in-memory store behavior and document the durable +persistence semantics user-engine consumers already rely on: users, accounts, +tenant accounts, external identities, applications, bindings, catalogs, +profile values, memberships, audit records, outbox events, readiness, and +schema version reporting. + +```task +id: USER-WP-0009-T2 +status: todo +priority: high +``` + +Create a consumer-facing requirements document for a Postgres durable store. +Cover connection handoff, tenant context, schema ownership, migrations, +transactions, isolation, constraints, query behavior, audit/outbox durability, +security, observability, backup/restore expectations, and acceptance tests. + +```task +id: USER-WP-0009-T3 +status: todo +priority: high +``` + +Define the boundary between user-engine and the future NetKingdom Postgres +provider repository. Specify which responsibilities belong to the provider, +which belong to the user-engine adapter, and which must remain external IAM, +secrets, authorization, or audit-platform concerns. + +```task +id: USER-WP-0009-T4 +status: todo +priority: medium +``` + +Identify required changes, if any, to the existing store protocol or migration +contract so durable implementations can satisfy the same service behavior as +the isolated MVP without leaking Postgres concepts into domain code. + +```task +id: USER-WP-0009-T5 +status: todo +priority: medium +``` + +Define conformance scenarios and failure-mode tests the future Postgres store +must pass. Include transaction rollback, duplicate identity prevention, +tenant-boundary enforcement, outbox exactly-once handoff semantics, migration +readiness, and redacted diagnostics. + +```task +id: USER-WP-0009-T6 +status: todo +priority: medium +``` + +Record open questions for the independent provider repository, including +tenant isolation model, credential lease model, schema-per-service or +database-per-tenant strategy, migration runner ownership, backup unit, PITR +expectations, encryption, and operational runbooks. + +## Acceptance Criteria + +- `docs/postgres-durable-store-consumer-requirements.md` exists and is clear + enough for an independent NetKingdom Postgres provider repo to implement + against. +- The document describes user-engine as a consumer of a secure Postgres + capability, not as the owner of Postgres provisioning or platform security. +- Requirements cover domain persistence, transactions, migrations, tenant + isolation, security, audit/outbox durability, operability, and acceptance + tests. +- The provider-repo boundary is explicit and avoids duplicating IAM, secrets, + authorization, audit-platform, or infrastructure ownership. +- No Postgres implementation code is added as part of this workplan. + +## Expected Outputs + +- `docs/postgres-durable-store-consumer-requirements.md` +- Store-boundary notes suitable for a future provider repo. +- Follow-up implementation workplan inputs for a Postgres adapter.