Established architecture blueprint

2026-05-22 19:05:13 +02:00
parent c4b6b83db5
commit 0a45c97486
1 changed files with 580 additions and 0 deletions
--- a/wiki/ArchitectureBlueprint.md
+++ b/wiki/ArchitectureBlueprint.md
@@ -0,0 +1,580 @@
+# User Engine Architecture Blueprint
+
+Status: initial architecture blueprint
+Date: 2026-05-22
+Related product docs: `../INTENT.md`, `ProductRequirementsDocument.md`
+Primary platform references:
+
+- `/home/worsch/net-kingdom/canon/standards/iam-profile_v0.2.md`
+- `/home/worsch/net-kingdom/docs/platform-identity-security-architecture.md`
+- `/home/worsch/net-kingdom/docs/responsibility-map.md`
+- `/home/worsch/key-cape/SCOPE.md`
+- `/home/worsch/flex-auth/docs/iam-profile-consumption.md`
+
+## 1. Architecture Decision
+
+`user-engine` should be implemented as the headless user-domain and profile
+service in the NetKingdom repository landscape.
+
+It owns user-domain records, account state, external identity links,
+profile/preference values, application registrations, customization catalogs,
+profile projections, and lifecycle/profile events.
+
+It does not own authentication, MFA, password/passkey credentials, OIDC token
+issuance, final resource authorization, runtime secret custody, or UI
+experience. Those remain with the existing NetKingdom-aligned components:
+
+- `key-cape` or Keycloak implements the NetKingdom IAM Profile.
+- `local-identity` supports local/bootstrap development identity.
+- `flex-auth` owns final authorization decisions and policy/audit envelopes.
+- OpenBao and Railiance platform services own runtime secrets and durable
+  platform backing services.
+- Future `user-account` and `user-manager` repos/apps own the UI surfaces.
+
+The useful product boundary is:
+
+```text
+Identity provider tells user-engine who the actor is.
+flex-auth tells user-engine what the actor may do.
+user-engine manages user-domain facts and profile projections.
+Applications and UIs consume user-engine through stable headless APIs.
+```
+
+## 2. Fit In The NetKingdom Landscape
+
+### 2.1 Identity Integration
+
+Applications and user-engine should target the NetKingdom IAM Profile, not a
+concrete provider. In production and platform mode, requests arrive with
+tokens issued by either:
+
+- key-cape lightweight mode, backed by Authelia/LLDAP/privacyIDEA where used;
+- Keycloak expanded mode, used for enterprise federation, SAML brokering, or
+  complex realm/delegated-admin needs.
+
+In local development, `local-identity` can issue local OIDC tokens that are
+accepted only in non-production environments. Production user-engine
+deployments must reject local or loopback issuers as required by the IAM
+Profile.
+
+Required token inputs for user-engine are the same provider-neutral facts used
+by flex-auth:
+
+- `iss`, `sub`, `aud`, `tenant`, `principal_type`
+- `groups`, `roles`, `scope` or `scp`
+- `assurance`
+- optional human display/contact claims such as `preferred_username`, `email`,
+  and `name`
+- optional agent/delegation claims for agentic use cases
+
+`user-engine` must treat `iss + sub` as the stable identity link key. Email,
+username, or display name must never be primary keys.
+
+### 2.2 Authorization Integration
+
+`user-engine` enforces access, but it should not become the policy decision
+point. For every protected action, it should present a normalized request to
+flex-auth and enforce the returned decision locally.
+
+Protected resource examples:
+
+| Resource | Example actions |
+| --- | --- |
+| `user-engine:user` | `read`, `create`, `update`, `deactivate`, `delete_request` |
+| `user-engine:identity-link` | `read`, `link`, `unlink` |
+| `user-engine:profile` | `read`, `update`, `resolve`, `project` |
+| `user-engine:membership` | `read`, `assign`, `remove` |
+| `user-engine:application` | `register`, `update`, `deactivate` |
+| `user-engine:catalog` | `register`, `activate`, `deprecate`, `migrate` |
+| `user-engine:audit` | `read` |
+
+The authorization flow should be:
+
+1. Validate or receive a validated IAM Profile token.
+2. Normalize identity claims into an actor envelope.
+3. Resolve the actor's linked user where needed.
+4. Call flex-auth with actor, tenant, resource, action, scope, assurance, and
+   contextual attributes.
+5. Enforce deny/allow/obligations locally.
+6. Apply attribute-level projection filtering for visibility, mutability, and
+   sensitivity.
+7. Emit audit and domain events.
+
+Memberships and profile facts can be authorization-relevant inputs, but they
+are not authorization decisions. `user-engine` may export subject, tenant,
+team, and membership facts to flex-auth subject manifests or resolver adapters;
+flex-auth remains the decision boundary.
+
+### 2.3 Deployment And Operations Integration
+
+`user-engine` can run in three modes:
+
+| Mode | Purpose | Dependencies |
+| --- | --- | --- |
+| Standalone | Small app, prototype, local dev, single service | Local DB, local/external OIDC, optional in-process policy adapter |
+| Platform | Shared service for multiple NetKingdom applications | IAM Profile issuer, flex-auth, PostgreSQL, event/audit sink, OpenBao-backed secrets |
+| Enterprise | Federated, provisioned, multi-tenant landscape | Keycloak expanded mode, SCIM/import adapters, stronger audit retention, directory/group resolvers |
+
+Production platform mode should use Railiance-provided backing services:
+
+- PostgreSQL for canonical user-engine state.
+- OpenBao for database credentials, API client secrets, signing keys, and
+  webhook secrets.
+- Railiance platform event or message services for outbox/event delivery.
+- Cluster ingress, network policy, TLS, and observability from the Railiance
+  stack.
+
+NetKingdom should meta-orchestrate the trust and responsibility boundaries.
+Railiance should execute deployment mechanics. If `user-engine` becomes a
+shared platform service that holds user, account, catalog, and membership
+resources across the landscape, NetKingdom's responsibility map should be
+updated there, not in `user-engine` intent, to preserve self-coherent repo
+intent.
+
+## 3. Responsibility Boundaries
+
+| Concern | Owner | user-engine role |
+| --- | --- | --- |
+| OIDC discovery, login, token issuance | key-cape, Keycloak, local-identity in dev | Consume tokens and identity facts |
+| Passwords, MFA, passkeys, sessions | key-cape/Keycloak/MFA stack | No storage or credential lifecycle |
+| External identity links | user-engine | Map `iss + sub` identities to local user records |
+| Canonical user/account/profile data | user-engine | Source of truth |
+| Application profile catalogs | user-engine | Source of truth and validation authority |
+| Groups/roles in tokens | IAM provider, mapped to IAM Profile | Consume as identity facts; do not make final decisions |
+| Membership/domain facts | user-engine, optionally synced with IAM/flex-auth | Manage user-domain relationships; export facts where useful |
+| Final authorization | flex-auth and PDP adapters | Ask, enforce, audit obligations |
+| Runtime secrets | OpenBao/Railiance platform | Consume scoped secrets only |
+| Self-service UI | future `user-account` | Provide APIs/projections |
+| Admin UI | future `user-manager` | Provide APIs/projections |
+
+## 4. Conceptual Component Model
+
+```text
+               +---------------------+
+               | key-cape / Keycloak |
+               | IAM Profile issuer  |
+               +----------+----------+
+                          |
+                          | OIDC token / normalized claims
+                          v
+-------------------------+--------------------------+
+|                    user-engine                     |
+|                                                    |
+|  API adapters                                      |
+|    - REST/HTTP API                                 |
+|    - internal service API                          |
+|    - future SDKs                                   |
+|                                                    |
+|  Domain core                                       |
+|    - users and accounts                            |
+|    - identity links                                |
+|    - tenants, applications, teams, memberships     |
+|    - global, tenant, app, and membership profiles  |
+|    - preference values                             |
+|                                                    |
+|  Catalog and projection engine                     |
+|    - catalog registry                              |
+|    - attribute governance                          |
+|    - effective profile resolver                    |
+|    - projection filters                            |
+|                                                    |
+|  Integration layer                                 |
+|    - flex-auth check client                        |
+|    - IAM Profile claim adapter                     |
+|    - event outbox                                  |
+|    - audit writer                                  |
+------------+--------------------------+------------+
+             |                          |
+             | policy checks            | state/events/audit
+             v                          v
+       +-----------+           +----------------------+
+       | flex-auth |           | PostgreSQL / outbox  |
+       +-----------+           | event/audit sinks    |
+                               +----------------------+
+```
+
+The domain core should be usable without HTTP. API adapters should call domain
+services rather than placing lifecycle or profile resolution logic in route
+handlers.
+
+## 5. Domain Model Blueprint
+
+### 5.1 Stable Identifiers
+
+Use stable internal identifiers and explicit external links.
+
+| Entity | Identifier rule |
+| --- | --- |
+| User | Internal `user_id`, opaque and stable |
+| External identity | Unique `(issuer, subject)` plus provider metadata |
+| Tenant | NetKingdom-aligned identifiers such as `tenant:platform`, `tenant:coulomb`, `tenant:customer:<id>` |
+| Application | Stable application id, preferably namespaced |
+| Catalog | `(application_id, namespace, catalog_id, version)` |
+| Attribute | Catalog-qualified key, not a global loose string |
+| Membership | `(user_id, scope_type, scope_id, role_or_relation)` where applicable |
+
+### 5.2 Core Entities
+
+Minimum persistent entities:
+
+- `User`: stable person/human-representing subject.
+- `Account`: lifecycle state and platform/application relationship.
+- `ExternalIdentity`: identity provider link, keyed by `iss + sub`.
+- `Tenant`: platform or tenant plane scope.
+- `Application`: profile-consuming application registration.
+- `Team` or `GroupScope`: optional team/project scope for membership profiles.
+- `Membership`: user relationship to tenant, application, team, project, or
+  other scope.
+- `ProfileValue`: scoped profile/preference values.
+- `Catalog`: application-owned customization descriptor.
+- `AttributeDefinition`: catalog-defined attribute metadata.
+- `ProjectionDefinition`: reusable projection rules.
+- `AuditRecord`: durable user-engine audit trail.
+- `OutboxEvent`: reliable integration event queue.
+
+### 5.3 Account Lifecycle
+
+Initial account states should align with the PRD while staying compatible with
+external provisioning:
+
+- `invited`
+- `active`
+- `disabled`
+- `suspended`
+- `deletion_pending`
+- `deleted`
+
+Account records should separately track management mode:
+
+- `local`
+- `externally_provisioned`
+- `federated`
+- `service_managed`
+
+State transitions must be explicit, audited, and authorized. Deletion should
+default to a staged `deletion_pending` flow so downstream projections and
+events can settle before irreversible erasure.
+
+## 6. Catalog And Profile Architecture
+
+### 6.1 Catalog Descriptor
+
+The first catalog descriptor should be JSON/YAML with JSON Schema-compatible
+validation plus user-engine governance metadata.
+
+Required catalog-level fields:
+
+- namespace
+- catalog id
+- semantic version
+- owning application id
+- lifecycle state: `draft`, `active`, `deprecated`, `retired`
+- allowed profile scopes
+- projection types
+- migration metadata
+
+Required attribute-level fields:
+
+- key
+- value type and validation schema
+- defaulting behavior
+- allowed scopes
+- owner
+- visibility
+- mutability
+- sensitivity
+- inheritance/override rule
+- projection eligibility
+- UI hints for future generated forms
+
+Catalog activation should validate:
+
+- namespace ownership
+- attribute key uniqueness within the catalog version
+- valid scope references
+- supported validation schemas
+- no incompatible sensitivity downgrade from previous active versions
+- migration requirement when changing type, scope, or sensitivity
+
+### 6.2 Layered Profile Scopes
+
+Initial layers:
+
+1. Global user profile
+2. Tenant user profile
+3. Application user profile
+4. Membership/team profile
+5. Admin override, only for attributes that explicitly allow it
+
+The effective profile resolver should be deterministic and inspectable. A
+projection response should be able to explain which layer supplied a value and
+which values were hidden by visibility/sensitivity policy.
+
+Suggested default precedence:
+
+```text
+catalog default
+  < global value
+  < tenant value
+  < application value
+  < membership/team value
+  < allowed admin override
+```
+
+Precedence should be configurable only through catalog-defined override rules,
+not per-request ad hoc logic.
+
+### 6.3 Projection Types
+
+Minimum projection types:
+
+- `self_service`: fields the current user may see or change.
+- `admin`: fields visible to a scope administrator.
+- `application_runtime`: fields a registered application may consume.
+- `oidc_claims`: fields safe to map into OIDC-style profile claims.
+- `audit`: redacted change summary for audit and diagnostics.
+- `agent_context`: policy-filtered profile/preferences for delegated or
+  autonomous agent use.
+
+Projection defaults should be deny-by-default for sensitive attributes.
+Sensitivity and mutability should be enforced in the projection layer even
+after flex-auth authorizes the broader action.
+
+## 7. API Surface Blueprint
+
+The first API should be headless and UI-neutral.
+
+Suggested high-level resources:
+
+```text
+GET    /health
+GET    /ready
+
+GET    /me
+PATCH  /me/profile
+GET    /me/applications/{application_id}/profile
+PATCH  /me/applications/{application_id}/profile
+GET    /me/identities
+
+GET    /users/{user_id}
+PATCH  /users/{user_id}
+POST   /users/{user_id}/lifecycle/{transition}
+
+GET    /tenants/{tenant_id}/users
+GET    /tenants/{tenant_id}/memberships
+POST   /tenants/{tenant_id}/memberships
+DELETE /tenants/{tenant_id}/memberships/{membership_id}
+
+GET    /applications
+POST   /applications
+GET    /applications/{application_id}
+PATCH  /applications/{application_id}
+
+GET    /applications/{application_id}/catalogs
+POST   /applications/{application_id}/catalogs
+POST   /applications/{application_id}/catalogs/{catalog_id}/activate
+POST   /applications/{application_id}/catalogs/{catalog_id}/deprecate
+
+GET    /profiles/effective
+GET    /projections/{projection_type}
+
+GET    /audit
+```
+
+API design rules:
+
+- Every mutating endpoint requires an authenticated IAM Profile actor.
+- Every scoped endpoint must carry an explicit tenant/application/team context.
+- Admin endpoints are scope-admin endpoints, not platform-root shortcuts.
+- The `me` API uses the linked current actor, not a client-supplied user id.
+- Applications may register catalogs only for namespaces they own.
+- Attribute updates must validate against the active catalog version.
+- Projection responses must not leak hidden sensitive values in validation
+  errors, explanations, or audit summaries.
+
+## 8. Event And Audit Blueprint
+
+`user-engine` should implement a transactional outbox so profile changes,
+membership changes, catalog changes, and lifecycle changes cannot commit
+without a corresponding event record.
+
+Initial domain events:
+
+- `user.created`
+- `user.updated`
+- `user.activated`
+- `user.disabled`
+- `user.suspended`
+- `user.deletion_requested`
+- `user.deleted`
+- `identity.linked`
+- `identity.unlinked`
+- `profile.updated`
+- `membership.changed`
+- `application.registered`
+- `catalog.registered`
+- `catalog.activated`
+- `catalog.deprecated`
+- `projection.changed`
+
+Audit records should include:
+
+- actor identity envelope
+- subject user or resource
+- action
+- tenant/application/scope
+- authorization decision id or correlation id from flex-auth where available
+- high-level change summary
+- sensitivity-aware redaction
+- timestamp and source client
+
+Audit is part of user-engine's own accountability, but platform-wide durable
+audit retention should integrate with the Railiance/NetKingdom audit sink when
+available.
+
+## 9. Standalone To Platform Progression
+
+### 9.1 Standalone Setup
+
+Standalone mode should prove reuse without a full platform:
+
+- local or external OIDC issuer;
+- SQLite or local PostgreSQL;
+- local config file for application registration;
+- in-process authorization adapter for development only, or a small allowlist
+  policy with clear production warnings;
+- same catalog and profile APIs as platform mode.
+
+Recommended decision: do not implement passwords or MFA in standalone mode.
+Use `local-identity` for local OIDC or an external OIDC provider. This keeps
+the standalone path useful without starting identity-provider creep.
+
+### 9.2 NetKingdom Platform Setup
+
+Platform mode should be the normal integrated setup:
+
+- key-cape or Keycloak issues IAM Profile tokens.
+- flex-auth authorizes user-engine actions.
+- PostgreSQL stores canonical state.
+- OpenBao provides runtime secrets.
+- event/audit outbox drains to platform services.
+- app and tenant identifiers follow NetKingdom tenant semantics.
+
+### 9.3 Enterprise Setup
+
+Enterprise mode adds:
+
+- Keycloak expanded mode for inbound enterprise federation and SAML/Entra/AD
+  brokering;
+- SCIM-aligned import/provisioning adapters;
+- directory/group resolver integration for group overage and freshness;
+- stronger audit retention, export, and review workflows;
+- tenant and admin delegation models.
+
+SCIM should initially be a compatibility target and adapter boundary, not the
+MVP's source of truth.
+
+## 10. Implementation Roadmap
+
+### Phase 0 - Contracts And Skeleton
+
+- Add `SCOPE.md` for user-engine.
+- Add API and catalog descriptor draft specs.
+- Define resource/action vocabulary for flex-auth.
+- Define user-engine event names and audit schema.
+- Choose implementation stack and repository layout.
+
+### Phase 1 - Headless Core MVP
+
+- Implement domain core for users, accounts, identity links, applications,
+  catalogs, profile values, and effective profile resolution.
+- Implement persistence with migration support.
+- Implement REST API and health/readiness endpoints.
+- Implement transactional audit and outbox tables.
+- Add SDK/client generation or a minimal typed client.
+
+### Phase 2 - NetKingdom Integration
+
+- Add IAM Profile claim adapter and production issuer validation.
+- Add flex-auth check client and protected resource/action manifests.
+- Add OpenBao/Railiance-ready secret configuration.
+- Add event sink adapter for the platform event path.
+- Add conformance tests with local-identity and IAM Profile fixtures.
+
+### Phase 3 - Catalog And Projection Hardening
+
+- Add catalog versioning and activation/deprecation rules.
+- Add migration checks for catalog changes.
+- Add projection definitions for self-service, admin, application runtime,
+  OIDC claims, audit, and agent context.
+- Add attribute-level sensitivity, mutability, visibility, and explanation
+  tests.
+
+### Phase 4 - Enterprise Adapters
+
+- Add SCIM-aligned inbound provisioning adapter.
+- Add Keycloak/expanded-mode provisioning integration where needed.
+- Add directory/group resolver integration for freshness/overage handling.
+- Add export/import and tenant onboarding diagnostics.
+
+### Phase 5 - UI Consumers
+
+- Build `user-account` as a self-service UI over the same APIs.
+- Build `user-manager` as a scoped admin UI over the same APIs.
+- Keep UI-specific form rendering downstream of catalog/projection metadata.
+
+## 11. Testing And Conformance
+
+Minimum test suites:
+
+- domain model unit tests;
+- catalog descriptor validation tests;
+- profile resolution precedence tests;
+- projection redaction and mutability tests;
+- IAM Profile token acceptance/rejection tests;
+- flex-auth allow/deny/obligation integration tests;
+- tenant isolation tests;
+- account lifecycle transition tests;
+- audit/outbox atomicity tests;
+- local standalone smoke test;
+- platform-mode smoke test with local-identity or fixture issuer.
+
+Production-readiness checks should include:
+
+- rejects local-development issuers in production;
+- validates issuer, audience, expiry, not-before, signature, and algorithm
+  either directly or through a trusted upstream gateway;
+- enforces tenant context on every scoped request;
+- denies profile access when flex-auth denies;
+- redacts sensitive attributes in all non-eligible projections;
+- persists audit records for administrative and lifecycle changes;
+- drains outbox without losing committed changes.
+
+## 12. Open Decisions
+
+| Decision | Recommendation |
+| --- | --- |
+| Implementation language/framework | Choose after first API/design spike; keep domain core independent from HTTP |
+| Standalone authentication | Use `local-identity` or external OIDC; do not implement passwords/MFA |
+| First database | PostgreSQL for platform, optional SQLite for local standalone |
+| Catalog descriptor format | JSON/YAML using JSON Schema-compatible validation plus user-engine governance metadata |
+| Membership source of truth | user-engine owns user-domain memberships; IAM/flex-auth consume or mirror only what they need |
+| SCIM timing | Adapter boundary after MVP, not initial source of truth |
+| NetKingdom responsibility-map update | Do in net-kingdom if/when user-engine becomes a shared platform service |
+
+## 13. First Implementation Slice
+
+The smallest useful slice should be:
+
+1. Authenticate a request with a local IAM Profile fixture or local-identity.
+2. Link `iss + sub` to a `user_id`.
+3. Register one application.
+4. Register one catalog with a few profile/preference attributes.
+5. Store global and application profile values.
+6. Resolve an effective application profile projection.
+7. Authorize the read/update path through a pluggable flex-auth interface,
+   using a local test adapter until flex-auth is wired.
+8. Emit audit and outbox events for every mutation.
+
+This slice proves the product thesis without duplicating identity provider,
+MFA, authorization engine, UI, or deployment-platform responsibilities.