flex-auth/docs/topaz-mapping-spike.md

# Topaz Alignment Spike

Date: 2026-05-16
Status: Spike output for FLEX-WP-0005 P5.4. Feeds FLEX-WP-0002 P2.1
(schemas) and FLEX-WP-0004 T01 (Topaz adapter).
Author: claude with Bernd

## Goal

Validate ADR-003's commitment to shape the flex-auth standalone core so
that the Topaz adapter is a small step. Concretely: confirm that
flex-auth's resource, subject, group, team, and relationship vocabulary
can map cleanly onto Topaz directory objects and relations, and that
flex-auth's Rego-in-Markdown policy packages decompose to Topaz policy
modules without translation.

## Topaz In Two Paragraphs

Topaz bundles three things behind one process: an OPA Rego evaluator,
an embedded Zanzibar-style directory ("EDS" — Edge Authorization
Database), and a gRPC/HTTP API that ties them together. The directory
stores objects (typed entities) and relations (typed edges between
objects). A `manifest` declares the object types, relation types, and
permission expressions. Policy modules in Rego consult both the request
input *and* directory data (via builtins like `ds.check`,
`ds.object`, `ds.relation`) to produce a decision.

Topaz exposes four service surfaces: **reader** (read directory),
**writer** (mutate directory), **model** (manage manifest), and
**authorizer** (evaluate policy). The `authorizer.Is` and
`authorizer.DecisionTree` RPCs are the primary integration points for a
policy decision point.

## Vocabulary Map

flex-auth's canonical vocabulary maps onto Topaz as follows.

| flex-auth concept | Topaz representation | Notes |
| --- | --- | --- |
| Subject (human) | `object{type:"user", id:<oidc subject>}` | `display_name` from `preferred_username` or claims |
| Subject (service account) | `object{type:"user", id:<sub>}` with `properties.principal_type:"service"` | Same type; principal_type carried in properties |
| Group | `object{type:"group", id:<group slug>}` | Members via `member` relation |
| Team | `object{type:"group", id:"team:<slug>"}` | Modeled as a kind of group; slug-prefixed for clarity |
| Tenant | `object{type:"tenant", id:<slug>}` | Resources reference tenant via `tenant` relation |
| Group membership | `relation{subject:user, name:"member", object:group}` | Recursive group→group also supported |
| Resource (Markitect knowledge_base / document / …) | `object{type:<flex_auth_type>, id:<resource id>}` | Type names match `FlexAuthResource.type` exactly |
| Resource parent | `relation{subject:child, name:"parent", object:parent}` | Walks inheritance |
| Resource owner team | `relation{subject:group, name:"owner_team", object:resource}` | Group-on-resource |
| Resource labels | `object.properties.labels: [...]` | Carried as JSON properties; consulted by Rego |
| Resource trust_zone | `object.properties.trust_zone: <str>` | Same |
| Action | Topaz permission name | `read`, `query`, `search`, `package`, `export`, `workflow_run`, `admin` |
| Relationship fact (subject↔resource) | `relation{subject, name:<relation>, object:resource}` | e.g. `reader`, `editor`, `steward` |
| Policy package (Rego-in-Markdown) | One Rego module per package | Extracted from the Markdown by flex-auth's loader (ADR-002) |
| Decision envelope | Topaz `authorizer.Is.Response` + Rego decision object | Topaz returns booleans per decision name; the Rego program returns the richer envelope and flex-auth wraps |
| Policy version | `metadata.version` in the package frontmatter | Threaded as `input.policy.version` for Topaz; recorded in audit |
| Provenance | `metadata` on Topaz objects/relations and on the decision | Topaz exposes `etag` for relations; flex-auth records both sides |

### Why these specific choices

- **Subject and service-account share `user` type.** Topaz's
  authorizer.Is treats `user` as the canonical identity subject. The
  `principal_type` distinction lives in `properties` so Rego rules
  can branch on it, but the directory shape stays uniform.
- **Team modeled as group.** Markitect (and most consumers) treat
  teams as groups for authorization purposes. A separate `team` type
  is possible but adds an indirection. The `team:` id-prefix keeps
  semantics legible without forking the type.
- **Labels and trust_zone in `properties` rather than as separate
  objects.** They are attributes of the resource, not first-class
  edges. Putting them in properties keeps the directory small and
  policies legible.
- **Owner is a relation, not a property.** A team owns multiple
  resources; the inverse query "what does team T own?" is a relation
  walk, not a property scan.

## Rego Module Shape

A flex-auth Rego-in-Markdown package (ADR-002) extracts to a single
Rego module. The same module runs in both flex-auth's embedded
evaluator and in Topaz, because Topaz exposes the same `data.<package>`
namespace and supports the same `ds.*` builtins flex-auth's adapter
will mock when running standalone.

### Standalone-mode contract

When evaluated by the flex-auth standalone evaluator
(`internal/policy`), the module sees:

```text
input.subject      # normalized subject (id, groups, roles, scopes, principal_type, assurance)
input.action       # action name
input.resource     # normalized resource (id, type, labels, trust_zone, owner, parent, …)
input.context      # request/environment/assurance/workflow attributes
data.policy        # registered package metadata table
data.directory     # standalone registry contents, exposed under the same shape ds.* would return
```

The `ds.*` builtins are **shimmed locally** in standalone mode:
flex-auth provides `ds.check`, `ds.object`, and `ds.relation`
implementations that consult `internal/registry` instead of Topaz.
The shim is a Go-side `rego.Function` registration.

### Delegated-mode contract

When evaluated by Topaz, the same module runs unchanged. `ds.*` calls
hit Topaz's directory. The decision envelope returned to the protected
system is identical because flex-auth's adapter wraps Topaz's response
into the same `decision_envelope.schema.json` shape (FLEX-WP-0002
P2.1).

### Decision object shape

The Rego package's `decision` rule produces:

```rego
decision := {
  "effect": "allow" | "deny" | "redact" | "audit_only" | "not_applicable",
  "reason": "<rule id>",
  "obligations": [...],     # optional, per ADR-002
  "diagnostics": {...},     # optional, free-form
}
```

flex-auth's adapter (standalone *or* Topaz) wraps this into the full
decision envelope by adding subject metadata, resource metadata,
matched policy version, matched rule, and provenance.

## Wire-Protocol Candidates

Ranked from primary to fallback:

1. **gRPC (Topaz's native API)** — primary. Type-safe, generated Go
   clients (`github.com/aserto-dev/topaz/pkg/cli` exposes the protos),
   low overhead. Authorizer.Is, DecisionTree, Reader/Writer/Model.
2. **HTTP/REST gateway** — fallback for environments where gRPC is
   awkward (e.g. browser tooling, debugging). Topaz exposes a REST
   gateway on a sibling port; same RPCs, JSON-encoded.
3. **Embedded library** — *not* recommended. Topaz's directory is
   tightly coupled to its process model and bolt/postgres storage;
   embedding it ties flex-auth's release cycle to Topaz's and erodes
   the adapter boundary. Reserved as a future optimization if a
   single-process deployment is ever required.

Adapter implementation under `internal/adapters/topaz/` consumes the
gRPC client. The HTTP gateway is documented but not implemented in
FLEX-WP-0004 T01.

## Adapter Surface (Preview For FLEX-WP-0004 T01)

```go
package topaz

type Adapter interface {
    // Check evaluates a single decision against Topaz.
    Check(ctx context.Context, req CheckRequest) (DecisionEnvelope, error)

    // BatchCheck evaluates multiple resources for one subject+action.
    BatchCheck(ctx context.Context, req BatchCheckRequest) ([]DecisionEnvelope, error)

    // ImportDirectory writes flex-auth registry data into Topaz.
    ImportDirectory(ctx context.Context, snapshot DirectorySnapshot) error

    // ImportPolicy installs a bundle (Rego modules extracted from
    // policy packages) into Topaz's OPA.
    ImportPolicy(ctx context.Context, bundle PolicyBundle) error
}
```

`DirectorySnapshot` and `PolicyBundle` are the canonical flex-auth
types from `pkg/api/`; the adapter translates them into Topaz's
proto types at the boundary.

## Recommendation: Schema Restatement, Not Embedding

flex-auth should **restate** its directory model in its own canonical
schemas (FLEX-WP-0002 P2.1), not embed Topaz's directory proto.
Reasoning:

- **The map is small.** flex-auth's vocabulary is six object kinds
  (user, group, tenant, resource, …) and a handful of relation
  conventions. The translation cost is tiny.
- **Vocabulary divergence is likely.** flex-auth wants to record
  things Topaz's directory does not natively model — e.g. policy
  package version on a decision, IAM Profile assurance bundles,
  decision-time provenance. Embedding would force those into
  Topaz-shaped properties bags.
- **Multi-backend goal.** OpenFGA and SpiceDB are next on the adapter
  list; both have similar-but-not-identical models. flex-auth's
  canonical schemas need to be the lingua franca, not Topaz's.
- **Stable contract for Markitect.** Markitect already emits
  `FlexAuthResourceManifest`; that is the canonical input. Topaz is a
  consumer of the translation, not the other way around.

Adopt Topaz's manifest *style* (object types + relation types +
permission expressions) for the standalone evaluator's directory
shim, but keep flex-auth's own schema files as the source of truth.

## Runnable Example

`examples/topaz/` ships:

- `manifest.yaml` — Topaz v3 directory manifest declaring `user`,
  `group`, `tenant`, `knowledge_base`, `document` object types and
  relations `member`, `parent`, `owner_team`, `reader`, `steward`,
  plus the permission expressions for `read`, `query`, `search`,
  `export`, `admin`.
- `policy/markitect.documents.rego` — Rego module matching the
  Rego-in-Markdown example in ADR-002 (internal document read gated
  on `reader` relation or `steward` role). This module shows the
  *flex-auth-shaped* input contract — bridging to Topaz's raw input
  is adapter scope (see Implementation Notes below).
- `data/objects.json`, `data/relations.json` — seed directory data
  derived from `examples/markitect/resource_manifest.yaml` plus a
  small subject/group set.
- `cfg/config.yaml` — Topaz v2 config (BoltDB at `/db`, TLS auto-
  managed in `/certs`, plaintext HTTP gateways for spike convenience,
  local bundle from `/bundle`).
- `docker-compose.yml` — Topaz service + a one-shot `seed` container
  (pushes the manifest, objects, relations via the directory REST
  gateway) + a one-shot `probe` container that calls the directory
  `check` API for three scenarios.
- `README.md` — usage.

**Verification status: green (2026-05-16).** `docker compose up
--abort-on-container-exit` produced all three expected outcomes:

| Scenario | Subject | Resource | Permission | Expected | Actual |
| --- | --- | --- | --- | --- | --- |
| steward allow | alice (steward) | document:internal-note | read | true | true |
| reader allow | bob (member of reader:platform-architecture) | document:internal-note | read | true | true |
| outsider deny | eve (no relation) | document:internal-note | read | false | false |

## Implementation Notes Surfaced By The Spike

1. **Rego input contract bridging is adapter scope.** Topaz's
   authorizer delivers a fixed input shape
   (`input.user`, `input.identity`, `input.policy`, `input.resource`)
   that comes from its identity-resolution flow. flex-auth's
   policy packages use the canonical input shape from ADR-002
   (`input.subject`, `input.action`, `input.resource`, `input.context`).
   The Topaz adapter (FLEX-WP-0004 T01) is responsible for the wrapper
   that translates between the two. The standalone evaluator (P2.4)
   produces the canonical shape directly. **Implication for P2.1:**
   the canonical input shape must be locked early so both code paths
   target it.
2. **Identity objects are Topaz-canonical, not flex-auth-canonical.**
   Topaz expects an `identity` object type with an `identifier`
   relation to a `user` object when `identity_context.type ==
   IDENTITY_TYPE_SUB`. flex-auth's subject model is flat (a Subject
   *is* the principal). The adapter materializes Topaz `identity`
   objects from the flex-auth subject registry at directory-import
   time, mapping subject id → identity → user. The standalone path
   doesn't need this indirection.
3. **Permission resolution from the directory alone is sufficient
   for the common case.** The probe demonstrates that the Topaz
   manifest's permission expressions (`reader | steward | parent->read`
   etc.) resolve correctly with only directory data. Rego is only
   needed when policy decisions consume request context (assurance
   claims, time-of-day, trust_zone interaction) that the directory
   can't express. **Implication for P2.3/P2.4:** the policy package
   loader and check API should accept "permission-only" packages
   (no Rego rules, just a permission expression referencing the
   directory manifest) as a first-class case alongside Rego packages.
4. **The Topaz config schema is not stable across versions.** Keys
   moved between `directory:` and `directory_service:`; relation
   rewrites use `->` only in `permissions:` not `relations:`;
   `disable_tls` is not honored as a server flag (use cert paths in
   a writable volume). The FLEX-WP-0004 T01 adapter implementation
   should pin a tested Topaz version and bump it deliberately.

## Implications for FLEX-WP-0002

- **P2.1 schemas** — define `directory_snapshot.schema.json` with the
  vocabulary above. Decision envelope unchanged from ADR-002 sketch.
- **P2.2 registry** — store objects+relations in the same canonical
  shape; the in-memory representation is what gets translated to
  Topaz tuples at adapter time.
- **P2.3 policy loader** — Markdown extractor produces *one* Rego
  module per package, identical for standalone and Topaz delivery.
  Standalone evaluator registers `ds.*` shim builtins.
- **P2.4 check API** — `Check` returns the canonical envelope; the
  same code path serves standalone evaluation and Topaz-delegated
  evaluation; only the evaluator and `ds.*` resolution differ.

## Implications for FLEX-WP-0004

- **T01 Topaz adapter** — implement against the surface preview
  above; consume the proto client from `github.com/aserto-dev/topaz`.
  No new schema work; adapter is a translator and request/response
  shaper.
- **T02 relationship PDP boundary** (OpenFGA, SpiceDB) — reuse the
  same `DirectorySnapshot` and decision envelope; only the directory
  protocol differs. The Topaz mapping above is reusable with small
  per-backend deltas.

## Open Items (Not Blocking P2.1)

- **Bundle distribution.** Topaz can load Rego from a remote bundle
  service (`opa.bundles`). Decision: load via local bundle path in
  the standalone deployment; switch to bundle service when running
  Topaz in clustered mode. Recorded for FLEX-WP-0004 T01.
- **Consistency tokens.** Topaz's relation writes return an `etag`.
  Decision envelopes should carry it for read-your-writes guarantees;
  envelope field `provenance.directory_etag` is reserved for this.
- **Group overage / freshness.** Out of scope for this spike; handled
  by directory resolver adapters in FLEX-WP-0004 T05.