audit-core/docs/audit-backend-contract.md

# Audit Backend Contract

Audit Core separates **event producers** (integrations, CLI, future HTTP API) from
**audit backends** (sinks that persist normalized events). This document defines
the replaceable backend interface, the current event schema, retention guarantees,
and how to migrate from the development mock file backend to durable custody.

## AuditBackend protocol

Every backend implements `audit_core.interface.AuditBackend`:

```python
class AuditBackend(Protocol):
    @property
    def retention_policy(self) -> RetentionPolicy: ...

    def emit(self, event: AuditEvent) -> str: ...
```

### `emit(event) -> str`

Persist one normalized `AuditEvent` and return a backend-specific reference.
Callers use the reference for debugging and correlation; it is not a stable
cross-backend identifier.

| Backend kind | Typical return value |
| --- | --- |
| Mock file | Absolute path to the hourly JSONL file |
| Archive (planned) | Batch URI or object key |
| Hot search (planned) | Stream offset or index document id |

Requirements:

- **Idempotent references:** Re-emitting the same logical event (same
  `event_id`) must not corrupt prior records. Backends may append duplicates
  unless deduplication is documented.
- **No secret dumping:** Backends must not log or persist plaintext secrets,
  tokens, keys, or passwords from `details` or future payload fields.
- **Visible failure:** Raise on persistence failure; do not silently drop events.
- **Normalization only:** Backends receive `AuditEvent` instances. Source-specific
  adapters run upstream.

### `retention_policy`

Each backend exposes a frozen `RetentionPolicy` describing what it guarantees.
Integrators and readiness checks use this to decide whether a sink satisfies a
scope's policy. See [Retention policy](#retention-policy).

## Event schema (`audit-core.event.v1alpha1`)

The first implementation uses a flat JSON record produced by `AuditEvent.as_record()`.
This is a deliberate simplification for local wiring; the long-term envelope in
`spec/ProductRequirementsDefinition.md` (`audit-core.event.v1`) nests source,
tenant, scope, actor, and result objects.

### Required fields

| Field | Type | Description |
| --- | --- | --- |
| `schema_version` | string | Must be `audit-core.event.v1alpha1` for this contract |
| `event_id` | string | UUID or source-stable identifier |
| `observed_at` | string | UTC ISO-8601 timestamp (no subsecond precision) |
| `tenant` | string | Tenant id; default `platform` for control-plane events |
| `scope` | string | Scope id; default `platform-control-plane` |
| `source` | string | Emitter id (e.g. `openbao`, `audit-core`) |
| `action` | string | Namespaced action (e.g. `openbao.audit.list`) |
| `resource` | string | Affected resource path or id |
| `outcome` | string | Result label (e.g. `success`, `failure`, `denied`) |

### Optional fields

| Field | Type | Description |
| --- | --- | --- |
| `actor` | string or null | Subject performing the action |
| `reason` | string or null | Human-readable result explanation |
| `details` | object | Source-specific extension map; must not contain secrets |

### Example record

```json
{
  "action": "openbao.authenticated_readiness_proof",
  "actor": null,
  "details": {"backend": "mock-file", "file_audit_visible": true},
  "event_id": "6f3e2b1a-4c5d-6e7f-8a9b-0c1d2e3f4a5b",
  "observed_at": "2026-06-01T20:30:00+00:00",
  "outcome": "success",
  "reason": null,
  "resource": "openbao/openbao-0",
  "schema_version": "audit-core.event.v1alpha1",
  "scope": "platform-control-plane",
  "source": "openbao",
  "tenant": "platform"
}
```

### Validation

Use `audit_core.interface.validate_event()` before emit. It checks required
fields, `schema_version`, and rejects empty strings on required identifiers.

### Evolution to `audit-core.event.v1`

Future backends will accept the nested v1 envelope at the HTTP ingestion layer.
Module-level backends may continue using `AuditEvent` with adapter translation.
Compatibility rules (not yet implemented):

- v1alpha1 records remain readable in archive export.
- New required v1 fields get sensible defaults during adapter migration.
- `schema_version` gates parser selection.

## Retention policy

`RetentionPolicy` is declarative metadata; enforcement is backend-specific.

| Field | Meaning |
| --- | --- |
| `custody_class` | `development`, `archive`, or `hot_search` |
| `retention_days` | Maximum age before eligible deletion; `None` means indefinite |
| `immutable` | Whether stored records are protected from in-place alteration |
| `tamper_evidence` | Whether manifests, hash chains, or signatures exist |
| `durable` | Whether survival is expected across process restarts and host reboots |

### Custody classes

| Class | Purpose | Guarantees |
| --- | --- | --- |
| `development` | Local integration and bootstrap wiring | Ephemeral local files; best-effort cleanup; **not audit custody** |
| `archive` | Long-term evidence (planned) | Durable object storage, batch manifests, explicit retention |
| `hot_search` | Operational investigation (planned) | Shorter retention; searchable; not the evidence record |

### Mock file backend policy

`MockFileAuditBackend.retention_policy`:

- `custody_class`: `development`
- `retention_days`: 7 (override via `AUDIT_CORE_MOCK_RETENTION_DAYS` or constructor)
- `immutable`: false
- `tamper_evidence`: false
- `durable`: false

Enforcement:

- Events append to hourly JSONL files under `AUDIT_CORE_MOCK_DIR` (default
  `/tmp/audit-core`).
- `cleanup_old_files()` deletes `audit-*.jsonl` whose mtime is older than
  `retention_days`. Cleanup runs on each `emit` and via `python3 -m audit_core cleanup`.
- Negative `retention_days` disables automatic deletion.

**Not guaranteed:** crash-safe writes, replication, encryption, tenant isolation,
integrity proofs, or survival of `/tmp` across reboots.

### Production archive policy (planned)

Target guarantees for the first durable backend:

- `custody_class`: `archive`
- `retention_days`: scope policy (often years, sometimes indefinite)
- `immutable`: true (WORM / object lock where available)
- `tamper_evidence`: true (batch manifests with content hashes)
- `durable`: true

## Migration path: mock file → durable backend

### Phase 0 — today (mock file)

Use `MockFileAuditBackend` or the CLI:

```bash
python3 -m audit_core emit \
  --source my-service \
  --action my_service.audit.smoke \
  --resource my-service/instance-0 \
  --outcome success
```

Integrations import the protocol, not the mock:

```python
from audit_core import AuditBackend, AuditEvent, MockFileAuditBackend

backend: AuditBackend = MockFileAuditBackend()
backend.emit(AuditEvent(source="...", action="...", resource="...", outcome="success"))
```

### Phase 1 — dual-write readiness (planned)

1. Register a durable archive backend implementing `AuditBackend`.
2. Configure routing: development scopes may keep mock; production scopes require
   `custody_class=archive`.
3. Readiness checks compare `backend.retention_policy` against scope policy and
   fail closed when custody is insufficient.

### Phase 2 — archive primary (planned)

1. Point `emit` calls (or HTTP ingestion) at the archive backend.
2. Retain mock only for local `make mock-audit-smoke` and unit tests.
3. Export historical mock JSONL into archive batches with manifest generation.

### Phase 3 — hot search adjunct (planned)

Add a second `AuditBackend` with `custody_class=hot_search` for investigation.
Archive remains the evidence record; hot search may use shorter `retention_days`.

### Code migration checklist

| Step | Action |
| --- | --- |
| 1 | Depend on `AuditBackend`, not `MockFileAuditBackend`, in integration code |
| 2 | Build `AuditEvent` with explicit `tenant`, `scope`, and `source` |
| 3 | Call `validate_event()` before emit |
| 4 | Inspect `retention_policy` in readiness gates |
| 5 | Replace mock construction with injected backend from configuration |
| 6 | Verify export/manifest workflow before decommissioning mock files |

## Reference implementations

| Backend | Module | Custody class |
| --- | --- | --- |
| Mock file JSONL | `audit_core.mock_file_backend.MockFileAuditBackend` | `development` |
| Archive (planned) | TBD | `archive` |
| Hot search (planned) | TBD | `hot_search` |

## Related documents

- `INTENT.md` — product purpose and principles
- `spec/ProductRequirementsDefinition.md` — full v1 envelope and API requirements
- `registry/capabilities/capability.audit.event-retain.md` — capability registry entry