Implement credentialed drill packaging workplan

This commit is contained in:
2026-05-19 01:27:59 +02:00
parent 022cd8d37e
commit 6e0372d21a
23 changed files with 924 additions and 43 deletions

View File

@@ -96,6 +96,7 @@ for retrieval and evaluation behavior, [docs/service-readiness.md](docs/service-
for service and adapter contracts, [docs/lifecycle-rules.md](docs/lifecycle-rules.md) for service and adapter contracts, [docs/lifecycle-rules.md](docs/lifecycle-rules.md)
for profile-driven lifecycle rules, [docs/external-adapter-packs.md](docs/external-adapter-packs.md) for profile-driven lifecycle rules, [docs/external-adapter-packs.md](docs/external-adapter-packs.md)
for fake external integration packs, [docs/operational-readiness.md](docs/operational-readiness.md) for fake external integration packs, [docs/operational-readiness.md](docs/operational-readiness.md)
for the local end-to-end operational recipe, [docs/api-compatibility.md](docs/api-compatibility.md) for the local end-to-end operational recipe, [docs/operator-readiness-runbook.md](docs/operator-readiness-runbook.md)
for operator readiness, [docs/api-compatibility.md](docs/api-compatibility.md)
for public API compatibility expectations, [docs/maturity-scorecard.md](docs/maturity-scorecard.md) for public API compatibility expectations, [docs/maturity-scorecard.md](docs/maturity-scorecard.md)
for the current maturity assessment, and [SCOPE.md](SCOPE.md) for repository boundaries. for the current maturity assessment, and [SCOPE.md](SCOPE.md) for repository boundaries.

View File

@@ -23,6 +23,11 @@ The snapshot in `tests/fixtures/public-api-snapshot.json` intentionally fails
when exports or service operations change. Update the snapshot only when the when exports or service operations change. Update the snapshot only when the
change is deliberate and documented. change is deliberate and documented.
The snapshot also points to `docs/release-note-template.md`. When a public API
snapshot changes, fill in that template or a release note derived from it so
operators can see changed exports, changed service operations, migration needs,
and required action.
## Adding Operations ## Adding Operations
When adding a service operation: When adding a service operation:

View File

@@ -26,24 +26,24 @@ to 5.
## Current Score ## Current Score
Overall maturity: **4.2 / 5** Overall maturity: **4.3 / 5**
Two sub-scores make the result easier to reason about: Two sub-scores make the result easier to reason about:
- Local integration maturity: **4.5 / 5** - Local integration maturity: **4.6 / 5**
- Operational maturity: **3.8 / 5** - Operational maturity: **4.0 / 5**
The repo is strong as a deterministic local library and service-boundary core. The repo is strong as a deterministic local library and service-boundary core.
It is not yet production-operational because adapter coverage is still It is not yet production-operational because adapter coverage is still
live-shaped rather than credentialed live integration, and service bindings are credential-gated rather than continuously exercised against live services, and
framework-neutral embedding surfaces rather than a deployed service. service packaging is stdlib/local rather than deployed to a managed environment.
## Dimension Scorecard ## Dimension Scorecard
| Dimension | Score | Target | Evidence | Needed Next | | Dimension | Score | Target | Evidence | Needed Next |
| --- | ---: | ---: | --- | --- | | --- | ---: | ---: | --- | --- |
| Intent and boundaries | 4.4 | 5.0 | `INTENT.md`, `SCOPE.md`, `README.md`, architecture docs, adjacent-repo boundary docs | Keep docs current as live adapters and service bindings clarify real ownership. | | Intent and boundaries | 4.4 | 5.0 | `INTENT.md`, `SCOPE.md`, `README.md`, architecture docs, adjacent-repo boundary docs | Keep docs current as live adapters and service bindings clarify real ownership. |
| Package and API foundation | 4.5 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot | Add release notes discipline and compatibility migration examples. | | Package and API foundation | 4.6 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot, release-note template | Add compatibility migration examples from a real release. |
| Markitect profile contract ingress | 3.7 | 4.5 | Profile loading, diagnostics, runtime envelopes, profile-derived config, local alias normalization | Add richer compatibility fixtures and schema drift diagnostics. | | Markitect profile contract ingress | 3.7 | 4.5 | Profile loading, diagnostics, runtime envelopes, profile-derived config, local alias normalization | Add richer compatibility fixtures and schema drift diagnostics. |
| Graph and event ingress | 4.0 | 4.5 | Graph loading, endpoint diagnostics, event model, JSONL log, export, repair checks, corrupt-record diagnostics, fake and live-shaped graph/event adapters | Add broader malformed/large graph fixtures and operator repair utilities. | | Graph and event ingress | 4.0 | 4.5 | Graph loading, endpoint diagnostics, event model, JSONL log, export, repair checks, corrupt-record diagnostics, fake and live-shaped graph/event adapters | Add broader malformed/large graph fixtures and operator repair utilities. |
| Phase domain model | 3.5 | 4.5 | Phases, lifecycle states, actions, paths, retention rules, profile-derived transition rules | Add migration semantics for profile/rule changes over durable stores. | | Phase domain model | 3.5 | 4.5 | Phases, lifecycle states, actions, paths, retention rules, profile-derived transition rules | Add migration semantics for profile/rule changes over durable stores. |
@@ -51,26 +51,26 @@ framework-neutral embedding surfaces rather than a deployed service.
| Lifecycle planning and apply | 4.1 | 4.5 | Dry-run lifecycle plans, profile rules, review-gated local apply, service `lifecycle.apply`, apply audit/export queries | Add richer apply rollback and repair drills. | | Lifecycle planning and apply | 4.1 | 4.5 | Dry-run lifecycle plans, profile rules, review-gated local apply, service `lifecycle.apply`, apply audit/export queries | Add richer apply rollback and repair drills. |
| Activation planning | 4.0 | 4.8 | Budgeted activation, selections, package request, graph neighborhoods, paths, ranking, metrics, multi-scenario evaluation fixtures | Wire semantic-index-assisted retrieval into runtime planning. | | Activation planning | 4.0 | 4.8 | Budgeted activation, selections, package request, graph neighborhoods, paths, ranking, metrics, multi-scenario evaluation fixtures | Wire semantic-index-assisted retrieval into runtime planning. |
| Local persistence | 4.0 | 4.5 | File-backed graph store, JSONL event log, audit sink, atomic JSON writes, executable metadata migrations, migration audit, export, repair diagnostics | Add compaction/retention utilities and stronger corruption recovery. | | Local persistence | 4.0 | 4.5 | File-backed graph store, JSONL event log, audit sink, atomic JSON writes, executable metadata migrations, migration audit, export, repair diagnostics | Add compaction/retention utilities and stronger corruption recovery. |
| Policy, review, and audit | 4.2 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and enforceable audit retention pruning. | | Policy, review, and audit | 4.4 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans and apply, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and credentialed telemetry pruning drill. |
| Observability and operations | 4.0 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, fake/live-shaped telemetry audit sinks, operational recipe | Add metrics/event export to external telemetry and deployable service packaging. | | Observability and operations | 4.3 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, stdlib service entrypoint, operator runbook, fake/live-shaped telemetry audit sinks | Add metrics/event export to external telemetry and managed deployment packaging. |
| Markitect interop | 4.0 | 4.5 | Local validation, package request/response envelopes, fake and live-shaped compiler fixtures | Add optional credentialed Markitect compiler adapter and schema drift suite. | | Markitect interop | 4.1 | 4.5 | Local validation, package request/response envelopes, fake/live-shaped compiler fixtures, credential-gated drill contract | Add credentialed Markitect compiler execution and schema drift suite. |
| Kontextual/Infospace interop | 3.7 | 4.5 | Delegation envelope, fake and live-shaped runtime registry, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual adapter drill and broader Infospace restart reports. | | Kontextual/Infospace interop | 3.9 | 4.5 | Delegation envelope, fake/live-shaped runtime registry, credential-gated drill contract, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual execution and broader Infospace restart reports. |
| Testing and evaluation | 4.3 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, API snapshots, and evaluation threshold reports | Add larger regression corpus and threshold trend reports. | | Testing and evaluation | 4.5 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, credential skip gates, API snapshots, evaluation threshold and trend reports | Add larger regression corpus and persisted trend history. |
| Service readiness | 4.5 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, health/readiness, config, adapter conformance | Add deployable packaging and operator readiness runbooks. | | Service readiness | 4.6 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, stdlib service entrypoint, health/readiness, config, adapter conformance | Add managed deployment packaging. |
| Developer experience | 4.3 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, API compatibility docs | Add troubleshooting matrix and release note templates. | | Developer experience | 4.5 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, operator runbook, API compatibility docs, release-note template | Add troubleshooting matrix from real operator feedback. |
## Assessment ## Assessment
The project has crossed the local integration-readiness threshold. The runtime The project has crossed the local integration-readiness threshold. The runtime
envelopes, policy/review model, profile-derived configuration, lifecycle rules, envelopes, policy/review model, profile-derived configuration, lifecycle rules,
local persistence migrations, queryable/exportable audit path, fake and local persistence migrations, queryable/exportable/prunable audit path, fake
live-shaped external pack manifests, service binding, API snapshots, and and live-shaped external pack manifests, credential-gated drills, service
binding and stdlib entrypoint, API snapshots, release discipline, and
conformance helpers form a solid integration boundary. conformance helpers form a solid integration boundary.
The biggest optimization opportunity is now the next operational layer: The biggest optimization opportunity is now the next operational layer:
moving from live-shaped local fixtures to credentialed live adapter drills, running the credential-gated drills against real services, adding managed
packaging the service binding for deployment, and growing evaluation thresholds deployment packaging, and growing evaluation trends into a historical corpus.
into trend reports.
## Completed Refinement Workplan ## Completed Refinement Workplan
@@ -97,20 +97,30 @@ into trend reports.
- evaluation threshold reports over the scenario corpus; - evaluation threshold reports over the scenario corpus;
- public API and service operation compatibility snapshots. - public API and service operation compatibility snapshots.
`PMEM-WP-0013` moved the score from 4.2 to 4.3 by adding:
- credential-gated adapter drill helpers and skipped smoke tests that list
required environment variables;
- stdlib `phase-memory-service` packaging with check mode and WSGI dispatch;
- operator readiness runbook for service startup, migrations, audit retention,
credentialed drills, and rollback;
- audit retention apply behavior with audit trace coverage;
- evaluation trend artifacts with threshold and regression deltas;
- release-note template gating for public API snapshot changes.
## Recommended Next Refinement ## Recommended Next Refinement
Create and execute `PMEM-WP-0013`: credentialed adapter drills and deployment Create and execute `PMEM-WP-0014`: live credential execution and managed
packaging. deployment hardening.
Highest-value tasks: Highest-value tasks:
- Add optional credentialed Markitect/Kontextual adapter smoke drills that are - Run the credential-gated drills against real Markitect/Kontextual endpoints
skipped unless credentials are present. in an operator environment.
- Package the service binding as a deployable local service with operator - Add managed deployment packaging and readiness probes.
readiness checks. - Persist evaluation trend reports across runs.
- Add audit retention pruning and telemetry export enforcement. - Add credentialed telemetry export and retention pruning drills.
- Grow evaluation reporting into historical threshold trends. - Expand troubleshooting from actual operator feedback.
- Add release note and migration-note templates for compatibility changes.
## Score Movement Gates ## Score Movement Gates
@@ -121,10 +131,11 @@ Achieved overall score **4.0** when:
- Local persistence has migration diagnostics. - Local persistence has migration diagnostics.
- Evaluation fixtures cover at least three profile/graph families. - Evaluation fixtures cover at least three profile/graph families.
Move overall score to **4.3+** when: Achieved overall score **4.3+** when:
- Credentialed optional Markitect or Kontextual adapter smoke drills run behind - Credentialed optional Markitect or Kontextual adapter smoke drills are
the same conformance suite as the fake/live-shaped packs. available behind the same conformance suite as the fake/live-shaped packs and
skip cleanly without credentials.
- Operational docs include deployable service packaging and an operator - Operational docs include deployable service packaging and an operator
readiness runbook. readiness runbook.

View File

@@ -84,6 +84,16 @@ The WSGI adapter returned by `binding.as_wsgi_app()` is also callable in tests
without opening a socket. Use this for deployment wrappers so the core service without opening a socket. Use this for deployment wrappers so the core service
operation contract stays framework-neutral. operation contract stays framework-neutral.
For the stdlib deployable entrypoint, use:
```bash
phase-memory-service --check --store .phase-memory-local
phase-memory-service --host 127.0.0.1 --port 8080 --store .phase-memory-local
```
See `docs/operator-readiness-runbook.md` for operator checks and rollback
guidance.
## Review-Gated Apply ## Review-Gated Apply
Lifecycle actions that require review are denied until an approval marker or Lifecycle actions that require review are denied until an approval marker or
@@ -143,8 +153,12 @@ retention = runner.runtime.audit_retention_plan(retention_days=30)
``` ```
The export batch includes matching audit events and sink retention metadata. The export batch includes matching audit events and sink retention metadata.
The retention plan identifies eligible operation ids but does not prune records; The retention plan identifies eligible operation ids. Retention apply prunes
retention apply is a follow-on operational task. eligible records and records `audit.retention.apply` after pruning:
```python
retention_apply = runner.runtime.apply_audit_retention(retention["plan"])
```
## Adapter Pack Compatibility ## Adapter Pack Compatibility

View File

@@ -0,0 +1,141 @@
# Operator Readiness Runbook
Updated: 2026-05-19
This runbook covers the operational path for `phase-memory` without requiring
credentials in the default test suite.
## Modes
| Mode | Purpose | Credentials | Network |
| --- | --- | --- | --- |
| Local fixture | Default deterministic runtime and tests. | No | No |
| Live-shaped | Adapter manifests and behavior that model live services locally. | No | No |
| Credentialed live drill | Operator-provided smoke drill for real endpoints. | Yes, via env only | Optional |
Credentialed drills require:
- `PHASE_MEMORY_MARKITECT_URL`
- `PHASE_MEMORY_MARKITECT_TOKEN`
- `PHASE_MEMORY_KONTEXTUAL_URL`
- `PHASE_MEMORY_KONTEXTUAL_TOKEN`
Do not store those values in Git, workplans, progress logs, or release notes.
## Service Startup
The deployable stdlib entrypoint is `phase-memory-service`.
Readiness check without listening:
```bash
phase-memory-service --check --store .phase-memory-local
```
Start the stdlib WSGI service:
```bash
phase-memory-service --host 127.0.0.1 --port 8080 --store .phase-memory-local
```
Routes:
- `GET /health`
- `GET /ready`
- `GET /contracts`
- `POST /operations/{operation}`
- `POST /operations` with `{"operation": "...", "payload": {...}}`
## Readiness Checks
Before accepting traffic:
1. Run `phase-memory-service --check`.
2. Verify `/ready` reports `ok: true`.
3. Verify `unsupported_operations` is empty.
4. Verify adapter diagnostics have no `error` severity.
5. Verify the public API snapshot test passes after any operation/export change.
## Migration Apply
Plan and apply local-store metadata migrations through the runtime:
```python
from phase_memory import RuntimeConfig, runtime_from_config
config = RuntimeConfig(local_store_path=".phase-memory-local")
runtime = runtime_from_config(config)
plan = runtime.plan_store_migration(source_ref=config.local_store_path)
result = runtime.apply_store_migration(
plan["data"]["migration_plan"],
actor="operator",
source_ref=config.local_store_path,
)
```
Expected:
- no `error` diagnostics in the plan;
- `result["valid"] is True`;
- metadata is updated atomically;
- `audit.query` can find the `store.migration.apply` event.
Rollback:
- stop the service;
- restore the previous local store directory from backup;
- rerun `phase-memory-service --check`;
- rerun `runtime.repair_diagnostics()`.
## Audit Export And Retention
Plan retention:
```python
plan = runtime.audit_retention_plan(retention_days=30)
```
Apply retention:
```python
result = runtime.apply_audit_retention(plan["plan"])
```
Expected:
- eligible operation ids are pruned;
- `audit.retention.apply` is recorded after pruning;
- no retention apply happens when the sink reports unsupported behavior.
Export a trace batch:
```python
export = runtime.export_audit_events({"operation": "package.compile"})
```
Use export batches for operator review, not as a credential or secret store.
## Credentialed Drill
Run the credentialed smoke test only from an operator environment:
```bash
PHASE_MEMORY_MARKITECT_URL=... \
PHASE_MEMORY_MARKITECT_TOKEN=... \
PHASE_MEMORY_KONTEXTUAL_URL=... \
PHASE_MEMORY_KONTEXTUAL_TOKEN=... \
python3 -m pytest tests/test_credentialed_drills.py
```
The report redacts tokens and uses a credential fingerprint rather than
persisting secrets.
## Compatibility Release Discipline
When public exports or service operations change:
1. Update `tests/fixtures/public-api-snapshot.json`.
2. Fill in `docs/release-note-template.md`.
3. Call out changed exports, changed service operations, migration needs, and
operator action.
4. Link the workplan or decision that authorized the change.

View File

@@ -0,0 +1,41 @@
# Release Note Template
## Summary
Briefly describe the release and the workplan or decision that authorized it.
## Changed Exports
- Added:
- Changed:
- Removed:
Reference the updated `tests/fixtures/public-api-snapshot.json` entry when the
public export list changes.
## Changed Service Operations
- Added:
- Changed:
- Removed:
Reference `service_contracts()["operations"]` and explain compatibility impact.
## Migration Needs
- Local store migrations:
- Runtime envelope migrations:
- Adapter manifest migrations:
Include whether migration apply is automatic, operator-triggered, or not
required.
## Operator Action
- Required environment variables:
- Service packaging changes:
- Readiness checks:
- Rollback note:
No credentials, tokens, or live endpoints should be included in the release
note.

View File

@@ -16,6 +16,7 @@ dependencies = []
[project.scripts] [project.scripts]
phase-memory = "phase_memory.cli:main" phase-memory = "phase_memory.cli:main"
phase-memory-service = "phase_memory.service_app:main"
[tool.setuptools.packages.find] [tool.setuptools.packages.find]
where = ["src"] where = ["src"]

View File

@@ -10,7 +10,15 @@ from .bridge import (
package_response_envelope, package_response_envelope,
) )
from .contracts import graph_from_markitect, profile_from_markitect from .contracts import graph_from_markitect, profile_from_markitect
from .evaluation import EVALUATION_REPORT_SCHEMA, evaluation_threshold_report from .credentialed_drills import (
CREDENTIALED_ADAPTER_ENV_VARS,
CREDENTIALED_DRILL_SCHEMA,
CredentialedDrillConfig,
credentialed_adapter_smoke_report,
credentialed_drill_config_from_env,
missing_credentialed_adapter_env,
)
from .evaluation import EVALUATION_REPORT_SCHEMA, EVALUATION_TREND_SCHEMA, evaluation_threshold_report, evaluation_trend_artifact
from .external_adapters import ( from .external_adapters import (
ADAPTER_PACK_MANIFEST_SCHEMA, ADAPTER_PACK_MANIFEST_SCHEMA,
ExternalAdapterPack, ExternalAdapterPack,
@@ -76,6 +84,7 @@ from .retrieval import (
select_event_path, select_event_path,
) )
from .service import LocalServiceRunner, RuntimeAdapterBundle, RuntimeConfig, health_report, resolve_runtime_adapters, runtime_from_config, service_contracts from .service import LocalServiceRunner, RuntimeAdapterBundle, RuntimeConfig, health_report, resolve_runtime_adapters, runtime_from_config, service_contracts
from .service_app import SERVICE_APP_SCHEMA, ServiceAppConfig, build_service_binding, create_wsgi_app, service_app_metadata
from .service_binding import READINESS_REPORT_SCHEMA, SERVICE_BINDING_SCHEMA, ServiceBinding, ServiceResponse, service_binding_from_config from .service_binding import READINESS_REPORT_SCHEMA, SERVICE_BINDING_SCHEMA, ServiceBinding, ServiceResponse, service_binding_from_config
from .planner import plan_profile_execution from .planner import plan_profile_execution
from .runtime import PhaseMemoryRuntime from .runtime import PhaseMemoryRuntime
@@ -83,9 +92,13 @@ from .runtime import PhaseMemoryRuntime
__all__ = [ __all__ = [
"ActivationPlan", "ActivationPlan",
"ADAPTER_PACK_MANIFEST_SCHEMA", "ADAPTER_PACK_MANIFEST_SCHEMA",
"CREDENTIALED_ADAPTER_ENV_VARS",
"CREDENTIALED_DRILL_SCHEMA",
"CredentialedDrillConfig",
"Diagnostic", "Diagnostic",
"ExternalAdapterPack", "ExternalAdapterPack",
"EVALUATION_REPORT_SCHEMA", "EVALUATION_REPORT_SCHEMA",
"EVALUATION_TREND_SCHEMA",
"FakeExternalEventLog", "FakeExternalEventLog",
"FakeExternalGraphStore", "FakeExternalGraphStore",
"FakeExternalPolicyGateway", "FakeExternalPolicyGateway",
@@ -130,8 +143,11 @@ __all__ = [
"branch_path", "branch_path",
"compact_path", "compact_path",
"create_path", "create_path",
"credentialed_adapter_smoke_report",
"credentialed_drill_config_from_env",
"graph_from_markitect", "graph_from_markitect",
"evaluation_threshold_report", "evaluation_threshold_report",
"evaluation_trend_artifact",
"merge_path", "merge_path",
"make_review_record", "make_review_record",
"plan_activation", "plan_activation",
@@ -147,6 +163,7 @@ __all__ = [
"fake_external_adapter_pack", "fake_external_adapter_pack",
"fake_external_runtime_config", "fake_external_runtime_config",
"live_shaped_adapter_pack", "live_shaped_adapter_pack",
"missing_credentialed_adapter_env",
"adapter_pack_manifest", "adapter_pack_manifest",
"validate_adapter_pack_manifest", "validate_adapter_pack_manifest",
"path_event", "path_event",
@@ -162,12 +179,17 @@ __all__ = [
"RuntimeAdapterBundle", "RuntimeAdapterBundle",
"READINESS_REPORT_SCHEMA", "READINESS_REPORT_SCHEMA",
"SERVICE_BINDING_SCHEMA", "SERVICE_BINDING_SCHEMA",
"SERVICE_APP_SCHEMA",
"ServiceBinding", "ServiceBinding",
"ServiceAppConfig",
"ServiceResponse", "ServiceResponse",
"build_service_binding",
"create_wsgi_app",
"health_report", "health_report",
"resolve_runtime_adapters", "resolve_runtime_adapters",
"runtime_from_config", "runtime_from_config",
"service_binding_from_config", "service_binding_from_config",
"service_app_metadata",
"service_contracts", "service_contracts",
] ]

View File

@@ -16,6 +16,7 @@ LOCAL_STORE_MIGRATION_PLAN_SCHEMA = "phase_memory.local_store.migration_plan.v1"
LOCAL_STORE_MIGRATION_RESULT_SCHEMA = "phase_memory.local_store.migration_result.v1" LOCAL_STORE_MIGRATION_RESULT_SCHEMA = "phase_memory.local_store.migration_result.v1"
AUDIT_EXPORT_BATCH_SCHEMA = "phase_memory.audit.export_batch.v1" AUDIT_EXPORT_BATCH_SCHEMA = "phase_memory.audit.export_batch.v1"
AUDIT_RETENTION_PLAN_SCHEMA = "phase_memory.audit.retention_plan.v1" AUDIT_RETENTION_PLAN_SCHEMA = "phase_memory.audit.retention_plan.v1"
AUDIT_RETENTION_RESULT_SCHEMA = "phase_memory.audit.retention_result.v1"
class InMemoryMemoryGraphStore: class InMemoryMemoryGraphStore:
@@ -435,6 +436,11 @@ class RecordingAuditSink:
def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]: def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]:
return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata()) return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata())
def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]:
result, retained = audit_retention_apply(self.events, plan, retention=self.retention_metadata())
self.events = retained
return result
def export_batch(self, **filters: Any) -> dict[str, Any]: def export_batch(self, **filters: Any) -> dict[str, Any]:
events = self.query(**filters) events = self.query(**filters)
return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) return audit_export_batch(events, filters=filters, retention=self.retention_metadata())
@@ -471,6 +477,15 @@ class JsonlAuditSink:
def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]: def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]:
return audit_retention_plan(self.query(), retention_days=retention_days, now=now, retention=self.retention_metadata()) return audit_retention_plan(self.query(), retention_days=retention_days, now=now, retention=self.retention_metadata())
def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]:
result, retained = audit_retention_apply(self.query(), plan, retention=self.retention_metadata())
tmp_path = self.path.with_name(f".{self.path.name}.tmp")
with tmp_path.open("w", encoding="utf-8") as handle:
for event in retained:
handle.write(json.dumps(event, sort_keys=True, separators=(",", ":")) + "\n")
tmp_path.replace(self.path)
return result
def export_batch(self, **filters: Any) -> dict[str, Any]: def export_batch(self, **filters: Any) -> dict[str, Any]:
events = self.query(**filters) events = self.query(**filters)
return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) return audit_export_batch(events, filters=filters, retention=self.retention_metadata())
@@ -600,6 +615,34 @@ def audit_retention_plan(
} }
def audit_retention_apply(
events: list[dict[str, Any]],
plan: dict[str, Any],
*,
retention: dict[str, Any] | None = None,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
eligible = {str(item) for item in plan.get("eligible_operation_ids", ())}
retained_events: list[dict[str, Any]] = []
pruned: list[str] = []
for event in events:
event_id = str(event.get("operation_id") or event.get("id") or stable_digest(event))
if event_id in eligible:
pruned.append(event_id)
else:
retained_events.append(dict(event))
result = {
"schema_version": AUDIT_RETENTION_RESULT_SCHEMA,
"plan_id": str(plan.get("id") or ""),
"applied": True,
"changed": bool(pruned),
"pruned_count": len(pruned),
"retained_count": len(retained_events),
"pruned_operation_ids": sorted(pruned),
"retention": dict(retention or plan.get("retention") or {}),
}
return result, retained_events
def _audit_event_matches(event: dict[str, Any], filters: dict[str, Any]) -> bool: def _audit_event_matches(event: dict[str, Any], filters: dict[str, Any]) -> bool:
operation = filters.get("operation") operation = filters.get("operation")
if operation is not None and event.get("operation") != operation: if operation is not None and event.get("operation") != operation:

View File

@@ -0,0 +1,86 @@
"""Credential-gated adapter drill helpers.
The helpers in this module never store or print credentials. They validate that
an operator supplied the expected environment contract, then run the same local
manifest/conformance path used by live-shaped adapter fixtures.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Mapping
from .external_adapters import live_shaped_adapter_pack, validate_adapter_pack_manifest
from .service import default_conformance_adapters
from .utils import stable_digest
CREDENTIALED_DRILL_SCHEMA = "phase_memory.credentialed_adapter_drill.v1"
CREDENTIALED_ADAPTER_ENV_VARS = (
"PHASE_MEMORY_MARKITECT_URL",
"PHASE_MEMORY_MARKITECT_TOKEN",
"PHASE_MEMORY_KONTEXTUAL_URL",
"PHASE_MEMORY_KONTEXTUAL_TOKEN",
)
@dataclass(frozen=True)
class CredentialedDrillConfig:
markitect_url: str
kontextual_url: str
def to_dict(self) -> dict[str, str]:
return {
"markitect_url": self.markitect_url,
"kontextual_url": self.kontextual_url,
"credential_fingerprint": stable_digest([self.markitect_url, self.kontextual_url]),
}
def missing_credentialed_adapter_env(environ: Mapping[str, str] | None = None) -> tuple[str, ...]:
environ = environ or {}
return tuple(name for name in CREDENTIALED_ADAPTER_ENV_VARS if not environ.get(name))
def credentialed_drill_config_from_env(environ: Mapping[str, str] | None = None) -> CredentialedDrillConfig:
environ = environ or {}
missing = missing_credentialed_adapter_env(environ)
if missing:
raise ValueError(f"Missing credentialed adapter drill environment: {', '.join(missing)}")
return CredentialedDrillConfig(
markitect_url=str(environ["PHASE_MEMORY_MARKITECT_URL"]),
kontextual_url=str(environ["PHASE_MEMORY_KONTEXTUAL_URL"]),
)
def credentialed_adapter_smoke_report(environ: Mapping[str, str] | None = None) -> dict:
environ = environ or {}
missing = missing_credentialed_adapter_env(environ)
if missing:
return {
"schema_version": CREDENTIALED_DRILL_SCHEMA,
"valid": False,
"skipped": True,
"missing_env": list(missing),
"diagnostics": [
{
"severity": "warn",
"code": "credential_env_missing",
"message": "Credentialed adapter drill skipped because required environment variables are missing.",
"metadata": {"required_env": list(CREDENTIALED_ADAPTER_ENV_VARS), "missing_env": list(missing)},
}
],
}
config = credentialed_drill_config_from_env(environ)
pack = live_shaped_adapter_pack()
diagnostics = validate_adapter_pack_manifest(pack)
conformance_adapters = default_conformance_adapters()
return {
"schema_version": CREDENTIALED_DRILL_SCHEMA,
"valid": not any(diagnostic.severity == "error" for diagnostic in diagnostics),
"skipped": False,
"config": config.to_dict(),
"adapter_pack": pack.manifest(),
"conformance_helpers": sorted(conformance_adapters),
"diagnostics": [diagnostic.to_dict() for diagnostic in diagnostics],
}

View File

@@ -10,8 +10,10 @@ from .contracts import graph_from_markitect
from .models import Diagnostic, MemoryPath from .models import Diagnostic, MemoryPath
from .retrieval import activation_quality_report, select_event_path from .retrieval import activation_quality_report, select_event_path
from .runtime import PhaseMemoryRuntime from .runtime import PhaseMemoryRuntime
from .utils import stable_digest, utc_now_iso
EVALUATION_REPORT_SCHEMA = "phase_memory.evaluation.threshold_report.v1" EVALUATION_REPORT_SCHEMA = "phase_memory.evaluation.threshold_report.v1"
EVALUATION_TREND_SCHEMA = "phase_memory.evaluation.trend_artifact.v1"
DEFAULT_THRESHOLDS = { DEFAULT_THRESHOLDS = {
"policy_denial_count": 1, "policy_denial_count": 1,
@@ -65,6 +67,54 @@ def evaluation_threshold_report(data: dict[str, Any], *, thresholds: dict[str, f
} }
def evaluation_trend_artifact(
report: dict[str, Any],
*,
previous_report: dict[str, Any] | None = None,
run_metadata: dict[str, Any] | None = None,
) -> dict[str, Any]:
run_metadata = {
"created_at": utc_now_iso(),
**dict(run_metadata or {}),
}
metrics = dict(report.get("metrics") or {})
thresholds = dict(report.get("thresholds") or {})
previous_metrics = dict((previous_report or {}).get("metrics") or {})
threshold_deltas = {
key: round(float(metrics.get(key) or 0) - float(threshold), 4)
for key, threshold in sorted(thresholds.items())
}
metric_deltas = {
key: round(float(value or 0) - float(previous_metrics.get(key) or 0), 4)
for key, value in sorted(metrics.items())
if key in previous_metrics
}
diagnostics = [dict(item) for item in report.get("diagnostics", ())]
for key, delta in metric_deltas.items():
if delta < 0:
diagnostics.append(
Diagnostic(
"warn",
"evaluation_metric_regressed",
"Evaluation metric declined from the previous report.",
key,
{"delta": delta, "current": metrics.get(key), "previous": previous_metrics.get(key)},
).to_dict()
)
artifact_id = f"evaluation-trend:{stable_digest([run_metadata, metrics, thresholds, previous_metrics])}"
return {
"schema_version": EVALUATION_TREND_SCHEMA,
"id": artifact_id,
"valid": not any(item.get("severity") == "error" for item in diagnostics),
"run": run_metadata,
"metrics": metrics,
"thresholds": thresholds,
"threshold_deltas": threshold_deltas,
"metric_deltas": metric_deltas,
"report": report,
"previous_report_id": (previous_report or {}).get("id", ""),
"diagnostics": diagnostics,
}
def _policy_scenario(scenario: dict[str, Any]) -> dict[str, Any]: def _policy_scenario(scenario: dict[str, Any]) -> dict[str, Any]:
runtime = PhaseMemoryRuntime() runtime = PhaseMemoryRuntime()
response = runtime.plan_activation( response = runtime.plan_activation(

View File

@@ -142,6 +142,13 @@ class FakeTelemetryAuditSink:
def retention_plan(self, *, retention_days: int | None = None, now=None) -> dict[str, Any]: def retention_plan(self, *, retention_days: int | None = None, now=None) -> dict[str, Any]:
return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata()) return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata())
def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]:
from .adapters import audit_retention_apply
result, retained = audit_retention_apply(self.events, plan, retention=self.retention_metadata())
self.events = retained
return result
def export_batch(self, **filters: Any) -> dict[str, Any]: def export_batch(self, **filters: Any) -> dict[str, Any]:
events = self.query(**filters) events = self.query(**filters)
return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) return audit_export_batch(events, filters=filters, retention=self.retention_metadata())

View File

@@ -24,6 +24,7 @@ class MemoryOperation(str, Enum):
AUDIT_QUERY = "audit.query" AUDIT_QUERY = "audit.query"
AUDIT_EXPORT = "audit.export" AUDIT_EXPORT = "audit.export"
AUDIT_RETENTION_PLAN = "audit.retention.plan" AUDIT_RETENTION_PLAN = "audit.retention.plan"
AUDIT_RETENTION_APPLY = "audit.retention.apply"
LIFECYCLE_APPLY = "lifecycle.apply" LIFECYCLE_APPLY = "lifecycle.apply"
STABILIZATION = "memory.stabilize" STABILIZATION = "memory.stabilize"
COMPACTION = "memory.compact" COMPACTION = "memory.compact"

View File

@@ -40,6 +40,7 @@ RUNTIME_ENVELOPE_SCHEMA = "phase_memory.runtime.envelope.v1"
AUDIT_QUERY_SCHEMA = "phase_memory.audit.query.v1" AUDIT_QUERY_SCHEMA = "phase_memory.audit.query.v1"
AUDIT_EXPORT_SCHEMA = "phase_memory.audit.export.v1" AUDIT_EXPORT_SCHEMA = "phase_memory.audit.export.v1"
AUDIT_RETENTION_SCHEMA = "phase_memory.audit.retention.v1" AUDIT_RETENTION_SCHEMA = "phase_memory.audit.retention.v1"
AUDIT_RETENTION_APPLY_SCHEMA = "phase_memory.audit.retention.apply.v1"
PACKAGE_REQUEST_SCHEMA = MARKITECT_PACKAGE_REQUEST_SCHEMA PACKAGE_REQUEST_SCHEMA = MARKITECT_PACKAGE_REQUEST_SCHEMA
@@ -399,6 +400,62 @@ class PhaseMemoryRuntime:
"source": {"ref": source_ref}, "source": {"ref": source_ref},
} }
def apply_audit_retention(
self,
retention_plan: dict[str, Any] | None = None,
*,
retention_days: int | None = None,
now: datetime | None = None,
source_ref: str = "audit",
) -> dict[str, Any]:
policy = self.policy_gateway.authorize(
action="audit.retention.apply",
resource="audit:events",
context={"source_ref": source_ref, "dry_run": False, "retention_days": retention_days},
)
diagnostics: tuple[Diagnostic, ...] = ()
if policy.allowed and hasattr(self.audit_sink, "apply_retention_plan"):
plan = retention_plan or self.audit_sink.retention_plan(retention_days=retention_days, now=now)
result = self.audit_sink.apply_retention_plan(plan)
elif policy.allowed:
plan = retention_plan or {}
result = {}
diagnostics = (
Diagnostic(
"error",
"audit_retention_apply_unsupported",
"Audit sink does not expose retention apply.",
self.audit_sink.__class__.__name__,
),
)
else:
plan = retention_plan or {}
result = {}
operation_id = f"op:{stable_digest(['audit.retention.apply', source_ref, retention_days, plan])}"
audit = self.audit_sink.record(
audit_event(
operation_id=operation_id,
operation="audit.retention.apply",
subject={"kind": "audit_events", "id": str(plan.get("id") or "retention")},
policy_decision=policy,
dry_run=False,
source_ref=source_ref,
)
)
return {
"schema_version": AUDIT_RETENTION_APPLY_SCHEMA,
"operation_id": operation_id,
"operation": "audit.retention.apply",
"dry_run": False,
"valid": policy.allowed and not any(diagnostic.severity == "error" for diagnostic in diagnostics),
"plan": plan,
"result": result,
"policy_decision": _policy_to_dict(policy),
"audit_receipt": audit,
"diagnostics": [diagnostic.to_dict() for diagnostic in diagnostics],
"source": {"ref": source_ref},
}
def export_graph(self, *, graph_id: str = "local", source_ref: str = "local-store") -> dict[str, Any]: def export_graph(self, *, graph_id: str = "local", source_ref: str = "local-store") -> dict[str, Any]:
events = self.event_log.list_events() events = self.event_log.list_events()
if hasattr(self.graph_store, "export_graph"): if hasattr(self.graph_store, "export_graph"):

View File

@@ -0,0 +1,77 @@
"""Deployable stdlib service entrypoint for phase-memory."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
from typing import Sequence
from wsgiref.simple_server import make_server
from .service import RuntimeConfig
from .service_binding import ServiceBinding, service_binding_from_config
SERVICE_APP_SCHEMA = "phase_memory.service.app.v1"
@dataclass(frozen=True)
class ServiceAppConfig:
host: str = "127.0.0.1"
port: int = 8080
local_store_path: str = ".phase-memory-local"
def runtime_config(self) -> RuntimeConfig:
return RuntimeConfig(local_store_path=self.local_store_path)
def to_dict(self) -> dict:
return {
"host": self.host,
"port": self.port,
"local_store_path": self.local_store_path,
}
def build_service_binding(config: ServiceAppConfig | None = None) -> ServiceBinding:
config = config or ServiceAppConfig()
return service_binding_from_config(config.runtime_config())
def service_app_metadata(config: ServiceAppConfig | None = None) -> dict:
config = config or ServiceAppConfig()
binding = build_service_binding(config)
readiness = binding.readiness()
return {
"schema_version": SERVICE_APP_SCHEMA,
"config": config.to_dict(),
"readiness": readiness,
"routes": {
"health": "/health",
"readiness": "/ready",
"contracts": "/contracts",
"operations": "/operations/{operation}",
},
}
def create_wsgi_app(config: ServiceAppConfig | None = None):
return build_service_binding(config).as_wsgi_app()
def serve(config: ServiceAppConfig | None = None) -> None:
config = config or ServiceAppConfig()
app = create_wsgi_app(config)
with make_server(config.host, config.port, app) as server:
server.serve_forever()
def main(argv: Sequence[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Run the phase-memory service binding.")
parser.add_argument("--host", default="127.0.0.1")
parser.add_argument("--port", type=int, default=8080)
parser.add_argument("--store", default=".phase-memory-local")
parser.add_argument("--check", action="store_true", help="Build the app and return readiness status without listening.")
args = parser.parse_args(list(argv) if argv is not None else None)
config = ServiceAppConfig(host=args.host, port=args.port, local_store_path=args.store)
if args.check:
return 0 if service_app_metadata(config)["readiness"]["ok"] else 1
serve(config)
return 0

View File

@@ -1,9 +1,16 @@
{ {
"compatibility": {
"release_note_template": "docs/release-note-template.md"
},
"exports": [ "exports": [
"ADAPTER_PACK_MANIFEST_SCHEMA", "ADAPTER_PACK_MANIFEST_SCHEMA",
"ActivationPlan", "ActivationPlan",
"CREDENTIALED_ADAPTER_ENV_VARS",
"CREDENTIALED_DRILL_SCHEMA",
"CredentialedDrillConfig",
"Diagnostic", "Diagnostic",
"EVALUATION_REPORT_SCHEMA", "EVALUATION_REPORT_SCHEMA",
"EVALUATION_TREND_SCHEMA",
"ExternalAdapterPack", "ExternalAdapterPack",
"FakeExternalEventLog", "FakeExternalEventLog",
"FakeExternalGraphStore", "FakeExternalGraphStore",
@@ -49,7 +56,9 @@
"ReviewRecord", "ReviewRecord",
"RuntimeAdapterBundle", "RuntimeAdapterBundle",
"RuntimeConfig", "RuntimeConfig",
"SERVICE_APP_SCHEMA",
"SERVICE_BINDING_SCHEMA", "SERVICE_BINDING_SCHEMA",
"ServiceAppConfig",
"ServiceBinding", "ServiceBinding",
"ServiceResponse", "ServiceResponse",
"WordCountTokenEstimator", "WordCountTokenEstimator",
@@ -57,9 +66,14 @@
"activation_quality_report", "activation_quality_report",
"adapter_pack_manifest", "adapter_pack_manifest",
"branch_path", "branch_path",
"build_service_binding",
"compact_path", "compact_path",
"create_path", "create_path",
"create_wsgi_app",
"credentialed_adapter_smoke_report",
"credentialed_drill_config_from_env",
"evaluation_threshold_report", "evaluation_threshold_report",
"evaluation_trend_artifact",
"fake_external_adapter_pack", "fake_external_adapter_pack",
"fake_external_runtime_config", "fake_external_runtime_config",
"graph_from_markitect", "graph_from_markitect",
@@ -67,6 +81,7 @@
"live_shaped_adapter_pack", "live_shaped_adapter_pack",
"make_review_record", "make_review_record",
"merge_path", "merge_path",
"missing_credentialed_adapter_env",
"package_request_from_selection", "package_request_from_selection",
"package_response_envelope", "package_response_envelope",
"path_event", "path_event",
@@ -85,6 +100,7 @@
"retrieve_graph_neighborhood", "retrieve_graph_neighborhood",
"runtime_from_config", "runtime_from_config",
"select_event_path", "select_event_path",
"service_app_metadata",
"service_binding_from_config", "service_binding_from_config",
"service_contracts", "service_contracts",
"validate_adapter_pack_manifest" "validate_adapter_pack_manifest"

View File

@@ -70,3 +70,58 @@ def test_audit_retention_plan_identifies_eligible_records() -> None:
assert plan["valid"] is True assert plan["valid"] is True
assert plan["plan"]["eligible_operation_ids"] == ["op:old"] assert plan["plan"]["eligible_operation_ids"] == ["op:old"]
assert plan["plan"]["eligible_count"] == 1 assert plan["plan"]["eligible_count"] == 1
def test_audit_retention_apply_prunes_eligible_records_and_records_apply() -> None:
runtime = PhaseMemoryRuntime()
runtime.audit_sink.record(
{
"schema_version": "phase_memory.audit.event.v1",
"operation_id": "op:old",
"operation": "manual",
"timestamp": "2026-01-01T00:00:00+00:00",
"subject": {"kind": "audit_events", "id": "old"},
"source": {"ref": "test"},
"dry_run": True,
"allowed": True,
}
)
runtime.audit_sink.record(
{
"schema_version": "phase_memory.audit.event.v1",
"operation_id": "op:new",
"operation": "manual",
"timestamp": "2026-05-18T00:00:00+00:00",
"subject": {"kind": "audit_events", "id": "new"},
"source": {"ref": "test"},
"dry_run": True,
"allowed": True,
}
)
plan = runtime.audit_retention_plan(retention_days=30, now=datetime(2026, 5, 19, tzinfo=timezone.utc))
applied = runtime.apply_audit_retention(plan["plan"])
remaining_ids = [event["operation_id"] for event in runtime.audit_sink.query()]
assert applied["valid"] is True
assert applied["result"]["pruned_operation_ids"] == ["op:old"]
assert "op:old" not in remaining_ids
assert "op:new" in remaining_ids
assert any(event["operation"] == "audit.retention.apply" for event in runtime.audit_sink.query())
def test_audit_retention_apply_noop_and_unsupported_paths() -> None:
runtime = PhaseMemoryRuntime()
noop = runtime.apply_audit_retention(retention_days=30, now=datetime(2026, 5, 19, tzinfo=timezone.utc))
assert noop["valid"] is True
assert noop["result"]["changed"] is False
class UnsupportedAuditSink:
def record(self, event):
return {"recorded": True, "event": event}
unsupported = PhaseMemoryRuntime(audit_sink=UnsupportedAuditSink())
result = unsupported.apply_audit_retention(retention_days=30)
assert result["valid"] is False
assert result["diagnostics"][0]["code"] == "audit_retention_apply_unsupported"

View File

@@ -0,0 +1,32 @@
import os
import pytest
from phase_memory.credentialed_drills import (
CREDENTIALED_ADAPTER_ENV_VARS,
credentialed_adapter_smoke_report,
missing_credentialed_adapter_env,
)
def test_credentialed_adapter_drill_reports_missing_env_without_secrets() -> None:
report = credentialed_adapter_smoke_report({})
assert report["valid"] is False
assert report["skipped"] is True
assert tuple(report["missing_env"]) == CREDENTIALED_ADAPTER_ENV_VARS
assert report["diagnostics"][0]["code"] == "credential_env_missing"
@pytest.mark.skipif(
missing_credentialed_adapter_env(os.environ),
reason="requires env vars: " + ", ".join(CREDENTIALED_ADAPTER_ENV_VARS),
)
def test_credentialed_adapter_drill_reuses_manifest_contract_when_env_is_present() -> None:
report = credentialed_adapter_smoke_report(os.environ)
assert report["valid"] is True
assert report["skipped"] is False
assert report["adapter_pack"]["name"] == "live-shaped"
assert report["config"]["credential_fingerprint"]
assert "PHASE_MEMORY_MARKITECT_TOKEN" not in str(report)

View File

@@ -4,7 +4,7 @@ from pathlib import Path
from phase_memory.adapters import InMemorySemanticIndex from phase_memory.adapters import InMemorySemanticIndex
from phase_memory.contracts import graph_from_markitect from phase_memory.contracts import graph_from_markitect
from phase_memory.evaluation import EVALUATION_REPORT_SCHEMA, evaluation_threshold_report from phase_memory.evaluation import EVALUATION_REPORT_SCHEMA, EVALUATION_TREND_SCHEMA, evaluation_threshold_report, evaluation_trend_artifact
from phase_memory.models import ActivationPlan, MemoryPath from phase_memory.models import ActivationPlan, MemoryPath
from phase_memory.retrieval import activation_quality_report, select_event_path from phase_memory.retrieval import activation_quality_report, select_event_path
from phase_memory.runtime import PhaseMemoryRuntime from phase_memory.runtime import PhaseMemoryRuntime
@@ -102,6 +102,30 @@ def test_evaluation_threshold_report_summarizes_all_scenarios() -> None:
assert report["diagnostics"] == [] assert report["diagnostics"] == []
def test_evaluation_trend_artifact_tracks_threshold_and_metric_deltas() -> None:
data = json.loads((FIXTURES / "evaluation-scenarios.json").read_text(encoding="utf-8"))
report = evaluation_threshold_report(data)
previous = {
"id": "previous",
"metrics": {
**report["metrics"],
"policy_denial_count": report["metrics"]["policy_denial_count"] + 1,
},
}
trend = evaluation_trend_artifact(
report,
previous_report=previous,
run_metadata={"run_id": "pytest", "created_at": "2026-05-19T00:00:00+00:00"},
)
assert trend["schema_version"] == EVALUATION_TREND_SCHEMA
assert trend["run"]["run_id"] == "pytest"
assert trend["threshold_deltas"]["policy_denial_count"] == 0.0
assert trend["metric_deltas"]["policy_denial_count"] == -1.0
assert trend["diagnostics"][0]["code"] == "evaluation_metric_regressed"
def _activation_plan(response): def _activation_plan(response):
data = response["data"]["activation_plan"] data = response["data"]["activation_plan"]
return ActivationPlan( return ActivationPlan(

View File

@@ -13,6 +13,10 @@ def test_public_api_snapshot_is_explicit() -> None:
assert sorted(phase_memory.__all__) == snapshot["exports"] assert sorted(phase_memory.__all__) == snapshot["exports"]
assert sorted(SERVICE_OPERATIONS) == snapshot["service_operations"] assert sorted(SERVICE_OPERATIONS) == snapshot["service_operations"]
release_note_template = Path(snapshot["compatibility"]["release_note_template"])
template_text = release_note_template.read_text(encoding="utf-8")
for heading in ("Changed Exports", "Changed Service Operations", "Migration Needs", "Operator Action"):
assert heading in template_text
def test_service_contract_catalog_matches_local_runner_supported_operations() -> None: def test_service_contract_catalog_matches_local_runner_supported_operations() -> None:

41
tests/test_service_app.py Normal file
View File

@@ -0,0 +1,41 @@
import json
from io import BytesIO
from phase_memory.service_app import ServiceAppConfig, create_wsgi_app, main, service_app_metadata
def test_service_app_metadata_exposes_deployable_routes_without_listener(tmp_path) -> None:
config = ServiceAppConfig(host="127.0.0.1", port=8123, local_store_path=str(tmp_path))
metadata = service_app_metadata(config)
assert metadata["schema_version"] == "phase_memory.service.app.v1"
assert metadata["config"]["port"] == 8123
assert metadata["readiness"]["ok"] is True
assert metadata["routes"]["operations"] == "/operations/{operation}"
def test_service_wsgi_app_can_dispatch_without_opening_listener(tmp_path) -> None:
app = create_wsgi_app(ServiceAppConfig(local_store_path=str(tmp_path)))
statuses: list[str] = []
payload = json.dumps({"selection": {"schema_version": "markitect.memory.selection.v1", "id": "svc", "nodes": [], "events": []}}).encode("utf-8")
body = b"".join(
app(
{
"REQUEST_METHOD": "POST",
"PATH_INFO": "/operations/package.compile",
"CONTENT_LENGTH": str(len(payload)),
"wsgi.input": BytesIO(payload),
},
lambda status, _headers: statuses.append(status),
)
)
response = json.loads(body.decode("utf-8"))
assert statuses == ["200 OK"]
assert response["operation"] == "package.compile"
def test_service_main_check_builds_app_without_serving(tmp_path) -> None:
assert main(["--check", "--store", str(tmp_path), "--port", "8124"]) == 0

View File

@@ -4,7 +4,7 @@ type: workplan
title: "Credentialed Adapter Drills And Deployment Packaging" title: "Credentialed Adapter Drills And Deployment Packaging"
domain: markitect domain: markitect
repo: phase-memory repo: phase-memory
status: ready status: finished
owner: codex owner: codex
topic_slug: phase-memory topic_slug: phase-memory
created: "2026-05-19" created: "2026-05-19"
@@ -37,7 +37,7 @@ reports, and public API snapshots. The scorecard now rates the repo at
```task ```task
id: PMEM-WP-0013-T01 id: PMEM-WP-0013-T01
status: todo status: done
priority: high priority: high
state_hub_task_id: "e4940a9d-130e-47ea-ba16-7b090841855c" state_hub_task_id: "e4940a9d-130e-47ea-ba16-7b090841855c"
``` ```
@@ -55,7 +55,7 @@ Acceptance:
```task ```task
id: PMEM-WP-0013-T02 id: PMEM-WP-0013-T02
status: todo status: done
priority: high priority: high
state_hub_task_id: "bf8d2159-761a-47f5-b7be-41ad52460b64" state_hub_task_id: "bf8d2159-761a-47f5-b7be-41ad52460b64"
``` ```
@@ -73,7 +73,7 @@ Acceptance:
```task ```task
id: PMEM-WP-0013-T03 id: PMEM-WP-0013-T03
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "7e39e894-8754-4977-abdd-00f3bf1a73d1" state_hub_task_id: "7e39e894-8754-4977-abdd-00f3bf1a73d1"
``` ```
@@ -92,7 +92,7 @@ Acceptance:
```task ```task
id: PMEM-WP-0013-T04 id: PMEM-WP-0013-T04
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "b23e3126-bbfa-44b1-b2a1-22cda968f5d8" state_hub_task_id: "b23e3126-bbfa-44b1-b2a1-22cda968f5d8"
``` ```
@@ -109,7 +109,7 @@ Acceptance:
```task ```task
id: PMEM-WP-0013-T05 id: PMEM-WP-0013-T05
status: todo status: done
priority: medium priority: medium
state_hub_task_id: "2e71fedd-aac6-42c9-822c-6305412ea064" state_hub_task_id: "2e71fedd-aac6-42c9-822c-6305412ea064"
``` ```
@@ -126,7 +126,7 @@ Acceptance:
```task ```task
id: PMEM-WP-0013-T06 id: PMEM-WP-0013-T06
status: todo status: done
priority: low priority: low
state_hub_task_id: "c1a8f699-9a0b-4983-8d35-e59cd124dd58" state_hub_task_id: "c1a8f699-9a0b-4983-8d35-e59cd124dd58"
``` ```
@@ -147,4 +147,26 @@ Acceptance:
## Closure Review ## Closure Review
Pending implementation. Completed on 2026-05-19.
Implemented:
- Credential-gated adapter drill helpers and a skipped smoke test that lists
required environment variables when credentials are absent.
- `phase-memory-service` stdlib service entrypoint with check mode, WSGI app
creation, and no-listener tests.
- Operator readiness runbook covering startup, readiness, migrations, audit
export/retention, credentialed drills, rollback, and compatibility release
discipline.
- Audit retention apply behavior for recording, JSONL, and telemetry sinks,
with runtime audit traces and unsupported-sink coverage.
- Evaluation trend artifacts with run metadata, threshold deltas, metric deltas,
and regression diagnostics.
- Release-note template and public API snapshot gate requiring compatibility
release notes for changed exports or service operations.
- Scorecard update from 4.2 to 4.3 and PMEM-WP-0014 as the next ready
refinement workplan.
Verification:
- Focused PMEM-WP-0013 tests passed: 18 passed, 1 skipped.

View File

@@ -0,0 +1,130 @@
---
id: PMEM-WP-0014
type: workplan
title: "Live Credential Execution And Managed Deployment Hardening"
domain: markitect
repo: phase-memory
status: ready
owner: codex
topic_slug: phase-memory
created: "2026-05-19"
updated: "2026-05-19"
state_hub_workstream_id: "312a04cb-124d-41b3-9fc0-292281f420ab"
---
# PMEM-WP-0014: Live Credential Execution And Managed Deployment Hardening
## Goal
Use the credential-gated drill and service packaging created in PMEM-WP-0013 to
exercise real operator environments, harden deployment packaging, and preserve
evaluation trend history.
## Current Evidence
`PMEM-WP-0013` added credential-gated drill helpers, stdlib service packaging,
operator readiness docs, audit retention apply, evaluation trend artifacts, and
release-note discipline. The scorecard now rates the repo at **4.3 / 5**.
## Non-Goals
- Commit credentials, tokens, or live endpoints.
- Make credentialed tests mandatory in default CI.
- Take ownership of Markitect or Kontextual service internals.
## T01 - Run credentialed adapter drills in operator mode
```task
id: PMEM-WP-0014-T01
status: todo
priority: high
state_hub_task_id: "1d0eb51c-60ce-47ad-bd91-6ce1ee91f0f8"
```
Exercise the credential-gated smoke drill against real operator-provided
Markitect/Kontextual endpoints.
Acceptance:
- Default suite still skips without credentials.
- Operator run records a redacted report with no tokens.
- Any live incompatibility is captured as explicit diagnostics.
## T02 - Add managed deployment packaging
```task
id: PMEM-WP-0014-T02
status: todo
priority: high
state_hub_task_id: "37b03680-fcc4-46c2-9ce2-f6bf1f2ef35b"
```
Add deployment packaging around the stdlib service entrypoint.
Acceptance:
- Health and readiness probes are documented.
- Packaging can be validated without live credentials.
- Rollback and local-store mount expectations are explicit.
## T03 - Persist evaluation trend history
```task
id: PMEM-WP-0014-T03
status: todo
priority: medium
state_hub_task_id: "a3260267-bc8f-4f17-abdd-2296ad2c6ed5"
```
Persist evaluation trend artifacts across runs for regression review.
Acceptance:
- Trend history format is deterministic.
- Deltas can be compared across commits or run ids.
- Regression diagnostics remain actionable.
## T04 - Add credentialed telemetry retention drill
```task
id: PMEM-WP-0014-T04
status: todo
priority: medium
state_hub_task_id: "b68478ce-90c2-4e21-b621-569cb6925f74"
```
Exercise audit export and retention apply against a credentialed telemetry
adapter or operator-approved fixture.
Acceptance:
- Tokens are never written to artifacts.
- Retention apply records an audit event.
- Pruned and retained operation ids are reviewable.
## T05 - Expand operator troubleshooting matrix
```task
id: PMEM-WP-0014-T05
status: todo
priority: medium
state_hub_task_id: "b0974113-debd-4823-929a-761510132c09"
```
Collect expected operator failures and remediations.
Acceptance:
- Matrix covers credentials, readiness, migrations, audit retention, and
adapter manifest failures.
- Each row includes diagnostic code, likely cause, and operator action.
## Acceptance Criteria
- Evidence moves the project toward the 4.7+ scorecard gate.
- Credentialed runs are reproducible but optional.
- Managed deployment packaging is ready for operator review.
## Closure Review
Pending implementation.