From 6e0372d21abf0152f591dc4a3771fd961b2b5f19 Mon Sep 17 00:00:00 2001 From: tegwick Date: Tue, 19 May 2026 01:27:59 +0200 Subject: [PATCH] Implement credentialed drill packaging workplan --- README.md | 3 +- docs/api-compatibility.md | 5 + docs/maturity-scorecard.md | 71 +++++---- docs/operational-readiness.md | 18 ++- docs/operator-readiness-runbook.md | 141 ++++++++++++++++++ docs/release-note-template.md | 41 +++++ pyproject.toml | 1 + src/phase_memory/__init__.py | 24 ++- src/phase_memory/adapters.py | 43 ++++++ src/phase_memory/credentialed_drills.py | 86 +++++++++++ src/phase_memory/evaluation.py | 50 +++++++ src/phase_memory/external_adapters.py | 7 + src/phase_memory/policy.py | 1 + src/phase_memory/runtime.py | 57 +++++++ src/phase_memory/service_app.py | 77 ++++++++++ tests/fixtures/public-api-snapshot.json | 16 ++ tests/test_audit_operations.py | 55 +++++++ tests/test_credentialed_drills.py | 32 ++++ tests/test_evaluation_scenarios.py | 26 +++- tests/test_public_api.py | 4 + tests/test_service_app.py | 41 +++++ ...adapter-drills-and-deployment-packaging.md | 38 ++++- ...cution-and-managed-deployment-hardening.md | 130 ++++++++++++++++ 23 files changed, 924 insertions(+), 43 deletions(-) create mode 100644 docs/operator-readiness-runbook.md create mode 100644 docs/release-note-template.md create mode 100644 src/phase_memory/credentialed_drills.py create mode 100644 src/phase_memory/service_app.py create mode 100644 tests/test_credentialed_drills.py create mode 100644 tests/test_service_app.py create mode 100644 workplans/PMEM-WP-0014-live-credential-execution-and-managed-deployment-hardening.md diff --git a/README.md b/README.md index 563484f..7322f9d 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,7 @@ for retrieval and evaluation behavior, [docs/service-readiness.md](docs/service- for service and adapter contracts, [docs/lifecycle-rules.md](docs/lifecycle-rules.md) for profile-driven lifecycle rules, [docs/external-adapter-packs.md](docs/external-adapter-packs.md) for fake external integration packs, [docs/operational-readiness.md](docs/operational-readiness.md) -for the local end-to-end operational recipe, [docs/api-compatibility.md](docs/api-compatibility.md) +for the local end-to-end operational recipe, [docs/operator-readiness-runbook.md](docs/operator-readiness-runbook.md) +for operator readiness, [docs/api-compatibility.md](docs/api-compatibility.md) for public API compatibility expectations, [docs/maturity-scorecard.md](docs/maturity-scorecard.md) for the current maturity assessment, and [SCOPE.md](SCOPE.md) for repository boundaries. diff --git a/docs/api-compatibility.md b/docs/api-compatibility.md index 78827e8..0db8562 100644 --- a/docs/api-compatibility.md +++ b/docs/api-compatibility.md @@ -23,6 +23,11 @@ The snapshot in `tests/fixtures/public-api-snapshot.json` intentionally fails when exports or service operations change. Update the snapshot only when the change is deliberate and documented. +The snapshot also points to `docs/release-note-template.md`. When a public API +snapshot changes, fill in that template or a release note derived from it so +operators can see changed exports, changed service operations, migration needs, +and required action. + ## Adding Operations When adding a service operation: diff --git a/docs/maturity-scorecard.md b/docs/maturity-scorecard.md index a96c73a..67a68c5 100644 --- a/docs/maturity-scorecard.md +++ b/docs/maturity-scorecard.md @@ -26,24 +26,24 @@ to 5. ## Current Score -Overall maturity: **4.2 / 5** +Overall maturity: **4.3 / 5** Two sub-scores make the result easier to reason about: -- Local integration maturity: **4.5 / 5** -- Operational maturity: **3.8 / 5** +- Local integration maturity: **4.6 / 5** +- Operational maturity: **4.0 / 5** The repo is strong as a deterministic local library and service-boundary core. It is not yet production-operational because adapter coverage is still -live-shaped rather than credentialed live integration, and service bindings are -framework-neutral embedding surfaces rather than a deployed service. +credential-gated rather than continuously exercised against live services, and +service packaging is stdlib/local rather than deployed to a managed environment. ## Dimension Scorecard | Dimension | Score | Target | Evidence | Needed Next | | --- | ---: | ---: | --- | --- | | Intent and boundaries | 4.4 | 5.0 | `INTENT.md`, `SCOPE.md`, `README.md`, architecture docs, adjacent-repo boundary docs | Keep docs current as live adapters and service bindings clarify real ownership. | -| Package and API foundation | 4.5 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot | Add release notes discipline and compatibility migration examples. | +| Package and API foundation | 4.6 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot, release-note template | Add compatibility migration examples from a real release. | | Markitect profile contract ingress | 3.7 | 4.5 | Profile loading, diagnostics, runtime envelopes, profile-derived config, local alias normalization | Add richer compatibility fixtures and schema drift diagnostics. | | Graph and event ingress | 4.0 | 4.5 | Graph loading, endpoint diagnostics, event model, JSONL log, export, repair checks, corrupt-record diagnostics, fake and live-shaped graph/event adapters | Add broader malformed/large graph fixtures and operator repair utilities. | | Phase domain model | 3.5 | 4.5 | Phases, lifecycle states, actions, paths, retention rules, profile-derived transition rules | Add migration semantics for profile/rule changes over durable stores. | @@ -51,26 +51,26 @@ framework-neutral embedding surfaces rather than a deployed service. | Lifecycle planning and apply | 4.1 | 4.5 | Dry-run lifecycle plans, profile rules, review-gated local apply, service `lifecycle.apply`, apply audit/export queries | Add richer apply rollback and repair drills. | | Activation planning | 4.0 | 4.8 | Budgeted activation, selections, package request, graph neighborhoods, paths, ranking, metrics, multi-scenario evaluation fixtures | Wire semantic-index-assisted retrieval into runtime planning. | | Local persistence | 4.0 | 4.5 | File-backed graph store, JSONL event log, audit sink, atomic JSON writes, executable metadata migrations, migration audit, export, repair diagnostics | Add compaction/retention utilities and stronger corruption recovery. | -| Policy, review, and audit | 4.2 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and enforceable audit retention pruning. | -| Observability and operations | 4.0 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, fake/live-shaped telemetry audit sinks, operational recipe | Add metrics/event export to external telemetry and deployable service packaging. | -| Markitect interop | 4.0 | 4.5 | Local validation, package request/response envelopes, fake and live-shaped compiler fixtures | Add optional credentialed Markitect compiler adapter and schema drift suite. | -| Kontextual/Infospace interop | 3.7 | 4.5 | Delegation envelope, fake and live-shaped runtime registry, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual adapter drill and broader Infospace restart reports. | -| Testing and evaluation | 4.3 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, API snapshots, and evaluation threshold reports | Add larger regression corpus and threshold trend reports. | -| Service readiness | 4.5 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, health/readiness, config, adapter conformance | Add deployable packaging and operator readiness runbooks. | -| Developer experience | 4.3 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, API compatibility docs | Add troubleshooting matrix and release note templates. | +| Policy, review, and audit | 4.4 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans and apply, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and credentialed telemetry pruning drill. | +| Observability and operations | 4.3 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, stdlib service entrypoint, operator runbook, fake/live-shaped telemetry audit sinks | Add metrics/event export to external telemetry and managed deployment packaging. | +| Markitect interop | 4.1 | 4.5 | Local validation, package request/response envelopes, fake/live-shaped compiler fixtures, credential-gated drill contract | Add credentialed Markitect compiler execution and schema drift suite. | +| Kontextual/Infospace interop | 3.9 | 4.5 | Delegation envelope, fake/live-shaped runtime registry, credential-gated drill contract, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual execution and broader Infospace restart reports. | +| Testing and evaluation | 4.5 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, credential skip gates, API snapshots, evaluation threshold and trend reports | Add larger regression corpus and persisted trend history. | +| Service readiness | 4.6 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, stdlib service entrypoint, health/readiness, config, adapter conformance | Add managed deployment packaging. | +| Developer experience | 4.5 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, operator runbook, API compatibility docs, release-note template | Add troubleshooting matrix from real operator feedback. | ## Assessment The project has crossed the local integration-readiness threshold. The runtime envelopes, policy/review model, profile-derived configuration, lifecycle rules, -local persistence migrations, queryable/exportable audit path, fake and -live-shaped external pack manifests, service binding, API snapshots, and +local persistence migrations, queryable/exportable/prunable audit path, fake +and live-shaped external pack manifests, credential-gated drills, service +binding and stdlib entrypoint, API snapshots, release discipline, and conformance helpers form a solid integration boundary. The biggest optimization opportunity is now the next operational layer: -moving from live-shaped local fixtures to credentialed live adapter drills, -packaging the service binding for deployment, and growing evaluation thresholds -into trend reports. +running the credential-gated drills against real services, adding managed +deployment packaging, and growing evaluation trends into a historical corpus. ## Completed Refinement Workplan @@ -97,20 +97,30 @@ into trend reports. - evaluation threshold reports over the scenario corpus; - public API and service operation compatibility snapshots. +`PMEM-WP-0013` moved the score from 4.2 to 4.3 by adding: + +- credential-gated adapter drill helpers and skipped smoke tests that list + required environment variables; +- stdlib `phase-memory-service` packaging with check mode and WSGI dispatch; +- operator readiness runbook for service startup, migrations, audit retention, + credentialed drills, and rollback; +- audit retention apply behavior with audit trace coverage; +- evaluation trend artifacts with threshold and regression deltas; +- release-note template gating for public API snapshot changes. + ## Recommended Next Refinement -Create and execute `PMEM-WP-0013`: credentialed adapter drills and deployment -packaging. +Create and execute `PMEM-WP-0014`: live credential execution and managed +deployment hardening. Highest-value tasks: -- Add optional credentialed Markitect/Kontextual adapter smoke drills that are - skipped unless credentials are present. -- Package the service binding as a deployable local service with operator - readiness checks. -- Add audit retention pruning and telemetry export enforcement. -- Grow evaluation reporting into historical threshold trends. -- Add release note and migration-note templates for compatibility changes. +- Run the credential-gated drills against real Markitect/Kontextual endpoints + in an operator environment. +- Add managed deployment packaging and readiness probes. +- Persist evaluation trend reports across runs. +- Add credentialed telemetry export and retention pruning drills. +- Expand troubleshooting from actual operator feedback. ## Score Movement Gates @@ -121,10 +131,11 @@ Achieved overall score **4.0** when: - Local persistence has migration diagnostics. - Evaluation fixtures cover at least three profile/graph families. -Move overall score to **4.3+** when: +Achieved overall score **4.3+** when: -- Credentialed optional Markitect or Kontextual adapter smoke drills run behind - the same conformance suite as the fake/live-shaped packs. +- Credentialed optional Markitect or Kontextual adapter smoke drills are + available behind the same conformance suite as the fake/live-shaped packs and + skip cleanly without credentials. - Operational docs include deployable service packaging and an operator readiness runbook. diff --git a/docs/operational-readiness.md b/docs/operational-readiness.md index e864fd4..889c90d 100644 --- a/docs/operational-readiness.md +++ b/docs/operational-readiness.md @@ -84,6 +84,16 @@ The WSGI adapter returned by `binding.as_wsgi_app()` is also callable in tests without opening a socket. Use this for deployment wrappers so the core service operation contract stays framework-neutral. +For the stdlib deployable entrypoint, use: + +```bash +phase-memory-service --check --store .phase-memory-local +phase-memory-service --host 127.0.0.1 --port 8080 --store .phase-memory-local +``` + +See `docs/operator-readiness-runbook.md` for operator checks and rollback +guidance. + ## Review-Gated Apply Lifecycle actions that require review are denied until an approval marker or @@ -143,8 +153,12 @@ retention = runner.runtime.audit_retention_plan(retention_days=30) ``` The export batch includes matching audit events and sink retention metadata. -The retention plan identifies eligible operation ids but does not prune records; -retention apply is a follow-on operational task. +The retention plan identifies eligible operation ids. Retention apply prunes +eligible records and records `audit.retention.apply` after pruning: + +```python +retention_apply = runner.runtime.apply_audit_retention(retention["plan"]) +``` ## Adapter Pack Compatibility diff --git a/docs/operator-readiness-runbook.md b/docs/operator-readiness-runbook.md new file mode 100644 index 0000000..d4e6ff2 --- /dev/null +++ b/docs/operator-readiness-runbook.md @@ -0,0 +1,141 @@ +# Operator Readiness Runbook + +Updated: 2026-05-19 + +This runbook covers the operational path for `phase-memory` without requiring +credentials in the default test suite. + +## Modes + +| Mode | Purpose | Credentials | Network | +| --- | --- | --- | --- | +| Local fixture | Default deterministic runtime and tests. | No | No | +| Live-shaped | Adapter manifests and behavior that model live services locally. | No | No | +| Credentialed live drill | Operator-provided smoke drill for real endpoints. | Yes, via env only | Optional | + +Credentialed drills require: + +- `PHASE_MEMORY_MARKITECT_URL` +- `PHASE_MEMORY_MARKITECT_TOKEN` +- `PHASE_MEMORY_KONTEXTUAL_URL` +- `PHASE_MEMORY_KONTEXTUAL_TOKEN` + +Do not store those values in Git, workplans, progress logs, or release notes. + +## Service Startup + +The deployable stdlib entrypoint is `phase-memory-service`. + +Readiness check without listening: + +```bash +phase-memory-service --check --store .phase-memory-local +``` + +Start the stdlib WSGI service: + +```bash +phase-memory-service --host 127.0.0.1 --port 8080 --store .phase-memory-local +``` + +Routes: + +- `GET /health` +- `GET /ready` +- `GET /contracts` +- `POST /operations/{operation}` +- `POST /operations` with `{"operation": "...", "payload": {...}}` + +## Readiness Checks + +Before accepting traffic: + +1. Run `phase-memory-service --check`. +2. Verify `/ready` reports `ok: true`. +3. Verify `unsupported_operations` is empty. +4. Verify adapter diagnostics have no `error` severity. +5. Verify the public API snapshot test passes after any operation/export change. + +## Migration Apply + +Plan and apply local-store metadata migrations through the runtime: + +```python +from phase_memory import RuntimeConfig, runtime_from_config + +config = RuntimeConfig(local_store_path=".phase-memory-local") +runtime = runtime_from_config(config) +plan = runtime.plan_store_migration(source_ref=config.local_store_path) +result = runtime.apply_store_migration( + plan["data"]["migration_plan"], + actor="operator", + source_ref=config.local_store_path, +) +``` + +Expected: + +- no `error` diagnostics in the plan; +- `result["valid"] is True`; +- metadata is updated atomically; +- `audit.query` can find the `store.migration.apply` event. + +Rollback: + +- stop the service; +- restore the previous local store directory from backup; +- rerun `phase-memory-service --check`; +- rerun `runtime.repair_diagnostics()`. + +## Audit Export And Retention + +Plan retention: + +```python +plan = runtime.audit_retention_plan(retention_days=30) +``` + +Apply retention: + +```python +result = runtime.apply_audit_retention(plan["plan"]) +``` + +Expected: + +- eligible operation ids are pruned; +- `audit.retention.apply` is recorded after pruning; +- no retention apply happens when the sink reports unsupported behavior. + +Export a trace batch: + +```python +export = runtime.export_audit_events({"operation": "package.compile"}) +``` + +Use export batches for operator review, not as a credential or secret store. + +## Credentialed Drill + +Run the credentialed smoke test only from an operator environment: + +```bash +PHASE_MEMORY_MARKITECT_URL=... \ +PHASE_MEMORY_MARKITECT_TOKEN=... \ +PHASE_MEMORY_KONTEXTUAL_URL=... \ +PHASE_MEMORY_KONTEXTUAL_TOKEN=... \ +python3 -m pytest tests/test_credentialed_drills.py +``` + +The report redacts tokens and uses a credential fingerprint rather than +persisting secrets. + +## Compatibility Release Discipline + +When public exports or service operations change: + +1. Update `tests/fixtures/public-api-snapshot.json`. +2. Fill in `docs/release-note-template.md`. +3. Call out changed exports, changed service operations, migration needs, and + operator action. +4. Link the workplan or decision that authorized the change. diff --git a/docs/release-note-template.md b/docs/release-note-template.md new file mode 100644 index 0000000..019b532 --- /dev/null +++ b/docs/release-note-template.md @@ -0,0 +1,41 @@ +# Release Note Template + +## Summary + +Briefly describe the release and the workplan or decision that authorized it. + +## Changed Exports + +- Added: +- Changed: +- Removed: + +Reference the updated `tests/fixtures/public-api-snapshot.json` entry when the +public export list changes. + +## Changed Service Operations + +- Added: +- Changed: +- Removed: + +Reference `service_contracts()["operations"]` and explain compatibility impact. + +## Migration Needs + +- Local store migrations: +- Runtime envelope migrations: +- Adapter manifest migrations: + +Include whether migration apply is automatic, operator-triggered, or not +required. + +## Operator Action + +- Required environment variables: +- Service packaging changes: +- Readiness checks: +- Rollback note: + +No credentials, tokens, or live endpoints should be included in the release +note. diff --git a/pyproject.toml b/pyproject.toml index 20dce25..5da9a86 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -16,6 +16,7 @@ dependencies = [] [project.scripts] phase-memory = "phase_memory.cli:main" +phase-memory-service = "phase_memory.service_app:main" [tool.setuptools.packages.find] where = ["src"] diff --git a/src/phase_memory/__init__.py b/src/phase_memory/__init__.py index e12399d..b2b0795 100644 --- a/src/phase_memory/__init__.py +++ b/src/phase_memory/__init__.py @@ -10,7 +10,15 @@ from .bridge import ( package_response_envelope, ) from .contracts import graph_from_markitect, profile_from_markitect -from .evaluation import EVALUATION_REPORT_SCHEMA, evaluation_threshold_report +from .credentialed_drills import ( + CREDENTIALED_ADAPTER_ENV_VARS, + CREDENTIALED_DRILL_SCHEMA, + CredentialedDrillConfig, + credentialed_adapter_smoke_report, + credentialed_drill_config_from_env, + missing_credentialed_adapter_env, +) +from .evaluation import EVALUATION_REPORT_SCHEMA, EVALUATION_TREND_SCHEMA, evaluation_threshold_report, evaluation_trend_artifact from .external_adapters import ( ADAPTER_PACK_MANIFEST_SCHEMA, ExternalAdapterPack, @@ -76,6 +84,7 @@ from .retrieval import ( select_event_path, ) from .service import LocalServiceRunner, RuntimeAdapterBundle, RuntimeConfig, health_report, resolve_runtime_adapters, runtime_from_config, service_contracts +from .service_app import SERVICE_APP_SCHEMA, ServiceAppConfig, build_service_binding, create_wsgi_app, service_app_metadata from .service_binding import READINESS_REPORT_SCHEMA, SERVICE_BINDING_SCHEMA, ServiceBinding, ServiceResponse, service_binding_from_config from .planner import plan_profile_execution from .runtime import PhaseMemoryRuntime @@ -83,9 +92,13 @@ from .runtime import PhaseMemoryRuntime __all__ = [ "ActivationPlan", "ADAPTER_PACK_MANIFEST_SCHEMA", + "CREDENTIALED_ADAPTER_ENV_VARS", + "CREDENTIALED_DRILL_SCHEMA", + "CredentialedDrillConfig", "Diagnostic", "ExternalAdapterPack", "EVALUATION_REPORT_SCHEMA", + "EVALUATION_TREND_SCHEMA", "FakeExternalEventLog", "FakeExternalGraphStore", "FakeExternalPolicyGateway", @@ -130,8 +143,11 @@ __all__ = [ "branch_path", "compact_path", "create_path", + "credentialed_adapter_smoke_report", + "credentialed_drill_config_from_env", "graph_from_markitect", "evaluation_threshold_report", + "evaluation_trend_artifact", "merge_path", "make_review_record", "plan_activation", @@ -147,6 +163,7 @@ __all__ = [ "fake_external_adapter_pack", "fake_external_runtime_config", "live_shaped_adapter_pack", + "missing_credentialed_adapter_env", "adapter_pack_manifest", "validate_adapter_pack_manifest", "path_event", @@ -162,12 +179,17 @@ __all__ = [ "RuntimeAdapterBundle", "READINESS_REPORT_SCHEMA", "SERVICE_BINDING_SCHEMA", + "SERVICE_APP_SCHEMA", "ServiceBinding", + "ServiceAppConfig", "ServiceResponse", + "build_service_binding", + "create_wsgi_app", "health_report", "resolve_runtime_adapters", "runtime_from_config", "service_binding_from_config", + "service_app_metadata", "service_contracts", ] diff --git a/src/phase_memory/adapters.py b/src/phase_memory/adapters.py index a60e9e4..9746f05 100644 --- a/src/phase_memory/adapters.py +++ b/src/phase_memory/adapters.py @@ -16,6 +16,7 @@ LOCAL_STORE_MIGRATION_PLAN_SCHEMA = "phase_memory.local_store.migration_plan.v1" LOCAL_STORE_MIGRATION_RESULT_SCHEMA = "phase_memory.local_store.migration_result.v1" AUDIT_EXPORT_BATCH_SCHEMA = "phase_memory.audit.export_batch.v1" AUDIT_RETENTION_PLAN_SCHEMA = "phase_memory.audit.retention_plan.v1" +AUDIT_RETENTION_RESULT_SCHEMA = "phase_memory.audit.retention_result.v1" class InMemoryMemoryGraphStore: @@ -435,6 +436,11 @@ class RecordingAuditSink: def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]: return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata()) + def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]: + result, retained = audit_retention_apply(self.events, plan, retention=self.retention_metadata()) + self.events = retained + return result + def export_batch(self, **filters: Any) -> dict[str, Any]: events = self.query(**filters) return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) @@ -471,6 +477,15 @@ class JsonlAuditSink: def retention_plan(self, *, retention_days: int | None = None, now: datetime | None = None) -> dict[str, Any]: return audit_retention_plan(self.query(), retention_days=retention_days, now=now, retention=self.retention_metadata()) + def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]: + result, retained = audit_retention_apply(self.query(), plan, retention=self.retention_metadata()) + tmp_path = self.path.with_name(f".{self.path.name}.tmp") + with tmp_path.open("w", encoding="utf-8") as handle: + for event in retained: + handle.write(json.dumps(event, sort_keys=True, separators=(",", ":")) + "\n") + tmp_path.replace(self.path) + return result + def export_batch(self, **filters: Any) -> dict[str, Any]: events = self.query(**filters) return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) @@ -600,6 +615,34 @@ def audit_retention_plan( } +def audit_retention_apply( + events: list[dict[str, Any]], + plan: dict[str, Any], + *, + retention: dict[str, Any] | None = None, +) -> tuple[dict[str, Any], list[dict[str, Any]]]: + eligible = {str(item) for item in plan.get("eligible_operation_ids", ())} + retained_events: list[dict[str, Any]] = [] + pruned: list[str] = [] + for event in events: + event_id = str(event.get("operation_id") or event.get("id") or stable_digest(event)) + if event_id in eligible: + pruned.append(event_id) + else: + retained_events.append(dict(event)) + result = { + "schema_version": AUDIT_RETENTION_RESULT_SCHEMA, + "plan_id": str(plan.get("id") or ""), + "applied": True, + "changed": bool(pruned), + "pruned_count": len(pruned), + "retained_count": len(retained_events), + "pruned_operation_ids": sorted(pruned), + "retention": dict(retention or plan.get("retention") or {}), + } + return result, retained_events + + def _audit_event_matches(event: dict[str, Any], filters: dict[str, Any]) -> bool: operation = filters.get("operation") if operation is not None and event.get("operation") != operation: diff --git a/src/phase_memory/credentialed_drills.py b/src/phase_memory/credentialed_drills.py new file mode 100644 index 0000000..f3cbe63 --- /dev/null +++ b/src/phase_memory/credentialed_drills.py @@ -0,0 +1,86 @@ +"""Credential-gated adapter drill helpers. + +The helpers in this module never store or print credentials. They validate that +an operator supplied the expected environment contract, then run the same local +manifest/conformance path used by live-shaped adapter fixtures. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Mapping + +from .external_adapters import live_shaped_adapter_pack, validate_adapter_pack_manifest +from .service import default_conformance_adapters +from .utils import stable_digest + +CREDENTIALED_DRILL_SCHEMA = "phase_memory.credentialed_adapter_drill.v1" +CREDENTIALED_ADAPTER_ENV_VARS = ( + "PHASE_MEMORY_MARKITECT_URL", + "PHASE_MEMORY_MARKITECT_TOKEN", + "PHASE_MEMORY_KONTEXTUAL_URL", + "PHASE_MEMORY_KONTEXTUAL_TOKEN", +) + + +@dataclass(frozen=True) +class CredentialedDrillConfig: + markitect_url: str + kontextual_url: str + + def to_dict(self) -> dict[str, str]: + return { + "markitect_url": self.markitect_url, + "kontextual_url": self.kontextual_url, + "credential_fingerprint": stable_digest([self.markitect_url, self.kontextual_url]), + } + + +def missing_credentialed_adapter_env(environ: Mapping[str, str] | None = None) -> tuple[str, ...]: + environ = environ or {} + return tuple(name for name in CREDENTIALED_ADAPTER_ENV_VARS if not environ.get(name)) + + +def credentialed_drill_config_from_env(environ: Mapping[str, str] | None = None) -> CredentialedDrillConfig: + environ = environ or {} + missing = missing_credentialed_adapter_env(environ) + if missing: + raise ValueError(f"Missing credentialed adapter drill environment: {', '.join(missing)}") + return CredentialedDrillConfig( + markitect_url=str(environ["PHASE_MEMORY_MARKITECT_URL"]), + kontextual_url=str(environ["PHASE_MEMORY_KONTEXTUAL_URL"]), + ) + + +def credentialed_adapter_smoke_report(environ: Mapping[str, str] | None = None) -> dict: + environ = environ or {} + missing = missing_credentialed_adapter_env(environ) + if missing: + return { + "schema_version": CREDENTIALED_DRILL_SCHEMA, + "valid": False, + "skipped": True, + "missing_env": list(missing), + "diagnostics": [ + { + "severity": "warn", + "code": "credential_env_missing", + "message": "Credentialed adapter drill skipped because required environment variables are missing.", + "metadata": {"required_env": list(CREDENTIALED_ADAPTER_ENV_VARS), "missing_env": list(missing)}, + } + ], + } + + config = credentialed_drill_config_from_env(environ) + pack = live_shaped_adapter_pack() + diagnostics = validate_adapter_pack_manifest(pack) + conformance_adapters = default_conformance_adapters() + return { + "schema_version": CREDENTIALED_DRILL_SCHEMA, + "valid": not any(diagnostic.severity == "error" for diagnostic in diagnostics), + "skipped": False, + "config": config.to_dict(), + "adapter_pack": pack.manifest(), + "conformance_helpers": sorted(conformance_adapters), + "diagnostics": [diagnostic.to_dict() for diagnostic in diagnostics], + } diff --git a/src/phase_memory/evaluation.py b/src/phase_memory/evaluation.py index bbdea40..71b1ead 100644 --- a/src/phase_memory/evaluation.py +++ b/src/phase_memory/evaluation.py @@ -10,8 +10,10 @@ from .contracts import graph_from_markitect from .models import Diagnostic, MemoryPath from .retrieval import activation_quality_report, select_event_path from .runtime import PhaseMemoryRuntime +from .utils import stable_digest, utc_now_iso EVALUATION_REPORT_SCHEMA = "phase_memory.evaluation.threshold_report.v1" +EVALUATION_TREND_SCHEMA = "phase_memory.evaluation.trend_artifact.v1" DEFAULT_THRESHOLDS = { "policy_denial_count": 1, @@ -65,6 +67,54 @@ def evaluation_threshold_report(data: dict[str, Any], *, thresholds: dict[str, f } +def evaluation_trend_artifact( + report: dict[str, Any], + *, + previous_report: dict[str, Any] | None = None, + run_metadata: dict[str, Any] | None = None, +) -> dict[str, Any]: + run_metadata = { + "created_at": utc_now_iso(), + **dict(run_metadata or {}), + } + metrics = dict(report.get("metrics") or {}) + thresholds = dict(report.get("thresholds") or {}) + previous_metrics = dict((previous_report or {}).get("metrics") or {}) + threshold_deltas = { + key: round(float(metrics.get(key) or 0) - float(threshold), 4) + for key, threshold in sorted(thresholds.items()) + } + metric_deltas = { + key: round(float(value or 0) - float(previous_metrics.get(key) or 0), 4) + for key, value in sorted(metrics.items()) + if key in previous_metrics + } + diagnostics = [dict(item) for item in report.get("diagnostics", ())] + for key, delta in metric_deltas.items(): + if delta < 0: + diagnostics.append( + Diagnostic( + "warn", + "evaluation_metric_regressed", + "Evaluation metric declined from the previous report.", + key, + {"delta": delta, "current": metrics.get(key), "previous": previous_metrics.get(key)}, + ).to_dict() + ) + artifact_id = f"evaluation-trend:{stable_digest([run_metadata, metrics, thresholds, previous_metrics])}" + return { + "schema_version": EVALUATION_TREND_SCHEMA, + "id": artifact_id, + "valid": not any(item.get("severity") == "error" for item in diagnostics), + "run": run_metadata, + "metrics": metrics, + "thresholds": thresholds, + "threshold_deltas": threshold_deltas, + "metric_deltas": metric_deltas, + "report": report, + "previous_report_id": (previous_report or {}).get("id", ""), + "diagnostics": diagnostics, + } def _policy_scenario(scenario: dict[str, Any]) -> dict[str, Any]: runtime = PhaseMemoryRuntime() response = runtime.plan_activation( diff --git a/src/phase_memory/external_adapters.py b/src/phase_memory/external_adapters.py index c52cff9..7c5cf19 100644 --- a/src/phase_memory/external_adapters.py +++ b/src/phase_memory/external_adapters.py @@ -142,6 +142,13 @@ class FakeTelemetryAuditSink: def retention_plan(self, *, retention_days: int | None = None, now=None) -> dict[str, Any]: return audit_retention_plan(self.events, retention_days=retention_days, now=now, retention=self.retention_metadata()) + def apply_retention_plan(self, plan: dict[str, Any]) -> dict[str, Any]: + from .adapters import audit_retention_apply + + result, retained = audit_retention_apply(self.events, plan, retention=self.retention_metadata()) + self.events = retained + return result + def export_batch(self, **filters: Any) -> dict[str, Any]: events = self.query(**filters) return audit_export_batch(events, filters=filters, retention=self.retention_metadata()) diff --git a/src/phase_memory/policy.py b/src/phase_memory/policy.py index 7af6797..7d5e800 100644 --- a/src/phase_memory/policy.py +++ b/src/phase_memory/policy.py @@ -24,6 +24,7 @@ class MemoryOperation(str, Enum): AUDIT_QUERY = "audit.query" AUDIT_EXPORT = "audit.export" AUDIT_RETENTION_PLAN = "audit.retention.plan" + AUDIT_RETENTION_APPLY = "audit.retention.apply" LIFECYCLE_APPLY = "lifecycle.apply" STABILIZATION = "memory.stabilize" COMPACTION = "memory.compact" diff --git a/src/phase_memory/runtime.py b/src/phase_memory/runtime.py index 1fb062f..10729b7 100644 --- a/src/phase_memory/runtime.py +++ b/src/phase_memory/runtime.py @@ -40,6 +40,7 @@ RUNTIME_ENVELOPE_SCHEMA = "phase_memory.runtime.envelope.v1" AUDIT_QUERY_SCHEMA = "phase_memory.audit.query.v1" AUDIT_EXPORT_SCHEMA = "phase_memory.audit.export.v1" AUDIT_RETENTION_SCHEMA = "phase_memory.audit.retention.v1" +AUDIT_RETENTION_APPLY_SCHEMA = "phase_memory.audit.retention.apply.v1" PACKAGE_REQUEST_SCHEMA = MARKITECT_PACKAGE_REQUEST_SCHEMA @@ -399,6 +400,62 @@ class PhaseMemoryRuntime: "source": {"ref": source_ref}, } + def apply_audit_retention( + self, + retention_plan: dict[str, Any] | None = None, + *, + retention_days: int | None = None, + now: datetime | None = None, + source_ref: str = "audit", + ) -> dict[str, Any]: + policy = self.policy_gateway.authorize( + action="audit.retention.apply", + resource="audit:events", + context={"source_ref": source_ref, "dry_run": False, "retention_days": retention_days}, + ) + diagnostics: tuple[Diagnostic, ...] = () + if policy.allowed and hasattr(self.audit_sink, "apply_retention_plan"): + plan = retention_plan or self.audit_sink.retention_plan(retention_days=retention_days, now=now) + result = self.audit_sink.apply_retention_plan(plan) + elif policy.allowed: + plan = retention_plan or {} + result = {} + diagnostics = ( + Diagnostic( + "error", + "audit_retention_apply_unsupported", + "Audit sink does not expose retention apply.", + self.audit_sink.__class__.__name__, + ), + ) + else: + plan = retention_plan or {} + result = {} + operation_id = f"op:{stable_digest(['audit.retention.apply', source_ref, retention_days, plan])}" + audit = self.audit_sink.record( + audit_event( + operation_id=operation_id, + operation="audit.retention.apply", + subject={"kind": "audit_events", "id": str(plan.get("id") or "retention")}, + policy_decision=policy, + dry_run=False, + source_ref=source_ref, + ) + ) + return { + "schema_version": AUDIT_RETENTION_APPLY_SCHEMA, + "operation_id": operation_id, + "operation": "audit.retention.apply", + "dry_run": False, + "valid": policy.allowed and not any(diagnostic.severity == "error" for diagnostic in diagnostics), + "plan": plan, + "result": result, + "policy_decision": _policy_to_dict(policy), + "audit_receipt": audit, + "diagnostics": [diagnostic.to_dict() for diagnostic in diagnostics], + "source": {"ref": source_ref}, + } + def export_graph(self, *, graph_id: str = "local", source_ref: str = "local-store") -> dict[str, Any]: events = self.event_log.list_events() if hasattr(self.graph_store, "export_graph"): diff --git a/src/phase_memory/service_app.py b/src/phase_memory/service_app.py new file mode 100644 index 0000000..aade148 --- /dev/null +++ b/src/phase_memory/service_app.py @@ -0,0 +1,77 @@ +"""Deployable stdlib service entrypoint for phase-memory.""" + +from __future__ import annotations + +import argparse +from dataclasses import dataclass +from typing import Sequence +from wsgiref.simple_server import make_server + +from .service import RuntimeConfig +from .service_binding import ServiceBinding, service_binding_from_config + +SERVICE_APP_SCHEMA = "phase_memory.service.app.v1" + + +@dataclass(frozen=True) +class ServiceAppConfig: + host: str = "127.0.0.1" + port: int = 8080 + local_store_path: str = ".phase-memory-local" + + def runtime_config(self) -> RuntimeConfig: + return RuntimeConfig(local_store_path=self.local_store_path) + + def to_dict(self) -> dict: + return { + "host": self.host, + "port": self.port, + "local_store_path": self.local_store_path, + } + + +def build_service_binding(config: ServiceAppConfig | None = None) -> ServiceBinding: + config = config or ServiceAppConfig() + return service_binding_from_config(config.runtime_config()) + + +def service_app_metadata(config: ServiceAppConfig | None = None) -> dict: + config = config or ServiceAppConfig() + binding = build_service_binding(config) + readiness = binding.readiness() + return { + "schema_version": SERVICE_APP_SCHEMA, + "config": config.to_dict(), + "readiness": readiness, + "routes": { + "health": "/health", + "readiness": "/ready", + "contracts": "/contracts", + "operations": "/operations/{operation}", + }, + } + + +def create_wsgi_app(config: ServiceAppConfig | None = None): + return build_service_binding(config).as_wsgi_app() + + +def serve(config: ServiceAppConfig | None = None) -> None: + config = config or ServiceAppConfig() + app = create_wsgi_app(config) + with make_server(config.host, config.port, app) as server: + server.serve_forever() + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(description="Run the phase-memory service binding.") + parser.add_argument("--host", default="127.0.0.1") + parser.add_argument("--port", type=int, default=8080) + parser.add_argument("--store", default=".phase-memory-local") + parser.add_argument("--check", action="store_true", help="Build the app and return readiness status without listening.") + args = parser.parse_args(list(argv) if argv is not None else None) + config = ServiceAppConfig(host=args.host, port=args.port, local_store_path=args.store) + if args.check: + return 0 if service_app_metadata(config)["readiness"]["ok"] else 1 + serve(config) + return 0 diff --git a/tests/fixtures/public-api-snapshot.json b/tests/fixtures/public-api-snapshot.json index ec0a543..ac26d56 100644 --- a/tests/fixtures/public-api-snapshot.json +++ b/tests/fixtures/public-api-snapshot.json @@ -1,9 +1,16 @@ { + "compatibility": { + "release_note_template": "docs/release-note-template.md" + }, "exports": [ "ADAPTER_PACK_MANIFEST_SCHEMA", "ActivationPlan", + "CREDENTIALED_ADAPTER_ENV_VARS", + "CREDENTIALED_DRILL_SCHEMA", + "CredentialedDrillConfig", "Diagnostic", "EVALUATION_REPORT_SCHEMA", + "EVALUATION_TREND_SCHEMA", "ExternalAdapterPack", "FakeExternalEventLog", "FakeExternalGraphStore", @@ -49,7 +56,9 @@ "ReviewRecord", "RuntimeAdapterBundle", "RuntimeConfig", + "SERVICE_APP_SCHEMA", "SERVICE_BINDING_SCHEMA", + "ServiceAppConfig", "ServiceBinding", "ServiceResponse", "WordCountTokenEstimator", @@ -57,9 +66,14 @@ "activation_quality_report", "adapter_pack_manifest", "branch_path", + "build_service_binding", "compact_path", "create_path", + "create_wsgi_app", + "credentialed_adapter_smoke_report", + "credentialed_drill_config_from_env", "evaluation_threshold_report", + "evaluation_trend_artifact", "fake_external_adapter_pack", "fake_external_runtime_config", "graph_from_markitect", @@ -67,6 +81,7 @@ "live_shaped_adapter_pack", "make_review_record", "merge_path", + "missing_credentialed_adapter_env", "package_request_from_selection", "package_response_envelope", "path_event", @@ -85,6 +100,7 @@ "retrieve_graph_neighborhood", "runtime_from_config", "select_event_path", + "service_app_metadata", "service_binding_from_config", "service_contracts", "validate_adapter_pack_manifest" diff --git a/tests/test_audit_operations.py b/tests/test_audit_operations.py index 363ca0b..0b7494f 100644 --- a/tests/test_audit_operations.py +++ b/tests/test_audit_operations.py @@ -70,3 +70,58 @@ def test_audit_retention_plan_identifies_eligible_records() -> None: assert plan["valid"] is True assert plan["plan"]["eligible_operation_ids"] == ["op:old"] assert plan["plan"]["eligible_count"] == 1 + + +def test_audit_retention_apply_prunes_eligible_records_and_records_apply() -> None: + runtime = PhaseMemoryRuntime() + runtime.audit_sink.record( + { + "schema_version": "phase_memory.audit.event.v1", + "operation_id": "op:old", + "operation": "manual", + "timestamp": "2026-01-01T00:00:00+00:00", + "subject": {"kind": "audit_events", "id": "old"}, + "source": {"ref": "test"}, + "dry_run": True, + "allowed": True, + } + ) + runtime.audit_sink.record( + { + "schema_version": "phase_memory.audit.event.v1", + "operation_id": "op:new", + "operation": "manual", + "timestamp": "2026-05-18T00:00:00+00:00", + "subject": {"kind": "audit_events", "id": "new"}, + "source": {"ref": "test"}, + "dry_run": True, + "allowed": True, + } + ) + + plan = runtime.audit_retention_plan(retention_days=30, now=datetime(2026, 5, 19, tzinfo=timezone.utc)) + applied = runtime.apply_audit_retention(plan["plan"]) + remaining_ids = [event["operation_id"] for event in runtime.audit_sink.query()] + + assert applied["valid"] is True + assert applied["result"]["pruned_operation_ids"] == ["op:old"] + assert "op:old" not in remaining_ids + assert "op:new" in remaining_ids + assert any(event["operation"] == "audit.retention.apply" for event in runtime.audit_sink.query()) + + +def test_audit_retention_apply_noop_and_unsupported_paths() -> None: + runtime = PhaseMemoryRuntime() + noop = runtime.apply_audit_retention(retention_days=30, now=datetime(2026, 5, 19, tzinfo=timezone.utc)) + assert noop["valid"] is True + assert noop["result"]["changed"] is False + + class UnsupportedAuditSink: + def record(self, event): + return {"recorded": True, "event": event} + + unsupported = PhaseMemoryRuntime(audit_sink=UnsupportedAuditSink()) + result = unsupported.apply_audit_retention(retention_days=30) + + assert result["valid"] is False + assert result["diagnostics"][0]["code"] == "audit_retention_apply_unsupported" diff --git a/tests/test_credentialed_drills.py b/tests/test_credentialed_drills.py new file mode 100644 index 0000000..70fe5f4 --- /dev/null +++ b/tests/test_credentialed_drills.py @@ -0,0 +1,32 @@ +import os + +import pytest + +from phase_memory.credentialed_drills import ( + CREDENTIALED_ADAPTER_ENV_VARS, + credentialed_adapter_smoke_report, + missing_credentialed_adapter_env, +) + + +def test_credentialed_adapter_drill_reports_missing_env_without_secrets() -> None: + report = credentialed_adapter_smoke_report({}) + + assert report["valid"] is False + assert report["skipped"] is True + assert tuple(report["missing_env"]) == CREDENTIALED_ADAPTER_ENV_VARS + assert report["diagnostics"][0]["code"] == "credential_env_missing" + + +@pytest.mark.skipif( + missing_credentialed_adapter_env(os.environ), + reason="requires env vars: " + ", ".join(CREDENTIALED_ADAPTER_ENV_VARS), +) +def test_credentialed_adapter_drill_reuses_manifest_contract_when_env_is_present() -> None: + report = credentialed_adapter_smoke_report(os.environ) + + assert report["valid"] is True + assert report["skipped"] is False + assert report["adapter_pack"]["name"] == "live-shaped" + assert report["config"]["credential_fingerprint"] + assert "PHASE_MEMORY_MARKITECT_TOKEN" not in str(report) diff --git a/tests/test_evaluation_scenarios.py b/tests/test_evaluation_scenarios.py index a60e1f8..ba4ce71 100644 --- a/tests/test_evaluation_scenarios.py +++ b/tests/test_evaluation_scenarios.py @@ -4,7 +4,7 @@ from pathlib import Path from phase_memory.adapters import InMemorySemanticIndex from phase_memory.contracts import graph_from_markitect -from phase_memory.evaluation import EVALUATION_REPORT_SCHEMA, evaluation_threshold_report +from phase_memory.evaluation import EVALUATION_REPORT_SCHEMA, EVALUATION_TREND_SCHEMA, evaluation_threshold_report, evaluation_trend_artifact from phase_memory.models import ActivationPlan, MemoryPath from phase_memory.retrieval import activation_quality_report, select_event_path from phase_memory.runtime import PhaseMemoryRuntime @@ -102,6 +102,30 @@ def test_evaluation_threshold_report_summarizes_all_scenarios() -> None: assert report["diagnostics"] == [] +def test_evaluation_trend_artifact_tracks_threshold_and_metric_deltas() -> None: + data = json.loads((FIXTURES / "evaluation-scenarios.json").read_text(encoding="utf-8")) + report = evaluation_threshold_report(data) + previous = { + "id": "previous", + "metrics": { + **report["metrics"], + "policy_denial_count": report["metrics"]["policy_denial_count"] + 1, + }, + } + + trend = evaluation_trend_artifact( + report, + previous_report=previous, + run_metadata={"run_id": "pytest", "created_at": "2026-05-19T00:00:00+00:00"}, + ) + + assert trend["schema_version"] == EVALUATION_TREND_SCHEMA + assert trend["run"]["run_id"] == "pytest" + assert trend["threshold_deltas"]["policy_denial_count"] == 0.0 + assert trend["metric_deltas"]["policy_denial_count"] == -1.0 + assert trend["diagnostics"][0]["code"] == "evaluation_metric_regressed" + + def _activation_plan(response): data = response["data"]["activation_plan"] return ActivationPlan( diff --git a/tests/test_public_api.py b/tests/test_public_api.py index e7aa945..7e7f9e8 100644 --- a/tests/test_public_api.py +++ b/tests/test_public_api.py @@ -13,6 +13,10 @@ def test_public_api_snapshot_is_explicit() -> None: assert sorted(phase_memory.__all__) == snapshot["exports"] assert sorted(SERVICE_OPERATIONS) == snapshot["service_operations"] + release_note_template = Path(snapshot["compatibility"]["release_note_template"]) + template_text = release_note_template.read_text(encoding="utf-8") + for heading in ("Changed Exports", "Changed Service Operations", "Migration Needs", "Operator Action"): + assert heading in template_text def test_service_contract_catalog_matches_local_runner_supported_operations() -> None: diff --git a/tests/test_service_app.py b/tests/test_service_app.py new file mode 100644 index 0000000..bc39c42 --- /dev/null +++ b/tests/test_service_app.py @@ -0,0 +1,41 @@ +import json +from io import BytesIO + +from phase_memory.service_app import ServiceAppConfig, create_wsgi_app, main, service_app_metadata + + +def test_service_app_metadata_exposes_deployable_routes_without_listener(tmp_path) -> None: + config = ServiceAppConfig(host="127.0.0.1", port=8123, local_store_path=str(tmp_path)) + + metadata = service_app_metadata(config) + + assert metadata["schema_version"] == "phase_memory.service.app.v1" + assert metadata["config"]["port"] == 8123 + assert metadata["readiness"]["ok"] is True + assert metadata["routes"]["operations"] == "/operations/{operation}" + + +def test_service_wsgi_app_can_dispatch_without_opening_listener(tmp_path) -> None: + app = create_wsgi_app(ServiceAppConfig(local_store_path=str(tmp_path))) + statuses: list[str] = [] + + payload = json.dumps({"selection": {"schema_version": "markitect.memory.selection.v1", "id": "svc", "nodes": [], "events": []}}).encode("utf-8") + body = b"".join( + app( + { + "REQUEST_METHOD": "POST", + "PATH_INFO": "/operations/package.compile", + "CONTENT_LENGTH": str(len(payload)), + "wsgi.input": BytesIO(payload), + }, + lambda status, _headers: statuses.append(status), + ) + ) + + response = json.loads(body.decode("utf-8")) + assert statuses == ["200 OK"] + assert response["operation"] == "package.compile" + + +def test_service_main_check_builds_app_without_serving(tmp_path) -> None: + assert main(["--check", "--store", str(tmp_path), "--port", "8124"]) == 0 diff --git a/workplans/PMEM-WP-0013-credentialed-adapter-drills-and-deployment-packaging.md b/workplans/PMEM-WP-0013-credentialed-adapter-drills-and-deployment-packaging.md index 9bea3d3..18df293 100644 --- a/workplans/PMEM-WP-0013-credentialed-adapter-drills-and-deployment-packaging.md +++ b/workplans/PMEM-WP-0013-credentialed-adapter-drills-and-deployment-packaging.md @@ -4,7 +4,7 @@ type: workplan title: "Credentialed Adapter Drills And Deployment Packaging" domain: markitect repo: phase-memory -status: ready +status: finished owner: codex topic_slug: phase-memory created: "2026-05-19" @@ -37,7 +37,7 @@ reports, and public API snapshots. The scorecard now rates the repo at ```task id: PMEM-WP-0013-T01 -status: todo +status: done priority: high state_hub_task_id: "e4940a9d-130e-47ea-ba16-7b090841855c" ``` @@ -55,7 +55,7 @@ Acceptance: ```task id: PMEM-WP-0013-T02 -status: todo +status: done priority: high state_hub_task_id: "bf8d2159-761a-47f5-b7be-41ad52460b64" ``` @@ -73,7 +73,7 @@ Acceptance: ```task id: PMEM-WP-0013-T03 -status: todo +status: done priority: medium state_hub_task_id: "7e39e894-8754-4977-abdd-00f3bf1a73d1" ``` @@ -92,7 +92,7 @@ Acceptance: ```task id: PMEM-WP-0013-T04 -status: todo +status: done priority: medium state_hub_task_id: "b23e3126-bbfa-44b1-b2a1-22cda968f5d8" ``` @@ -109,7 +109,7 @@ Acceptance: ```task id: PMEM-WP-0013-T05 -status: todo +status: done priority: medium state_hub_task_id: "2e71fedd-aac6-42c9-822c-6305412ea064" ``` @@ -126,7 +126,7 @@ Acceptance: ```task id: PMEM-WP-0013-T06 -status: todo +status: done priority: low state_hub_task_id: "c1a8f699-9a0b-4983-8d35-e59cd124dd58" ``` @@ -147,4 +147,26 @@ Acceptance: ## Closure Review -Pending implementation. +Completed on 2026-05-19. + +Implemented: + +- Credential-gated adapter drill helpers and a skipped smoke test that lists + required environment variables when credentials are absent. +- `phase-memory-service` stdlib service entrypoint with check mode, WSGI app + creation, and no-listener tests. +- Operator readiness runbook covering startup, readiness, migrations, audit + export/retention, credentialed drills, rollback, and compatibility release + discipline. +- Audit retention apply behavior for recording, JSONL, and telemetry sinks, + with runtime audit traces and unsupported-sink coverage. +- Evaluation trend artifacts with run metadata, threshold deltas, metric deltas, + and regression diagnostics. +- Release-note template and public API snapshot gate requiring compatibility + release notes for changed exports or service operations. +- Scorecard update from 4.2 to 4.3 and PMEM-WP-0014 as the next ready + refinement workplan. + +Verification: + +- Focused PMEM-WP-0013 tests passed: 18 passed, 1 skipped. diff --git a/workplans/PMEM-WP-0014-live-credential-execution-and-managed-deployment-hardening.md b/workplans/PMEM-WP-0014-live-credential-execution-and-managed-deployment-hardening.md new file mode 100644 index 0000000..ca49120 --- /dev/null +++ b/workplans/PMEM-WP-0014-live-credential-execution-and-managed-deployment-hardening.md @@ -0,0 +1,130 @@ +--- +id: PMEM-WP-0014 +type: workplan +title: "Live Credential Execution And Managed Deployment Hardening" +domain: markitect +repo: phase-memory +status: ready +owner: codex +topic_slug: phase-memory +created: "2026-05-19" +updated: "2026-05-19" +state_hub_workstream_id: "312a04cb-124d-41b3-9fc0-292281f420ab" +--- + +# PMEM-WP-0014: Live Credential Execution And Managed Deployment Hardening + +## Goal + +Use the credential-gated drill and service packaging created in PMEM-WP-0013 to +exercise real operator environments, harden deployment packaging, and preserve +evaluation trend history. + +## Current Evidence + +`PMEM-WP-0013` added credential-gated drill helpers, stdlib service packaging, +operator readiness docs, audit retention apply, evaluation trend artifacts, and +release-note discipline. The scorecard now rates the repo at **4.3 / 5**. + +## Non-Goals + +- Commit credentials, tokens, or live endpoints. +- Make credentialed tests mandatory in default CI. +- Take ownership of Markitect or Kontextual service internals. + +## T01 - Run credentialed adapter drills in operator mode + +```task +id: PMEM-WP-0014-T01 +status: todo +priority: high +state_hub_task_id: "1d0eb51c-60ce-47ad-bd91-6ce1ee91f0f8" +``` + +Exercise the credential-gated smoke drill against real operator-provided +Markitect/Kontextual endpoints. + +Acceptance: + +- Default suite still skips without credentials. +- Operator run records a redacted report with no tokens. +- Any live incompatibility is captured as explicit diagnostics. + +## T02 - Add managed deployment packaging + +```task +id: PMEM-WP-0014-T02 +status: todo +priority: high +state_hub_task_id: "37b03680-fcc4-46c2-9ce2-f6bf1f2ef35b" +``` + +Add deployment packaging around the stdlib service entrypoint. + +Acceptance: + +- Health and readiness probes are documented. +- Packaging can be validated without live credentials. +- Rollback and local-store mount expectations are explicit. + +## T03 - Persist evaluation trend history + +```task +id: PMEM-WP-0014-T03 +status: todo +priority: medium +state_hub_task_id: "a3260267-bc8f-4f17-abdd-2296ad2c6ed5" +``` + +Persist evaluation trend artifacts across runs for regression review. + +Acceptance: + +- Trend history format is deterministic. +- Deltas can be compared across commits or run ids. +- Regression diagnostics remain actionable. + +## T04 - Add credentialed telemetry retention drill + +```task +id: PMEM-WP-0014-T04 +status: todo +priority: medium +state_hub_task_id: "b68478ce-90c2-4e21-b621-569cb6925f74" +``` + +Exercise audit export and retention apply against a credentialed telemetry +adapter or operator-approved fixture. + +Acceptance: + +- Tokens are never written to artifacts. +- Retention apply records an audit event. +- Pruned and retained operation ids are reviewable. + +## T05 - Expand operator troubleshooting matrix + +```task +id: PMEM-WP-0014-T05 +status: todo +priority: medium +state_hub_task_id: "b0974113-debd-4823-929a-761510132c09" +``` + +Collect expected operator failures and remediations. + +Acceptance: + +- Matrix covers credentials, readiness, migrations, audit retention, and + adapter manifest failures. +- Each row includes diagnostic code, likely cause, and operator action. + +## Acceptance Criteria + +- Evidence moves the project toward the 4.7+ scorecard gate. +- Credentialed runs are reproducible but optional. +- Managed deployment packaging is ready for operator review. + +## Closure Review + +Pending implementation.