generated from coulomb/repo-seed
Implement credentialed drill packaging workplan
This commit is contained in:
@@ -26,24 +26,24 @@ to 5.
|
||||
|
||||
## Current Score
|
||||
|
||||
Overall maturity: **4.2 / 5**
|
||||
Overall maturity: **4.3 / 5**
|
||||
|
||||
Two sub-scores make the result easier to reason about:
|
||||
|
||||
- Local integration maturity: **4.5 / 5**
|
||||
- Operational maturity: **3.8 / 5**
|
||||
- Local integration maturity: **4.6 / 5**
|
||||
- Operational maturity: **4.0 / 5**
|
||||
|
||||
The repo is strong as a deterministic local library and service-boundary core.
|
||||
It is not yet production-operational because adapter coverage is still
|
||||
live-shaped rather than credentialed live integration, and service bindings are
|
||||
framework-neutral embedding surfaces rather than a deployed service.
|
||||
credential-gated rather than continuously exercised against live services, and
|
||||
service packaging is stdlib/local rather than deployed to a managed environment.
|
||||
|
||||
## Dimension Scorecard
|
||||
|
||||
| Dimension | Score | Target | Evidence | Needed Next |
|
||||
| --- | ---: | ---: | --- | --- |
|
||||
| Intent and boundaries | 4.4 | 5.0 | `INTENT.md`, `SCOPE.md`, `README.md`, architecture docs, adjacent-repo boundary docs | Keep docs current as live adapters and service bindings clarify real ownership. |
|
||||
| Package and API foundation | 4.5 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot | Add release notes discipline and compatibility migration examples. |
|
||||
| Package and API foundation | 4.6 | 4.8 | Python package, public exports, runtime facade, CLI, service runner export, service config, dependency-light tests, public API snapshot, release-note template | Add compatibility migration examples from a real release. |
|
||||
| Markitect profile contract ingress | 3.7 | 4.5 | Profile loading, diagnostics, runtime envelopes, profile-derived config, local alias normalization | Add richer compatibility fixtures and schema drift diagnostics. |
|
||||
| Graph and event ingress | 4.0 | 4.5 | Graph loading, endpoint diagnostics, event model, JSONL log, export, repair checks, corrupt-record diagnostics, fake and live-shaped graph/event adapters | Add broader malformed/large graph fixtures and operator repair utilities. |
|
||||
| Phase domain model | 3.5 | 4.5 | Phases, lifecycle states, actions, paths, retention rules, profile-derived transition rules | Add migration semantics for profile/rule changes over durable stores. |
|
||||
@@ -51,26 +51,26 @@ framework-neutral embedding surfaces rather than a deployed service.
|
||||
| Lifecycle planning and apply | 4.1 | 4.5 | Dry-run lifecycle plans, profile rules, review-gated local apply, service `lifecycle.apply`, apply audit/export queries | Add richer apply rollback and repair drills. |
|
||||
| Activation planning | 4.0 | 4.8 | Budgeted activation, selections, package request, graph neighborhoods, paths, ranking, metrics, multi-scenario evaluation fixtures | Wire semantic-index-assisted retrieval into runtime planning. |
|
||||
| Local persistence | 4.0 | 4.5 | File-backed graph store, JSONL event log, audit sink, atomic JSON writes, executable metadata migrations, migration audit, export, repair diagnostics | Add compaction/retention utilities and stronger corruption recovery. |
|
||||
| Policy, review, and audit | 4.2 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and enforceable audit retention pruning. |
|
||||
| Observability and operations | 4.0 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, fake/live-shaped telemetry audit sinks, operational recipe | Add metrics/event export to external telemetry and deployable service packaging. |
|
||||
| Markitect interop | 4.0 | 4.5 | Local validation, package request/response envelopes, fake and live-shaped compiler fixtures | Add optional credentialed Markitect compiler adapter and schema drift suite. |
|
||||
| Kontextual/Infospace interop | 3.7 | 4.5 | Delegation envelope, fake and live-shaped runtime registry, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual adapter drill and broader Infospace restart reports. |
|
||||
| Testing and evaluation | 4.3 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, API snapshots, and evaluation threshold reports | Add larger regression corpus and threshold trend reports. |
|
||||
| Service readiness | 4.5 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, health/readiness, config, adapter conformance | Add deployable packaging and operator readiness runbooks. |
|
||||
| Developer experience | 4.3 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, API compatibility docs | Add troubleshooting matrix and release note templates. |
|
||||
| Policy, review, and audit | 4.4 | 5.0 | Operation points, review records, audit schema, queryable/exportable audit sinks, retention plans and apply, denials, redaction, fake/live-shaped policy/audit adapters | Add live policy adapter boundary and credentialed telemetry pruning drill. |
|
||||
| Observability and operations | 4.3 | 4.8 | Health report, readiness report, config diagnostics, adapter status, service binding, stdlib service entrypoint, operator runbook, fake/live-shaped telemetry audit sinks | Add metrics/event export to external telemetry and managed deployment packaging. |
|
||||
| Markitect interop | 4.1 | 4.5 | Local validation, package request/response envelopes, fake/live-shaped compiler fixtures, credential-gated drill contract | Add credentialed Markitect compiler execution and schema drift suite. |
|
||||
| Kontextual/Infospace interop | 3.9 | 4.5 | Delegation envelope, fake/live-shaped runtime registry, credential-gated drill contract, activation quality report fixture, adapter compatibility manifests | Add credentialed Kontextual execution and broader Infospace restart reports. |
|
||||
| Testing and evaluation | 4.5 | 4.7 | Deterministic tests over runtime, CLI, adapters, policy, activation, lifecycle, service, fakes, live-shaped packs, credential skip gates, API snapshots, evaluation threshold and trend reports | Add larger regression corpus and persisted trend history. |
|
||||
| Service readiness | 4.6 | 4.8 | Service contracts, full local runner parity, framework-neutral service binding, WSGI adapter, stdlib service entrypoint, health/readiness, config, adapter conformance | Add managed deployment packaging. |
|
||||
| Developer experience | 4.5 | 4.7 | README, package map, CLI examples, persistence/policy/interop/service/lifecycle/fake-pack docs, operational recipe, operator runbook, API compatibility docs, release-note template | Add troubleshooting matrix from real operator feedback. |
|
||||
|
||||
## Assessment
|
||||
|
||||
The project has crossed the local integration-readiness threshold. The runtime
|
||||
envelopes, policy/review model, profile-derived configuration, lifecycle rules,
|
||||
local persistence migrations, queryable/exportable audit path, fake and
|
||||
live-shaped external pack manifests, service binding, API snapshots, and
|
||||
local persistence migrations, queryable/exportable/prunable audit path, fake
|
||||
and live-shaped external pack manifests, credential-gated drills, service
|
||||
binding and stdlib entrypoint, API snapshots, release discipline, and
|
||||
conformance helpers form a solid integration boundary.
|
||||
|
||||
The biggest optimization opportunity is now the next operational layer:
|
||||
moving from live-shaped local fixtures to credentialed live adapter drills,
|
||||
packaging the service binding for deployment, and growing evaluation thresholds
|
||||
into trend reports.
|
||||
running the credential-gated drills against real services, adding managed
|
||||
deployment packaging, and growing evaluation trends into a historical corpus.
|
||||
|
||||
## Completed Refinement Workplan
|
||||
|
||||
@@ -97,20 +97,30 @@ into trend reports.
|
||||
- evaluation threshold reports over the scenario corpus;
|
||||
- public API and service operation compatibility snapshots.
|
||||
|
||||
`PMEM-WP-0013` moved the score from 4.2 to 4.3 by adding:
|
||||
|
||||
- credential-gated adapter drill helpers and skipped smoke tests that list
|
||||
required environment variables;
|
||||
- stdlib `phase-memory-service` packaging with check mode and WSGI dispatch;
|
||||
- operator readiness runbook for service startup, migrations, audit retention,
|
||||
credentialed drills, and rollback;
|
||||
- audit retention apply behavior with audit trace coverage;
|
||||
- evaluation trend artifacts with threshold and regression deltas;
|
||||
- release-note template gating for public API snapshot changes.
|
||||
|
||||
## Recommended Next Refinement
|
||||
|
||||
Create and execute `PMEM-WP-0013`: credentialed adapter drills and deployment
|
||||
packaging.
|
||||
Create and execute `PMEM-WP-0014`: live credential execution and managed
|
||||
deployment hardening.
|
||||
|
||||
Highest-value tasks:
|
||||
|
||||
- Add optional credentialed Markitect/Kontextual adapter smoke drills that are
|
||||
skipped unless credentials are present.
|
||||
- Package the service binding as a deployable local service with operator
|
||||
readiness checks.
|
||||
- Add audit retention pruning and telemetry export enforcement.
|
||||
- Grow evaluation reporting into historical threshold trends.
|
||||
- Add release note and migration-note templates for compatibility changes.
|
||||
- Run the credential-gated drills against real Markitect/Kontextual endpoints
|
||||
in an operator environment.
|
||||
- Add managed deployment packaging and readiness probes.
|
||||
- Persist evaluation trend reports across runs.
|
||||
- Add credentialed telemetry export and retention pruning drills.
|
||||
- Expand troubleshooting from actual operator feedback.
|
||||
|
||||
## Score Movement Gates
|
||||
|
||||
@@ -121,10 +131,11 @@ Achieved overall score **4.0** when:
|
||||
- Local persistence has migration diagnostics.
|
||||
- Evaluation fixtures cover at least three profile/graph families.
|
||||
|
||||
Move overall score to **4.3+** when:
|
||||
Achieved overall score **4.3+** when:
|
||||
|
||||
- Credentialed optional Markitect or Kontextual adapter smoke drills run behind
|
||||
the same conformance suite as the fake/live-shaped packs.
|
||||
- Credentialed optional Markitect or Kontextual adapter smoke drills are
|
||||
available behind the same conformance suite as the fake/live-shaped packs and
|
||||
skip cleanly without credentials.
|
||||
- Operational docs include deployable service packaging and an operator
|
||||
readiness runbook.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user