can-you-assist/docs/phase-memory-optimization-suggestions.md

Optimization Suggestions & Missing Functionality for phase-memory

Date: 2026-05-28 (finalized in CYA-WP-0005 T04)
Source: CYA-WP-0005 (Agentic Memory Profiles) + persisted research history/2026-05-28-CYA-Agentic-Memory-Research-Variations.md
Audience: phase-memory / markitect owners (sister repo)
Purpose: Actionable, prioritized feedback from the cya consumer so that Profiles 0–3 (and the self-improving loops they enable) can be realized cleanly, safely, and with excellent user control / explainability.

Updated after T03: Cross-references added to the crisp profile definitions and Capability Matrix now living in MemoryVision.md. The 9 categories below remain the concrete interface asks needed to enable the three variations.

Context (Why This Feedback Exists)

can-you-assist (cya) is a consumer of the phase-memory profile-driven memory layer. After CYA-WP-0002/0003 we have a high-quality local approximation (Profile 0) behind explicit ports that already uses profile, kinds, activation_context, retrospection outcomes as higher-order memory, provenance, and phase hints. This delivers real directory/project-bound activation and a cya retrospect continuous-optimization loop today.

To evolve to true agentic self-improving behavior we need three progressive profiles (defined in the research doc and CYA-WP-0005):

• Profile 1 (Reflexion-style verbal): Store and preferentially activate natural-language self-reflections / lessons (builds directly on existing retrospection kinds and remember_retrospection_outcome).
• Profile 2 (Generative-Agents-style hierarchical): Episodic stream → LLM synthesis of abstractions (conventions, patterns) with citations → storage in stabilized phases + multi-factor retrieval.
• Profile 3 (Procedural / meta-policy): First-class evolvable "how I behave" rules/procedures; meta-reflection proposes changes under strong audit + safety guardrails.

Profile 0 is already shipped (local JSON + activation + retrospection). The seam (4 ports + kinds + activation_context + provenance) is deliberately stable. The suggestions below are the concrete extensions we need from phase-memory to make Profiles 1–3 natural, safe, and explainable without cya owning storage or lifecycle policy.

All suggestions respect cya invariants:

• Memory may only increase caution (rule-based RiskClassifier in cya is primary; memory feeds it but never relaxes it).
• Everything is user-inspectable, exportable, and veto-able.
• Full provenance for every item (raw or synthesized).
• Dry-run / proposal semantics for any memory evolution.
• cya never bypasses explicit user confirmation for non-SAFE actions.

Prioritized Suggestions (9 Categories)

See the "Profiles 1–3: Definitions and cya Integration Plans" section + Capability Matrix in MemoryVision.md for the detailed intent and cya mappings that these interface asks are designed to enable.

1. Refined Port Signatures & Activation Context (Must-have for Profile 1, baseline for 2/3)

• Make activation_context: dict[str, Any] (cwd, git_root, task_class, recent_kinds, token_budget_hint) a first-class, documented parameter on recall_preferences / context-package requests (cya already prototypes this locally).
• Support kinds: list[str] filter + boost natively (cya has KIND_PREFERENCE | RETROSPECTION | INTERACTION_GOAL; will add REFLECTION, SYNTHESIZED_CONVENTION, PROCEDURAL_RULE, etc.).
• Return structure from recall/export must include: items[] with per-item provenance + phase + citations (for synthesized), provenance[] array, dry_run_plan (when relevant), profile, phase_summary.
• Add a distinct remember_event / ingest_retrospection_outcome (or keep thin wrapper) so planners receive high-quality, typed input from cya's guided reflection sessions.

Why: Enables the activation boost and kind-based preferential treatment that 0003 already proved valuable, and that all three profiles rely on.

2. Synthesis / Reflection Planner Hooks (Required for Profile 2, nice for 1)

• Expose (or allow cya to request) a "run_reflection_pass(profile, scope, recent_memories, budget)" that returns proposed abstractions with citations, confidence, and suggested phase (fluid/stabilized).
• Dry-run first: every synthesis returns a proposal object (before any write) that cya can present to the user for review/edit/approve ("phase-memory proposes 4 stabilized conventions from your last 12 sessions — accept?").
• Recursive / hierarchical synthesis support (reflections on reflections) with citation chains.

Why: Directly implements the Generative-Agents reflection module and the "retrospection outputs as higher-order memory" model from 0003.

3. First-Class Procedural / Meta-Policy Support (Required for Profile 3)

• Dedicated collection or kind namespace for procedural items (procedural_rule, meta_policy, explanation_strategy, safety_tuning).
• Policy-evolution planner: input = recent outcomes + current procedural set + safety incidents; output = structured patch proposal + safety-impact analysis ("this change tightens risk posture on destructive commands").
• Guardrail primitives that the PolicyGateway can enforce: "procedural changes may not relax risk classification without explicit user + operator override."

Why: Highest-leverage self-improvement layer (the assistant gets better at being an assistant). Highest risk if uncontrolled — needs the strongest dry-run + veto UX.

4. Activation & Multi-Factor Retrieval (Strong for 1 & 2, essential for 3)

• Configurable composite scoring for retrieval: recency (decay), importance (LLM-inferred or user-pinned), relevance (to current activation_context or query), phase_weight, kind_boost.
• Profile can declare defaults ("for terminal assistance in Rust projects, weight retrospection + procedural at 2.0").
• cya can request "context package for a console turn, max 4k tokens, prioritize X kinds, under profile Y" and receive a planned package + explanation.

Why: Replaces cya's current simple boost logic with something phase-memory can optimize consistently across consumers.

5. Lifecycle, Phase Transitions & Compaction (Baseline for all profiles)

• Explicit phase-transition proposals (fluid → stabilized → rigid) with human-readable diffs and citation back to source memories.
• Compaction / eviction planner with dry-run + rationale ("these 17 near-duplicate reflections can be merged into 3; confidence 0.85").
• TTL, review_date, last_used on items; cya can surface "this convention has not been reviewed in 90 days — re-validate in retrospect?"

Why: Realizes the ephemeral/fluid/stabilized/rigid model in MemoryVision and prevents unbounded growth while keeping user in control.

6. Explainability & Audit (Non-negotiable for all profiles)

• Every memory item (raw, synthesized, or procedural) carries machine-readable + human-readable provenance: source memories, reflection prompt id or planner id, timestamps, author (user vs system), last review.
• Structured "memory influence" objects that cya can render directly in --explain-context and final responses without fragile parsing.
• Full export of a profile/scope includes the procedural rule set, recent synthesis activity, and token budget accounting by phase.

Why: Matches cya's core promise ("I can see exactly what it knows and why it used it") and is required for user trust when the system starts synthesizing and evolving its own behavior.

7. Safety & Policy Integration (Critical for 2 & 3)

• Memory evolution operations declare safety impact: none / tightens / potentially relaxes.
• Hook or query so cya's rule-based RiskClassifier can consult memory policy / procedural rules before activation (or to inject forced-confirmation signals).
• "Memory evolution veto mode" (per-profile or global): all proposals require explicit approval; no auto-stabilization or auto-procedural changes.
• Rollback / versioning for the procedural layer itself (user can say "revert to the rules from 2026-05-20").

Why: Preserves the hard invariant that memory never weakens safety. Procedural evolution is powerful but must never become a covert policy bypass.

8. Observability & Introspection (Strongly desired for 1–3)

• Profile/scope introspection endpoint that returns: active procedural rules (with provenance), recent synthesis activity + citations, token usage by phase/kind, items nearing review or compaction.
• Event log query so cya can answer "show me the memory events that caused this preference to be activated."
• Cost / token / LLM-call accounting surfaced per recall or synthesis request (helps users understand the price of richer memory).

Why: Makes the self-improving loop transparent and debuggable from the terminal.

9. Retrospection / Continuous Optimization Interop (Direct enabler for 1 + foundation for 2/3)

• Standard event schema or ingestion path for "retrospection session completed" (with captured goals, interaction_goals, reflections, explicit stabilization decisions) that phase-memory planners consume natively.
• cya can tag a retrospection outcome as "high-importance for stabilization" or "candidate for procedural rule extraction."
• Planners can propose "based on 3 recent retrospections in this project, I recommend promoting these 2 patterns to stabilized + adding 1 procedural rule — review?"

Why: cya retrospect (0003) is already the highest-quality way new, user-vetted memory enters the system. phase-memory should treat it as privileged input rather than generic events.

Handoff Notes for phase-memory / markitect

• Reference artifacts (please read in this order):

  1. MemoryVision.md (especially the 0002 T01 integration contract with refined port signatures, phases, primary kinds, profile-driven behavior examples, safety/explainability requirements).
  2. history/2026-05-28-CYA-Agentic-Memory-Research-Variations.md (full research + the three profile definitions + cya-specific mappings).
  3. docs/cya-memory-activation-and-retrospection-concept.md (0003 T01 — the activation layers + retrospection-as-higher-order-memory model that Profiles 1–3 build on).
  4. CYA-WP-0005 (this workplan) — tasks T01–T04 are the direct consumers of the feedback.
  5. Current cya port implementation: src/cya/memory/__init__.py (Profile 0; the signatures and return shapes we want to keep stable or evolve in place).

• Coordination channel: State Hub (topic 64418556-3206-457a-ba29-6884b5b12cf3 for can-you-assist / capabilities domain). Workstream for CYA-WP-0005 will be registered. Progress events and decisions will be posted there. Direct messages or shared docs also fine.

• Prioritization guidance:

  • Items in categories 1, 6, 7 are foundational — they protect user control, explainability, and safety even for Profile 0 today.
  • Categories 2, 4, 5, 9 enable Profile 1 + 2 with high leverage.
  • Category 3 + strong guardrails in 7 are the big unlock (and biggest risk) for Profile 3.
  • Observability (8) makes the whole story usable and debuggable.

• Non-requests (for clarity): We are not asking phase-memory to own the cya CLI, the rule-based risk classifier, or terminal-specific prompting. We want the memory operating layer (profiles, planners, phases, graph/event store, policy) to be excellent so cya can be a thin, safe, highly-explainable consumer.

Success Criteria (from cya perspective)

phase-memory + these suggestions are successful when a cya user can say:

• "I ran cya retrospect, captured three lessons, and now the assistant reliably surfaces the right one in this project — and I can see exactly why."
• "phase-memory proposed stabilizing four patterns from my last month of work; I reviewed the citations and accepted two. The assistant now behaves differently and I know why."
• "I defined a procedural rule for how I like Rust error handling explained. The assistant follows it, and when it suggested changing the rule it showed me the safety impact and asked for approval."
• "I exported my memory for project X, saw the procedural layer and the synthesis history, and deleted two items. The assistant respected it immediately."

These map directly to the long-term success criteria already in MemoryVision.md.

---

Document status: Living feedback artifact. Will be updated in CYA-WP-0005 T04 (polish + extraction) and as we learn more during profile implementation spikes. Please treat the numbered categories above as the actionable input for planning and interface design in phase-memory.

Contact / next step: After CYA-WP-0005 T04/T08 complete (workplan registered + suggestions doc finalized), the cya side will open a State Hub coordination message or workstream handoff with this doc + the research + MemoryVision contract attached.

Thank you — the profile model and self-improving loops are only as good as the memory operating system they run on. We are excited to collaborate on making phase-memory the excellent, user-controlled foundation for the whole ecosystem.