can-you-assist/history/2026-05-28-CYA-Agentic-Memory-Research-Variations.md

Agentic Memory & Self-Improving Loops: Research Synthesis and Three Variations for can-you-assist

Date: 2026-05-28
Context: Post CYA-WP-0004 (packaging), during planning for memory profile evolution (CYA-WP-0005)
Related: MemoryVision.md, docs/cya-memory-activation-and-retrospection-concept.md, CYA-WP-0002/0003, history/2026-05-28-CYA-Intent-Scope-Gap-Analysis-Post-0004.md, phase-memory (sister repo)
Author: Grok (research + synthesis for cya)

Executive Summary

Deep research into agentic memory architectures for LLM agents reveals mature patterns for closed self-improving loops: Trial/Interaction → Evaluation/Feedback (success/failure signals, user outcomes) → Reflection/Synthesis/Critique (verbal or structured) → Memory Update/Consolidation (store abstractions, rules, or critiques) → Improved future behavior via retrieval/activation.

Key exemplars:

• Reflexion (Shinn et al., NeurIPS 2023): Verbal reinforcement via stored self-reflections. Dramatic gains with minimal infra.
• Generative Agents (Park et al., SIGGRAPH 2023): Memory stream + LLM-powered reflection module synthesizing abstractions + multi-factor scored retrieval.
• Procedural / Meta-policy evolution (LangMem, A-Mem, LEGOMem patterns): Agents dynamically rewrite their own high-level rules, procedures, or system instructions based on experience.

These map cleanly to cya's existing foundation (post-0003: real JSON memory with kinds, activation_context, retrospection outcomes as higher-order memory, directory/project scoping, provenance, safety integration, cya retrospect guided loop) and the phase-memory vision (phases: ephemeral/fluid/stabilized/rigid; profile-driven planners; explicit ports).

Recommendation for cya: Establish current (post-0003 local JSON + activation + retrospection) as explicit Profile 0 baseline. Then implement Profiles 1–3 incrementally, each adding one dimension of agentic self-improvement while preserving non-negotiables: user control/inspectability, explainability (provenance + --explain-context), rule-based safety (memory never downgrades risk or bypasses confirmation), and the cya/phase-memory boundary (cya consumes; phase-memory owns storage/lifecycle/planners).

This document persists the research and directly feeds CYA-WP-0005 (profile definitions + phase-memory interface feedback).

Research Sources & Core Patterns

Reflexion (Shinn et al. 2023)

• Core idea: After each trial (e.g., code gen + test feedback, or assistant suggestion + user outcome), the agent generates a natural-language self-reflection ("what went wrong / what should I remember for next time?") using the LLM itself.
• Store these reflections in an episodic memory buffer (list of (trajectory, reflection, outcome)).
• On next similar task: prepend the most relevant past reflections + trajectories to the prompt.
• Feedback sources flexible: unit tests, environment rewards, human critique, or (in cya) explicit user "accept / revise / reject" + retrospection.
• Results (paper): +11–34% absolute gains on HumanEval / HotpotQA / AlfWorld with GPT-4 / Claude-1; works with open models too.
• Key properties: lightweight (no fine-tuning, no embeddings required for MVP), highly explainable (reflections are human-readable English), verbal reinforcement learning.
• Limitations: Can accumulate noise if reflections are low-quality; needs good relevance filtering or recency/importance scoring.

Primary citation: Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning", https://arxiv.org/abs/2303.11366 (NeurIPS 2023). Additional analysis: Lilian Weng "LLM Powered Autonomous Agents" (https://lilianweng.github.io/posts/2023-06-23-agent/), LangChain Reflexion tutorial.

Generative Agents (Park et al. 2023)

• Memory stream: All experiences stored as natural-language "memory objects" (timestamped observations, actions, thoughts).
• Reflection module (periodic, often at night or low-load): LLM is prompted to generate questions about recent memories, then synthesize high-level abstractions (e.g., "I prefer concise git status output and always want alternatives listed") with citations back to source memories. Recursive: reflections can themselves be reflected upon → hierarchy.
• Retrieval: For a new situation, score every memory by a composite:
  • Recency (exponential decay)
  • Importance (LLM infers 1–10 saliency for the agent)
  • Relevance (embedding cosine similarity to current context/query)
• Retrieved memories + top reflections fed into planning / action.
• Used to drive believable agents in a Sims-like sandbox (25 agents, 2 weeks simulated).
• Strengths: Produces rich, human-interpretable higher-order knowledge; natural hierarchy aligns with phase-memory's fluid→stabilized→rigid phases.
• Cost: More LLM calls for synthesis; needs embedding store or smart sampling for scale.

Primary citation: Park et al., "Generative Agents: Interactive Simulacra of Human Behavior", https://arxiv.org/abs/2304.03442 (SIGGRAPH 2023). Excellent overview in Lilian Weng post above.

Procedural Memory & Meta-Policy Evolution (LangMem, A-Mem, LEGOMem, MemGPT patterns)

• Procedural memory: Not "what happened" or "what I prefer", but "how I should behave / what procedure to follow".
  • LangMem (LangChain): Three stores — episodic (events), semantic (facts), procedural (evolving instructions / system prompts / rules the agent rewrites for itself).
  • Agents observe outcomes, run "meta-reflection", and propose patches to their own high-level policy or few-shot examples.
• A-Mem (Agentic Memory): Dynamic graph where memories are nodes with auto-generated keywords/tags/context; agents autonomously evolve the graph structure and connections over time. No fixed schema.
• LEGOMem: Procedural memory specifically for reusable workflows / "recipes" (e.g., "when debugging a failing test in this project: run X, inspect Y, propose minimal patch").
• MemGPT: OS-inspired virtual context management — "main context" (RAM), recall (disk), archival (cold store); paging + function calls to manage what is in the LLM window. Hierarchical + procedural control.
• Common thread: The agent treats its own operating policy / instructions / strategies as first-class mutable memory that improves via experience.
• Highest upside for self-improvement (the system literally gets "smarter at being an assistant" rather than just knowing more facts).
• Highest risk: Uncontrolled policy drift, safety regressions, user surprise ("why did it start doing X differently?").

Key references:

• LangMem: https://blog.langchain.dev/langmem-building-production-ready-agents-with-long-term-memory/ and docs.
• A-Mem: https://arxiv.org/abs/2409.16166 (autonomous memory evolution).
• MemGPT: https://arxiv.org/abs/2310.08560.
• Lilian Weng survey (covers procedural aspects).

Cross-Cutting Architectural Patterns

• Episodic stream (time-stamped records) scored by recency/importance/relevance + activation context (directory, project, task class).
• Reflection / synthesis as a distinct, often offline or user-triggered step (natural fit for cya retrospect).
• Hierarchical / phased storage: raw events → summaries → abstractions → rigid core (aligns perfectly with phase-memory phases).
• Procedural layer on top: meta-rules that govern retrieval, synthesis thresholds, safety posture, explanation style, etc.
• Provenance + audit everywhere: every synthesized item cites sources; every activation explains why.
• Dry-run / proposal semantics for any memory evolution (user veto before commit).
• Safety firewall: memory-derived signals can only increase caution (cya RiskClassifier invariant).

Three Variations for cya + phase-memory

These are ordered by increasing agentic power and implementation cost. All preserve cya's core contract (user-controlled, explainable, safety-first, explicit seam to phase-memory).

Variation 1: Reflexion-Style Verbal Self-Improvement Loop (Profile 1 candidate)

Core loop:

1. Normal assistance (memory activated via current 0003 activation_context + kinds).
2. Outcome capture: user accepts/revises/rejects, or runs cya retrospect.
3. Verbal reflection: LLM (or guided prompt) produces 1–3 concise "lessons" in natural language, e.g. "In this Rust project the user always wants cargo clippy before suggesting fixes; remember to surface that early."
4. Store via remember_retrospection_outcome (or new remember_reflection) with kind="reflection" or kind="verbal_lesson".
5. Future recall: boost kinds=["reflection", "retrospection", "interaction_goal"] + activation_context match; prepend top-N to context envelope (high salience).
6. Explainability: surface "3 verbal reflections from prior sessions in this project influenced this suggestion" + the actual text in --explain-context.

cya mapping (builds directly on 0003):

• Existing KIND_RETROSPECTION + remember_retrospection_outcome + cya retrospect flow is ~80% of the infrastructure.
• Add a lightweight "capture lesson" step at end of retrospect (or auto after notable interactions).
• New kind values + filter support already in ports.
• Preferential activation for reflection kinds (small change to current boost logic).
• Safety: reflections that touch risky patterns still route through RiskClassifier.

Pros: Lowest infra cost, maximum explainability (plain English), immediate value, fits terminal workflow perfectly. Cons: Reflections can be noisy or contradictory without synthesis (addressed in Var 2). Phase-memory fit: Minimal new requirements — just good support for kind filtering + provenance on recall. Later: a "reflection planner" that can suggest compaction of duplicate lessons.

See also: Shinn paper, current docs/cya-memory-activation-and-retrospection-concept.md (retrospection outputs as higher-order memory).

Variation 2: Generative-Agents-Style Hierarchical Memory Stream + Synthesis (Profile 2 candidate)

Core loop:

1. Episodic capture: every assistance turn, outcome, retrospection, explicit remember → structured memory record (timestamp, kind, scope, activation_context, raw text or structured payload, provenance).
2. Periodic synthesis (user-triggered via cya retrospect --synthesize, or background/low-load, or phase-memory planner): LLM clusters recent fluid memories for the scope/profile, generates abstractions (preferences, project conventions, recurring workflows, "in this repo we always..."), with citations.
3. Store synthesized items with higher phase hint ("stabilized") and new kinds (e.g. KIND_SYNTHESIZED_CONVENTION, KIND_PROJECT_PATTERN).
4. Retrieval/activation: multi-factor scoring (recency + explicit importance + relevance to current cwd/git/task_class + profile match) + the existing activation_context boost.
5. Hierarchy: raw retrospection → synthesized convention → (later) higher-order "this user values X across all projects".
6. Compaction: phase-memory (or cya helper) can propose merging/evicting with dry-run + user review.

cya mapping:

• Current local JSON can serve as the episodic stream (already has ts, scope, kind, profile, value).
• export_memory + recall_preferences(..., activation_context=...) already return provenance.
• Orchestrator already injects activated memory into ContextEnvelope.
• Extend retrospect flow to offer "synthesize patterns from recent sessions" as an explicit user choice.
• Add simple importance scoring (LLM call or heuristic) on remember.
• Wire synthesized items back through the same recall path (they just have different kinds/phases).

Pros: Richer longitudinal value, directly realizes MemoryVision "Project / Directory Memory" + "Workflow Recipes", natural fit for phase-memory's phases and planners. Cons: More LLM calls (synthesis cost); requires good citation tracking to keep explainability. Phase-memory fit: Strong alignment. phase-memory should expose:

• Synthesis / reflection planner entrypoint (or allow cya to request "run reflection pass for this profile/scope").
• Structured memory objects with citation/provenance fields.
• Phase transition proposals (fluid → stabilized) with dry-run.
• Multi-factor retrieval API that cya can parameterize (recency weight, importance weight, relevance query).

See also: Park paper, MemoryVision phases + "profile-driven behavior" section, 0003 activation layers.

Variation 3: Procedural / Meta-Policy Evolution (Profile 3 candidate — aspirational)

Core loop:

1. Base behavior defined by a small set of first-class procedural memory items (kinds: procedural_rule, meta_policy, explanation_strategy, safety_tuning).
   • Example: "When user is in a Rust project with recent compilation errors, always surface cargo check output early and propose minimal patches before broad refactors."
2. Meta-reflection (triggered after retrospection, or on explicit "improve my assistant rules"): LLM reviews recent outcomes + current procedural rules + safety incidents, proposes patches or new rules.
3. Proposal + audit: phase-memory (or cya) presents the diff ("+1 procedural rule, -0, changed confidence on 2") in human review form; user can edit/approve/veto.
4. On approval: the new rule is stored with high phase/stability and becomes part of future activation / prompt construction / risk hints.
5. Guardrails: all procedural changes are additive or tightening only by default (never relax safety without explicit user action); every rule carries provenance + last-review date; cya RiskClassifier treats procedural memory as a strong "force confirmation" signal source.

cya mapping:

• New dedicated kinds + a remember_procedural_rule helper (thin wrapper).
• A new cya improve-rules or extension to retrospect that runs the meta-reflection.
• Orchestrator / safety layer reads procedural items with highest priority for injection (system prompt augmentation or risk context).
• Export/inspect must make procedural layer very prominent ("these 7 rules govern how I behave for you in this project").
• Strong integration with the existing rule-based RiskClassifier (procedural memory can only raise the assessed risk level).

Pros: Highest self-improvement leverage — the assistant literally gets better at being an assistant for this user. Cons: Highest risk of drift or unintended behavior; requires the strongest audit, veto, and rollback UX. Phase-memory fit (most demanding):

• First-class procedural memory kind + dedicated planner (policy evolution planner).
• Proposal / diff semantics for memory changes (dry-run + structured review objects).
• Explicit hooks for "meta-reflection" passes that cya can invoke with bounded context.
• Safety / policy gateway that can enforce "procedural changes may not weaken risk posture without user override".
• Versioning / rollback for the procedural layer itself.
• Profile can declare "aggressiveness of self-evolution" (conservative / balanced / bold) with corresponding planner behavior.

See also: LangMem procedural store, A-Mem autonomous evolution, LEGOMem workflow recipes, safety invariants in cya's classifier.py (T04 0002).

Profile 0 Baseline (Current Post-0003 Reality)

Before defining 1–3, the workplan must explicitly document and stabilize Profile 0 as the shipped baseline so future profiles have a clear "from here" point.

Exact current implementation (as of post-0003 + 0004):

• Backing: user-controlled ~/.config/cya/memory/<scope>.json (list of records). Inspectable, editable, deletable by user. No hidden state.
• Ports (src/cya/memory/init.py):
  • KIND_PREFERENCE, KIND_RETROSPECTION, KIND_INTERACTION_GOAL
  • remember_preference(key, value, scope="cwd", *, profile=None, ttl=None, kind=KIND_PREFERENCE)
  • recall_preferences(scope="cwd", task_class=None, *, kinds=None, profile=None, limit=50, activation_context=None) → returns items + rich provenance + phase hint + note about local json
  • forget(scope, keys=None, *, profile=None)
  • export_memory(scope, *, profile=None, kinds=None) → includes by_kind counts, provenance_summary, phases list
  • remember_retrospection_outcome(...) convenience (sets kind automatically)
• Activation (0003): simple but effective boost — items whose scope/profile match activation_context (populated by ContextEnvelope with cwd + git_root) are moved to front of results. Wired in orchestrator.py for every assistance request + --explain-context panel.
• Retrospection (0003): cya retrospect subcommand (guided questions, review recent memory usage, capture goals → stored as first-class retrospection records with preferential future activation).
• Safety integration (unchanged invariant): memory signals only append rationale or force confirmation in RiskClassifier; never downgrade risk level or bypass mandatory confirmation.
• Observability: --explain-context surfaces activated memory items, provenance, phase, activation reason.
• Explainability: every recall/export carries "source: cya-local-memory-json (T02+0003; activation + retrospection support)".
• Limitations (intentional): no embeddings, no synthesis, no procedural layer, no phase transitions, local only (no graph/event store), no profile execution planner yet.

This is deliberately a high-quality local approximation that delivers real user value today (contextual activation + continuous optimization loop via retrospect) while keeping the seam (profile/kind/activation_context parameters) ready for replacement by full phase-memory.

Profile 0 success criteria (for 0005 T0x): The above is clearly documented in MemoryVision.md (add "What CYA-WP-0003 Delivered" + "Profile 0 Baseline" sections), tests cover kinds + activation_context + retrospection roundtrips, README/AGENTS mention the current loop, and the new workplan treats it as the stable starting point for 1–3.

Recommendations & Next Steps (for CYA-WP-0005)

1. Persist this research (this document) and reference it from the workplan.
2. Create CYA-WP-0005 with tasks for:
   • Formalize Profile 0 baseline (update MemoryVision, SCOPE, code comments, tests).
   • Define + implement Profile 1 (Reflexion verbal) as small delta on existing retrospect/kinds machinery.
   • Profile 2 skeleton (episodic capture + synthesis entrypoint + hierarchical kinds).
   • Profile 3 exploration spike (procedural kind + proposal UX, strong guardrails).
   • Standalone or section "Optimization Suggestions & Missing Functionality for phase-memory" (see below).
   • Update docs, examples, MemoryVision with profile matrix.
   • Register workplan, commit, sync via fix-consistency.
3. Do not start full implementation of 1–3 until the workplan is reviewed and activated (per AGENTS convention).

Optimization Suggestions & Missing Functionality for phase-memory (Sister Repo)

(This section is the direct input for the "provide optimization suggestions to the sister repo" part of the request. It will be extracted or expanded as a deliverable in 0005.)

High-priority interface / contract improvements (to make Profiles 1–3 natural):

1. Refined / extended port signatures (build on the CYA-WP-0002 T01 contract already in MemoryVision):

   • Add activation_context: dict (cwd, git_root, task_class, recent_kinds) as first-class param to recall (already prototyped in cya local).
   • Support kinds: list[str] filter/boost in recall (already in cya; make phase-memory native).
   • Return richer provenance + dry_run_plan + phase + synthesis_citations in recall/export results.
   • Add remember_event(...) or ingest_retrospection_outcome(...) as distinct from generic remember (for planner input).

2. Synthesis / Reflection planner hooks:

   • Expose (or allow cya to request) a "run_reflection_pass(profile, scope, recent_memories)" that returns proposed abstractions with citations and confidence.
   • Dry-run semantics for all synthesis proposals (cya can present "phase-memory proposes these 4 stabilized conventions — review?").

3. First-class procedural / meta-policy support (for Profile 3):

   • Dedicated kind values or a separate procedural_rules collection.
   • Policy evolution planner: input = recent outcomes + current rules; output = proposed patches + impact analysis (esp. safety impact).
   • Guardrail primitives: "this change may not relax risk posture" that the PolicyGateway can enforce.

4. Activation & retrieval enhancements:

   • Multi-factor scoring API (recency, importance, relevance, user_pinned, phase_weight) that cya can parameterize per request or per profile.
   • Profile-aware activation requests: cya says "give me a context package for a terminal assistance turn in this cwd under profile X, budget 4k tokens, prioritize retrospection + procedural kinds".

5. Lifecycle & phase management:

   • Explicit phase transition proposals (fluid → stabilized) with user-reviewable diffs.
   • Compaction / eviction planner with dry-run + rationale.
   • TTL + review_date on memory items (cya can surface "this convention has not been reviewed in 90 days").

6. Explainability & audit:

   • Every returned memory item (raw or synthesized) must carry machine + human readable provenance (source memories, reflection prompt id, timestamp, author).
   • Structured "memory influence" objects that cya can render cleanly in --explain-context and final output without custom parsing.

7. Safety & policy integration:

   • Memory evolution operations must be able to declare "safety impact: none / tightens / potentially relaxes".
   • Hook for cya's rule-based RiskClassifier to consult memory policy before activation (or to inject forced-confirmation signals from procedural rules).
   • User-level "memory evolution veto mode" (all proposals require explicit approval; no auto-stabilization).

8. Observability & introspection:

   • export_memory / profile introspection that includes "active procedural rules", "recent synthesis activity", "token budget usage by phase".
   • Event log query API so cya can show "memory events that led to this activated preference".

9. Retrospection / continuous optimization interop:

   • Standard event schema for "retrospection session completed" that phase-memory planners can consume directly.
   • Ability for cya to say "this retrospection outcome should be treated as high-importance for stabilization".

Non-goals for near-term phase-memory feedback: full embedding/semantic index details, voice, cross-user sharing, production scale numbers. Focus on the profile + planner + procedural + dry-run proposal surface that lets cya deliver the three self-improving variations safely and explainably.

These suggestions will be turned into a clean, reviewable artifact (standalone doc or PR-ready section) in CYA-WP-0005 and shared with the phase-memory / markitect owners via State Hub or direct coordination.

---

End of persisted research document.
This artifact, together with the three gap analyses in history/, MemoryVision.md, and the activation concept doc, forms the knowledge base for CYA-WP-0005.

Next: Create the workplan that turns this research into executable tasks (profile 0 baseline + 1–3 + phase-memory feedback deliverable).