can-you-assist/history/2026-05-28-CYA-Agentic-Memory-Research-Variations.md

# Agentic Memory & Self-Improving Loops: Research Synthesis and Three Variations for can-you-assist

**Date:** 2026-05-28  
**Context:** Post CYA-WP-0004 (packaging), during planning for memory profile evolution (CYA-WP-0005)  
**Related:** MemoryVision.md, docs/cya-memory-activation-and-retrospection-concept.md, CYA-WP-0002/0003, history/2026-05-28-CYA-Intent-Scope-Gap-Analysis-Post-0004.md, phase-memory (sister repo)  
**Author:** Grok (research + synthesis for cya)

## Executive Summary

Deep research into agentic memory architectures for LLM agents reveals mature patterns for **closed self-improving loops**: Trial/Interaction → Evaluation/Feedback (success/failure signals, user outcomes) → Reflection/Synthesis/Critique (verbal or structured) → Memory Update/Consolidation (store abstractions, rules, or critiques) → Improved future behavior via retrieval/activation.

Key exemplars:
- **Reflexion (Shinn et al., NeurIPS 2023)**: Verbal reinforcement via stored self-reflections. Dramatic gains with minimal infra.
- **Generative Agents (Park et al., SIGGRAPH 2023)**: Memory stream + LLM-powered reflection module synthesizing abstractions + multi-factor scored retrieval.
- **Procedural / Meta-policy evolution** (LangMem, A-Mem, LEGOMem patterns): Agents dynamically rewrite their own high-level rules, procedures, or system instructions based on experience.

These map cleanly to cya's existing foundation (post-0003: real JSON memory with `kinds`, `activation_context`, retrospection outcomes as higher-order memory, directory/project scoping, provenance, safety integration, `cya retrospect` guided loop) and the phase-memory vision (phases: ephemeral/fluid/stabilized/rigid; profile-driven planners; explicit ports).

**Recommendation for cya**: Establish current (post-0003 local JSON + activation + retrospection) as explicit **Profile 0** baseline. Then implement **Profiles 1–3** incrementally, each adding one dimension of agentic self-improvement while preserving non-negotiables: user control/inspectability, explainability (provenance + `--explain-context`), rule-based safety (memory never downgrades risk or bypasses confirmation), and the cya/phase-memory boundary (cya consumes; phase-memory owns storage/lifecycle/planners).

This document persists the research and directly feeds CYA-WP-0005 (profile definitions + phase-memory interface feedback).

## Research Sources & Core Patterns

### Reflexion (Shinn et al. 2023)
- Core idea: After each trial (e.g., code gen + test feedback, or assistant suggestion + user outcome), the agent generates a **natural-language self-reflection** ("what went wrong / what should I remember for next time?") using the LLM itself.
- Store these reflections in an **episodic memory buffer** (list of (trajectory, reflection, outcome)).
- On next similar task: prepend the most relevant past reflections + trajectories to the prompt.
- Feedback sources flexible: unit tests, environment rewards, human critique, or (in cya) explicit user "accept / revise / reject" + retrospection.
- Results (paper): +11–34% absolute gains on HumanEval / HotpotQA / AlfWorld with GPT-4 / Claude-1; works with open models too.
- Key properties: **lightweight** (no fine-tuning, no embeddings required for MVP), **highly explainable** (reflections are human-readable English), **verbal reinforcement learning**.
- Limitations: Can accumulate noise if reflections are low-quality; needs good relevance filtering or recency/importance scoring.

**Primary citation**: Shinn et al., "Reflexion: Language Agents with Verbal Reinforcement Learning", https://arxiv.org/abs/2303.11366 (NeurIPS 2023). Additional analysis: Lilian Weng "LLM Powered Autonomous Agents" (https://lilianweng.github.io/posts/2023-06-23-agent/), LangChain Reflexion tutorial.

### Generative Agents (Park et al. 2023)
- **Memory stream**: All experiences stored as natural-language "memory objects" (timestamped observations, actions, thoughts).
- **Reflection module** (periodic, often at night or low-load): LLM is prompted to generate questions about recent memories, then synthesize **high-level abstractions** (e.g., "I prefer concise git status output and always want alternatives listed") with citations back to source memories. Recursive: reflections can themselves be reflected upon → hierarchy.
- **Retrieval**: For a new situation, score every memory by a composite:
  - Recency (exponential decay)
  - Importance (LLM infers 1–10 saliency for the agent)
  - Relevance (embedding cosine similarity to current context/query)
- Retrieved memories + top reflections fed into planning / action.
- Used to drive believable agents in a Sims-like sandbox (25 agents, 2 weeks simulated).
- Strengths: Produces **rich, human-interpretable** higher-order knowledge; natural hierarchy aligns with phase-memory's fluid→stabilized→rigid phases.
- Cost: More LLM calls for synthesis; needs embedding store or smart sampling for scale.

**Primary citation**: Park et al., "Generative Agents: Interactive Simulacra of Human Behavior", https://arxiv.org/abs/2304.03442 (SIGGRAPH 2023). Excellent overview in Lilian Weng post above.

### Procedural Memory & Meta-Policy Evolution (LangMem, A-Mem, LEGOMem, MemGPT patterns)
- **Procedural memory**: Not "what happened" or "what I prefer", but **"how I should behave / what procedure to follow"**.
  - LangMem (LangChain): Three stores — episodic (events), semantic (facts), **procedural** (evolving instructions / system prompts / rules the agent rewrites for itself).
  - Agents observe outcomes, run "meta-reflection", and propose patches to their own high-level policy or few-shot examples.
- **A-Mem (Agentic Memory)**: Dynamic graph where memories are nodes with auto-generated keywords/tags/context; agents autonomously evolve the graph structure and connections over time. No fixed schema.
- **LEGOMem**: Procedural memory specifically for reusable workflows / "recipes" (e.g., "when debugging a failing test in this project: run X, inspect Y, propose minimal patch").
- **MemGPT**: OS-inspired virtual context management — "main context" (RAM), recall (disk), archival (cold store); paging + function calls to manage what is in the LLM window. Hierarchical + procedural control.
- Common thread: The agent treats its own **operating policy / instructions / strategies** as first-class mutable memory that improves via experience.
- Highest upside for self-improvement (the system literally gets "smarter at being an assistant" rather than just knowing more facts).
- Highest risk: Uncontrolled policy drift, safety regressions, user surprise ("why did it start doing X differently?").

**Key references**:
- LangMem: https://blog.langchain.dev/langmem-building-production-ready-agents-with-long-term-memory/ and docs.
- A-Mem: https://arxiv.org/abs/2409.16166 (autonomous memory evolution).
- MemGPT: https://arxiv.org/abs/2310.08560.
- Lilian Weng survey (covers procedural aspects).

### Cross-Cutting Architectural Patterns
- **Episodic stream** (time-stamped records) scored by recency/importance/relevance + activation context (directory, project, task class).
- **Reflection / synthesis** as a distinct, often offline or user-triggered step (natural fit for `cya retrospect`).
- **Hierarchical / phased storage**: raw events → summaries → abstractions → rigid core (aligns perfectly with phase-memory phases).
- **Procedural layer** on top: meta-rules that govern retrieval, synthesis thresholds, safety posture, explanation style, etc.
- **Provenance + audit everywhere**: every synthesized item cites sources; every activation explains why.
- **Dry-run / proposal semantics** for any memory evolution (user veto before commit).
- **Safety firewall**: memory-derived signals can only **increase** caution (cya RiskClassifier invariant).

## Three Variations for cya + phase-memory

These are ordered by increasing agentic power and implementation cost. All preserve cya's core contract (user-controlled, explainable, safety-first, explicit seam to phase-memory).

### Variation 1: Reflexion-Style Verbal Self-Improvement Loop (Profile 1 candidate)
**Core loop**:
1. Normal assistance (memory activated via current 0003 activation_context + kinds).
2. Outcome capture: user accepts/revises/rejects, or runs `cya retrospect`.
3. Verbal reflection: LLM (or guided prompt) produces 1–3 concise "lessons" in natural language, e.g. "In this Rust project the user always wants `cargo clippy` before suggesting fixes; remember to surface that early."
4. Store via `remember_retrospection_outcome` (or new `remember_reflection`) with `kind="reflection"` or `kind="verbal_lesson"`.
5. Future recall: boost `kinds=["reflection", "retrospection", "interaction_goal"]` + activation_context match; prepend top-N to context envelope (high salience).
6. Explainability: surface "3 verbal reflections from prior sessions in this project influenced this suggestion" + the actual text in `--explain-context`.

**cya mapping (builds directly on 0003)**:
- Existing `KIND_RETROSPECTION` + `remember_retrospection_outcome` + `cya retrospect` flow is ~80% of the infrastructure.
- Add a lightweight "capture lesson" step at end of retrospect (or auto after notable interactions).
- New `kind` values + filter support already in ports.
- Preferential activation for reflection kinds (small change to current boost logic).
- Safety: reflections that touch risky patterns still route through RiskClassifier.

**Pros**: Lowest infra cost, maximum explainability (plain English), immediate value, fits terminal workflow perfectly.
**Cons**: Reflections can be noisy or contradictory without synthesis (addressed in Var 2).
**Phase-memory fit**: Minimal new requirements — just good support for `kind` filtering + provenance on recall. Later: a "reflection planner" that can suggest compaction of duplicate lessons.

**See also**: Shinn paper, current `docs/cya-memory-activation-and-retrospection-concept.md` (retrospection outputs as higher-order memory).

### Variation 2: Generative-Agents-Style Hierarchical Memory Stream + Synthesis (Profile 2 candidate)
**Core loop**:
1. Episodic capture: every assistance turn, outcome, retrospection, explicit remember → structured memory record (timestamp, kind, scope, activation_context, raw text or structured payload, provenance).
2. Periodic synthesis (user-triggered via `cya retrospect --synthesize`, or background/low-load, or phase-memory planner): LLM clusters recent fluid memories for the scope/profile, generates abstractions (preferences, project conventions, recurring workflows, "in this repo we always..."), with citations.
3. Store synthesized items with higher phase hint ("stabilized") and new kinds (e.g. `KIND_SYNTHESIZED_CONVENTION`, `KIND_PROJECT_PATTERN`).
4. Retrieval/activation: multi-factor scoring (recency + explicit importance + relevance to current cwd/git/task_class + profile match) + the existing activation_context boost.
5. Hierarchy: raw retrospection → synthesized convention → (later) higher-order "this user values X across all projects".
6. Compaction: phase-memory (or cya helper) can propose merging/evicting with dry-run + user review.

**cya mapping**:
- Current local JSON can serve as the episodic stream (already has ts, scope, kind, profile, value).
- `export_memory` + `recall_preferences(..., activation_context=...)` already return provenance.
- Orchestrator already injects activated memory into ContextEnvelope.
- Extend retrospect flow to offer "synthesize patterns from recent sessions" as an explicit user choice.
- Add simple importance scoring (LLM call or heuristic) on remember.
- Wire synthesized items back through the same recall path (they just have different kinds/phases).

**Pros**: Richer longitudinal value, directly realizes MemoryVision "Project / Directory Memory" + "Workflow Recipes", natural fit for phase-memory's phases and planners.
**Cons**: More LLM calls (synthesis cost); requires good citation tracking to keep explainability.
**Phase-memory fit**: Strong alignment. phase-memory should expose:
- Synthesis / reflection planner entrypoint (or allow cya to request "run reflection pass for this profile/scope").
- Structured memory objects with citation/provenance fields.
- Phase transition proposals (fluid → stabilized) with dry-run.
- Multi-factor retrieval API that cya can parameterize (recency weight, importance weight, relevance query).

**See also**: Park paper, MemoryVision phases + "profile-driven behavior" section, 0003 activation layers.

### Variation 3: Procedural / Meta-Policy Evolution (Profile 3 candidate — aspirational)
**Core loop**:
1. Base behavior defined by a small set of **first-class procedural memory items** (kinds: `procedural_rule`, `meta_policy`, `explanation_strategy`, `safety_tuning`).
   - Example: "When user is in a Rust project with recent compilation errors, always surface `cargo check` output early and propose minimal patches before broad refactors."
2. Meta-reflection (triggered after retrospection, or on explicit "improve my assistant rules"): LLM reviews recent outcomes + current procedural rules + safety incidents, proposes **patches or new rules**.
3. Proposal + audit: phase-memory (or cya) presents the diff ("+1 procedural rule, -0, changed confidence on 2") in human review form; user can edit/approve/veto.
4. On approval: the new rule is stored with high phase/stability and becomes part of future activation / prompt construction / risk hints.
5. Guardrails: all procedural changes are **additive or tightening only** by default (never relax safety without explicit user action); every rule carries provenance + last-review date; cya RiskClassifier treats procedural memory as a strong "force confirmation" signal source.

**cya mapping**:
- New dedicated kinds + a `remember_procedural_rule` helper (thin wrapper).
- A new `cya improve-rules` or extension to `retrospect` that runs the meta-reflection.
- Orchestrator / safety layer reads procedural items with highest priority for injection (system prompt augmentation or risk context).
- Export/inspect must make procedural layer very prominent ("these 7 rules govern how I behave for you in this project").
- Strong integration with the existing rule-based RiskClassifier (procedural memory can only raise the assessed risk level).

**Pros**: Highest self-improvement leverage — the assistant literally gets better at *being an assistant* for this user.
**Cons**: Highest risk of drift or unintended behavior; requires the strongest audit, veto, and rollback UX.
**Phase-memory fit** (most demanding):
- First-class **procedural memory** kind + dedicated planner (policy evolution planner).
- Proposal / diff semantics for memory changes (dry-run + structured review objects).
- Explicit hooks for "meta-reflection" passes that cya can invoke with bounded context.
- Safety / policy gateway that can enforce "procedural changes may not weaken risk posture without user override".
- Versioning / rollback for the procedural layer itself.
- Profile can declare "aggressiveness of self-evolution" (conservative / balanced / bold) with corresponding planner behavior.

**See also**: LangMem procedural store, A-Mem autonomous evolution, LEGOMem workflow recipes, safety invariants in cya's classifier.py (T04 0002).

## Profile 0 Baseline (Current Post-0003 Reality)

Before defining 1–3, the workplan must explicitly document and stabilize **Profile 0** as the shipped baseline so future profiles have a clear "from here" point.

**Exact current implementation** (as of post-0003 + 0004):
- Backing: user-controlled `~/.config/cya/memory/<scope>.json` (list of records). Inspectable, editable, deletable by user. No hidden state.
- Ports (src/cya/memory/__init__.py):
  - `KIND_PREFERENCE`, `KIND_RETROSPECTION`, `KIND_INTERACTION_GOAL`
  - `remember_preference(key, value, scope="cwd", *, profile=None, ttl=None, kind=KIND_PREFERENCE)`
  - `recall_preferences(scope="cwd", task_class=None, *, kinds=None, profile=None, limit=50, activation_context=None)` → returns items + rich provenance + phase hint + note about local json
  - `forget(scope, keys=None, *, profile=None)`
  - `export_memory(scope, *, profile=None, kinds=None)` → includes `by_kind` counts, provenance_summary, phases list
  - `remember_retrospection_outcome(...)` convenience (sets kind automatically)
- Activation (0003): simple but effective boost — items whose scope/profile match `activation_context` (populated by ContextEnvelope with cwd + git_root) are moved to front of results. Wired in orchestrator.py for every assistance request + `--explain-context` panel.
- Retrospection (0003): `cya retrospect` subcommand (guided questions, review recent memory usage, capture goals → stored as first-class retrospection records with preferential future activation).
- Safety integration (unchanged invariant): memory signals only append rationale or force confirmation in RiskClassifier; never downgrade risk level or bypass mandatory confirmation.
- Observability: `--explain-context` surfaces activated memory items, provenance, phase, activation reason.
- Explainability: every recall/export carries "source: cya-local-memory-json (T02+0003; activation + retrospection support)".
- Limitations (intentional): no embeddings, no synthesis, no procedural layer, no phase transitions, local only (no graph/event store), no profile execution planner yet.

This is deliberately a **high-quality local approximation** that delivers real user value today (contextual activation + continuous optimization loop via retrospect) while keeping the seam (profile/kind/activation_context parameters) ready for replacement by full phase-memory.

**Profile 0 success criteria** (for 0005 T0x): The above is clearly documented in MemoryVision.md (add "What CYA-WP-0003 Delivered" + "Profile 0 Baseline" sections), tests cover kinds + activation_context + retrospection roundtrips, README/AGENTS mention the current loop, and the new workplan treats it as the stable starting point for 1–3.

## Recommendations & Next Steps (for CYA-WP-0005)

1. **Persist this research** (this document) and reference it from the workplan.
2. Create CYA-WP-0005 with tasks for:
   - Formalize Profile 0 baseline (update MemoryVision, SCOPE, code comments, tests).
   - Define + implement Profile 1 (Reflexion verbal) as small delta on existing retrospect/kinds machinery.
   - Profile 2 skeleton (episodic capture + synthesis entrypoint + hierarchical kinds).
   - Profile 3 exploration spike (procedural kind + proposal UX, strong guardrails).
   - Standalone or section "Optimization Suggestions & Missing Functionality for phase-memory" (see below).
   - Update docs, examples, MemoryVision with profile matrix.
   - Register workplan, commit, sync via fix-consistency.
3. **Do not** start full implementation of 1–3 until the workplan is reviewed and activated (per AGENTS convention).

## Optimization Suggestions & Missing Functionality for phase-memory (Sister Repo)

(This section is the direct input for the "provide optimization suggestions to the sister repo" part of the request. It will be extracted or expanded as a deliverable in 0005.)

**High-priority interface / contract improvements** (to make Profiles 1–3 natural):

1. **Refined / extended port signatures** (build on the CYA-WP-0002 T01 contract already in MemoryVision):
   - Add `activation_context: dict` (cwd, git_root, task_class, recent_kinds) as first-class param to recall (already prototyped in cya local).
   - Support `kinds: list[str]` filter/boost in recall (already in cya; make phase-memory native).
   - Return richer `provenance` + `dry_run_plan` + `phase` + `synthesis_citations` in recall/export results.
   - Add `remember_event(...)` or `ingest_retrospection_outcome(...)` as distinct from generic remember (for planner input).

2. **Synthesis / Reflection planner hooks**:
   - Expose (or allow cya to request) a "run_reflection_pass(profile, scope, recent_memories)" that returns proposed abstractions with citations and confidence.
   - Dry-run semantics for all synthesis proposals (cya can present "phase-memory proposes these 4 stabilized conventions — review?").

3. **First-class procedural / meta-policy support** (for Profile 3):
   - Dedicated `kind` values or a separate `procedural_rules` collection.
   - Policy evolution planner: input = recent outcomes + current rules; output = proposed patches + impact analysis (esp. safety impact).
   - Guardrail primitives: "this change may not relax risk posture" that the PolicyGateway can enforce.

4. **Activation & retrieval enhancements**:
   - Multi-factor scoring API (recency, importance, relevance, user_pinned, phase_weight) that cya can parameterize per request or per profile.
   - Profile-aware activation requests: cya says "give me a context package for a terminal assistance turn in this cwd under profile X, budget 4k tokens, prioritize retrospection + procedural kinds".

5. **Lifecycle & phase management**:
   - Explicit phase transition proposals (fluid → stabilized) with user-reviewable diffs.
   - Compaction / eviction planner with dry-run + rationale.
   - TTL + review_date on memory items (cya can surface "this convention has not been reviewed in 90 days").

6. **Explainability & audit**:
   - Every returned memory item (raw or synthesized) must carry machine + human readable provenance (source memories, reflection prompt id, timestamp, author).
   - Structured "memory influence" objects that cya can render cleanly in `--explain-context` and final output without custom parsing.

7. **Safety & policy integration**:
   - Memory evolution operations must be able to declare "safety impact: none / tightens / potentially relaxes".
   - Hook for cya's rule-based RiskClassifier to consult memory policy before activation (or to inject forced-confirmation signals from procedural rules).
   - User-level "memory evolution veto mode" (all proposals require explicit approval; no auto-stabilization).

8. **Observability & introspection**:
   - `export_memory` / profile introspection that includes "active procedural rules", "recent synthesis activity", "token budget usage by phase".
   - Event log query API so cya can show "memory events that led to this activated preference".

9. **Retrospection / continuous optimization interop**:
   - Standard event schema for "retrospection session completed" that phase-memory planners can consume directly.
   - Ability for cya to say "this retrospection outcome should be treated as high-importance for stabilization".

**Non-goals for near-term phase-memory feedback**: full embedding/semantic index details, voice, cross-user sharing, production scale numbers. Focus on the **profile + planner + procedural + dry-run proposal** surface that lets cya deliver the three self-improving variations safely and explainably.

These suggestions will be turned into a clean, reviewable artifact (standalone doc or PR-ready section) in CYA-WP-0005 and shared with the phase-memory / markitect owners via State Hub or direct coordination.

---

**End of persisted research document.**  
This artifact, together with the three gap analyses in history/, MemoryVision.md, and the activation concept doc, forms the knowledge base for CYA-WP-0005.

Next: Create the workplan that turns this research into executable tasks (profile 0 baseline + 1–3 + phase-memory feedback deliverable).