diff --git a/docs/cya-memory-activation-and-retrospection-concept.md b/docs/cya-memory-activation-and-retrospection-concept.md new file mode 100644 index 0000000..d32fe58 --- /dev/null +++ b/docs/cya-memory-activation-and-retrospection-concept.md @@ -0,0 +1,131 @@ +# Conceptual Model: Contextual Memory Activation and Retrospection Loops (CYA-WP-0003 T01) + +**Date:** 2026-05-27 +**Workplan:** CYA-WP-0003 +**Related:** INTENT.md, MemoryVision.md, history/2026-05-27-CYA-Intent-Scope-Gap-Analysis-Post-0002.md, CYA-WP-0002 results + +## Guiding Principle + +Memory in `cya` must be **proactively useful** and **user-steerable over time**. + +Two complementary mechanisms achieve this: + +1. **Contextual Activation** — Relevant memory is automatically surfaced based on the user's current working context (especially directory/project), without the user having to explicitly recall it every time. +2. **Retrospection Loops** — Structured, recurring reflection sessions allow the user to review how memory was used, reflect on effectiveness, and explicitly set or refine goals, preferences, and "rules for future assistance." + +Together they create a user-controlled continuous optimization loop: Use → Surface memory in context → Reflect → Evolve memory/goals → Better future context and behavior. + +This directly addresses the remaining gaps after CYA-WP-0002: moving from passive storage to active, contextually-aware, longitudinally improving assistance. + +## Alignment with INTENT.md and MemoryVision.md + +**From INTENT.md:** +- "Personalized Console Helper" that adapts to habits, preferred styles, repository conventions, recurring workflows, safety preferences. +- `phase-memory` must support "project-specific memory", "distinguishing global, local, temporary, and contextual memory". +- Core principle: user’s working memory belongs to the user. + +**From MemoryVision.md:** +- Primary relevant kinds: Project/Directory Memory, Workflow Recipes & Patterns, Learned Preferences & Conventions, Safety & Trust Signals. +- `cya` is a consumer that requests activation of context packages. +- Every response influenced by memory must explain **which** memory items and why. +- Support for profile-driven behavior (what to stabilize, compact, etc.). + +The proposed model is a direct, practical realization of these for the current and near-term phase (local JSON backing evolving toward full phase-memory). + +## Model 1: Memory Activation + +### Definition +**Memory Activation** is the process by which `cya` automatically selects and injects relevant items from the user's memory store into the assistance context for a given request, based on contextual signals. + +### Primary Activation Signal (MVP for this workplan) +- **Current working directory + git root** (the `scope` already used in the T02 implementation). +- When the user runs `cya` in a directory, the system preferentially activates items whose `scope` matches the cwd or any ancestor directory up to the git root. +- Project identity (git remote or root path) can serve as a stronger key for "project memory". + +### Activation Layers (in priority order) +1. Explicitly user-pinned or high-priority items for the current scope. +2. Recently used or high-confidence stabilized preferences for the project/directory. +3. Workflow patterns and recipes previously approved/refined in this context. +4. Safety & trust signals (e.g., "never auto-run" patterns remembered for this project). +5. General (global) user preferences as fallback. + +### Activation Metadata (for explainability) +Every activated item must carry: +- `provenance`: where it came from (retrospection, explicit remember, etc.) +- `confidence` / stability phase hint +- `reason_for_activation`: why it was selected for this context/request + +This must be surfaced cleanly in `--explain-context` and (when relevant) in the final response. + +### User Control +- `--no-memory` or per-request opt-out. +- Ability to say "always include X in this project" or "never activate Y here". +- `cya memory export/inspect/forget` for the current scope. + +## Model 2: Retrospection as a First-Class Interaction Pattern + +### Definition +A **Retrospection** is a deliberate, bounded session (user-initiated or lightly prompted) in which the user and `cya` jointly review recent interactions, memory usage, and outcomes, then explicitly capture reflection and forward-looking intent. + +### Retrospection Flow (High-Level) +1. Trigger: `cya retrospect` (or equivalent special prompt/mode). +2. Review phase: + - Show recent assistance cycles and which memory items were activated. + - Surface outcomes (was the suggestion accepted? did it require extra clarification?). +3. Reflection phase: + - User answers guided questions: "What worked well?", "What should I have remembered?", "How do I want to be helped differently going forward?" +4. Goal-setting / Intent capture: + - New or updated preferences. + - Interaction goals (e.g., "Prefer concise answers with one alternative", "Always warn about destructive commands even if I have approved the pattern before"). + - Explicit stabilization or forgetting decisions. +5. Commit phase: + - Outcomes are stored as first-class memory records (with `kind: "retrospection"` or "interaction_goal"). + - These receive preferential treatment in future activation (higher weight for the same scope). + +### Retrospection Outputs as Higher-Order Memory +- `kind: "interaction_goal"` or "retrospection_outcome" +- Fields: goal_text, applies_to (scope, kinds, task_classes), priority, review_date +- These can influence: + - What gets activated. + - Explanation depth and style passed in context to the LLM. + - Safety tuning (e.g., "treat this pattern more conservatively in this project"). + +### Relationship to Activation +Retrospection is one of the most powerful ways new memory enters the system in a high-quality, user-vetted form. Activated memory during normal use can trigger a lightweight retrospection prompt ("Would you like to capture this as a recurring preference?"). + +## Integration with Existing Ports and Future phase-memory + +### Minimal Extensions Identified for T02 +Current ports (post-0002) already provide a good foundation via the `scope` and `profile` parameters. + +Recommended small evolutions (to be detailed and implemented in T02): +- Add support for a `kind` parameter / filter in `recall_preferences` (to boost or specifically request retrospection outcomes, interaction goals, etc.). +- Add `activation_context` or richer hinting when calling recall (so the orchestrator can say "current cwd + recent task class"). +- Standardize a `retrospection` / `interaction_goal` kind in the stored records. +- Add a lightweight `remember_retrospection_outcome(...)` helper (or just use `remember_preference` with special kind) for now. +- Ensure `export_memory` can filter and pretty-print retrospection records. + +Longer term (when wiring to full phase-memory): +- Retrospection outcomes become natural input to profile execution and stabilization planners. +- Activation becomes a proper request for a "context package" under user profile constraints. + +### Explainability & Safety (Non-Negotiable) +- Any memory-activated context (including from retrospection goals) must be explainable. +- Retrospection-derived goals that would affect risky commands must still pass through the rule-based risk classifier (T03/T04 invariants preserved). +- Users can always review, edit, or delete retrospection records. + +## Proposed Design Artifacts to Produce in Later Tasks +- Retrospection prompt templates / guided flow (T04). +- Changes to orchestrator context assembly for automatic activation (T03). +- Observability for "memory activated from retrospection goals" (T06). +- User-facing examples in README (T07). + +## Acceptance Criteria for T01 (Self-Check) + +- [x] The two ideas (activation + retrospection) are clearly defined and shown to reinforce each other. +- [x] Models are consistent with INTENT.md (personalized helper, project-specific memory, user control) and MemoryVision.md (kinds, activation, explainability, profile direction). +- [x] Concrete, minimal extensions to the current port surface have been identified. +- [x] Safety and explainability requirements are explicitly preserved. +- [ ] (To be validated in review) The concepts feel natural in a terminal workflow. + +This document serves as the primary output of T01. It can be referenced and evolved in subsequent tasks and eventually folded into MemoryVision.md or a dedicated user guide. \ No newline at end of file diff --git a/workplans/CYA-WP-0003-contextual-memory-activation-and-retrospection.md b/workplans/CYA-WP-0003-contextual-memory-activation-and-retrospection.md index 5f3136e..1dff7b9 100644 --- a/workplans/CYA-WP-0003-contextual-memory-activation-and-retrospection.md +++ b/workplans/CYA-WP-0003-contextual-memory-activation-and-retrospection.md @@ -48,21 +48,26 @@ This workplan directly addresses two remaining gaps identified in the post-0002 ```task id: CYA-WP-0003-T01 -status: todo +status: done priority: high state_hub_task_id: "335595b4-96fe-401f-ab4c-0824a6ba3f05" +started: "2026-05-27 ralph iter 1" +completed: "2026-05-27" ``` -- Deeply align the two ideas (directory-bound activation + regular retrospections) with INTENT.md and MemoryVision.md. -- Define clear models for: - - "Memory activation" (what gets recalled automatically, when, and why). - - Retrospection as a first-class interaction pattern (what is captured, how goals/preferences are updated). -- Produce a short design note (or section in MemoryVision) describing the interaction loop. -- Identify minimal extensions needed to the existing memory ports and data model. +**Done** — produced `docs/cya-memory-activation-and-retrospection-concept.md`. -**Acceptance criteria**: -- Clear, written concept exists and is consistent with INTENT and MemoryVision. -- The two ideas are shown to reinforce each other rather than compete. +- Deeply aligned the two ideas with INTENT.md (Personalized Console Helper, project-specific memory, recurring workflows, user control) and MemoryVision.md (Project/Directory Memory, Workflow Recipes, activation planning, explainability requirements). +- Defined clear models: + - **Memory Activation**: Automatic, scope-aware (cwd + git root) surfacing of relevant items (preferences, patterns, safety signals) with strong provenance and user control. + - **Retrospection Loops**: Deliberate reflection sessions that produce higher-order memory (`interaction_goal`, retrospection outcomes) which preferentially influence future activation and behavior. +- The two mechanisms reinforce each other: normal use generates candidates for retrospection; retrospection produces high-quality memory that improves future activation. +- Identified minimal port/data model extensions for T02 (richer `kind` support and activation hints in recall, standardized retrospection record kinds). +- Non-negotiables (explainability + safety invariants) explicitly preserved. + +**Acceptance criteria met**: +- Clear, written concept document exists and is consistent with INTENT and MemoryVision. +- The ideas are shown to reinforce each other in a user-controlled continuous optimization loop. ### T02 — Extend memory ports and data model for activation and retrospection