can-you-assist/workplans/CYA-WP-0003-contextual-memory-activation-and-retrospection.md

id, type, title, domain, repo, status, owner, topic_slug, created, updated, state_hub_workstream_id

id	type	title	domain	repo	status	owner	topic_slug	created	updated	state_hub_workstream_id
CYA-WP-0003	workplan	Contextual Memory Activation and Retrospection Loops for Continuous Optimization	capabilities	can-you-assist	done	grok	foerster-capabilities	2026-05-27	2026-05-27	ac9a9d42-4db9-44d0-9b41-5e16d87c65c8

CYA-WP-0003: Contextual Memory Activation and Retrospection Loops for Continuous Optimization

Goal

Build on the real memory foundation delivered in CYA-WP-0002 to make memory actively useful for context-aware assistance, with two primary advances:

1. Directory- and project-bound memory activation: Automatically and intelligently surface relevant memory when the user is working in a specific directory or project, without requiring explicit recall every time.
2. Regular retrospection loops: Enable structured, recurring reflection sessions between the user and cya that turn memory into a vehicle for continuous improvement — reviewing what worked, setting interaction goals, and evolving preferences over time.

This workplan directly addresses two remaining gaps identified in the post-0002 Intent-vs-Scope analysis:

• Moving from passive memory storage to proactive, contextually activated memory.
• Establishing the foundation for true longitudinal adaptation and a user-driven optimization loop.

Background & References

• INTENT.md — especially the "Personalized Console Helper" section and the explicit call for phase-memory to support "project-specific memory" and "recurring workflows".
• MemoryVision.md — defines primary memory kinds relevant to cya, including Project/Directory Memory and Workflow Recipes.
• history/2026-05-27-CYA-Intent-Scope-Gap-Analysis-Post-0002.md — identifies remaining gaps in contextual activation and longitudinal value after 0002.
• CYA-WP-0002 — delivered the real (persisting) memory implementation and explicit ports.
• Current memory ports in src/cya/memory/__init__.py (the scope parameter is the natural starting point for directory binding).
• phase-memory concepts (phases, profiles, activation planning, dry-run-first).

Non-Goals (for this slice)

• Full autonomous agentic behavior or background memory harvesting.
• Deep semantic understanding of code or notes (still out of scope).
• Building retrospection as a fully automated scheduled process (user-initiated or explicitly prompted is preferred for control).
• Replacing the explicit port seam with direct phase-memory calls (we continue to evolve through the ports).
• Rich multi-turn conversational state beyond what scoped memory + retrospection can provide.

Task Breakdown

T01 — Refine the conceptual model for contextual activation and retrospection loops

id: CYA-WP-0003-T01
status: done
priority: high
state_hub_task_id: "335595b4-96fe-401f-ab4c-0824a6ba3f05"
started: "2026-05-27 ralph iter 1"
completed: "2026-05-27"

Done — produced docs/cya-memory-activation-and-retrospection-concept.md.

• Deeply aligned the two ideas with INTENT.md (Personalized Console Helper, project-specific memory, recurring workflows, user control) and MemoryVision.md (Project/Directory Memory, Workflow Recipes, activation planning, explainability requirements).
• Defined clear models:
  • Memory Activation: Automatic, scope-aware (cwd + git root) surfacing of relevant items (preferences, patterns, safety signals) with strong provenance and user control.
  • Retrospection Loops: Deliberate reflection sessions that produce higher-order memory (interaction_goal, retrospection outcomes) which preferentially influence future activation and behavior.
• The two mechanisms reinforce each other: normal use generates candidates for retrospection; retrospection produces high-quality memory that improves future activation.
• Identified minimal port/data model extensions for T02 (richer kind support and activation hints in recall, standardized retrospection record kinds).
• Non-negotiables (explainability + safety invariants) explicitly preserved.

Acceptance criteria met:

• Clear, written concept document exists and is consistent with INTENT and MemoryVision.
• The ideas are shown to reinforce each other in a user-controlled continuous optimization loop.

T02 — Extend memory ports and data model for activation and retrospection

id: CYA-WP-0003-T02
status: done
priority: high
state_hub_task_id: "6f50bdf4-6252-4b93-9697-407ef432cd90"
started: "2026-05-27 ralph continuation (after T01 review)"
completed: "2026-05-27"

Done — implemented in src/cya/memory/__init__.py.

• Added kind parameter to remember_preference (defaults to "preference" for full backward compat).
• Added KIND_* constants and remember_retrospection_outcome() convenience helper.
• Enhanced recall_preferences with better kinds filtering + new activation_context parameter for smarter directory/project-aware activation.
• Improved export_memory with optional kinds filter and by_kind summary.
• All changes maintain 100% backward compatibility with existing call sites.
• Verified manually (roundtrips for preferences, retrospection records, kind filtering, and activation hints all work).

Acceptance criteria met:

• Ports and data model now support the T01 concepts.
• Everything remains fully user-visible/editable (still plain JSON under ~/.config/cya/memory/).
• No breaking changes.

T03 — Implement directory- and project-bound memory activation

id: CYA-WP-0003-T03
status: done
priority: high
state_hub_task_id: "45731b48-74a5-485b-bd56-72f387db3846"
started: "2026-05-27 ralph continuation (after T02)"
completed: "2026-05-27"

Done — implemented in src/cya/orchestrator.py.

• Memory recall now passes activation_context containing cwd + git_root (when available from the envelope).
• The T02-enhanced recall_preferences uses this for smarter directory/project-bound activation (boosting matching scopes).
• Improved --explain-context panel now shows activation context and sample activated keys.
• Final output line simplified and updated to reflect activation.
• Users can influence via normal remember_preference(..., scope=...) and the existing export/forget tools (more UX in T04/T07).

Acceptance criteria met (for this slice):

• Memory is now automatically activated based on working directory/project.
• Activation is visible (in explain panel + provenance) and controllable via existing memory tools.

T04 — Build the retrospection interaction flow

id: CYA-WP-0003-T04
status: done
priority: high
state_hub_task_id: "fb63edc4-1e3f-4964-a2b6-19b29d00ffd8"
started: "2026-05-27 ralph continuation (after T03)"
completed: "2026-05-27"

Done — implemented.

• Added cya retrospect subcommand in src/cya/cli/main.py.
• Implemented run_retrospection() in src/cya/orchestrator.py:
  • Reviews recent memory in the given scope.
  • Guides the user through reflection questions.
  • Records outcomes using the T02 remember_retrospection_outcome helper (stored as retrospection / interaction_goal kinds).
  • Uses rich panels for a pleasant terminal experience.
• The command is discoverable (cya --help shows the retrospect subcommand).
• Outcomes are stored in the same user-controlled memory store and will be activated in future normal sessions (per T03 activation logic).

Acceptance criteria met (MVP):

• A user can run cya retrospect and capture goals/preferences that affect future assistance.
• The flow is natural, guided, and fully respects user control (they decide what to record).

T05 — Tests, observability, and graceful degradation

id: CYA-WP-0003-T05
status: done
priority: medium
state_hub_task_id: "f17a4f42-1630-4244-bdd4-c8d732e8de9b"
started: "2026-05-27 ralph continuation (after T04)"
completed: "2026-05-27"

• Added comprehensive tests in tests/test_memory.py for:
  • Activation logic with activation_context (T03)
  • Retrospection outcomes and kind-specific recall/export (T04)
  • Observability (provenance, by_kind, activation_context recording)
  • Graceful degradation on bad context or storage issues
• All tests are hermetic (using the existing isolated_memory fixture) and introduce no new external dependencies.
• Full suite passes cleanly: pytest tests/test_memory.py -q

Acceptance criteria met:

• Strong test coverage for the new activation + retrospection behaviors.
• pytest remains clean with no new external dependencies.

T06 — Tests, observability, and graceful degradation (completed in T05)

id: CYA-WP-0003-T06
status: done
priority: high
state_hub_task_id: "c7da1a0d-647f-48d9-aba3-d2e62791e05f"
started: "2026-05-27"
completed: "2026-05-27"

Tests were completed as part of the previous step (strong coverage for activation, retrospection, observability, and graceful degradation; all tests pass cleanly).

T07 — Documentation, examples, and handoff

id: CYA-WP-0003-T07
status: done
priority: medium
state_hub_task_id: "d120efff-eaf6-4630-a3d3-3e70a3db4e09"
started: "2026-05-27 ralph continuation (final tasks)"
completed: "2026-05-27"

Done.

• Heavily expanded the Memory section in README.md with clear before/after examples for:
  • Automatic directory/project-bound activation
  • cya retrospect guided sessions and goal capture
• Updated AGENTS.md "Commands" section with the new cya retrospect command and 0003 workplan reference.
• Added a substantial "What CYA-WP-0003 Delivered" section to MemoryVision.md documenting the new capabilities and their alignment with the original vision.
• Registered technical debt (retrospection/activation still on local JSON; deeper phase-memory integration is future work).

Acceptance criteria met:

• A reader of the README can now understand and try the new memory features (activation + retrospection) with concrete examples.
• The concepts are clearly connected back to INTENT.md and MemoryVision.md.
• Extension points and debt are documented.

Dependencies & Cross-Repo Coordination

• phase-memory: The concepts here are designed to be consumable and eventually enriched by richer phase-memory capabilities (profiles, phases, activation planning). Close coordination recommended during T01/T02.
• markitect-tool: May become relevant if retrospection outcomes or directory contexts are expressed as memory profiles.
• State Hub: For tracking this as the direct follow-on to CYA-WP-0002.

Completion

Status: done — all 7 tasks completed via ralph loop. The workplan delivered contextual memory activation (T03), a user-driven retrospection flow (T04), supporting port extensions (T02), tests (T05), and full documentation (T07).

This workplan successfully implemented the two main ideas requested: directory/project-bound memory activation and regular retrospection loops for continuous optimization — directly realizing deeper parts of INTENT.md and MemoryVision.md.

Status note: CYA-WP-0003 is complete. The explicit memory seam is now significantly more powerful while remaining fully user-controlled and transparent.