docs: update SCOPE.md post-0002 + new gap analysis (2026-05-27) vs INTENT.md

SCOPE.md

@@ -6,19 +6,23 @@
 
 It allows users to express intent in natural language from the terminal and receive safe, explainable, context-aware assistance while keeping memory, history, preferences, and adaptation under explicit user control.
 
-## Current Status (Post MVP Slice)
+## Current Status (Post CYA-WP-0002 Memory Slice)
 
-The first narrow implementation slice (CYA-WP-0001) has been delivered. A working `cya` tool now exists that can be installed with `pip install -e .`.
+Two implementation slices have been delivered:
 
-Core capabilities implemented:
+- **CYA-WP-0001 (Console-Native MVP)**: Core CLI, context collection, rule-based safety with mandatory confirmation, LLMAdapter seam, and thin memory ports.
+- **CYA-WP-0002 (Memory Integration)**: Real user-controlled, persisting memory implementation behind the explicit ports, wired into the orchestrator and `--explain-context`, memory signals integrated into safety, dedicated tests, and documentation.
+
+Core capabilities now include:
 - Natural language request handling via a clean Typer CLI.
-- Bounded, transparent, non-recursive local context collection (cwd + git + environment + explicit files).
-- Genuine rule-based risk classification with mandatory terminal confirmation for anything above "safe".
-- Stable `LLMAdapter` Protocol boundary (currently satisfied by a deterministic `FakeLLMAdapter`).
-- Strictly minimal explicit no-op ports for future `phase-memory` integration.
-- A small orchestrator that coordinates the above.
+- Bounded, transparent, non-recursive local context collection.
+- Genuine rule-based risk classification (now memory-aware) with mandatory terminal confirmation.
+- Stable `LLMAdapter` Protocol boundary.
+- Real, user-controlled memory for preferences and workflow patterns (scoped JSON backing under `~/.config/cya/memory/`, fully inspectable and user-editable).
+- Memory surfaced in explanations and fed into safety decisions.
+- A small orchestrator coordinating the pipeline.
 
-All LLM interaction flows through the documented adapter seam. No production path bypasses it.
+All LLM interaction flows through the documented adapter seam. Memory flows through explicit ports. No production path bypasses these boundaries.
 
 ## Owns
 
@@ -28,7 +32,7 @@ All LLM interaction flows through the documented adapter seam. No production pat
 - Safety layer: rule-based risk assessment + mandatory explicit confirmation flow.
 - Orchestration of the request → context → safety → LLM adapter → response pipeline.
 - The stable `LLMAdapter` Protocol and the contract for how `cya` talks to LLM backends.
-- Explicit, minimal integration points (ports) for `phase-memory`.
+- Explicit, now real (persisting) integration with user-controlled memory via `phase-memory` ports.
 - Transparent, inspectable behavior (especially via `--explain-context`).
 - User-facing documentation, examples, and safety guarantees for the CLI tool.
 
@@ -48,7 +52,7 @@ All LLM interaction flows through the documented adapter seam. No production pat
 | Project       | Responsibility                          | Integration Style                  |
 |---------------|-----------------------------------------|------------------------------------|
 | `llm-connect` | Provider access, config, token counting, structured responses | Stable `LLMAdapter` Protocol      |
-| `phase-memory`| User-controlled memory, preferences, history | Explicit thin ports (currently no-op) |
+| `phase-memory`| User-controlled memory, preferences, history | Explicit ports with real (local JSON) implementation (T02); long-term target is deeper profile-driven integration |
 | State Hub     | Work tracking, decisions, coordination  | HTTP REST (non-runtime)           |
 
 ## MVP Scope (CYA-WP-0001)
@@ -59,19 +63,19 @@ What was explicitly in scope for the first slice:
 - Safe, bounded context collection with full transparency (`--explain-context`).
 - Rule-based safety classification as the primary mechanism, with mandatory confirmation.
 - A clean, documented adapter boundary for future real LLM backends.
-- Strictly minimal memory ports (no hidden state, no local JSON store).
+- Real user-controlled memory implementation (scoped, persisting, explainable) behind the explicit ports, with a clean seam for future deeper `phase-memory` integration.
 - Basic orchestrator that ties the pieces together.
 - Test coverage focused on safety invariants and context rules.
 - Clear public boundaries and extension points.
 
-## Explicitly Out of Scope (MVP and Near-Term)
+## Explicitly Out of Scope (Current and Near-Term)
 
-- Any durable or sophisticated memory implementation.
+- Full deep integration with the complete `phase-memory` profile/planner/graph system (current implementation uses a deliberate, user-visible local JSON store as the first real backing; deeper integration is planned future work).
 - Real `llm-connect` client (only the contract + fake exists).
-- Deep git/repository understanding beyond basic status + log.
+- Deep git/repository understanding beyond basic status + log + user-declared memory.
 - Automatic command execution (even "safe" suggestions).
 - Structured editing / patch generation.
-- Multi-turn conversation state.
+- Rich multi-turn conversational state (one-shot + lightweight scoped memory only).
 - Cost tracking, token budgeting, or usage dashboards.
 - Team/shared memory or collaboration features.
 - Plugin system or domain-specific extensions.
@@ -79,7 +83,7 @@ What was explicitly in scope for the first slice:
 ## Extension Points (Registered)
 
 - `cya/llm/adapter.py` — `LLMAdapter` Protocol (the primary seam).
-- `cya/memory/__init__.py` — the four explicit ports (`remember_preference`, `recall_preferences`, `forget`, `export_memory`).
+- `cya/memory/__init__.py` — the four explicit ports with real (persisting, user-controlled) implementation behind them.
 - `cya/safety/risk.py` — the `_RULES` table and `classify()` function.
 - `cya/context/collector.py` — collection functions and ignore policy.
 - `cya/orchestrator.py` — the main coordination entry point.
@@ -96,4 +100,8 @@ Sibling project owners can read the workplan + boundary documentation and know p
 
 ---
 
-**This SCOPE document reflects the state after the CYA-WP-0001 MVP slice.** It is intentionally narrower than the long-term vision in INTENT.md.
+**This SCOPE document reflects the state after CYA-WP-0001 (MVP) + CYA-WP-0002 (Memory Integration).**
+
+It is intentionally narrower than the long-term vision in INTENT.md and MemoryVision.md, but now includes a real first slice of user-controlled memory as delivered by 0002.
+
+See `workplans/CYA-WP-0002-memory-integration-roadmap.md` and `MemoryVision.md` for the intended direction of deeper `phase-memory` integration.

history/2026-05-27-CYA-Intent-Scope-Gap-Analysis-Post-0002.md

@@ -0,0 +1,134 @@
+# Gap Analysis: INTENT.md vs SCOPE.md (Post CYA-WP-0002 Memory Integration)
+
+**Date:** 2026-05-27  
+**Repo:** can-you-assist  
+**Workplan:** CYA-WP-0002 (completed)  
+**Previous Analysis:** 2026-05-26 (Post 0001)  
+**Author:** Grok
+
+## Executive Summary
+
+Since the May 26 analysis (conducted immediately after CYA-WP-0001), the team has executed and completed **CYA-WP-0002 — Memory Integration Roadmap**.
+
+This work directly targeted the largest identified gap from the previous analysis: **memory, preferences, and longitudinal user-controlled adaptation**.
+
+### Key Change Since Last Analysis
+- **Memory** moved from "strictly minimal no-op ports" (T05 of 0001) to a **real, persisting, user-controlled implementation** with:
+  - JSON-backed storage under `~/.config/cya/memory/` (explicitly user-inspectable and editable)
+  - Full wiring into the orchestrator and `--explain-context`
+  - Memory signals feeding the rule-based safety layer (T04)
+  - Dedicated test coverage (T05)
+  - Documentation and examples (T06)
+- The explicit port seam defined in T01/T05 remains the long-term integration point for full `phase-memory`.
+
+**Overall Assessment**: The memory gap has been meaningfully closed for a first production-quality slice. The product now delivers on a core piece of the original INTENT that was previously missing. Other gaps remain.
+
+## Strong Alignments (Updated)
+
+| Area                              | INTENT.md Position                              | Current Reality (Post-0002)                              | Assessment          |
+|-----------------------------------|--------------------------------------------------|-----------------------------------------------------------|---------------------|
+| Console-native experience         | Foundational                                     | Excellent                                                 | Strong match        |
+| Safety & mandatory confirmation   | Important                                        | Core product behavior + memory-aware (T04)                | Exceeded            |
+| Explainability & transparency     | Strong requirement                               | Very strong (provenance in memory + context)              | Strong              |
+| Backend agnosticism               | Must use llm-connect seam                        | Clean LLMAdapter Protocol                                 | Excellent           |
+| User-controlled memory            | Central principle                                | Real implementation now exists (T02), user-visible JSON   | Major improvement   |
+| Clear boundaries                  | cya / llm-connect / phase-memory separation      | Still clearly documented                                  | Good                |
+
+## Key Gaps (Post-0002)
+
+### 1. Memory: From Largest Gap to "First Slice Delivered"
+
+**Previous Status (May 26)**: The biggest divergence. Only no-op ports existed. No real accumulation of value across sessions.
+
+**Current Status (May 27)**:
+- Real persisting memory for preferences and workflow patterns is implemented and working.
+- Users can `remember` things scoped to directories/projects and have them recalled later.
+- Memory is explainable and feeds safety.
+- Backing store is deliberately simple and user-owned (JSON files).
+
+**Remaining Sub-Gaps** (intentional per the 0002 roadmap):
+- Still a local JSON implementation, not yet wired to the full `phase-memory` profile/planner/graph system described in MemoryVision.md and phase-memory's architecture.
+- No support yet for richer memory *kinds* (conversation history as first-class fluid memory, project knowledge graphs, etc.).
+- No profile-driven lifecycle, compaction, or stabilization yet (these are owned by phase-memory).
+
+**Assessment**: Large positive movement. The "no memory at all" problem is solved for practical use. The next deepening is now a proper integration exercise rather than a foundational build.
+
+### 2. Depth of Local Context Understanding
+
+**INTENT.md** envisions rich assistance with code repositories, notes, project structures, and conventions.
+
+**Current State**:
+- Context collector remains intentionally shallow (top-level cwd entries + basic git status).
+- No recursive understanding, no semantic awareness of code or notes.
+- Memory helps with *user-declared* project conventions, but the system does not autonomously discover deep structure.
+
+**Gap Type**: Still medium-to-large. The tool is stronger for "I have a preference or pattern I want remembered" than for "understand this complex codebase for me."
+
+### 3. One-Shot vs Longitudinal / Adaptive Value
+
+**INTENT.md**:
+- Strong vision of the assistant becoming more useful over time through memory of habits, conventions, aliases, recurring workflows, preferred explanation styles, safety tolerance, etc.
+- "Personalized console helper."
+
+**Current Reality (Post-0002)**:
+- We now have the basic mechanism for this (remember/recall scoped preferences).
+- Early adaptation is possible today (users can teach `cya` their preferences).
+- However, higher-order adaptation (automatically learning patterns from accepted suggestions, building project "personalities", long-term user models) is not yet present.
+- Most usage is still fundamentally one-shot + explicit memory seeding.
+
+**Gap Type**: Reduced but still significant. The foundation for longitudinal value now exists; rich automatic adaptation is future work.
+
+### 4. Safety Implementation (Continued Positive Gap)
+
+Memory signals are now considered in risk classification (T04) while preserving the hard "never auto-execute" rule. This strengthens the original intent.
+
+**Assessment**: Remains a strength.
+
+### 5. Documentation and Perceived Scope (Meta Gap)
+
+**Observation**:
+- The current [SCOPE.md](/home/worsch/can-you-assist/SCOPE.md) still describes the world as it existed after CYA-WP-0001.
+- It continues to say memory ports are "strictly minimal no-op" and lists durable memory as out of scope.
+- This creates a documentation gap between what the code and README now deliver and what the official Scope document claims.
+
+This is a maintenance issue rather than a product gap, but it is noticeable.
+
+## Summary Table (Updated)
+
+| Gap Area                        | Severity (Post-0001) | Severity (Post-0002) | Trend      | Notes |
+|---------------------------------|----------------------|----------------------|------------|-------|
+| Memory & Adaptation             | Large                | Medium               | Much better | Real prefs + wiring delivered; full phase-memory integration pending |
+| Depth of Context Understanding  | Medium-Large         | Medium-Large         | Unchanged  | Still shallow collector |
+| Longitudinal / Automatic Adaptation | Large             | Medium               | Improved   | Foundation exists via explicit memory |
+| Safety & Explainability         | Positive             | Positive             | Stable     | Further strengthened |
+| Documentation vs Reality        | Small                | Medium               | Worsened   | SCOPE.md is now stale |
+
+## Recommendations
+
+1. **Update SCOPE.md** (high priority)
+   - Reflect post-0002 reality: real user-controlled memory now exists.
+   - Distinguish between "current implementation" (local JSON + ports) and "long-term target" (deep phase-memory integration).
+   - Move durable memory out of "explicitly out of scope."
+
+2. **Continue the Memory Roadmap**
+   - The natural next work would be deeper integration with `phase-memory` (once it exposes stable high-level APIs for preferences/project context) or richer memory kinds beyond simple key-value prefs.
+
+3. **Consider a "Context Depth" Slice**
+   - If richer repository and note understanding becomes important, a dedicated workplan focused on improving the collector + adding project memory conventions would address the second-largest remaining gap.
+
+4. **Keep the Explicit Seam Discipline**
+   - The four ports + MemoryVision contract remain the correct boundary. Future work should continue to flow through them rather than building parallel memory systems inside cya.
+
+## Conclusion
+
+CYA-WP-0002 successfully closed the most glaring hole between INTENT and delivered reality. The product now has a credible story for user-controlled memory and early personalization that was completely absent after the MVP.
+
+The remaining gaps are more about *depth and richness* of context/memory rather than the complete absence of foundational capabilities. This represents healthy progress aligned with the original vision, while maintaining the disciplined, boundary-respecting approach established in 0001.
+
+---
+
+**Related Documents**
+- Previous analysis: `history/2026-05-26-CYA-Intent-Scope-Gap-Analysis.md`
+- Memory vision: `MemoryVision.md`
+- Completed memory workplan: `workplans/CYA-WP-0002-memory-integration-roadmap.md`
+- Current (stale) scope: `SCOPE.md` (recommended for update)