phase-memory/workplans/PMEM-WP-0006-retrieval-activation-quality-and-evaluation.md

---
id: PMEM-WP-0006
type: workplan
title: "Retrieval, Activation Quality, And Evaluation"
domain: markitect
repo: phase-memory
status: proposed
owner: phase-memory
topic_slug: activation-quality
planning_priority: P2
planning_order: 60
related_workplans:
  - PMEM-WP-0002
  - PMEM-WP-0005
created: "2026-05-18"
updated: "2026-05-18"
state_hub_workstream_id: "68417e46-d0a4-4168-bdd6-9ba6ac7847c1"
---

# PMEM-WP-0006: Retrieval, Activation Quality, And Evaluation

## Goal

Move activation planning beyond deterministic id ordering into explainable,
policy-aware memory retrieval and evaluation.

INTENT.md expects activation memory to compile graph neighborhoods, decision
paths, conversation episodes, and knowledge slices into bounded context
packages. The current activation planner selects nodes under item and token
budgets, but it does not yet model graph neighborhoods, event paths, freshness,
confidence, or retrieval quality.

## Current Evidence

Current activation behavior:

- prioritizes explicitly requested node ids
- sorts remaining nodes by id
- estimates tokens with word count
- includes events only when they reference packages or activations
- emits Markitect-compatible selection dictionaries
- explains budget omissions

This is enough for a first handoff, but not enough to satisfy the full
activation-memory intent.

## Non-Goals

- Do not require embeddings or vector stores in the default path.
- Do not use live LLM ranking in deterministic tests.
- Do not own benchmark dashboards that belong in `infospace-bench`.
- Do not optimize for a single application domain.

## T01 - Add deterministic graph-neighborhood retrieval

```task
id: PMEM-WP-0006-T01
status: todo
priority: high
state_hub_task_id: "8ed0909f-9e8e-4d49-9312-dca267df29f5"
```

Add selection strategies that can expand from seed nodes across graph edges:

- one-hop neighbors
- bounded multi-hop neighbors
- edge-kind filters
- source and target direction filters
- phase filters
- memory kind filters

Output: retrieval planner and tests for stable graph-neighborhood selection.

## T02 - Add event-path activation

```task
id: PMEM-WP-0006-T02
status: todo
priority: high
state_hub_task_id: "5d48ba91-fef0-4d4f-a560-836abed1c527"
```

Select conversational path episodes and event windows as first-class
activation inputs.

Output: event-path selection support, path budget behavior, and tests around
active, abandoned, and merged paths.

## T03 - Add ranking signals and explanations

```task
id: PMEM-WP-0006-T03
status: todo
priority: high
state_hub_task_id: "0f6340ef-f7bd-408b-b98e-6d90188c5969"
```

Rank activation candidates using deterministic local signals:

- explicit priority
- graph distance from seed
- lifecycle state
- phase
- freshness
- confidence
- policy allowance
- source-backed status
- prior activation references

Output: scoring model, per-item selection reason, and omitted-item reason.

## T04 - Improve token and budget accounting

```task
id: PMEM-WP-0006-T04
status: todo
priority: medium
state_hub_task_id: "12d83382-a767-45a8-b7cc-8c3f6f3f4c37"
```

Replace rough word-count behavior with a pluggable budget estimator that can
stay deterministic locally and later delegate to provider-specific tokenizers.

Output: estimator protocol, default estimator, and tests for node, event, and
package budget pressure.

## T05 - Add evaluation fixture scenarios

```task
id: PMEM-WP-0006-T05
status: todo
priority: medium
state_hub_task_id: "509e9417-3aa7-4899-aed5-20749372fe00"
```

Add small evaluation fixtures inspired by adjacent `infospace-bench` pilots:

- restart package
- decision recall
- stale source refresh
- policy-denied activation
- compacted trace window
- conflicting preference update

Output: fixture set and expected activation plans.

## T06 - Add maturity metrics for activation quality

```task
id: PMEM-WP-0006-T06
status: todo
priority: medium
state_hub_task_id: "477a896a-8013-42a5-b965-b1ccd2577fec"
```

Define local metrics that can be exported to `infospace-bench` later:

- selected expected nodes
- omitted required nodes
- policy-denied required nodes
- token budget utilization
- stale item activation count
- provenance coverage
- source span coverage
- explanation coverage

Output: metrics helper and JSON report fixture.

## T07 - Document retrieval and evaluation behavior

```task
id: PMEM-WP-0006-T07
status: todo
priority: medium
state_hub_task_id: "551432e4-2551-49fa-b17b-f762853a6a50"
```

Document activation strategy, scoring inputs, limitations, and evaluation
fixtures.

Output: activation planning guide and scorecard update.

## Acceptance Criteria

- `python3 -m pytest` passes.
- Activation can select graph neighborhoods and event paths under budget.
- Every selected and omitted item has a machine-readable reason.
- Evaluation fixtures produce deterministic activation quality reports.
- Optional semantic indexes remain behind the `SemanticIndex` port.