chore(WP-0013): create Phase 12 workplan — Platform Memory and Continuous Learning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 22:59:56 +00:00
parent 4a32593990
commit eb0803b8d7
1 changed files with 532 additions and 0 deletions
--- a/workplans/IHUB-WP-0013-ihf-phase12-platform-memory.md
+++ b/workplans/IHUB-WP-0013-ihf-phase12-platform-memory.md
@@ -0,0 +1,532 @@
+---
+id: IHUB-WP-0013
+type: workplan
+title: "IHF Phase 12 — Platform Memory and Continuous Learning"
+domain: inter_hub
+repo: inter-hub
+status: todo
+owner: custodian
+topic_slug: inter_hub
+created: "2026-04-01"
+updated: "2026-04-01"
+state_hub_sync: done
+state_hub_workstream_id: "baeb2891-2136-4ac5-aa03-b635e87285dd"
+---
+
+# IHF Phase 12 — Platform Memory and Continuous Learning
+
+## Goal
+
+Close the longest feedback loop in the IHF: from deployed outcome signals and
+accumulated governance history back to improved distillation, better routing,
+and sharper AI proposals. Phase 12 makes the IHF a learning platform, not
+merely a record-keeping one.
+
+## Background
+
+Phases 1–11 and IHUB-WP-0013 entry gates are satisfied:
+
+- Phase 4 `OutcomeSignal` append-only table operational ✓
+- Phase 7 `FrictionScore` + `BottleneckRecord` + `HubHealthSnapshot` operational ✓
+- Phase 10 `WidgetPattern` + `PatternAdoption` with aggregate panel ✓
+- Phase 11 `AgentRegistration` + `ModelRoutingPolicy` + `AiGovernancePolicy` operational ✓
+- Full traceability chain: Widget → Annotation → RequirementCandidate →
+  Requirement → DecisionRecord → DeploymentRecord → OutcomeSignal ✓
+- GAAF scorecard at 3.61 (Strong) ✓
+
+Reference: `specs/InteractionHubFrameworkSpecification_v0.2.md` §Phase 12.
+
+## GAAF Architectural Constraints
+
+1. `outcome_correlations.correlation_type` must carry a CHECK constraint
+   (`annotation_predictor`, `routing_quality`, `pattern_quality`).
+2. `learning_insights.insight_type` must carry a CHECK constraint
+   (`annotation_predictor`, `threshold_calibration`, `pattern_ranking`,
+   `routing_improvement`).
+3. **Core table extensions** — `decision_records` and `requirement_candidates`
+   gain `outcome_summary JSONB NULL` columns via ALTER TABLE. This requires
+   updating `/contracts/core/` (GAAF rule 3).
+4. The outcome_signals enrichment trigger is **read-only on core tables** —
+   it may UPDATE outcome_summary on non-append-only columns; it must never
+   UPDATE outcome_signals or interaction_events.
+5. The knowledge base uses PostgreSQL GIN full-text search over
+   `institutional_knowledge_entries.summary`, not a vector database.
+   Simple, dependency-free, works with the existing Postgres stack.
+
+## Data Artifacts Introduced
+
+`OutcomeCorrelation`, `PatternPerformanceRecord`, `AdaptiveThresholdConfig`,
+`InstitutionalKnowledgeEntry`, `LearningInsight`
+
+### Schema additions
+
+```sql
+-- outcome_correlations: links annotation signals to downstream outcome quality
+-- GAAF: correlation_type CHECK constraint
+CREATE TABLE outcome_correlations (
+    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
+    hub_id UUID NOT NULL REFERENCES hubs(id),
+    annotation_category TEXT NOT NULL REFERENCES annotation_category_registry(name),
+    correlation_type TEXT NOT NULL DEFAULT 'annotation_predictor',
+    correlation_score DOUBLE PRECISION NOT NULL,
+    -- score = fraction of positive outcomes for this category in this hub
+    sample_count INTEGER NOT NULL DEFAULT 0,
+    computed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
+    CHECK (correlation_type IN ('annotation_predictor', 'routing_quality', 'pattern_quality'))
+);
+
+CREATE INDEX outcome_correlations_hub_idx ON outcome_correlations (hub_id);
+CREATE INDEX outcome_correlations_score_idx ON outcome_correlations (correlation_score DESC);
+
+-- pattern_performance_records: per-pattern historical outcome quality
+CREATE TABLE pattern_performance_records (
+    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
+    widget_pattern_id UUID NOT NULL REFERENCES widget_patterns(id),
+    hub_id UUID NOT NULL REFERENCES hubs(id),
+    adoption_count INTEGER NOT NULL DEFAULT 0,
+    positive_outcome_count INTEGER NOT NULL DEFAULT 0,
+    total_outcome_count INTEGER NOT NULL DEFAULT 0,
+    mean_outcome_value DOUBLE PRECISION,
+    outcome_rank INTEGER,
+    calibrated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
+    UNIQUE (widget_pattern_id, hub_id)
+);
+
+CREATE INDEX pattern_performance_pattern_idx ON pattern_performance_records (widget_pattern_id);
+CREATE INDEX pattern_performance_rank_idx ON pattern_performance_records (hub_id, outcome_rank);
+
+-- adaptive_threshold_configs: per-hub friction weight overrides
+CREATE TABLE adaptive_threshold_configs (
+    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
+    hub_id UUID NOT NULL REFERENCES hubs(id) UNIQUE,
+    weight_overrides JSONB NOT NULL DEFAULT '{}',
+    -- keys: friction component names; values: multiplier floats
+    bottleneck_threshold_override DOUBLE PRECISION,
+    calibration_date TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
+    notes TEXT
+);
+
+CREATE INDEX adaptive_threshold_hub_idx ON adaptive_threshold_configs (hub_id);
+
+-- institutional_knowledge_entries: distilled decision summaries
+-- GIN index for full-text search
+CREATE TABLE institutional_knowledge_entries (
+    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
+    hub_id UUID NOT NULL REFERENCES hubs(id),
+    decision_record_id UUID REFERENCES decision_records(id),
+    summary TEXT NOT NULL,
+    summary_tsv TSVECTOR GENERATED ALWAYS AS (to_tsvector('english', summary)) STORED,
+    tags JSONB NOT NULL DEFAULT '[]',
+    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
+    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL
+);
+
+CREATE INDEX institutional_knowledge_hub_idx ON institutional_knowledge_entries (hub_id);
+CREATE INDEX institutional_knowledge_fts_idx ON institutional_knowledge_entries USING GIN (summary_tsv);
+
+-- learning_insights: platform-level insights with evidence
+-- GAAF: insight_type CHECK constraint
+CREATE TABLE learning_insights (
+    id UUID DEFAULT uuid_generate_v4() PRIMARY KEY NOT NULL,
+    hub_id UUID NOT NULL REFERENCES hubs(id),
+    insight_type TEXT NOT NULL,
+    title TEXT NOT NULL,
+    body TEXT NOT NULL,
+    evidence_links JSONB NOT NULL DEFAULT '[]',
+    -- array of {type, id, label} objects
+    is_actioned BOOLEAN NOT NULL DEFAULT FALSE,
+    computed_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() NOT NULL,
+    CHECK (insight_type IN (
+        'annotation_predictor',
+        'threshold_calibration',
+        'pattern_ranking',
+        'routing_improvement'
+    ))
+);
+
+CREATE INDEX learning_insights_hub_idx ON learning_insights (hub_id);
+CREATE INDEX learning_insights_type_idx ON learning_insights (insight_type);
+
+-- Extend core tables with outcome_summary (retroactive lineage enrichment)
+-- GAAF: requires /contracts/core/ update (T06)
+ALTER TABLE decision_records
+    ADD COLUMN outcome_summary JSONB NULL;
+
+ALTER TABLE requirement_candidates
+    ADD COLUMN outcome_summary JSONB NULL;
+```
+
+---
+
+## Tasks
+
+### T01 — Schema: all Phase 12 tables + migration
+
+```task
+id: IHUB-WP-0013-T01
+status: todo
+priority: high
+state_hub_task_id: "7bef90d5-7efc-488b-80ba-7f1a2220f75a"
+```
+
+Add all Phase 12 tables to `Application/Schema.sql` and write migration
+`Application/Migration/<timestamp>-ihf-phase12-platform-memory.sql`.
+
+Includes ALTER TABLE on `decision_records` and `requirement_candidates` for
+`outcome_summary JSONB NULL`. Update `/contracts/core/` to document the new
+columns per GAAF rule 3.
+
+---
+
+### T02 — Outcome correlation engine
+
+```task
+id: IHUB-WP-0013-T02
+status: todo
+priority: high
+state_hub_task_id: "589bf316-1a44-4726-b88c-cc7940f4dc53"
+```
+
+**`Application/Helper/CorrelationEngine.hs`** — pure computation:
+
+```haskell
+module Application.Helper.CorrelationEngine where
+
+import IHP.Prelude
+import Generated.Types
+import Database.PostgreSQL.Simple (Only(..))
+
+-- | For a hub, compute the correlation score per annotation category:
+-- fraction of traceability chains that end in a positive outcome signal
+-- (signal_type IN ('success', 'adoption', 'satisfaction')).
+computeAnnotationCorrelations ::
+    (?modelContext :: ModelContext) =>
+    Id Hub -> IO [(Text, Double, Int)]
+    -- ^ [(category, score, sample_count)]
+computeAnnotationCorrelations hubId =
+    sqlQuery
+        "SELECT a.category, \
+        \       COALESCE(AVG(CASE WHEN os.signal_type IN ('success','adoption','satisfaction') \
+        \                        THEN 1.0 ELSE 0.0 END), 0) AS score, \
+        \       COUNT(os.id)::int AS sample_count \
+        \ FROM annotations a \
+        \ JOIN widgets w ON w.id = a.widget_id \
+        \ JOIN requirement_candidates rc ON rc.source_widget_id = w.id \
+        \ JOIN requirements r ON r.candidate_id = rc.id \
+        \ JOIN decision_records dr ON dr.requirement_id = r.id \
+        \ JOIN deployment_records dep ON dep.decision_id = dr.id \
+        \ JOIN outcome_signals os ON os.deployment_id = dep.id \
+        \ WHERE w.hub_id = ? \
+        \ GROUP BY a.category \
+        \ ORDER BY score DESC"
+        [hubId]
+```
+
+**`Web/Controller/OutcomeCorrelations.hs`**:
+- `ComputeCorrelationsAction { hubId }` — runs engine, upserts `OutcomeCorrelation`
+  records, generates a `LearningInsight` of type `annotation_predictor` for
+  the top-scoring category
+- `OutcomeCorrelationsAction` — index view sorted by score
+
+**Views:** `Web/View/OutcomeCorrelations/{Index,Show}.hs`
+
+---
+
+### T03 — Pattern performance memory
+
+```task
+id: IHUB-WP-0013-T03
+status: todo
+priority: high
+state_hub_task_id: "3790e9da-a28b-4287-a0bb-0083e2af42f7"
+```
+
+**`ComputePatternPerformanceAction { hubId }`** in a new
+`Web/Controller/PatternPerformance.hs`:
+
+```sql
+SELECT
+    wp.id AS pattern_id,
+    COUNT(DISTINCT pa.id) AS adoption_count,
+    COUNT(os.id) AS total_outcome_count,
+    COUNT(os.id) FILTER (
+        WHERE os.signal_type IN ('success','adoption','satisfaction')
+    ) AS positive_outcome_count,
+    AVG(os.value) AS mean_outcome_value
+FROM widget_patterns wp
+JOIN pattern_adoptions pa ON pa.widget_pattern_id = wp.id
+JOIN widgets w ON w.hub_id = pa.adopting_hub_id
+    AND w.widget_type = wp.widget_type
+JOIN deployment_records dep ON dep.id IN (
+    SELECT dep2.id FROM deployment_records dep2
+    JOIN decision_records dr2 ON dr2.id = dep2.decision_id
+    JOIN requirements r2 ON r2.id = dr2.requirement_id
+    JOIN requirement_candidates rc2 ON rc2.id = r2.candidate_id
+    WHERE rc2.source_widget_id = w.id
+)
+JOIN outcome_signals os ON os.deployment_id = dep.id
+WHERE pa.adopting_hub_id = ?
+GROUP BY wp.id
+```
+
+Writes `PatternPerformanceRecord` per pattern, computes `outcome_rank` via
+`RANK() OVER (ORDER BY positive_outcome_count::float / NULLIF(total_outcome_count,0) DESC)`.
+
+Update `Web/Controller/MarketplaceDashboard.hs`: if `PatternPerformanceRecord`
+exists for a pattern, use `outcome_rank` for sort order.
+
+---
+
+### T04 — Adaptive friction thresholds
+
+```task
+id: IHUB-WP-0013-T04
+status: todo
+priority: medium
+state_hub_task_id: "a1de1a6b-14aa-4a3c-a103-d2630b762d30"
+```
+
+**`CalibrateThresholdsAction { hubId }`** in
+`Web/Controller/AdaptiveThresholds.hs`:
+
+1. Query `OutcomeCorrelation` records for the hub — find which annotation
+   categories have `correlation_score < 0.3` (weak predictors)
+2. Compute a `bottleneck_threshold_override` = mean `friction_score` for
+   widgets with negative outcomes only
+3. Upsert `AdaptiveThresholdConfig` for the hub
+4. Write `LearningInsight` of type `threshold_calibration`
+
+Update `Application/Helper/FrictionScore.hs`:
+
+```haskell
+-- | Read per-hub AdaptiveThresholdConfig and apply weight_overrides
+-- to friction component scores before summing. Falls back to global
+-- defaults when no config exists for the hub.
+applyAdaptiveWeights ::
+    (?modelContext :: ModelContext) =>
+    Id Hub -> FrictionComponents -> IO Double
+applyAdaptiveWeights hubId components = do
+    mConfig <- query @AdaptiveThresholdConfig
+        |> filterWhere (#hubId, hubId)
+        |> fetchOneOrNothing
+    let overrides = maybe mempty (.weightOverrides) mConfig
+    pure $ computeWeightedScore overrides components
+```
+
+**Views:** `Web/View/AdaptiveThresholds/{Index,Show}.hs`
+
+---
+
+### T05 — Institutional knowledge base
+
+```task
+id: IHUB-WP-0013-T05
+status: todo
+priority: medium
+state_hub_task_id: "16f03f8e-e664-4589-bdba-45cfed638595"
+```
+
+**`DistilDecisionAction { decisionRecordId }`** — appended to
+`Web/Controller/DecisionRecords.hs`:
+
+```haskell
+action DistilDecisionAction { decisionRecordId } = do
+    record <- fetch decisionRecordId
+    outcomes <- sqlQuery
+        "SELECT os.signal_type, os.value FROM outcome_signals os
+         JOIN deployment_records dep ON dep.id = os.deployment_id
+         WHERE dep.decision_id = ?" [decisionRecordId]
+    let prompt = "Distil this decision into a 2-3 sentence institutional
+                  knowledge entry. Include the outcome data.\n\nDecision: "
+               <> record.title <> "\nRationale: " <> record.rationale
+               <> "\nOutcome: " <> record.outcome
+               <> "\nSignals: " <> show (outcomes :: [(Text, Double)])
+    mAgent <- resolveAgent record.hubId "synthesis"
+    ...
+    newRecord @InstitutionalKnowledgeEntry
+        |> set #hubId record.hubId
+        |> set #decisionRecordId (Just decisionRecordId)
+        |> set #summary content
+        |> set #tags (A.toJSON ["decision" :: Text])
+        |> createRecord
+```
+
+**`QueryKnowledgeBaseAction`** — full-text search:
+
+```sql
+SELECT id, hub_id, decision_record_id, summary, tags, created_at
+FROM institutional_knowledge_entries
+WHERE hub_id = ?
+  AND summary_tsv @@ plainto_tsquery('english', ?)
+ORDER BY ts_rank(summary_tsv, plainto_tsquery('english', ?)) DESC
+LIMIT 20
+```
+
+**Views:** `Web/View/InstitutionalKnowledge/{Index,Show}.hs` with search form.
+
+---
+
+### T06 — Retroactive lineage enrichment
+
+```task
+id: IHUB-WP-0013-T06
+status: todo
+priority: medium
+state_hub_task_id: "cad61a11-7fdb-4e69-9dba-bb0176b2afdb"
+```
+
+**PL/pgSQL trigger** (in migration SQL):
+
+```sql
+CREATE OR REPLACE FUNCTION enrich_lineage_on_outcome()
+RETURNS TRIGGER LANGUAGE plpgsql AS $$
+DECLARE
+    v_dep_id  UUID;
+    v_dec_id  UUID;
+    v_req_id  UUID;
+    v_cand_id UUID;
+    v_summary JSONB;
+BEGIN
+    -- Walk chain upward from the new outcome_signal
+    SELECT decision_id INTO v_dec_id
+    FROM deployment_records WHERE id = NEW.deployment_id;
+
+    IF v_dec_id IS NOT NULL THEN
+        v_summary := jsonb_build_object(
+            'signal_type', NEW.signal_type,
+            'value',       NEW.value,
+            'observed_at', NEW.observed_at
+        );
+
+        UPDATE decision_records
+        SET outcome_summary = COALESCE(outcome_summary, '[]'::jsonb) || v_summary
+        WHERE id = v_dec_id;
+
+        SELECT requirement_id INTO v_req_id
+        FROM decision_records WHERE id = v_dec_id;
+
+        IF v_req_id IS NOT NULL THEN
+            SELECT candidate_id INTO v_cand_id
+            FROM requirements WHERE id = v_req_id;
+
+            IF v_cand_id IS NOT NULL THEN
+                UPDATE requirement_candidates
+                SET outcome_summary = COALESCE(outcome_summary, '[]'::jsonb) || v_summary
+                WHERE id = v_cand_id;
+            END IF;
+        END IF;
+    END IF;
+
+    RETURN NEW;
+END;
+$$;
+
+CREATE TRIGGER trg_enrich_lineage
+AFTER INSERT ON outcome_signals
+FOR EACH ROW EXECUTE FUNCTION enrich_lineage_on_outcome();
+```
+
+**`EnrichLineageAction { hubId }`** in `Web/Controller/LineageEnrichment.hs`
+— batch on-demand version: queries existing outcome_signals for the hub and
+calls the same enrichment logic via a SQL function call.
+
+Update `/contracts/core/append-only-events-v1.md` to note that
+`outcome_signals` now has an AFTER INSERT trigger that enriches upstream
+records (read: the trigger never modifies outcome_signals itself).
+
+---
+
+### T07 — Learning dashboard
+
+```task
+id: IHUB-WP-0013-T07
+status: todo
+priority: medium
+state_hub_task_id: "4445282e-e87c-48fe-87ba-484da4121195"
+```
+
+**`Web/Controller/LearningDashboard.hs`** with `autoRefresh`:
+
+```haskell
+data ShowView = ShowView
+    { topCorrelations  :: ![OutcomeCorrelation]
+    , patternRankings  :: ![PatternPerformanceRecord]
+    , thresholdStatus  :: ![(Hub, Maybe AdaptiveThresholdConfig)]
+    , recentInsights   :: ![LearningInsight]
+    , knowledgeHighlights :: ![InstitutionalKnowledgeEntry]
+    }
+```
+
+Five panels:
+
+1. **Top annotation predictors** — `OutcomeCorrelation` top 10 by score,
+   with colour-coded bars (green ≥ 0.7, amber 0.4–0.7, red < 0.4)
+2. **Pattern performance ranking** — `PatternPerformanceRecord` top 10 by
+   `positive_outcome_rate`, with link to pattern show page
+3. **Adaptive threshold status** — per hub: last calibration date, drift
+   indicator (days since last calibration > 30 = amber)
+4. **Recent learning insights** — last 10 `LearningInsight` with type badge
+   and evidence link count
+5. **Knowledge base highlights** — 5 most recent
+   `InstitutionalKnowledgeEntry` with excerpt and link to full entry
+
+Add "Learning" nav link in `Web/FrontController.hs`.
+
+---
+
+### T08 — API v2: /outcome-correlations, /pattern-performance, /knowledge-base
+
+```task
+id: IHUB-WP-0013-T08
+status: todo
+priority: medium
+state_hub_task_id: "2b3e7f84-c8f6-42fc-bb0a-4c524efd1688"
+```
+
+**`Web/Controller/Api/V2/Learning.hs`**:
+
+- `GET /api/v2/outcome-correlations` — paginated; filter `?hub_id=`, `?category=`
+- `GET /api/v2/pattern-performance` — paginated; sort by `positive_outcome_rate`
+- `GET /api/v2/knowledge-base` — full-text search via `?q=`; paginated
+- `GET /api/v2/knowledge-base/{id}` — single entry
+
+All require `BearerAuth`. Add three schemas to `Web/Controller/Api/V2/OpenApi.hs`:
+`OutcomeCorrelation`, `PatternPerformanceRecord`, `InstitutionalKnowledgeEntry`.
+
+Update `Web/Types.hs`:
+```haskell
+data ApiV2LearningController
+    = ApiV2IndexOutcomeCorrelationsAction
+    | ApiV2IndexPatternPerformanceAction
+    | ApiV2IndexKnowledgeBaseAction
+    | ApiV2ShowKnowledgeBaseAction { knowledgeEntryId :: !(Id InstitutionalKnowledgeEntry) }
+    deriving (Eq, Show, Data)
+```
+
+---
+
+### T09 — GAAF scorecard + CLAUDE.md + workplan done
+
+```task
+id: IHUB-WP-0013-T09
+status: todo
+priority: medium
+state_hub_task_id: "a9048aeb-5e4b-49e5-b8f5-159ede9ab04c"
+```
+
+**`ARCHITECTURE-LAYERS.md`** scorecard update:
+
+- Core: 3.8 → 3.9 (lineage enrichment trigger + outcome_summary columns;
+  contracts updated to document them)
+- Functional: 3.6 → 3.8 (outcome correlation + adaptive thresholds close
+  the long-range feedback loop; learning dashboard makes insights visible)
+- Target overall: ≥ 3.75
+
+Decisions Log entries:
+- Trigger-based lineage enrichment over polling (AFTER INSERT, zero app-layer overhead)
+- GIN tsvector over pgvector for knowledge base search (no extension dependency, sufficient for keyword queries)
+- `outcome_summary` as JSONB append (not a normalised table) to avoid joins on already-deep traceability queries
+
+**`CLAUDE.md`**: Phase 12 complete → active workplan cleared (IHF v0.2 complete).
+
+**Commit** all changes. **Mark workplan `status: done`.**