Document measurement loop plan and ecosystem integration strategy.
Persist INTENT and ecosystem assessments in history/, add ADR-004 for project metrics with Helix Forge correlation, and register WP-0003 and WP-0004 workplans with State Hub. Update SCOPE, README, and agency-framework docs to reflect the two-layer measurement model.
This commit is contained in:
142
history/2026-06-16-ecosystem-assessment.md
Normal file
142
history/2026-06-16-ecosystem-assessment.md
Normal file
@@ -0,0 +1,142 @@
|
||||
# KaizenAgentic Ecosystem Assessment
|
||||
|
||||
**Date:** 2026-06-16
|
||||
**Compared repos:** info-tech-canon, agentic-resources, activity-core, llm-connect, identity-canon, phase-memory, artifact-store, domain-tree, kontextual-engine, tele-mcp
|
||||
**Against:** `INTENT.md`, `wiki/`, WP-0003 measurement loop plan
|
||||
|
||||
---
|
||||
|
||||
## Strategic Insight
|
||||
|
||||
INTENT's vision is **distributed across the ecosystem**, not missing from a single repo:
|
||||
|
||||
| INTENT promise | Primary owner |
|
||||
|----------------|---------------|
|
||||
| Agent definitions + deployment | kaizen-agentic |
|
||||
| Project memory + Coach | kaizen-agentic |
|
||||
| Per-agent metrics + optimizer | kaizen-agentic (WP-0003) |
|
||||
| Session capture + fleet metrics | agentic-resources (Helix Forge) |
|
||||
| Scheduled improvement triggers | activity-core |
|
||||
| Evidence retention | artifact-store |
|
||||
| Rich memory graphs | phase-memory (future) |
|
||||
| Guidance as knowledge | kontextual-engine + info-tech-canon |
|
||||
| Semantic vocabulary | info-tech-canon, identity-canon |
|
||||
| Org placement | domain-tree |
|
||||
| Runtime telemetry MCP | tele-mcp (unassessed — not cloned) |
|
||||
|
||||
KaizenAgentic matures by **stabilizing conventions and composing adjacent services**, consistent with INTENT boundaries.
|
||||
|
||||
---
|
||||
|
||||
## Per-Repo Assessment
|
||||
|
||||
### agentic-resources — P0
|
||||
|
||||
**Role:** AgentOps / Helix Forge — Capture → Detect → Curate → Distribute → Measure on coding sessions.
|
||||
|
||||
**Use:** Fleet-level session metrics (`session_memory/measure/`), JSONL baselines, cross-agent adapters (Claude/Codex/Grok). Complements project-scoped `.kaizen/metrics/`.
|
||||
|
||||
**Action:** ADR-004 correlation fields; WP-0004 integration; do not re-implement session ingestion here.
|
||||
|
||||
### activity-core — P1
|
||||
|
||||
**Role:** Event bridge — cron/NATS → task emission.
|
||||
|
||||
**Use:** Scheduled `metrics optimize`, retention hygiene, metrics scaffold validation after agent install.
|
||||
|
||||
**Action:** WP-0004 ActivityDefinitions after WP-0003 Part 2.
|
||||
|
||||
### artifact-store — P1
|
||||
|
||||
**Role:** Artifact registry + retention gateway.
|
||||
|
||||
**Use:** Persist optimizer `analysis.json`, recommendations, e2e evidence packages.
|
||||
|
||||
**Action:** WP-0004 pilot registration with `raw-evidence` retention class.
|
||||
|
||||
### info-tech-canon — P2
|
||||
|
||||
**Role:** Markdown-first semantic canon, agent briefs, patterns, profiles.
|
||||
|
||||
**Use:** Map KaizenAgentTemplate → canon profiles; publish per-agent briefs; validation rules for `kaizen-agentic validate`.
|
||||
|
||||
**Action:** WP-0004 Part 4 (later phase).
|
||||
|
||||
### phase-memory — P2
|
||||
|
||||
**Role:** Profile-driven memory graphs (ephemeral → rigid).
|
||||
|
||||
**Use:** Upgrade path from flat `.kaizen/agents/*/memory.md`.
|
||||
|
||||
**Action:** Future WP after WP-0004; no WP-0003 blocker.
|
||||
|
||||
### kontextual-engine — P2
|
||||
|
||||
**Role:** Knowledge operations engine.
|
||||
|
||||
**Use:** Ingest `wiki/` and `agents/` as knowledge assets; KaizenGuidance catalog runtime.
|
||||
|
||||
**Action:** WP-0004 Part 4 (guidance pilot).
|
||||
|
||||
### llm-connect — P3
|
||||
|
||||
**Role:** Provider-neutral LLM adapter.
|
||||
|
||||
**Use:** Automated Coach/optimizer narration when LLM synthesis moves beyond CLI context assembly.
|
||||
|
||||
**Action:** Reference pattern; adopt when WP-0003+ adds LLM-powered recommendations.
|
||||
|
||||
### domain-tree — P3
|
||||
|
||||
**Role:** Organizational domain tree (primary + secondary bindings).
|
||||
|
||||
**Use:** Register kaizen-agentic and agent categories in org structure.
|
||||
|
||||
**Action:** When capability catalog matures.
|
||||
|
||||
### identity-canon — P3
|
||||
|
||||
**Role:** Identity/agent terminology research.
|
||||
|
||||
**Use:** Distinguish agent persona vs instance vs session actor for "digital talent agency" framing.
|
||||
|
||||
**Action:** Glossary alignment in wiki.
|
||||
|
||||
### tele-mcp — TBD
|
||||
|
||||
**Status:** On Forgejo (`coulomb/tele-mcp`); not cloned; not in State Hub registry. Name suggests telemetry MCP.
|
||||
|
||||
**Action:** Clone and assess before integration; candidate for WP-0001 T04 telemetry adapter.
|
||||
|
||||
---
|
||||
|
||||
## Two-Layer Measurement Model
|
||||
|
||||
| Layer | Scope | Owner | Storage |
|
||||
|-------|-------|-------|---------|
|
||||
| **Fleet** | Cross-repo session outcomes | agentic-resources | Helix Forge store + `measure/baselines.jsonl` |
|
||||
| **Project** | Per-agent persona performance in one repo | kaizen-agentic | `.kaizen/metrics/<agent>/executions.jsonl` |
|
||||
|
||||
Correlation via shared fields defined in ADR-004 (`helix_session_uid`, `repo`, `success`, `tokens`, `execution_time_s`).
|
||||
|
||||
See `wiki/EcosystemIntegration.md` for integration contracts.
|
||||
|
||||
---
|
||||
|
||||
## Priority Matrix
|
||||
|
||||
| Priority | Repo | WP |
|
||||
|----------|------|-----|
|
||||
| P0 | agentic-resources | WP-0004 Part 1 |
|
||||
| P1 | activity-core | WP-0004 Part 2 |
|
||||
| P1 | artifact-store | WP-0004 Part 3 |
|
||||
| P2 | info-tech-canon, kontextual-engine, phase-memory | WP-0004 Part 4 / future |
|
||||
| P3 | llm-connect, domain-tree, identity-canon | Adopt as needed |
|
||||
| TBD | tele-mcp | Assess when cloned |
|
||||
|
||||
---
|
||||
|
||||
## Follow-Up Workplans
|
||||
|
||||
- **KAIZEN-WP-0003** — measurement loop (active, State Hub registered)
|
||||
- **KAIZEN-WP-0004** — ecosystem integration (active, depends on WP-0003 Part 3)
|
||||
87
history/2026-06-16-intent-gap-analysis.md
Normal file
87
history/2026-06-16-intent-gap-analysis.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# KaizenAgentic Intent Gap Analysis
|
||||
|
||||
**Date:** 2026-06-16
|
||||
**Scope:** `INTENT.md`, `wiki/`, codebase (`agents/`, `src/kaizen_agentic/`, `docs/`, workplans)
|
||||
**Author:** kaizen-agentic session assessment
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
Kaizen-agentic is in a **two-layer state**: the strategic/conceptual layer (`INTENT.md`, `wiki/`) is well-developed; the operational layer (agents, CLI, agency framework) is substantial but implements a **deployment and memory** product more than a **measurable continuous-improvement engine**.
|
||||
|
||||
The largest gap: the **measurement → optimization → specification refinement loop** described in INTENT is largely unbuilt. Addressed by **KAIZEN-WP-0003** (registered 2026-06-16).
|
||||
|
||||
---
|
||||
|
||||
## Alignment
|
||||
|
||||
| INTENT asset | Status |
|
||||
|--------------|--------|
|
||||
| Mission and conceptual model | `wiki/` established |
|
||||
| KaizenAgent definition template | `wiki/KaizenAgentTemplate.md` — not enforced in agents |
|
||||
| Meta-optimizer concept | `wiki/AgentKaizenOptimizer.md` + `agent-optimization.md` — no data pipeline |
|
||||
| Idempotent/measurable principles | Documented; not in agent implementations |
|
||||
| Codebase improvement guidance | `wiki/KaizenGuidance.md` — vision only |
|
||||
| Prompts/experiments/mantras | `wiki/KaizenPrompting.md` — not operationalized |
|
||||
| Product/pricing/brand | `wiki/` complete |
|
||||
| Agency memory + Coach | WP-0002 shipped |
|
||||
| CLI deployment | Functional (21 agents) |
|
||||
|
||||
---
|
||||
|
||||
## Critical Gaps
|
||||
|
||||
### 1. Kaizen loop not closed
|
||||
|
||||
INTENT requires evidence-based refinement with before/after deltas. Reality: `OptimizationLoop` exists but is unwired; no `.kaizen/metrics/`; WP-0001 telemetry unstarted.
|
||||
|
||||
### 2. Agent template not enforced
|
||||
|
||||
Agents use minimal YAML frontmatter; `wiki/KaizenAgentTemplate.md` (metrics, idempotency, testing, evolution) is reference only.
|
||||
|
||||
### 3. KaizenGuidance unbuilt
|
||||
|
||||
No guide catalog, manifests, codemods, or Parse→Measure pipeline.
|
||||
|
||||
### 4. Coach vs optimizer not integrated
|
||||
|
||||
Qualitative memory (Coach) and quantitative optimization (optimizer) are separate paths.
|
||||
|
||||
### 5. Agent implementation boundary undeclared
|
||||
|
||||
INTENT says repo should not own all concrete agent implementations; 21 agents live here as reference fleet — interim state needs explicit policy.
|
||||
|
||||
---
|
||||
|
||||
## Design Principles Scorecard
|
||||
|
||||
| Principle | Status |
|
||||
|-----------|--------|
|
||||
| Continuous Improvement | Partial (memory; no automated refinement) |
|
||||
| Measurable by Default | Gap |
|
||||
| Idempotent Operations | Gap |
|
||||
| Evidence over Intuition | Gap |
|
||||
| Separation of Concerns | Partial |
|
||||
| Composable Capabilities | Gap |
|
||||
| Human-Readable + Machine-Executable | Gap (guidance) |
|
||||
| Rollback-Ready Evolution | Partial |
|
||||
| Compounding Value | Partial (memory only) |
|
||||
|
||||
---
|
||||
|
||||
## Remediation Sequence
|
||||
|
||||
1. **WP-0003** — metrics convention, CLI, optimizer wiring, Coach bridge (active)
|
||||
2. **WP-0004** — ecosystem integration (agentic-resources, activity-core, artifact-store)
|
||||
3. Future — KaizenGuidance catalog, phase-memory upgrade, full template conformance
|
||||
|
||||
---
|
||||
|
||||
## Related Artifacts
|
||||
|
||||
- `SCOPE.md` — updated 2026-06-16
|
||||
- `workplans/kaizen-agentic-WP-0003-measurement-loop.md`
|
||||
- `history/2026-06-16-ecosystem-assessment.md`
|
||||
- `wiki/EcosystemIntegration.md`
|
||||
- `docs/adr/ADR-004-project-metrics-convention.md`
|
||||
11
history/README.md
Normal file
11
history/README.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# History
|
||||
|
||||
Persisted assessments, gap analyses, and ecosystem reviews for KaizenAgentic.
|
||||
|
||||
| Date | Document | Summary |
|
||||
|------|----------|---------|
|
||||
| 2026-06-16 | [2026-06-16-intent-gap-analysis.md](2026-06-16-intent-gap-analysis.md) | INTENT.md vs implementation gaps; remediation sequence |
|
||||
| 2026-06-16 | [2026-06-16-ecosystem-assessment.md](2026-06-16-ecosystem-assessment.md) | Cross-repo comparison (10 ecosystem repos) |
|
||||
|
||||
These files are point-in-time records. Living conventions live in `INTENT.md`,
|
||||
`SCOPE.md`, `wiki/`, and `docs/adr/`.
|
||||
Reference in New Issue
Block a user