generated from coulomb/repo-seed
Compare commits
4 Commits
4b7a628b6f
...
70433cda61
| Author | SHA1 | Date | |
|---|---|---|---|
| 70433cda61 | |||
| 56b2f576de | |||
| d06791f070 | |||
| 519e76442a |
25
AGENTS.md
25
AGENTS.md
@@ -11,6 +11,31 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Dev Workflow
|
||||||
|
|
||||||
|
The deliverable code lives in `session_memory/` (the Helix Forge coding-session
|
||||||
|
memory system). It is **pure-stdlib Python 3.11+** — `tomllib`, `sqlite3`,
|
||||||
|
`dataclasses`; no third-party runtime dependencies and no build step. `pytest` is
|
||||||
|
the only dev dependency. Run everything from the repo root.
|
||||||
|
|
||||||
|
| Need | Command |
|
||||||
|
|------|---------|
|
||||||
|
| Python | `python3` (3.11+ required for `tomllib`; developed on 3.12) |
|
||||||
|
| Install deps | none at runtime; for tests: `pip install pytest` (or `uv pip install pytest`) |
|
||||||
|
| Test | `python3 -m pytest` (full suite) · `python3 -m pytest tests/test_curate_review.py` (one file) · `-q` for quiet |
|
||||||
|
| Lint / build | none configured — keep changes matching surrounding style |
|
||||||
|
| Run: ingest sweep | `python3 -m session_memory.ingest` (`--dry-run`, `--config PATH`) |
|
||||||
|
| Run: detect | `python3 -m session_memory.detect` (`--json`, `--min-frequency N`) |
|
||||||
|
| Run: curate | `python3 -m session_memory.curate` (`--auto-approve`, `--json`) |
|
||||||
|
| Config | `session_memory/config.toml`; local store under `session_memory/.store/` (gitignored) |
|
||||||
|
|
||||||
|
**Verify a change before declaring it done:** run `python3 -m pytest` (expect all
|
||||||
|
green), and for pipeline changes do a live `ingest → detect → curate` pass against
|
||||||
|
the local store. See `session_memory/README.md` for the full layout and the
|
||||||
|
detect → curate → distribute flow.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## State Hub Integration
|
## State Hub Integration
|
||||||
|
|
||||||
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
|
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
|
||||||
|
|||||||
@@ -255,12 +255,26 @@ record:
|
|||||||
three flavors?
|
three flavors?
|
||||||
- **OQ3** Where does detection logic run — local batch jobs, hub-side, or a dedicated
|
- **OQ3** Where does detection logic run — local batch jobs, hub-side, or a dedicated
|
||||||
service? What volume do we actually expect?
|
service? What volume do we actually expect?
|
||||||
- **OQ4** Pattern format: how do we keep one agnostic representation while giving each
|
- ~~**OQ4** Pattern format: how do we keep one agnostic representation while giving each
|
||||||
distributor enough to render high-quality native artifacts?
|
distributor enough to render high-quality native artifacts?~~ **Resolved (Phase 2,
|
||||||
- **OQ5** What's the minimum trustworthy evidence bar before a pattern is allowed to be
|
AGENTIC-WP-0004):** the `SolutionPattern` core is flavor-agnostic (problem,
|
||||||
distributed to live agent environments?
|
resolutions, scope, provenance) and carries per-flavor knowledge only in a separate
|
||||||
- **OQ6** How do we prevent pattern bloat — too many low-value instructions degrading
|
`rendering_hints` sub-structure keyed by flavor — distributors read the hints, the
|
||||||
agent context budgets (cf. the token-budget policy in global instructions)?
|
core stays neutral. Catalogued as versioned files-first artifacts (FR-U3).
|
||||||
|
- ~~**OQ5** What's the minimum trustworthy evidence bar before a pattern is allowed to be
|
||||||
|
distributed to live agent environments?~~ **Resolved (Phase 2):** a two-tier
|
||||||
|
evidence bar (`[curate.gate]`). A *promote* floor (frequency / distinct sessions /
|
||||||
|
cost-impact) admits a candidate as `provisional`; a stricter *distribution* floor
|
||||||
|
(higher frequency, optional cross-flavor requirement, cost-impact) is required to
|
||||||
|
mark a pattern `approved` + `distribution_ready`. Defaults are conservative and
|
||||||
|
config-tunable.
|
||||||
|
- ~~**OQ6** How do we prevent pattern bloat — too many low-value instructions degrading
|
||||||
|
agent context budgets (cf. the token-budget policy in global instructions)?~~
|
||||||
|
**Resolved (Phase 2):** a bloat guard flags duplicate (same id) and near-duplicate
|
||||||
|
(same signal-type+locus) candidates at review time, and the catalog dedups
|
||||||
|
structurally on the source-candidate key so re-promotion never multiplies entries.
|
||||||
|
Thin candidates stay `provisional` (not distributed) rather than padding live
|
||||||
|
context.
|
||||||
|
|
||||||
## 13. Risks
|
## 13. Risks
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,14 @@ session_memory/
|
|||||||
detect/signals.py # signal extractors over digests
|
detect/signals.py # signal extractors over digests
|
||||||
detect/cluster.py # cluster signals -> candidate patterns + cross-flavor flag
|
detect/cluster.py # cluster signals -> candidate patterns + cross-flavor flag
|
||||||
detect/__main__.py # python -m session_memory.detect (ranked report)
|
detect/__main__.py # python -m session_memory.detect (ranked report)
|
||||||
config.toml # store paths, retention caps, sources, repo->domain map
|
curate/schema.py # SolutionPattern artifact + per-flavor rendering hints
|
||||||
|
curate/catalog.py # versioned, files-first Pattern Catalog (dedup on id)
|
||||||
|
curate/gating.py # promotion evidence bar + bloat guard
|
||||||
|
curate/review.py # discuss/approve/reject -> promote workflow
|
||||||
|
curate/decisions.py # hub decision audit trail (graceful local-queue fallback)
|
||||||
|
curate/__main__.py # python -m session_memory.curate (interactive / --auto-approve)
|
||||||
|
catalog/ # the committed Pattern Catalog (source of truth)
|
||||||
|
config.toml # store paths, retention caps, sources, repo->domain map, curate gate
|
||||||
```
|
```
|
||||||
|
|
||||||
The local store lives under `session_memory/.store/` (gitignored).
|
The local store lives under `session_memory/.store/` (gitignored).
|
||||||
@@ -71,6 +78,42 @@ Candidates are persisted to a Tier 2 `patterns` table and are the input to the
|
|||||||
Curate phase (Phase 2). Patterns whose evidence spans more than one agent flavor
|
Curate phase (Phase 2). Patterns whose evidence spans more than one agent flavor
|
||||||
are flagged `[CROSS-FLAVOR]` — the highest-value reuse targets.
|
are flagged `[CROSS-FLAVOR]` — the highest-value reuse targets.
|
||||||
|
|
||||||
|
## Curate candidates into the Pattern Catalog
|
||||||
|
|
||||||
|
Review detect candidates into versioned **Solution Patterns** held in the
|
||||||
|
files-first catalog (`session_memory/catalog/`). The flow is **detect → curate →
|
||||||
|
(Phase 3) distribute**; `curate` refreshes candidates by running detect first.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m session_memory.curate # interactive review (a/r/d per candidate)
|
||||||
|
python -m session_memory.curate --auto-approve # batch: promote all that clear the evidence bar
|
||||||
|
python -m session_memory.curate --json # machine-readable result
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Promotion** writes a `SolutionPattern` file (id = source candidate key, so
|
||||||
|
re-promoting the same candidate dedups; content changes bump the semver and
|
||||||
|
archive the prior version to `<id>.history.jsonl`).
|
||||||
|
- The **evidence bar** (`[curate.gate]`) sets two floors: a promote floor and a
|
||||||
|
stricter *distribution* floor. A thin-but-real candidate lands `provisional`;
|
||||||
|
one clearing the distribution floor lands `approved` + `distribution_ready`.
|
||||||
|
- A **bloat guard** flags duplicate / near-duplicate candidates so the catalog
|
||||||
|
stays lean.
|
||||||
|
- Re-review is **idempotent** — a remembered decision is skipped unless the
|
||||||
|
candidate's evidence changed; a prior reject is not re-surfaced.
|
||||||
|
- Each final promote/reject is recorded as a **hub decision**; if the hub is
|
||||||
|
offline the decision is queued to `[curate].decision_queue` for later sync
|
||||||
|
(the same after-the-fact pattern used in Phase 1).
|
||||||
|
|
||||||
|
### Curate knobs (`[curate]` / `[curate.gate]` in config.toml)
|
||||||
|
|
||||||
|
| Key | Meaning |
|
||||||
|
|-----|---------|
|
||||||
|
| `catalog_dir` | committed Pattern Catalog dir (source of truth) |
|
||||||
|
| `review_log` / `decision_queue` | remembered decisions + pending hub decisions (gitignored) |
|
||||||
|
| `min_frequency` / `min_sessions` / `min_cost_impact` | floor to promote at all |
|
||||||
|
| `dist_require_cross_flavor` | require cross-flavor evidence to be distribution-eligible |
|
||||||
|
| `dist_min_frequency` / `dist_min_cost_impact` | stricter floor for `distribution_ready` |
|
||||||
|
|
||||||
## Retention knobs (`[retention]` in config.toml)
|
## Retention knobs (`[retention]` in config.toml)
|
||||||
|
|
||||||
| Key | Meaning |
|
| Key | Meaning |
|
||||||
@@ -86,7 +129,7 @@ exists, except the explicitly-reported hard-cap overflow path.
|
|||||||
## Tests
|
## Tests
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python -m pytest # 26 tests: schema, adapter, store, digest, retention, ingest
|
python -m pytest # schema, adapters, store, digest, retention, ingest, detect, curate
|
||||||
```
|
```
|
||||||
|
|
||||||
## Status
|
## Status
|
||||||
@@ -95,5 +138,7 @@ python -m pytest # 26 tests: schema, adapter, store, digest, retention,
|
|||||||
adapter, ingest sweep.
|
adapter, ingest sweep.
|
||||||
- **Phase 1** (AGENTIC-WP-0003): Codex + Grok adapters, multi-file session merge,
|
- **Phase 1** (AGENTIC-WP-0003): Codex + Grok adapters, multi-file session merge,
|
||||||
and the Detect pipeline (signals → clustering → cross-flavor candidate patterns).
|
and the Detect pipeline (signals → clustering → cross-flavor candidate patterns).
|
||||||
- **Next — Phase 2 (Curate):** review/approve candidates into a versioned pattern
|
- **Phase 2** (AGENTIC-WP-0004): Curate — Solution Pattern schema, versioned
|
||||||
catalog. **Phase 3 (Distribute) / Phase 4 (Measure)** follow per the PRD.
|
files-first Pattern Catalog, discuss/approve/reject review with an evidence bar +
|
||||||
|
bloat guard, and hub-decision audit trail.
|
||||||
|
- **Next — Phase 3 (Distribute) / Phase 4 (Measure)** follow per the PRD.
|
||||||
|
|||||||
79
session_memory/catalog/sp-problem-abandoned-outcome.json
Normal file
79
session_memory/catalog/sp-problem-abandoned-outcome.json
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
{
|
||||||
|
"created_at": "2026-06-07T08:02:03Z",
|
||||||
|
"distribution_ready": true,
|
||||||
|
"id": "sp-problem-abandoned-outcome",
|
||||||
|
"name": "cross-flavor problem: abandoned",
|
||||||
|
"polarity": "problem",
|
||||||
|
"problem": "cross-flavor problem: abandoned",
|
||||||
|
"provenance": {
|
||||||
|
"detected_at": null,
|
||||||
|
"evidence": {
|
||||||
|
"cost_impact": 13.0,
|
||||||
|
"cross_flavor": true,
|
||||||
|
"flavors": [
|
||||||
|
"claude",
|
||||||
|
"grok"
|
||||||
|
],
|
||||||
|
"frequency": 13,
|
||||||
|
"key": "problem:abandoned:outcome",
|
||||||
|
"locus": "outcome",
|
||||||
|
"polarity": "problem",
|
||||||
|
"repos": [
|
||||||
|
"can-you-assist",
|
||||||
|
"llm-connect"
|
||||||
|
],
|
||||||
|
"score": 253.5,
|
||||||
|
"sessions": [
|
||||||
|
"claude:0510d5f4-956d-430a-9e89-6abc54f95b6a",
|
||||||
|
"claude:106fd234-949e-470d-a208-fe5ed8f14562",
|
||||||
|
"claude:377aba4f-8bbf-4760-90e9-469486ab0518",
|
||||||
|
"claude:4c606c31-beff-4a41-a325-ef63c9f8fb0e",
|
||||||
|
"claude:5bffe081-39fb-44cd-9966-4006f9235a0e",
|
||||||
|
"claude:60d3c947-eacf-49e9-b12c-ff8eb6b1c20b",
|
||||||
|
"claude:8f50f5b4-fbc4-4abe-9a7c-b25b2a713671",
|
||||||
|
"claude:95b1fe00-5d2e-482f-9618-fddf9cdbeb51",
|
||||||
|
"claude:c3e782ad-96b9-4cf1-9eb5-defdf3578426",
|
||||||
|
"claude:d75b2084-faec-40cf-aaf8-d7e0c026bde6",
|
||||||
|
"claude:f282058a-0a43-4fb8-87fc-1e67eaa3533c",
|
||||||
|
"grok:019e6103-af11-7a92-8e0b-5f40465d8223",
|
||||||
|
"grok:019e611e-0728-77d3-bb7a-8c5983e5058a"
|
||||||
|
],
|
||||||
|
"signal_type": "abandoned",
|
||||||
|
"title": "cross-flavor problem: abandoned"
|
||||||
|
},
|
||||||
|
"promoted_at": "2026-06-07T08:02:03Z",
|
||||||
|
"source_key": "problem:abandoned:outcome"
|
||||||
|
},
|
||||||
|
"rendering_hints": {
|
||||||
|
"claude": {
|
||||||
|
"note": "TODO: refine rendering",
|
||||||
|
"target": "CLAUDE.md"
|
||||||
|
},
|
||||||
|
"grok": {
|
||||||
|
"note": "TODO: refine rendering",
|
||||||
|
"target": "instructions"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"resolutions": [
|
||||||
|
{
|
||||||
|
"detail": "",
|
||||||
|
"steps": [],
|
||||||
|
"summary": "TODO: capture the recommended resolution"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"schema_version": 1,
|
||||||
|
"scope": {
|
||||||
|
"domains": [],
|
||||||
|
"flavors": [
|
||||||
|
"claude",
|
||||||
|
"grok"
|
||||||
|
],
|
||||||
|
"repos": [
|
||||||
|
"can-you-assist",
|
||||||
|
"llm-connect"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status": "approved",
|
||||||
|
"updated_at": "2026-06-07T08:02:03Z",
|
||||||
|
"version": "1.0.0"
|
||||||
|
}
|
||||||
78
session_memory/catalog/sp-problem-budget_overrun-tokens.json
Normal file
78
session_memory/catalog/sp-problem-budget_overrun-tokens.json
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
{
|
||||||
|
"created_at": "2026-06-07T08:02:03Z",
|
||||||
|
"distribution_ready": true,
|
||||||
|
"id": "sp-problem-budget_overrun-tokens",
|
||||||
|
"name": "problem: budget overrun",
|
||||||
|
"polarity": "problem",
|
||||||
|
"problem": "problem: budget overrun",
|
||||||
|
"provenance": {
|
||||||
|
"detected_at": null,
|
||||||
|
"evidence": {
|
||||||
|
"cost_impact": 27.135,
|
||||||
|
"cross_flavor": false,
|
||||||
|
"flavors": [
|
||||||
|
"claude"
|
||||||
|
],
|
||||||
|
"frequency": 8,
|
||||||
|
"key": "problem:budget_overrun:tokens",
|
||||||
|
"locus": "tokens",
|
||||||
|
"polarity": "problem",
|
||||||
|
"repos": [
|
||||||
|
"activity-core",
|
||||||
|
"artifact-store",
|
||||||
|
"citation-evidence",
|
||||||
|
"flex-auth",
|
||||||
|
"infospace-bench",
|
||||||
|
"railiance-apps",
|
||||||
|
"vergabe-teilnahme"
|
||||||
|
],
|
||||||
|
"score": 217.08,
|
||||||
|
"sessions": [
|
||||||
|
"claude:0ef1b45c-5c27-4e20-88b3-37daeaa24eca",
|
||||||
|
"claude:2c0d14e1-d089-4076-bf35-b134737a261d",
|
||||||
|
"claude:6e0d3d68-872b-4d93-bb09-0691e091314b",
|
||||||
|
"claude:8313f946-f008-4e98-9915-31950380e39e",
|
||||||
|
"claude:8fabd5ce-6a20-4412-9a8b-0f0763394a78",
|
||||||
|
"claude:a7b4a9b3-0942-4899-b502-e76b0013fc42",
|
||||||
|
"claude:b4ae9631-a7eb-42a6-acb1-c65b660c4b74",
|
||||||
|
"claude:bbcf1c2b-14be-40e4-826b-4b2b49b9d212"
|
||||||
|
],
|
||||||
|
"signal_type": "budget_overrun",
|
||||||
|
"title": "problem: budget overrun"
|
||||||
|
},
|
||||||
|
"promoted_at": "2026-06-07T08:02:03Z",
|
||||||
|
"source_key": "problem:budget_overrun:tokens"
|
||||||
|
},
|
||||||
|
"rendering_hints": {
|
||||||
|
"claude": {
|
||||||
|
"note": "TODO: refine rendering",
|
||||||
|
"target": "CLAUDE.md"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"resolutions": [
|
||||||
|
{
|
||||||
|
"detail": "",
|
||||||
|
"steps": [],
|
||||||
|
"summary": "TODO: capture the recommended resolution"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"schema_version": 1,
|
||||||
|
"scope": {
|
||||||
|
"domains": [],
|
||||||
|
"flavors": [
|
||||||
|
"claude"
|
||||||
|
],
|
||||||
|
"repos": [
|
||||||
|
"activity-core",
|
||||||
|
"artifact-store",
|
||||||
|
"citation-evidence",
|
||||||
|
"flex-auth",
|
||||||
|
"infospace-bench",
|
||||||
|
"railiance-apps",
|
||||||
|
"vergabe-teilnahme"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status": "approved",
|
||||||
|
"updated_at": "2026-06-07T08:02:03Z",
|
||||||
|
"version": "1.0.0"
|
||||||
|
}
|
||||||
106
session_memory/catalog/sp-success-clean_pass-outcome.json
Normal file
106
session_memory/catalog/sp-success-clean_pass-outcome.json
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
{
|
||||||
|
"created_at": "2026-06-07T08:02:03Z",
|
||||||
|
"distribution_ready": true,
|
||||||
|
"id": "sp-success-clean_pass-outcome",
|
||||||
|
"name": "cross-flavor success: clean pass",
|
||||||
|
"polarity": "success",
|
||||||
|
"problem": "cross-flavor success: clean pass",
|
||||||
|
"provenance": {
|
||||||
|
"detected_at": null,
|
||||||
|
"evidence": {
|
||||||
|
"cost_impact": 20.0,
|
||||||
|
"cross_flavor": true,
|
||||||
|
"flavors": [
|
||||||
|
"claude",
|
||||||
|
"grok"
|
||||||
|
],
|
||||||
|
"frequency": 20,
|
||||||
|
"key": "success:clean_pass:outcome",
|
||||||
|
"locus": "outcome",
|
||||||
|
"polarity": "success",
|
||||||
|
"repos": [
|
||||||
|
"activity-core",
|
||||||
|
"agentic-resources",
|
||||||
|
"artifact-store",
|
||||||
|
"can-you-assist",
|
||||||
|
"citation-evidence",
|
||||||
|
"infospace-bench",
|
||||||
|
"issue-facade",
|
||||||
|
"ops-bridge",
|
||||||
|
"railiance-apps",
|
||||||
|
"state-hub",
|
||||||
|
"the-custodian",
|
||||||
|
"vergabe-teilnahme"
|
||||||
|
],
|
||||||
|
"score": 600.0,
|
||||||
|
"sessions": [
|
||||||
|
"claude:0ef1b45c-5c27-4e20-88b3-37daeaa24eca",
|
||||||
|
"claude:16bdbec4-b018-4902-9fb5-336f8f3d61c8",
|
||||||
|
"claude:2c0d14e1-d089-4076-bf35-b134737a261d",
|
||||||
|
"claude:30dbad62-c042-41f2-80c1-5953a1100e7f",
|
||||||
|
"claude:39dd33b1-d156-4d6a-8c33-c359b6f841d8",
|
||||||
|
"claude:4307eff6-cd39-4189-be58-79a3acb69d6c",
|
||||||
|
"claude:4340b160-2fb6-47d0-897c-3cac0a8855d8",
|
||||||
|
"claude:631de76e-fdee-43b5-b091-7b7675467ad1",
|
||||||
|
"claude:63fd4df2-5add-4748-af21-c1544825e006",
|
||||||
|
"claude:6e0d3d68-872b-4d93-bb09-0691e091314b",
|
||||||
|
"claude:8313f946-f008-4e98-9915-31950380e39e",
|
||||||
|
"claude:8fabd5ce-6a20-4412-9a8b-0f0763394a78",
|
||||||
|
"claude:99e9c5af-043f-4b97-8d92-14189da8716b",
|
||||||
|
"claude:a7b4a9b3-0942-4899-b502-e76b0013fc42",
|
||||||
|
"claude:a9483f07-c9dc-4f71-9fa0-831790ea965e",
|
||||||
|
"claude:b4ae9631-a7eb-42a6-acb1-c65b660c4b74",
|
||||||
|
"claude:eb837dd1-5b8e-472e-b9e1-4537b10e03e6",
|
||||||
|
"claude:ee9e84f2-bc35-4eb5-a7ad-aaec5f31d965",
|
||||||
|
"claude:f1b25697-0e5f-45f0-81d1-af0f1762c438",
|
||||||
|
"grok:019e6122-00c0-79f3-b4e5-9c70b77c015d"
|
||||||
|
],
|
||||||
|
"signal_type": "clean_pass",
|
||||||
|
"title": "cross-flavor success: clean pass"
|
||||||
|
},
|
||||||
|
"promoted_at": "2026-06-07T08:02:03Z",
|
||||||
|
"source_key": "success:clean_pass:outcome"
|
||||||
|
},
|
||||||
|
"rendering_hints": {
|
||||||
|
"claude": {
|
||||||
|
"note": "TODO: refine rendering",
|
||||||
|
"target": "CLAUDE.md"
|
||||||
|
},
|
||||||
|
"grok": {
|
||||||
|
"note": "TODO: refine rendering",
|
||||||
|
"target": "instructions"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"resolutions": [
|
||||||
|
{
|
||||||
|
"detail": "",
|
||||||
|
"steps": [],
|
||||||
|
"summary": "TODO: capture the recommended resolution"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"schema_version": 1,
|
||||||
|
"scope": {
|
||||||
|
"domains": [],
|
||||||
|
"flavors": [
|
||||||
|
"claude",
|
||||||
|
"grok"
|
||||||
|
],
|
||||||
|
"repos": [
|
||||||
|
"activity-core",
|
||||||
|
"agentic-resources",
|
||||||
|
"artifact-store",
|
||||||
|
"can-you-assist",
|
||||||
|
"citation-evidence",
|
||||||
|
"infospace-bench",
|
||||||
|
"issue-facade",
|
||||||
|
"ops-bridge",
|
||||||
|
"railiance-apps",
|
||||||
|
"state-hub",
|
||||||
|
"the-custodian",
|
||||||
|
"vergabe-teilnahme"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"status": "approved",
|
||||||
|
"updated_at": "2026-06-07T08:02:03Z",
|
||||||
|
"version": "1.0.0"
|
||||||
|
}
|
||||||
@@ -31,10 +31,19 @@ enabled = true
|
|||||||
root = "~/.grok/sessions"
|
root = "~/.grok/sessions"
|
||||||
glob = "*/*/chat_history.jsonl"
|
glob = "*/*/chat_history.jsonl"
|
||||||
|
|
||||||
|
# Detect phase (AGENTIC-WP-0005): quality filter — drop non-coding/trivial sessions
|
||||||
|
# before signals form, so health-checks don't mint false-positive patterns.
|
||||||
|
[detect.quality]
|
||||||
|
min_events = 20 # below this many events, not a real coding session
|
||||||
|
min_substantive = 3 # require >= this many substantive (edit/read/shell) tool calls
|
||||||
|
min_prompt_len = 25 # first prompt shorter than this is treated as trivial
|
||||||
|
|
||||||
# Curate phase (AGENTIC-WP-0004): catalog location + promotion evidence bar.
|
# Curate phase (AGENTIC-WP-0004): catalog location + promotion evidence bar.
|
||||||
[curate]
|
[curate]
|
||||||
catalog_dir = "session_memory/catalog" # files-first Pattern Catalog (committed)
|
catalog_dir = "session_memory/catalog" # files-first Pattern Catalog (committed)
|
||||||
review_log = "session_memory/.store/reviews.jsonl" # remembered decisions (gitignored)
|
review_log = "session_memory/.store/reviews.jsonl" # remembered decisions (gitignored)
|
||||||
|
decision_queue = "session_memory/.store/decisions.queue.jsonl" # hub decisions pending sync
|
||||||
|
state_hub_workstream_id = "b3703684-f60e-42f3-b03e-dabe3e8ce3f4" # AGENTIC-WP-0004
|
||||||
|
|
||||||
# Evidence bar (OQ5): floors to promote at all, and stricter floors to be
|
# Evidence bar (OQ5): floors to promote at all, and stricter floors to be
|
||||||
# distribution-eligible (status=approved, distribution_ready=true).
|
# distribution-eligible (status=approved, distribution_ready=true).
|
||||||
|
|||||||
130
session_memory/curate/__main__.py
Normal file
130
session_memory/curate/__main__.py
Normal file
@@ -0,0 +1,130 @@
|
|||||||
|
"""Curate entrypoint (T06): review detect candidates into the Pattern Catalog.
|
||||||
|
|
||||||
|
python -m session_memory.curate [--config PATH] [--auto-approve] [--json]
|
||||||
|
[--workstream-id ID]
|
||||||
|
|
||||||
|
Refreshes candidate patterns (runs the detect pipeline), then drives them through
|
||||||
|
the review workflow — **interactive** by default, or **batch** with
|
||||||
|
``--auto-approve`` (promote everything clearing the evidence bar, reject the rest)
|
||||||
|
for kaizen-agent runs. Candidates are presented cross-flavor first (detect's
|
||||||
|
ranking). Emits a catalog diff summary and, with ``--json``, a machine-readable
|
||||||
|
result. Approvals land in the files-first catalog; each final decision is logged
|
||||||
|
as a hub decision (queued if the hub is down).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
from ..detect.__main__ import run_detect
|
||||||
|
from ..ingest import _expand, load_config
|
||||||
|
from .catalog import Catalog
|
||||||
|
from .decisions import DecisionRecorder
|
||||||
|
from .gating import bloat_warnings, evaluate, gate_config
|
||||||
|
from .review import APPROVE, DISCUSS, REJECT, ReviewLog, review
|
||||||
|
|
||||||
|
|
||||||
|
def _curate_paths(config: dict):
|
||||||
|
c = config.get("curate", {})
|
||||||
|
catalog_dir = _expand(c.get("catalog_dir", "session_memory/catalog"))
|
||||||
|
review_log = _expand(c.get("review_log", "session_memory/.store/reviews.jsonl"))
|
||||||
|
queue = _expand(c.get("decision_queue", "session_memory/.store/decisions.queue.jsonl"))
|
||||||
|
ws_id = c.get("state_hub_workstream_id")
|
||||||
|
return catalog_dir, review_log, queue, ws_id
|
||||||
|
|
||||||
|
|
||||||
|
def _render_candidate(cand: dict, gate, existing) -> str:
|
||||||
|
g = evaluate(cand, gate)
|
||||||
|
flag = " [CROSS-FLAVOR]" if cand.get("cross_flavor") else ""
|
||||||
|
lines = [
|
||||||
|
f"\n{cand['title']}{flag}",
|
||||||
|
f" key={cand['key']} score={cand.get('score')} freq={cand['frequency']} "
|
||||||
|
f"impact={cand.get('cost_impact')}",
|
||||||
|
f" flavors={','.join(cand.get('flavors', []))} "
|
||||||
|
f"repos={','.join(cand.get('repos', [])) or '-'} sessions={len(cand.get('sessions', []))}",
|
||||||
|
f" gate: promotable={g.promotable} distribution_ready={g.distribution_ready}"
|
||||||
|
+ (f" ({'; '.join(g.reasons)})" if g.reasons else ""),
|
||||||
|
]
|
||||||
|
for w in bloat_warnings(cand, existing):
|
||||||
|
lines.append(f" bloat: {w}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _interactive_decider(gate, catalog):
|
||||||
|
def decide(cand):
|
||||||
|
print(_render_candidate(cand, gate, catalog.list()))
|
||||||
|
while True:
|
||||||
|
choice = input(" [a]pprove / [r]eject / [d]iscuss ? ").strip().lower()
|
||||||
|
if choice in ("a", "approve"):
|
||||||
|
return (APPROVE, input(" rationale: ").strip() or "approved")
|
||||||
|
if choice in ("r", "reject"):
|
||||||
|
return (REJECT, input(" rationale: ").strip() or "rejected")
|
||||||
|
if choice in ("d", "discuss"):
|
||||||
|
return (DISCUSS, "deferred for discussion")
|
||||||
|
return decide
|
||||||
|
|
||||||
|
|
||||||
|
def _auto_decider(gate):
|
||||||
|
"""Batch policy: approve candidates clearing the promote floor, reject the rest."""
|
||||||
|
def decide(cand):
|
||||||
|
g = evaluate(cand, gate)
|
||||||
|
if g.promotable:
|
||||||
|
return (APPROVE, "auto-approved: clears evidence bar")
|
||||||
|
return (REJECT, "auto-rejected: " + "; ".join(g.reasons))
|
||||||
|
return decide
|
||||||
|
|
||||||
|
|
||||||
|
def _summary(result, n_candidates: int) -> str:
|
||||||
|
added = [k for k, a in result.approved if a in ("added", "versioned", "updated")]
|
||||||
|
lines = [
|
||||||
|
f"# Curate summary ({n_candidates} candidates reviewed)",
|
||||||
|
f" approved : {len(result.approved)} ({', '.join(f'{k}:{a}' for k, a in result.approved) or '-'})",
|
||||||
|
f" rejected : {len(result.rejected)} ({', '.join(result.rejected) or '-'})",
|
||||||
|
f" deferred : {len(result.deferred)} ({', '.join(result.deferred) or '-'})",
|
||||||
|
f" skipped : {len(result.skipped)} (already decided)",
|
||||||
|
f" catalog writes: {len(added)}",
|
||||||
|
]
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None) -> int:
|
||||||
|
here = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
ap = argparse.ArgumentParser(description="Curate detect candidates into the Pattern Catalog.")
|
||||||
|
ap.add_argument("--config", default=os.path.join(here, "config.toml"))
|
||||||
|
ap.add_argument("--auto-approve", action="store_true",
|
||||||
|
help="batch mode: promote everything clearing the evidence bar")
|
||||||
|
ap.add_argument("--min-frequency", type=int, default=2)
|
||||||
|
ap.add_argument("--workstream-id", default=None, help="hub workstream for decisions")
|
||||||
|
ap.add_argument("--json", action="store_true", help="emit machine-readable JSON")
|
||||||
|
args = ap.parse_args(argv)
|
||||||
|
|
||||||
|
config = load_config(args.config)
|
||||||
|
candidates = run_detect(config, min_frequency=args.min_frequency)
|
||||||
|
|
||||||
|
catalog_dir, review_log_path, queue_path, ws_id = _curate_paths(config)
|
||||||
|
gate = gate_config(config)
|
||||||
|
catalog = Catalog(catalog_dir)
|
||||||
|
log = ReviewLog(review_log_path)
|
||||||
|
recorder = DecisionRecorder(queue_path, workstream_id=args.workstream_id or ws_id)
|
||||||
|
|
||||||
|
decide = _auto_decider(gate) if args.auto_approve else _interactive_decider(gate, catalog)
|
||||||
|
result = review(candidates, decide, catalog, log, gate=gate, recorder=recorder)
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({
|
||||||
|
"approved": result.approved, "rejected": result.rejected,
|
||||||
|
"deferred": result.deferred, "skipped": result.skipped,
|
||||||
|
"decisions_queued": len(recorder.pending()),
|
||||||
|
}, indent=2))
|
||||||
|
else:
|
||||||
|
print(_summary(result, len(candidates)))
|
||||||
|
if recorder.pending():
|
||||||
|
print(f" decisions queued (hub offline): {len(recorder.pending())} "
|
||||||
|
f"-> {queue_path}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -16,13 +16,14 @@ import os
|
|||||||
from ..core.store import Store
|
from ..core.store import Store
|
||||||
from ..ingest import _expand, load_config
|
from ..ingest import _expand, load_config
|
||||||
from .cluster import cluster
|
from .cluster import cluster
|
||||||
|
from .quality import filter_real, quality_config
|
||||||
from .signals import extract_signals
|
from .signals import extract_signals
|
||||||
|
|
||||||
|
|
||||||
def run_detect(config: dict, *, min_frequency: int = 2) -> list[dict]:
|
def run_detect(config: dict, *, min_frequency: int = 2) -> list[dict]:
|
||||||
store_cfg = config.get("store", {})
|
store_cfg = config.get("store", {})
|
||||||
store = Store(_expand(store_cfg["db_path"]), _expand(store_cfg["blob_dir"]))
|
store = Store(_expand(store_cfg["db_path"]), _expand(store_cfg["blob_dir"]))
|
||||||
digests = store.list_digests()
|
digests = filter_real(store.list_digests(), quality_config(config))
|
||||||
signals = extract_signals(digests)
|
signals = extract_signals(digests)
|
||||||
patterns = [p.to_dict() for p in cluster(signals, min_frequency=min_frequency)]
|
patterns = [p.to_dict() for p in cluster(signals, min_frequency=min_frequency)]
|
||||||
store.save_patterns(patterns)
|
store.save_patterns(patterns)
|
||||||
@@ -56,7 +57,8 @@ def main(argv=None) -> int:
|
|||||||
|
|
||||||
config = load_config(args.config)
|
config = load_config(args.config)
|
||||||
store_cfg = config.get("store", {})
|
store_cfg = config.get("store", {})
|
||||||
n = len(Store(_expand(store_cfg["db_path"]), _expand(store_cfg["blob_dir"])).list_digests())
|
all_digests = Store(_expand(store_cfg["db_path"]), _expand(store_cfg["blob_dir"])).list_digests()
|
||||||
|
n = len(filter_real(all_digests, quality_config(config)))
|
||||||
patterns = run_detect(config, min_frequency=args.min_frequency)
|
patterns = run_detect(config, min_frequency=args.min_frequency)
|
||||||
|
|
||||||
if args.json:
|
if args.json:
|
||||||
|
|||||||
75
session_memory/detect/quality.py
Normal file
75
session_memory/detect/quality.py
Normal file
@@ -0,0 +1,75 @@
|
|||||||
|
"""Session-quality filter (T01).
|
||||||
|
|
||||||
|
The capture layer ingests *every* session it finds — including API health-checks,
|
||||||
|
smoke-tests, and interrupted runs (e.g. ``llm-connect`` firing "Say hello in one
|
||||||
|
word", or a transcript that is just ``[Request interrupted by user]``). These are
|
||||||
|
not real coding work, but the outcome heuristic labels the short ones ``abandoned``
|
||||||
|
and the clusterer then mints false-positive "problem" patterns from them.
|
||||||
|
|
||||||
|
:func:`is_real_coding_session` gates those out so Detect signals/clusters form only
|
||||||
|
over genuine coding sessions. It is intentionally conservative — a session counts
|
||||||
|
as real if it shows substantive activity, and is dropped only on clear trivial
|
||||||
|
markers. Thresholds come from ``[detect.quality]`` in ``config.toml``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
# Prompt prefixes/markers that indicate a non-coding or interrupted session.
|
||||||
|
_TRIVIAL_PROMPTS = (
|
||||||
|
"say hello", "hello", "[request interrupted", "return only this json",
|
||||||
|
"ping", "ok", "<system-reminder>",
|
||||||
|
)
|
||||||
|
|
||||||
|
# Tool buckets that count as "substantive" coding activity.
|
||||||
|
_SUBSTANTIVE_TOOLS = (
|
||||||
|
"Edit", "Write", "Read", "Bash", "search_replace", "write", "read_file",
|
||||||
|
"run_terminal_command", "grep", "Grep", "glob", "Glob", "NotebookEdit",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class QualityConfig:
|
||||||
|
min_events: int = 20 # below this, not a real coding session
|
||||||
|
min_substantive: int = 3 # >= this many substantive tool calls required
|
||||||
|
min_prompt_len: int = 25 # first prompt shorter than this is suspect
|
||||||
|
|
||||||
|
|
||||||
|
def quality_config(config: Optional[dict] = None) -> QualityConfig:
|
||||||
|
d = (config or {}).get("detect", {}).get("quality", {}) if config else {}
|
||||||
|
return QualityConfig(
|
||||||
|
min_events=d.get("min_events", 20),
|
||||||
|
min_substantive=d.get("min_substantive", 3),
|
||||||
|
min_prompt_len=d.get("min_prompt_len", 25),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _substantive_calls(digest: dict) -> int:
|
||||||
|
hist = digest.get("tool_histogram") or {}
|
||||||
|
return sum(n for t, n in hist.items() if t in _SUBSTANTIVE_TOOLS)
|
||||||
|
|
||||||
|
|
||||||
|
def is_real_coding_session(digest: dict, config: Optional[QualityConfig] = None) -> bool:
|
||||||
|
cfg = config or QualityConfig()
|
||||||
|
|
||||||
|
if not digest.get("repo"):
|
||||||
|
return False
|
||||||
|
if digest.get("event_count", 0) < cfg.min_events:
|
||||||
|
return False
|
||||||
|
if _substantive_calls(digest) < cfg.min_substantive:
|
||||||
|
return False
|
||||||
|
|
||||||
|
prompt = (digest.get("first_prompt") or "").strip().lower()
|
||||||
|
if len(prompt) < cfg.min_prompt_len:
|
||||||
|
return False
|
||||||
|
if any(prompt.startswith(p) for p in _TRIVIAL_PROMPTS):
|
||||||
|
return False
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def filter_real(digests: list[dict], config: Optional[QualityConfig] = None) -> list[dict]:
|
||||||
|
cfg = config or QualityConfig()
|
||||||
|
return [d for d in digests if is_real_coding_session(d, cfg)]
|
||||||
84
tests/test_curate_entrypoint.py
Normal file
84
tests/test_curate_entrypoint.py
Normal file
@@ -0,0 +1,84 @@
|
|||||||
|
"""Curate entrypoint tests (T06): batch auto-approve end-to-end via the store."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
from session_memory.core.store import Store # noqa: E402
|
||||||
|
from session_memory.curate.__main__ import main # noqa: E402
|
||||||
|
from session_memory.curate.catalog import Catalog # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(uid, flavor, repo, **markers):
|
||||||
|
return {
|
||||||
|
"session_uid": uid, "flavor": flavor, "repo": repo, "outcome": "fail",
|
||||||
|
"cost": {"input_tokens": 10, "output_tokens": 1},
|
||||||
|
"markers": {"errors": markers.get("errors", 0), "retries": markers.get("retries", 0),
|
||||||
|
"test_runs": 0, "edits": 0, "human_interventions": 0},
|
||||||
|
# real coding session per the quality filter (WP-0005 T01)
|
||||||
|
"event_count": 40, "first_prompt": "Fix the failing build and retry the suite",
|
||||||
|
"tool_histogram": {"Bash": 20, "Edit": 12, "Read": 8},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _write_config(tmp_path) -> str:
|
||||||
|
store = tmp_path / ".store"
|
||||||
|
catalog = tmp_path / "catalog"
|
||||||
|
cfg = f"""
|
||||||
|
[store]
|
||||||
|
db_path = "{store / 'm.db'}"
|
||||||
|
blob_dir = "{store / 'blobs'}"
|
||||||
|
cursor = "{store / 'c.json'}"
|
||||||
|
|
||||||
|
[curate]
|
||||||
|
catalog_dir = "{catalog}"
|
||||||
|
review_log = "{store / 'reviews.jsonl'}"
|
||||||
|
decision_queue = "{store / 'decisions.queue.jsonl'}"
|
||||||
|
|
||||||
|
[curate.gate]
|
||||||
|
min_frequency = 2
|
||||||
|
min_sessions = 2
|
||||||
|
"""
|
||||||
|
path = tmp_path / "config.toml"
|
||||||
|
path.write_text(cfg)
|
||||||
|
return str(path), str(store), str(catalog)
|
||||||
|
|
||||||
|
|
||||||
|
def test_auto_approve_promotes_cross_flavor(tmp_path, capsys):
|
||||||
|
cfg_path, store_dir, catalog_dir = _write_config(tmp_path)
|
||||||
|
st = Store(os.path.join(store_dir, "m.db"), os.path.join(store_dir, "blobs"))
|
||||||
|
st.write_digest("claude:a", _digest("claude:a", "claude", "r1", retries=5))
|
||||||
|
st.write_digest("codex:b", _digest("codex:b", "codex", "r2", retries=4))
|
||||||
|
st.close()
|
||||||
|
|
||||||
|
rc = main(["--config", cfg_path, "--auto-approve"])
|
||||||
|
assert rc == 0
|
||||||
|
|
||||||
|
cat = Catalog(catalog_dir)
|
||||||
|
patterns = cat.list()
|
||||||
|
assert len(patterns) == 1
|
||||||
|
assert patterns[0].polarity == "problem"
|
||||||
|
# clears the promote floor (freq>=2) but below the default distribution
|
||||||
|
# floor (freq>=3) -> promoted as provisional, not distribution-ready
|
||||||
|
assert patterns[0].status == "provisional"
|
||||||
|
assert patterns[0].distribution_ready is False
|
||||||
|
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "Curate summary" in out
|
||||||
|
# hub offline in tests -> decision queued
|
||||||
|
assert "decisions queued" in out
|
||||||
|
|
||||||
|
|
||||||
|
def test_rerun_is_idempotent(tmp_path):
|
||||||
|
cfg_path, store_dir, catalog_dir = _write_config(tmp_path)
|
||||||
|
st = Store(os.path.join(store_dir, "m.db"), os.path.join(store_dir, "blobs"))
|
||||||
|
st.write_digest("claude:a", _digest("claude:a", "claude", "r1", retries=5))
|
||||||
|
st.write_digest("codex:b", _digest("codex:b", "codex", "r2", retries=4))
|
||||||
|
st.close()
|
||||||
|
|
||||||
|
main(["--config", cfg_path, "--auto-approve"])
|
||||||
|
main(["--config", cfg_path, "--auto-approve"]) # second pass: already decided
|
||||||
|
cat = Catalog(catalog_dir)
|
||||||
|
assert len(cat.list()) == 1
|
||||||
|
assert cat.load(cat.list()[0].id).version == "1.0.0" # no spurious bump
|
||||||
@@ -15,6 +15,9 @@ def _digest(uid, flavor, repo, **markers):
|
|||||||
"cost": {"input_tokens": 10, "output_tokens": 1},
|
"cost": {"input_tokens": 10, "output_tokens": 1},
|
||||||
"markers": {"errors": markers.get("errors", 0), "retries": markers.get("retries", 0),
|
"markers": {"errors": markers.get("errors", 0), "retries": markers.get("retries", 0),
|
||||||
"test_runs": 0, "edits": 0, "human_interventions": 0},
|
"test_runs": 0, "edits": 0, "human_interventions": 0},
|
||||||
|
# fields the quality filter (WP-0005 T01) checks — real coding session
|
||||||
|
"event_count": 40, "first_prompt": "Fix the failing build and retry the suite",
|
||||||
|
"tool_histogram": {"Bash": 20, "Edit": 12, "Read": 8},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
61
tests/test_detect_quality.py
Normal file
61
tests/test_detect_quality.py
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
"""Session-quality filter tests (T01)."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
|
||||||
|
from session_memory.detect.quality import ( # noqa: E402
|
||||||
|
QualityConfig,
|
||||||
|
filter_real,
|
||||||
|
is_real_coding_session,
|
||||||
|
quality_config,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _digest(repo="agentic-resources", events=60, prompt="Implement the curate entrypoint",
|
||||||
|
tools=None):
|
||||||
|
return {
|
||||||
|
"session_uid": "claude:x", "flavor": "claude", "repo": repo,
|
||||||
|
"event_count": events, "first_prompt": prompt,
|
||||||
|
"tool_histogram": tools if tools is not None else {"Bash": 20, "Edit": 15, "Read": 8},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_session_passes():
|
||||||
|
assert is_real_coding_session(_digest()) is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_healthcheck_prompt_dropped():
|
||||||
|
assert is_real_coding_session(_digest(events=3, prompt="Say hello in one word.",
|
||||||
|
tools={})) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_interrupted_dropped():
|
||||||
|
assert is_real_coding_session(_digest(events=1, prompt="[Request interrupted by user]",
|
||||||
|
tools={})) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_too_short_dropped():
|
||||||
|
assert is_real_coding_session(_digest(events=5)) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_repo_dropped():
|
||||||
|
assert is_real_coding_session(_digest(repo=None)) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_substantive_tools_dropped():
|
||||||
|
# plenty of events but only plumbing calls -> not real coding
|
||||||
|
assert is_real_coding_session(
|
||||||
|
_digest(tools={"mcp__state-hub__update_task_status": 40})) is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_filter_real_keeps_only_real():
|
||||||
|
digs = [_digest(), _digest(events=3, prompt="hello", tools={}), _digest(repo=None)]
|
||||||
|
assert len(filter_real(digs)) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_quality_config_from_toml():
|
||||||
|
cfg = quality_config({"detect": {"quality": {"min_events": 50}}})
|
||||||
|
assert cfg.min_events == 50
|
||||||
|
assert cfg.min_substantive == 3 # default preserved
|
||||||
@@ -4,11 +4,11 @@ type: workplan
|
|||||||
title: "Bootstrap State Hub integration"
|
title: "Bootstrap State Hub integration"
|
||||||
domain: helix_forge
|
domain: helix_forge
|
||||||
repo: agentic-resources
|
repo: agentic-resources
|
||||||
status: ready
|
status: finished
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: helix-forge
|
topic_slug: helix-forge
|
||||||
created: "2026-06-06"
|
created: "2026-06-06"
|
||||||
updated: "2026-06-06"
|
updated: "2026-06-07"
|
||||||
state_hub_workstream_id: "bb9a43a3-a54f-434b-97c2-e1c7142b52f5"
|
state_hub_workstream_id: "bb9a43a3-a54f-434b-97c2-e1c7142b52f5"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -20,7 +20,7 @@ Iterating towards optimal agentic performance.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: AGENTIC-WP-0001-T01
|
id: AGENTIC-WP-0001-T01
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "3ad7b7a9-ee5b-40e7-bd83-d6f0d1db7867"
|
state_hub_task_id: "3ad7b7a9-ee5b-40e7-bd83-d6f0d1db7867"
|
||||||
```
|
```
|
||||||
@@ -32,7 +32,7 @@ Replace generated placeholders with repo-specific facts where needed.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: AGENTIC-WP-0001-T02
|
id: AGENTIC-WP-0001-T02
|
||||||
status: todo
|
status: done
|
||||||
priority: high
|
priority: high
|
||||||
state_hub_task_id: "db248d57-8e1b-41ef-8386-c04ba490bc6d"
|
state_hub_task_id: "db248d57-8e1b-41ef-8386-c04ba490bc6d"
|
||||||
```
|
```
|
||||||
@@ -45,7 +45,7 @@ changes confidently.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: AGENTIC-WP-0001-T03
|
id: AGENTIC-WP-0001-T03
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "9cbb7aa5-dd49-48e2-b5a0-1670584aecf7"
|
state_hub_task_id: "9cbb7aa5-dd49-48e2-b5a0-1670584aecf7"
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -4,11 +4,11 @@ type: workplan
|
|||||||
title: "Coding Session Memory — Phase 2 (Curate: review workflow + Pattern Catalog)"
|
title: "Coding Session Memory — Phase 2 (Curate: review workflow + Pattern Catalog)"
|
||||||
domain: helix_forge
|
domain: helix_forge
|
||||||
repo: agentic-resources
|
repo: agentic-resources
|
||||||
status: ready
|
status: finished
|
||||||
owner: codex
|
owner: codex
|
||||||
topic_slug: helix-forge
|
topic_slug: helix-forge
|
||||||
created: "2026-06-06"
|
created: "2026-06-06"
|
||||||
updated: "2026-06-06"
|
updated: "2026-06-07"
|
||||||
state_hub_workstream_id: "b3703684-f60e-42f3-b03e-dabe3e8ce3f4"
|
state_hub_workstream_id: "b3703684-f60e-42f3-b03e-dabe3e8ce3f4"
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -129,7 +129,7 @@ audit trail.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: AGENTIC-WP-0004-T06
|
id: AGENTIC-WP-0004-T06
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "95d7747e-8407-41af-9a60-b919a4ee5e06"
|
state_hub_task_id: "95d7747e-8407-41af-9a60-b919a4ee5e06"
|
||||||
```
|
```
|
||||||
@@ -146,7 +146,7 @@ detect → curate → (Phase 3) distribute flow.
|
|||||||
|
|
||||||
```task
|
```task
|
||||||
id: AGENTIC-WP-0004-T07
|
id: AGENTIC-WP-0004-T07
|
||||||
status: todo
|
status: done
|
||||||
priority: medium
|
priority: medium
|
||||||
state_hub_task_id: "20407007-0a8b-4999-a470-fa3c84e17eba"
|
state_hub_task_id: "20407007-0a8b-4999-a470-fa3c84e17eba"
|
||||||
```
|
```
|
||||||
@@ -156,10 +156,22 @@ Unit tests for schema/catalog/review/gating on synthetic candidates, plus an
|
|||||||
the live detect output (the Claude+Grok "clean pass" / "abandoned" patterns from
|
the live detect output (the Claude+Grok "clean pass" / "abandoned" patterns from
|
||||||
the WP-0003 verification) into the catalog and confirms a hub decision is logged
|
the WP-0003 verification) into the catalog and confirms a hub decision is logged
|
||||||
(or queued if the API is down). Confirm catalog round-trips and versioning is
|
(or queued if the API is down). Confirm catalog round-trips and versioning is
|
||||||
idempotent on re-run. Refresh design open questions **OQ4/OQ5/OQ6** in
|
idempotent on re-run. Refresh design open questions **OQ4/OQ5/OQ6** (PRD §12).
|
||||||
[DESIGN-session-memory.md](../docs/DESIGN-session-memory.md). After workplan file
|
After workplan file updates, notify the custodian operator to run from
|
||||||
updates, notify the custodian operator to run from `~/state-hub`:
|
`~/state-hub`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
make fix-consistency REPO=agentic-resources
|
make fix-consistency REPO=agentic-resources
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Verification results (2026-06-07):** full suite 72/72 green (26 new curate
|
||||||
|
tests across schema/catalog/review/gating/decisions/entrypoint). Live pipeline
|
||||||
|
over real local sessions: fresh ingest 94→93 → 72 digests; detect surfaced 3
|
||||||
|
candidates, **2 cross-flavor** (Claude+Grok). `curate --auto-approve` promoted
|
||||||
|
all 3 into the files-first catalog — `sp-success-clean_pass-outcome` and
|
||||||
|
`sp-problem-abandoned-outcome` (both cross-flavor, `approved`/`distribution_ready`)
|
||||||
|
plus `sp-problem-budget_overrun-tokens` (Claude-only). 3 hub decisions queued
|
||||||
|
(API offline). Re-run was fully idempotent (3 skipped, 0 catalog writes, no
|
||||||
|
version bump). PRD §12 OQ4/OQ5/OQ6 resolved. The 3 catalog artifacts are
|
||||||
|
committed as the source of truth; operator runs `make fix-consistency` to index
|
||||||
|
them in the hub.
|
||||||
|
|||||||
88
workplans/AGENTIC-WP-0005-detect-hardening.md
Normal file
88
workplans/AGENTIC-WP-0005-detect-hardening.md
Normal file
@@ -0,0 +1,88 @@
|
|||||||
|
---
|
||||||
|
id: AGENTIC-WP-0005
|
||||||
|
type: workplan
|
||||||
|
title: "Coding Session Memory — Detect Hardening (quality filter + infra signals)"
|
||||||
|
domain: helix_forge
|
||||||
|
repo: agentic-resources
|
||||||
|
status: ready
|
||||||
|
owner: codex
|
||||||
|
topic_slug: helix-forge
|
||||||
|
created: "2026-06-07"
|
||||||
|
updated: "2026-06-07"
|
||||||
|
state_hub_workstream_id: "d8b7b8d1-1d85-4d2a-8ccd-7b0366a9442d"
|
||||||
|
---
|
||||||
|
|
||||||
|
# Coding Session Memory — Detect Hardening
|
||||||
|
|
||||||
|
A focused hardening pass (call it Phase 1.5) so the Detect output is trustworthy
|
||||||
|
enough to drive an **infrastructure assessment**. Triggered by ad-hoc analysis of
|
||||||
|
the live store after Phase 2:
|
||||||
|
|
||||||
|
- Of **72 captured sessions, only 31 are real coding sessions**; the rest are
|
||||||
|
health-checks / smoke-tests / interrupted runs (mostly `llm-connect` *"Say hello
|
||||||
|
in one word"*). The `abandoned` outcome heuristic mislabels these, and Phase 2
|
||||||
|
cataloged a **false-positive** "cross-flavor abandoned" pattern as
|
||||||
|
`approved`/`distribution_ready`.
|
||||||
|
- All 31 real sessions read as `success`, so the current signal set
|
||||||
|
(outcome + markers + cost) surfaces almost no genuine friction.
|
||||||
|
- The already-captured `tool_histogram` tells the real story: **~17% of tool
|
||||||
|
activity in real sessions is State Hub MCP + task plumbing + `ToolSearch`
|
||||||
|
schema-loading**, concentrated to 40–70% in some sessions — but `signals.py`
|
||||||
|
never looks at it.
|
||||||
|
|
||||||
|
No new capture is needed — this is analysis the data already supports.
|
||||||
|
|
||||||
|
## Session-Quality Filter
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: AGENTIC-WP-0005-T01
|
||||||
|
status: done
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "9f8b4304-0a37-4f66-ad34-d93e12fba0d8"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add `detect/quality.py` with `is_real_coding_session(digest)` that filters out
|
||||||
|
health-checks, smoke-tests, interrupted, and trivially-short sessions (event-count
|
||||||
|
floor, repo present, substantive edit/tool activity, not a single hello/interrupt
|
||||||
|
prompt). Wire it into the detect pipeline so signals/clusters only form over real
|
||||||
|
sessions — fixing the `abandoned` false-positive. Knobs under `[detect]` in
|
||||||
|
`config.toml`. Unit-tested on synthetic trivial-vs-real digests.
|
||||||
|
|
||||||
|
## Infra-Overhead + Thrash Signals
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: AGENTIC-WP-0005-T02
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "10d57b05-a731-4ece-bf45-f6a98ac77555"
|
||||||
|
```
|
||||||
|
|
||||||
|
Add `tool_histogram`-based extractors to `detect/signals.py`: a shared tool-bucket
|
||||||
|
helper (`shell` / `edit` / `read` / `statehub_mcp` / `task_mgmt` / `schema_load` /
|
||||||
|
`other`); `sig_infra_overhead` (PROBLEM when the statehub+task+schema share of tool
|
||||||
|
calls exceeds a threshold; magnitude = share; locus `infra_overhead`);
|
||||||
|
`sig_schema_thrash` (`ToolSearch` count over threshold; locus `schema_load`);
|
||||||
|
`sig_tool_thrash` (extreme single-tool repetition). Pure functions over digests;
|
||||||
|
thresholds configurable. Unit-tested.
|
||||||
|
|
||||||
|
## Re-run Live, Purge False Positives, Ranked Friction Report
|
||||||
|
|
||||||
|
```task
|
||||||
|
id: AGENTIC-WP-0005-T03
|
||||||
|
status: todo
|
||||||
|
priority: high
|
||||||
|
state_hub_task_id: "8b9d029a-60d0-4caf-af62-4fcc9c9a645c"
|
||||||
|
```
|
||||||
|
|
||||||
|
Re-run `ingest → detect` over the real local sessions with the filter + new
|
||||||
|
signals. Purge the false-positive catalog entries seeded in Phase 2 (the
|
||||||
|
health-check `abandoned` pattern) and re-curate so the catalog reflects real
|
||||||
|
friction. Produce a ranked **friction assessment** (`docs/ASSESSMENT-infra-friction.md`)
|
||||||
|
of the major infrastructure problems — quantified per repo/flavor, infra-overhead
|
||||||
|
share, schema-thrash — with recommendations (incl. the State Hub / MCP skill
|
||||||
|
hypothesis). After workplan file updates, notify the operator to run from
|
||||||
|
`~/state-hub`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make fix-consistency REPO=agentic-resources
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user