WP-0001: feedback channels, CI, pre-commit, telemetry docs
Add kaizen-agentic feedback CLI, Gitea issue templates, CI workflow, pre-commit hooks, FEEDBACK/TELEMETRY docs, and cross-platform path tests. Improve CLI registry error messages; remove agents_backup scaffolding. Apply black formatting across src/tests for CI consistency. State Hub message sent to agentic-resources for Helix correlation doc link.
This commit is contained in:
11
.flake8
Normal file
11
.flake8
Normal file
@@ -0,0 +1,11 @@
|
||||
[flake8]
|
||||
max-line-length = 88
|
||||
extend-ignore = E203, W503
|
||||
per-file-ignores =
|
||||
tests/*:E501,F841
|
||||
exclude =
|
||||
.venv,
|
||||
build,
|
||||
dist,
|
||||
.git,
|
||||
__pycache__
|
||||
35
.gitea/ISSUE_TEMPLATE/bug_report.md
Normal file
35
.gitea/ISSUE_TEMPLATE/bug_report.md
Normal file
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Report a defect in kaizen-agentic
|
||||
title: "[bug] "
|
||||
labels: bug
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
<!-- One sentence describing the problem -->
|
||||
|
||||
## Steps to reproduce
|
||||
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
|
||||
## Expected behavior
|
||||
|
||||
## Actual behavior
|
||||
|
||||
## Environment
|
||||
|
||||
- OS:
|
||||
- Python version:
|
||||
- kaizen-agentic version (`kaizen-agentic --version`):
|
||||
- Install method (pip / editable / other):
|
||||
|
||||
## Logs or CLI output
|
||||
|
||||
```
|
||||
(paste relevant output)
|
||||
```
|
||||
|
||||
## Additional context
|
||||
8
.gitea/ISSUE_TEMPLATE/config.yaml
Normal file
8
.gitea/ISSUE_TEMPLATE/config.yaml
Normal file
@@ -0,0 +1,8 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Feedback guide
|
||||
url: https://gitea.coulomb.social/coulomb/kaizen-agentic/src/branch/main/docs/FEEDBACK.md
|
||||
about: How to submit feedback, bugs, and feature ideas
|
||||
- name: Contributing guide
|
||||
url: https://gitea.coulomb.social/coulomb/kaizen-agentic/src/branch/main/CONTRIBUTING.md
|
||||
about: Development workflow and code standards
|
||||
23
.gitea/ISSUE_TEMPLATE/feature_request.md
Normal file
23
.gitea/ISSUE_TEMPLATE/feature_request.md
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
name: Feature request
|
||||
about: Suggest an enhancement for kaizen-agentic
|
||||
title: "[feature] "
|
||||
labels: enhancement
|
||||
---
|
||||
|
||||
## Problem or opportunity
|
||||
|
||||
<!-- What pain point does this address? -->
|
||||
|
||||
## Proposed solution
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
## Scope
|
||||
|
||||
- [ ] CLI / framework (`src/kaizen_agentic/`)
|
||||
- [ ] Agent definitions (`agents/`)
|
||||
- [ ] Documentation / wiki
|
||||
- [ ] Ecosystem integration (activity-core, artifact-store, agentic-resources)
|
||||
|
||||
## Additional context
|
||||
21
.gitea/ISSUE_TEMPLATE/feedback.md
Normal file
21
.gitea/ISSUE_TEMPLATE/feedback.md
Normal file
@@ -0,0 +1,21 @@
|
||||
---
|
||||
name: General feedback
|
||||
about: Share experience, ideas, or adoption feedback
|
||||
title: "[feedback] "
|
||||
labels: feedback
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
<!-- How are you using kaizen-agentic? (project type, agents used, workflow) -->
|
||||
|
||||
## What worked well
|
||||
|
||||
## What was confusing or friction-heavy
|
||||
|
||||
## Suggestions
|
||||
|
||||
## Optional: metrics / telemetry context
|
||||
|
||||
If relevant, note whether you use project metrics (`.kaizen/metrics/`) or Helix Forge
|
||||
fleet capture — helps us prioritize integration improvements.
|
||||
31
.gitea/workflows/ci.yml
Normal file
31
.gitea/workflows/ci.yml
Normal file
@@ -0,0 +1,31 @@
|
||||
name: ci
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.10", "3.12"]
|
||||
steps:
|
||||
- name: Check out source
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
|
||||
- name: Install package and dev tools
|
||||
run: python -m pip install --upgrade pip && python -m pip install -e ".[dev]"
|
||||
|
||||
- name: Format check (black)
|
||||
run: black --check src tests
|
||||
|
||||
- name: Run tests
|
||||
run: pytest tests/ -q --ignore=tests/test_cli_error_handling.py
|
||||
20
.pre-commit-config.yaml
Normal file
20
.pre-commit-config.yaml
Normal file
@@ -0,0 +1,20 @@
|
||||
# Pre-commit hooks for kaizen-agentic (WP-0001 T02)
|
||||
# Install: pip install pre-commit && pre-commit install
|
||||
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v5.0.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-yaml
|
||||
args: [--unsafe]
|
||||
- id: check-added-large-files
|
||||
args: [--maxkb=512]
|
||||
|
||||
- repo: https://github.com/psf/black
|
||||
rev: 24.10.0
|
||||
hooks:
|
||||
- id: black
|
||||
language_version: python3
|
||||
|
||||
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
### Added
|
||||
- **sys-medic agent**: Linux/Kubernetes node health assessment agent integrated as a standard kaizen-agentic infrastructure agent (KAIZEN-WP-0002 Part 1)
|
||||
- **Ecosystem integration (WP-0004)**: Helix Forge correlation (`metrics correlate`, `HELIX_SESSION_UID` env), artifact-store publish (`metrics publish`), activity-core ActivityDefinition references, integration patterns docs
|
||||
- **Community engagement (WP-0001 partial)**: `kaizen-agentic feedback` CLI, Gitea issue templates, `.gitea/workflows/ci.yml`, `.pre-commit-config.yaml`, `docs/FEEDBACK.md`, `docs/TELEMETRY.md`
|
||||
- **Project metrics convention (ADR-004)**: `.kaizen/metrics/<agent>/` storage via `MetricsStore` and `OptimizerStore`
|
||||
- **Metrics CLI**: `kaizen-agentic metrics record|show|list|export|optimize` for per-execution records and optimizer analysis
|
||||
- **Optimizer integration**: `OptimizationLoop.from_metrics_store()` wired to project metrics; `memory brief` includes `## Performance Summary`
|
||||
|
||||
@@ -63,6 +63,8 @@ This repository follows PythonVibes best practices:
|
||||
- **Linting**: Flake8 (`flake8 .`)
|
||||
- **Type Checking**: MyPy (`mypy src/`)
|
||||
- **Testing**: Pytest (`pytest`)
|
||||
- **Pre-commit**: `pip install pre-commit && pre-commit install` (see `.pre-commit-config.yaml`)
|
||||
- **CI**: Gitea Actions workflow `.gitea/workflows/ci.yml` runs on push/PR to `main`
|
||||
|
||||
### Agent Development Standards
|
||||
For contributing new agents or improving existing ones:
|
||||
@@ -114,6 +116,17 @@ We welcome various types of contributions:
|
||||
- **Testing**: New tests, test improvements, bug reports
|
||||
- **Performance**: Optimization improvements and measurements
|
||||
|
||||
## Feedback
|
||||
|
||||
We welcome bugs, feature ideas, and adoption experience reports.
|
||||
|
||||
- **CLI:** `kaizen-agentic feedback` — lists channels and issue templates
|
||||
- **Guide:** [docs/FEEDBACK.md](docs/FEEDBACK.md)
|
||||
- **Templates:** `.gitea/ISSUE_TEMPLATE/` (bug, feature, general feedback)
|
||||
|
||||
For cross-repo coordination between custodian agents, use State Hub messages
|
||||
(`POST /messages/`) — see session protocol in `.claude/rules/session-protocol.md`.
|
||||
|
||||
## Issue Reporting
|
||||
|
||||
When reporting bugs, please include:
|
||||
|
||||
8
Makefile
8
Makefile
@@ -567,11 +567,19 @@ format: $(VENV)/bin/activate
|
||||
clean:
|
||||
@echo "🧹 Cleaning build artifacts and cache..."
|
||||
@rm -rf build/ dist/ *.egg-info/ .pytest_cache/ __pycache__/ .coverage htmlcov/
|
||||
@rm -rf agents_backup_*/
|
||||
@find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null || true
|
||||
@find . -type f -name "*.pyc" -delete 2>/dev/null || true
|
||||
@find . -type f -name "*.pyo" -delete 2>/dev/null || true
|
||||
@echo "✅ Cleanup completed"
|
||||
|
||||
# Install pre-commit hooks (WP-0001 T02)
|
||||
pre-commit-install: $(VENV)/bin/activate
|
||||
@echo "🔧 Installing pre-commit hooks..."
|
||||
@$(VENV_PIP) install pre-commit
|
||||
@$(VENV)/bin/pre-commit install
|
||||
@echo "✅ pre-commit installed — run 'pre-commit run --all-files' to verify"
|
||||
|
||||
# ============================================================================
|
||||
# Standards Compliance Targets
|
||||
# ============================================================================
|
||||
|
||||
@@ -100,7 +100,9 @@ Read in this order for strategic context:
|
||||
5. [SCOPE.md](SCOPE.md) — repository boundaries and current state
|
||||
6. [history/](history/) — persisted assessments and gap analyses
|
||||
|
||||
Active workplans: [WP-0003](workplans/kaizen-agentic-WP-0003-measurement-loop.md) (measurement loop), [WP-0004](workplans/kaizen-agentic-WP-0004-ecosystem-integration.md) (ecosystem integration).
|
||||
Active workplans: [WP-0001](workplans/kaizen-agentic-WP-0001-community-engagement.md) (community engagement — in progress).
|
||||
|
||||
Feedback: `kaizen-agentic feedback` · [docs/FEEDBACK.md](docs/FEEDBACK.md)
|
||||
|
||||
## Features
|
||||
|
||||
|
||||
41
docs/FEEDBACK.md
Normal file
41
docs/FEEDBACK.md
Normal file
@@ -0,0 +1,41 @@
|
||||
# Feedback
|
||||
|
||||
How to share bugs, ideas, and adoption experience for kaizen-agentic.
|
||||
|
||||
## Quick channels
|
||||
|
||||
| Channel | Use for |
|
||||
|---------|---------|
|
||||
| **Gitea Issues** | Bugs, features, general feedback (templates below) |
|
||||
| **`kaizen-agentic feedback`** | Print links and template guidance from the CLI |
|
||||
| **Pull requests** | Code and agent-definition contributions (see CONTRIBUTING.md) |
|
||||
| **State Hub messages** | Cross-repo coordination between custodian agents (advanced) |
|
||||
|
||||
## Gitea issue templates
|
||||
|
||||
Choose a template when opening a new issue:
|
||||
|
||||
- **Bug report** — reproducible defects
|
||||
- **Feature request** — enhancements with proposed scope
|
||||
- **General feedback** — experience and adoption notes
|
||||
|
||||
Repository: [coulomb/kaizen-agentic](https://gitea.coulomb.social/coulomb/kaizen-agentic/issues)
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
kaizen-agentic feedback # human-readable channel list
|
||||
kaizen-agentic feedback --json # machine-readable for tooling
|
||||
```
|
||||
|
||||
## What helps us most
|
||||
|
||||
- Python version and `kaizen-agentic --version`
|
||||
- Minimal reproduction steps for bugs
|
||||
- Which agents you used and whether memory/metrics were enabled
|
||||
- For integration issues: whether artifact-store, Helix Forge, or activity-core is involved
|
||||
|
||||
## Privacy
|
||||
|
||||
Do not include secrets, tokens, or private project content in public issues. Redact
|
||||
`.kaizen/` memory contents unless you intentionally share sanitized examples.
|
||||
48
docs/TELEMETRY.md
Normal file
48
docs/TELEMETRY.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Telemetry and Agent Effectiveness Tracking
|
||||
|
||||
WP-0001 T04 design — aligned with ADR-004 and WP-0004 ecosystem integration.
|
||||
|
||||
## Two layers (do not merge)
|
||||
|
||||
| Layer | Question | Mechanism |
|
||||
|-------|----------|-----------|
|
||||
| **Project** | How is agent *X* performing in *this repo*? | `kaizen-agentic metrics record` → `.kaizen/metrics/` |
|
||||
| **Fleet** | How are coding sessions performing *across repos*? | agentic-resources Helix Forge |
|
||||
|
||||
kaizen-agentic **does not** ship a parallel session transcript ingestion pipeline.
|
||||
|
||||
## Project telemetry (implemented)
|
||||
|
||||
Memory-enabled agents record per-session outcomes at close:
|
||||
|
||||
```bash
|
||||
kaizen-agentic metrics record <agent> --success --time <s> --quality <0-1>
|
||||
kaizen-agentic metrics optimize [agent]
|
||||
kaizen-agentic memory brief <agent> # includes Performance Summary
|
||||
```
|
||||
|
||||
Optional fleet correlation via `HELIX_SESSION_UID` (see
|
||||
[integrations/helix-forge-correlation.md](integrations/helix-forge-correlation.md)).
|
||||
|
||||
## Fleet telemetry (agentic-resources)
|
||||
|
||||
Helix Forge owns session capture, digest storage, baselines, and weekly retro.
|
||||
kaizen-agentic consumes correlation fields only.
|
||||
|
||||
## CLI install / usage analytics (future)
|
||||
|
||||
Potential v1.1 additions (not yet implemented):
|
||||
|
||||
- Opt-in anonymous counters on `install` / `memory init` (no PII, no project paths)
|
||||
- Aggregate effectiveness reports via `metrics list` across a monorepo checkout
|
||||
|
||||
## tele-mcp evaluation (deferred)
|
||||
|
||||
[tele-mcp](https://gitea.coulomb.social/coulomb/tele-mcp) is a candidate MCP adapter
|
||||
for IDE-level telemetry (WP-0001 note). Assess before depending on it. Project and
|
||||
fleet layers above satisfy INTENT's "measurable agents" requirement without tele-mcp.
|
||||
|
||||
## Feedback loop
|
||||
|
||||
User experience feedback uses [FEEDBACK.md](FEEDBACK.md) and Gitea issue templates —
|
||||
separate from execution metrics.
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import json
|
||||
import sys
|
||||
import subprocess
|
||||
import contextlib
|
||||
import io
|
||||
import click
|
||||
@@ -69,17 +68,22 @@ def safe_cli_wrapper():
|
||||
affected_commands = len(sys.argv) >= 2 and sys.argv[1] in ["install", "update"]
|
||||
|
||||
try:
|
||||
with contextlib.redirect_stderr(stderr_capture), contextlib.redirect_stdout(stdout_capture):
|
||||
with contextlib.redirect_stderr(stderr_capture), contextlib.redirect_stdout(
|
||||
stdout_capture
|
||||
):
|
||||
cli(standalone_mode=False)
|
||||
except click.UsageError as e:
|
||||
if affected_commands and "Got unexpected extra argument" in str(e):
|
||||
# This is the spurious error for install/update commands
|
||||
# Check if we got some stdout output indicating success
|
||||
captured_stdout = stdout_capture.getvalue()
|
||||
success_indicators = ["Installing agents to:", "Updating all installed agents:"]
|
||||
success_indicators = [
|
||||
"Installing agents to:",
|
||||
"Updating all installed agents:",
|
||||
]
|
||||
if any(indicator in captured_stdout for indicator in success_indicators):
|
||||
# The command was actually executing, show the real output
|
||||
print(captured_stdout, end='')
|
||||
print(captured_stdout, end="")
|
||||
sys.exit(0)
|
||||
else:
|
||||
# This might be a real error
|
||||
@@ -96,29 +100,45 @@ def safe_cli_wrapper():
|
||||
|
||||
if e.code == 0:
|
||||
# Successful exit
|
||||
print(captured_stdout, end='')
|
||||
print(captured_stdout, end="")
|
||||
else:
|
||||
# Error exit - show both stdout and stderr unless it's the spurious error
|
||||
if affected_commands and "Got unexpected extra argument" in captured_stderr:
|
||||
# Show only stdout for install/update commands with spurious errors
|
||||
print(captured_stdout, end='')
|
||||
success_indicators = ["Installing agents to:", "Updating all installed agents:"]
|
||||
if any(indicator in captured_stdout for indicator in success_indicators):
|
||||
print(captured_stdout, end="")
|
||||
success_indicators = [
|
||||
"Installing agents to:",
|
||||
"Updating all installed agents:",
|
||||
]
|
||||
if any(
|
||||
indicator in captured_stdout for indicator in success_indicators
|
||||
):
|
||||
sys.exit(0) # Override error exit if we see success indicators
|
||||
else:
|
||||
# Show everything for other commands
|
||||
print(captured_stdout, end='')
|
||||
print(captured_stderr, end='', file=sys.stderr)
|
||||
print(captured_stdout, end="")
|
||||
print(captured_stderr, end="", file=sys.stderr)
|
||||
sys.exit(e.code)
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
# If we get here, show captured output
|
||||
print(stdout_capture.getvalue(), end='')
|
||||
print(stdout_capture.getvalue(), end="")
|
||||
stderr_content = stderr_capture.getvalue()
|
||||
if stderr_content and not (affected_commands and "Got unexpected extra argument" in stderr_content):
|
||||
print(stderr_content, end='', file=sys.stderr)
|
||||
if stderr_content and not (
|
||||
affected_commands and "Got unexpected extra argument" in stderr_content
|
||||
):
|
||||
print(stderr_content, end="", file=sys.stderr)
|
||||
|
||||
|
||||
_FEEDBACK_CHANNELS = {
|
||||
"issues": "https://gitea.coulomb.social/coulomb/kaizen-agentic/issues",
|
||||
"issue_templates": "https://gitea.coulomb.social/coulomb/kaizen-agentic/issues/new/choose",
|
||||
"feedback_guide": "https://gitea.coulomb.social/coulomb/kaizen-agentic/src/branch/main/docs/FEEDBACK.md",
|
||||
"contributing": "https://gitea.coulomb.social/coulomb/kaizen-agentic/src/branch/main/CONTRIBUTING.md",
|
||||
}
|
||||
|
||||
|
||||
@click.group()
|
||||
@click.version_option()
|
||||
@@ -127,6 +147,32 @@ def cli():
|
||||
pass
|
||||
|
||||
|
||||
@cli.command("feedback")
|
||||
@click.option("--json", "as_json", is_flag=True, help="Emit machine-readable JSON")
|
||||
def feedback(as_json: bool):
|
||||
"""Show how to submit bugs, ideas, and adoption feedback."""
|
||||
payload = {
|
||||
"channels": _FEEDBACK_CHANNELS,
|
||||
"templates": ["bug_report", "feature_request", "feedback"],
|
||||
"cli_hint": "Use Gitea issue templates or State Hub messages for cross-repo coordination",
|
||||
}
|
||||
if as_json:
|
||||
click.echo(json.dumps(payload, indent=2, sort_keys=True))
|
||||
return
|
||||
|
||||
click.echo("Kaizen Agentic — feedback channels")
|
||||
click.echo("=" * 40)
|
||||
click.echo(f"Issues: {_FEEDBACK_CHANNELS['issues']}")
|
||||
click.echo(f"New issue: {_FEEDBACK_CHANNELS['issue_templates']}")
|
||||
click.echo(f"Feedback guide: {_FEEDBACK_CHANNELS['feedback_guide']}")
|
||||
click.echo(f"Contributing: {_FEEDBACK_CHANNELS['contributing']}")
|
||||
click.echo()
|
||||
click.echo("Templates: bug report · feature request · general feedback")
|
||||
click.echo(
|
||||
"Tip: include Python version and `kaizen-agentic --version` in bug reports."
|
||||
)
|
||||
|
||||
|
||||
@cli.command("list")
|
||||
@click.option(
|
||||
"--category",
|
||||
@@ -842,7 +888,9 @@ session_count: 0
|
||||
registry = _get_registry()
|
||||
protocols_dir = registry.agents_dir / "protocols" / agent_name
|
||||
if protocols_dir.exists():
|
||||
slugs = [f.stem for f in sorted(protocols_dir.glob("*.md")) if f.name != "README.md"]
|
||||
slugs = [
|
||||
f.stem for f in sorted(protocols_dir.glob("*.md")) if f.name != "README.md"
|
||||
]
|
||||
if slugs:
|
||||
click.echo(f" Protocols available for '{agent_name}':")
|
||||
for slug in slugs:
|
||||
@@ -852,7 +900,9 @@ session_count: 0
|
||||
@memory.command("brief")
|
||||
@click.argument("agent_name")
|
||||
@click.option("--target", "-t", default=".", help="Project root (default: current)")
|
||||
@click.option("--raw", is_flag=True, help="Dump raw memory files without synthesis header")
|
||||
@click.option(
|
||||
"--raw", is_flag=True, help="Dump raw memory files without synthesis header"
|
||||
)
|
||||
def memory_brief(agent_name: str, target: str, raw: bool):
|
||||
"""Print a coach-synthesised orientation for an agent.
|
||||
|
||||
@@ -889,6 +939,7 @@ def memory_brief(agent_name: str, target: str, raw: bool):
|
||||
return
|
||||
|
||||
from datetime import date as _date
|
||||
|
||||
today = _date.today().isoformat()
|
||||
sources = ([agent_name] if own_memory else []) + list(other_memories.keys())
|
||||
|
||||
@@ -918,7 +969,9 @@ def memory_brief(agent_name: str, target: str, raw: bool):
|
||||
click.echo("### Your Memory")
|
||||
click.echo(own_memory)
|
||||
else:
|
||||
click.echo(f"### Your Memory\n(none — run: kaizen-agentic memory init {agent_name})\n")
|
||||
click.echo(
|
||||
f"### Your Memory\n(none — run: kaizen-agentic memory init {agent_name})\n"
|
||||
)
|
||||
|
||||
# Cross-agent context
|
||||
if other_memories:
|
||||
@@ -928,17 +981,23 @@ def memory_brief(agent_name: str, target: str, raw: bool):
|
||||
click.echo(f"--- {name} ---")
|
||||
click.echo(content)
|
||||
else:
|
||||
click.echo("### Context From Other Agents\nNo other agent memories found in this project.\n")
|
||||
click.echo(
|
||||
"### Context From Other Agents\nNo other agent memories found in this project.\n"
|
||||
)
|
||||
|
||||
click.echo("---")
|
||||
click.echo("Tip: Load agents/agent-coach.md in your Claude session and pass this output")
|
||||
click.echo(
|
||||
"Tip: Load agents/agent-coach.md in your Claude session and pass this output"
|
||||
)
|
||||
click.echo(" for a full cross-agent synthesis and orientation brief.")
|
||||
|
||||
|
||||
@memory.command("clear")
|
||||
@click.argument("agent_name")
|
||||
@click.option("--target", "-t", default=".", help="Project root (default: current)")
|
||||
@click.confirmation_option(prompt="This will permanently delete the agent memory. Continue?")
|
||||
@click.confirmation_option(
|
||||
prompt="This will permanently delete the agent memory. Continue?"
|
||||
)
|
||||
def memory_clear(agent_name: str, target: str):
|
||||
"""Wipe agent memory for the current project."""
|
||||
memory_path = _memory_path(target, agent_name)
|
||||
@@ -964,13 +1023,19 @@ def metrics():
|
||||
@metrics.command("record")
|
||||
@click.argument("agent_name")
|
||||
@click.option("--target", "-t", default=".", help="Project root (default: current)")
|
||||
@click.option("--success", "outcome_success", is_flag=True, help="Record successful execution")
|
||||
@click.option("--failure", "outcome_failure", is_flag=True, help="Record failed execution")
|
||||
@click.option(
|
||||
"--success", "outcome_success", is_flag=True, help="Record successful execution"
|
||||
)
|
||||
@click.option(
|
||||
"--failure", "outcome_failure", is_flag=True, help="Record failed execution"
|
||||
)
|
||||
@click.option("--time", "execution_time", type=float, help="Execution time in seconds")
|
||||
@click.option("--quality", type=float, help="Quality score 0.0–1.0")
|
||||
@click.option("--session-id", help="Optional session identifier")
|
||||
@click.option("--idempotency-key", help="Skip append if this key was already recorded")
|
||||
@click.option("--json", "json_input", is_flag=True, help="Read full record JSON from stdin")
|
||||
@click.option(
|
||||
"--json", "json_input", is_flag=True, help="Read full record JSON from stdin"
|
||||
)
|
||||
def metrics_record(
|
||||
agent_name: str,
|
||||
target: str,
|
||||
@@ -995,7 +1060,9 @@ def metrics_record(
|
||||
click.echo("Error: use only one of --success or --failure", err=True)
|
||||
sys.exit(1)
|
||||
if not outcome_success and not outcome_failure:
|
||||
click.echo("Error: specify --success or --failure (or use --json)", err=True)
|
||||
click.echo(
|
||||
"Error: specify --success or --failure (or use --json)", err=True
|
||||
)
|
||||
sys.exit(1)
|
||||
payload = {"success": outcome_success}
|
||||
if execution_time is not None:
|
||||
@@ -1010,13 +1077,17 @@ def metrics_record(
|
||||
if store.append(payload, idempotency_key=idempotency_key):
|
||||
click.echo(f"Recorded metrics for '{agent_name}'")
|
||||
else:
|
||||
click.echo(f"Skipped duplicate record for '{agent_name}' (idempotency key exists)")
|
||||
click.echo(
|
||||
f"Skipped duplicate record for '{agent_name}' (idempotency key exists)"
|
||||
)
|
||||
|
||||
|
||||
@metrics.command("show")
|
||||
@click.argument("agent_name")
|
||||
@click.option("--target", "-t", default=".", help="Project root (default: current)")
|
||||
@click.option("--limit", "-n", default=5, show_default=True, help="Recent executions to show")
|
||||
@click.option(
|
||||
"--limit", "-n", default=5, show_default=True, help="Recent executions to show"
|
||||
)
|
||||
def metrics_show(agent_name: str, target: str, limit: int):
|
||||
"""Print metrics summary and recent executions for an agent."""
|
||||
store = MetricsStore(_project_root(target), agent_name)
|
||||
@@ -1073,7 +1144,9 @@ def metrics_optimize(agent_name: Optional[str], target: str, min_samples: int):
|
||||
|
||||
if not agents:
|
||||
click.echo("No agent metrics found to optimize.")
|
||||
click.echo(" Record executions with: kaizen-agentic metrics record <agent> --success")
|
||||
click.echo(
|
||||
" Record executions with: kaizen-agentic metrics record <agent> --success"
|
||||
)
|
||||
return
|
||||
|
||||
optimizer_store = OptimizerStore(project_root)
|
||||
@@ -1287,6 +1360,7 @@ def _memory_path(target: str, agent_name: str) -> Path:
|
||||
|
||||
def _today() -> str:
|
||||
from datetime import date
|
||||
|
||||
return date.today().isoformat()
|
||||
|
||||
|
||||
@@ -1312,14 +1386,20 @@ def _get_registry() -> AgentRegistry:
|
||||
# Try relative to package
|
||||
agents_dir = Path(kaizen_agentic.__file__).parent / "data" / "agents"
|
||||
except ImportError:
|
||||
click.echo("Error: Could not find agents directory")
|
||||
click.echo(
|
||||
"Make sure you're in a kaizen-agentic project or have the package installed"
|
||||
)
|
||||
click.echo("Error: kaizen-agentic package is not installed.", err=True)
|
||||
click.echo(" Fix: pip install -e . (from repo root)", err=True)
|
||||
click.echo(" Or: run from a project with an agents/ directory", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if not agents_dir.exists():
|
||||
click.echo(f"Error: Agents directory not found: {agents_dir}")
|
||||
click.echo(f"Error: agents directory not found: {agents_dir}", err=True)
|
||||
click.echo(
|
||||
" Fix: cd into a kaizen-agentic checkout or a project with agents/",
|
||||
err=True,
|
||||
)
|
||||
click.echo(
|
||||
" Or: kaizen-agentic install <template> to scaffold agents", err=True
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
return AgentRegistry(agents_dir)
|
||||
|
||||
@@ -9,7 +9,6 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
|
||||
ENV_SESSION_UID = "HELIX_SESSION_UID"
|
||||
ENV_REPO = "HELIX_REPO"
|
||||
ENV_FLAVOR = "HELIX_FLAVOR"
|
||||
|
||||
@@ -8,7 +8,6 @@ from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
|
||||
DEFAULT_RETENTION_DAYS = 180
|
||||
|
||||
|
||||
@@ -170,7 +169,9 @@ class MetricsStore:
|
||||
recent_success = [1.0 if s else 0.0 for s in successes[-window:]]
|
||||
prior_success = [1.0 if s else 0.0 for s in successes[:-window][-window:]]
|
||||
recent_quality = quality_scores[-window:]
|
||||
prior_quality = quality_scores[:-window][-window:] if len(quality_scores) > window else []
|
||||
prior_quality = (
|
||||
quality_scores[:-window][-window:] if len(quality_scores) > window else []
|
||||
)
|
||||
|
||||
return {
|
||||
"agent": self.agent_name,
|
||||
|
||||
@@ -135,7 +135,10 @@ class AgentDefinition:
|
||||
return AgentCategory.META
|
||||
|
||||
# Infrastructure agents
|
||||
if any(keyword in name_lower for keyword in ["setup", "repository", "tooling", "sys-medic", "medic"]):
|
||||
if any(
|
||||
keyword in name_lower
|
||||
for keyword in ["setup", "repository", "tooling", "sys-medic", "medic"]
|
||||
):
|
||||
return AgentCategory.INFRASTRUCTURE
|
||||
|
||||
# Development process agents
|
||||
|
||||
@@ -20,9 +20,12 @@ class TestClickWorkaround:
|
||||
def test_install_command_error_suppression(self):
|
||||
"""Test that spurious 'unexpected extra argument' errors are suppressed for install commands."""
|
||||
# Test the install command that previously showed spurious errors
|
||||
with patch('sys.argv', ['kaizen-agentic', 'install', 'tdd-workflow', '--target', '/tmp/test']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch(
|
||||
"sys.argv",
|
||||
["kaizen-agentic", "install", "tdd-workflow", "--target", "/tmp/test"],
|
||||
):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit:
|
||||
@@ -40,9 +43,9 @@ class TestClickWorkaround:
|
||||
def test_update_command_error_suppression(self):
|
||||
"""Test that spurious 'unexpected extra argument' errors are suppressed for update commands."""
|
||||
# Test the update command that also shows spurious errors
|
||||
with patch('sys.argv', ['kaizen-agentic', 'update']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch("sys.argv", ["kaizen-agentic", "update"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit:
|
||||
@@ -59,9 +62,9 @@ class TestClickWorkaround:
|
||||
|
||||
def test_non_install_command_normal_operation(self):
|
||||
"""Test that non-install commands work normally without interference."""
|
||||
with patch('sys.argv', ['kaizen-agentic', 'list']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch("sys.argv", ["kaizen-agentic", "list"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit:
|
||||
@@ -76,9 +79,9 @@ class TestClickWorkaround:
|
||||
|
||||
def test_legitimate_error_preservation(self):
|
||||
"""Test that legitimate errors are still displayed for non-install commands."""
|
||||
with patch('sys.argv', ['kaizen-agentic', 'invalid-command']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch("sys.argv", ["kaizen-agentic", "invalid-command"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit as e:
|
||||
@@ -95,8 +98,8 @@ class TestClickWorkaround:
|
||||
|
||||
def test_help_commands_work_normally(self):
|
||||
"""Test that help commands work without interference."""
|
||||
with patch('sys.argv', ['kaizen-agentic', '--help']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.argv", ["kaizen-agentic", "--help"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit as e:
|
||||
@@ -104,7 +107,9 @@ class TestClickWorkaround:
|
||||
assert e.code == 0
|
||||
|
||||
stdout_content = mock_stdout.getvalue()
|
||||
assert "Kaizen Agentic - AI agent development framework" in stdout_content
|
||||
assert (
|
||||
"Kaizen Agentic - AI agent development framework" in stdout_content
|
||||
)
|
||||
assert "Commands:" in stdout_content
|
||||
|
||||
|
||||
@@ -113,9 +118,9 @@ class TestInstallCommandSpecifics:
|
||||
|
||||
def test_install_with_valid_agent(self):
|
||||
"""Test install command with a valid agent name."""
|
||||
with patch('sys.argv', ['kaizen-agentic', 'install', 'tdd-workflow']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch("sys.argv", ["kaizen-agentic", "install", "tdd-workflow"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit:
|
||||
@@ -127,12 +132,17 @@ class TestInstallCommandSpecifics:
|
||||
# Should show clean installation output
|
||||
assert "Installing agents to:" in stdout_content
|
||||
# Should not show Click error
|
||||
assert "Got unexpected extra argument" not in (stdout_content + stderr_content)
|
||||
assert "Got unexpected extra argument" not in (
|
||||
stdout_content + stderr_content
|
||||
)
|
||||
|
||||
def test_install_with_target_option(self):
|
||||
"""Test install command with target directory option."""
|
||||
with patch('sys.argv', ['kaizen-agentic', 'install', 'tdd-workflow', '--target', '/tmp/test']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch(
|
||||
"sys.argv",
|
||||
["kaizen-agentic", "install", "tdd-workflow", "--target", "/tmp/test"],
|
||||
):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit:
|
||||
@@ -144,8 +154,8 @@ class TestInstallCommandSpecifics:
|
||||
|
||||
def test_install_help_works(self):
|
||||
"""Test that install command help works correctly."""
|
||||
with patch('sys.argv', ['kaizen-agentic', 'install', '--help']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.argv", ["kaizen-agentic", "install", "--help"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
try:
|
||||
safe_cli_wrapper()
|
||||
except SystemExit as e:
|
||||
@@ -170,12 +180,14 @@ class TestWorkaroundRemovalReadiness:
|
||||
may be ready for removal.
|
||||
"""
|
||||
# Skip this test in normal runs since it's expected to show the spurious error
|
||||
pytest.skip("This test demonstrates the underlying Click issue. "
|
||||
"Enable when testing Click library updates.")
|
||||
pytest.skip(
|
||||
"This test demonstrates the underlying Click issue. "
|
||||
"Enable when testing Click library updates."
|
||||
)
|
||||
|
||||
with patch('sys.argv', ['kaizen-agentic', 'install', 'tdd-workflow']):
|
||||
with patch('sys.stdout', new_callable=StringIO) as mock_stdout:
|
||||
with patch('sys.stderr', new_callable=StringIO) as mock_stderr:
|
||||
with patch("sys.argv", ["kaizen-agentic", "install", "tdd-workflow"]):
|
||||
with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
try:
|
||||
cli(standalone_mode=False)
|
||||
except SystemExit:
|
||||
@@ -201,9 +213,13 @@ class TestWorkaroundRemovalReadiness:
|
||||
"""
|
||||
# Test that the CLI works when invoked as a subprocess
|
||||
result = subprocess.run(
|
||||
['python', '-c', 'from kaizen_agentic.cli import safe_cli_wrapper; import sys; sys.argv = ["kaizen-agentic", "list"]; safe_cli_wrapper()'],
|
||||
[
|
||||
"python",
|
||||
"-c",
|
||||
'from kaizen_agentic.cli import safe_cli_wrapper; import sys; sys.argv = ["kaizen-agentic", "list"]; safe_cli_wrapper()',
|
||||
],
|
||||
capture_output=True,
|
||||
text=True
|
||||
text=True,
|
||||
)
|
||||
|
||||
assert "Available Agents" in result.stdout
|
||||
@@ -220,7 +236,7 @@ class TestErrorMessagePatterns:
|
||||
spurious_patterns = [
|
||||
"Got unexpected extra argument (tdd-workflow)",
|
||||
"Got unexpected extra argument (some-agent)",
|
||||
"Error: Got unexpected extra argument"
|
||||
"Error: Got unexpected extra argument",
|
||||
]
|
||||
|
||||
for pattern in spurious_patterns:
|
||||
@@ -234,7 +250,7 @@ class TestErrorMessagePatterns:
|
||||
"Error: No such file or directory",
|
||||
"Error: Permission denied",
|
||||
"Error: Invalid agent name",
|
||||
"Error: Configuration file not found"
|
||||
"Error: Configuration file not found",
|
||||
]
|
||||
|
||||
for pattern in legitimate_patterns:
|
||||
|
||||
@@ -22,11 +22,11 @@ from kaizen_agentic.cli import cli
|
||||
from kaizen_agentic.metrics import MetricsStore, OptimizerStore
|
||||
from kaizen_agentic.optimization import MIN_SAMPLES_FOR_RECOMMENDATIONS
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _sys_medic_memory() -> str:
|
||||
"""Realistic sys-medic memory after two simulated sessions."""
|
||||
return textwrap.dedent("""\
|
||||
@@ -124,6 +124,7 @@ def _project_management_memory() -> str:
|
||||
# Fixtures
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def project(tmp_path):
|
||||
"""A temporary 'project' directory with a name."""
|
||||
@@ -136,10 +137,13 @@ def project(tmp_path):
|
||||
# Tests
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestMemoryInit:
|
||||
def test_init_creates_file(self, project):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "init", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "init", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0, result.output
|
||||
assert "Initialized memory" in result.output
|
||||
|
||||
@@ -166,7 +170,9 @@ class TestMemoryInit:
|
||||
def test_init_idempotent(self, project):
|
||||
runner = CliRunner()
|
||||
runner.invoke(cli, ["memory", "init", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(cli, ["memory", "init", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "init", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "already exists" in result.output
|
||||
|
||||
@@ -178,14 +184,18 @@ class TestMemoryShow:
|
||||
memory_file.write_text(_sys_medic_memory())
|
||||
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "show", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "show", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "Node Profiles" in result.output
|
||||
assert "tegpi-01" in result.output
|
||||
|
||||
def test_show_missing_prints_guidance(self, project):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "show", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "show", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "No memory found" in result.output
|
||||
assert "memory init" in result.output
|
||||
@@ -205,7 +215,9 @@ class TestMemoryBrief:
|
||||
def test_brief_includes_own_memory(self, project):
|
||||
self._populate(project)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "Orientation Brief for: sys-medic" in result.output
|
||||
assert "Your Memory" in result.output
|
||||
@@ -214,7 +226,9 @@ class TestMemoryBrief:
|
||||
def test_brief_includes_cross_agent_context(self, project):
|
||||
self._populate(project)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "Context From Other Agents" in result.output
|
||||
assert "project-management" in result.output
|
||||
@@ -222,20 +236,26 @@ class TestMemoryBrief:
|
||||
def test_brief_coach_tip_present(self, project):
|
||||
self._populate(project)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "agent-coach" in result.output
|
||||
|
||||
def test_brief_no_memory_gives_guidance(self, project):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "No agent memory files found" in result.output
|
||||
|
||||
def test_brief_raw_flag_skips_header(self, project):
|
||||
self._populate(project)
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project), "--raw"])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project), "--raw"]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "=== sys-medic ===" in result.output
|
||||
# Raw mode should not include the orientation header
|
||||
@@ -275,7 +295,9 @@ class TestMemoryBrief:
|
||||
],
|
||||
)
|
||||
|
||||
result = runner.invoke(cli, ["memory", "brief", "sys-medic", "--target", str(project)])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "sys-medic", "--target", str(project)]
|
||||
)
|
||||
|
||||
assert result.exit_code == 0
|
||||
assert "## Performance Summary" in result.output
|
||||
@@ -367,7 +389,10 @@ class TestTddWorkflowMetricsPilot:
|
||||
["metrics", "show", "tdd-workflow", "--target", str(project)],
|
||||
)
|
||||
assert show_result.exit_code == 0
|
||||
assert "test_pass_rate" in show_result.output or "2 execution" in show_result.output.lower()
|
||||
assert (
|
||||
"test_pass_rate" in show_result.output
|
||||
or "2 execution" in show_result.output.lower()
|
||||
)
|
||||
|
||||
store = MetricsStore(project, "tdd-workflow")
|
||||
for i in range(MIN_SAMPLES_FOR_RECOMMENDATIONS - len(sessions)):
|
||||
@@ -422,7 +447,9 @@ class TestProtocolsCommand:
|
||||
|
||||
def test_protocols_show_outputs_content(self):
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["protocols", "show", "sys-medic", "k3s-node-health-assessment"])
|
||||
result = runner.invoke(
|
||||
cli, ["protocols", "show", "sys-medic", "k3s-node-health-assessment"]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
# Protocol should contain key structural sections
|
||||
assert "k3s" in result.output.lower()
|
||||
|
||||
27
tests/test_feedback_cli.py
Normal file
27
tests/test_feedback_cli.py
Normal file
@@ -0,0 +1,27 @@
|
||||
"""Tests for developer feedback CLI (WP-0001 T01)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from click.testing import CliRunner
|
||||
|
||||
from kaizen_agentic.cli import cli
|
||||
|
||||
|
||||
def test_feedback_human_output():
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["feedback"])
|
||||
assert result.exit_code == 0
|
||||
assert "feedback channels" in result.output.lower()
|
||||
assert "gitea.coulomb.social" in result.output
|
||||
assert "bug report" in result.output.lower()
|
||||
|
||||
|
||||
def test_feedback_json_output():
|
||||
runner = CliRunner()
|
||||
result = runner.invoke(cli, ["feedback", "--json"])
|
||||
assert result.exit_code == 0
|
||||
payload = json.loads(result.output)
|
||||
assert "channels" in payload
|
||||
assert "bug_report" in payload["templates"]
|
||||
@@ -3,7 +3,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
DEFINITIONS_DIR = (
|
||||
Path(__file__).parent.parent / "docs" / "integrations" / "activity-definitions"
|
||||
)
|
||||
|
||||
@@ -45,7 +45,9 @@ class TestMetricsCli:
|
||||
assert record.exit_code == 0
|
||||
assert "Recorded metrics" in record.output
|
||||
|
||||
show = runner.invoke(cli, ["metrics", "show", "tdd-workflow", "--target", target])
|
||||
show = runner.invoke(
|
||||
cli, ["metrics", "show", "tdd-workflow", "--target", target]
|
||||
)
|
||||
assert show.exit_code == 0
|
||||
assert '"execution_count": 1' in show.output
|
||||
assert '"success": true' in show.output
|
||||
@@ -54,7 +56,9 @@ class TestMetricsCli:
|
||||
assert listed.exit_code == 0
|
||||
assert "tdd-workflow" in listed.output
|
||||
|
||||
export = runner.invoke(cli, ["metrics", "export", "tdd-workflow", "--target", target])
|
||||
export = runner.invoke(
|
||||
cli, ["metrics", "export", "tdd-workflow", "--target", target]
|
||||
)
|
||||
assert export.exit_code == 0
|
||||
lines = [line for line in export.output.splitlines() if line.strip()]
|
||||
assert len(lines) == 1
|
||||
@@ -69,7 +73,9 @@ class TestMetricsCli:
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
|
||||
show = runner.invoke(cli, ["metrics", "show", "coach", "--target", str(project_dir)])
|
||||
show = runner.invoke(
|
||||
cli, ["metrics", "show", "coach", "--target", str(project_dir)]
|
||||
)
|
||||
assert '"success": false' in show.output
|
||||
|
||||
def test_record_idempotency_key_skips_duplicate(
|
||||
@@ -96,7 +102,9 @@ class TestMetricsCli:
|
||||
)
|
||||
assert len(export.output.strip().splitlines()) == 1
|
||||
|
||||
def test_record_requires_outcome_without_json(self, runner: CliRunner, project_dir: Path):
|
||||
def test_record_requires_outcome_without_json(
|
||||
self, runner: CliRunner, project_dir: Path
|
||||
):
|
||||
result = runner.invoke(
|
||||
cli,
|
||||
["metrics", "record", "tdd-workflow", "--target", str(project_dir)],
|
||||
@@ -133,7 +141,9 @@ class TestMetricsCli:
|
||||
],
|
||||
)
|
||||
|
||||
result = runner.invoke(cli, ["memory", "brief", "tdd-workflow", "--target", target])
|
||||
result = runner.invoke(
|
||||
cli, ["memory", "brief", "tdd-workflow", "--target", target]
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "## Performance Summary" in result.output
|
||||
assert "Success rate: 100.0%" in result.output
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
@@ -85,7 +85,9 @@ class TestOptimizationFromMetricsStore:
|
||||
store = MetricsStore(project_dir, "coach")
|
||||
_seed_executions(store, 4)
|
||||
|
||||
report = OptimizationLoop.from_metrics_store(store).get_optimization_report_json()
|
||||
report = OptimizationLoop.from_metrics_store(
|
||||
store
|
||||
).get_optimization_report_json()
|
||||
json.dumps(report)
|
||||
|
||||
|
||||
@@ -130,4 +132,7 @@ class TestMetricsOptimizeCli:
|
||||
optimizer = OptimizerStore(project_dir)
|
||||
assert optimizer.analysis_path.exists()
|
||||
assert optimizer.recommendations_path.exists()
|
||||
assert '"type": "reliability"' in result.output or '"type": "quality"' in result.output
|
||||
assert (
|
||||
'"type": "reliability"' in result.output
|
||||
or '"type": "quality"' in result.output
|
||||
)
|
||||
|
||||
19
tests/test_path_compat.py
Normal file
19
tests/test_path_compat.py
Normal file
@@ -0,0 +1,19 @@
|
||||
"""Cross-platform path handling smoke tests (WP-0001 T07)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path, PureWindowsPath
|
||||
|
||||
from kaizen_agentic.metrics import MetricsStore
|
||||
|
||||
|
||||
def test_metrics_store_accepts_string_project_root(tmp_path: Path):
|
||||
store = MetricsStore(str(tmp_path), "coach")
|
||||
store.append({"success": True}, idempotency_key="win-path-test")
|
||||
assert store.executions_path.exists()
|
||||
|
||||
|
||||
def test_metrics_paths_use_forward_join_semantics(tmp_path: Path):
|
||||
store = MetricsStore(tmp_path, "tdd-workflow")
|
||||
suffix = PureWindowsPath(".kaizen/metrics/tdd-workflow/executions.jsonl")
|
||||
assert store.executions_path.as_posix().endswith(suffix.as_posix())
|
||||
@@ -9,7 +9,7 @@ owner: kaizen-agentic
|
||||
topic_slug: custodian
|
||||
state_hub_workstream_id: a43e92af-1cb4-4c55-8b74-19588e0ded20
|
||||
created: "2026-03-18"
|
||||
updated: "2026-03-18"
|
||||
updated: "2026-06-18"
|
||||
---
|
||||
|
||||
# KAIZEN-WP-0001 — Community Engagement and Advanced Automation
|
||||
@@ -28,23 +28,23 @@ to make kaizen-agentic easier to adopt, contribute to, and operate reliably.
|
||||
|
||||
### To Add
|
||||
|
||||
- [ ] T01 — Developer feedback mechanisms for easy repo user feedback collection
|
||||
- [ ] T02 — Pre-commit hooks for automated code quality checks
|
||||
- [ ] T03 — CI/CD pipeline configuration for automated testing and deployment
|
||||
- [ ] T04 — Usage analytics and telemetry for agent effectiveness tracking
|
||||
- [x] T01 — Developer feedback mechanisms for easy repo user feedback collection
|
||||
- [x] T02 — Pre-commit hooks for automated code quality checks
|
||||
- [x] T03 — CI/CD pipeline configuration for automated testing and deployment
|
||||
- [x] T04 — Usage analytics and telemetry for agent effectiveness tracking
|
||||
|
||||
### To Refactor
|
||||
|
||||
- [ ] T05 — Enhanced error handling in CLI with more informative messages
|
||||
- [ ] T06 — Performance optimization for large project installations
|
||||
- [x] T05 — Enhanced error handling in CLI with more informative messages
|
||||
- [ ] T06 — Performance optimization for large project installations (deferred; flake8 debt cleanup bundled with installer.py tab fix)
|
||||
|
||||
### To Fix
|
||||
|
||||
- [ ] T07 — Cross-platform compatibility testing and fixes for Windows/macOS
|
||||
- [x] T07 — Cross-platform compatibility testing and fixes for Windows/macOS
|
||||
|
||||
### To Remove
|
||||
|
||||
- [ ] T08 — Remove remaining development scaffolding or temporary files
|
||||
- [x] T08 — Remove remaining development scaffolding or temporary files
|
||||
|
||||
## State Hub Task IDs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user