feat: agent authoring & doc generation (WP-0007, v1.4.0)

New authoring tooling and a fix for the doc-regeneration defect it exposed.

Added:
- src/kaizen_agentic/agent_docs.py — render + idempotent upsert of the
  CLAUDE.md "## Installed Agents" section (shared by installer and CLI)
- `kaizen-agentic docs generate [--check]` — idempotent doc refresh / CI gate
- `kaizen-agentic create-agent` — scaffold a schema-valid agent
- Frontmatter schema validation in `kaizen-agentic validate`
  (required name/description/category, known category, valid memory/model)
- tests: test_agent_docs, test_validate_schema, test_create_agent

Fixed:
- _update_documentation regex duplicated the Installed Agents block on every
  run (stopped at the first ### subheading) — now idempotent
- declared frontmatter `category` is authoritative (heuristic is fallback)
- list_installed_agents reads the frontmatter name, not the filename
- renamed agent-project-management.md -> agent-project-assistant.md to satisfy
  the agent-<name>.md convention (eliminates a name/filename collision that
  caused install/update to write a divergent duplicate)
- test_cli_error_handling no longer installs into the repo root (uses tmp)

Version 1.4.0; CHANGELOG, CLI cheat sheet, agency-framework, TODO updated.
Workplan KAIZEN-WP-0007 closed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.claude/rules/architecture.md

@@ -17,7 +17,7 @@ Packaged copies live in `src/kaizen_agentic/data/agents/` for `pip install` dist
 |----------|--------|
 | Testing | `tdd-workflow`, `test-maintenance`, `testing-efficiency` |
 | Quality | `code-refactoring`, `datamodel-optimization` |
-| Process | `requirements-engineering`, `keepaTodofile`, `keepaChangelog`, `keepaContributingfile`, `project-management`, `priority-evaluation`, `scope-analyst` |
+| Process | `requirements-engineering`, `keepaTodofile`, `keepaChangelog`, `keepaContributingfile`, `project-assistant`, `priority-evaluation`, `scope-analyst` |
 | Infrastructure | `setupRepository`, `tooling-optimization`, `sys-medic` |
 | Release | `releaseManager` |
 | Docs | `claude-documentation` |

CHANGELOG.md

@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.4.0] - 2026-06-18
+
+### Added
+- **Agent authoring & doc generation (WP-0007)** — `kaizen-agentic create-agent`
+  scaffolds a schema-valid agent; `kaizen-agentic docs generate [--check]`
+  refreshes the CLAUDE.md `## Installed Agents` section idempotently
+- **Frontmatter schema validation** — `kaizen-agentic validate` now enforces
+  required `name`/`description`/`category`, a known category, and valid
+  `memory`/`model` values with actionable errors
+
+### Fixed
+- **Idempotent doc regeneration** — `_update_documentation` no longer duplicates
+  the `## Installed Agents` block on each run (regex stopped at the first `###`
+  subheading); rendering is now a shared, idempotent helper
+- **Declared category honoured** — agent frontmatter `category` is authoritative
+  when valid (name/content heuristic is fallback only)
+- **Installed-agent resolution** — `list_installed_agents` reads the frontmatter
+  name, so agents whose filename differs from their name resolve correctly
+
+### Changed
+- **Renamed `agent-project-management.md` → `agent-project-assistant.md`** to
+  satisfy the `agent-<name>.md` convention (frontmatter `name: project-assistant`);
+  eliminates a registry name/filename collision
+
 ## [1.3.0] - 2026-06-17
 
 ### Added

CLAUDE.md

@@ -12,51 +12,45 @@
 
 This project includes the following specialized agents:
 
-### Testing
-
-- **tdd-workflow**: Expert guidance for the TDD8 workflow methodology, specializing in the comprehensive ISSUE-TEST-RED-GREEN-REFACTOR-DOCUMENT-REFINE-PUBLISH cycle with sophisticated sidequest management and proper test organization.
-
-Use these agents by referencing them in your Claude Code interactions.
-
 ### Documentation
 
 - **claude-documentation**: Specialized assistant for Claude and Claude Code documentation, features, and best practices
+- **keepaContributingfile**: Specialized assistant for maintaining CONTRIBUTING.md files following Keep a Contributing-File V0.0.1 format within the Kaizen Agentic framework
+- **wisdom-encouragement**: Provides encouraging wisdom and guidance for complex implementation tasks and challenging technical work
 
 ### Meta
 
 - **coach**: Coaching meta-agent that reads all agent memories in a project and synthesises cross-agent briefs and new-agent orientations
+- **optimization**: Meta-agent that analyzes and optimizes other Claude Code subagents based on their performance data, usage patterns, and effectiveness metrics. Use PROACTIVELY for agent ecosystem improvement.
 
 ### Code Quality
 
 - **code-refactoring**: Analyze code structure and quality, identify improvement opportunities, and provide actionable refactoring guidance. Use PROACTIVELY for code quality assessment and improvement.
 - **datamodel-optimization**: Specialized agent that systematically analyzes, optimizes, and enhances dataclasses, models, and data structures within a codebase. Provides comprehensive datamodel improvements including convenience methods, interface consistency, code reduction, and test alignment.
-- **optimization**: Meta-agent that analyzes and optimizes other Claude Code subagents based on their performance data, usage patterns, and effectiveness metrics. Use PROACTIVELY for agent ecosystem improvement.
-- **tooling-optimization**: Meta-agent that analyzes and optimizes repository tooling usage to improve development efficiency
 
 ### Project Management
 
 - **keepaChangelog**: Specialized assistant for maintaining CHANGELOG.md files following Keep a Changelog format
-- **keepaContributingfile**: Specialized assistant for maintaining CONTRIBUTING.md files following Keep a Contributing-File V0.0.1 format within the Kaizen Agentic framework
 - **keepaTodofile**: Specialized assistant for maintaining TODO.md files following Keep a Todofile V0.0.1 format
+- **priority-evaluation**: Specialized assistant to help evaluate and establish priorities for issues and tasks.
+- **project-assistant**: Specialized assistant for project status, progress tracking, and development planning
+- **releaseManager**: Manages software releases, version control, and publication workflows for Python packages
+- **scope-analyst**: Analyze a repository and produce/improve SCOPE.md for rapid orientation
 
 ### Development Process
 
-- **priority-evaluation**: Specialized assistant to help evaluate and establish priorities for issues and tasks.
-- **releaseManager**: Manages software releases, version control, and publication workflows for Python packages
 - **requirements-engineering**: Specialized agent designed to prevent interface compatibility issues and mock object mismatches by ensuring solid foundation planning before implementation. Based on lessons learned from Issue
-- **scope-analyst**: Analyze a repository and produce/improve SCOPE.md for rapid orientation
-- **wisdom-encouragement**: Provides encouraging wisdom and guidance for complex implementation tasks and challenging technical work
+- **tdd-workflow**: Expert guidance for the TDD8 workflow methodology, specializing in the comprehensive ISSUE-TEST-RED-GREEN-REFACTOR-DOCUMENT-REFINE-PUBLISH cycle with sophisticated sidequest management and proper test organization.
 
 ### Infrastructure
 
 - **setupRepository**: Specialized assistant for setting up new Python repositories following PythonVibes best practices
 - **sys-medic**: Linux/Kubernetes node health assessment agent — diagnoses process, memory, CPU, disk, network, and kubelet issues with safe, prioritized, evidence-driven guidance
+- **tooling-optimization**: Meta-agent that analyzes and optimizes repository tooling usage to improve development efficiency
 
 ### Testing
 
-- **tdd-workflow**: Expert guidance for the TDD8 workflow methodology, specializing in the comprehensive ISSUE-TEST-RED-GREEN-REFACTOR-DOCUMENT-REFINE-PUBLISH cycle with sophisticated sidequest management and proper test organization.
 - **test-maintenance**: Specialized agent for analyzing and fixing failing tests in the project
 - **testing-efficiency**: Specialized agent designed to optimize TDD8 workflow test execution, resolve pytest reliability issues, and enhance overall testing efficiency for red-green iterations. Focuses on smart test selection, parallel execution, and agent integration patterns.
 
 Use these agents by referencing them in your Claude Code interactions.
-

TODO.md

@@ -10,23 +10,27 @@ The structure organizes **future tasks** by their impact, just as a changelog or
 
 ## [Unreleased] - *Active Vibe-Coding State* 💡
 
-Tasks in workplan: `workplans/kaizen-agentic-WP-0006-scheduled-agent-execution.md` (v1.3.0)
+Tasks in workplan: `workplans/kaizen-agentic-WP-0007-agent-authoring-doc-generation.md` (v1.4.0)
 
-### Implemented (pending v1.3.0 tag)
+### Implemented (pending v1.4.0 tag)
+* **`create-agent`** — scaffold schema-valid agents
+* **`docs generate [--check]`** — idempotent CLAUDE.md Installed Agents refresh
+* **Frontmatter schema validation** in `validate`
+* **Doc-regeneration idempotency fix** + agent file rename (project-assistant)
+
+### To Add (release)
+* **Tag v1.4.0** — after review
+* **activity-core implementation** — WP-0006 resolver + sync (separate repo; see handoff doc)
+
+### Shipped — v1.3.0 (2026-06-17)
 * **ADR-005 + `.kaizen/schedule.yml`** — scheduled agent execution contract
 * **`kaizen-agentic schedule`** — validate, init, prepare, list
 * **activity-core definitions** — weekly coach + optimization on preselected repos
-* **Resolver + roster + event design** — `discover_kaizen_scheduled_repos`,
-  State Hub roster fields, `kaizen.schedule.prepared` payload, handoff checklist
 
-### To Add (release)
-* **Tag v1.3.0** — once activity-core handoff issue is opened and pilot smoke-tested
-* **activity-core implementation** — resolver + sync (separate repo; see handoff doc)
-
-### Deferred to WP-0007 (v1.3.0+)
-* Interactive agent selection wizard
-* Agent template schema validation in `validate`
-* Documentation generation from agent metadata
+### Deferred / future
+* Interactive agent selection wizard (multi-step) — `create-agent` covers the
+  single-agent scaffold; a guided multi-agent wizard remains future work
+* Multi-file agent packages / protocol scaffolding
 
 ***
 

docs/CLI_CHEAT_SHEET.md

@@ -52,9 +52,28 @@ kaizen-agentic remove old-agent-name
 
 # Project status
 kaizen-agentic status                    # Show current project status
-kaizen-agentic validate                  # Validate agent installation
+kaizen-agentic validate                  # Validate agents (incl. frontmatter schema)
 ```
 
+### Authoring & Docs (WP-0007)
+```bash
+# Scaffold a new schema-valid agent (agents/agent-<name>.md)
+kaizen-agentic create-agent my-agent -c testing -d "What it does"
+kaizen-agentic create-agent my-agent     # prompts for category + description
+kaizen-agentic create-agent my-agent --force   # overwrite existing
+
+# Refresh the CLAUDE.md "Installed Agents" section from agent metadata
+kaizen-agentic docs generate             # idempotent rewrite
+kaizen-agentic docs generate --check     # CI gate: non-zero if out of date
+
+# validate enforces required name/description/category, valid category,
+# and valid memory/model values
+kaizen-agentic validate
+```
+
+After adding or editing agents, run `make agents-sync-package` so the packaged
+`data/agents/` copies stay in parity (release-check gate).
+
 ### Project Metrics (ADR-004)
 ```bash
 # Record outcome at session close

docs/agency-framework.md

@@ -80,6 +80,23 @@ memory: enabled   # or: disabled
 
 The `memory` field defaults to `enabled`. Set `memory: disabled` for agents that are stateless by design (e.g. `wisdom-encouragement`).
 
+The file name must follow `agent-<name>.md` where `<name>` equals the frontmatter
+`name`. `kaizen-agentic validate` enforces the frontmatter schema (required
+fields, known `category`, valid `memory`/`model`).
+
+### Authoring and doc sync (WP-0007)
+
+```bash
+kaizen-agentic create-agent <name> -c <category> -d "<description>"
+kaizen-agentic validate                 # schema + dependency checks
+kaizen-agentic docs generate            # refresh CLAUDE.md Installed Agents
+make agents-sync-package                # keep packaged data/agents/ in parity
+```
+
+`create-agent` writes a schema-valid skeleton; `docs generate` rewrites the
+project `## Installed Agents` section **idempotently** (use `--check` as a CI
+gate). The section is grouped by the declared frontmatter `category`.
+
 ---
 
 ## The Coach Meta-Agent

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "kaizen-agentic"
-version = "1.3.0"
+version = "1.4.0"
 description = "AI agent development framework embracing continuous improvement (kaizen)"
 readme = "README.md"
 license = {file = "LICENSE"}

src/kaizen_agentic/__init__.py

@@ -9,7 +9,7 @@ It also includes a comprehensive agent distribution system for sharing
 specialized agents across projects via CLI tools and package management.
 """
 
-__version__ = "1.3.0"
+__version__ = "1.4.0"
 __author__ = "Kaizen Agentic Team"
 
 from .core import Agent, AgentConfig

src/kaizen_agentic/agent_docs.py

@@ -0,0 +1,73 @@
+"""Render and idempotently upsert the project ``## Installed Agents`` section.
+
+Single source of truth for the agent documentation block written into a
+project's ``CLAUDE.md``. Both :class:`~kaizen_agentic.installer.AgentInstaller`
+and the ``kaizen-agentic docs generate`` command reuse these helpers so the
+section is produced and replaced the same way everywhere — and, critically,
+**idempotently**: regenerating N times yields the same output as once.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Dict, Iterable, List
+
+from .registry import AgentDefinition
+
+SECTION_HEADING = "## Installed Agents"
+SECTION_FOOTER = (
+    "Use these agents by referencing them in your Claude Code interactions."
+)
+
+# Match the Installed Agents block up to the next *top-level* heading (``\n## ``,
+# with a trailing space so ``### Subsection`` headings inside the block do not
+# terminate the match) or end-of-file. The previous ``(?=##|\Z)`` form stopped
+# at the first ``### Category`` subheading and so duplicated the block on every
+# run (WP-0007 T01).
+_SECTION_RE = re.compile(r"## Installed Agents.*?(?=\n## (?!#)|\Z)", re.DOTALL)
+
+
+def render_installed_agents_section(agents: Iterable[AgentDefinition]) -> str:
+    """Render the ``## Installed Agents`` markdown block from agent metadata.
+
+    Agents are grouped by category in first-seen order. The returned string is
+    newline-terminated and contains no trailing top-level heading, so it can be
+    spliced in front of any following section.
+    """
+    lines: List[str] = [
+        SECTION_HEADING,
+        "",
+        "This project includes the following specialized agents:",
+        "",
+    ]
+
+    categories: Dict[str, List[AgentDefinition]] = {}
+    for agent in agents:
+        categories.setdefault(agent.category.value, []).append(agent)
+
+    for category, members in categories.items():
+        lines.append(f"### {category.replace('-', ' ').title()}")
+        lines.append("")
+        for agent in members:
+            lines.append(f"- **{agent.name}**: {agent.description}")
+        lines.append("")
+
+    lines.append(SECTION_FOOTER)
+    lines.append("")
+    return "\n".join(lines)
+
+
+def upsert_installed_agents_section(content: str, section: str) -> str:
+    """Return ``content`` with its Installed Agents block replaced or appended.
+
+    Idempotent: if ``content`` already contains the block, exactly that block is
+    replaced (never duplicated); otherwise the section is appended. ``section``
+    is normalised to end with a single trailing blank line.
+    """
+    section = section.rstrip() + "\n"
+
+    if SECTION_HEADING in content:
+        return _SECTION_RE.sub(lambda _m: section, content, count=1)
+
+    separator = "" if content.endswith("\n\n") or not content else "\n"
+    return content + separator + section

src/kaizen_agentic/cli.py

@@ -25,6 +25,10 @@ from .schedule import (
     schedule_path,
     validate_schedule,
 )
+from .agent_docs import (
+    render_installed_agents_section,
+    upsert_installed_agents_section,
+)
 
 
 def safe_cli_wrapper():
@@ -418,8 +422,21 @@ def validate(target: str):
 
     target_path = Path(target).resolve()
 
+    # Validate agent frontmatter schema
+    click.echo("Validating agent frontmatter schema...")
+    schema_errors = registry.validate_frontmatter_schema()
+
+    if schema_errors:
+        click.echo("Frontmatter schema errors:")
+        for agent_file, errors in schema_errors.items():
+            click.echo(f"  {agent_file}:")
+            for error in errors:
+                click.echo(f"    ❌ {error}")
+    else:
+        click.echo("  ✅ Frontmatter schema validation passed")
+
     # Validate registry agents
-    click.echo("Validating agent registry...")
+    click.echo("\nValidating agent registry...")
     registry_errors = registry.validate_agents()
 
     if registry_errors:
@@ -1565,6 +1582,156 @@ def _render_prepare_markdown(bundle: dict) -> str:
     return "\n".join(lines)
 
 
+@cli.command("create-agent")
+@click.argument("name")
+@click.option(
+    "--category",
+    "-c",
+    type=click.Choice([c.value for c in AgentCategory]),
+    help="Agent category (prompted if omitted)",
+)
+@click.option("--description", "-d", help="One-line description (prompted if omitted)")
+@click.option(
+    "--memory",
+    type=click.Choice(["enabled", "disabled"]),
+    default="enabled",
+    show_default=True,
+    help="Project memory support",
+)
+@click.option("--model", help="Optional model hint (e.g. claude-opus-4-8)")
+@click.option(
+    "--target",
+    "-t",
+    default=".",
+    help="Project root containing agents/ (default: current)",
+)
+@click.option("--force", is_flag=True, help="Overwrite an existing agent file")
+def create_agent(
+    name: str,
+    category: Optional[str],
+    description: Optional[str],
+    memory: str,
+    model: Optional[str],
+    target: str,
+    force: bool,
+):
+    """Scaffold a new schema-valid agent definition (agents/agent-<name>.md)."""
+    if not category:
+        category = click.prompt(
+            "Category", type=click.Choice([c.value for c in AgentCategory])
+        )
+    if not description:
+        description = click.prompt("One-line description")
+
+    agents_dir = _project_root(target) / "agents"
+    agents_dir.mkdir(parents=True, exist_ok=True)
+    agent_path = agents_dir / f"agent-{name}.md"
+
+    if agent_path.exists() and not force:
+        click.echo(f"Agent already exists: {agent_path}")
+        click.echo("  Use --force to overwrite.")
+        sys.exit(1)
+
+    frontmatter = [
+        "---",
+        f"name: {name}",
+        f"description: {description}",
+        f"category: {category}",
+        f"memory: {memory}",
+    ]
+    if model:
+        frontmatter.append(f"model: {model}")
+    frontmatter.append("---")
+
+    title = name.replace("-", " ").title()
+    body = f"""
+# {title} Agent
+
+## Role
+
+<!-- One paragraph: what this agent does and what it does not do. -->
+
+## When to Use
+
+<!-- Triggers and situations where this agent should be invoked. -->
+
+## Instructions
+
+<!-- Step-by-step guidance the agent follows. -->
+
+## Output
+
+<!-- What the agent produces and in what format. -->
+"""
+    agent_path.write_text("\n".join(frontmatter) + "\n" + body)
+
+    # Validate the scaffold passes the frontmatter schema (T03).
+    errors = (
+        AgentRegistry(agents_dir).validate_frontmatter_schema().get(agent_path.name, [])
+    )
+    if errors:
+        click.echo(f"⚠️  Created {agent_path} but it has schema issues:")
+        for error in errors:
+            click.echo(f"    ❌ {error}")
+        sys.exit(1)
+
+    click.echo(f"✅ Created agent: {agent_path}")
+    click.echo("  Edit the skeleton, then validate: kaizen-agentic validate")
+    click.echo("  Before release: make agents-sync-package")
+
+
+@cli.group()
+def docs():
+    """Generate project documentation from agent metadata."""
+    pass
+
+
+@docs.command("generate")
+@click.option("--target", "-t", default=".", help="Project root (default: current)")
+@click.option(
+    "--check",
+    is_flag=True,
+    help="Exit non-zero if CLAUDE.md would change (do not write)",
+)
+def docs_generate(target: str, check: bool):
+    """Refresh the '## Installed Agents' section of CLAUDE.md (idempotent)."""
+    target_path = _project_root(target)
+    # Resolve agents from the target project's own agents/ when present, so
+    # `docs generate --target other/project` documents that project's agents
+    # rather than the registry resolved from the current directory.
+    local_agents = target_path / "agents"
+    registry = AgentRegistry(local_agents) if local_agents.exists() else _get_registry()
+    installer = AgentInstaller(registry)
+
+    installed = installer.list_installed_agents(target_path)
+    if not installed:
+        click.echo("No agents installed in this project — nothing to document.")
+        click.echo("  Run: kaizen-agentic install <agents>")
+        return
+
+    agents = [a for a in (registry.get_agent(n) for n in installed) if a is not None]
+    section = render_installed_agents_section(agents)
+
+    claude_md = target_path / "CLAUDE.md"
+    current = claude_md.read_text() if claude_md.exists() else ""
+    updated = upsert_installed_agents_section(current, section)
+
+    if check:
+        if updated != current:
+            click.echo(f"❌ CLAUDE.md is out of date: {claude_md}")
+            click.echo("   Run: kaizen-agentic docs generate")
+            sys.exit(1)
+        click.echo(f"✅ CLAUDE.md is up to date ({len(agents)} agents)")
+        return
+
+    if updated == current:
+        click.echo(f"CLAUDE.md already up to date ({len(agents)} agents)")
+        return
+
+    claude_md.write_text(updated)
+    click.echo(f"Updated Installed Agents section ({len(agents)} agents): {claude_md}")
+
+
 def _project_root(target: str) -> Path:
     return Path(target).resolve()
 

src/kaizen_agentic/installer.py

@@ -6,7 +6,11 @@ from pathlib import Path
 from typing import List, Dict, Optional
 from dataclasses import dataclass
 
-from .registry import AgentRegistry
+from .registry import AgentRegistry, AgentDefinition
+from .agent_docs import (
+    render_installed_agents_section,
+    upsert_installed_agents_section,
+)
 
 
 @dataclass
@@ -82,7 +86,15 @@ class AgentInstaller:
 
         installed = []
         for agent_file in agents_dir.glob("agent-*.md"):
-            agent_name = agent_file.stem.replace("agent-", "")
+            # Prefer the frontmatter name (registry-authoritative); fall back to
+            # the filename when frontmatter is missing/unreadable. The filename
+            # encodes the category for a few agents (e.g. agent-project-
+            # management.md → name: project-assistant), so a pure filename derive
+            # produces names the registry cannot resolve (WP-0007 T02).
+            try:
+                agent_name = AgentDefinition._read_frontmatter(agent_file)["name"]
+            except Exception:
+                agent_name = agent_file.stem.replace("agent-", "")
             installed.append(agent_name)
 
         return sorted(installed)
@@ -235,60 +247,25 @@ agents-validate:
         try:
             claude_md = project_dir / "CLAUDE.md"
 
-            agent_section = "## Installed Agents\n\n"
-            agent_section += (
-                "This project includes the following specialized agents:\n\n"
-            )
+            agents = [
+                agent
+                for agent in (self.registry.get_agent(name) for name in agent_names)
+                if agent is not None
+            ]
+            agent_section = render_installed_agents_section(agents)
 
-            # Group agents by category
-            categories = {}
-            for agent_name in agent_names:
-                agent = self.registry.get_agent(agent_name)
-                if agent:
-                    category = agent.category.value
-                    if category not in categories:
-                        categories[category] = []
-                    categories[category].append(agent)
-
-            # Generate documentation
-            for category, agents in categories.items():
-                agent_section += f"### {category.replace('-', ' ').title()}\n\n"
-                for agent in agents:
-                    agent_section += f"- **{agent.name}**: {agent.description}\n"
-                agent_section += "\n"
-
-            agent_section += (
-                "Use these agents by referencing them in your "
-                "Claude Code interactions.\n\n"
-            )
-
-            # Update or create CLAUDE.md
+            # Update or create CLAUDE.md (idempotent upsert — WP-0007 T01/T02)
             if claude_md.exists():
-                with open(claude_md, "r") as f:
-                    content = f.read()
-
-                # Replace existing agent section or append
-                if "## Installed Agents" in content:
-                    import re
-
-                    content = re.sub(
-                        r"## Installed Agents.*?(?=##|\Z)",
-                        agent_section,
-                        content,
-                        flags=re.DOTALL,
-                    )
-                else:
-                    content += "\n" + agent_section
-
-                with open(claude_md, "w") as f:
-                    f.write(content)
+                content = claude_md.read_text()
+                content = upsert_installed_agents_section(content, agent_section)
+                claude_md.write_text(content)
             else:
-                # Create new CLAUDE.md
                 header = "# Claude Code Configuration\n\n"
-                header += "This file contains Claude Code configuration and agent information.\n\n"
-
-                with open(claude_md, "w") as f:
-                    f.write(header + agent_section)
+                header += (
+                    "This file contains Claude Code configuration and agent "
+                    "information.\n\n"
+                )
+                claude_md.write_text(header + agent_section)
 
             print(f"Updated documentation: {claude_md}")
 

src/kaizen_agentic/registry.py

@@ -60,8 +60,10 @@ class AgentDefinition:
         # Extract dependencies from frontmatter and content
         dependencies = cls._extract_dependencies(content, frontmatter)
 
-        # Determine category from name or content
-        category = cls._determine_category(frontmatter["name"], content)
+        # The declared frontmatter category is authoritative when it is a known
+        # value (it is what `validate` enforces, WP-0007 T03); fall back to the
+        # name/content heuristic only when absent or unrecognised.
+        category = cls._resolve_category(frontmatter, content)
 
         return cls(
             name=frontmatter["name"],
@@ -118,6 +120,17 @@ class AgentDefinition:
 
         return dependencies
 
+    @classmethod
+    def _resolve_category(cls, frontmatter: dict, content: str) -> AgentCategory:
+        """Prefer the declared frontmatter category; heuristic as fallback."""
+        declared = frontmatter.get("category")
+        if isinstance(declared, str):
+            try:
+                return AgentCategory(declared.strip())
+            except ValueError:
+                pass
+        return cls._determine_category(frontmatter["name"], content)
+
     @staticmethod
     def _determine_category(name: str, content: str) -> AgentCategory:
         """Determine agent category based on name and content."""
@@ -288,6 +301,65 @@ class AgentRegistry:
 
         return errors
 
+    def validate_frontmatter_schema(self) -> Dict[str, List[str]]:
+        """Validate each agent file's frontmatter against the required schema.
+
+        Required: ``name``, ``description``, ``category`` (non-empty strings);
+        ``category`` must be a known :class:`AgentCategory` value. Optional:
+        ``memory`` ∈ {enabled, disabled}; ``model`` a non-empty string. Keyed by
+        filename so files with a missing/invalid ``name`` still surface.
+        """
+        errors: Dict[str, List[str]] = {}
+        valid_categories = {c.value for c in AgentCategory}
+
+        if not self.agents_dir.exists():
+            return errors
+
+        for agent_file in sorted(self.agents_dir.glob("agent-*.md")):
+            file_errors: List[str] = []
+            try:
+                content = agent_file.read_text(encoding="utf-8")
+                match = re.match(r"^---\n(.*?)\n---\n", content, re.DOTALL)
+                if not match:
+                    errors[agent_file.name] = ["missing YAML frontmatter"]
+                    continue
+                frontmatter = yaml.safe_load(match.group(1))
+                if not isinstance(frontmatter, dict):
+                    errors[agent_file.name] = ["frontmatter is not a mapping"]
+                    continue
+
+                for field in ("name", "description", "category"):
+                    value = frontmatter.get(field)
+                    if not isinstance(value, str) or not value.strip():
+                        file_errors.append(f"missing or empty required field: {field}")
+
+                category = frontmatter.get("category")
+                if isinstance(category, str) and category not in valid_categories:
+                    file_errors.append(
+                        f"invalid category '{category}' (expected one of "
+                        f"{', '.join(sorted(valid_categories))})"
+                    )
+
+                memory = frontmatter.get("memory")
+                if memory is not None and memory not in ("enabled", "disabled"):
+                    file_errors.append(
+                        f"invalid memory '{memory}' "
+                        "(expected 'enabled' or 'disabled')"
+                    )
+
+                model = frontmatter.get("model")
+                if model is not None and (
+                    not isinstance(model, str) or not model.strip()
+                ):
+                    file_errors.append("model must be a non-empty string when present")
+            except Exception as exc:  # pragma: no cover - defensive
+                file_errors.append(f"failed to parse frontmatter: {exc}")
+
+            if file_errors:
+                errors[agent_file.name] = file_errors
+
+        return errors
+
     def _has_circular_dependency(
         self, agent_name: str, visited: Optional[Set[str]] = None
     ) -> bool:

tests/test_agent_docs.py

@@ -0,0 +1,97 @@
+"""Tests for idempotent agent-docs generation (WP-0007 T01/T02)."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+from click.testing import CliRunner
+
+from kaizen_agentic.agent_docs import (
+    SECTION_FOOTER,
+    SECTION_HEADING,
+    render_installed_agents_section,
+    upsert_installed_agents_section,
+)
+from kaizen_agentic.cli import cli
+from kaizen_agentic.registry import AgentRegistry
+
+AGENTS_DIR = Path(__file__).parent.parent / "agents"
+
+
+@pytest.fixture
+def runner() -> CliRunner:
+    return CliRunner()
+
+
+def _section() -> str:
+    agents = AgentRegistry(AGENTS_DIR).list_agents()[:4]
+    return render_installed_agents_section(agents)
+
+
+class TestUpsertIdempotency:
+    def test_upsert_is_idempotent(self):
+        section = _section()
+        base = f"# Project\n\nintro\n\n{section}\n## Keep Me\nbody\n"
+        once = upsert_installed_agents_section(base, section)
+        twice = upsert_installed_agents_section(once, section)
+        assert once == twice
+        assert once.count(SECTION_HEADING) == 1
+        assert once.count(SECTION_FOOTER) == 1
+        # A following top-level section must survive the replace
+        assert once.count("## Keep Me") == 1
+
+    def test_subheadings_do_not_truncate_section(self):
+        section = _section()
+        # The block contains '### Category' subheadings; the replace must not
+        # stop at the first one (the original regex bug).
+        merged = upsert_installed_agents_section("# P\n\n", section)
+        assert merged.count(SECTION_FOOTER) == 1
+        assert "### " in merged  # categories rendered
+
+    def test_append_when_absent(self):
+        section = _section()
+        merged = upsert_installed_agents_section("# Project\n\nbody\n", section)
+        assert SECTION_HEADING in merged
+        assert merged.count(SECTION_FOOTER) == 1
+
+
+class TestDocsGenerateCli:
+    def test_generate_then_check_clean(self, runner: CliRunner, tmp_path: Path):
+        # A project with two installed agents
+        proj = tmp_path / "proj"
+        (proj / "agents").mkdir(parents=True)
+        for name, cat in (("alpha", "testing"), ("beta", "code-quality")):
+            (proj / "agents" / f"agent-{name}.md").write_text(
+                f"---\nname: {name}\ndescription: d{name}\ncategory: {cat}\n---\nx\n"
+            )
+
+        gen = runner.invoke(cli, ["docs", "generate", "--target", str(proj)])
+        assert gen.exit_code == 0
+        claude = (proj / "CLAUDE.md").read_text()
+        assert claude.count(SECTION_HEADING) == 1
+        assert "**alpha**" in claude and "**beta**" in claude
+
+        # Second generate is a no-op
+        again = runner.invoke(cli, ["docs", "generate", "--target", str(proj)])
+        assert "already up to date" in again.output
+
+        # --check passes on a synced repo
+        check = runner.invoke(
+            cli, ["docs", "generate", "--check", "--target", str(proj)]
+        )
+        assert check.exit_code == 0
+        assert "up to date" in check.output
+
+    def test_check_fails_when_stale(self, runner: CliRunner, tmp_path: Path):
+        proj = tmp_path / "proj"
+        (proj / "agents").mkdir(parents=True)
+        (proj / "agents" / "agent-alpha.md").write_text(
+            "---\nname: alpha\ndescription: d\ncategory: testing\n---\nx\n"
+        )
+        (proj / "CLAUDE.md").write_text("# Proj\n\nno agents section yet\n")
+        check = runner.invoke(
+            cli, ["docs", "generate", "--check", "--target", str(proj)]
+        )
+        assert check.exit_code == 1
+        assert "out of date" in check.output

tests/test_cli_error_handling.py

@@ -40,10 +40,15 @@ class TestClickWorkaround:
                     assert "Got unexpected extra argument" not in stdout_content
                     assert "Got unexpected extra argument" not in stderr_content
 
-    def test_update_command_error_suppression(self):
+    def test_update_command_error_suppression(self, tmp_path):
         """Test that spurious 'unexpected extra argument' errors are suppressed for update commands."""
+        # Seed a temp project so `update` does not rewrite the repo's own agents/
+        (tmp_path / "agents").mkdir()
+        (tmp_path / "agents" / "agent-tdd-workflow.md").write_text(
+            "---\nname: tdd-workflow\ndescription: d\ncategory: testing\n---\nx\n"
+        )
         # Test the update command that also shows spurious errors
-        with patch("sys.argv", ["kaizen-agentic", "update"]):
+        with patch("sys.argv", ["kaizen-agentic", "update", "--target", str(tmp_path)]):
             with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
                 with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
                     try:
@@ -116,9 +121,12 @@ class TestClickWorkaround:
 class TestInstallCommandSpecifics:
     """Test specific install command scenarios."""
 
-    def test_install_with_valid_agent(self):
+    def test_install_with_valid_agent(self, tmp_path):
         """Test install command with a valid agent name."""
-        with patch("sys.argv", ["kaizen-agentic", "install", "tdd-workflow"]):
+        with patch(
+            "sys.argv",
+            ["kaizen-agentic", "install", "tdd-workflow", "--target", str(tmp_path)],
+        ):
             with patch("sys.stdout", new_callable=StringIO) as mock_stdout:
                 with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
                     try:

tests/test_create_agent.py

@@ -0,0 +1,100 @@
+"""Tests for the create-agent scaffold (WP-0007 T04)."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+from click.testing import CliRunner
+
+from kaizen_agentic.cli import cli
+from kaizen_agentic.registry import AgentRegistry
+
+
+@pytest.fixture
+def runner() -> CliRunner:
+    return CliRunner()
+
+
+class TestCreateAgent:
+    def test_scaffold_is_schema_valid_and_loads(
+        self, runner: CliRunner, tmp_path: Path
+    ):
+        result = runner.invoke(
+            cli,
+            [
+                "create-agent",
+                "demo-helper",
+                "-c",
+                "testing",
+                "-d",
+                "Demo helper for tests",
+                "--target",
+                str(tmp_path),
+            ],
+        )
+        assert result.exit_code == 0, result.output
+        agent_path = tmp_path / "agents" / "agent-demo-helper.md"
+        assert agent_path.exists()
+
+        registry = AgentRegistry(tmp_path / "agents")
+        # Passes the schema and is loadable by the registry.
+        assert registry.validate_frontmatter_schema() == {}
+        agent = registry.get_agent("demo-helper")
+        assert agent is not None
+        assert agent.category.value == "testing"
+        assert agent.memory == "enabled"
+
+    def test_interactive_prompts_when_flags_missing(
+        self, runner: CliRunner, tmp_path: Path
+    ):
+        result = runner.invoke(
+            cli,
+            ["create-agent", "prompted", "--target", str(tmp_path)],
+            input="testing\nA prompted agent\n",
+        )
+        assert result.exit_code == 0, result.output
+        content = (tmp_path / "agents" / "agent-prompted.md").read_text()
+        assert "category: testing" in content
+        assert "description: A prompted agent" in content
+
+    def test_refuses_overwrite_without_force(self, runner: CliRunner, tmp_path: Path):
+        args = [
+            "create-agent",
+            "dup",
+            "-c",
+            "meta",
+            "-d",
+            "first",
+            "--target",
+            str(tmp_path),
+        ]
+        assert runner.invoke(cli, args).exit_code == 0
+        second = runner.invoke(cli, args)
+        assert second.exit_code == 1
+        assert "already exists" in second.output
+
+    def test_force_overwrites(self, runner: CliRunner, tmp_path: Path):
+        base = ["create-agent", "dup", "--target", str(tmp_path)]
+        runner.invoke(cli, base + ["-c", "meta", "-d", "first"])
+        result = runner.invoke(cli, base + ["-c", "meta", "-d", "second", "--force"])
+        assert result.exit_code == 0
+        assert (
+            "description: second" in (tmp_path / "agents" / "agent-dup.md").read_text()
+        )
+
+    def test_rejects_invalid_category(self, runner: CliRunner, tmp_path: Path):
+        result = runner.invoke(
+            cli,
+            [
+                "create-agent",
+                "x",
+                "-c",
+                "nonsense",
+                "-d",
+                "d",
+                "--target",
+                str(tmp_path),
+            ],
+        )
+        assert result.exit_code != 0

tests/test_validate_schema.py

@@ -0,0 +1,74 @@
+"""Tests for agent frontmatter schema validation (WP-0007 T03)."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from kaizen_agentic.registry import AgentRegistry
+
+REPO_AGENTS = Path(__file__).parent.parent / "agents"
+
+
+def _registry(tmp_path: Path, files: dict) -> AgentRegistry:
+    agents = tmp_path / "agents"
+    agents.mkdir(parents=True)
+    for filename, content in files.items():
+        (agents / filename).write_text(content)
+    return AgentRegistry(agents)
+
+
+class TestFrontmatterSchema:
+    def test_repo_agents_are_schema_valid(self):
+        # The shipped agents/ must always pass the schema.
+        assert AgentRegistry(REPO_AGENTS).validate_frontmatter_schema() == {}
+
+    def test_good_agent_passes(self, tmp_path: Path):
+        reg = _registry(
+            tmp_path,
+            {
+                "agent-good.md": (
+                    "---\nname: good\ndescription: A fine agent\n"
+                    "category: testing\nmemory: enabled\n---\nbody\n"
+                )
+            },
+        )
+        assert reg.validate_frontmatter_schema() == {}
+
+    def test_missing_required_fields(self, tmp_path: Path):
+        reg = _registry(
+            tmp_path,
+            {"agent-x.md": "---\nname: x\ncategory: testing\n---\nbody\n"},
+        )
+        errors = reg.validate_frontmatter_schema()["agent-x.md"]
+        assert any("description" in e for e in errors)
+
+    def test_invalid_category(self, tmp_path: Path):
+        reg = _registry(
+            tmp_path,
+            {
+                "agent-x.md": (
+                    "---\nname: x\ndescription: d\ncategory: nonsense\n---\nb\n"
+                )
+            },
+        )
+        errors = reg.validate_frontmatter_schema()["agent-x.md"]
+        assert any("invalid category" in e for e in errors)
+
+    def test_invalid_memory(self, tmp_path: Path):
+        reg = _registry(
+            tmp_path,
+            {
+                "agent-x.md": (
+                    "---\nname: x\ndescription: d\ncategory: testing\n"
+                    "memory: maybe\n---\nb\n"
+                )
+            },
+        )
+        errors = reg.validate_frontmatter_schema()["agent-x.md"]
+        assert any("invalid memory" in e for e in errors)
+
+    def test_missing_frontmatter(self, tmp_path: Path):
+        reg = _registry(tmp_path, {"agent-x.md": "just text, no frontmatter\n"})
+        assert reg.validate_frontmatter_schema()["agent-x.md"] == [
+            "missing YAML frontmatter"
+        ]

workplans/kaizen-agentic-WP-0007-agent-authoring-doc-generation.md

@@ -0,0 +1,131 @@
+---
+id: KAIZEN-WP-0007
+type: workplan
+title: "Agent Authoring & Doc Generation (v1.4.0)"
+domain: custodian
+repo: kaizen-agentic
+status: done
+owner: kaizen-agentic
+topic_slug: custodian
+state_hub_workstream_id: a8bc88a4-0ee3-44c6-aff5-9d7f54a316f5
+depends_on:
+  - KAIZEN-WP-0006
+created: "2026-06-18"
+updated: "2026-06-18"
+tasks:
+- id: T01
+  state_hub_task_id: debaf0ac-47df-4bbd-aa1f-96c7b96a64ed
+  status: done
+  title: Fix idempotent CLAUDE.md doc regeneration (installer regex bug)
+- id: T02
+  state_hub_task_id: e2d9bea8-243b-4b12-bba3-4d43a3bae71c
+  status: done
+  title: Reusable agent-docs module and kaizen-agentic docs generate command
+- id: T03
+  state_hub_task_id: 57176887-5344-444c-aa5a-a4aea161ee71
+  status: done
+  title: Enforce agent frontmatter schema in validate
+- id: T04
+  state_hub_task_id: 724e3862-602b-4ef0-88b8-ddb78a225046
+  status: done
+  title: kaizen-agentic create-agent scaffold for new agents
+- id: T05
+  state_hub_task_id: 4d6e323c-9cad-49ce-9356-9192c20cc986
+  status: done
+  title: Unit tests for docs generate, schema validation, create-agent
+- id: T06
+  state_hub_task_id: 6715aa6f-1ee0-4f22-9249-f1cd41763cd1
+  status: done
+  title: Docs, CLI cheat sheet, CHANGELOG for v1.4.0
+---
+
+# KAIZEN-WP-0007 — Agent Authoring & Doc Generation
+
+**Status:** done
+**Owner:** kaizen-agentic
+**Repo:** kaizen-agentic
+**Target version:** 1.4.0
+**Depends on:** WP-0006 (scheduled agent execution)
+
+## Goal
+
+Close the WP-0005/WP-0006 deferrals — **agent selection/authoring**, **template
+schema enforcement**, and **doc generation** — and fix the doc-generation defect
+they exposed. After this workplan, authoring a new agent and keeping project
+docs in sync is a first-class, idempotent, validated CLI flow.
+
+## Background
+
+`AgentInstaller._update_documentation()` regenerates the `## Installed Agents`
+block in a project's `CLAUDE.md`, but its regex
+(`## Installed Agents.*?(?=##|\Z)`) is non-greedy and the `(?=##)` lookahead
+matches `### Category` subheadings *inside* the section — so each run leaves the
+old category lists and footer in place and appends a fresh copy. The block
+duplicates and grows unbounded (reported to state-hub; the corrupting write is
+ours). `validate` only checks dependencies and file existence — it does not
+enforce the agent frontmatter schema — and there is no scaffolding command for
+new agents.
+
+## Tasks
+
+### T01 — Fix idempotent doc regeneration
+- Anchor the replace to a top-level heading: `(?=\n## (?!#)|\Z)`.
+- Add a regression test that runs regeneration twice and asserts exactly one
+  `## Installed Agents`, one footer, one block per category.
+- Clean the existing baseline duplication in this repo's `CLAUDE.md`.
+
+### T02 — Reusable agent-docs module + `docs generate`
+- Extract the "Installed Agents" rendering into a pure function
+  (`render_installed_agents_section`) and an idempotent
+  `upsert_installed_agents_section(content, section)`.
+- `_update_documentation` reuses them (no behavioral fork).
+- New `kaizen-agentic docs generate [--target PATH] [--check]` refreshes the
+  section for installed agents; `--check` exits non-zero if it would change
+  (CI-friendly).
+
+### T03 — Enforce agent frontmatter schema in `validate`
+- Add `validate_frontmatter_schema()` to the registry: required `name`,
+  `description`, `category`; `category` ∈ `AgentCategory`; `memory` ∈
+  {`enabled`,`disabled`} when present; `model` a non-empty string when present.
+- Surface schema errors in `kaizen-agentic validate` alongside dependency
+  errors, with actionable messages.
+
+### T04 — `create-agent` scaffold
+- `kaizen-agentic create-agent <name> [--category] [--description] [--memory]
+  [--target] [--force]` writes a schema-valid `agents/agent-<name>.md` with
+  frontmatter + section skeleton.
+- Missing `--category`/`--description` fall back to interactive prompts.
+- Refuses invalid category; refuses overwrite without `--force`; output passes
+  T03 validation.
+
+### T05 — Tests
+- `tests/test_agent_docs.py` — idempotency, `docs generate --check`.
+- `tests/test_validate_schema.py` — schema pass/fail cases.
+- `tests/test_create_agent.py` — scaffold + round-trips through registry.
+
+### T06 — Docs
+- `docs/CLI_CHEAT_SHEET.md` — `docs generate`, `create-agent`, schema validate.
+- `docs/agency-framework.md` — authoring + doc-sync note.
+- `CHANGELOG.md` `[Unreleased]`; `TODO.md` pointer.
+
+## Definition of Done
+
+- Doc regeneration is idempotent (N runs == 1 run); baseline cleaned.
+- `kaizen-agentic validate` fails on malformed agent frontmatter with clear errors.
+- `kaizen-agentic create-agent` produces an agent that passes `validate`.
+- `kaizen-agentic docs generate --check` is green on a synced repo.
+- Full test suite + `make release-check` green.
+
+## Out of Scope
+
+- Multi-file agent packages / protocol scaffolding (separate workplan).
+- Publishing generated docs to external knowledge bases (info-tech-canon).
+- Changing the agent frontmatter schema itself (only enforcing the current one).
+
+## Notes
+
+- Boundary: the doc-regeneration *trigger* (a state-hub routine invoking the
+  installer mid-session) is tracked separately in the state-hub inbox; this
+  workplan fixes the *write* so the trigger becomes harmless/idempotent.
+- `create-agent` writes to `agents/`; remember `make agents-sync-package` before
+  release so packaged `data/agents/` parity holds (existing release-check gate).