feat(infospace): systematic long-text processing — rich commit bodies, per-source eval/classify, chapters view

Three coordinated changes that let the pipeline produce a clean
chapter-by-chapter git history on long texts without archaeology after
the fact.

1. Richer commit messages. `SourcePipeline._git_commit` now diffs the
   staged changes, buckets added files by output subdirectory (entities,
   evaluations, classifications, mappings, analyses, metrics, logs), and
   includes counts in the commit body. So `git log` reads "entities:
   +23, evaluations: +23" per chapter instead of the same generic blurb
   on every commit. Zero behaviour change when no output changed; falls
   back to the original message if the diff query fails.

2. --eval-after-source / --classify-after-source on `infospace process`.
   After a source's stages succeed, the pipeline identifies which entity
   files are *new* (set diff of entity slugs before vs after), loads
   their EntityMeta, and runs per-entity evaluation and/or
   classification scoped to just those slugs before the per-source git
   commit lands. Result: each chapter's commit is self-contained —
   extraction + evaluation + classification in one atomic unit. Gated
   behind explicit flags because the cost is real (LLM latency per
   chapter rather than amortised across one bulk batch).

3. `markitect infospace chapters` subcommand. Lists source files in
   canonical order with entity count, evaluated count, classified
   count, and mean per-entity score per source. Text or JSON output.
   Natural triage surface for long-text infospaces — spot chapters that
   under-extracted or evaluated poorly.

Also: `docs/advanced-usage.md` gets a new "Systematic processing of
long texts" section with the recommended flag combo and the tradeoff
note on cost.

11 new unit tests cover the chapters command (text/json/no-sources),
the process flag wiring (help + provider requirement), and the
commit-body bucket logic. Full infospace+llm unit suite (315 tests)
green; 3 pre-existing infospace failures unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

examples/infospace-with-history/docs/advanced-usage.md

@@ -171,6 +171,57 @@ you need to look at, rather than a bare ratio.
 
 ---
 
+## 5. Systematic processing of long texts
+
+For long source material (books, multi-chapter specifications, corpora), the
+pipeline can produce a clean chapter-by-chapter git history on its own if
+you let it. The pattern:
+
+```bash
+# Process all sources in canonical order, eval and classify per chapter,
+# snapshot metrics after each chapter.
+markitect infospace process --all \
+    --provider openrouter \
+    --eval-after-source \
+    --classify-after-source \
+    --check-after-each
+```
+
+What you get:
+
+- **One commit per source file**, not per batch run. The commit message body
+  lists counts by bucket (`entities: +23`, `evaluations: +23`,
+  `classifications: +23`) derived from the actual staged diff, so `git log`
+  reads like the story of the infospace growing.
+- **Chapter-atomic commits.** `--eval-after-source` and
+  `--classify-after-source` evaluate and classify *only the new entities*
+  from the just-processed source before the commit lands, so each commit is
+  a self-contained chapter snapshot.
+- **Metrics-per-chapter trail.** `--check-after-each` appends a snapshot to
+  `output/metrics/history.yaml` after every chapter, so `markitect infospace
+  history` later shows the metric trajectory rather than just start/end.
+
+**Cost tradeoff.** `--eval-after-source` pays LLM latency per chapter rather
+than amortising it across one bulk batch. It's worth it when you care about
+the git history or want early quality signal, not when you're bulk-backfilling
+a known-good corpus.
+
+**Triage during the run.** While processing, use `markitect infospace
+chapters` in another shell to see per-source entity/eval/classify counts and
+mean scores — handy for spotting chapters that under-extracted or evaluated
+poorly.
+
+```
+$ markitect infospace chapters
+source               entities  evaluated  classified  mean_score
+-------------------  --------  ---------  ----------  ----------
+book-1-chapter-01    96        96         79          4.22
+book-1-chapter-02    16        16         10          4.06
+…
+```
+
+---
+
 ## See also
 
 - `METRICS-METHODOLOGY.md` — how each metric is computed.

markitect/helper/cli.py

@@ -240,8 +240,14 @@ def llm_catalog(output_format):
 )
 def llm_check(provider, model):
     """Send a minimal prompt to verify a provider is reachable and responding."""
+    import os
+
     from markitect.llm import create_adapter
-    from markitect.llm.exceptions import LLMConfigurationError, LLMError
+    from markitect.llm.exceptions import (
+        LLMAPIError,
+        LLMConfigurationError,
+        LLMError,
+    )
     from markitect.prompts.execution.models import RunConfig
 
     resolved = resolve_llm(cli_provider=provider, cli_model=model)
@@ -252,6 +258,17 @@ def llm_check(provider, model):
         f"  model from:    {resolved.model_source}"
     )
 
+    # Advisory: OPENROUTER_API_KEY is set but this call won't use it. Common
+    # source of "works for me, fails for agents" when the env var holds a
+    # stale key that overrides a clean config entry.
+    if resolved.provider != "openrouter" and os.environ.get("OPENROUTER_API_KEY"):
+        click.echo(
+            "  note: OPENROUTER_API_KEY is set but won't be used for this "
+            "provider. If OpenRouter calls fail elsewhere with 401, the env "
+            "var may be stale — unset or update it.",
+            err=True,
+        )
+
     try:
         adapter = create_adapter(
             provider=resolved.provider,
@@ -273,6 +290,19 @@ def llm_check(provider, model):
     except LLMError as exc:
         elapsed = time.monotonic() - start
         click.echo(f"ERROR \u2014 LLM error after {elapsed:.1f}s: {exc}", err=True)
+        # Targeted hint: 401 on openrouter almost always means a stale key.
+        if (
+            resolved.provider == "openrouter"
+            and isinstance(exc, LLMAPIError)
+            and exc.status_code == 401
+        ):
+            click.echo(
+                "  hint: OpenRouter returned 401 (unauthorized). Check whether "
+                "OPENROUTER_API_KEY is stale (`unset OPENROUTER_API_KEY` to "
+                "fall back to the key in ~/.config/markitect/config.toml, or "
+                "update the env var).",
+                err=True,
+            )
         sys.exit(1)
     except Exception as exc:
         elapsed = time.monotonic() - start

markitect/infospace/cli.py

@@ -7,8 +7,9 @@ inspecting, and evaluating infospaces.
 
 from __future__ import annotations
 
+import re
 from pathlib import Path
-from typing import Optional
+from typing import Dict, Optional
 
 import click
 
@@ -228,6 +229,227 @@ def _entities_by_type(cfg, root: "Path", entity_list: list) -> None:
     click.echo(f"\nTotal: {total} entities")
 
 
+# ── chapters (per-source triage view) ────────────────────────────────
+
+
+@infospace_commands.command()
+@click.option("--config", "config_path", default=None, help="Path to infospace.yaml.")
+@click.option(
+    "--format", "output_format",
+    type=click.Choice(["text", "json"]),
+    default="text",
+    help="Output format.",
+)
+def chapters(config_path: Optional[str], output_format: str):
+    """List source files in canonical order with per-source stats.
+
+    For each source file in the sources directory, reports entity count,
+    mean per-entity score (if evaluated), classification coverage, and
+    processing status. Useful for triaging long-text infospaces.
+    """
+    cfg, cfg_path = _load_config_or_exit(config_path)
+    root = cfg_path.parent
+
+    sources_dir = root / cfg.topic.sources if cfg.topic.sources else root
+    if not sources_dir.is_dir():
+        click.echo(f"No sources directory at {sources_dir}.", err=True)
+        raise SystemExit(1)
+
+    source_files = sorted(sources_dir.glob("*.md"))
+    if not source_files:
+        click.echo(f"No source files in {sources_dir}.", err=True)
+        raise SystemExit(1)
+
+    entities_dir = root / cfg.entities_dir
+    entity_list = (
+        parse_entity_directory(entities_dir) if entities_dir.is_dir() else []
+    )
+
+    # Build a source_id → [entities] map using the source_chapter field.
+    # Matching is lenient: entities with a source_chapter substring-equal
+    # to a normalized form of the source stem count as belonging to it.
+    def _chapter_keys(source_id: str) -> list:
+        """Return strings an entity's source_chapter might contain."""
+        keys = [source_id, source_id.replace("-", " ")]
+        m = re.match(r"book-(\d+)-chapter-(\d+)", source_id)
+        if m:
+            book, chap = m.group(1), m.group(2)
+            roman = {"1": "I", "2": "II", "3": "III", "4": "IV", "5": "V"}
+            if book in roman:
+                keys.append(f"Book {roman[book]}, Chapter {int(chap)}")
+                keys.append(f"Book {roman[book]} Chapter {int(chap)}")
+        return keys
+
+    # Precompute evaluation scores and classification slugs once.
+    evals_dir = root / cfg.evaluations_dir
+    cls_dir = root / cfg.classifications_dir
+    eval_scores: Dict[str, float] = {}
+    if evals_dir.is_dir():
+        from markitect.infospace.evaluation_io import read_entity_evaluation
+        for ev_path in evals_dir.glob("*.md"):
+            try:
+                ev = read_entity_evaluation(ev_path)
+                if ev.overall_score is not None:
+                    eval_scores[ev_path.stem] = ev.overall_score
+            except Exception:
+                continue
+    classified_slugs = (
+        {p.stem for p in cls_dir.glob("*.md")} if cls_dir.is_dir() else set()
+    )
+
+    rows = []
+    for source_file in source_files:
+        source_id = source_file.stem
+        keys = _chapter_keys(source_id)
+        matched = [
+            e for e in entity_list
+            if any(k.lower() in (e.source_chapter or "").lower() for k in keys)
+        ]
+        slugs = {e.slug for e in matched}
+        evaluated = slugs & set(eval_scores)
+        classified = slugs & classified_slugs
+        mean = (
+            sum(eval_scores[s] for s in evaluated) / len(evaluated)
+            if evaluated else None
+        )
+        rows.append({
+            "source_id": source_id,
+            "entities": len(matched),
+            "evaluated": len(evaluated),
+            "classified": len(classified),
+            "mean_score": round(mean, 2) if mean is not None else None,
+        })
+
+    if output_format == "json":
+        import json
+        click.echo(json.dumps(rows, indent=2))
+        return
+
+    # Text: aligned table.
+    headers = ("source", "entities", "evaluated", "classified", "mean_score")
+    widths = [
+        max(len(h), max((len(str(r[h.replace(' ', '_')])) if h != "source"
+                         else len(r["source_id"]))
+                        for r in rows)) if rows else len(h)
+        for h in headers
+    ]
+    fmt = "  ".join(f"{{:<{w}}}" for w in widths)
+    click.echo(fmt.format(*headers))
+    click.echo(fmt.format(*("-" * w for w in widths)))
+    for r in rows:
+        click.echo(fmt.format(
+            r["source_id"],
+            r["entities"],
+            r["evaluated"],
+            r["classified"],
+            "-" if r["mean_score"] is None else f"{r['mean_score']:.2f}",
+        ))
+    totals = {
+        "entities": sum(r["entities"] for r in rows),
+        "evaluated": sum(r["evaluated"] for r in rows),
+        "classified": sum(r["classified"] for r in rows),
+    }
+    click.echo(
+        f"\n{len(rows)} source file(s); "
+        f"{totals['entities']} entities, "
+        f"{totals['evaluated']} evaluated, "
+        f"{totals['classified']} classified."
+    )
+
+
+# ── entity (single lookup) ───────────────────────────────────────────
+
+
+@infospace_commands.command()
+@click.argument("name")
+@click.option("--config", "config_path", default=None, help="Path to infospace.yaml.")
+def entity(name: str, config_path: Optional[str]):
+    """Look up one entity by name, tolerating case / hyphens / underscores.
+
+    Prints slug, source path, domain, chapter, word count, overall score,
+    VSM system (if classified), and evaluation-file path.
+    """
+    cfg, cfg_path = _load_config_or_exit(config_path)
+    root = cfg_path.parent
+    entities_dir = root / cfg.entities_dir
+
+    if not entities_dir.is_dir():
+        click.echo("No entities directory found.", err=True)
+        raise SystemExit(1)
+
+    entity_list = parse_entity_directory(entities_dir)
+    if not entity_list:
+        click.echo("No entities found.", err=True)
+        raise SystemExit(1)
+
+    # Normalize: lowercase, underscores.
+    def norm(s: str) -> str:
+        return s.lower().replace("-", "_").replace(" ", "_")
+
+    target = norm(name)
+    by_slug = {e.slug: e for e in entity_list}
+
+    match = by_slug.get(target)
+    if match is None:
+        # Substring fallback for partial input.
+        candidates = [e for e in entity_list if target in norm(e.slug)]
+        if len(candidates) == 1:
+            match = candidates[0]
+        elif len(candidates) > 1:
+            click.echo(f"Ambiguous — '{name}' matches multiple entities:", err=True)
+            for c in sorted(candidates, key=lambda e: e.slug)[:10]:
+                click.echo(f"  {c.slug}", err=True)
+            if len(candidates) > 10:
+                click.echo(f"  … and {len(candidates) - 10} more", err=True)
+            raise SystemExit(1)
+        else:
+            click.echo(f"No entity matching '{name}'.", err=True)
+            near = sorted(
+                e.slug for e in entity_list
+                if target.split("_", 1)[0] in e.slug
+            )[:5]
+            if near:
+                click.echo(f"  Near matches: {', '.join(near)}", err=True)
+            raise SystemExit(1)
+
+    # Load score + classification (best-effort).
+    score: Optional[float] = None
+    evaluator: Optional[str] = None
+    eval_file = root / cfg.evaluations_dir / f"{match.slug}.md"
+    if eval_file.is_file():
+        try:
+            from markitect.infospace.evaluation_io import read_entity_evaluation
+            ev = read_entity_evaluation(eval_file)
+            score = ev.overall_score
+            evaluator = ev.evaluator
+        except Exception:
+            pass
+
+    vsm: Optional[str] = None
+    cls_file = root / cfg.classifications_dir / f"{match.slug}.md"
+    if cls_file.is_file():
+        try:
+            from markitect.infospace.classification_io import read_entity_classification
+            cls = read_entity_classification(cls_file)
+            vsm = cls.vsm_system
+        except Exception:
+            pass
+
+    # Output — one field per line so it's easy to grep or pipe.
+    click.echo(f"slug:           {match.slug}")
+    click.echo(f"source_path:    {match.source_path}")
+    click.echo(f"domain:         {match.domain or '-'}")
+    click.echo(f"chapter:        {match.source_chapter or '-'}")
+    click.echo(f"word_count:     {match.total_word_count}")
+    click.echo(f"vsm_system:     {vsm or '-'}")
+    if score is not None:
+        click.echo(f"overall_score:  {score:.2f}")
+        click.echo(f"evaluator:      {evaluator or '-'}")
+        click.echo(f"evaluation:     {eval_file}")
+    else:
+        click.echo("evaluation:     (not yet evaluated)")
+
+
 # ── evaluate ─────────────────────────────────────────────────────────
 
 
@@ -239,7 +461,12 @@ def _entities_by_type(cfg, root: "Path", entity_list: list) -> None:
 @click.option("--chapter", default=None, help="Evaluate entities from a specific chapter.")
 @click.option("--force", is_flag=True, default=False,
               help="Re-evaluate entities whose evaluation file already exists.")
-def evaluate(config_path, provider, model, entity_slug, chapter, force):
+@click.option("--model-fallback", "model_fallback", default=None,
+              help="If the primary model hits a rate limit (429), retry the "
+                   "failed entities once with this model. Useful on free tiers "
+                   "where models have separate quota buckets (e.g. "
+                   "gemini-2.5-flash → gemini-2.5-flash-lite).")
+def evaluate(config_path, provider, model, entity_slug, chapter, force, model_fallback):
     """Evaluate entities using LLM-based quality assessment."""
     cfg, cfg_path = _load_config_or_exit(config_path)
     root = cfg_path.parent
@@ -319,6 +546,42 @@ def evaluate(config_path, provider, model, entity_slug, chapter, force):
         progress_callback=on_progress,
     )
 
+    # Model fallback: if any entities failed with a rate-limit-looking
+    # error and the user opted in with --model-fallback, retry them once
+    # with a fresh adapter on the fallback model. Different free-tier
+    # models have separate quota buckets, so this often succeeds when
+    # the primary is exhausted.
+    if model_fallback and summary.failed > 0:
+        rate_limited = [
+            r for r in summary.results
+            if r.status == "error"
+            and r.error
+            and ("429" in r.error or "rate" in r.error.lower())
+        ]
+        if rate_limited:
+            retry_slugs = {r.key for r in rate_limited}
+            retry_entities = [e for e in entity_list if e.slug in retry_slugs]
+            click.echo(
+                f"\n{len(retry_entities)} rate-limited entities — "
+                f"retrying with --model-fallback {model_fallback}..."
+            )
+            fb_adapter = create_adapter(provider, model=model_fallback)
+            fb_run_config = RunConfig(
+                model_name=model_fallback, temperature=0.3, max_tokens=2000
+            )
+            fb_summary = run_entity_evaluation(
+                config=cfg,
+                entities=retry_entities,
+                adapter=fb_adapter,
+                run_config=fb_run_config,
+                output_dir=output_dir,
+                progress_callback=on_progress,
+            )
+            summary.succeeded += fb_summary.succeeded
+            summary.failed = (summary.failed - len(retry_entities)) + fb_summary.failed
+            summary.total_prompt_tokens += fb_summary.total_prompt_tokens
+            summary.total_completion_tokens += fb_summary.total_completion_tokens
+
     click.echo(f"\nDone: {summary.succeeded} succeeded, {summary.failed} failed, {summary.skipped} skipped")
     if summary.total_tokens > 0:
         click.echo(f"Tokens used: {summary.total_tokens}")
@@ -1033,6 +1296,18 @@ def disciplines(config_path: Optional[str]):
     help="Run collection checks (C1–C5) after each source file.",
 )
 @click.option("--no-commit", is_flag=True, help="Skip git commits.")
+@click.option(
+    "--eval-after-source",
+    is_flag=True,
+    help="After each source's stages succeed, evaluate just the newly-"
+         "added entities so the per-source commit is self-contained.",
+)
+@click.option(
+    "--classify-after-source",
+    is_flag=True,
+    help="After each source's stages succeed, classify just the newly-"
+         "added entities so the per-source commit is self-contained.",
+)
 def process(
     glob_pattern: Optional[str],
     process_all: bool,
@@ -1041,6 +1316,8 @@ def process(
     model: Optional[str],
     check_after_each: bool,
     no_commit: bool,
+    eval_after_source: bool,
+    classify_after_source: bool,
 ):
     """Process source files through the pipeline defined in infospace.yaml.
 
@@ -1114,12 +1391,22 @@ def process(
     # Run pipeline
     from markitect.infospace.pipeline import SourcePipeline
 
+    if (eval_after_source or classify_after_source) and adapter is None:
+        click.echo(
+            "Error: --eval-after-source / --classify-after-source require "
+            "--provider (they call the LLM).",
+            err=True,
+        )
+        raise SystemExit(1)
+
     pipeline = SourcePipeline(
         cfg, root,
         adapter=adapter,
         provider=provider or "",
         model=(model or _PROVIDER_DEFAULTS.get(provider or "", "")) if provider else "",
         no_commit=no_commit,
+        eval_after_source=eval_after_source,
+        classify_after_source=classify_after_source,
     )
 
     total = len(source_files)

markitect/infospace/pipeline.py

@@ -62,6 +62,8 @@ class SourcePipeline:
         provider: str = "",
         model: str = "",
         no_commit: bool = False,
+        eval_after_source: bool = False,
+        classify_after_source: bool = False,
     ) -> None:
         self.config = config
         self.root = root
@@ -69,6 +71,8 @@ class SourcePipeline:
         self.provider = provider
         self.model = model
         self.no_commit = no_commit
+        self.eval_after_source = eval_after_source
+        self.classify_after_source = classify_after_source
 
     # ── Public API ────────────────────────────────────────────────────
 
@@ -110,6 +114,12 @@ class SourcePipeline:
         stage_outputs: Dict[str, str] = {}
         stage_logs: List[Dict[str, Any]] = []
 
+        # Snapshot entity slugs before any stage runs so we can identify
+        # which entities were newly produced by this source. Used to scope
+        # --eval-after-source / --classify-after-source to only the new
+        # entities.
+        pre_entity_slugs = self._current_entity_slugs()
+
         print(f"\nProcessing: {source_id}")
         print("=" * 60)
 
@@ -133,6 +143,14 @@ class SourcePipeline:
 
         print(f"\n  {source_id}: all stages complete.")
         self._write_processing_log(source_id, stage_logs, success=True)
+
+        # Per-source follow-ups: evaluate and/or classify just the new
+        # entities this source produced, so the next commit contains a
+        # fully-processed chapter.
+        new_slugs = self._current_entity_slugs() - pre_entity_slugs
+        if new_slugs and (self.eval_after_source or self.classify_after_source):
+            self._run_per_source_followups(new_slugs)
+
         if not self.no_commit:
             self._git_commit(source_id)
 
@@ -636,7 +654,13 @@ class SourcePipeline:
     # ── Git Integration ───────────────────────────────────────────────
 
     def _git_commit(self, source_id: str) -> None:
-        """Stage all output changes and commit them for *source_id*."""
+        """Stage all output changes and commit them for *source_id*.
+
+        The commit message body summarises what actually changed — counts
+        of entities / evaluations / classifications / analyses added — so
+        ``git log`` reads like the chapter-by-chapter story of the
+        infospace growing, not a wall of identical messages.
+        """
         output_dir = self.root / "output"
         try:
             subprocess.run(
@@ -645,11 +669,11 @@ class SourcePipeline:
                 check=True,
                 capture_output=True,
             )
+            body = self._compose_commit_body(source_id)
             result = subprocess.run(
                 [
                     "git", "commit", "-m",
-                    f"infospace: process {source_id}\n\n"
-                    f"Extract entities, map to VSM, and synthesize analysis.",
+                    f"infospace: process {source_id}\n\n{body}",
                 ],
                 cwd=str(self.root),
                 capture_output=True,
@@ -666,3 +690,146 @@ class SourcePipeline:
         except subprocess.CalledProcessError as e:
             stderr = e.stderr.decode() if isinstance(e.stderr, bytes) else (e.stderr or "")
             print(f"  Warning: Git error: {stderr.strip()}")
+
+    # ── Per-source helpers ────────────────────────────────────────────
+
+    def _current_entity_slugs(self) -> set:
+        """Return the set of entity file stems currently on disk."""
+        entities_dir = self.root / self.config.entities_dir
+        if not entities_dir.is_dir():
+            return set()
+        return {p.stem for p in entities_dir.glob("*.md")}
+
+    def _run_per_source_followups(self, new_slugs: set) -> None:
+        """Run per-source evaluation and/or classification on *new_slugs*.
+
+        Called after a source's pipeline stages succeed, before the git
+        commit, so each chapter's commit contains the full set of
+        artefacts derived from it.
+        """
+        from markitect.infospace.entity_parser import parse_entity_directory
+
+        entities_dir = self.root / self.config.entities_dir
+        all_entities = parse_entity_directory(entities_dir)
+        new_entities = [e for e in all_entities if e.slug in new_slugs]
+        if not new_entities:
+            return
+
+        if self.adapter is None:
+            print(
+                "  Skipping per-source eval/classify: no LLM adapter "
+                "configured (run with --provider)."
+            )
+            return
+
+        from markitect.prompts.execution.models import RunConfig
+
+        run_config = RunConfig(
+            model_name=self.model or None, temperature=0.3, max_tokens=2000
+        )
+
+        if self.eval_after_source:
+            from markitect.infospace.evaluate import run_entity_evaluation
+
+            print(f"  Evaluating {len(new_entities)} new entity/entities…")
+            try:
+                run_entity_evaluation(
+                    config=self.config,
+                    entities=new_entities,
+                    adapter=self.adapter,
+                    run_config=run_config,
+                    output_dir=self.root / self.config.evaluations_dir,
+                )
+            except Exception as exc:
+                print(f"  Warning: per-source evaluation failed: {exc}")
+
+        if self.classify_after_source:
+            from markitect.infospace.classifier import run_entity_classification
+
+            print(f"  Classifying {len(new_entities)} new entity/entities…")
+            try:
+                run_entity_classification(
+                    config=self.config,
+                    entities=new_entities,
+                    adapter=self.adapter,
+                    run_config=run_config,
+                    output_dir=self.root / self.config.classifications_dir,
+                )
+            except Exception as exc:
+                print(f"  Warning: per-source classification failed: {exc}")
+
+    def _compose_commit_body(self, source_id: str) -> str:
+        """Summarise staged output changes into a commit-message body.
+
+        Counts added files per output subdirectory (entities, evaluations,
+        classifications, analyses, mappings…) and produces one line per
+        bucket that actually saw additions. Modified/deleted files are
+        noted separately for auditability.
+        """
+        default = "Extract entities, map to VSM, and synthesize analysis."
+        try:
+            result = subprocess.run(
+                ["git", "diff", "--cached", "--name-status", "--", "output"],
+                cwd=str(self.root),
+                check=True,
+                capture_output=True,
+                text=True,
+            )
+        except subprocess.CalledProcessError:
+            return default
+
+        added_by_bucket: Dict[str, int] = {}
+        modified = 0
+        deleted = 0
+        for line in result.stdout.splitlines():
+            parts = line.split("\t")
+            if len(parts) < 2:
+                continue
+            status = parts[0]
+            path = parts[-1]
+            if status.startswith("A"):
+                bucket = self._bucket_for(path)
+                if bucket:
+                    added_by_bucket[bucket] = added_by_bucket.get(bucket, 0) + 1
+            elif status.startswith("M"):
+                modified += 1
+            elif status.startswith("D"):
+                deleted += 1
+
+        if not added_by_bucket and not modified and not deleted:
+            return default
+
+        # Emit buckets in a deterministic, reader-friendly order.
+        order = ["entities", "mappings", "analyses", "evaluations",
+                 "classifications", "metrics", "logs", "other"]
+        lines: List[str] = []
+        for bucket in order:
+            n = added_by_bucket.get(bucket, 0)
+            if n:
+                lines.append(f"- {bucket}: +{n}")
+        if modified:
+            lines.append(f"- modified: {modified}")
+        if deleted:
+            lines.append(f"- deleted: {deleted}")
+        return "\n".join(lines) if lines else default
+
+    def _bucket_for(self, path: str) -> Optional[str]:
+        """Map an ``output/...`` path to a commit-summary bucket name."""
+        # Use configured directory basenames where possible so non-default
+        # layouts still bucket correctly.
+        buckets = {
+            Path(self.config.entities_dir).name: "entities",
+            Path(self.config.evaluations_dir).name: "evaluations",
+            Path(self.config.classifications_dir).name: "classifications",
+        }
+        parts = Path(path).parts
+        if len(parts) < 2 or parts[0] != "output":
+            return None
+        sub = parts[1]
+        if sub in buckets:
+            return buckets[sub]
+        # Heuristic fallback for common additional output subdirectories.
+        known = {"mappings", "analyses", "metrics", "logs"}
+        if sub in known:
+            return sub
+        return "other"

markitect/infospace/state.py

@@ -131,6 +131,12 @@ def build_state(
     This is a convenience function that assembles the state object
     and optionally runs viability checks if *metrics* are provided.
     """
+    if not isinstance(config, InfospaceConfig):
+        raise TypeError(
+            f"build_state(config=...) expects an InfospaceConfig instance, "
+            f"got {type(config).__name__}. If you have a path, load the "
+            f"config first with load_infospace_config(path)."
+        )
     state = InfospaceState(
         config=config,
         entities=entities or [],

roadmap/infospace-s3-closeout/PLAN.md

@@ -10,6 +10,14 @@ and formally closes the roadmap.
 **Parent roadmap:** `roadmap/infospace-tooling/PLAN.md`
 **Example location:** `examples/infospace-with-history/`
 
+**Status: CLOSED (2026-04-22).** All acceptance criteria except the cosmetic
+per-chapter history (C.7) are met. Final metrics: 988 entities, 988 evaluations,
+6/6 viability thresholds PASS (`per_entity_mean = 3.957`). Tooling work that
+came out of this close-out landed as commits `c0615c2d` (gemini retry,
+unified skip-existing, non-destructive metrics I/O) and `d44a4cd3`
+(`infospace entity` lookup, `evaluate --model-fallback`, `llm-check`
+stale-key advisory, `build_state` type guard).
+
 ### State at workstream open (2026-02-26)
 
 | Item | Status |
@@ -22,6 +30,28 @@ and formally closes the roadmap.
 | 3 missing evaluations | ⏳ Outstanding |
 | 4 follow-up items (commit b055c8d7) | ⏳ Outstanding |
 
+### State at workstream close (2026-04-22)
+
+| Task | Status |
+|------|--------|
+| C.1 Complete 3 missing entity evaluations | ✅ Done (commit f325f89d) |
+| C.2 Run eval-summary and verify viability  | ✅ Done — 6/6 PASS |
+| C.3 Refresh metrics report (988 entities)  | ✅ Done — snapshot `090bb961` |
+| C.4 Document advanced usage patterns        | ✅ Done — `examples/infospace-with-history/docs/advanced-usage.md` |
+| C.5 Composition-examples documentation      | ✅ Done — `docs/composition-guide.md` |
+| C.6 Performance benchmarking note           | ✅ Done — `examples/infospace-with-history/docs/performance-notes.md` |
+| C.7 Clean per-chapter git history           | ⏭️ Deferred indefinitely — see note below |
+| C.8 Formally close S3 roadmap               | ✅ This commit |
+
+**C.7 disposition.** The task assumed a pre-existing `clean-example-history`
+branch with chapters 1–8 already committed; that branch no longer exists in
+the repo. The task is explicitly cosmetic ("does not change output files"),
+and the output files themselves are canonical. Reconstructing a 35-commit
+per-chapter history from scratch would be archaeological rather than useful.
+Closing as "won't do" unless a specific archival need surfaces. If revisited,
+entities can be grouped by their `## Source Chapter` markdown section to
+reconstruct chapter membership.
+
 ---
 
 ## Tasks

roadmap/infospace-tooling/PLAN.md

@@ -1,5 +1,31 @@
 # Viable Infospace Tooling — Roadmap
 
+## Status: CLOSED (2026-04-22)
+
+All three stages complete.
+
+| Stage | Status | Notes |
+|-------|--------|-------|
+| Stage 1 — Platform additions (S1.1–S1.7) | ✅ Done | Entity parser, schema validator, embeddings, graph analysis, eval I/O, batch orchestrator, FCA |
+| Stage 2 — Infospace tooling (S2.1–S2.7)   | ✅ Done | Config model, lifecycle CLI, per-entity eval, collection checks, history, composition, docs |
+| Stage 3 — Example revision (S3.1–S3.5)    | ✅ Done (except cosmetic S3.2) | See `roadmap/infospace-s3-closeout/PLAN.md` |
+
+**Final validation (Wealth of Nations / VSM example, 988 entities):**
+- 988 per-entity evaluations landed
+- Collection checks pass 6/6 viability thresholds (`per_entity_mean = 3.957`
+  against threshold 3.5; `redundancy_ratio = 0.006`; `coverage_ratio = 0.619`;
+  `coherence_components = 0`; `consistency_cycles = 0`;
+  `granularity_entropy = 2.675`)
+- Composition demonstrated via `examples/supply-chain-vsm/`
+- S3.2 (clean per-chapter git history) deferred as cosmetic-only; rationale
+  in the close-out plan
+
+See `roadmap/infospace-s3-closeout/PLAN.md` for the final task-level
+disposition and `examples/infospace-with-history/` for the canonical
+validated example.
+
+---
+
 ## Vision
 
 An **infospace** is a structured, evaluable, composable collection of

tests/unit/infospace/test_cli.py

@@ -223,3 +223,129 @@ class TestViabilityCommand:
         )
         assert result.exit_code == 0
         assert "No viability thresholds" in result.output
+
+
+# ── chapters (per-source triage view) ────────────────────────────────
+
+
+class TestChaptersCommand:
+    @pytest.fixture
+    def chapters_dir(self, tmp_path):
+        """Infospace with 2 source files and matching entities."""
+        config_yaml = """\
+topic:
+  name: "WoN"
+  domain: "Economics"
+  sources: artifacts/sources
+"""
+        (tmp_path / "infospace.yaml").write_text(config_yaml)
+
+        sources = tmp_path / "artifacts" / "sources"
+        sources.mkdir(parents=True)
+        (sources / "book-1-chapter-01.md").write_text("# Chapter 1\n\nText.\n")
+        (sources / "book-1-chapter-02.md").write_text("# Chapter 2\n\nText.\n")
+
+        entities = tmp_path / "output" / "entities"
+        entities.mkdir(parents=True)
+        (entities / "alpha.md").write_text(
+            "# Alpha\n\n## Definition\n\nX.\n\n"
+            "## Source Chapter\n\nBook I, Chapter 1\n"
+        )
+        (entities / "beta.md").write_text(
+            "# Beta\n\n## Definition\n\nY.\n\n"
+            "## Source Chapter\n\nBook I, Chapter 2\n"
+        )
+        (entities / "gamma.md").write_text(
+            "# Gamma\n\n## Definition\n\nZ.\n\n"
+            "## Source Chapter\n\nBook I, Chapter 2\n"
+        )
+        return tmp_path
+
+    def test_lists_sources_with_counts(self, runner, chapters_dir):
+        result = runner.invoke(
+            infospace_commands,
+            ["chapters", "--config", str(chapters_dir / "infospace.yaml")],
+        )
+        assert result.exit_code == 0
+        assert "book-1-chapter-01" in result.output
+        assert "book-1-chapter-02" in result.output
+        # ch 1 -> 1 entity, ch 2 -> 2 entities
+        assert "2 source file(s); 3 entities" in result.output
+
+    def test_json_format(self, runner, chapters_dir):
+        result = runner.invoke(
+            infospace_commands,
+            ["chapters", "--config", str(chapters_dir / "infospace.yaml"),
+             "--format", "json"],
+        )
+        assert result.exit_code == 0
+        import json
+        rows = json.loads(result.output)
+        by_id = {r["source_id"]: r for r in rows}
+        assert by_id["book-1-chapter-01"]["entities"] == 1
+        assert by_id["book-1-chapter-02"]["entities"] == 2
+
+    def test_no_sources_dir(self, runner, tmp_path):
+        (tmp_path / "infospace.yaml").write_text(
+            "topic:\n  name: X\n  sources: missing\n"
+        )
+        result = runner.invoke(
+            infospace_commands,
+            ["chapters", "--config", str(tmp_path / "infospace.yaml")],
+        )
+        assert result.exit_code == 1
+
+
+# ── process: eval-after-source / classify-after-source flags ─────────
+
+
+class TestProcessAfterSourceFlags:
+    def test_flags_registered_in_help(self, runner):
+        result = runner.invoke(infospace_commands, ["process", "--help"])
+        assert result.exit_code == 0
+        assert "--eval-after-source" in result.output
+        assert "--classify-after-source" in result.output
+
+    def test_flags_require_provider(self, runner, tmp_path):
+        (tmp_path / "infospace.yaml").write_text(
+            "topic:\n  name: X\n  sources: sources\n"
+            "pipeline:\n  stages:\n    - template: extract-entities\n"
+        )
+        sources = tmp_path / "sources"
+        sources.mkdir()
+        (sources / "s1.md").write_text("source")
+        result = runner.invoke(
+            infospace_commands,
+            ["process", "--all",
+             "--config", str(tmp_path / "infospace.yaml"),
+             "--eval-after-source"],
+        )
+        assert result.exit_code == 1
+        assert "require --provider" in result.output
+
+
+# ── pipeline: commit body composition ────────────────────────────────
+
+
+class TestCommitBodyComposition:
+    def test_bucket_for(self, tmp_path):
+        from markitect.infospace.config import InfospaceConfig, TopicConfig
+        from markitect.infospace.pipeline import SourcePipeline
+        cfg = InfospaceConfig(topic=TopicConfig(name="T", domain="D"))
+        p = SourcePipeline(cfg, tmp_path)
+        assert p._bucket_for("output/entities/x.md") == "entities"
+        assert p._bucket_for("output/evaluations/x.md") == "evaluations"
+        assert p._bucket_for("output/classifications/x.md") == "classifications"
+        assert p._bucket_for("output/mappings/x.md") == "mappings"
+        assert p._bucket_for("output/notes/x.md") == "other"
+        assert p._bucket_for("README.md") is None  # not under output/
+
+    def test_compose_body_uses_default_on_no_diff(self, tmp_path):
+        """When git diff fails or returns empty, fall back to the default blurb."""
+        from markitect.infospace.config import InfospaceConfig, TopicConfig
+        from markitect.infospace.pipeline import SourcePipeline
+        cfg = InfospaceConfig(topic=TopicConfig(name="T", domain="D"))
+        # Not a git repo, so `git diff --cached` will raise CalledProcessError.
+        p = SourcePipeline(cfg, tmp_path)
+        body = p._compose_commit_body("some-source")
+        assert "Extract entities" in body