# Generic Source Generator Date: 2026-05-14 ## Purpose `infospace-bench generate` turns a local article, ebook-like file, or folder of knowledge sources into a manifest-backed infospace. It generalizes the Wealth/VSM pilot into an explicit workflow path with deterministic fixture support and an optional OpenRouter provider. ## Deterministic Run Use fixture responses for repeatable tests and demos: ```bash infospace-bench generate from-source ./examples/article.md \ --workspace . \ --slug article-space \ --name "Article Space" \ --profile general-knowledge \ --fixture-responses ./examples/responses.yaml \ --apply ``` The command creates normalized source chunks, installs the selected profile, runs the declared workflows, writes entities, relations, evaluations, metrics, history, and a generation report, then registers artifacts in `artifacts/index.yaml`. ## Stepwise Workflow ```bash infospace-bench generate init ./book.epub \ --workspace . \ --slug book-space \ --name "Book Space" \ --profile general-knowledge \ --max-chunks 3 infospace-bench generate plan ./infospaces/book-space --stage all infospace-bench generate run ./infospaces/book-space \ --fixture-responses ./responses.yaml infospace-bench generate status ./infospaces/book-space ``` `--max-chunks` caps early experiments and provider cost. `generate status` shows chunk counts, generated artifact counts, evaluations, metrics, history, and stale source/profile inputs. ### Live OpenRouter runs (handle with care) A single-chapter live run is the only OpenRouter shape the test suite covers today. Use `--chapter` (or `--from-chapter` / `--to-chapter`) on `generate init` or `generate from-source` to scope what gets registered before any provider calls happen: ```bash export OPENROUTER_API_KEY=... # Preview the cost first infospace-bench generate plan ./infospaces/foo --chapter I --cost-per-1k 0.30 # Run only Chapter I against a cheap model infospace-bench generate from-source ./LEFEVRE.epub \ --workspace ./infospaces \ --slug reminiscences-ch1 \ --name "Reminiscences (Ch I)" \ --profile trading-literature \ --provider openrouter \ --model openai/gpt-4o-mini \ --chapter I \ --apply ``` `output/budget/plans.yaml`, `usage.yaml`, and `summary.yaml` record what was estimated, what was actually spent, and the plan-vs-actual delta. `output/workflows/runs/*.yaml` carry the OpenRouter request_id, model, token usage, retry count, and per-call duration; the same metadata reaches the entity/relation/evaluation artifacts via `provenance.provider_metadata`. Before scaling to the full book: - Inspect each chapter's outputs and `generation-summary.md` - Multiply the per-chapter `total_provider_calls_estimate` and `estimated_cost_usd` by the chapter count and compare to your budget - Decide on a final model and confirm the rate-table entry exists in `src/infospace_bench/model_rates.yaml` or your workspace override The optional live-smoke test in `tests/test_openrouter_live.py` is skipped unless both `OPENROUTER_API_KEY` and `INFOSPACE_BENCH_ENABLE_LIVE_OPENROUTER=1` are set. It runs a single chapter through the same path and asserts the provider metadata plumb-through. ### Live runs with `--provider routing` When the routing CLI is what you want to exercise live, swap `--provider openrouter --model ...` for the routing pair: ```bash infospace-bench generate from-source ./LEFEVRE.epub \ --workspace ./infospaces \ --slug reminiscences-routed \ --name "Reminiscences (Routed)" \ --profile trading-literature \ --provider routing \ --routing-config ./examples/routing/trading-literature.yaml \ --chapter I \ --apply ``` `examples/routing/trading-literature.yaml` is a checked-in starting config: cheap candidates for summary/evaluation, smart candidates for entity/relation, a `claude_code` baseline rule for future shadow sampling, and a workspace-relative `output/routing/quality.jsonl` ledger so adaptive observations stay with the workspace. `--quality-floor ` on the same command overrides the config's `default_quality_floor` for a single invocation — useful for tightening the bar for a specific run without editing the file. The ledger fills up as the `AdaptiveRoutingPolicy` records each observation; later runs against the same workspace get the benefit without re-grading from scratch. The parallel live-smoke test (`test_provider_routing_one_chapter_live_smoke`) is also gated on `INFOSPACE_BENCH_ENABLE_LIVE_OPENROUTER=1` + `OPENROUTER_API_KEY` and asserts the per-stage adapter-choices report section names the routed model. ### Budget and usage registry Every `generate plan` invocation appends a compact snapshot to `output/budget/plans.yaml` (deterministic 12-char `snapshot_id`, 50-entry sliding retention). Every `generate run` invocation appends a usage rollup to `output/budget/usage.yaml`, bucketed by `(workflow_id, stage_id, provider, model)` with prompt and completion token counts, known cost (when the adapter returned it), and estimated cost (when a rate table entry matches the model). The default rate table is bundled at `src/infospace_bench/model_rates.yaml` and covers a handful of common OpenRouter models at list price (see the file for the captured-at timestamp). A workspace can override or extend entries by placing `model-rates.yaml` next to its `infospaces/` directory; the workspace file is overlaid on top of the package default so partial overrides are fine. Cost resolution order on each run: adapter-returned `cost` first, then the rate table, then `cost_status="unknown"` (recorded explicitly, never silently zeroed). The plan-vs-actual variance summary lands in follow-on task T04. ### Profiles Two profiles ship today: - `general-knowledge` — durable concepts, claims, methods, people, places, works, and objects across any source - `trading-literature` — trading memoirs and market-structure texts; tunes entity categories (`trader`, `market`, `strategy`, `error`, `psychological_pattern`, `institution`, `instrument`, `evidence_bearing_claim`), relation types (`cause_effect`, `lesson_evidence`, `risk_mitigation`, `actor_venue`, `strategy_outcome`), and evaluation criteria (`groundedness`, `lesson_clarity`, `historical_context`, `overgeneralization_risk`) Select via `--profile trading-literature` on `generate init` or `generate from-source`. The generic profile remains the default. ### Scale-aware plan `generate plan` returns a compact estimate by default — counts of selected chunks, calls per workflow, prompt-word and token estimates, and a rough USD cost when `--cost-per-1k` is supplied. Long corpora no longer dump hundreds of full prompts unless `--full` is set. ```bash infospace-bench generate plan ./infospaces/book-space \ --from-chapter 1 --to-chapter 3 \ --cost-per-1k 0.30 \ --max-calls 50 \ --cost-cap 2.00 ``` Selection filters: - `--chapter LABEL` (repeatable) — match a chapter by roman/arabic label or numeric value (e.g. `--chapter I` or `--chapter 2`) - `--from-chapter N` / `--to-chapter N` — numeric chapter range - `--chunk ID` (repeatable) — exact source chunk id (e.g. `chapter-01-part-002`) Budget flags `--max-calls` and `--cost-cap` are reported as `exceeds_max_calls` / `exceeds_cost_cap` booleans in the summary, so a caller can fail fast before invoking `run`. Use `--full` to opt back into the full per-workflow plan with prompts for deep inspection. ## OpenRouter Live model calls are explicit: ```bash export OPENROUTER_API_KEY=... infospace-bench generate run ./infospaces/book-space \ --provider openrouter \ --model openai/gpt-4o-mini \ --stage all ``` Choose the `--model` value from OpenRouter model IDs. The API key is read from `OPENROUTER_API_KEY`; it is not written to `infospace.yaml`. Default tests never make live provider calls. ## Resume Use resume for interrupted or reviewed runs: ```bash infospace-bench generate resume ./infospaces/book-space \ --provider openrouter \ --model openai/gpt-4o-mini ``` Unchanged completed runs are skipped. Use `--force` when you intentionally want to rerun completed work. Stale status is reported when source artifact digests or installed profile/template files change. ## Review Path After generation: - inspect `artifacts/sources/` for normalized input chunks - inspect `artifacts/entities/` and `artifacts/relations/` for generated claims - inspect `output/evaluations/` for rubric output - run `infospace-bench validate ` and `infospace-bench graph ` - review `reports/generation-summary.md` Move from the generic profile to a specialized profile when the source domain needs stricter terminology, narrower extraction granularity, or a discipline lens such as VSM.