generated from coulomb/repo-seed
tegwick
df87e212a2
Ship a specialized profile for trading memoirs and market-structure texts. The profile names eight entity categories (trader, market, strategy, error, psychological_pattern, institution, instrument, evidence_bearing_claim), five relation types (cause_effect, lesson_evidence, risk_mitigation, actor_venue, strategy_outcome), and four evaluation criteria (groundedness, lesson_clarity, historical_context, overgeneralization_risk). Each is reflected in the prompts and contracts so the LLM is steered toward operator-level findings rather than biographical detail or moralising. The generic profile remains the default. A 2-chapter Lefevre smoke run with --profile trading-literature completes end-to-end with viable metrics; 93 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
140 lines
4.4 KiB
Markdown
140 lines
4.4 KiB
Markdown
# Generic Source Generator
|
|
|
|
Date: 2026-05-14
|
|
|
|
## Purpose
|
|
|
|
`infospace-bench generate` turns a local article, ebook-like file, or folder of
|
|
knowledge sources into a manifest-backed infospace. It generalizes the
|
|
Wealth/VSM pilot into an explicit workflow path with deterministic fixture
|
|
support and an optional OpenRouter provider.
|
|
|
|
## Deterministic Run
|
|
|
|
Use fixture responses for repeatable tests and demos:
|
|
|
|
```bash
|
|
infospace-bench generate from-source ./examples/article.md \
|
|
--workspace . \
|
|
--slug article-space \
|
|
--name "Article Space" \
|
|
--profile general-knowledge \
|
|
--fixture-responses ./examples/responses.yaml \
|
|
--apply
|
|
```
|
|
|
|
The command creates normalized source chunks, installs the selected profile,
|
|
runs the declared workflows, writes entities, relations, evaluations, metrics,
|
|
history, and a generation report, then registers artifacts in
|
|
`artifacts/index.yaml`.
|
|
|
|
## Stepwise Workflow
|
|
|
|
```bash
|
|
infospace-bench generate init ./book.epub \
|
|
--workspace . \
|
|
--slug book-space \
|
|
--name "Book Space" \
|
|
--profile general-knowledge \
|
|
--max-chunks 3
|
|
|
|
infospace-bench generate plan ./infospaces/book-space --stage all
|
|
infospace-bench generate run ./infospaces/book-space \
|
|
--fixture-responses ./responses.yaml
|
|
infospace-bench generate status ./infospaces/book-space
|
|
```
|
|
|
|
`--max-chunks` caps early experiments and provider cost. `generate status`
|
|
shows chunk counts, generated artifact counts, evaluations, metrics, history,
|
|
and stale source/profile inputs.
|
|
|
|
### Profiles
|
|
|
|
Two profiles ship today:
|
|
|
|
- `general-knowledge` — durable concepts, claims, methods, people,
|
|
places, works, and objects across any source
|
|
- `trading-literature` — trading memoirs and market-structure texts;
|
|
tunes entity categories (`trader`, `market`, `strategy`, `error`,
|
|
`psychological_pattern`, `institution`, `instrument`,
|
|
`evidence_bearing_claim`), relation types (`cause_effect`,
|
|
`lesson_evidence`, `risk_mitigation`, `actor_venue`,
|
|
`strategy_outcome`), and evaluation criteria (`groundedness`,
|
|
`lesson_clarity`, `historical_context`, `overgeneralization_risk`)
|
|
|
|
Select via `--profile trading-literature` on `generate init` or
|
|
`generate from-source`. The generic profile remains the default.
|
|
|
|
### Scale-aware plan
|
|
|
|
`generate plan` returns a compact estimate by default — counts of selected
|
|
chunks, calls per workflow, prompt-word and token estimates, and a rough
|
|
USD cost when `--cost-per-1k` is supplied. Long corpora no longer dump
|
|
hundreds of full prompts unless `--full` is set.
|
|
|
|
```bash
|
|
infospace-bench generate plan ./infospaces/book-space \
|
|
--from-chapter 1 --to-chapter 3 \
|
|
--cost-per-1k 0.30 \
|
|
--max-calls 50 \
|
|
--cost-cap 2.00
|
|
```
|
|
|
|
Selection filters:
|
|
|
|
- `--chapter LABEL` (repeatable) — match a chapter by roman/arabic label
|
|
or numeric value (e.g. `--chapter I` or `--chapter 2`)
|
|
- `--from-chapter N` / `--to-chapter N` — numeric chapter range
|
|
- `--chunk ID` (repeatable) — exact source chunk id (e.g.
|
|
`chapter-01-part-002`)
|
|
|
|
Budget flags `--max-calls` and `--cost-cap` are reported as
|
|
`exceeds_max_calls` / `exceeds_cost_cap` booleans in the summary, so a
|
|
caller can fail fast before invoking `run`. Use `--full` to opt back into
|
|
the full per-workflow plan with prompts for deep inspection.
|
|
|
|
## OpenRouter
|
|
|
|
Live model calls are explicit:
|
|
|
|
```bash
|
|
export OPENROUTER_API_KEY=...
|
|
|
|
infospace-bench generate run ./infospaces/book-space \
|
|
--provider openrouter \
|
|
--model openai/gpt-4o-mini \
|
|
--stage all
|
|
```
|
|
|
|
Choose the `--model` value from OpenRouter model IDs. The API key is read from
|
|
`OPENROUTER_API_KEY`; it is not written to `infospace.yaml`. Default tests never
|
|
make live provider calls.
|
|
|
|
## Resume
|
|
|
|
Use resume for interrupted or reviewed runs:
|
|
|
|
```bash
|
|
infospace-bench generate resume ./infospaces/book-space \
|
|
--provider openrouter \
|
|
--model openai/gpt-4o-mini
|
|
```
|
|
|
|
Unchanged completed runs are skipped. Use `--force` when you intentionally want
|
|
to rerun completed work. Stale status is reported when source artifact digests
|
|
or installed profile/template files change.
|
|
|
|
## Review Path
|
|
|
|
After generation:
|
|
|
|
- inspect `artifacts/sources/` for normalized input chunks
|
|
- inspect `artifacts/entities/` and `artifacts/relations/` for generated claims
|
|
- inspect `output/evaluations/` for rubric output
|
|
- run `infospace-bench validate <root>` and `infospace-bench graph <root>`
|
|
- review `reports/generation-summary.md`
|
|
|
|
Move from the generic profile to a specialized profile when the source domain
|
|
needs stricter terminology, narrower extraction granularity, or a discipline
|
|
lens such as VSM.
|