Files

tegwick 001b64d67b IB-WP-0016: refresh validation baseline after T01/T02 smoke run

Run a fixture-backed end-to-end smoke against the real Lefevre EPUB
(max-chunks 3) and capture the result in the validation note and the
workplan. The pipeline produces a complete infospace with stable
chapter-01-part-NNN source IDs, full chapter/book/anchor provenance on
every source artifact, viable metrics, and exact-title entity dedupe.

Refresh the workplan validation baseline to reflect the post-T01/T02
state, and add a remaining-gaps section that maps the open issues to the
right follow-on tasks: cost/scope controls and plan preview to T03, the
trading-literature profile to T04, chunk-level resume to T06, and a
richer generation-summary report (entity titles, chapter coverage,
anchor links) to T07.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-17 16:13:39 +02:00

5.5 KiB

Raw Blame History

Lefevre EPUB3 Validation

Date: 2026-05-14

Source

Local source file:

/mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub

The EPUB is Project Gutenberg edition 60979, EPUB package version 3.0. The OPF metadata identifies:

title: Reminiscences of a Stock Operator
creator: Edwin Lefevre
subjects: Speculation, New York Stock Exchange, Investments
rights: public domain in the USA

Current Infrastructure Result

The current generic generator can initialize a disposable infospace from the file and run non-provider metrics:

disposable root: /tmp/infospace-bench-lefevre-583mopy_/infospaces/reminiscences-stock-operator
source chunks: 155
entities: 0
relations: 0
evaluations: 0
stale status: false
metrics snapshot: 5978ece0

The source-only metrics were:

redundancy ratio: 0.9225806451612903
coverage ratio: 1.0
coherence components: 155.0
consistency cycles: 0.0
granularity entropy: -0.0

Findings

The EPUB intake works mechanically, but it is not ready for a serious full-book OpenRouter generation run.

EPUB spine order is visible in OEBPS/content.opf, but current intake reads XHTML files by archive-name sorting.
Current titles mostly collapse to the same long Gutenberg page title instead of chapter labels such as I, II, and III.
Current intake includes non-body material such as cover/header/footer/license candidates unless the caller manually filters after import.
generate plan is not yet a compact cost/risk plan for a long book; a full all-stage run would imply hundreds of provider calls.
Resume state is run-level enough for the small generic path, but a long ebook needs chunk-level retry, stale, and skip policy.
Cross-chunk entity deduplication and merge/review policy are needed before a full narrative book becomes a coherent infospace.

Desired Readiness Bar

Before building the real Lefevre infospace with OpenRouter, the CLI should be able to show:

book metadata and selected source sections
body-only chapter order
stable chapter/chunk IDs
estimated provider call count and token/cost budget
selected chapter or chunk filters for smoke runs
deterministic fixture acceptance on a small Lefevre-like subset
optional live one-chapter smoke run with explicit provider/model/cost caps

T01 Result (2026-05-17)

Spine-aware EPUB3 intake landed. Re-running the local Lefevre EPUB through normalize_source(...) now yields:

148 body chunks (default), down from the original 155 mixed chunks
Spine reading order: indices 0..27 in declared order, not archive-name sort
Full OPF metadata on every chunk's book_metadata: title, creator, language, subjects, rights, identifier, source_url, modified
Section roles classified across the 154 spine items: body=148, footer=4, cover=1, header=1
The four Gutenberg footer/license/notes sections and the *** START OF… header section are now excluded from generation input by default and available via include_non_body=True for inspection
The legacy zip-without-OPF fallback path is preserved for malformed EPUBs

The remaining gap is title collapse: all body sections still share the Project Gutenberg page title because chapter headings are not yet read from in-document <h1> content. That collapse is T02's scope (chapter-aware chunking and stable IDs from in-document headings).

T02 Result (2026-05-17)

Chapter-aware chunking and stable IDs landed. The same local Lefevre EPUB now produces:

67 body chunks (default max_words=800 collapses to 24 single-chunk chapters once max_words=2000)
All 24 roman-numeral chapters detected and assigned stable IDs chapter-01 .. chapter-24; multi-part chapters get chapter-NN-part-001, chapter-NN-part-002, ...
Chapter labels resolved from the EPUB nav doc (when present) and from the first in-document <h2>/<h1> heading
Project Gutenberg page-title collapse is gone: each chunk's title is the chapter label, not the shared book title
TOC body section ("Contents") reclassified to toc; transcriber's notes section reclassified to notes; section-role histogram is now body=67, cover=1, header=1, toc=1, notes=1, footer=2
Page anchors of the form id="Page_N" are preserved per chunk via the page_anchors provenance field (e.g. chapter-01 carries Page_1..Page_14 distributed across its three parts)
Optional overlap_words parameter supports evidence-window context between adjacent parts of the same chapter without duplicating headings

Fixture Smoke Run (2026-05-17)

generate from-source ... --fixture-responses ... --max-chunks 3 --apply against the real EPUB produced a complete infospace:

3 source chunks (chapter-01-part-001..003) and 3 entities/relations/ evaluations plus the generation-summary report
artifacts/index.yaml carries full T01/T02 provenance on every source artifact (chapter_label, chapter_number, page_anchors, OPF book_metadata)
Metrics viable: coverage=1.0, redundancy=0.0, granularity_entropy ≈ 1.79; viability gates pass
Repeated same-title entities upserted to single artifact files — basic exact-title dedupe works; near-duplicate dedupe is still open

Gaps the smoke surfaced for follow-on tasks:

generation-summary.md is just counts + metrics; needs entity titles, chapter coverage, page-anchor links for review (T07)
No plan cost preview, no chapter/cost cap selection — running the full book at default max_words is ~335 provider calls (T03)
Generic profile shaped the output, not Lefevre's trading vocabulary (T04)

5.5 KiB Raw Blame History