Files

tegwick 5b6a63fb7a IB-WP-0016-T01: spine-aware EPUB3 intake

Parse META-INF/container.xml and the OPF package document, then iterate
documents in spine reading order instead of archive-name sort. Classify
each spine item (body, cover, nav, toc, header, footer, notes, license,
auxiliary) and exclude non-body sections by default; include_non_body=True
opts them back in for inspection. Capture OPF book metadata (title,
creator, language, subjects, rights, identifier, source_url, modified)
onto every chunk and propagate it through source artifact provenance.
Preserve the legacy zip-without-OPF fallback for malformed EPUBs.

Real Lefevre EPUB now yields 148 body chunks in spine order (was 155
mixed, archive-sorted) with cover=1, header=1, footer=4 detected and
dropped. 78 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-17 13:52:24 +02:00

3.4 KiB

Raw Blame History

Lefevre EPUB3 Validation

Date: 2026-05-14

Source

Local source file:

/mnt/c/Users/bernd.worsch/Downloads/LefevreEdwin-ReminiscencesOfAStockOperator.epub

The EPUB is Project Gutenberg edition 60979, EPUB package version 3.0. The OPF metadata identifies:

title: Reminiscences of a Stock Operator
creator: Edwin Lefevre
subjects: Speculation, New York Stock Exchange, Investments
rights: public domain in the USA

Current Infrastructure Result

The current generic generator can initialize a disposable infospace from the file and run non-provider metrics:

disposable root: /tmp/infospace-bench-lefevre-583mopy_/infospaces/reminiscences-stock-operator
source chunks: 155
entities: 0
relations: 0
evaluations: 0
stale status: false
metrics snapshot: 5978ece0

The source-only metrics were:

redundancy ratio: 0.9225806451612903
coverage ratio: 1.0
coherence components: 155.0
consistency cycles: 0.0
granularity entropy: -0.0

Findings

The EPUB intake works mechanically, but it is not ready for a serious full-book OpenRouter generation run.

EPUB spine order is visible in OEBPS/content.opf, but current intake reads XHTML files by archive-name sorting.
Current titles mostly collapse to the same long Gutenberg page title instead of chapter labels such as I, II, and III.
Current intake includes non-body material such as cover/header/footer/license candidates unless the caller manually filters after import.
generate plan is not yet a compact cost/risk plan for a long book; a full all-stage run would imply hundreds of provider calls.
Resume state is run-level enough for the small generic path, but a long ebook needs chunk-level retry, stale, and skip policy.
Cross-chunk entity deduplication and merge/review policy are needed before a full narrative book becomes a coherent infospace.

Desired Readiness Bar

Before building the real Lefevre infospace with OpenRouter, the CLI should be able to show:

book metadata and selected source sections
body-only chapter order
stable chapter/chunk IDs
estimated provider call count and token/cost budget
selected chapter or chunk filters for smoke runs
deterministic fixture acceptance on a small Lefevre-like subset
optional live one-chapter smoke run with explicit provider/model/cost caps

T01 Result (2026-05-17)

Spine-aware EPUB3 intake landed. Re-running the local Lefevre EPUB through normalize_source(...) now yields:

148 body chunks (default), down from the original 155 mixed chunks
Spine reading order: indices 0..27 in declared order, not archive-name sort
Full OPF metadata on every chunk's book_metadata: title, creator, language, subjects, rights, identifier, source_url, modified
Section roles classified across the 154 spine items: body=148, footer=4, cover=1, header=1
The four Gutenberg footer/license/notes sections and the *** START OF… header section are now excluded from generation input by default and available via include_non_body=True for inspection
The legacy zip-without-OPF fallback path is preserved for malformed EPUBs

The remaining gap is title collapse: all body sections still share the Project Gutenberg page title because chapter headings are not yet read from in-document <h1> content. That collapse is T02's scope (chapter-aware chunking and stable IDs from in-document headings).

3.4 KiB Raw Blame History