coulomb/markitect-main
2
0
Fork 0
Code Issues 135 Pull Requests Actions 1 Projects 12 Releases 1 Wiki Activity

feat(llm): add Gemini adapter and process book-1-chapter-05

Browse Source 
Add GeminiAdapter calling Google's Generative Language REST API
(default model: gemini-2.5-flash). Register "gemini" as third
provider in the factory and CLI. Add rate-limit retry with
exponential backoff to the pipeline's _call_llm helper. Increase
default max_tokens from 2000 to 4096.

Process book-1-chapter-05 via Gemini free tier — 1 new entity
extracted (necessaries-conveniencies-and-amusements-of-life),
41 existing entities correctly skipped by dedup. Canonical set
now at 42 unique entities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Bernd Worsch
2026-02-11 22:54:37 +01:00
parent 2d1282a61e
commit 880c1d1374
22 changed files with 12008 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
examples/infospace-with-history/TUTORIAL.md
View File

@@ -283,11 +283,15 @@ python process_chapters.py --chapter book-1-chapter-05 --no-commit
# Automatic mode via OpenRouter (recommended — fast, real token counts):
python process_chapters.py --chapter book-1-chapter-05 --provider openrouter --no-commit
# Automatic mode via Gemini free tier:
python process_chapters.py --chapter book-1-chapter-05 --provider gemini --no-commit
# Automatic mode via Claude Code CLI:
python process_chapters.py --chapter book-1-chapter-05 --provider claude-code --no-commit
# With a specific model:
python process_chapters.py --chapter book-1-chapter-05 --provider openrouter --model anthropic/claude-haiku-4-5-20251001 --no-commit
python process_chapters.py --chapter book-1-chapter-05 --provider gemini --model gemini-2.5-flash-lite --no-commit
```
### Running a whole book
@@ -321,10 +325,10 @@ Available chapters (35):
book-1-chapter-02 done (7) done done
book-1-chapter-03 done (18) done done
book-1-chapter-04 done (5) done done
book-1-chapter-05 - - -
book-1-chapter-05 done (1) done done
...
Canonical entity set: 41 unique entities
Canonical entity set: 42 unique entities
```
### Assessing metrics
@@ -348,13 +352,14 @@ python process_chapters.py --stats
## 7. How the LLM Integration Works
The pipeline uses MarkiTect's `markitect.llm` module, which provides two
The pipeline uses MarkiTect's `markitect.llm` module, which provides three
adapter backends that implement the `LLMAdapter` interface:
| Backend | How it works | Pros | Cons |
|---------|-------------|------|------|
| `openrouter` | HTTP POST to OpenRouter API | Fast, real token counts, any model | Needs API key |
| `claude-code` | Shells out to `claude --print` | No API key needed if CLI installed | Slower, estimated token counts |
| `gemini` | HTTP POST to Google Generative Language API | Free tier available, real token counts | Rate limits on free tier |
### API key setup (OpenRouter)
@@ -364,6 +369,16 @@ Place your key in one of these locations (checked in order):
2. Set `OPENROUTER_API_KEY` environment variable
3. Create `apikey-openrouter.txt` in the project root (git-ignored)
### API key setup (Gemini)
Place your Google AI Studio key in one of these locations (checked in order):
1. Set `GEMINI_API_KEY` environment variable
2. Create `apikey-geminifree.txt` in the project root (git-ignored)
Get a free API key at https://aistudio.google.com/apikey. The free tier
supports `gemini-2.5-flash` with generous rate limits.
### What happens per stage
1. The pipeline **resolves** macro placeholders by looking up artifacts
@@ -427,33 +442,39 @@ git commit -m "infospace: process book-1-chapter-05"
## 9. Cost and Performance
From our measurements processing chapters 3 and 4:
From our measurements processing chapters 3-5:
| | Claude Code CLI | OpenRouter |
|---|---|---|
| Time per chapter | ~5 minutes | ~2 minutes |
| Token counts | Estimated (4 chars/tok) | Real (from API) |
| Cost per chapter | ~$0.35 est. | ~$0.07 est. |
| | Claude Code CLI | OpenRouter | Gemini (free) |
|---|---|---|---|
| Time per chapter | ~5 minutes | ~2 minutes | ~45 seconds |
| Token counts | Estimated (4 chars/tok) | Real (from API) | Real (from API) |
| Cost per chapter | ~$0.35 est. | ~$0.07 est. | Free |
| Default model | claude-sonnet-4 | anthropic/claude-sonnet-4 | gemini-2.5-flash |
**Projected cost for all 35 chapters via OpenRouter:** ~$2.50
(varies by chapter length; Book V chapters are longer).
**Gemini free tier** is a zero-cost option for experimentation and
development. The `gemini-2.5-flash` model produces good results.
Rate limits apply — the pipeline retries automatically on 429 responses.
To reduce costs further, use a cheaper model:
```bash
--provider openrouter --model anthropic/claude-haiku-4-5-20251001
--provider gemini --model gemini-2.5-flash-lite
```
---
## 10. Completing the Remaining Chapters
As of now, 4 of 35 chapters are processed (Book I, Chapters 1-4). Here is
As of now, 5 of 35 chapters are processed (Book I, Chapters 1-5). Here is
how to complete the rest.
### Step-by-step
**1. Process remaining Book I chapters (5-11):**
**1. Process remaining Book I chapters (6-11):**
```bash
python process_chapters.py --book 1 --provider openrouter --no-commit