research: ikiwiki deep dive (compile-to-static, git-distributed); UC-79

SHARD-WP-0003 T4. Wiki compiler: git-canonical Markdown source -> static HTML
(derived publish/projection, not the canonical store). Two new insights atop
the git-IS-store home case: compile-to-static = clean canonical-source vs
derived-output separation (attach source repo, UC-37/UC-56), and a third
federation flavor -- git replication + XML-RPC change-ping (pinger), beside
fedwiki fork/journal (UC-72) and Wikibase SERVICE (UC-74). UC-79. Enriched
UC-31/56/37/33. Marks T4 done. Feeds SHARD-WP-0002 T4/T6/T14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

SCOPE.md

@@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary.
 |-------|-------|
 | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) |
 | Intent | `INTENT.md` established; authorization-in-core amendments drafted |
-| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
+| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis + TiddlyWiki + ikiwiki deep dives & shard-spectrum synthesis (`research/260614-*/`) |
 | Demand | NetKingdom integration asks captured, not yet negotiated |
-| Spec | Architecture blueprint drafted; UseCaseCatalog 78 UCs from research; PRD/TSD scaffolds |
+| Spec | Architecture blueprint drafted; UseCaseCatalog 79 UCs from research; PRD/TSD scaffolds |
 | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract); `SHARD-WP-0003` active (9 engine dives); `SHARD-WP-0004` active (8 computational-knowledge dives) |
 
 ## In Scope (today)

research/260614-ikiwiki-deep-dive/README.md

@@ -0,0 +1,15 @@
+# 260614 — ikiwiki deep dive
+
+Deep dive on **ikiwiki** (Joey Hess): a **wiki compiler** — it compiles a **VCS-backed**
+(usually **git**) tree of Markdown source into **static HTML**, supports **distributed**
+operation (clone/edit/push between wiki instances, change **pings**), and treats web edits as
+commits to the same repo.
+
+- `findings.md` — the compiler model, git-distributed federation + pinger, static output,
+  capability profile, INTENT mapping, UC seed (UC-79), architecture notes for SHARD-WP-0002,
+  open questions, sources, traceability.
+
+Catalog yield: UC-79 (attach a git-backed **compile-to-static** wiki — git Markdown source
+is the shard, compiled static HTML is a derived publish/projection; participate in
+git-distributed clone federation with change-pings). Enriched UC-31/56/37/33. Feeds
+SHARD-WP-0002 T4 (federation), T6 (publish/projection).

research/260614-ikiwiki-deep-dive/findings.md

@@ -0,0 +1,153 @@
+# ikiwiki — deep dive (findings)
+
+**Date:** 2026-06-14 · **Source:** SHARD-WP-0003 T4 · **Subject:** ikiwiki, Joey Hess's
+VCS-backed wiki compiler.
+
+## Why this dive
+
+The forge-wiki dive (T5) established *git-IS-the-store* for hosted Markdown wikis. ikiwiki
+takes the same git-canonical source but adds two ideas shard-wiki cares about directly:
+**compile-to-static** (the wiki is *built*, not served from a DB) and **git-distributed
+federation** (wiki instances clone, pull, push, and **ping** each other). It is referenced
+in `research/260608-federation-concepts/`; here we go into the model.
+
+## 1. The wiki-compiler model
+
+ikiwiki is fundamentally a **compiler**: input is a directory of **source pages** (Markdown
+by default; also other formats) held in a **version-control repo**; output is a tree of
+**static HTML**. A rebuild is triggered by a **VCS post-commit/post-update hook**, so:
+
+- **The VCS repo is canonical**; the HTML is *derived build output* (regenerable, disposable).
+- **Web edits are commits.** The CGI edit interface writes the change *into the repo* (a
+  commit) and triggers a rebuild — so browser edits and `git push` edits converge on one
+  history. (Same convergence as forge wikis, but here the canonical store is *your* git repo,
+  not a forge's.)
+- **VCS-agnostic**: git is usual, but svn/bzr/mercurial/darcs are supported via a VCS
+  plugin layer — an early "pluggable backend behind a stable interface" (adapter-contract
+  echo).
+- **Plugins** (Perl) provide directives, feeds, auth (`openid`), and the federation hooks
+  below.
+
+## 2. Git-distributed federation + the pinger
+
+Because the source is an ordinary VCS repo, ikiwiki instances federate the way *git* does:
+
+- **Clone-and-diverge**: you can `git clone` a wiki, edit offline, and `push`/`pull` between
+  instances — **multiple wiki clones that reconcile via git merge**. A wiki is a branch-space
+  (UC-33).
+- **`pinger` / `pingee` plugins**: an instance can send an **XML-RPC ping** to another
+  ikiwiki when it changes, prompting the other to **pull and rebuild** — a lightweight
+  **subscribe/notify** primitive over the git-distributed mesh (UC-31).
+- **`aggregate` plugin**: pulls external **RSS/Atom feeds** into the wiki as pages — an
+  inbound projection of remote content.
+
+So ikiwiki is *federation by git plus a ping* — distinct from fedwiki's fork/journal
+(UC-72) and from Wikibase's query-time `SERVICE` (UC-74): a **third federation flavor**,
+*VCS-replication federation with change notification*.
+
+## 3. Static output as a publish/projection target
+
+The compiled static HTML is a **read-only, regenerable projection** of the source:
+
+- It is a natural **outbound publish target** (UC-56): render the union (or a shard) to a
+  static site for hosting/backup, no server needed.
+- It is also the **read-only backup** end (UC-37): a static snapshot that survives the engine.
+
+The key shard-wiki framing: **source (git Markdown) is the attachable shard; static HTML is a
+derived projection** — never confuse the build output for the canonical content.
+
+## 4. Capability profile
+
+| Dimension (synthesis spectrum) | ikiwiki |
+|--------------------------------|---------|
+| Attachment mode | **file-store** (VCS-backed git Markdown source) |
+| Addressing granularity | page = source file; path = identity |
+| Content identity | path/filename (placement-bound) |
+| Structure | directory tree of Markdown source + directives |
+| History | **native VCS (git) history** |
+| Merge model | **git** (clone/pull/push/merge across instances) |
+| Native query | none; directives + plugins compute derived pages at build |
+| Translation | Markdown source → static HTML (build-time render) |
+| Write granularity | **file (page)** per commit |
+| Operational envelope | a compiler + VCS hook; static hosting for output |
+| Access grant | VCS/file perms; `openid` for web edits |
+| Content opacity | transparent Markdown |
+| Provenance | git author/timestamp; aggregated feeds carry source |
+| Federation | **git replication + XML-RPC ping**; RSS aggregation |
+
+## 5. INTENT mapping
+
+### Reinforcements
+
+- **Git-canonical Markdown source** = the home case (shared with forge wikis UC-76/40):
+  attach the source repo, adopt its git log as the journal, write by commit.
+- **Coordination layer is git** (INTENT): ikiwiki's whole federation *is* git replication +
+  a ping — the most literal realization of "Git-addressable coordination layer."
+- **Projection vs canonical**: compile-to-static cleanly separates **canonical source** from
+  **derived output** — exactly shard-wiki's projection principle (static HTML = a lazy/cache
+  projection that is regenerable, never the source of truth).
+- **Graceful degradation / publish**: static output is the trivial read-only backup and
+  outbound publish target (UC-37/UC-56).
+- **Subscribe/notify mechanism, not policy**: the pinger is a *mechanism* (notify a peer to
+  pull); *which* peers, when, and conflict policy stay configurable.
+
+### Divergences (boundaries / notes)
+
+- ikiwiki is mostly a **reinforcement** of git-canonical-Markdown (UC-76) — its *new*
+  contributions are (a) **compile-to-static** as a distinct projection/publish direction and
+  (b) the **git-distributed-clone + ping** federation flavor. The static output is *not* a
+  shard to attach (it's derived); attach the **source repo**.
+- VCS-agnostic backend layer is an interesting adapter-contract echo, but for shard-wiki the
+  git case dominates.
+
+### What to keep
+
+1. **Source-repo-as-shard, static-output-as-derived-projection** — never attach the build
+   output as canonical (UC-79; relates UC-37/UC-56).
+2. **Git-replication + change-ping** as a named federation flavor beside fork/journal
+   (fedwiki) and query-SERVICE (Wikibase) — a **subscribe/notify over a git mesh** (UC-31/UC-33).
+3. **Inbound feed aggregation** (RSS/Atom → pages) as an inbound projection pattern.
+
+## 6. UC seed
+
+| # | Seed | Disposition |
+|---|------|-------------|
+| UC-79 | Attach a git-backed **compile-to-static** wiki — the **git Markdown source is the shard**, compiled **static HTML is a derived publish/projection**; participate in **git-distributed clone federation** with change-**pings** | **new** |
+| — | XML-RPC pinger = subscribe/notify over a git mesh | enrich **UC-31** |
+| — | render union/shard to a static site (publish) | enrich **UC-56** |
+| — | static HTML as read-only regenerable backup | enrich **UC-37** |
+| — | wiki as a git branch-space; clones reconcile via merge | enrich **UC-33** |
+
+## 7. Architecture notes for SHARD-WP-0002
+
+- **T4 (federation):** record **git-replication + change-ping** as a federation flavor — peers
+  hold git clones, reconcile via merge, and notify with a ping to pull/rebuild. Distinct from
+  fedwiki fork/journal and Wikibase `SERVICE`; complements the "Git-addressable coordination
+  layer" mandate most literally.
+- **T6 (publish/projection):** **compile-to-static** is the canonical *outbound* projection —
+  source (git) is canonical, static HTML is a regenerable derived view (publish/backup
+  target). Reinforces projection-vs-canonical separation.
+- **Attach binding (T14):** attach the **source repo** (git Markdown), not the build output;
+  shares the forge-wiki / `wiki/`-subdir git+Markdown adapter (parameterized by repo/path).
+
+## 8. Open questions
+
+1. Is the **pinger** (notify-peer-to-pull) modeled as shard-wiki's own subscribe/notify
+   primitive, or only recognized when bridging two ikiwiki instances?
+2. Does shard-wiki ever *drive* a compile-to-static **publish** of the union (act as the
+   ikiwiki-like compiler), or only attach existing ikiwiki source repos? (UC-56 scope.)
+3. Is **feed aggregation** (RSS→pages) an inbound projection mode shard-wiki offers generally
+   (a feed-shard), or an ikiwiki-internal feature?
+
+## 9. Sources
+
+- ikiwiki.info — *ikiwiki* overview, *rcs* (VCS backends), *plugins* (`pinger`/`pingee`,
+  `aggregate`, `openid`), *setup* / post-commit hooks
+- prior: `research/260608-federation-concepts/` (ikiwiki reference);
+  `research/260614-forge-wikis-deep-dive/` (git-canonical Markdown contrast)
+
+## 10. Traceability
+
+New UC **UC-79** carries the marker **⊟** in the wikiengines column of
+`spec/UseCaseCatalog.md`. Enriched: UC-31, UC-56, UC-37, UC-33. Architecture cross-refs:
+SHARD-WP-0002 T4 (git-replication+ping federation), T6 (compile-to-static publish), T14.

research/README.md

@@ -31,4 +31,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and
 | 2026-06-14 | `260614-federated-wiki-deep-dive/` | Federated Wiki — fork-with-provenance, per-page semantic-action journal (story=replay), neighborhood/roster + chorus; prior art for our coordination journal / overlay / union pillars; UC-70/71/72 |
 | 2026-06-14 | `260614-wikibase-deep-dive/` | Wikibase/Wikidata — typed entity-statement knowledge graph (claim+qualifiers+refs+rank), SPARQL/RDF + federated SERVICE, opaque stable IDs, statement-level provenance; structure & query far-end; UC-73/74/75 |
 | 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 |
-| 2026-06-14 | `260614-tiddlywiki-deep-dive/` | TiddlyWiki — entire wiki (content + app) in one self-contained HTML file = whole-file write-granularity extreme; Node `.tid` file-per-tiddler substrate (git-diffable); tiddler/field records, filter-expression query; UC-78 |
+| 2026-06-14 | `260614-tiddlywiki-deep-dive/` | TiddlyWiki — entire wiki (content + app) in one self-contained HTML file = whole-file write-granularity extreme; Node `.tid` file-per-tiddler substrate (git-diffable); tiddler/field records, filter-expression query; UC-78 |
+| 2026-06-14 | `260614-ikiwiki-deep-dive/` | ikiwiki — wiki compiler: git-canonical Markdown source → static HTML (derived publish/projection); git-distributed clone federation + XML-RPC pinger (third federation flavor); UC-79 |

spec/UseCaseCatalog.md

@@ -15,7 +15,8 @@ Promoted from `research/260608-c2-wiki-origins/`,
 `research/260614-federated-wiki-deep-dive/`, and
 `research/260614-wikibase-deep-dive/`, and
 `research/260614-forge-wikis-deep-dive/`, and
-`research/260614-tiddlywiki-deep-dive/`.
+`research/260614-tiddlywiki-deep-dive/`, and
+`research/260614-ikiwiki-deep-dive/`.
 See InfoTechPrimers on coulomb.social for use-case catalog conventions.
 
 ## Conventions
@@ -197,7 +198,11 @@ concrete push transport: an engine event bus (`ObservationManager`
 eventual consistency and the rate limit (`research/260614-notion-deep-dive/findings.md`
 §4). **Joplin** is poll-based and turns concurrent edits into **conflict notes**
 (keep-both, not silent overwrite) — conflict-as-data, links UC-07
-(`research/260614-joplin-deep-dive/findings.md` §2).  
+(`research/260614-joplin-deep-dive/findings.md` §2). The **ikiwiki** dive details the
+canonical pinger: an instance sends an **XML-RPC ping** to a peer when it changes, prompting
+the peer to **pull and rebuild** — a subscribe/notify *mechanism* over a **git-distributed
+mesh** (UC-79, `research/260614-ikiwiki-deep-dive/findings.md` §2), the federation flavor
+beside fedwiki fork/journal (UC-72) and Wikibase `SERVICE` (UC-74).  
 **Priority:** Later
 
 ### UC-32 — Transclude remote span with live freshness
@@ -1075,6 +1080,23 @@ ignore the embedded engine). Feeds SHARD-WP-0002 T11 (whole-file tier), T14 (sin
 `.tid` binding; cf. UC-43/UC-62).  
 **Priority:** Later
 
+### UC-79 — Attach a git-backed compile-to-static wiki (source is the shard)
+
+**Actor:** Orchestrator / adapter  
+**Goal:** Attach a **compile-to-static** wiki (ikiwiki) where the **git Markdown source is
+the shard** and the **compiled static HTML is a derived publish/projection**; optionally
+participate in **git-distributed clone federation** with change-**pings**.  
+**Source:** wikiengines, federation, intent  
+**Notes:** ikiwiki compiles a VCS-backed (git) Markdown tree to static HTML; web edits are
+commits, so git is canonical and HTML is regenerable build output
+(`research/260614-ikiwiki-deep-dive/findings.md` §1, §3). **Attach the source repo, never
+the build output** (the static site is a projection, UC-37/UC-56). Its federation is **git
+replication + an XML-RPC pinger** (notify a peer to pull/rebuild, UC-31) — a third
+federation flavor beside fedwiki fork/journal (UC-72) and Wikibase `SERVICE` (UC-74). Shares
+the git+Markdown adapter with forge wikis (UC-76). Feeds SHARD-WP-0002 T4 (federation), T6
+(publish/projection).  
+**Priority:** Later
+
 ---
 
 ## B. Knowledge work and collaboration
@@ -1350,6 +1372,7 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD.
 | UC-76 | | | | ⎇ | ✓ |
 | UC-77 | | | | ⎇ | ✓ |
 | UC-78 | | | | ⊡ | ✓ |
+| UC-79 | | | ✓ | ⊟ | ✓ |
 | UC-08 | ✓ | | |
 | UC-09 | ✓ | | |
 | UC-10 | ✓ | | |
@@ -1881,6 +1904,33 @@ are a real native-query tier (UC-52). **Boundary recorded:** treat the single HT
 Architecture logged for `SHARD-WP-0002` (T11/T14): whole-file as the coarsest write tier,
 single-file-container vs `.tid`-dir dual binding.
 
+### ikiwiki mapping
+
+(⊟ UC-79 lineage = the **ikiwiki deep dive**, `research/260614-ikiwiki-deep-dive/findings.md`;
+marked in both federation and wikiengines columns.)
+
+| ikiwiki mechanism (findings §) | Catalog UC |
+|--------------------------------|------------|
+| VCS(git)-backed Markdown source compiled to static HTML; web edits = commits (§1, §3) | UC-79 (new) |
+| `pinger`/`pingee` XML-RPC = notify a peer to pull/rebuild (§2) | UC-31 (enriched) |
+| Compile union/shard to a static site = outbound publish (§3) | UC-56 (enriched) |
+| Static HTML = read-only regenerable backup (§3) | UC-37 (enriched) |
+| Clone/pull/push between instances; wiki = git branch-space (§2) | UC-33 (enriched) |
+| `aggregate` (RSS/Atom → pages) = inbound feed projection (§2) | links UC-03 |
+| VCS-agnostic backend behind a plugin layer (§1) | links UC-43 |
+
+Note: ikiwiki is a **wiki compiler** — git-canonical Markdown **source** built into **static
+HTML** — so it mostly **reinforces** the git-IS-store home case (UC-76/40), but adds two
+genuinely new things: **compile-to-static** as a clean *canonical-source vs derived-output*
+separation (attach the **source repo**, treat static HTML as a regenerable
+projection/publish target, UC-37/UC-56), and a **third federation flavor** — **git
+replication + an XML-RPC change-ping** (peers hold clones, merge to reconcile, notify to
+pull) beside fedwiki fork/journal (UC-72) and Wikibase `SERVICE` (UC-74). **Boundary
+recorded:** never attach the build output as canonical; the pinger is a subscribe/notify
+*mechanism* (peers/timing/conflict stay policy). Architecture logged for `SHARD-WP-0002`
+(T4/T6/T14): git-replication+ping federation, compile-to-static publish, source-repo attach
+sharing the git+Markdown adapter.
+
 ---
 
 ## Open questions
@@ -1951,5 +2001,9 @@ single-file-container vs `.tid`-dir dual binding.
 26. For a **single-file wiki** (UC-78), does shard-wiki support per-page overlays by
     buffering and re-serializing the whole file, or require the Node `.tid` substrate for
     write-through and treat single-file as read/projection/backup only? (TiddlyWiki dive §9.)
+27. For a **compile-to-static** wiki (UC-79), does shard-wiki ever *drive* the build to
+    **publish the union** as a static site (act as the compiler), or only attach existing
+    source repos — and is the **pinger** modeled as shard-wiki's own subscribe/notify
+    primitive or only recognized when bridging two ikiwiki instances? (ikiwiki dive §8.)
 23. How does shard-wiki **honor/surface a shard's path-based access rules** (UC-06) in a
     projection without re-implementing its ACL engine? (Wiki.js dive §9.)

workplans/SHARD-WP-0003-engine-dives-batch.md

@@ -100,7 +100,7 @@ model. The portability/granularity extreme. Enrich UC-35 (write granularity), UC
 
 ```task
 id: SHARD-WP-0003-T4
-status: todo
+status: done
 priority: medium
 state_hub_task_id: "5748d246-66a8-4cf2-9b22-427cc4507b63"
 ```