diff --git a/.claude/ralph-loop.local.md b/.claude/ralph-loop.local.md index 12fb5c6..1a414bf 100644 --- a/.claude/ralph-loop.local.md +++ b/.claude/ralph-loop.local.md @@ -1,6 +1,6 @@ --- active: true -iteration: 1 +iteration: 2 session_id: b24b53ca-a25b-4a18-8331-925c4100d39f max_iterations: 20 completion_promise: "HEUREKA" diff --git a/SCOPE.md b/SCOPE.md index 6594480..be3b7a0 100644 --- a/SCOPE.md +++ b/SCOPE.md @@ -19,9 +19,9 @@ Learnings update both SCOPE and INTENT where necessary. |-------|-------| | Code | Python package scaffold (`src/shard_wiki/`, smoke tests only) | | Intent | `INTENT.md` established; authorization-in-core amendments drafted | -| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase deep dives & shard-spectrum synthesis (`research/260614-*/`) | +| Research | yawex prior art; c2 origins; federation concepts; wikiengines overview (`research/260608-*/`); XWiki/TWiki/Foswiki deep dives (`research/260613-*/`); Xanadu + ZigZag + Roam + Obsidian + Notion + Joplin + Logseq + local-first workspaces (Anytype/AFFiNE/AppFlowy) + Trilium + Wiki.js + Federated Wiki + Wikibase + git-forge wikis deep dives & shard-spectrum synthesis (`research/260614-*/`) | | Demand | NetKingdom integration asks captured, not yet negotiated | -| Spec | Architecture blueprint drafted; UseCaseCatalog 75 UCs from research; PRD/TSD scaffolds | +| Spec | Architecture blueprint drafted; UseCaseCatalog 77 UCs from research; PRD/TSD scaffolds | | Work | `SHARD-WP-0001` active (6 tasks); `SHARD-WP-0002` active (16 tasks: T1–T10 federation + T11–T16 adapter contract); `SHARD-WP-0003` active (9 engine dives); `SHARD-WP-0004` active (8 computational-knowledge dives) | ## In Scope (today) diff --git a/research/260614-forge-wikis-deep-dive/README.md b/research/260614-forge-wikis-deep-dive/README.md new file mode 100644 index 0000000..9e555a4 --- /dev/null +++ b/research/260614-forge-wikis-deep-dive/README.md @@ -0,0 +1,17 @@ +# 260614 — git-forge wikis deep dive (Gitea · GitLab · GitHub) + +Deep dive on the **git-forge-hosted Markdown wikis** as one grouped memo: each is a +**dedicated git repository of Markdown** exposed through a forge, attachable by **cloning +the wiki repo directly** *or* (where offered) through the **forge's wiki API**. INTENT names +**Gitea wikis** explicitly as a shard participant — this dive confirms the **git-native +file-store** as a first-class, and the *simplest*, backend. + +- `findings.md` — the three forges compared, the `.wiki.git` model, API matrix, capability + profile, INTENT mapping, UC seeds (UC-76/77), architecture notes for SHARD-WP-0002, open + questions (resolves the UC-68 source-of-truth question for this case), sources, + traceability. + +Catalog yield: UC-76 (attach a forge wiki by cloning its dedicated `.wiki.git` — git **is** +the native store and the coordination journal), UC-77 (attach/write via the forge wiki API +where git-clone is unavailable/undesired — capability varies by forge). Enriched +UC-40/02/68/38. Feeds SHARD-WP-0002 T14 (attachment binding). diff --git a/research/260614-forge-wikis-deep-dive/findings.md b/research/260614-forge-wikis-deep-dive/findings.md new file mode 100644 index 0000000..940f159 --- /dev/null +++ b/research/260614-forge-wikis-deep-dive/findings.md @@ -0,0 +1,173 @@ +# git-forge wikis (Gitea · GitLab · GitHub) — deep dive (findings) + +**Date:** 2026-06-14 · **Source:** SHARD-WP-0003 T5 · **Subject:** the Markdown wikis +hosted by the three major git forges — **Gitea**, **GitLab**, **GitHub** — treated as one +family because they share one architecture: *a wiki is a separate git repo of Markdown.* + +## Why this dive + +INTENT names **Gitea wikis** as a shard participant, and the whole project is "a **Git-based +Markdown** wiki orchestrator." The forge wikis are therefore the **least exotic, highest- +fit** backend in the entire study: the page store is *literally a git repository of Markdown +files*. After fourteen dives into DBs, CRDTs, graphs and SaaS, this one confirms the +**home case** — and sharpens it by contrasting *git-IS-the-store* (forge wikis) against +*git-is-a-mirror* (Wiki.js, UC-68). + +## 1. The shared architecture — a wiki is a `.wiki.git` repo + +All three forges implement a project/repo wiki as a **second, dedicated git repository** +alongside the code repo, addressable as `.wiki.git`: + +- `git@host:owner/project.wiki.git` (GitLab), `…/owner/repo.wiki.git` (Gitea/GitHub). +- **Pages are Markdown files** (`Home.md`, `Some-Page.md`), one file per page; the **page + title ↔ filename** (spaces ↔ hyphens by convention). Other markups are accepted + (AsciiDoc, Textile, reStructuredText, Org) — GitHub/Gitea via **Gollum** (the Ruby + git-backed wiki library), GitLab via its own renderer. +- **History is git history** — every page edit (web or pushed) is a git commit with + author/timestamp/message. *The wiki's revision history is a real git log.* +- **Special pages** by convention: `_Sidebar`, `_Footer`, `_Header` (GitHub/Gitea), + `_sidebar` (GitLab) — engine-rendered chrome stored as ordinary files. +- **Subdirectories / nested pages**: GitLab and Gitea support directory structure; GitHub + wikis are historically flat (Gollum supports paths but the GitHub UI is shallow). + +The decisive property: **you can `git clone` the wiki repo, edit files, commit, and push**, +and the forge UI reflects it — *and vice versa*. Git is **a** (often **the**) first-class +write path. This is exactly shard-wiki's native medium with no impedance. + +## 2. Where they differ — the API matrix + +| | git clone/push of `.wiki.git` | wiki content **API** | nested dirs | markups | +|--|--|--|--|--| +| **Gitea** | ✅ yes | ✅ **REST wiki endpoints** (list/get/create/edit/delete pages) | ✅ | Markdown (+Gollum-style) | +| **GitLab** | ✅ yes | ✅ **REST Wikis API** (project & group wikis) | ✅ | Markdown/AsciiDoc/RDoc/Org | +| **GitHub** | ✅ yes | ❌ **no wiki REST API** — wiki is **git-only** (Gollum) | ⚠️ flat UI | Markdown + Gollum markups | + +The key asymmetry: **GitHub exposes wiki content *only* through git** (the REST/GraphQL API +covers issues/PRs/code but **not** wiki pages); **GitLab and Gitea offer both** a wiki API +*and* git access. So the **git-clone path is the universal one** (works for all three); the +API path is an *optional, capability-varying* alternative. + +## 3. git-IS-the-store vs git-is-a-mirror (the UC-68 contrast) + +Wiki.js (UC-68) keeps a **DB as canonical** and *maintains a git mirror* — so writing by +commit risks **racing the engine's DB↔git sync** (catalog open-Q22). Forge wikis are the +opposite: **the git repo IS the canonical store**; there is *no* separate DB of record for +wiki content. Therefore: + +- **The source-of-truth question (Q22) is resolved for this case:** the `.wiki.git` repo is + authoritative. shard-wiki can **write by commit/push directly** with no engine to race — + the forge merely *renders* what git holds. +- The forge **API** (GitLab/Gitea), where present, is a *convenience over the same git + repo*, not a competing store — so API-write and git-write converge on one history. + +This makes forge wikis the **cleanest possible write-through file-store shard**: clone = +projection/mirror, commit = overlay-applied/write, git log = the coordination journal *as +is*. + +## 4. Capability profile + +| Dimension (synthesis spectrum) | Gitea / GitLab / GitHub wiki | +|--------------------------------|------------------------------| +| Attachment mode | **file-store (native: git clone)** + optional **external-API** (GitLab/Gitea wiki REST) | +| Addressing granularity | **page = file**; sub-page = path (GitLab/Gitea) | +| Content identity | path/filename within the wiki repo (title-derived) | +| Identity vs placement | placement-bound (path = identity), like a plain git repo | +| Structure | flat or directory tree of Markdown files; `_Sidebar`/`_Footer` chrome | +| History | **native git history** (real commits, authors, messages) | +| Merge model | **git** (3-way merge, branches) — though wiki repos are usually single-branch | +| Native query | none (it's files); forge full-text search over the wiki | +| Translation | **Markdown-native** (+ AsciiDoc/Org via renderer) — minimal/no translation needed | +| Attachment/write granularity | **file (page)** per commit | +| Operational envelope | ordinary git + forge; clone is cheap; API rate limits apply to API path | +| Access grant | **forge repo permissions** (delegated auth; per-repo/role ACL) | +| Content opacity | transparent Markdown in git | +| Provenance | git author/committer/timestamp per commit — native | + +## 5. INTENT mapping + +### Reinforcements (this is the home case) + +- **Git-based Markdown orchestrator** (INTENT core): forge wikis *are* git repos of + Markdown. The **wiki page model** (Markdown-first, path-addressed, git-versioned) maps 1:1 + — minimal adapter, maximal fit. +- **Coordination journal = git** (INTENT): the wiki repo's **git log is already the + coordination journal** — no synthesis needed; adopt it directly. +- **Overlay before mutation**: overlays are **branches/commits** on the cloned wiki repo; + applying = push (or open an MR/PR where the forge supports wiki MRs — GitLab does not for + wikis, so push-to-branch + manual is the path). +- **Graceful degradation**: even GitHub (no wiki API) is fully usable via git-clone — the + *universal* path means a limited forge is still a first-class read/write shard. +- **No silent remote mutation**: writes are explicit git pushes (or explicit API calls) + under the user's forge credentials and repo permissions. + +### Divergences (boundaries / notes — minor) + +- **Capability varies by forge**: GitHub = git-only (no content API); GitLab/Gitea = git + + API. The adapter must **model the API as an optional capability**, defaulting to the + universal git path (T11/T14). Not a bug — exactly the capability-awareness INTENT mandates. +- **Wiki repos rarely use branches/MRs for review**: forge wikis usually edit a single + branch directly; the rich PR-review flow is on the *code* repo, not the wiki. So + "overlay → review → merge" needs shard-wiki to provide the review layer, not the forge. +- **Identity = path** (like any git repo) — cross-shard identity (T16) is layered above, as + for plain git/`wiki/` subdir shards. + +### What to keep + +1. **git-clone as the universal, canonical file-store attach** for forge wikis — Markdown + + git history directly as page model + coordination journal (UC-76). The reference + easy-case backend. +2. **Forge wiki API as an optional capability** (GitLab/Gitea), with **git-only fallback** + (GitHub) — capability-aware binding (UC-77). +3. **git-IS-store ⇒ write-by-commit is safe** (no engine race) — record this as the + resolution of the Wiki.js mirror dilemma (Q22) for forge wikis. + +## 6. UC seeds + +| # | Seed | Disposition | +|---|------|-------------| +| UC-76 | Attach a **git-forge wiki** by **cloning its dedicated `.wiki.git`** — git is the native store; Markdown files = pages, git log = coordination journal; commit/push = write (no engine to race) | **new** | +| UC-77 | Attach/write a forge wiki via the **forge's wiki API** (GitLab/Gitea REST) where git-clone is unavailable or API-write is preferred; **git-only fallback** for GitHub — capability varies by forge | **new** | +| — | git-native file-store as the *canonical store* (not mirror) | enrich **UC-40** | +| — | dual-path attach (git clone vs forge API) | enrich **UC-02** | +| — | git-IS-store vs engine-maintained mirror (resolves Q22) | enrich **UC-68** | +| — | forge as an API host for the wiki resource | enrich **UC-38** | + +## 7. Architecture notes for SHARD-WP-0002 + +- **T14 (adapter binding / attach path):** forge wikis are the canonical **file-store + attach** — bind to the `.wiki.git` clone as the universal path; model the **wiki API as an + optional, forge-specific capability** (present: GitLab, Gitea; absent: GitHub). One shard, + two possible bindings converging on the same git history. +- **T11 (capability model):** "has-content-API" is a **per-forge capability flag**; git + clone/push is the baseline every forge satisfies. Minimal adapter profile — near the + Oddmuse-simple end but Markdown-native and git-versioned. +- **Coordination journal:** adopt the wiki repo's **git log directly** — the one backend + where INTENT's git-backed journal needs *zero* synthesis. +- **Resolves Q22 (UC-68):** because git **is** the store (not a mirror), **write-by-commit + is safe** — no engine DB↔git sync to race. Record the distinction *engine-mirror* + (Wiki.js: DB canonical, careful) vs *git-canonical* (forge wikis: commit freely). + +## 8. Open questions + +1. For overlay → **review** → apply, does shard-wiki supply the review layer over a forge + wiki (which lacks wiki-MRs), e.g. via a branch + its own diff/approve, or push directly? +2. When a forge offers **both** git and a wiki API (GitLab/Gitea), which does the adapter + prefer by default — git (universal, full history) with API as a fallback for hosts where + clone is disabled? (cf. UC-43 backend-swap under stable binding.) +3. Should the **code-repo `wiki/` subdir** shard and the **forge wiki repo** shard share one + adapter (both git+Markdown) with a "which repo / which path" parameter, or stay distinct? + +## 9. Sources + +- GitLab Docs — *Wiki* (separate git repo; web/git/API; `.wiki.git`) — docs.gitlab.com +- Gitea — wiki via git clone + repository **wiki API**; forum/issue threads on + `.wiki.git` clone (go-gitea/gitea #1426, #15420) — gitea.com / github.com/go-gitea +- GitHub — wiki = Gollum git repo (`.wiki.git`), no wiki REST API — docs.github.com +- Gollum (git-based wiki library) — github.com/gollum/gollum +- prior: `research/260614-wikijs-deep-dive/` (engine-maintained mirror contrast, UC-68) + +## 10. Traceability + +New UCs **UC-76–UC-77** carry the marker **⎇** in the wikiengines column of +`spec/UseCaseCatalog.md`. Enriched: UC-40, UC-02, UC-68, UC-38. Architecture cross-refs: +SHARD-WP-0002 T14, T11; coordination-journal-from-git; resolves catalog open-Q22. diff --git a/research/README.md b/research/README.md index 1311a81..c88fe4c 100644 --- a/research/README.md +++ b/research/README.md @@ -29,4 +29,5 @@ when multiple files or sources are involved. Findings here inform `spec/` and | 2026-06-14 | `260614-trilium-deep-dive/` | Trilium/TriliumNext — note cloning (DAG hierarchy), attribute inheritance/templates, HTML-native, scripting+ETAPI; UC-66/67 | | 2026-06-14 | `260614-wikijs-deep-dive/` | Wiki.js — storage-module engine (DB↔Git Markdown), GraphQL API, pluggable modules ≈ adapter-contract prior art; UC-68/69 | | 2026-06-14 | `260614-federated-wiki-deep-dive/` | Federated Wiki — fork-with-provenance, per-page semantic-action journal (story=replay), neighborhood/roster + chorus; prior art for our coordination journal / overlay / union pillars; UC-70/71/72 | -| 2026-06-14 | `260614-wikibase-deep-dive/` | Wikibase/Wikidata — typed entity-statement knowledge graph (claim+qualifiers+refs+rank), SPARQL/RDF + federated SERVICE, opaque stable IDs, statement-level provenance; structure & query far-end; UC-73/74/75 | \ No newline at end of file +| 2026-06-14 | `260614-wikibase-deep-dive/` | Wikibase/Wikidata — typed entity-statement knowledge graph (claim+qualifiers+refs+rank), SPARQL/RDF + federated SERVICE, opaque stable IDs, statement-level provenance; structure & query far-end; UC-73/74/75 | +| 2026-06-14 | `260614-forge-wikis-deep-dive/` | Gitea · GitLab · GitHub wikis — a wiki is a separate `.wiki.git` of Markdown; git-clone universal, wiki API capability-varying (GitHub git-only); git IS the store (resolves UC-68 race); the home case; UC-76/77 | \ No newline at end of file diff --git a/spec/UseCaseCatalog.md b/spec/UseCaseCatalog.md index 0fb1415..f96b483 100644 --- a/spec/UseCaseCatalog.md +++ b/spec/UseCaseCatalog.md @@ -13,7 +13,8 @@ Promoted from `research/260608-c2-wiki-origins/`, `research/260614-localfirst-workspaces-deep-dive/`, `research/260614-trilium-deep-dive/`, `research/260614-wikijs-deep-dive/`, and `research/260614-federated-wiki-deep-dive/`, and -`research/260614-wikibase-deep-dive/`. +`research/260614-wikibase-deep-dive/`, and +`research/260614-forge-wikis-deep-dive/`. See InfoTechPrimers on coulomb.social for use-case catalog conventions. ## Conventions @@ -899,7 +900,12 @@ interchange mirror (UC-60) — it is **plain Markdown** — and richer than a re store (UC-40): the bidirectional ingest makes **git commit a write path** (overlay/patch as a commit, no API). Caution: the **engine owns the DB↔git sync** — don't double-sync; coordinate source-of-truth (configurable). Gives **git history natively** (UC-36, adopt -via mirror). Open: mirror-vs-DB source of truth; write-by-commit races (findings §9 Q1). +via mirror). Open: mirror-vs-DB source of truth; write-by-commit races (findings §9 Q1). +**Contrast (resolves the race for forge wikis):** a **git-forge wiki** (UC-76) is the +opposite — **git IS the canonical store, not a mirror**, so there is no engine DB↔git sync +to race and **write-by-commit is safe** +(`research/260614-forge-wikis-deep-dive/findings.md` §3). The dilemma here is specific to +*engine-maintained mirrors* (DB canonical), not to git-canonical stores. **Priority:** Later ### UC-69 — Attach via a typed, introspectable API (schema discovery + selective projection) @@ -1015,6 +1021,36 @@ sub-page/per-assertion provenance even if MVP records per page. Enriches UC-24; SHARD-WP-0002 T12 + provenance model. **Priority:** Later +### UC-76 — Attach a git-forge wiki by cloning its dedicated wiki repo + +**Actor:** Orchestrator / adapter +**Goal:** Attach a **Gitea / GitLab / GitHub wiki** by **cloning its dedicated +`.wiki.git`** — Markdown files are pages, the **git log is the coordination journal**, +and **commit/push is the write path** with no engine to race. +**Source:** wikiengines, intent +**Notes:** A forge wiki is a *separate git repo of Markdown* +(`research/260614-forge-wikis-deep-dive/findings.md` §1, §3) — the **home case**: page +model, history, and journal map 1:1 to shard-wiki's medium with minimal adapter. Unlike a +Wiki.js engine-maintained mirror (UC-68), **git IS the canonical store**, so write-by-commit +is safe (resolves catalog open-Q22). The **universal** attach path across all three forges. +INTENT names Gitea wikis explicitly. Feeds SHARD-WP-0002 T14 (file-store attach), T11. +**Priority:** MVP + +### UC-77 — Attach/write a forge wiki via the forge's wiki API (capability varies) + +**Actor:** Orchestrator / adapter +**Goal:** Attach or write a forge wiki via the **forge's wiki API** (GitLab Wikis API, +Gitea wiki endpoints) where git-clone is unavailable or API-write is preferred — with a +**git-only fallback** for forges lacking a wiki API (**GitHub**). +**Source:** wikiengines, intent +**Notes:** Capability **varies by forge**: GitLab and Gitea expose REST wiki endpoints +(list/get/create/edit/delete); **GitHub has no wiki content API** — git-only +(`research/260614-forge-wikis-deep-dive/findings.md` §2). The adapter models the API as an +**optional capability** over the universal git path; both converge on one git history. An +external-API host sub-mode (UC-38) beside the file-store attach (UC-76). Feeds +SHARD-WP-0002 T11 (capability flag), T14 (binding). +**Priority:** Later + --- ## B. Knowledge work and collaboration @@ -1287,6 +1323,8 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD. | UC-73 | | | | ⬡ | ✓ | | UC-74 | | | | ⬡ | ✓ | | UC-75 | | | | ⬡ | ✓ | +| UC-76 | | | | ⎇ | ✓ | +| UC-77 | | | | ⎇ | ✓ | | UC-08 | ✓ | | | | UC-09 | ✓ | | | | UC-10 | ✓ | | | @@ -1759,6 +1797,36 @@ graph-query tier (UC-74), not the union default. Architecture logged for `SHARD- (T12/T16): typed-graph page payload, opaque identity, native-query tiering (SPARQL + federated SERVICE), sub-page provenance model. +### forge-wikis mapping + +(⎇ UC-76–UC-77 are placed in the **wikiengines** matrix column; lineage = the **git-forge +wikis deep dive**, `research/260614-forge-wikis-deep-dive/findings.md`.) + +| Forge-wiki mechanism (findings §) | Catalog UC | +|-----------------------------------|------------| +| Wiki = separate `.wiki.git` of Markdown; git log = journal; clone/push = write (§1) | UC-76 (new) | +| Wiki content API varies: GitLab/Gitea yes, GitHub git-only (§2) | UC-77 (new) | +| git **is** the canonical store (not a mirror) → write-by-commit safe (§3) | UC-40 (enriched); resolves UC-68 Q22 | +| Dual-path attach: git clone vs forge wiki API (§2) | UC-02 (enriched) | +| git-IS-store vs Wiki.js engine-maintained git mirror (§3) | UC-68 (enriched) | +| Forge as API host for the wiki resource (GitLab/Gitea) (§2) | UC-38 (enriched) | +| Page = file; sub-page = path; identity = path (§1, §4) | links UC-25 | +| Overlay = branch/commit; forge lacks wiki-MRs → shard-wiki supplies review (§5) | links UC-04 | + +Note: The git-forge wikis (Gitea, GitLab, GitHub) are the **home case** — a wiki that is +*literally a separate git repository of Markdown* (`.wiki.git`), so the **page model, +history, and coordination journal map 1:1** to shard-wiki's medium with a near-zero adapter. +The **git-clone path is universal** (all three); a **wiki content API** is an *optional, +capability-varying* alternative (GitLab/Gitea have one, **GitHub is git-only**) — exactly +the capability-awareness INTENT mandates (UC-77). The decisive contrast with Wiki.js +(UC-68): there the DB is canonical and git is a *mirror* (write carefully, open-Q22); here +**git IS the store**, so **write-by-commit is safe with no engine to race** — which +*resolves Q22 for this case*. **Boundary recorded:** identity = path (cross-shard identity +layered above, like a `wiki/` subdir shard); forge wikis lack wiki-MRs, so the +overlay→review→apply flow is shard-wiki's to provide. Architecture logged for +`SHARD-WP-0002` (T14/T11): `.wiki.git` clone as the canonical file-store attach, wiki API as +an optional per-forge capability, git log adopted directly as the journal. + --- ## Open questions @@ -1820,5 +1888,11 @@ federated SERVICE), sub-page provenance model. query** (UC-74) a union-level capability or a pass-through to graph-capable shards? At what granularity is **provenance** recorded — per page (MVP) or per statement (UC-75)? (Wikibase dive §8.) +25. For a **git-forge wiki** (UC-76/77), when a forge offers **both** git-clone and a wiki + API (GitLab/Gitea), which does the adapter prefer by default — git (universal, full + history) with API as fallback? And does the **code-repo `wiki/` subdir** shard share one + git+Markdown adapter with the **forge wiki repo** shard (parameterized by repo/path), or + stay distinct? Since forge wikis lack wiki-MRs, does shard-wiki supply the + overlay→review→apply layer? (Forge-wikis dive §8.) 23. How does shard-wiki **honor/surface a shard's path-based access rules** (UC-06) in a projection without re-implementing its ACL engine? (Wiki.js dive §9.) \ No newline at end of file diff --git a/workplans/SHARD-WP-0003-engine-dives-batch.md b/workplans/SHARD-WP-0003-engine-dives-batch.md index d65ad8a..ccd566b 100644 --- a/workplans/SHARD-WP-0003-engine-dives-batch.md +++ b/workplans/SHARD-WP-0003-engine-dives-batch.md @@ -116,7 +116,7 @@ T4/T6. ```task id: SHARD-WP-0003-T5 -status: todo +status: done priority: high state_hub_task_id: "9719ce23-5edd-4110-a428-3cf4df753cdc" ```