research: Logseq deep dive (block-graph on plain Markdown files, in-file block IDs, derived Datalog index); UC-62/63

Occupies the point the other modern tools leave empty: block-graph
semantics (UUID-addressable, embeddable, queryable blocks) stored as plain
Markdown/Org files on disk, with a DataScript graph derived from the files
(files canonical, index derived). The bridge between Roam (block-DB) and
Obsidian (file-over-app). Headline finding: Logseq resolves the
addressing-spectrum tension — block-level addressing that is also
git-diffable in-file text (id:: property) — and proves a file-backed shard
can serve rich Datalog queries via a derived index. Also: file->SQLite
"DB graph" migration is a live UC-43 (substrate swap under stable
identity); whiteboards = non-Markdown content; dual-attachable (file-store
direct with a Logseq format profile, or in-app plugin). Added UC-62
(attach block-graph-on-plain-files shard), UC-63 (serve structured
queries over a file shard via a derived index shard-wiki builds — converse
of UC-52); enriched UC-32/34/43/50/51/52/55. Catalog now 63 UCs.
Architecture for SHARD-WP-0002 T11/T14/T16: Logseq format profile,
derived-query-index capability, substrate-migration tolerance, in-file
block addressing as the T16 span-address target.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-14 15:27:37 +02:00
parent e00c160a43
commit 6ccf349209
5 changed files with 396 additions and 12 deletions

View File

@@ -8,7 +8,8 @@ Promoted from `research/260608-c2-wiki-origins/`,
`research/260613-twiki-deep-dive/`, `research/260613-foswiki-deep-dive/`,
`research/260614-xanadu-deep-dive/`, `research/260614-zigzag-deep-dive/`,
`research/260614-roam-deep-dive/`, `research/260614-obsidian-deep-dive/`,
`research/260614-notion-deep-dive/`, and `research/260614-joplin-deep-dive/`.
`research/260614-notion-deep-dive/`, `research/260614-joplin-deep-dive/`, and
`research/260614-logseq-deep-dive/`.
See InfoTechPrimers on coulomb.social for use-case catalog conventions.
## Conventions
@@ -201,7 +202,8 @@ UC-44 for the whole-page composition-manifest form. **Roam ships this**: block e
transclude by `:block/uid` live (not copied), proving transclusion is a cheap data-layer
capability over an addressable store (`research/260614-roam-deep-dive/findings.md` §3,
§7) — argues for transclusion in core over the union, surfaced by UI. Depends on span
addressing (UC-51).
addressing (UC-51). **Logseq** does the same on *files*: `{{embed ((uuid))}}` transcludes
by an in-file block ID (`research/260614-logseq-deep-dive/findings.md` §2).
**Priority:** Later
### UC-33 — Git-branch an information space
@@ -238,7 +240,10 @@ sidecar metadata, and a candidate for query delegation (UC-52). Obsidian shows t
and are queried by the Dataview plugin (`research/260614-obsidian-deep-dive/findings.md`
§3) — structure as portable text, not DB state. **Notion** is the apex of the DB-locked
end: typed database properties with relations/rollups/formulas, lossy to Markdown —
see UC-58 (`research/260614-notion-deep-dive/findings.md` §2).
see UC-58 (`research/260614-notion-deep-dive/findings.md` §2). **Logseq** sits between:
`key:: value` block/page properties live in the Markdown text (git-diffable) yet are
queried via a derived Datalog graph (`research/260614-logseq-deep-dive/findings.md` §2§3)
— in-text structure at block granularity.
**Priority:** Later
### UC-35 — Attach a shard with coarse write granularity
@@ -406,7 +411,10 @@ versioned `Foswiki::Store` interface (`research/260613-foswiki-deep-dive/finding
§2). shard-wiki orchestration robustness: a shard's identity/attachment is **not** its
storage format. Open: detect the swap and preserve history across it (findings §9 Q3).
Relates to UC-40 (attach path) and UC-41 (history import) but is about *change under a
live attachment*, not initial attach.
live attachment*, not initial attach. Live modern instance: **Logseq** is migrating its
substrate from Markdown-files to a SQLite "DB graph" under a stable graph identity
(`research/260614-logseq-deep-dive/findings.md` §5) — bind to capabilities, not to "it's
files".
**Priority:** Later
### UC-44 — Compose a page by reference (xanadoc / EDL manifest)
@@ -508,7 +516,9 @@ else degrade to read/projection/overlay-target (findings §11 Q2). **Notion** is
external-API variant of the same block-DB family: a server Postgres store with a real
**external** REST API, so write-through needs **no in-engine hosting** — but is bounded
by rate limits and eventual consistency (UC-57,
`research/260614-notion-deep-dive/findings.md` §4).
`research/260614-notion-deep-dive/findings.md` §4). **Logseq** is the *file-store* variant
of the block-tool family — block-graph semantics over plain Markdown files, not a DB — see
UC-62 (`research/260614-logseq-deep-dive/findings.md` §1§2).
**Priority:** Later
### UC-51 — Adopt a shard's native span IDs as portable span addresses
@@ -531,7 +541,10 @@ UID. Notion is a second store-minted case: per-block **UUID v4**, exposed via th
API (`research/260614-notion-deep-dive/findings.md` §1). **Joplin** marks the spectrum's
middle: a store-minted **page-level** 32-char ID with `:/<id>` links that survive
rename/move (`research/260614-joplin-deep-dive/findings.md` §1) — coarser than a block
UUID, more stable than a path.
UUID, more stable than a path. **Logseq** is the **sweet spot**: a block-level `id:: uuid`
stored **in the Markdown text** — fine-grained *and* git-diffable/portable at once,
resolving the Roam(DB-minted) vs Obsidian(page-level) tension
(`research/260614-logseq-deep-dive/findings.md` §2).
**Priority:** Later
### UC-52 — Delegate derived views to a shard's native query engine
@@ -549,7 +562,10 @@ nuance: query can be an *ecosystem plugin* (Dataview), **not core** — so "nati
is adapter/plugin-provided and must not be assumed
(`research/260614-obsidian-deep-dive/findings.md` §7). **Notion** ships a native
database **query API** (filters/sorts) — a strong delegation target
(`research/260614-notion-deep-dive/findings.md` §2, §4).
(`research/260614-notion-deep-dive/findings.md` §2, §4). **Logseq** shows the third path:
Datalog over a **derived** index built from plain files — neither native-DB nor plugin;
when no engine exists, shard-wiki can build the index itself (UC-63,
`research/260614-logseq-deep-dive/findings.md` §3).
**Priority:** Later
### UC-53 — Attach a local-first Markdown vault with a live concurrent native editor
@@ -598,7 +614,8 @@ Pushes the **wiki page model**: page vs. typed asset vs. opaque blob — a page-
decision, not just adapter config (findings §10). JSON Canvas is an open format worth
first-class support. **Joplin** resources (attachments, each with an ID, linked `:/<id>`)
are the same demand in a sync-mirror shard (`research/260614-joplin-deep-dive/findings.md`
§5).
§5). **Logseq whiteboards** (tldraw JSON) are the same in a file-store block shard
(`research/260614-logseq-deep-dive/findings.md` §6).
**Priority:** Later
### UC-56 — Publish a curated projection to an external read-only target
@@ -704,6 +721,40 @@ undecryptable content. Open: does shard-wiki ever hold keys, or only treat such
opaque backups (findings §9 Q1)? Ties [[shard-wiki-auth-in-core-decision]].
**Priority:** Later
### UC-62 — Attach a block-graph-on-plain-files shard
**Actor:** Maintainer
**Goal:** Attach a Logseq-style local outliner — block-graph semantics stored as plain
Markdown/Org files — preserving its in-file block IDs, block/page properties, and outline
tree, with a derivable query index.
**Source:** logseq, intent
**Notes:** Logseq is **file-over-app *and* block-graph**: blocks carry an in-file
`id:: <uuid>` property, `((uuid))` refs, and `key:: value` properties, with a DataScript
graph **derived** from the files (`research/260614-logseq-deep-dive/findings.md` §1§2).
The addressing **sweet spot** — block-level *and* git-diffable (UC-51) — resolving the
Roam(DB-minted)/Obsidian(page-level) tension. Attach **file-store direct with a Logseq
format profile** (parse `id::`/`((uuid))`/`key::`/outline) or via the in-app plugin
(`logseq.Editor`/`logseq.DB`). Distinct from UC-50 (Roam: block-DB, not files) and UC-53
(Obsidian: page-level, path-addressed). Watch the file→SQLite substrate migration
(UC-43).
**Priority:** Later
### UC-63 — Serve structured queries over a file-backed shard via a derived index
**Actor:** Reader or orchestrator
**Goal:** Run structured queries (tasks, tagged blocks, typed properties) over a
file-backed shard that exposes no native query engine, using an index the orchestrator or
adapter builds and rebuilds from the files.
**Source:** logseq, intent
**Notes:** Logseq proves a **file-backed** store supports rich **Datalog** queries via a
**derived DataScript index** (files canonical, graph derived —
`research/260614-logseq-deep-dive/findings.md` §1, §3). The converse of UC-52 (delegate to
a *native* engine): here shard-wiki **builds** the index over the projection when none
exists. Operationalizes the ZigZag dimensions / Roam-Notion query model on a file shard.
Open: built by adapter (per-shard) or core (over the union); persisted vs rebuilt
(findings §10 Q2). Belongs to the T16 navigation layer + T11 capabilities.
**Priority:** Later
---
## B. Knowledge work and collaboration
@@ -953,6 +1004,8 @@ CamelCase and `[[free links]]`. Markdown-first link semantics TBD.
| UC-59 | | | | ◊ | ✓ |
| UC-60 | | | | ‖ | ✓ |
| UC-61 | | | | ‖ | ✓ |
| UC-62 | | | | ※ | ✓ |
| UC-63 | | | | ※ | ✓ |
| UC-08 | ✓ | | |
| UC-09 | ✓ | | |
| UC-10 | ✓ | | |
@@ -1235,6 +1288,35 @@ never re-syncs, and treats the DB-local store and encrypted content as opaque
`SHARD-WP-0002` (T11/T14): interchange-format attach, content-opacity field, local-REST
sub-mode, format-aware file-store profiles.
### logseq mapping
(※ UC-62UC-63 are placed in the **wikiengines** matrix column as the nearest existing
source — Logseq is a shipped tool — but their true lineage is the **Logseq deep dive**,
`research/260614-logseq-deep-dive/findings.md`.)
| Logseq mechanism (findings §) | Catalog UC |
|-------------------------------|------------|
| Block-graph semantics on plain MD/Org files; in-file `id::`/`((uuid))`/`key::` (§1§2) | UC-62 (new) |
| Datalog over a **derived** DataScript index built from files (§1, §3) | UC-63 (new) |
| In-file block `id::` = block-level **and** git-diffable addressing (the sweet spot) (§2) | UC-51 (enriched) |
| File→SQLite "DB graph" substrate migration under stable identity (§5) | UC-43 (enriched) |
| Block-graph that is **files, not a DB** — file-store variant of the block-tool family (§1) | UC-50 (enriched) |
| Datalog over a derived index (third path: not native-DB, not plugin) (§3) | UC-52 (enriched) |
| `key:: value` block/page properties in-text (§2) | UC-34 (enriched) |
| Whiteboards (tldraw JSON) = non-Markdown content (§6) | UC-55 (enriched) |
| Block embeds `{{embed ((uuid))}}` = in-file transclusion (§2) | links UC-32 |
Note: Logseq is the **block-graph-on-plain-files** point the other modern tools leave
empty — the bridge between Roam (block-DB) and Obsidian (file-over-app) — and it
**resolves the addressing-spectrum tension**: block-level addressing that is also
git-diffable in-file text (`id::`). It confirms a **file-backed shard can serve rich
Datalog queries via a derived index** (files canonical, graph derived). **Boundary
recorded:** one block-graph-on-files candidate shard (the addressing sweet spot), best
attached file-store-direct with a Logseq **format profile**; not the federation layer;
substrate migrating file→SQLite (UC-43). Architecture logged for `SHARD-WP-0002`
(T11/T14/T16): Logseq format profile, derived-query-index capability, substrate-migration
tolerance, in-file block addressing as the concrete T16 span-address target.
---
## Open questions
@@ -1268,4 +1350,9 @@ sub-mode, format-aware file-store profiles.
14. For an **encrypted shard** (UC-61), what is visible without keys (IDs/structure vs
nothing), and does shard-wiki ever hold keys or only treat it as an opaque backup?
15. Is writing to a tool's **sync mirror** (UC-60) ever safe, or are interchange-format
shards read/projection/overlay-only by policy? (Joplin dive §9.)
shards read/projection/overlay-only by policy? (Joplin dive §9.)
16. Is the **derived query index** (UC-63) built by the adapter (per-shard) or the core
orchestrator (over the union), and is it persisted or rebuilt? (Logseq dive §10.)
17. When a tool offers **both file and DB substrates** (Logseq), does shard-wiki prefer
the git-diffable file graph or the richer DB graph, and can one binding follow the
migration? (UC-43, UC-62.)