shard-wiki/research/260614-logseq-deep-dive/findings.md

# Findings — Logseq: block-graph semantics on plain Markdown files

Date: 2026-06-14
Source kind: **modern shipped product** — an open-source, local-first outliner; a
*candidate shard* occupying the point none of the prior tools do: **block-graph + file-
over-app**
Lens: shard-wiki — the convergence of block-level addressing with git-diffable files, a
derived query index over canonical files, and a live file→DB storage migration

> Why Logseq matters distinctly. The modern-tool dives staked out the corners: **Roam**
> = block-graph but client-DB / in-app-API / store-minted UUID; **Obsidian** =
> file-over-app but page-level / path-addressed (block `^id` opt-in); **Notion** =
> block-graph but closed hosted DB / external-API. **Logseq sits in the middle they
> leave empty: block-graph *semantics* (UUID-addressable, embeddable, queryable blocks)
> stored as *plain Markdown/Org files on disk you own*, with a DataScript index
> **derived** from those files.** It is the system that proves you do not have to choose
> between Roam's fine-grained addressing and Obsidian's file sovereignty — you can have
> block-level addressing *that is also git-diffable text*. That resolves a tension the
> synthesis flagged (addressing spectrum) and is the reason it earns a memo rather than
> a footnote.

Contrast set: Roam (block-DB, in-app), Obsidian (file, page-level), Notion (hosted DB),
Joplin (SQLite-local, files-on-sync). Logseq = **block-graph, files-canonical,
index-derived** — and now *also* migrating toward SQLite (§5), a live UC-43 case.

---

## 1. Core architecture — files canonical, DataScript derived

- **Storage:** plain-text **Markdown or Org-mode files on disk** — `pages/<Page>.md` and
  `journals/<date>.md` — fully usable even if the app disappears (local-first, open
  source, ClojureScript).
- **Index:** a **DataScript** in-memory graph database, **derived** from the files: an
  `mldoc` parser turns Markdown into an AST, extracts references/properties, and
  transforms it into **DataScript entities** with tree-position attributes. Files are the
  source of truth; the graph DB is a rebuildable projection. (In current versions a **DB
  Worker** manages **both DataScript and SQLite** connections — see §5.)
- This is the **files-canonical / index-derived** architecture (Obsidian's MetadataCache,
  Git's working tree) — but here the derived index is a *full Datalog-queryable graph*,
  not just a metadata cache. Logseq is the strongest evidence that **a file-backed shard
  can support rich structured queries via a derived index** (UC-52, UC-63).

---

## 2. The block-graph, in the file text

Everything is an **atomic block** (a bullet), individually referenceable, embeddable, and
queryable — Roam's model — but the addressing lives **in the Markdown**:

- **Block IDs are in-file properties.** When a block is referenced, Logseq writes an
  `id:: <uuid>` property line into the Markdown. So the block's stable address is
  **git-diffable text that survives a file copy**, not a DB-minted hidden key. This is the
  **sweet spot of the addressing spectrum**: block-level (like Roam) *and* in-file/
  portable (like Obsidian's `^id`), without choosing.
- **References:** `[[page]]`, **`((block-uuid))`**, `#tag` — all extracted into the graph;
  block embeds `{{embed ((uuid))}}` are transclusion at block granularity (UC-32).
- **Properties:** `key:: value` lines attach typed metadata to blocks (block properties)
  and pages (first-block/page properties) — **git-diffable structured data at block
  granularity** (UC-34), queried by Datalog.
- **Outline tree:** a page is a **nested tree of blocks** (indent = structure), with
  outliner operations — indent/outdent, move subtree, **zoom/narrow** into a block. The
  page is not flat prose; it is an addressable, reorderable block tree (UC-63).

---

## 3. Queries — Datalog over the derived graph

Logseq exposes **`{{query}}`** (simple) and **advanced Datalog queries** over the
DataScript graph (and via the plugin API, `logseq.DB.datascriptQuery`). Because the graph
is derived from files, **query-defined pages** (UC-54) and structured aggregation (tasks,
tagged blocks) run over a *file-backed* store. Key lesson: query capability here is
**neither native-DB (Notion) nor a third-party plugin (Obsidian Dataview)** — it is
**built into the tool over its own files**, demonstrating that *a derived query index is a
viable adapter/orchestrator capability for file shards* (UC-63).

---

## 4. Extension surface — plugin API + marketplace

Logseq has a JS **plugin API** (marketplace ~486 plugins, `logseq/marketplace`):

- `logseq.Editor` — block/page CRUD, insert/move/update/delete, get current block/page.
- `logseq.DB` — **`datascriptQuery`** (Datalog), plus DB change subscriptions.
- `logseq.App` — app-level ops/events; `logseq.UI`; `logseq.Git` (git ops on the graph);
  `logseq.Assets`; `logseq.FileStorage`.
- Slash commands, block/page context menus, `provideModel`/`provideStyle`, hooks/events.

So, like Obsidian, Logseq is **dual-attachable**: (a) **file-store direct** — read the
`pages/`+`journals/` Markdown with a **Logseq format profile** (parse `id::`, `((uuid))`,
`key::`, the outline tree); (b) **in-app plugin host** — `logseq.Editor` write +
`logseq.DB` query + change events for live write-through. A notable extra: a built-in
**`logseq.Git`** surface — the tool treats git as a first-class companion to the file
graph (validating the coordination journal).

---

## 5. The file→DB migration — a live UC-43

Logseq is **migrating its storage model from Markdown-files to a SQLite "DB graph"**
(the DB Worker already manages SQLite alongside DataScript; the plugin API has a distinct
"DB graph" mode with tags/classes/typed properties). This is a real-world instance of
**UC-43 (backend-swap under stable identity)**: the *same tool and graph identity* moving
from a file substrate to a DB substrate, trading git-diffability for richer typed
structure (toward the Notion/XWiki end). For shard-wiki it is both a caution (a shard's
substrate can change under it) and a confirmation that the **addressing/structure/history
spectra** are trajectories tools actually travel — an adapter keyed to a fixed substrate
will break.

---

## 6. Logseq as a shard — capability profile

| Capability | Logseq (MD-file graph) | Notes for the adapter contract |
|------------|------------------------|--------------------------------|
| Read | **yes** | file-store direct (pages/journals MD) or `logseq.Editor`/`logseq.DB` in-app |
| Write | **yes** | direct file write (format-aware) or plugin; block-level |
| Write granularity | **per-block** (in a per-file page) | finer than Obsidian (per-file), like Roam — but on files |
| Identity / addressing | **in-file block `id:: uuid` + `((uuid))`** | block-level **and** git-diffable — the addressing sweet spot (UC-51) |
| Structure | `key:: value` block/page properties; outline tree | git-diffable structured data at block granularity (UC-34) |
| History | none native; **`logseq.Git`** + files = git-native | git-friendly out of the box (adopt, not supplement) |
| Native query | **Datalog over derived DataScript** | derived index over files → delegate or rebuild (UC-52, UC-63) |
| Transclusion | **block embeds `{{embed ((uuid))}}`** | in-file, addressable (UC-32) |
| Backlinks | linked + unlinked references | derived (UC-05/18) |
| Content types | Markdown/Org + **whiteboards** (tldraw JSON) | non-Markdown spatial content (UC-55) |
| Substrate | **files now, SQLite "DB graph" emerging** | live backend-swap (UC-43) |
| Attach modes | file-store direct (format profile) · in-app plugin | dual, per-binding (UC-62) |

Verdict: **the most shard-wiki-friendly block tool** — block-graph power with file-over-
app sovereignty and git-diffable addressing/structure. Best attach: **file-store direct
with a Logseq format profile** (offline, git-native), with the plugin for live write-
through. Watch the **DB-graph migration** (UC-43).

---

## 7. Mapping to shard-wiki INTENT (compare, do not equate)

### 7.1 Reinforcements

- **Resolves the addressing tension.** Block-level addressing *can* be git-diffable
  in-file (`id::`), not forced to choose Roam-DB vs Obsidian-page (UC-51, synthesis §2).
- **Confirms files-canonical / index-derived at full power** — a Datalog graph derived
  from files, not just a metadata cache (UC-52, UC-63).
- **Structure as git-diffable text** (`key::`) reinforces "prefer in-text structure"
  (UC-34, synthesis through-line).
- **Outliner block tree** validates the page-as-addressable-block-tree demand (UC-50,
  UC-63) on a *file* backend.

### 7.2 Deliberate divergences (design bugs if conflated)

1. **One graph = one shard.** Local-first single graph; not the federation layer.
2. **The MD files carry Logseq-specific syntax** (`id::`, `((uuid))`, `key::`, outline
   semantics) — a **format profile**, not plain CommonMark. A naïve Markdown reader will
   mangle block IDs/properties (cf. UC-42; lighter than Notion's lossy case).
3. **The substrate is moving (file→SQLite).** Don't hard-code the file model; gate on the
   substrate capability and tolerate the swap (UC-43).
4. **Whiteboards are not Markdown** — typed/opaque assets, not flattened (UC-55).

### 7.3 What Logseq teaches that shard-wiki should keep

- **In-file block addressing is the target shape** for a portable span address where the
  backend cooperates — adopt `id::`-style schemes; they are git-diffable and survive copy.
- **A derived query index over files is a first-class capability** — shard-wiki can build
  one over a file shard (UC-63) when the backend exposes none, or delegate when it does.
- **Expect substrate migration** — bind to capabilities, not to "it's files."

---

## 8. Use-case seeds → catalog (promoted 2026-06-14)

Last existing UC is **UC-61**. New UCs **UC-62, UC-63** added; existing UCs enriched.

| Seed | Catalog action |
|------|----------------|
| **Attach a block-graph-on-plain-files shard** (Logseq-style): file-over-app Markdown carrying in-file block IDs (`id::`), block refs, and `key::` properties, with the outline tree and a derivable query index | **UC-62 (new)** |
| **Serve structured queries over a file-backed shard via a derived index** the orchestrator/adapter builds (Logseq DataScript-over-files) when the backend exposes none | **UC-63 (new)** |
| In-file block `id::` = block-level **and** git-diffable addressing (the spectrum sweet spot) | **enriches UC-51** |
| Live **file→SQLite substrate migration** under stable graph identity | **enriches UC-43** |
| Block-graph that is **files, not a DB** — the file-store variant of the block tool family | **enriches UC-50** |
| Datalog over a **derived** index built from files | **enriches UC-52** |
| `key:: value` block/page properties in-text | **enriches UC-34** |
| Whiteboards (tldraw JSON) = non-Markdown content | **enriches UC-55** |
| Block embeds `{{embed ((uuid))}}` = in-file transclusion | links UC-32 |

---

## 9. Architecture notes for SHARD-WP-0002 (no UC)

- **Format-aware file-store profiles** (already flagged by Joplin, UC-60) gain a strong
  case here: a Logseq profile parses `id::`/`((uuid))`/`key::`/outline from otherwise-
  plain Markdown. The contract should let a file-store adapter declare its **format
  profile** (plain MD / Obsidian / Logseq / Joplin-item / Foswiki-PlainFile). (T11/T14.)
- **Derived-query-index as an orchestrator capability** (UC-63): when a file shard has no
  native query engine, build a DataScript-like index over the projection; when it does
  (Roam/Notion/XWiki), delegate (UC-52). Both belong in T16's navigation layer + T11.
- **Substrate-migration tolerance** (UC-43, Logseq file→SQLite): T14 binding should treat
  substrate as a capability that can change under a live attachment, preserving identity.
- **In-file block addressing** is the concrete realization of T16's span-address thread —
  prefer `id::`-style git-diffable IDs over DB-minted where the backend allows.

---

## 10. Open questions (for spec / workplans)

1. When the same tool offers **file** and **DB** substrates (Logseq now), does shard-wiki
   prefer the file graph (git-diffable, UC-62) or the DB graph (richer typed structure),
   and can one binding follow the migration (UC-43)?
2. Is the **derived query index** (UC-63) built by the **adapter** (per-shard) or the
   **core orchestrator** (over the union), and is it persisted or rebuilt?
3. How much **Logseq outline semantics** (zoom, subtree move) must shard-wiki preserve vs.
   present as a flat page (UC-63 vs. Markdown-first page model)?
4. Does the **Logseq format profile** round-trip overlays (write `id::`/`key::` back) or
   only read them? (cf. UC-42 round-trip question.)

---

## 11. Sources

| Source | Used for |
|--------|----------|
| DeepWiki — logseq/logseq (https://deepwiki.com/logseq/logseq) | DataScript+Datalog graph, mldoc AST parse, block entities/tree, DB Worker managing DataScript **and** SQLite |
| logseq/docs (https://deepwiki.com/logseq/docs) | Plain-text MD/Org files; pages/journals; references ([[ ]], (( )), #tag); properties |
| Logseq Plugin API docs (https://plugins-doc.logseq.com/) | `logseq.App/Editor/DB/Git/UI/Assets/FileStorage`; `DB.datascriptQuery` |
| logseq/marketplace (https://github.com/logseq/marketplace) | Plugin distribution; ~486 plugins |
| kerim/logseq-db-plugin-api-skill (https://github.com/kerim/logseq-db-plugin-api-skill) | DB-graph version: tags/classes, typed properties, file→DB migration |
| pangea.app glossary — Logseq (https://pangea.app/glossary/logseq) | Local-first framing, outliner, plain-text control |

Cross-references: `research/260614-roam-deep-dive/findings.md` (block-DB contrast),
`research/260614-obsidian-deep-dive/findings.md` (file-over-app, derived index),
`research/260614-joplin-deep-dive/findings.md` (format-aware file-store profiles),
`research/260614-shard-spectrum-synthesis/findings.md` (addressing spectrum this
resolves), `spec/UseCaseCatalog.md` (UC-32, UC-34, UC-43, UC-50, UC-51, UC-52, UC-55),
`workplans/SHARD-WP-0002-federation-architecture.md` (T11, T14, T16).

---

## 12. Traceability

- New UCs: **UC-62, UC-63** → `spec/UseCaseCatalog.md`.
- Enriched UCs: **UC-32, UC-34, UC-43, UC-50, UC-51, UC-52, UC-55**.
- Architecture (no UC): format-aware file-store profiles (Logseq profile); derived-query-
  index capability; substrate-migration tolerance; in-file block addressing as the T16
  span-address target → `SHARD-WP-0002` (T11, T14, T16).
- Boundary recorded: Logseq is **one block-graph-on-files candidate shard** (the
  addressing sweet spot), best attached file-store-direct with a format profile; not the
  federation layer; substrate is migrating file→SQLite (UC-43).
</content>