shard-wiki/INTENT.md

INTENT

Purpose

This repository exists to provide a Git-based Markdown wiki orchestrator and federation layer.

It enables multiple wiki-shaped knowledge stores, called shards, to attach to the same root entity and participate in a joined information space.

The goal is to allow independently stored and differently implemented wikis, page stores, directories, and content backends to behave as a union of pages, while preserving their separate storage, provenance, capabilities, and histories.

shard-wiki turns fragmented wiki storage into a coordinated, versionable, patchable, inspectable, and synchronizable information space.

---

Primary Utility

The repository provides a shard orchestration layer for interconnected Markdown and markup-based wiki content.

It allows wiki-like systems to:

• Attach heterogeneous page stores as shards of a shared information space
• Normalize pages, paths, links, metadata, revisions, overlays, and provenance across backends
• Present the union of pages from multiple shards as a coherent wiki graph
• Use Git as the coordination, history, backup, review, patching, and merge substrate for an information space
• Lazy-load remote or external pages as cache-like projections into local views
• Support remote editing through overlays, drafts, commits, patches, or merge requests
• Detect and reconcile divergence between equivalent pages stored in different shards
• Keep selected representations of the same wiki content synchronized where appropriate
• Provide backup, versioning, and recovery for local, remote, and projected wiki content
• Run fully standalone with open read/write access and complete change history, then progressively layer multi-tenant enterprise access control through external identity integration
• Allow existing wiki engines to become federation-capable through a shared API
• Allow non-federation-aware systems to participate through adapters and projections

It transforms disconnected wiki engines, Git repositories, local folders, WebDAV directories, application-specific content stores, and desktop editing workflows into a composable federated wiki space.

---

Intended Users

• People maintaining Markdown-based knowledge bases across multiple tools
• Projects that use Git repositories as durable knowledge stores
• Wiki users who want local, web, mobile, and repository-backed views of the same pages
• Developers building wiki engines that should interoperate with other wiki systems
• Systems such as Coulomb spaces that want to grow into wiki-like capabilities
• Gitea or Git-hosted wiki users who want repository-level access to wiki pages
• Obsidian users who want local editing over federated or synchronized wiki content
• Automation systems and agents operating over structured project knowledge
• Administrators who need backup, versioning, mirroring, and recovery of wiki content

---

Strategic Role in the System

This repository acts as the wiki federation and content orchestration layer within the broader system.

It decouples:

• Wiki user interfaces from storage backends
• Wiki engines from federation mechanics
• Markdown content from any single hosting environment
• Local editing workflows from remote publication workflows
• File synchronization from semantic wiki-page synchronization
• Content storage from change review, patching, and reconciliation

shard-wiki provides the shared coordination layer between systems such as:

• Coulomb spaces
• Git-backed repositories
• Gitea wiki storage
• Repository subdirectories such as wiki/
• Local filesystem directories
• WebDAV or Nextcloud directories
• Desktop tools such as Obsidian
• Future wiki engines that expose or consume the shard-wiki API

Its strategic role is to let higher-level systems treat wiki content as a federated, pluggable, versioned capability rather than as data locked inside one engine, one database, one directory layout, or one synchronization mechanism.

Each joined information space should have a Git-addressable coordination layer. Individual shards do not all need to be Git-native, but the information space as a whole should be representable through Git-backed history, patches, commits, reviews, backups, and merge operations wherever possible.

A mature shard-wiki should allow each participating shard to see the others as part of the same information space while preserving the provenance, freshness, capabilities, and limitations of every underlying backend.

---

Strategic Boundaries

This repository is not intended to:

• Replace all wiki engines with a single canonical wiki implementation
• Force every shard to use the same backend, database, directory layout, or storage format
• Require every participating system to become federation-aware
• Require every participating shard to be Git-native
• Define a universal ontology for all wiki knowledge
• Own domain-specific page semantics, project policies, or editorial rules
• Hide authorship, provenance, conflicts, freshness, or backend limitations
• Silently mutate remote systems without explicit adapter support and user intent
• Act as a generic file synchronization daemon independent of wiki semantics
• Guarantee real-time consistency across all shards and backends
• Implement its own authentication, identity store, user directory, or secret storage — these are delegated to an external identity provider (in the reference deployment, user-engine for accounts/memberships and net-kingdom for IAM/SSO). shard-wiki does own an authorization model (who may read/write/patch/merge a page or shard) and the change history that makes edits recoverable; it does not own authentication or identity lifecycle.
• Collapse all shards into one permanently canonical source of truth

Its responsibility is limited to orchestrating wiki-shaped content across shards, providing normalized interfaces, projections, overlays, synchronization mechanisms, Git-backed coordination, and versioned change flows.

Policy decisions such as conflict preference, canonical source selection, publication rules, access control, retention, editorial governance, and synchronization strategy should remain explicit and configurable rather than being hard-coded into the core abstraction.

---

Design Principles

• Federation by attachment
  A wiki or wiki-like store should be able to join an information space by being attached as a shard of a root entity.

• Git-addressable coordination
  Each information space should have a Git-backed coordination layer for history, patches, review, backup, reconciliation, and recovery, even when some attached shards are not Git-native.

• Shard sovereignty
  Each shard remains an independently meaningful store with its own capabilities, limitations, history, identity model, and access constraints.

• Union without erasure
  The joined information space should expose the union of pages while preserving where each page, revision, projection, overlay, and edit came from.

• Markdown-first, backend-neutral
  Markdown should be the primary content format, but the orchestration model should not assume that every backend stores pages in the same way.

• Lazy projection over eager copying
  Remote pages and external shard content should be loadable on demand and presented as local projections or caches where appropriate.

• Overlay before mutation
  Edits to remote, read-only, or capability-limited shards should be representable as overlays, drafts, patches, commits, or merge requests before being applied destructively.

• Patchable change flows
  The system should prefer auditable change proposals over opaque synchronization whenever a backend supports review, versioning, or merge semantics.

• Explicit provenance and freshness
  Users and systems should be able to see which shard a page came from, how fresh it is, whether it is cached, whether it has local overlays, and whether it diverges from another representation.

• Capability-aware adapters
  Different shards may support different operations. The core should model capabilities explicitly instead of assuming that all backends can read, write, diff, merge, lock, version, publish, or accept patches in the same way.

• Mechanism over policy
  The repository should provide primitives for federation, synchronization, overlays, patching, conflict detection, projection, and reconciliation, but avoid hard-coding one editorial or synchronization policy.

• Graceful degradation
  Backends with limited capabilities should still be useful as read-only shards, cache sources, projection targets, backup targets, or patch destinations.

• Local-first usefulness
  Local directories, Obsidian vaults, rsync workflows, and WebDAV-backed folders should be treated as legitimate participants in the information space.

• Composable integration
  Wiki engines should be able to use the shard-wiki API to become federation-enabled without reimplementing federation internally.

• Open by default, progressively governed
  A standalone shard-wiki must be runnable with zero external dependencies in a classic Ward Cunningham / c2-style open read/write-for-all mode. Access control is an additive capability, not a precondition: the same core progresses — without re-architecture — to authenticated single-user, to group/role-based, to multi-tenant enterprise access control, mirroring the NetKingdom capability ladder (lightweight → expanded).

• History as the safety net
  Because the open mode trusts all writers, every change must be captured in a complete, Git-backed history so that accidental loss, vandalism, or mistaken edits are always recoverable. Recoverability — not gatekeeping — is the baseline protection; access control hardens it but never replaces it.

• Authorization in core, authentication delegated
  shard-wiki owns the authorization model (capability-based decisions about who may read/write/patch/merge a page or shard) and expresses identity through a pluggable provider interface. It never implements its own authentication, credential store, or user directory; those are supplied by an external identity provider such as user-engine (accounts, memberships, profiles, audit) over a net-kingdom IAM backend (OIDC/PKCE in lightweight mode, Keycloak/SAML in expanded mode).

---

Maturity Target

A mature version of this repository should:

• Provide a stable, versioned shard adapter contract
• Provide a stable, versioned wiki page model for Markdown-oriented content
• Provide a stable, Git-backed coordination journal for each joined information space
• Support attachment of multiple shards to a shared root entity
• Support union views of pages, links, metadata, and revisions across all attached shards
• Preserve provenance for pages, revisions, links, projections, overlays, patches, and conflicts
• Support lazy loading and cache invalidation for remote shard content
• Support local overlays for edits against remote, read-only, or capability-limited shards
• Generate Git commits, patches, or merge requests from proposed edits where possible
• Detect divergence between equivalent pages stored in different shards
• Provide configurable reconciliation and synchronization mechanisms
• Support read-only, write-through, mirrored, projected, cached, and canonical shard modes
• Integrate with Git repositories and repository subdirectories such as wiki/
• Integrate with Gitea wiki storage or Gitea-backed wiki workflows
• Integrate with Coulomb spaces as wiki-capable knowledge shards
• Integrate with local directories for tools such as Obsidian
• Integrate with WebDAV or Nextcloud-backed directories for desktop and mobile workflows
• Provide a UI for browsing, editing, projecting, comparing, and reconciling the full information space
• Provide an API that allows wiki engines to become federation-enabled
• Provide backup and recovery workflows for attached wiki shards
• Run standalone with zero external dependencies in an open, c2-style read/write-for-all mode
• Preserve a complete, Git-backed change history that enables recovery from accidental loss, vandalism, or mistaken edits
• Provide a pluggable authorization model that spans the full range from anonymous-open, through authenticated and group/role-based, to multi-tenant enterprise access control — without changing the core
• Integrate with user-engine (accounts, memberships, profiles, audit, events) and its net-kingdom IAM backend (OIDC/PKCE lightweight mode, Keycloak/SAML expanded mode) to deliver enterprise-grade multi-tenant access control
• Serve as the default orchestration layer for federated Markdown wiki content across dependent systems

A mature shard-wiki should make the practical mixture of repository wikis, application databases, synced directories, local Obsidian vaults, WebDAV folders, projected pages, and Git-backed content feel like a deliberate architecture rather than an accidental workaround.

---

Stability Note

Changes to this file represent a deliberate shift in the abstraction boundaries or strategic role of this repository.

Such changes should be rare, because they affect all downstream systems relying on shard-wiki as a federation, orchestration, synchronization, projection, patching, backup, and reconciliation layer for wiki-shaped knowledge.

In particular, changes that redefine what counts as a shard, what role Git plays, how root entities are modeled, or whether shard-wiki is an orchestrator rather than a wiki engine should be treated as architectural changes.