Compare commits
20 Commits
c0615c2d50
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| cd8339ecef | |||
| f8ab58edbe | |||
| 2b5e9743fe | |||
| 753c3d4fc6 | |||
| 94e84f0db9 | |||
| a765ccda21 | |||
| 4472fa6c7f | |||
| 526fa1e3bc | |||
| 86de18c247 | |||
| ca9d0d7030 | |||
| bc527ec09a | |||
| ce984482e2 | |||
| 9266f124e6 | |||
| 8740a66611 | |||
| b7e9edbb4b | |||
| 479fa95fdf | |||
| eb9b622499 | |||
| e3e5b8ecc1 | |||
| 9e8d73fa7d | |||
| d44a4cd3df |
20
.claude/rules/agents.md
Normal file
20
.claude/rules/agents.md
Normal file
@@ -0,0 +1,20 @@
|
||||
## Kaizen Agents
|
||||
|
||||
Specialized agent personas available on demand via the state-hub MCP.
|
||||
|
||||
**Discover:** `list_kaizen_agents()` — returns all agents with name, description, category
|
||||
**Load:** `get_kaizen_agent("tdd-workflow")` — returns full instructions; read and follow them
|
||||
|
||||
Common agents:
|
||||
|
||||
| Agent | Category | When to use |
|
||||
|-------|----------|-------------|
|
||||
| `tdd-workflow` | testing | Step-by-step TDD8 workflow for any feature |
|
||||
| `code-refactoring` | quality | Code quality analysis and safe refactoring |
|
||||
| `test-maintenance` | testing | Diagnose and fix failing tests |
|
||||
| `requirements-engineering` | process | Prevent interface/mock mismatches upfront |
|
||||
| `keepaTodofile` | process | Maintain TODO.md during work |
|
||||
| `project-management` | process | Track status, determine next steps |
|
||||
| `datamodel-optimization` | quality | Optimize dataclasses and data structures |
|
||||
|
||||
All 17 agents: call `list_kaizen_agents()` for the full list.
|
||||
8
.claude/rules/architecture.md
Normal file
8
.claude/rules/architecture.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## Architecture
|
||||
|
||||
<!-- TODO: Describe the key design decisions and component structure.
|
||||
Key modules, data flows, external integrations, state machines, etc. -->
|
||||
|
||||
## Quick Reference
|
||||
|
||||
`~/state-hub/mcp_server/TOOLS.md` — MCP tool reference
|
||||
50
.claude/rules/credential-routing.md
Normal file
50
.claude/rules/credential-routing.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Credential and access routing
|
||||
|
||||
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||
|
||||
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||
other credential need belongs to another subsystem. **Do not** message
|
||||
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||
|
||||
### Lookup (do this first)
|
||||
|
||||
```bash
|
||||
warden route find "<describe your need>" --json
|
||||
warden route show <catalog-id> --json
|
||||
```
|
||||
|
||||
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||
|
||||
| Agent runtime | How to orient |
|
||||
| --- | --- |
|
||||
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=markitect-main` is for coordination, not secret vending |
|
||||
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||
|
||||
### Quick routing table
|
||||
|
||||
| I need… | Owner | ops-warden executes? |
|
||||
| --- | --- | --- |
|
||||
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||
| Authorization decision | flex-auth | No — route only |
|
||||
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||
|
||||
### Anti-patterns (do not do these)
|
||||
|
||||
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||
|
||||
### Other capabilities (reuse-surface)
|
||||
|
||||
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||
get wrong.
|
||||
|
||||
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||
38
.claude/rules/first-session.md
Normal file
38
.claude/rules/first-session.md
Normal file
@@ -0,0 +1,38 @@
|
||||
## First Session Protocol
|
||||
|
||||
Triggered when `get_domain_summary("communication")` shows **no workstreams**.
|
||||
The project is registered but work has not yet been structured.
|
||||
|
||||
**Step 1 — Read, don't write**
|
||||
- `~/the-custodian/canon/projects/communication/project_charter_v0.1.md` — purpose, scope
|
||||
- `~/the-custodian/canon/projects/communication/roadmap_v0.1.md` — planned phases
|
||||
- Scan repo root: README, directory structure, existing code or docs
|
||||
|
||||
**Step 2 — Survey in-progress work**
|
||||
Look for TODOs, open branches, half-finished files. Note done vs. started but incomplete.
|
||||
|
||||
**Step 3 — Propose workstreams to Bernd**
|
||||
Propose 1–3 workstreams — each a coherent strand, weeks to months, anchored to a
|
||||
roadmap phase. **Wait for approval before creating.**
|
||||
|
||||
**Step 4 — Create workplan file first, then DB record (ADR-001)**
|
||||
```
|
||||
workplans/MARKITECT-WP-NNNN-<slug>.md ← write this first
|
||||
```
|
||||
Then register in the hub:
|
||||
```
|
||||
create_workstream(topic_id="36c7421b-c537-4723-bf75-42a3ebc6a1dc", title="...", owner="...", description="...")
|
||||
create_task(workstream_id="<id>", title="...", priority="high|medium|low")
|
||||
```
|
||||
|
||||
**Step 5 — Record the setup**
|
||||
```
|
||||
add_progress_event(
|
||||
summary="First session: structured communication into N workstreams, M tasks",
|
||||
event_type="milestone",
|
||||
topic_id="36c7421b-c537-4723-bf75-42a3ebc6a1dc",
|
||||
detail={"workstreams": [...], "tasks_created": M}
|
||||
)
|
||||
```
|
||||
|
||||
<!-- Delete or archive this file once past first session -->
|
||||
8
.claude/rules/repo-boundary.md
Normal file
8
.claude/rules/repo-boundary.md
Normal file
@@ -0,0 +1,8 @@
|
||||
## Repo boundary
|
||||
|
||||
This repo owns **Markitect Main** only. It does not own:
|
||||
|
||||
<!-- TODO: List what belongs in adjacent repos, e.g.:
|
||||
- SSH key management → railiance-infra/
|
||||
- State hub code → state-hub/
|
||||
-->
|
||||
5
.claude/rules/repo-identity.md
Normal file
5
.claude/rules/repo-identity.md
Normal file
@@ -0,0 +1,5 @@
|
||||
**Purpose:** Markitect Main - (fill in purpose)
|
||||
|
||||
**Domain:** communication
|
||||
**Repo slug:** markitect-main
|
||||
**Topic ID:** 36c7421b-c537-4723-bf75-42a3ebc6a1dc
|
||||
85
.claude/rules/session-protocol.md
Normal file
85
.claude/rules/session-protocol.md
Normal file
@@ -0,0 +1,85 @@
|
||||
## Session Protocol
|
||||
|
||||
Dev Hub (State Hub API): http://127.0.0.1:8000
|
||||
MCP server name in `~/.claude.json`: `dev-hub`
|
||||
|
||||
**Step 1 — Orient**
|
||||
|
||||
Read the offline-safe brief first — it works without a live hub connection:
|
||||
```bash
|
||||
cat .custodian-brief.md
|
||||
```
|
||||
Then call the MCP tool for richer cross-domain context when MCP tools are exposed:
|
||||
```
|
||||
get_domain_summary("communication")
|
||||
```
|
||||
If MCP tools are unavailable in the current agent session, use the REST API:
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:8000/state/summary" | python3 -m json.tool
|
||||
```
|
||||
If the hub is offline: `cd ~/state-hub && make api`
|
||||
|
||||
**Step 2 — Check inbox**
|
||||
With MCP tools:
|
||||
```
|
||||
get_messages(to_agent="markitect-main", unread_only=True)
|
||||
```
|
||||
Mark read with `mark_message_read(message_id)`. Reply or act on coordination
|
||||
requests before proceeding.
|
||||
|
||||
Without MCP tools:
|
||||
```bash
|
||||
curl -s "http://127.0.0.1:8000/messages/?to_agent=markitect-main&unread_only=true" \
|
||||
| python3 -m json.tool
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||
-H "Content-Type: application/json" -d '{}'
|
||||
```
|
||||
|
||||
**Step 3 — Scan workplans**
|
||||
```bash
|
||||
ls workplans/
|
||||
```
|
||||
For each file with `status: ready`, `active`, or `blocked`, note pending
|
||||
`wait`/`todo`/`progress` tasks.
|
||||
|
||||
**Step 4 — Present brief**
|
||||
|
||||
1. **Active workstreams** for `communication` — title, task counts, blocking decisions
|
||||
2. **Pending tasks** from `workplans/` + any `[repo:markitect-main]` hub tasks
|
||||
3. **Goal guidance** — if `goal_guidance` in summary:
|
||||
- `needs_workplan`: surface as top action — *"Repo goal '{title}' has no workplan yet"*
|
||||
- `alignment_warnings`: flag if active work is not aligned with current goal
|
||||
4. **Suggested next action** — highest-priority open item
|
||||
5. **SBOM status** — flag if `last_sbom_at` is unset for this repo
|
||||
|
||||
If no workstreams: follow First Session Protocol (`first-session.md`).
|
||||
|
||||
**During work:** `record_decision()` · `add_progress_event()` · `resolve_decision()`
|
||||
|
||||
> State Hub is a *read model*. Bootstrap tools (`create_workstream`, `create_task`)
|
||||
> are First Session Protocol only. Work structure belongs in repo files (ADR-001).
|
||||
|
||||
**Session close:**
|
||||
With MCP tools:
|
||||
```
|
||||
add_progress_event(summary="...", topic_id="36c7421b-c537-4723-bf75-42a3ebc6a1dc", workstream_id="<uuid>")
|
||||
```
|
||||
Without MCP tools:
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"topic_id":"36c7421b-c537-4723-bf75-42a3ebc6a1dc","workstream_id":"<uuid>","event_type":"note","summary":"what changed","author":"codex"}'
|
||||
```
|
||||
If workplan files were modified, ensure the local copy is up to date first:
|
||||
```bash
|
||||
git -C <repo_path> pull --ff-only
|
||||
cd ~/state-hub && make fix-consistency REPO=markitect-main
|
||||
```
|
||||
For repos where implementation runs on a remote machine (e.g. CoulombCore),
|
||||
use the combined target which pulls before fixing:
|
||||
```bash
|
||||
cd ~/state-hub && make fix-consistency-remote REPO=markitect-main
|
||||
```
|
||||
**C-15** (DB task ahead of file) is normal in multi-machine workflows — writeback
|
||||
will sync the file to match DB. **C-16** (repo behind remote) blocks all writes
|
||||
until you pull — intentional to prevent clobbering remote progress.
|
||||
16
.claude/rules/stack-and-commands.md
Normal file
16
.claude/rules/stack-and-commands.md
Normal file
@@ -0,0 +1,16 @@
|
||||
## Stack
|
||||
|
||||
- **Language:** Python 3.12+ (monorepo) + JavaScript UI (testdrive-jsui)
|
||||
- **Key deps:** uv/pip, pytest, npm; see `pyproject.toml`, `package.json`, `Makefile`
|
||||
|
||||
## Dev Commands
|
||||
|
||||
```bash
|
||||
make setup
|
||||
make test
|
||||
make test-js
|
||||
make test-all
|
||||
make lint
|
||||
make build
|
||||
make help
|
||||
```
|
||||
40
.claude/rules/workplan-convention.md
Normal file
40
.claude/rules/workplan-convention.md
Normal file
@@ -0,0 +1,40 @@
|
||||
## Workplan Convention (ADR-001)
|
||||
|
||||
File location: `workplans/MARKITECT-WP-NNNN-<slug>.md`
|
||||
ID prefix: `MARKITECT-WP-`
|
||||
|
||||
Work items originate as files in this repo **before** being registered in the hub.
|
||||
|
||||
Canonical workplan/workstream frontmatter statuses are:
|
||||
`proposed`, `ready`, `active`, `blocked`, `backlog`, `finished`, `archived`.
|
||||
Use `proposed` for a newly drafted plan, `ready` after review against current
|
||||
repo state, and `finished` when implementation is complete. `stalled` and
|
||||
`needs_review` are derived health labels, not stored statuses.
|
||||
|
||||
Closed workplans may be moved to `workplans/archived/` with a completion-date
|
||||
prefix: `YYMMDD-MARKITECT-WP-NNNN-<slug>.md`. The frontmatter id remains
|
||||
unchanged; the prefix is only for quick visual reference.
|
||||
|
||||
Small opportunistic tasks discovered during another session use **Ad Hoc Tasks**:
|
||||
`workplans/ADHOC-YYYY-MM-DD.md`, workstream slug `adhoc-YYYY-MM-DD`, and task ids
|
||||
`ADHOC-YYYY-MM-DD-T01`, `T02`, etc. Use adhocs only for low-risk work completed
|
||||
directly. Promote anything requiring analysis, design, approval, dependencies, or
|
||||
multiple planned phases into a normal workplan.
|
||||
|
||||
Ecosystem todos from other agents arrive as `[repo:markitect-main]` hub tasks —
|
||||
visible at session start. Pick one up by creating the workplan file, then registering
|
||||
the workstream.
|
||||
|
||||
Task blocks use this shape:
|
||||
|
||||
```task
|
||||
id: MARKITECT-WP-NNNN-T01
|
||||
status: wait | todo | progress | done | cancel
|
||||
priority: high | medium | low
|
||||
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
```
|
||||
|
||||
Status progression is `todo` → `progress` → `done`; use `wait` for waiting or
|
||||
blocked work and `cancel` for stopped work.
|
||||
|
||||
<!-- Ralph Loop rules and HEUREKA sequence: ~/.claude/CLAUDE.md — do not duplicate here -->
|
||||
@@ -1,42 +1,18 @@
|
||||
<!-- custodian-brief: generated by fix-consistency — do not edit manually -->
|
||||
# Custodian Brief — markitect-project
|
||||
# Custodian Brief — markitect-main
|
||||
|
||||
**Domain:** markitect
|
||||
**Last synced:** 2026-04-21 22:28 UTC
|
||||
**Domain:** communication
|
||||
**Last synced:** 2026-06-22 21:32 UTC
|
||||
**State Hub:** http://127.0.0.1:8000 *(adjust if running on a remote machine)*
|
||||
|
||||
## Active Workstreams
|
||||
|
||||
### TestDrive-JSUI — npm Publication
|
||||
Progress: 0/9 done | workstream_id: `e203d487-01f1-494a-b14d-a436241a4c01`
|
||||
|
||||
**Open tasks:**
|
||||
- · P.1 — Decide repository structure (monorepo vs standalone) `81c03377`
|
||||
- · P.2 — Verify Markitect integration still works `f518601b`
|
||||
- · P.3 — Resolve STANDALONE_PLAN.md status `d700098c`
|
||||
- · P.4 — Pack and dry-run publish `94dd2a30`
|
||||
- · P.5 — Create v1.0.0 release tag `d7c2ce00`
|
||||
- · P.6 — Publish to npm and verify CDN `8bcde75e`
|
||||
- · P.7 — Fresh install test in clean environment `61d14b53`
|
||||
- … and 2 more open tasks
|
||||
|
||||
### Infospace Tooling — Stage 3 Close-out
|
||||
Progress: 0/8 done | workstream_id: `830c888e-e1d4-43e6-8093-9b61e7578257`
|
||||
|
||||
**Open tasks:**
|
||||
- · C.1 — Evaluate the 3 missing entities `0fa8f461`
|
||||
- · C.2 — Run eval-summary and verify viability (6/6 PASS) `ba7d992c`
|
||||
- · C.3 — Refresh metrics report from full 988-entity set `84a59244`
|
||||
- · C.4 — Document advanced usage patterns `0ef75ee5`
|
||||
- · C.5 — Add composition guide referencing supply-chain-vsm `864977db`
|
||||
- · C.6 — Write performance notes `414496b0`
|
||||
- · C.7 — S3.2: Complete clean per-chapter git history `21c865c1`
|
||||
- … and 1 more open tasks
|
||||
*(none — repo may need first-session setup)*
|
||||
|
||||
---
|
||||
## MCP Orientation (when available)
|
||||
|
||||
If the state-hub MCP server is reachable, call:
|
||||
`get_domain_summary("markitect")`
|
||||
`get_domain_summary("communication")`
|
||||
This provides richer cross-domain context.
|
||||
If the MCP call fails, use this file as your orientation source.
|
||||
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -91,6 +91,8 @@ debug_*.py
|
||||
|
||||
# Claude Code local settings (user-specific permissions)
|
||||
.claude/settings.local.json
|
||||
# Claude Code runtime session locks (per-session, not content)
|
||||
.claude/*.lock
|
||||
|
||||
.aider*
|
||||
|
||||
|
||||
25
.repo-classification.yaml
Normal file
25
.repo-classification.yaml
Normal file
@@ -0,0 +1,25 @@
|
||||
repo_classification:
|
||||
standard: Repo Classification Standard
|
||||
version: '1.0'
|
||||
classified_at: '2026-06-22'
|
||||
classified_by: human
|
||||
category: product
|
||||
domain: communication
|
||||
secondary_domains:
|
||||
- infotech
|
||||
- agents
|
||||
capability_tags:
|
||||
- knowledge
|
||||
- documentation
|
||||
- product-development
|
||||
- platform
|
||||
business_stake:
|
||||
- product
|
||||
- technology
|
||||
- execution
|
||||
business_mechanics:
|
||||
- intention
|
||||
- coordination
|
||||
- operation
|
||||
- adaptation
|
||||
notes: Markitect successor to archived markitect-project; human confirmed.
|
||||
219
AGENTS.md
Normal file
219
AGENTS.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# Markitect Main — Agent Instructions
|
||||
|
||||
## Repo Identity
|
||||
|
||||
**Purpose:** Markitect Main - (fill in purpose)
|
||||
|
||||
**Domain:** communication
|
||||
**Repo slug:** markitect-main
|
||||
**Topic ID:** `36c7421b-c537-4723-bf75-42a3ebc6a1dc`
|
||||
**Workplan prefix:** `MARKITECT-WP-`
|
||||
|
||||
---
|
||||
|
||||
## State Hub Integration
|
||||
|
||||
The Custodian State Hub tracks work across all domains. Interact via HTTP REST —
|
||||
there is no MCP server for Codex agents.
|
||||
|
||||
| Context | URL |
|
||||
|---------|-----|
|
||||
| Local workstation | `http://127.0.0.1:8000` |
|
||||
| Remote via tunnel | `http://127.0.0.1:18000` |
|
||||
|
||||
### Orient at session start
|
||||
|
||||
```bash
|
||||
# Offline brief — works without hub connection
|
||||
cat .custodian-brief.md
|
||||
|
||||
# Active workstreams for this domain
|
||||
curl -s "http://127.0.0.1:8000/workstreams/?topic_id=36c7421b-c537-4723-bf75-42a3ebc6a1dc&status=active" \
|
||||
| python3 -m json.tool
|
||||
|
||||
# Check inbox
|
||||
curl -s "http://127.0.0.1:8000/messages/?to_agent=markitect-main&unread_only=true" \
|
||||
| python3 -m json.tool
|
||||
```
|
||||
|
||||
Mark a message read:
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/messages/<id>/read" \
|
||||
-H "Content-Type: application/json" -d '{}'
|
||||
```
|
||||
|
||||
### Log progress (required at session close)
|
||||
|
||||
```bash
|
||||
curl -s -X POST http://127.0.0.1:8000/progress/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"summary": "what was done",
|
||||
"event_type": "note",
|
||||
"author": "codex",
|
||||
"workstream_id": "<uuid>",
|
||||
"task_id": "<uuid>"
|
||||
}'
|
||||
```
|
||||
|
||||
Omit `workstream_id` / `task_id` when not applicable.
|
||||
|
||||
### Update task status
|
||||
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"status": "progress"}'
|
||||
# values: wait | todo | progress | done | cancel
|
||||
```
|
||||
|
||||
### Flag a task for human review
|
||||
|
||||
```bash
|
||||
curl -s -X PATCH "http://127.0.0.1:8000/tasks/<task_id>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"needs_human": true, "intervention_note": "reason"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Session Protocol
|
||||
|
||||
**Start:**
|
||||
1. `cat .custodian-brief.md` — domain goal and open workstreams (offline-safe)
|
||||
2. Check inbox: `GET /messages/?to_agent=markitect-main&unread_only=true`; mark read
|
||||
3. Scan workplans: `ls workplans/` — note `status: ready`, `active`, or `blocked` files and open tasks
|
||||
4. Check human-needed tasks: `GET /tasks/?needs_human=true`
|
||||
|
||||
**During work:**
|
||||
- Update task statuses in workplan files as tasks progress
|
||||
- Record significant decisions via `POST /decisions/`
|
||||
|
||||
**Close:**
|
||||
1. Update workplan file task statuses to reflect progress
|
||||
2. Log: `POST /progress/` with a summary of what changed
|
||||
3. Note for the custodian operator: after workplan file changes, run from
|
||||
`~/state-hub`:
|
||||
```bash
|
||||
make fix-consistency REPO=markitect-main
|
||||
```
|
||||
This syncs task status from files into the hub DB.
|
||||
|
||||
---
|
||||
|
||||
## Credential and access routing
|
||||
|
||||
**Audience:** Codex, Claude Code, Grok, and custodian agents that call **llm-connect**
|
||||
for inference. Run this check **before** requesting secrets, API keys, SSH access,
|
||||
login tokens, or database passwords — in any repo, not only `ops-warden`.
|
||||
|
||||
ops-warden **issues SSH certificates only** (`warden sign`, `cert_command`). Every
|
||||
other credential need belongs to another subsystem. **Do not** message
|
||||
`ops-warden` on State Hub expecting a secret value; the reply is a pointer, not a key.
|
||||
|
||||
### Lookup (do this first)
|
||||
|
||||
```bash
|
||||
warden route find "<describe your need>" --json
|
||||
warden route show <catalog-id> --json
|
||||
```
|
||||
|
||||
Requires the `warden` CLI from `~/ops-warden` (`uv tool install .` or `uv run warden`).
|
||||
|
||||
| Agent runtime | How to orient |
|
||||
| --- | --- |
|
||||
| **Codex / Grok** (shell, HTTP State Hub) | `warden route` commands above; inbox `to_agent=markitect-main` is for coordination, not secret vending |
|
||||
| **Claude Code** (MCP when available) | `get_domain_summary("custodian")` for workstreams; **still** use `warden route` for credential ownership |
|
||||
| **llm-connect** (inference service) | Never put secret retrieval in prompts; route custody to OpenBao/operator paths surfaced by `warden route` |
|
||||
|
||||
### Quick routing table
|
||||
|
||||
| I need… | Owner | ops-warden executes? |
|
||||
| --- | --- | --- |
|
||||
| SSH cert (`adm`/`agt`/`atm`) | ops-warden | **Yes** — `warden sign` |
|
||||
| API key, DB password, provider token | OpenBao (`railiance-platform`) | No — route only |
|
||||
| Login / OIDC / MFA | key-cape / Keycloak | No — route only |
|
||||
| Authorization decision | flex-auth | No — route only |
|
||||
| activity-core → issue-core emission | activity-core + issue-core | No — `warden route show activity-core-issue-sink` |
|
||||
| SSH tunnel | ops-bridge (+ `cert_command` from warden) | No — route only |
|
||||
|
||||
### Anti-patterns (do not do these)
|
||||
|
||||
- `POST /messages/` to `ops-warden` asking for `ISSUE_CORE_API_KEY`, `OPENROUTER_API_KEY`, etc.
|
||||
- Inventing `warden secret`, `warden login`, `warden bao`, `warden tunnel` — they do not exist
|
||||
- Pasting secrets into Git, State Hub, workplans, logs, or chat
|
||||
|
||||
### Other capabilities (reuse-surface)
|
||||
|
||||
Non-credential capabilities are usually discovered through **reuse-surface** federation
|
||||
(`reuse-surface` registry / `capability.*` indexes). Credential routing is inlined in
|
||||
every repo's agent instructions because it is high-frequency, high-risk, and easy to
|
||||
get wrong.
|
||||
|
||||
**Canon:** `~/ops-warden/wiki/CredentialRouting.md` · catalog `~/ops-warden/registry/routing/catalog.yaml`
|
||||
|
||||
<!-- REPO-AGENTS-EXTENSIONS -->
|
||||
<!-- Append repo-specific agent instructions below this marker.
|
||||
The state-hub template sync preserves content after this line. -->
|
||||
|
||||
---
|
||||
|
||||
## Workplan Convention (ADR-001)
|
||||
|
||||
Work items originate as files in this repo — not in the hub. The hub is a
|
||||
read/cache/index layer that rebuilds from files.
|
||||
|
||||
**File location:** `workplans/MARKITECT-WP-NNNN-<slug>.md`
|
||||
|
||||
**Archived location:** finished workplans may move to
|
||||
`workplans/archived/YYMMDD-MARKITECT-WP-NNNN-<slug>.md`. The `YYMMDD` prefix is
|
||||
the completion/archive date; the frontmatter `id` does not change.
|
||||
|
||||
**Ad Hoc Tasks:** small opportunistic fixes discovered during a session use
|
||||
`workplans/ADHOC-YYYY-MM-DD.md` with task ids `ADHOC-YYYY-MM-DD-T01`, etc. Use
|
||||
this only for low-risk work completed directly; create a normal workplan for
|
||||
anything needing analysis, design, approval, dependencies, or multiple phases.
|
||||
|
||||
**Frontmatter:**
|
||||
|
||||
```yaml
|
||||
---
|
||||
id: MARKITECT-WP-NNNN
|
||||
type: workplan
|
||||
title: "..."
|
||||
domain: communication
|
||||
repo: markitect-main
|
||||
status: proposed | ready | active | blocked | backlog | finished | archived
|
||||
owner: codex
|
||||
topic_slug: ...
|
||||
created: "YYYY-MM-DD"
|
||||
updated: "YYYY-MM-DD"
|
||||
state_hub_workstream_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
---
|
||||
```
|
||||
|
||||
Use `proposed` for a new draft, `ready` after review against current repo
|
||||
state, and `finished` after implementation. `stalled` and `needs_review` are
|
||||
derived health labels, not frontmatter statuses.
|
||||
|
||||
**Task block format** (one per `##` section):
|
||||
|
||||
```
|
||||
## Task Title
|
||||
|
||||
` ` `task
|
||||
id: MARKITECT-WP-NNNN-T01
|
||||
status: wait | todo | progress | done | cancel
|
||||
priority: high | medium | low
|
||||
state_hub_task_id: "<uuid>" # written by fix-consistency — do not edit
|
||||
` ` `
|
||||
|
||||
Task description text.
|
||||
```
|
||||
|
||||
Status progression: `todo` → `progress` → `done`; use `wait` for waiting/blocked work and `cancel` for stopped work.
|
||||
|
||||
To create a new workplan:
|
||||
1. Write the file following the format above
|
||||
2. Notify the custodian operator to run `make fix-consistency REPO=markitect-main`
|
||||
(or send a message to the hub agent via `POST /messages/`)
|
||||
132
CLAUDE.md
132
CLAUDE.md
@@ -1,120 +1,12 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Custodian State Hub Integration
|
||||
|
||||
This project is tracked as the **markitect** domain in the Custodian State Hub.
|
||||
Hub topic ID: `5571d954-0d30-4950-980d-7bcaaad8e3e2`
|
||||
|
||||
**Session start:** Call `get_domain_summary("markitect")` via the `state-hub` MCP tool.
|
||||
If the hub is not reachable: `cd ~/the-custodian/state-hub && make api`
|
||||
|
||||
**Session end:** Call `add_progress_event()` with `topic_id` above, a `summary`, and `event_type` (`note` / `milestone` / `blocker`).
|
||||
|
||||
**Available state-hub MCP tools:** `get_state_summary`, `get_domain_summary`, `add_progress_event`, `create_workstream`, `create_task`, `update_task_status`, `record_decision`, `resolve_decision`.
|
||||
|
||||
If the hub API is unavailable, use `curl` against `http://127.0.0.1:8000/docs`.
|
||||
|
||||
---
|
||||
|
||||
## Development Commands
|
||||
|
||||
All commands assume the `markitect-venv` virtual environment is active (`source ~/.venvs/markitect-venv/bin/activate` or equivalent). The package is installed in editable mode.
|
||||
|
||||
```bash
|
||||
# Run the core test suite
|
||||
pytest
|
||||
|
||||
# Run a single test file
|
||||
pytest tests/test_issue_17_batch_processing.py
|
||||
|
||||
# Run tests by marker
|
||||
pytest -m unit
|
||||
pytest -m "not slow"
|
||||
|
||||
# Run with coverage
|
||||
pytest --cov=markitect
|
||||
|
||||
# Run only fast unit tests (TDD inner loop)
|
||||
pytest tests/unit/
|
||||
|
||||
# CLI entry point
|
||||
markitect --help
|
||||
|
||||
# Infospace subcommands (primary active development area)
|
||||
markitect infospace --help
|
||||
markitect infospace eval-summary --update-metrics
|
||||
|
||||
# LLM helper commands
|
||||
markitect llm-helper "<question>"
|
||||
markitect llm-catalog
|
||||
markitect llm-check
|
||||
markitect llm-default
|
||||
markitect llm-preference
|
||||
|
||||
# Install / reinstall in dev mode
|
||||
pip install -e ".[analysis]"
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
MarkiTect is a CLI-driven markdown engine that treats documents as structured, queryable information spaces. Entry point: `markitect/cli.py` → `main()` (Click group). All subcommand groups are registered at the bottom of `cli.py`.
|
||||
|
||||
### Core Modules (`markitect/`)
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `spaces/` | InformationSpace model — composability, events, history, transclusion, rendering, sync |
|
||||
| `prompts/` | Prompt resolver, compiler, execution engine, quality gates, dependency graph, traceability |
|
||||
| `llm/` | LLM adapter layer — 4 text providers + embedding adapter; 7-layer config resolution |
|
||||
| `infospace/` | Infospace lifecycle — init, entity parsing, evaluation, composition, graph export |
|
||||
| `analysis/` | Graph analysis (networkx) and FCA (Formal Concept Analysis, pure Python) |
|
||||
| `schema*/` | Schema generation, validation, naming, refinement, metaschema |
|
||||
| `core/` | Foundational parser, AST, serializer, workspace |
|
||||
|
||||
### LLM Configuration (`markitect/llm/`)
|
||||
|
||||
Resolution order (highest → lowest priority):
|
||||
1. CLI flags (`--provider`, `--model`)
|
||||
2. `MARKITECT_HELPER_MODEL` env var (model only)
|
||||
3. User preference (`[llm.preference]` in `~/.config/markitect/config.toml`)
|
||||
4. Directory preference (`[llm.preference]` in `.markitect.toml`)
|
||||
5. Directory default (`[llm.default]` in `.markitect.toml`)
|
||||
6. User default (`[llm.default]` in `~/.config/markitect/config.toml`)
|
||||
7. Hardcoded fallback: `gemini/gemini-2.5-flash`
|
||||
|
||||
Canonical models: `markitect/llm/models.py` (`RunConfig`, `LLMResponse`) and `markitect/llm/adapter.py` (`LLMAdapter`). These are mirrored in the standalone `/home/worsch/llm-connect/` package (`llm_connect`), which is installed as a local path dependency. `markitect/prompts/execution/{models,llm_adapter}.py` are re-export shims for backward compatibility.
|
||||
|
||||
**Gotcha:** The `OPENROUTER_API_KEY` env var may hold a stale key — `unset OPENROUTER_API_KEY` before running LLM commands if you get auth errors. `claude-code` provider fails inside Claude Code sessions (nested session restriction).
|
||||
|
||||
### Infospace (`markitect/infospace/`)
|
||||
|
||||
An infospace is a directory-based collection of typed entities governed by an `infospace.yaml` config. Key files:
|
||||
- `config.py` — config loading, `find_infospace_config()`, `DisciplineBinding`
|
||||
- `entity_parser.py` — parse entity markdown files from `output/entities/<slug>.md`
|
||||
- `evaluation.py` / `evaluation_io.py` — per-entity LLM evaluation (5 dimensions)
|
||||
- `state.py` — `build_state()` aggregates entity + evaluation state
|
||||
- `history.py` / `checks/` — viability tracking and collection-level checks
|
||||
|
||||
Slugs use **underscores** (e.g., `accumulation_of_stock`). Active example: `examples/infospace-with-history/` (988 entities).
|
||||
|
||||
### Layered Architecture
|
||||
|
||||
The project has a partially-implemented layered architecture separate from `markitect/`:
|
||||
- `domain/` — domain models (issues, projects)
|
||||
- `infrastructure/` — config, repositories, logging, connection management
|
||||
- `application/` — application layer (mostly empty)
|
||||
- `capabilities/` — pluggable capability packages (`issue-facade`, `release-management`, `testdrive-jsui`, `markitect-utils`, `markitect-content`, `kaizen-agentic`)
|
||||
|
||||
### Tests
|
||||
|
||||
Tests live in `tests/`. Many test files are named `test_issue_NNN_*.py` (TDD by issue). Layered tests follow `test_l{N}_*.py` naming. The `tests/unit/` subtree has the cleanest structure and covers `prompts/`, `spaces/`, `llm/`, `infospace/`, `analysis/`.
|
||||
|
||||
Pytest config is in `pytest.ini`. Relevant markers: `unit`, `integration`, `e2e`, `slow`, `performance`.
|
||||
|
||||
### Active Roadmap Files
|
||||
|
||||
- `roadmap/infospace-tooling/PLAN.md` — main infospace roadmap (S1 ✅ S2 ✅ S3 nearly done)
|
||||
- `roadmap/infospace-s3-closeout/PLAN.md` — S3 close-out tasks (C.1–C.8)
|
||||
- `roadmap/llm-shared-library/PLAN.md` — llm-connect extraction (S1+S2 done, S3 pending)
|
||||
# Markitect Main — Claude Code Instructions
|
||||
|
||||
@SCOPE.md
|
||||
@.claude/rules/repo-identity.md
|
||||
@.claude/rules/session-protocol.md
|
||||
@.claude/rules/first-session.md
|
||||
@.claude/rules/workplan-convention.md
|
||||
@.claude/rules/stack-and-commands.md
|
||||
@.claude/rules/architecture.md
|
||||
@.claude/rules/repo-boundary.md
|
||||
@.claude/rules/credential-routing.md
|
||||
@.claude/rules/agents.md
|
||||
|
||||
2
SCOPE.md
2
SCOPE.md
@@ -89,7 +89,7 @@ MarkiTect turns fragmented knowledge (scattered docs, chats, notes) into structu
|
||||
|
||||
---
|
||||
|
||||
## Related / Overlapping Repositories
|
||||
## Related / Overlapping
|
||||
|
||||
- `llm-connect` — standalone LLM adapter extracted from MarkiTect (dependency)
|
||||
- `the-custodian` — tracks markitect workstreams; custodian canon includes a markitect domain charter
|
||||
|
||||
141
docs/successor-gap-assessment.md
Normal file
141
docs/successor-gap-assessment.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# markitect-main → Successor Repos: Gap Assessment
|
||||
|
||||
**Date:** 2026-05-23
|
||||
**Author:** Claude (custodian session)
|
||||
**Status:** Draft — awaiting Bernd's decisions on items A/B/C below
|
||||
|
||||
## Purpose
|
||||
|
||||
Bernd is retiring `markitect-main` and has transferred most functionality to
|
||||
sibling repos. This document identifies what was provided by `markitect-main`
|
||||
that is **not addressed** in those successors, and flags candidates that may
|
||||
not fit any successor's intent.
|
||||
|
||||
## Successor Ecosystem (5 repos, not 3)
|
||||
|
||||
| Repo | Role |
|
||||
|---|---|
|
||||
| `markitect-tool` | Markdown syntax layer + structured-document primitives; defines source-adapter and render-adapter contracts. CLI: `mkt`. |
|
||||
| `kontextual-engine` | Headless knowledge operations engine: artifacts, collections, persistence, relationships, workflow runs/manifests, query, quality/assessment, API. |
|
||||
| `infospace-bench` | Application layer — concrete infospaces, evaluation methodology, reference pilots. |
|
||||
| `markitect-filter` | Source-format ingestion adapters (`source.epub3`, `source.pdf`) implementing the markitect-tool source-adapter contract. |
|
||||
| `markitect-quarkdown` | Render/export adapter — implements the markitect-tool render-adapter contract via Quarkdown. |
|
||||
|
||||
## Method
|
||||
|
||||
Analysis is grounded in each successor's own assessment docs (recent, May 2026):
|
||||
|
||||
- `markitect-tool/docs/markitect-main-scope-assessment.md`
|
||||
- `kontextual-engine/docs/markitect-main-scope-assessment.md`
|
||||
- `kontextual-engine/docs/system-layer-extraction-inventory.md`
|
||||
- `kontextual-engine/docs/system-layer-migration-backlog.md`
|
||||
- `infospace-bench/docs/markitect-main-scope-assessment.md`
|
||||
- `infospace-bench/docs/legacy-infospace-feature-inventory.md`
|
||||
- `infospace-bench/docs/replacement-acceptance-matrix.md`
|
||||
|
||||
Cross-checked against actual `markitect-main` module sizing (Python LOC) and
|
||||
`__init__.py` docstrings.
|
||||
|
||||
**Confidence:** These successor docs are authoritative on *intent*. They have
|
||||
**not** been line-verified to confirm every "reimplement"-classified item
|
||||
actually landed in the successor. Where verification matters, it's flagged.
|
||||
|
||||
---
|
||||
|
||||
## A. Doesn't fit any successor's intent — needs a new home or explicit retirement
|
||||
|
||||
These are explicitly pushed away by tool/engine/bench and are unrelated to
|
||||
filter/quarkdown.
|
||||
|
||||
| markitect-main area | LOC | What it is | Status |
|
||||
|---|---|---|---|
|
||||
| `markitect/finance/` | ~8,100 | Cost-tracking system: cost items, period allocation to issues, financial reports, audit trails | **Orphan.** markitect-main's own SCOPE.md lists "financial transactions" as out-of-scope. Belongs with issue/project-ops, not knowledge tooling. |
|
||||
| `issue_tracker/` + `_issue-tracking/` + `.issues/` | ~1,200 | Issue tracking (finance allocates costs to these issues) | **Orphan to the five** — but likely already superseded by the `issue-facade` capability / `use-issues` skill. **Verify before retiring.** |
|
||||
| `markitect/profile/` | ~1,600 | User-profile CRUD, multi-profile, DB-backed | **Orphan.** Unrelated to all five. (Distinct from quarkdown's *render* "profile".) |
|
||||
| `markitect/production/` | ~3,800 | Deployment-readiness validation, cross-platform checks, perf benchmarking | Engine keeps only "structured error/audit *ideas*". Deployment-validation bulk is orphan. |
|
||||
| `tools/`, `services/`, gitea/tddai glue | ~5,500 | Project-ops tooling | Out-of-scope everywhere. |
|
||||
| `markitect/legacy/` + `legacy_compat.py` | ~2,700 | Backward-compat shims | Retire by definition. |
|
||||
|
||||
## B. Rendering / asset / plugin layer — only *partially* covered, real residual gap
|
||||
|
||||
**This is the most consequential gap.** `SCOPE.md` lists "Rendering: markdown
|
||||
→ interactive HTML via plugin system (testdrive-jsui)" as an in-scope
|
||||
capability of markitect-main.
|
||||
|
||||
| Area | LOC | Covered? |
|
||||
|---|---|---|
|
||||
| `markitect/plugins/` (generic processor/formatter/validator/exporter plugin system) | ~8,000 | **No.** tool defines a render-adapter *contract* and an *extension* point, but the general plugin runtime isn't carried. |
|
||||
| `markitect/assets/` (content-addressable asset store, dedup, `.mdpkg` ZIP packaging, symlink handling) + `asset_registry.json` (277 KB) | ~6,000 | **No.** Bench says "leave behind unless a concrete export needs assets." |
|
||||
| Interactive-HTML / testdrive-jsui rendering, `static/`, `themes/`, `templates/document.html`, JS UI | — | **Partial only.** quarkdown covers a *Quarkdown* export path; the interactive-HTML / JS-UI path has no home. |
|
||||
|
||||
**Decision needed:** spin these into a dedicated render/asset repo (sibling to
|
||||
quarkdown), fold the asset store into one of the existing repos, or retire the
|
||||
interactive-HTML path.
|
||||
|
||||
## C. The other "Information Space" lineage — `markitect/spaces/` (~11,000 LOC)
|
||||
|
||||
**Distinct from `markitect/infospace/`** (which infospace-bench inherited).
|
||||
`spaces/` is an older/parallel abstraction with features bench did *not* take:
|
||||
|
||||
- event-driven change tracking & notifications
|
||||
- persistent transclusion context with cross-space references
|
||||
- bidirectional directory synchronization
|
||||
- HTML rendering of spaces with caching/themes
|
||||
|
||||
Engine takes generic persistence concepts and bench takes infospace semantics,
|
||||
but **these specific `spaces/` behaviors (bidirectional sync, event
|
||||
notifications, cross-space transclusion context) aren't mapped anywhere.**
|
||||
|
||||
Likely intended as dead/superseded — but 11k LOC warrants an explicit "retire
|
||||
vs salvage" call.
|
||||
|
||||
## D. Declined-by-design (confirm retirement, don't re-extract)
|
||||
|
||||
| Area | LOC | Disposition |
|
||||
|---|---|---|
|
||||
| `markitect/graphql/` | ~4,000 | All three explicitly declined GraphQL ("evidence of API need, not a commitment"). |
|
||||
| `markitect/query_paradigms/` | ~3,500 | Engine/tool keep the *QueryResult envelope* concept but say "do not port the registry wholesale." |
|
||||
| `markitect/proxy/` | ~870 | Non-markdown→md proxy with checksum/freshness tracking. **Overlaps markitect-filter.** Freshness/staleness-tracking mechanism may be worth checking against bench's deferred "stale-mappings." |
|
||||
| `capabilities/` (top-level) | ~8,300 | Capability-packaging architecture; partially maps to tool (schema generation) but the packaging approach itself isn't carried. |
|
||||
|
||||
---
|
||||
|
||||
## What this means
|
||||
|
||||
The successors are, by their own assessments, **near complete for the
|
||||
in-scope core** (parsing/schema → tool; persistence/workflow → engine;
|
||||
infospace lifecycle → bench; ingestion → filter; one render path →
|
||||
quarkdown). The truly unaddressed functionality is almost entirely the stuff
|
||||
markitect-main accreted **beyond** its stated scope: finance, issue tracking,
|
||||
user profiles, production/deployment validation, the asset/plugin/interactive-HTML
|
||||
rendering stack, and the older `spaces/` abstraction.
|
||||
|
||||
## Decisions for Bernd
|
||||
|
||||
Three live decisions, not a long extraction backlog:
|
||||
|
||||
### Decision 1 — Render/asset stack (Section B)
|
||||
The one with genuine product value left.
|
||||
- **Option 1a:** new repo (sibling to quarkdown) for plugin runtime + asset store + interactive-HTML
|
||||
- **Option 1b:** fold the asset store into an existing repo (most likely markitect-tool, behind a flag); retire interactive-HTML
|
||||
- **Option 1c:** retire the interactive-HTML path entirely; trust quarkdown export as the single render story
|
||||
|
||||
### Decision 2 — `markitect/spaces/` (Section C)
|
||||
- **Option 2a:** salvage bidirectional-sync / event-tracking / cross-space transclusion into engine (engine has the persistence story to support it)
|
||||
- **Option 2b:** retire wholesale as superseded by infospace
|
||||
|
||||
### Decision 3 — Project-ops cluster (Section A: finance + issues + profile)
|
||||
- **Option 3a:** confirm `issue-facade` already replaces `issue_tracker/` + `finance/`; retire both
|
||||
- **Option 3b:** identify a home for any pieces worth keeping
|
||||
|
||||
---
|
||||
|
||||
## Suggested verification before deciding
|
||||
|
||||
If verification matters before committing:
|
||||
|
||||
- **For Decision 1:** grep the five repos for any render/asset adapter that already covers the HTML path beyond Quarkdown.
|
||||
- **For Decision 2:** check whether engine's `OperationRun` + collection model can express bidirectional-sync semantics, or whether new primitives would be needed.
|
||||
- **For Decision 3:** confirm whether `issue-facade` truly replaces `issue_tracker/` + `finance/` end-to-end.
|
||||
|
||||
Happy to do any of these focused passes when you're ready to decide.
|
||||
@@ -171,6 +171,57 @@ you need to look at, rather than a bare ratio.
|
||||
|
||||
---
|
||||
|
||||
## 5. Systematic processing of long texts
|
||||
|
||||
For long source material (books, multi-chapter specifications, corpora), the
|
||||
pipeline can produce a clean chapter-by-chapter git history on its own if
|
||||
you let it. The pattern:
|
||||
|
||||
```bash
|
||||
# Process all sources in canonical order, eval and classify per chapter,
|
||||
# snapshot metrics after each chapter.
|
||||
markitect infospace process --all \
|
||||
--provider openrouter \
|
||||
--eval-after-source \
|
||||
--classify-after-source \
|
||||
--check-after-each
|
||||
```
|
||||
|
||||
What you get:
|
||||
|
||||
- **One commit per source file**, not per batch run. The commit message body
|
||||
lists counts by bucket (`entities: +23`, `evaluations: +23`,
|
||||
`classifications: +23`) derived from the actual staged diff, so `git log`
|
||||
reads like the story of the infospace growing.
|
||||
- **Chapter-atomic commits.** `--eval-after-source` and
|
||||
`--classify-after-source` evaluate and classify *only the new entities*
|
||||
from the just-processed source before the commit lands, so each commit is
|
||||
a self-contained chapter snapshot.
|
||||
- **Metrics-per-chapter trail.** `--check-after-each` appends a snapshot to
|
||||
`output/metrics/history.yaml` after every chapter, so `markitect infospace
|
||||
history` later shows the metric trajectory rather than just start/end.
|
||||
|
||||
**Cost tradeoff.** `--eval-after-source` pays LLM latency per chapter rather
|
||||
than amortising it across one bulk batch. It's worth it when you care about
|
||||
the git history or want early quality signal, not when you're bulk-backfilling
|
||||
a known-good corpus.
|
||||
|
||||
**Triage during the run.** While processing, use `markitect infospace
|
||||
chapters` in another shell to see per-source entity/eval/classify counts and
|
||||
mean scores — handy for spotting chapters that under-extracted or evaluated
|
||||
poorly.
|
||||
|
||||
```
|
||||
$ markitect infospace chapters
|
||||
source entities evaluated classified mean_score
|
||||
------------------- -------- --------- ---------- ----------
|
||||
book-1-chapter-01 96 96 79 4.22
|
||||
book-1-chapter-02 16 16 10 4.06
|
||||
…
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- `METRICS-METHODOLOGY.md` — how each metric is computed.
|
||||
|
||||
@@ -240,8 +240,14 @@ def llm_catalog(output_format):
|
||||
)
|
||||
def llm_check(provider, model):
|
||||
"""Send a minimal prompt to verify a provider is reachable and responding."""
|
||||
import os
|
||||
|
||||
from markitect.llm import create_adapter
|
||||
from markitect.llm.exceptions import LLMConfigurationError, LLMError
|
||||
from markitect.llm.exceptions import (
|
||||
LLMAPIError,
|
||||
LLMConfigurationError,
|
||||
LLMError,
|
||||
)
|
||||
from markitect.prompts.execution.models import RunConfig
|
||||
|
||||
resolved = resolve_llm(cli_provider=provider, cli_model=model)
|
||||
@@ -252,6 +258,17 @@ def llm_check(provider, model):
|
||||
f" model from: {resolved.model_source}"
|
||||
)
|
||||
|
||||
# Advisory: OPENROUTER_API_KEY is set but this call won't use it. Common
|
||||
# source of "works for me, fails for agents" when the env var holds a
|
||||
# stale key that overrides a clean config entry.
|
||||
if resolved.provider != "openrouter" and os.environ.get("OPENROUTER_API_KEY"):
|
||||
click.echo(
|
||||
" note: OPENROUTER_API_KEY is set but won't be used for this "
|
||||
"provider. If OpenRouter calls fail elsewhere with 401, the env "
|
||||
"var may be stale — unset or update it.",
|
||||
err=True,
|
||||
)
|
||||
|
||||
try:
|
||||
adapter = create_adapter(
|
||||
provider=resolved.provider,
|
||||
@@ -273,6 +290,19 @@ def llm_check(provider, model):
|
||||
except LLMError as exc:
|
||||
elapsed = time.monotonic() - start
|
||||
click.echo(f"ERROR \u2014 LLM error after {elapsed:.1f}s: {exc}", err=True)
|
||||
# Targeted hint: 401 on openrouter almost always means a stale key.
|
||||
if (
|
||||
resolved.provider == "openrouter"
|
||||
and isinstance(exc, LLMAPIError)
|
||||
and exc.status_code == 401
|
||||
):
|
||||
click.echo(
|
||||
" hint: OpenRouter returned 401 (unauthorized). Check whether "
|
||||
"OPENROUTER_API_KEY is stale (`unset OPENROUTER_API_KEY` to "
|
||||
"fall back to the key in ~/.config/markitect/config.toml, or "
|
||||
"update the env var).",
|
||||
err=True,
|
||||
)
|
||||
sys.exit(1)
|
||||
except Exception as exc:
|
||||
elapsed = time.monotonic() - start
|
||||
|
||||
@@ -7,8 +7,9 @@ inspecting, and evaluating infospaces.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from typing import Dict, Optional
|
||||
|
||||
import click
|
||||
|
||||
@@ -228,6 +229,227 @@ def _entities_by_type(cfg, root: "Path", entity_list: list) -> None:
|
||||
click.echo(f"\nTotal: {total} entities")
|
||||
|
||||
|
||||
# ── chapters (per-source triage view) ────────────────────────────────
|
||||
|
||||
|
||||
@infospace_commands.command()
|
||||
@click.option("--config", "config_path", default=None, help="Path to infospace.yaml.")
|
||||
@click.option(
|
||||
"--format", "output_format",
|
||||
type=click.Choice(["text", "json"]),
|
||||
default="text",
|
||||
help="Output format.",
|
||||
)
|
||||
def chapters(config_path: Optional[str], output_format: str):
|
||||
"""List source files in canonical order with per-source stats.
|
||||
|
||||
For each source file in the sources directory, reports entity count,
|
||||
mean per-entity score (if evaluated), classification coverage, and
|
||||
processing status. Useful for triaging long-text infospaces.
|
||||
"""
|
||||
cfg, cfg_path = _load_config_or_exit(config_path)
|
||||
root = cfg_path.parent
|
||||
|
||||
sources_dir = root / cfg.topic.sources if cfg.topic.sources else root
|
||||
if not sources_dir.is_dir():
|
||||
click.echo(f"No sources directory at {sources_dir}.", err=True)
|
||||
raise SystemExit(1)
|
||||
|
||||
source_files = sorted(sources_dir.glob("*.md"))
|
||||
if not source_files:
|
||||
click.echo(f"No source files in {sources_dir}.", err=True)
|
||||
raise SystemExit(1)
|
||||
|
||||
entities_dir = root / cfg.entities_dir
|
||||
entity_list = (
|
||||
parse_entity_directory(entities_dir) if entities_dir.is_dir() else []
|
||||
)
|
||||
|
||||
# Build a source_id → [entities] map using the source_chapter field.
|
||||
# Matching is lenient: entities with a source_chapter substring-equal
|
||||
# to a normalized form of the source stem count as belonging to it.
|
||||
def _chapter_keys(source_id: str) -> list:
|
||||
"""Return strings an entity's source_chapter might contain."""
|
||||
keys = [source_id, source_id.replace("-", " ")]
|
||||
m = re.match(r"book-(\d+)-chapter-(\d+)", source_id)
|
||||
if m:
|
||||
book, chap = m.group(1), m.group(2)
|
||||
roman = {"1": "I", "2": "II", "3": "III", "4": "IV", "5": "V"}
|
||||
if book in roman:
|
||||
keys.append(f"Book {roman[book]}, Chapter {int(chap)}")
|
||||
keys.append(f"Book {roman[book]} Chapter {int(chap)}")
|
||||
return keys
|
||||
|
||||
# Precompute evaluation scores and classification slugs once.
|
||||
evals_dir = root / cfg.evaluations_dir
|
||||
cls_dir = root / cfg.classifications_dir
|
||||
eval_scores: Dict[str, float] = {}
|
||||
if evals_dir.is_dir():
|
||||
from markitect.infospace.evaluation_io import read_entity_evaluation
|
||||
for ev_path in evals_dir.glob("*.md"):
|
||||
try:
|
||||
ev = read_entity_evaluation(ev_path)
|
||||
if ev.overall_score is not None:
|
||||
eval_scores[ev_path.stem] = ev.overall_score
|
||||
except Exception:
|
||||
continue
|
||||
classified_slugs = (
|
||||
{p.stem for p in cls_dir.glob("*.md")} if cls_dir.is_dir() else set()
|
||||
)
|
||||
|
||||
rows = []
|
||||
for source_file in source_files:
|
||||
source_id = source_file.stem
|
||||
keys = _chapter_keys(source_id)
|
||||
matched = [
|
||||
e for e in entity_list
|
||||
if any(k.lower() in (e.source_chapter or "").lower() for k in keys)
|
||||
]
|
||||
slugs = {e.slug for e in matched}
|
||||
evaluated = slugs & set(eval_scores)
|
||||
classified = slugs & classified_slugs
|
||||
mean = (
|
||||
sum(eval_scores[s] for s in evaluated) / len(evaluated)
|
||||
if evaluated else None
|
||||
)
|
||||
rows.append({
|
||||
"source_id": source_id,
|
||||
"entities": len(matched),
|
||||
"evaluated": len(evaluated),
|
||||
"classified": len(classified),
|
||||
"mean_score": round(mean, 2) if mean is not None else None,
|
||||
})
|
||||
|
||||
if output_format == "json":
|
||||
import json
|
||||
click.echo(json.dumps(rows, indent=2))
|
||||
return
|
||||
|
||||
# Text: aligned table.
|
||||
headers = ("source", "entities", "evaluated", "classified", "mean_score")
|
||||
widths = [
|
||||
max(len(h), max((len(str(r[h.replace(' ', '_')])) if h != "source"
|
||||
else len(r["source_id"]))
|
||||
for r in rows)) if rows else len(h)
|
||||
for h in headers
|
||||
]
|
||||
fmt = " ".join(f"{{:<{w}}}" for w in widths)
|
||||
click.echo(fmt.format(*headers))
|
||||
click.echo(fmt.format(*("-" * w for w in widths)))
|
||||
for r in rows:
|
||||
click.echo(fmt.format(
|
||||
r["source_id"],
|
||||
r["entities"],
|
||||
r["evaluated"],
|
||||
r["classified"],
|
||||
"-" if r["mean_score"] is None else f"{r['mean_score']:.2f}",
|
||||
))
|
||||
totals = {
|
||||
"entities": sum(r["entities"] for r in rows),
|
||||
"evaluated": sum(r["evaluated"] for r in rows),
|
||||
"classified": sum(r["classified"] for r in rows),
|
||||
}
|
||||
click.echo(
|
||||
f"\n{len(rows)} source file(s); "
|
||||
f"{totals['entities']} entities, "
|
||||
f"{totals['evaluated']} evaluated, "
|
||||
f"{totals['classified']} classified."
|
||||
)
|
||||
|
||||
|
||||
# ── entity (single lookup) ───────────────────────────────────────────
|
||||
|
||||
|
||||
@infospace_commands.command()
|
||||
@click.argument("name")
|
||||
@click.option("--config", "config_path", default=None, help="Path to infospace.yaml.")
|
||||
def entity(name: str, config_path: Optional[str]):
|
||||
"""Look up one entity by name, tolerating case / hyphens / underscores.
|
||||
|
||||
Prints slug, source path, domain, chapter, word count, overall score,
|
||||
VSM system (if classified), and evaluation-file path.
|
||||
"""
|
||||
cfg, cfg_path = _load_config_or_exit(config_path)
|
||||
root = cfg_path.parent
|
||||
entities_dir = root / cfg.entities_dir
|
||||
|
||||
if not entities_dir.is_dir():
|
||||
click.echo("No entities directory found.", err=True)
|
||||
raise SystemExit(1)
|
||||
|
||||
entity_list = parse_entity_directory(entities_dir)
|
||||
if not entity_list:
|
||||
click.echo("No entities found.", err=True)
|
||||
raise SystemExit(1)
|
||||
|
||||
# Normalize: lowercase, underscores.
|
||||
def norm(s: str) -> str:
|
||||
return s.lower().replace("-", "_").replace(" ", "_")
|
||||
|
||||
target = norm(name)
|
||||
by_slug = {e.slug: e for e in entity_list}
|
||||
|
||||
match = by_slug.get(target)
|
||||
if match is None:
|
||||
# Substring fallback for partial input.
|
||||
candidates = [e for e in entity_list if target in norm(e.slug)]
|
||||
if len(candidates) == 1:
|
||||
match = candidates[0]
|
||||
elif len(candidates) > 1:
|
||||
click.echo(f"Ambiguous — '{name}' matches multiple entities:", err=True)
|
||||
for c in sorted(candidates, key=lambda e: e.slug)[:10]:
|
||||
click.echo(f" {c.slug}", err=True)
|
||||
if len(candidates) > 10:
|
||||
click.echo(f" … and {len(candidates) - 10} more", err=True)
|
||||
raise SystemExit(1)
|
||||
else:
|
||||
click.echo(f"No entity matching '{name}'.", err=True)
|
||||
near = sorted(
|
||||
e.slug for e in entity_list
|
||||
if target.split("_", 1)[0] in e.slug
|
||||
)[:5]
|
||||
if near:
|
||||
click.echo(f" Near matches: {', '.join(near)}", err=True)
|
||||
raise SystemExit(1)
|
||||
|
||||
# Load score + classification (best-effort).
|
||||
score: Optional[float] = None
|
||||
evaluator: Optional[str] = None
|
||||
eval_file = root / cfg.evaluations_dir / f"{match.slug}.md"
|
||||
if eval_file.is_file():
|
||||
try:
|
||||
from markitect.infospace.evaluation_io import read_entity_evaluation
|
||||
ev = read_entity_evaluation(eval_file)
|
||||
score = ev.overall_score
|
||||
evaluator = ev.evaluator
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
vsm: Optional[str] = None
|
||||
cls_file = root / cfg.classifications_dir / f"{match.slug}.md"
|
||||
if cls_file.is_file():
|
||||
try:
|
||||
from markitect.infospace.classification_io import read_entity_classification
|
||||
cls = read_entity_classification(cls_file)
|
||||
vsm = cls.vsm_system
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Output — one field per line so it's easy to grep or pipe.
|
||||
click.echo(f"slug: {match.slug}")
|
||||
click.echo(f"source_path: {match.source_path}")
|
||||
click.echo(f"domain: {match.domain or '-'}")
|
||||
click.echo(f"chapter: {match.source_chapter or '-'}")
|
||||
click.echo(f"word_count: {match.total_word_count}")
|
||||
click.echo(f"vsm_system: {vsm or '-'}")
|
||||
if score is not None:
|
||||
click.echo(f"overall_score: {score:.2f}")
|
||||
click.echo(f"evaluator: {evaluator or '-'}")
|
||||
click.echo(f"evaluation: {eval_file}")
|
||||
else:
|
||||
click.echo("evaluation: (not yet evaluated)")
|
||||
|
||||
|
||||
# ── evaluate ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@@ -239,7 +461,12 @@ def _entities_by_type(cfg, root: "Path", entity_list: list) -> None:
|
||||
@click.option("--chapter", default=None, help="Evaluate entities from a specific chapter.")
|
||||
@click.option("--force", is_flag=True, default=False,
|
||||
help="Re-evaluate entities whose evaluation file already exists.")
|
||||
def evaluate(config_path, provider, model, entity_slug, chapter, force):
|
||||
@click.option("--model-fallback", "model_fallback", default=None,
|
||||
help="If the primary model hits a rate limit (429), retry the "
|
||||
"failed entities once with this model. Useful on free tiers "
|
||||
"where models have separate quota buckets (e.g. "
|
||||
"gemini-2.5-flash → gemini-2.5-flash-lite).")
|
||||
def evaluate(config_path, provider, model, entity_slug, chapter, force, model_fallback):
|
||||
"""Evaluate entities using LLM-based quality assessment."""
|
||||
cfg, cfg_path = _load_config_or_exit(config_path)
|
||||
root = cfg_path.parent
|
||||
@@ -319,6 +546,42 @@ def evaluate(config_path, provider, model, entity_slug, chapter, force):
|
||||
progress_callback=on_progress,
|
||||
)
|
||||
|
||||
# Model fallback: if any entities failed with a rate-limit-looking
|
||||
# error and the user opted in with --model-fallback, retry them once
|
||||
# with a fresh adapter on the fallback model. Different free-tier
|
||||
# models have separate quota buckets, so this often succeeds when
|
||||
# the primary is exhausted.
|
||||
if model_fallback and summary.failed > 0:
|
||||
rate_limited = [
|
||||
r for r in summary.results
|
||||
if r.status == "error"
|
||||
and r.error
|
||||
and ("429" in r.error or "rate" in r.error.lower())
|
||||
]
|
||||
if rate_limited:
|
||||
retry_slugs = {r.key for r in rate_limited}
|
||||
retry_entities = [e for e in entity_list if e.slug in retry_slugs]
|
||||
click.echo(
|
||||
f"\n{len(retry_entities)} rate-limited entities — "
|
||||
f"retrying with --model-fallback {model_fallback}..."
|
||||
)
|
||||
fb_adapter = create_adapter(provider, model=model_fallback)
|
||||
fb_run_config = RunConfig(
|
||||
model_name=model_fallback, temperature=0.3, max_tokens=2000
|
||||
)
|
||||
fb_summary = run_entity_evaluation(
|
||||
config=cfg,
|
||||
entities=retry_entities,
|
||||
adapter=fb_adapter,
|
||||
run_config=fb_run_config,
|
||||
output_dir=output_dir,
|
||||
progress_callback=on_progress,
|
||||
)
|
||||
summary.succeeded += fb_summary.succeeded
|
||||
summary.failed = (summary.failed - len(retry_entities)) + fb_summary.failed
|
||||
summary.total_prompt_tokens += fb_summary.total_prompt_tokens
|
||||
summary.total_completion_tokens += fb_summary.total_completion_tokens
|
||||
|
||||
click.echo(f"\nDone: {summary.succeeded} succeeded, {summary.failed} failed, {summary.skipped} skipped")
|
||||
if summary.total_tokens > 0:
|
||||
click.echo(f"Tokens used: {summary.total_tokens}")
|
||||
@@ -1033,6 +1296,18 @@ def disciplines(config_path: Optional[str]):
|
||||
help="Run collection checks (C1–C5) after each source file.",
|
||||
)
|
||||
@click.option("--no-commit", is_flag=True, help="Skip git commits.")
|
||||
@click.option(
|
||||
"--eval-after-source",
|
||||
is_flag=True,
|
||||
help="After each source's stages succeed, evaluate just the newly-"
|
||||
"added entities so the per-source commit is self-contained.",
|
||||
)
|
||||
@click.option(
|
||||
"--classify-after-source",
|
||||
is_flag=True,
|
||||
help="After each source's stages succeed, classify just the newly-"
|
||||
"added entities so the per-source commit is self-contained.",
|
||||
)
|
||||
def process(
|
||||
glob_pattern: Optional[str],
|
||||
process_all: bool,
|
||||
@@ -1041,6 +1316,8 @@ def process(
|
||||
model: Optional[str],
|
||||
check_after_each: bool,
|
||||
no_commit: bool,
|
||||
eval_after_source: bool,
|
||||
classify_after_source: bool,
|
||||
):
|
||||
"""Process source files through the pipeline defined in infospace.yaml.
|
||||
|
||||
@@ -1114,12 +1391,22 @@ def process(
|
||||
# Run pipeline
|
||||
from markitect.infospace.pipeline import SourcePipeline
|
||||
|
||||
if (eval_after_source or classify_after_source) and adapter is None:
|
||||
click.echo(
|
||||
"Error: --eval-after-source / --classify-after-source require "
|
||||
"--provider (they call the LLM).",
|
||||
err=True,
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
pipeline = SourcePipeline(
|
||||
cfg, root,
|
||||
adapter=adapter,
|
||||
provider=provider or "",
|
||||
model=(model or _PROVIDER_DEFAULTS.get(provider or "", "")) if provider else "",
|
||||
no_commit=no_commit,
|
||||
eval_after_source=eval_after_source,
|
||||
classify_after_source=classify_after_source,
|
||||
)
|
||||
|
||||
total = len(source_files)
|
||||
|
||||
@@ -62,6 +62,8 @@ class SourcePipeline:
|
||||
provider: str = "",
|
||||
model: str = "",
|
||||
no_commit: bool = False,
|
||||
eval_after_source: bool = False,
|
||||
classify_after_source: bool = False,
|
||||
) -> None:
|
||||
self.config = config
|
||||
self.root = root
|
||||
@@ -69,6 +71,8 @@ class SourcePipeline:
|
||||
self.provider = provider
|
||||
self.model = model
|
||||
self.no_commit = no_commit
|
||||
self.eval_after_source = eval_after_source
|
||||
self.classify_after_source = classify_after_source
|
||||
|
||||
# ── Public API ────────────────────────────────────────────────────
|
||||
|
||||
@@ -110,6 +114,12 @@ class SourcePipeline:
|
||||
stage_outputs: Dict[str, str] = {}
|
||||
stage_logs: List[Dict[str, Any]] = []
|
||||
|
||||
# Snapshot entity slugs before any stage runs so we can identify
|
||||
# which entities were newly produced by this source. Used to scope
|
||||
# --eval-after-source / --classify-after-source to only the new
|
||||
# entities.
|
||||
pre_entity_slugs = self._current_entity_slugs()
|
||||
|
||||
print(f"\nProcessing: {source_id}")
|
||||
print("=" * 60)
|
||||
|
||||
@@ -133,6 +143,14 @@ class SourcePipeline:
|
||||
|
||||
print(f"\n {source_id}: all stages complete.")
|
||||
self._write_processing_log(source_id, stage_logs, success=True)
|
||||
|
||||
# Per-source follow-ups: evaluate and/or classify just the new
|
||||
# entities this source produced, so the next commit contains a
|
||||
# fully-processed chapter.
|
||||
new_slugs = self._current_entity_slugs() - pre_entity_slugs
|
||||
if new_slugs and (self.eval_after_source or self.classify_after_source):
|
||||
self._run_per_source_followups(new_slugs)
|
||||
|
||||
if not self.no_commit:
|
||||
self._git_commit(source_id)
|
||||
|
||||
@@ -636,7 +654,13 @@ class SourcePipeline:
|
||||
# ── Git Integration ───────────────────────────────────────────────
|
||||
|
||||
def _git_commit(self, source_id: str) -> None:
|
||||
"""Stage all output changes and commit them for *source_id*."""
|
||||
"""Stage all output changes and commit them for *source_id*.
|
||||
|
||||
The commit message body summarises what actually changed — counts
|
||||
of entities / evaluations / classifications / analyses added — so
|
||||
``git log`` reads like the chapter-by-chapter story of the
|
||||
infospace growing, not a wall of identical messages.
|
||||
"""
|
||||
output_dir = self.root / "output"
|
||||
try:
|
||||
subprocess.run(
|
||||
@@ -645,11 +669,11 @@ class SourcePipeline:
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
body = self._compose_commit_body(source_id)
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git", "commit", "-m",
|
||||
f"infospace: process {source_id}\n\n"
|
||||
f"Extract entities, map to VSM, and synthesize analysis.",
|
||||
f"infospace: process {source_id}\n\n{body}",
|
||||
],
|
||||
cwd=str(self.root),
|
||||
capture_output=True,
|
||||
@@ -666,3 +690,146 @@ class SourcePipeline:
|
||||
except subprocess.CalledProcessError as e:
|
||||
stderr = e.stderr.decode() if isinstance(e.stderr, bytes) else (e.stderr or "")
|
||||
print(f" Warning: Git error: {stderr.strip()}")
|
||||
|
||||
# ── Per-source helpers ────────────────────────────────────────────
|
||||
|
||||
def _current_entity_slugs(self) -> set:
|
||||
"""Return the set of entity file stems currently on disk."""
|
||||
entities_dir = self.root / self.config.entities_dir
|
||||
if not entities_dir.is_dir():
|
||||
return set()
|
||||
return {p.stem for p in entities_dir.glob("*.md")}
|
||||
|
||||
def _run_per_source_followups(self, new_slugs: set) -> None:
|
||||
"""Run per-source evaluation and/or classification on *new_slugs*.
|
||||
|
||||
Called after a source's pipeline stages succeed, before the git
|
||||
commit, so each chapter's commit contains the full set of
|
||||
artefacts derived from it.
|
||||
"""
|
||||
from markitect.infospace.entity_parser import parse_entity_directory
|
||||
|
||||
entities_dir = self.root / self.config.entities_dir
|
||||
all_entities = parse_entity_directory(entities_dir)
|
||||
new_entities = [e for e in all_entities if e.slug in new_slugs]
|
||||
if not new_entities:
|
||||
return
|
||||
|
||||
if self.adapter is None:
|
||||
print(
|
||||
" Skipping per-source eval/classify: no LLM adapter "
|
||||
"configured (run with --provider)."
|
||||
)
|
||||
return
|
||||
|
||||
from markitect.prompts.execution.models import RunConfig
|
||||
|
||||
run_config = RunConfig(
|
||||
model_name=self.model or None, temperature=0.3, max_tokens=2000
|
||||
)
|
||||
|
||||
if self.eval_after_source:
|
||||
from markitect.infospace.evaluate import run_entity_evaluation
|
||||
|
||||
print(f" Evaluating {len(new_entities)} new entity/entities…")
|
||||
try:
|
||||
run_entity_evaluation(
|
||||
config=self.config,
|
||||
entities=new_entities,
|
||||
adapter=self.adapter,
|
||||
run_config=run_config,
|
||||
output_dir=self.root / self.config.evaluations_dir,
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f" Warning: per-source evaluation failed: {exc}")
|
||||
|
||||
if self.classify_after_source:
|
||||
from markitect.infospace.classifier import run_entity_classification
|
||||
|
||||
print(f" Classifying {len(new_entities)} new entity/entities…")
|
||||
try:
|
||||
run_entity_classification(
|
||||
config=self.config,
|
||||
entities=new_entities,
|
||||
adapter=self.adapter,
|
||||
run_config=run_config,
|
||||
output_dir=self.root / self.config.classifications_dir,
|
||||
)
|
||||
except Exception as exc:
|
||||
print(f" Warning: per-source classification failed: {exc}")
|
||||
|
||||
def _compose_commit_body(self, source_id: str) -> str:
|
||||
"""Summarise staged output changes into a commit-message body.
|
||||
|
||||
Counts added files per output subdirectory (entities, evaluations,
|
||||
classifications, analyses, mappings…) and produces one line per
|
||||
bucket that actually saw additions. Modified/deleted files are
|
||||
noted separately for auditability.
|
||||
"""
|
||||
default = "Extract entities, map to VSM, and synthesize analysis."
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", "diff", "--cached", "--name-status", "--", "output"],
|
||||
cwd=str(self.root),
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
except subprocess.CalledProcessError:
|
||||
return default
|
||||
|
||||
added_by_bucket: Dict[str, int] = {}
|
||||
modified = 0
|
||||
deleted = 0
|
||||
for line in result.stdout.splitlines():
|
||||
parts = line.split("\t")
|
||||
if len(parts) < 2:
|
||||
continue
|
||||
status = parts[0]
|
||||
path = parts[-1]
|
||||
if status.startswith("A"):
|
||||
bucket = self._bucket_for(path)
|
||||
if bucket:
|
||||
added_by_bucket[bucket] = added_by_bucket.get(bucket, 0) + 1
|
||||
elif status.startswith("M"):
|
||||
modified += 1
|
||||
elif status.startswith("D"):
|
||||
deleted += 1
|
||||
|
||||
if not added_by_bucket and not modified and not deleted:
|
||||
return default
|
||||
|
||||
# Emit buckets in a deterministic, reader-friendly order.
|
||||
order = ["entities", "mappings", "analyses", "evaluations",
|
||||
"classifications", "metrics", "logs", "other"]
|
||||
lines: List[str] = []
|
||||
for bucket in order:
|
||||
n = added_by_bucket.get(bucket, 0)
|
||||
if n:
|
||||
lines.append(f"- {bucket}: +{n}")
|
||||
if modified:
|
||||
lines.append(f"- modified: {modified}")
|
||||
if deleted:
|
||||
lines.append(f"- deleted: {deleted}")
|
||||
return "\n".join(lines) if lines else default
|
||||
|
||||
def _bucket_for(self, path: str) -> Optional[str]:
|
||||
"""Map an ``output/...`` path to a commit-summary bucket name."""
|
||||
# Use configured directory basenames where possible so non-default
|
||||
# layouts still bucket correctly.
|
||||
buckets = {
|
||||
Path(self.config.entities_dir).name: "entities",
|
||||
Path(self.config.evaluations_dir).name: "evaluations",
|
||||
Path(self.config.classifications_dir).name: "classifications",
|
||||
}
|
||||
parts = Path(path).parts
|
||||
if len(parts) < 2 or parts[0] != "output":
|
||||
return None
|
||||
sub = parts[1]
|
||||
if sub in buckets:
|
||||
return buckets[sub]
|
||||
# Heuristic fallback for common additional output subdirectories.
|
||||
known = {"mappings", "analyses", "metrics", "logs"}
|
||||
if sub in known:
|
||||
return sub
|
||||
return "other"
|
||||
|
||||
@@ -131,6 +131,12 @@ def build_state(
|
||||
This is a convenience function that assembles the state object
|
||||
and optionally runs viability checks if *metrics* are provided.
|
||||
"""
|
||||
if not isinstance(config, InfospaceConfig):
|
||||
raise TypeError(
|
||||
f"build_state(config=...) expects an InfospaceConfig instance, "
|
||||
f"got {type(config).__name__}. If you have a path, load the "
|
||||
f"config first with load_infospace_config(path)."
|
||||
)
|
||||
state = InfospaceState(
|
||||
config=config,
|
||||
entities=entities or [],
|
||||
|
||||
12
registry/README.md
Normal file
12
registry/README.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Capability Registry
|
||||
|
||||
Markdown-first capability index for federation and reuse planning.
|
||||
|
||||
## Authoring
|
||||
|
||||
1. Copy a capability entry template (see reuse-surface `templates/capability-entry.template.md`).
|
||||
2. Add the row to `indexes/capabilities.yaml`.
|
||||
3. Run `reuse-surface validate` from a checkout with the CLI installed.
|
||||
4. Merge to `main` and verify publish with `reuse-surface establish --publish-check`.
|
||||
|
||||
Federation contract: reuse-surface `docs/RegistryFederation.md`.
|
||||
0
registry/capabilities/.gitkeep
Normal file
0
registry/capabilities/.gitkeep
Normal file
4
registry/indexes/capabilities.yaml
Normal file
4
registry/indexes/capabilities.yaml
Normal file
@@ -0,0 +1,4 @@
|
||||
version: 1
|
||||
updated: '2026-06-16'
|
||||
domain: helix_forge
|
||||
capabilities: []
|
||||
@@ -10,6 +10,14 @@ and formally closes the roadmap.
|
||||
**Parent roadmap:** `roadmap/infospace-tooling/PLAN.md`
|
||||
**Example location:** `examples/infospace-with-history/`
|
||||
|
||||
**Status: CLOSED (2026-04-22).** All acceptance criteria except the cosmetic
|
||||
per-chapter history (C.7) are met. Final metrics: 988 entities, 988 evaluations,
|
||||
6/6 viability thresholds PASS (`per_entity_mean = 3.957`). Tooling work that
|
||||
came out of this close-out landed as commits `c0615c2d` (gemini retry,
|
||||
unified skip-existing, non-destructive metrics I/O) and `d44a4cd3`
|
||||
(`infospace entity` lookup, `evaluate --model-fallback`, `llm-check`
|
||||
stale-key advisory, `build_state` type guard).
|
||||
|
||||
### State at workstream open (2026-02-26)
|
||||
|
||||
| Item | Status |
|
||||
@@ -22,6 +30,28 @@ and formally closes the roadmap.
|
||||
| 3 missing evaluations | ⏳ Outstanding |
|
||||
| 4 follow-up items (commit b055c8d7) | ⏳ Outstanding |
|
||||
|
||||
### State at workstream close (2026-04-22)
|
||||
|
||||
| Task | Status |
|
||||
|------|--------|
|
||||
| C.1 Complete 3 missing entity evaluations | ✅ Done (commit f325f89d) |
|
||||
| C.2 Run eval-summary and verify viability | ✅ Done — 6/6 PASS |
|
||||
| C.3 Refresh metrics report (988 entities) | ✅ Done — snapshot `090bb961` |
|
||||
| C.4 Document advanced usage patterns | ✅ Done — `examples/infospace-with-history/docs/advanced-usage.md` |
|
||||
| C.5 Composition-examples documentation | ✅ Done — `docs/composition-guide.md` |
|
||||
| C.6 Performance benchmarking note | ✅ Done — `examples/infospace-with-history/docs/performance-notes.md` |
|
||||
| C.7 Clean per-chapter git history | ⏭️ Deferred indefinitely — see note below |
|
||||
| C.8 Formally close S3 roadmap | ✅ This commit |
|
||||
|
||||
**C.7 disposition.** The task assumed a pre-existing `clean-example-history`
|
||||
branch with chapters 1–8 already committed; that branch no longer exists in
|
||||
the repo. The task is explicitly cosmetic ("does not change output files"),
|
||||
and the output files themselves are canonical. Reconstructing a 35-commit
|
||||
per-chapter history from scratch would be archaeological rather than useful.
|
||||
Closing as "won't do" unless a specific archival need surfaces. If revisited,
|
||||
entities can be grouped by their `## Source Chapter` markdown section to
|
||||
reconstruct chapter membership.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
@@ -1,5 +1,31 @@
|
||||
# Viable Infospace Tooling — Roadmap
|
||||
|
||||
## Status: CLOSED (2026-04-22)
|
||||
|
||||
All three stages complete.
|
||||
|
||||
| Stage | Status | Notes |
|
||||
|-------|--------|-------|
|
||||
| Stage 1 — Platform additions (S1.1–S1.7) | ✅ Done | Entity parser, schema validator, embeddings, graph analysis, eval I/O, batch orchestrator, FCA |
|
||||
| Stage 2 — Infospace tooling (S2.1–S2.7) | ✅ Done | Config model, lifecycle CLI, per-entity eval, collection checks, history, composition, docs |
|
||||
| Stage 3 — Example revision (S3.1–S3.5) | ✅ Done (except cosmetic S3.2) | See `roadmap/infospace-s3-closeout/PLAN.md` |
|
||||
|
||||
**Final validation (Wealth of Nations / VSM example, 988 entities):**
|
||||
- 988 per-entity evaluations landed
|
||||
- Collection checks pass 6/6 viability thresholds (`per_entity_mean = 3.957`
|
||||
against threshold 3.5; `redundancy_ratio = 0.006`; `coverage_ratio = 0.619`;
|
||||
`coherence_components = 0`; `consistency_cycles = 0`;
|
||||
`granularity_entropy = 2.675`)
|
||||
- Composition demonstrated via `examples/supply-chain-vsm/`
|
||||
- S3.2 (clean per-chapter git history) deferred as cosmetic-only; rationale
|
||||
in the close-out plan
|
||||
|
||||
See `roadmap/infospace-s3-closeout/PLAN.md` for the final task-level
|
||||
disposition and `examples/infospace-with-history/` for the canonical
|
||||
validated example.
|
||||
|
||||
---
|
||||
|
||||
## Vision
|
||||
|
||||
An **infospace** is a structured, evaluable, composable collection of
|
||||
|
||||
@@ -223,3 +223,129 @@ class TestViabilityCommand:
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "No viability thresholds" in result.output
|
||||
|
||||
|
||||
# ── chapters (per-source triage view) ────────────────────────────────
|
||||
|
||||
|
||||
class TestChaptersCommand:
|
||||
@pytest.fixture
|
||||
def chapters_dir(self, tmp_path):
|
||||
"""Infospace with 2 source files and matching entities."""
|
||||
config_yaml = """\
|
||||
topic:
|
||||
name: "WoN"
|
||||
domain: "Economics"
|
||||
sources: artifacts/sources
|
||||
"""
|
||||
(tmp_path / "infospace.yaml").write_text(config_yaml)
|
||||
|
||||
sources = tmp_path / "artifacts" / "sources"
|
||||
sources.mkdir(parents=True)
|
||||
(sources / "book-1-chapter-01.md").write_text("# Chapter 1\n\nText.\n")
|
||||
(sources / "book-1-chapter-02.md").write_text("# Chapter 2\n\nText.\n")
|
||||
|
||||
entities = tmp_path / "output" / "entities"
|
||||
entities.mkdir(parents=True)
|
||||
(entities / "alpha.md").write_text(
|
||||
"# Alpha\n\n## Definition\n\nX.\n\n"
|
||||
"## Source Chapter\n\nBook I, Chapter 1\n"
|
||||
)
|
||||
(entities / "beta.md").write_text(
|
||||
"# Beta\n\n## Definition\n\nY.\n\n"
|
||||
"## Source Chapter\n\nBook I, Chapter 2\n"
|
||||
)
|
||||
(entities / "gamma.md").write_text(
|
||||
"# Gamma\n\n## Definition\n\nZ.\n\n"
|
||||
"## Source Chapter\n\nBook I, Chapter 2\n"
|
||||
)
|
||||
return tmp_path
|
||||
|
||||
def test_lists_sources_with_counts(self, runner, chapters_dir):
|
||||
result = runner.invoke(
|
||||
infospace_commands,
|
||||
["chapters", "--config", str(chapters_dir / "infospace.yaml")],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
assert "book-1-chapter-01" in result.output
|
||||
assert "book-1-chapter-02" in result.output
|
||||
# ch 1 -> 1 entity, ch 2 -> 2 entities
|
||||
assert "2 source file(s); 3 entities" in result.output
|
||||
|
||||
def test_json_format(self, runner, chapters_dir):
|
||||
result = runner.invoke(
|
||||
infospace_commands,
|
||||
["chapters", "--config", str(chapters_dir / "infospace.yaml"),
|
||||
"--format", "json"],
|
||||
)
|
||||
assert result.exit_code == 0
|
||||
import json
|
||||
rows = json.loads(result.output)
|
||||
by_id = {r["source_id"]: r for r in rows}
|
||||
assert by_id["book-1-chapter-01"]["entities"] == 1
|
||||
assert by_id["book-1-chapter-02"]["entities"] == 2
|
||||
|
||||
def test_no_sources_dir(self, runner, tmp_path):
|
||||
(tmp_path / "infospace.yaml").write_text(
|
||||
"topic:\n name: X\n sources: missing\n"
|
||||
)
|
||||
result = runner.invoke(
|
||||
infospace_commands,
|
||||
["chapters", "--config", str(tmp_path / "infospace.yaml")],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
|
||||
|
||||
# ── process: eval-after-source / classify-after-source flags ─────────
|
||||
|
||||
|
||||
class TestProcessAfterSourceFlags:
|
||||
def test_flags_registered_in_help(self, runner):
|
||||
result = runner.invoke(infospace_commands, ["process", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "--eval-after-source" in result.output
|
||||
assert "--classify-after-source" in result.output
|
||||
|
||||
def test_flags_require_provider(self, runner, tmp_path):
|
||||
(tmp_path / "infospace.yaml").write_text(
|
||||
"topic:\n name: X\n sources: sources\n"
|
||||
"pipeline:\n stages:\n - template: extract-entities\n"
|
||||
)
|
||||
sources = tmp_path / "sources"
|
||||
sources.mkdir()
|
||||
(sources / "s1.md").write_text("source")
|
||||
result = runner.invoke(
|
||||
infospace_commands,
|
||||
["process", "--all",
|
||||
"--config", str(tmp_path / "infospace.yaml"),
|
||||
"--eval-after-source"],
|
||||
)
|
||||
assert result.exit_code == 1
|
||||
assert "require --provider" in result.output
|
||||
|
||||
|
||||
# ── pipeline: commit body composition ────────────────────────────────
|
||||
|
||||
|
||||
class TestCommitBodyComposition:
|
||||
def test_bucket_for(self, tmp_path):
|
||||
from markitect.infospace.config import InfospaceConfig, TopicConfig
|
||||
from markitect.infospace.pipeline import SourcePipeline
|
||||
cfg = InfospaceConfig(topic=TopicConfig(name="T", domain="D"))
|
||||
p = SourcePipeline(cfg, tmp_path)
|
||||
assert p._bucket_for("output/entities/x.md") == "entities"
|
||||
assert p._bucket_for("output/evaluations/x.md") == "evaluations"
|
||||
assert p._bucket_for("output/classifications/x.md") == "classifications"
|
||||
assert p._bucket_for("output/mappings/x.md") == "mappings"
|
||||
assert p._bucket_for("output/notes/x.md") == "other"
|
||||
assert p._bucket_for("README.md") is None # not under output/
|
||||
|
||||
def test_compose_body_uses_default_on_no_diff(self, tmp_path):
|
||||
"""When git diff fails or returns empty, fall back to the default blurb."""
|
||||
from markitect.infospace.config import InfospaceConfig, TopicConfig
|
||||
from markitect.infospace.pipeline import SourcePipeline
|
||||
cfg = InfospaceConfig(topic=TopicConfig(name="T", domain="D"))
|
||||
# Not a git repo, so `git diff --cached` will raise CalledProcessError.
|
||||
p = SourcePipeline(cfg, tmp_path)
|
||||
body = p._compose_commit_body("some-source")
|
||||
assert "Extract entities" in body
|
||||
|
||||
67
workplans/MARKITECT-WP-0001-statehub-bootstrap.md
Normal file
67
workplans/MARKITECT-WP-0001-statehub-bootstrap.md
Normal file
@@ -0,0 +1,67 @@
|
||||
---
|
||||
id: MARKITECT-WP-0001
|
||||
type: workplan
|
||||
title: "Bootstrap State Hub integration"
|
||||
domain: communication
|
||||
repo: markitect-main
|
||||
status: finished
|
||||
owner: codex
|
||||
topic_slug: communication
|
||||
created: "2026-06-22"
|
||||
updated: "2026-06-22"
|
||||
state_hub_workstream_id: "dfc40b03-fe8e-49fe-b8d4-86eb1fe26b4a"
|
||||
---
|
||||
|
||||
# Bootstrap State Hub integration
|
||||
|
||||
Knowledge artifact management and markdown engine platform.
|
||||
|
||||
## Review Generated Integration Files
|
||||
|
||||
```task
|
||||
id: MARKITECT-WP-0001-T01
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7455a381-a93d-4220-8f80-3b6ccf953cff"
|
||||
|
||||
```
|
||||
|
||||
Result 2026-06-22: SCOPE.md and INTRODUCTION.md reviewed; AGENTS.md confirmed.
|
||||
|
||||
Review `INTENT.md`, `SCOPE.md`, `AGENTS.md`, and `.custodian-brief.md`.
|
||||
Replace generated placeholders with repo-specific facts where needed.
|
||||
|
||||
## Verify Local Developer Workflow
|
||||
|
||||
```task
|
||||
id: MARKITECT-WP-0001-T02
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "7e34bdab-aa49-49ca-b28a-b254725dd8db"
|
||||
|
||||
```
|
||||
|
||||
Result 2026-06-22: Documented make-based Python/JS workflow.
|
||||
|
||||
Identify the repo's install, test, lint, build, and run commands. Add or refine
|
||||
those commands in the agent instructions so future coding sessions can verify
|
||||
changes confidently.
|
||||
|
||||
## Seed First Real Workplan
|
||||
|
||||
```task
|
||||
id: MARKITECT-WP-0001-T03
|
||||
status: done
|
||||
priority: medium
|
||||
state_hub_task_id: "35a64da7-dda9-4315-901d-88c6827432d9"
|
||||
|
||||
```
|
||||
|
||||
Result 2026-06-22: MARKITECT-WP-0002 already exists (TestDrive npm publication).
|
||||
|
||||
Create the first implementation workplan for the repository's most important
|
||||
next change. After workplan file updates, run from `~/state-hub`:
|
||||
|
||||
```bash
|
||||
make fix-consistency REPO=markitect-main
|
||||
```
|
||||
28
workplans/MARKITECT-WP-0002-testdrive-jsui-publication.md
Normal file
28
workplans/MARKITECT-WP-0002-testdrive-jsui-publication.md
Normal file
@@ -0,0 +1,28 @@
|
||||
---
|
||||
id: MARKITECT-WP-0002
|
||||
type: workplan
|
||||
title: "TestDrive-JSUI — npm Publication"
|
||||
domain: communication
|
||||
repo: markitect-main
|
||||
status: backlog
|
||||
owner: codex
|
||||
topic_slug: communication
|
||||
created: "2026-06-22"
|
||||
updated: "2026-06-22"
|
||||
state_hub_workstream_id: "e203d487-01f1-494a-b14d-a436241a4c01"
|
||||
---
|
||||
|
||||
# TestDrive-JSUI — npm Publication
|
||||
|
||||
Backlog workstream for publishing the TestDrive JSUI package to npm.
|
||||
|
||||
## Publication Readiness
|
||||
|
||||
```task
|
||||
id: MARKITECT-WP-0002-T01
|
||||
status: todo
|
||||
priority: medium
|
||||
state_hub_task_id: "88b3c206-4d45-4bb3-bbb3-47443cdf2123"
|
||||
```
|
||||
|
||||
Define package scope, versioning, and publication checklist for TestDrive-JSUI.
|
||||
Reference in New Issue
Block a user