generated from coulomb/repo-seed
acceptance matrix and workflow generation
This commit is contained in:
@@ -22,9 +22,13 @@ infospace.
|
||||
|
||||
```text
|
||||
artifacts/sources/
|
||||
artifacts/entities/
|
||||
artifacts/relations/
|
||||
artifacts/generated/
|
||||
workflows/templates/
|
||||
output/evaluations/
|
||||
output/metrics/
|
||||
output/workflows/runs/
|
||||
reports/
|
||||
exports/
|
||||
```
|
||||
|
||||
@@ -15,17 +15,17 @@ considered a replacement for each in-scope legacy infospace behavior from
|
||||
| Create an infospace config | File-backed `infospaces/<slug>/infospace.yaml` and manifest | Lifecycle tests, CLI create/inspect docs | `IB-WP-0002` | baseline done |
|
||||
| Load and inspect an infospace | Structured config/artifact model and JSON CLI | Tests for load, missing config, structured errors | `IB-WP-0002` | baseline done |
|
||||
| Add/list artifacts | Manifest-backed artifact registration | Tests for add, duplicate rejection, inspect output | `IB-WP-0002` | baseline done |
|
||||
| Parse entity Markdown | Entity parser built on `markitect-tool` document sections | Entity fixture tests, diagnostics for missing sections | `IB-WP-0006`, `IB-WP-0007` | planned |
|
||||
| Validate entity/relation shape | Contract/schema validation through `markitect-tool` | Valid/invalid fixture tests, diagnostic mapping | `IB-WP-0006`, `IB-WP-0007` | planned |
|
||||
| List entities | Entity listing from parsed artifact model | CLI/API example and fixture tests | `IB-WP-0007` | planned |
|
||||
| Parse and list relations | Relation triplet model and endpoint checks | Relation fixture tests, graph edge tests | `IB-WP-0007` | planned |
|
||||
| Parse entity Markdown | Entity parser built on `markitect-tool` document sections | Entity fixture tests, diagnostics for missing sections | `IB-WP-0006`, `IB-WP-0007` | done |
|
||||
| Validate entity/relation shape | Contract/schema validation through `markitect-tool` | Valid/invalid fixture tests, diagnostic mapping | `IB-WP-0006`, `IB-WP-0007` | baseline done |
|
||||
| List entities | Entity listing from parsed artifact model | CLI/API example and fixture tests | `IB-WP-0007` | done |
|
||||
| Parse and list relations | Relation triplet model and endpoint checks | Relation fixture tests, graph edge tests | `IB-WP-0007` | done |
|
||||
| Export semantic graph | Infospace graph model with Mermaid/DOT output | Graph output tests and pilot report | `IB-WP-0007`, `IB-WP-0008` | partial baseline |
|
||||
| Run collection checks | Methodology-owned metrics in `infospace-bench` | Deterministic metric tests and fixture output | `IB-WP-0008` | partial baseline |
|
||||
| Check viability | Threshold report from metrics | Viability tests and CLI/report output | `IB-WP-0008` | partial baseline |
|
||||
| Write evaluation results | Evaluation files with structured metadata | Round-trip tests and pilot fixture | `IB-WP-0008` | planned |
|
||||
| Maintain metrics history | Snapshot history append/read/diff | History and history-diff tests | `IB-WP-0008` | planned |
|
||||
| Evaluate with LLM assistance | Provider-neutral assisted evaluation workflow | Dry-run plan, adapter contract, audited output | `IB-WP-0009` | planned |
|
||||
| Process source chapters | Explicit infospace workflow stages | Deterministic runner tests, generated artifact provenance | `IB-WP-0009` | planned |
|
||||
| Write evaluation results | Evaluation files with structured metadata | Round-trip tests and pilot fixture | `IB-WP-0008` | done |
|
||||
| Maintain metrics history | Snapshot history append/read/diff | History and history-diff tests | `IB-WP-0008` | done |
|
||||
| Evaluate with LLM assistance | Provider-neutral assisted evaluation workflow | Dry-run plan, adapter contract, audited output | `IB-WP-0009` | boundary done |
|
||||
| Process source chapters | Explicit infospace workflow stages | Deterministic runner tests, generated artifact provenance | `IB-WP-0009` | deterministic baseline done |
|
||||
| Track stale outputs | Digest/provenance comparison | Tests after workflow provenance exists | `IB-WP-0009` | deferred |
|
||||
| Persist durable assets | Optional engine-backed repository adapter | Dry-run sync tests and integration design | `IB-WP-0010` | planned |
|
||||
| Run a legacy-derived pilot | Pruned `infospace-with-history` migration | Pilot corpus, migration report, parity comparison | `IB-WP-0011` | planned |
|
||||
|
||||
84
docs/workflow-generation-pipeline.md
Normal file
84
docs/workflow-generation-pipeline.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# Workflow Generation Pipeline
|
||||
|
||||
`infospace-bench` replaces the old `markitect infospace process` concept with
|
||||
explicit workflow declarations in `infospace.yaml`.
|
||||
|
||||
The boundary is intentionally narrow:
|
||||
|
||||
- `infospace-bench` owns concrete workflow declarations, source artifact
|
||||
selection, generated artifact provenance, run records, and CLI behavior.
|
||||
- `markitect-tool` owns deterministic Markdown rendering and parsing. The
|
||||
workflow layer calls it only through `infospace_bench.markdown_adapter`.
|
||||
- Assisted generation is represented as provider-neutral requests. Actual
|
||||
model/provider calls must arrive through an explicit adapter, not through the
|
||||
workflow definition itself.
|
||||
|
||||
## Declaration Shape
|
||||
|
||||
```yaml
|
||||
workflows:
|
||||
- id: source-summary
|
||||
description: Render deterministic summaries for source artifacts.
|
||||
inputs:
|
||||
source:
|
||||
kind: source
|
||||
static_macros:
|
||||
discipline: Viable System Model
|
||||
stages:
|
||||
- id: render-summary
|
||||
kind: template
|
||||
input: source
|
||||
template: workflows/templates/summary.md
|
||||
output:
|
||||
path: artifacts/generated/{{ input.slug }}-summary.md
|
||||
artifact_id: generated/{{ input.slug }}-summary.md
|
||||
kind: generated
|
||||
title: "{{ input.title }} Summary"
|
||||
expected_evaluations:
|
||||
- metrics
|
||||
```
|
||||
|
||||
Workflow template files use the `markitect-tool` `{{ variable.path }}` template
|
||||
syntax. Current deterministic stage data includes:
|
||||
|
||||
- `input`: selected artifact metadata and content
|
||||
- `macros`: workflow-level plus stage-level static macros
|
||||
- `workflow`: the workflow declaration
|
||||
- `stage`: the current stage declaration
|
||||
- `stages`: previous deterministic stage outputs, keyed by stage ID
|
||||
|
||||
## Deterministic Runs
|
||||
|
||||
`kind: template` stages render Markdown and write generated artifacts. Each
|
||||
generated artifact is registered in `artifacts/index.yaml` with provenance:
|
||||
|
||||
- `workflow_id`
|
||||
- `stage_id`
|
||||
- `input_artifact_id`
|
||||
|
||||
Runs write trace records under:
|
||||
|
||||
```text
|
||||
output/workflows/runs/<run-id>.yaml
|
||||
```
|
||||
|
||||
These records are file-backed evidence for later `kontextual-engine` integration
|
||||
without making this repo a workflow engine.
|
||||
|
||||
## Assisted Boundary
|
||||
|
||||
`kind: assisted` stages render a provider-neutral prompt request during planning.
|
||||
Running an assisted stage requires an explicit `AssistedGenerationAdapter`.
|
||||
Without one, the runner raises `assisted_stage_requires_adapter`, which keeps
|
||||
provider behavior optional, auditable, and outside the application workflow
|
||||
declaration.
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
python3 -m infospace_bench workflow inspect infospaces/bootstrap-pilot
|
||||
python3 -m infospace_bench workflow plan infospaces/bootstrap-pilot source-summary
|
||||
python3 -m infospace_bench workflow run infospaces/bootstrap-pilot source-summary
|
||||
```
|
||||
|
||||
All commands emit JSON for scripted migration and parity checks.
|
||||
Reference in New Issue
Block a user