generated from coulomb/repo-seed
Initial implementation
This commit is contained in:
42
docs/evaluation-and-inspection.md
Normal file
42
docs/evaluation-and-inspection.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Evaluation And Inspection
|
||||
|
||||
`infospace-bench` now has a deterministic baseline for evaluation and
|
||||
inspection. It is intentionally small: the repo can produce structured quality
|
||||
objects and relationship summaries before any LLM or engine integration is
|
||||
introduced.
|
||||
|
||||
## Evaluation Objects
|
||||
|
||||
- `ScoreEntry`
|
||||
- `EntityEvaluation`
|
||||
- `MetricValue`
|
||||
- `EvaluationSnapshot`
|
||||
- `SnapshotDiff`
|
||||
|
||||
Snapshots are serializable through `to_dict()` / `from_dict()` and can be
|
||||
compared with `diff_snapshots()`.
|
||||
|
||||
## Collection Checks
|
||||
|
||||
`run_collection_checks()` produces five baseline metrics:
|
||||
|
||||
- `redundancy_ratio`
|
||||
- `coverage_ratio`
|
||||
- `coherence_components`
|
||||
- `consistency_cycles`
|
||||
- `granularity_entropy`
|
||||
|
||||
These metrics are deliberately deterministic and file-backed. Later work can
|
||||
replace or extend their internals with embeddings, richer graph analysis, or
|
||||
agent-assisted evaluation without changing the result contract.
|
||||
|
||||
## Viability
|
||||
|
||||
`evaluate_viability()` compares metric values against declared
|
||||
`ViabilityThreshold` values. Missing metrics fail visibly.
|
||||
|
||||
## Relationship Inspection
|
||||
|
||||
`relationship_summary()` extracts nodes, edges, and relationship type counts
|
||||
from artifact manifests. `export_mermaid()` provides the first graph-friendly
|
||||
representation.
|
||||
55
docs/infospace-layout.md
Normal file
55
docs/infospace-layout.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Infospace Layout
|
||||
|
||||
An infospace is a file-backed project rooted at:
|
||||
|
||||
```text
|
||||
infospaces/<slug>/
|
||||
```
|
||||
|
||||
## Required Files
|
||||
|
||||
```text
|
||||
infospace.yaml
|
||||
artifacts/index.yaml
|
||||
```
|
||||
|
||||
`infospace.yaml` declares identity, topic, schema references, workflow
|
||||
references, discipline bindings, and viability thresholds. `artifacts/index.yaml`
|
||||
is the deterministic manifest of artifacts that have been added to the
|
||||
infospace.
|
||||
|
||||
## Required Directories
|
||||
|
||||
```text
|
||||
artifacts/sources/
|
||||
artifacts/generated/
|
||||
output/evaluations/
|
||||
output/metrics/
|
||||
reports/
|
||||
exports/
|
||||
```
|
||||
|
||||
## Artifact Manifest
|
||||
|
||||
Artifacts are represented by stable IDs such as `source/chapter-01.md`.
|
||||
|
||||
Each manifest entry records:
|
||||
|
||||
- `id`
|
||||
- `path`
|
||||
- `kind`
|
||||
- `title`
|
||||
- `provenance`
|
||||
- `relationships`
|
||||
|
||||
The manifest is intentionally plain YAML so it can be inspected, diffed, and
|
||||
regenerated by tools or agents.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
python3 -m infospace_bench create . pilot --name "Pilot Infospace"
|
||||
python3 -m infospace_bench add-artifact infospaces/pilot ./source.md --kind source
|
||||
python3 -m infospace_bench inspect infospaces/pilot
|
||||
python3 -m infospace_bench export infospaces/pilot
|
||||
```
|
||||
24
docs/reference-pilot-decision.md
Normal file
24
docs/reference-pilot-decision.md
Normal file
@@ -0,0 +1,24 @@
|
||||
# Reference Pilot Decision
|
||||
|
||||
Date: 2026-05-14
|
||||
|
||||
## Decision
|
||||
|
||||
Use a small purpose-built corpus as the first maintained reference infospace.
|
||||
|
||||
## Rationale
|
||||
|
||||
`markitect-main/examples/infospace-with-history/` remains the primary migration
|
||||
candidate for a larger pilot, but it contains a large public-domain book corpus
|
||||
and substantial generated output. Pulling it in before the lifecycle and
|
||||
evaluation baseline exists would make the new repo noisy before it is useful.
|
||||
|
||||
The bootstrap pilot uses this repo's own PRD/FRS intent as a compact corpus. It
|
||||
proves the expected file layout, artifact manifest, relationship inspection,
|
||||
collection metrics, and viability thresholding with minimal bulk.
|
||||
|
||||
## Follow-up
|
||||
|
||||
After the baseline is stable, migrate a pruned Wealth of Nations/VSM fixture or
|
||||
a similarly representative slice from `markitect-main` under a separate
|
||||
workplan.
|
||||
Reference in New Issue
Block a user