generated from coulomb/repo-seed
tegwick
f818acfc62
T03 — wrap_with_shadow_sampling() helper in routing.py: builds a llm-connect ShadowingAdapter around any candidate LLMAdapter with a caller-supplied baseline, grader, and QualityLedger. async_shadow=True by default so production load is not doubled; on_shadow_error escape hatch keeps caller logs informed when a baseline outage swallows the shadow path. The returned adapter is still an LLMAdapter so it slots into a RoutingPolicy rule without further code change. T04 — generation report enrichment plus a small CLI helper: - _collect_adapter_choices walks artifact provenance, groups by (stage_id, adapter_id), and surfaces calls + prompt/completion tokens per (stage, adapter) pair in a new ## Per-stage adapter choices section. Runs that did not go through the bridge have no provider_metadata.adapter_id and emit an empty list, so fixture-only reports stay terse. - summarise_quality_ledger() rolls a llm-connect QualityLedger up by (task_type, adapter_id) with mean quality, mean cost, observations, and cumulative tokens. - infospace-bench routing ledger <path> CLI prints the rollup as JSON. Five new tests cover shadow happy-path, shadow failure isolation, ledger rollup, the routing CLI, and the report's adapter-choice aggregation. Closes IB-WP-0018: T01-T05 are all done and the workplan status flips from blocked to done now that LLM-WP-0004's primitives have shipped. 144 tests pass, 1 skipped (the OpenRouter live smoke, gated as before). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
infospace-bench
Workspace and service for creating, developing, evaluating, and inspecting structured knowledge spaces.
This repo is the application-layer successor for the infospace work that began
inside markitect-main. It focuses on concrete infospaces and their lifecycle,
while lower-level markdown tooling and runtime orchestration remain in sibling
projects.
Start with:
INTENT.mdwiki/ProductRequirementsDocument.mdwiki/FunctionalRequirementsSpecification.mdSCOPE.mddocs/infospace-layout.mddocs/evaluation-and-inspection.mddocs/reference-pilot-decision.mddocs/markitect-main-scope-assessment.mddocs/markitect-tool-adapter.mddocs/entity-relation-model.mddocs/evaluation-history-and-metrics.mddocs/workflow-generation-pipeline.mddocs/kontextual-engine-boundary.mddocs/orthogonal-successor-roadmap.mddocs/legacy-infospace-feature-inventory.mddocs/successor-boundary-interface-map.mddocs/replacement-acceptance-matrix.mddocs/legacy-command-parity.mddocs/legacy-infospace-migration-guide.mddocs/replacement-readiness-decision.mddocs/wealth-vsm-generation-pipeline.mddocs/generic-source-generator.mddocs/agentic-memory-profile-pilot.mddocs/lefevre-epub3-validation.mdinfospaces/bootstrap-pilot/infospaces/wealth-vsm-legacy-slice/infospaces/wealth-vsm-generation-pilot/infospaces/agentic-memory-profile-pilot/workplans/
Current development command:
python3 -m pytest