diff --git a/README.md b/README.md index 55082de..6c92752 100644 --- a/README.md +++ b/README.md @@ -42,9 +42,10 @@ The same CLI contracts are packaged by the container baseline. See [docs/CONTAINER.md](docs/CONTAINER.md). The dependency-light local API wraps those contracts for service and container operation; see [docs/LOCAL-SERVICE-API.md](docs/LOCAL-SERVICE-API.md). -New runs write a richer `sources.lock.json` and -`reports/submission-package.json` alongside the assessment package so reviewers -can inspect source, metadata, artifact, and boundary references. +New runs write a richer `sources.lock.json`, +`reports/submission-package.json`, extension report fragments, and +`exports/export-manifest.json` alongside the assessment package so reviewers can +inspect source, metadata, artifact, export, and boundary references. The `sample-noop` extension exercises the guide-board contracts without invoking an external harness. `sdk-fixture` demonstrates the extension SDK contracts for diff --git a/docs/ARCHITECTURE-BLUEPRINT.md b/docs/ARCHITECTURE-BLUEPRINT.md index 0934989..e698ed6 100644 --- a/docs/ARCHITECTURE-BLUEPRINT.md +++ b/docs/ARCHITECTURE-BLUEPRINT.md @@ -653,6 +653,18 @@ building complex runtime code. - `reported_metadata` - `certification_boundary` +### `ExportManifest` + +- `export_type` +- `source_package_ref` +- `source_lock_ref` +- `summary` +- `policy_summary` +- `mapping_summary` +- `report_fragments` +- `counts` +- `certification_boundary` + ## Result Vocabulary The evidence model should allow these statuses: @@ -740,9 +752,11 @@ runs// mappings.json reports/ report.md + fragments.json assessment-package.json submission-package.json exports/ + export-manifest.json ``` ## Container And Service Model @@ -836,6 +850,8 @@ hooks for runners and normalizers. 6. Add container design after the CLI baseline is stable. 7. Add optional service API around the CLI job model. 8. Add OSCAL export and procedural evidence-pack support after the internal - evidence model proves itself with executable extensions. + evidence model proves itself with executable extensions. The first generic + export is `exports/export-manifest.json`; authority-specific interchange + remains extension-owned until the internal model is stable. The first extension SDK contract is documented in `docs/EXTENSION-SDK.md`. diff --git a/docs/ASSESSMENT-OPERATIONS.md b/docs/ASSESSMENT-OPERATIONS.md index ac8083c..d16dc0e 100644 --- a/docs/ASSESSMENT-OPERATIONS.md +++ b/docs/ASSESSMENT-OPERATIONS.md @@ -79,6 +79,7 @@ A completed CLI command prints a JSON result with: - `assessment_package`: JSON assessment package path, - `report`: Markdown report path, - `submission_package`: portable submission package manifest path, +- `export_manifest`: portable generic JSON export manifest path, - `retention_summary`: compact durable summary path. The output directory uses this contract: @@ -95,7 +96,9 @@ normalized/findings.json normalized/mappings.json reports/assessment-package.json reports/report.md +reports/fragments.json reports/submission-package.json +exports/export-manifest.json artifacts/ ``` @@ -107,6 +110,12 @@ raw artifact manifest, and repeats the certification boundary. It is a portable handoff manifest for preparation evidence, not an authority-specific final submission. +Extension report fragments are recorded in `reports/fragments.json`, embedded in +`reports/assessment-package.json`, and rendered into the Markdown report. +`exports/export-manifest.json` is the first generic portable export surface. It +is derived from the assessment package and carries summary, policy, mapping, +fragment, count, source-lock, and boundary references. + Use the retained run helpers for history: ```sh @@ -121,6 +130,10 @@ PYTHONPATH=src python3 -m guide_board runs trend --runs-dir runs PYTHONPATH=src python3 -m guide_board runs gate --runs-dir runs ``` +Trend summaries include status changes, unexpected finding deltas, unresolved +review deltas, mapping target deltas, evidence result deltas, and a compact +human-readable summary string for each target/assessment pair. + ## Local Service Flow Start the service from the guide-board repository: diff --git a/docs/CONTAINER.md b/docs/CONTAINER.md index c698424..dac7dac 100644 --- a/docs/CONTAINER.md +++ b/docs/CONTAINER.md @@ -62,7 +62,8 @@ The script: - builds `guide-board-core:smoke`, - mounts a host output directory at `/runs`, - runs the bundled sample assessment, -- verifies that the expected run artifacts are present on the host. +- verifies that the expected run artifacts, report fragments, submission + manifest, and generic export manifest are present on the host. Override the runtime, image name, or output directory when needed: diff --git a/docs/EXTENSION-SDK.md b/docs/EXTENSION-SDK.md index cf5738e..d23fa9e 100644 --- a/docs/EXTENSION-SDK.md +++ b/docs/EXTENSION-SDK.md @@ -83,6 +83,8 @@ The key runtime fields are: - `normalizers`: optional plug-ins that convert native runner output into the stable runner-result shape before evidence is written. - `mappings`: mapping set IDs under `mappings/.json`. +- `report_fragments`: optional Markdown file or Python module descriptors for + extension-owned report content. - `certification_boundary`: explicit statement of what the extension does not certify. @@ -209,6 +211,53 @@ to extension-owned mappings and writes normalized mapping records to: runs//normalized/mappings.json ``` +## Report Fragments + +Extensions can contribute report fragments through `report_fragments`. + +Static Markdown file: + +```json +{ + "id": "overview", + "kind": "markdown_file", + "path": "reports/overview.md", + "title": "Overview" +} +``` + +Dynamic Python fragment: + +```json +{ + "id": "sdk-fixture-summary", + "kind": "python_module", + "module_path": "reports/sdk_fixture_summary.py", + "callable": "build_fragment", + "path": null, + "title": "SDK Fixture Summary" +} +``` + +Fragment paths are resolved relative to the extension root and must stay inside +that root. A Python fragment receives `root`, `run_dir`, `run_id`, `plan`, +`evidence`, `findings`, `mappings`, `assessment_package`, `policy_summary`, +`source_lock`, `extension_path`, and `report_fragment`. + +It returns: + +```python +def build_fragment(context: dict) -> dict: + return { + "markdown": "### Extension Summary\n\n- evidence items: 2", + "structured": {"evidence_count": 2}, + } +``` + +Fragments are written to `reports/fragments.json`, embedded in the assessment +package, rendered in `reports/report.md`, and summarized in +`exports/export-manifest.json`. + ## Evidence Request Sets Procedural and hybrid compliance extensions may include evidence request sets @@ -402,9 +451,9 @@ profiles. ## Source Lock And Submission Package -Every new run writes `sources.lock.json` and -`reports/submission-package.json`. Extension authors should treat source -metadata as part of the evidence contract: +Every new run writes `sources.lock.json`, `reports/submission-package.json`, +and the generic portable export manifest at `exports/export-manifest.json`. +Extension authors should treat source metadata as part of the evidence contract: - declare extension, authority, framework, runner, and normalizer metadata in `extension.json` when it is static; diff --git a/docs/LOCAL-SERVICE-API.md b/docs/LOCAL-SERVICE-API.md index 2f687ec..579b36b 100644 --- a/docs/LOCAL-SERVICE-API.md +++ b/docs/LOCAL-SERVICE-API.md @@ -98,8 +98,8 @@ errors. ### `GET /runs/{job_id}/reports` Returns the Markdown report content, assessment package JSON, retention summary, -submission package JSON when present, and their filesystem paths after a job has -succeeded. +submission package JSON, export manifest JSON when present, and their filesystem +paths after a job has succeeded. ### `GET /retained-runs` diff --git a/docs/schemas/assessment-package.schema.json b/docs/schemas/assessment-package.schema.json index 54a7c91..75cf269 100644 --- a/docs/schemas/assessment-package.schema.json +++ b/docs/schemas/assessment-package.schema.json @@ -19,6 +19,7 @@ "waivers", "challenges", "exclusions", + "report_fragments", "certification_boundary", "created_at" ], @@ -38,6 +39,7 @@ "waivers": { "type": "array", "items": { "type": "object" } }, "challenges": { "type": "array", "items": { "type": "object" } }, "exclusions": { "type": "array", "items": { "type": "object" } }, + "report_fragments": { "type": "array", "items": { "type": "object" } }, "certification_boundary": { "type": "string" }, "created_at": { "type": "string" } } diff --git a/docs/schemas/export-manifest.schema.json b/docs/schemas/export-manifest.schema.json new file mode 100644 index 0000000..e763971 --- /dev/null +++ b/docs/schemas/export-manifest.schema.json @@ -0,0 +1,36 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "Guide Board Export Manifest", + "type": "object", + "additionalProperties": false, + "required": [ + "id", + "schema_version", + "export_type", + "run_id", + "created_at", + "source_package_ref", + "source_lock_ref", + "summary", + "policy_summary", + "mapping_summary", + "report_fragments", + "counts", + "certification_boundary" + ], + "properties": { + "id": { "type": "string" }, + "schema_version": { "type": "string" }, + "export_type": { "type": "string" }, + "run_id": { "type": "string" }, + "created_at": { "type": "string" }, + "source_package_ref": { "type": "string" }, + "source_lock_ref": { "type": "string" }, + "summary": { "type": "object" }, + "policy_summary": { "type": "object" }, + "mapping_summary": { "type": "object" }, + "report_fragments": { "type": "array", "items": { "type": "object" } }, + "counts": { "type": "object" }, + "certification_boundary": { "type": "string" } + } +} diff --git a/docs/schemas/extension-manifest.schema.json b/docs/schemas/extension-manifest.schema.json index d08c569..7aa6bf2 100644 --- a/docs/schemas/extension-manifest.schema.json +++ b/docs/schemas/extension-manifest.schema.json @@ -142,7 +142,23 @@ } }, "mappings": { "type": "array", "items": { "type": "string" } }, - "report_fragments": { "type": "array", "items": { "type": "string" } }, + "report_fragments": { + "type": "array", + "items": { + "type": ["string", "object"], + "additionalProperties": false, + "required": ["id", "kind"], + "properties": { + "id": { "type": "string" }, + "kind": { "type": "string", "enum": ["markdown_file", "python_module"] }, + "path": { "type": ["string", "null"] }, + "module_path": { "type": ["string", "null"] }, + "callable": { "type": ["string", "null"] }, + "title": { "type": ["string", "null"] }, + "description": { "type": ["string", "null"] } + } + } + }, "dependencies": { "type": "array", "items": { "type": "string" } }, "restricted_assets": { "type": "array", "items": { "type": "string" } }, "certification_boundary": { "type": "string" } diff --git a/docs/schemas/retention-summary.schema.json b/docs/schemas/retention-summary.schema.json index adf4e94..6a878c3 100644 --- a/docs/schemas/retention-summary.schema.json +++ b/docs/schemas/retention-summary.schema.json @@ -21,6 +21,7 @@ "created_at": { "type": "string" }, "summary": { "type": "object" }, "report_refs": { "type": "array", "items": { "type": "string" } }, + "export_refs": { "type": "array", "items": { "type": "string" } }, "artifact_retention": { "type": "object" } } } diff --git a/extensions/sdk-fixture/extension.json b/extensions/sdk-fixture/extension.json index f08d4a6..a2a9b4a 100644 --- a/extensions/sdk-fixture/extension.json +++ b/extensions/sdk-fixture/extension.json @@ -74,7 +74,17 @@ "mappings": [ "sdk-fixture-map" ], - "report_fragments": [], + "report_fragments": [ + { + "id": "sdk-fixture-summary", + "kind": "python_module", + "module_path": "reports/sdk_fixture_summary.py", + "callable": "build_fragment", + "path": null, + "title": "SDK Fixture Summary", + "description": "Summarizes SDK fixture evidence for report fragment tests." + } + ], "dependencies": [], "restricted_assets": [], "certification_boundary": "SDK fixture only. It does not certify any product, process, or repository." diff --git a/extensions/sdk-fixture/reports/sdk_fixture_summary.py b/extensions/sdk-fixture/reports/sdk_fixture_summary.py new file mode 100644 index 0000000..aa023e3 --- /dev/null +++ b/extensions/sdk-fixture/reports/sdk_fixture_summary.py @@ -0,0 +1,26 @@ +"""Report fragment for the SDK fixture extension.""" + +from __future__ import annotations + + +def build_fragment(context: dict) -> dict: + evidence_count = len(context["evidence"]) + finding_count = len(context["findings"]) + source_lock = context["source_lock"] + markdown = "\n".join( + [ + "### SDK Fixture Summary", + "", + f"- evidence items: {evidence_count}", + f"- findings: {finding_count}", + f"- source lock: {source_lock.get('id')}", + ] + ) + return { + "markdown": markdown, + "structured": { + "evidence_count": evidence_count, + "finding_count": finding_count, + "source_lock_ref": source_lock.get("id"), + }, + } diff --git a/scripts/container_smoke.sh b/scripts/container_smoke.sh index 2fd9692..c5797d0 100755 --- a/scripts/container_smoke.sh +++ b/scripts/container_smoke.sh @@ -42,7 +42,9 @@ for path in \ "$RUNS_DIR/sample-noop/normalized/mappings.json" \ "$RUNS_DIR/sample-noop/reports/assessment-package.json" \ "$RUNS_DIR/sample-noop/reports/report.md" \ - "$RUNS_DIR/sample-noop/reports/submission-package.json" + "$RUNS_DIR/sample-noop/reports/fragments.json" \ + "$RUNS_DIR/sample-noop/reports/submission-package.json" \ + "$RUNS_DIR/sample-noop/exports/export-manifest.json" do if [ ! -f "$path" ]; then echo "ERROR: expected artifact missing: $path" >&2 diff --git a/src/guide_board/execution.py b/src/guide_board/execution.py index 7ee6e35..52f6d4c 100644 --- a/src/guide_board/execution.py +++ b/src/guide_board/execution.py @@ -8,11 +8,13 @@ from pathlib import Path from typing import Any from guide_board.artifacts import build_artifact_manifest, build_submission_manifest +from guide_board.exports import build_export_manifest from guide_board.io import load_json, write_json from guide_board.mapping import build_mapping_records, summarize_mappings from guide_board.normalizers import normalize_step_result from guide_board.planning import build_run_plan from guide_board.policy import apply_policy +from guide_board.reports import build_report_fragments, markdown_for_fragments from guide_board.retention import build_retention_summary from guide_board.runners import run_step from guide_board.schema import assert_valid @@ -64,7 +66,19 @@ def run_assessment( applied_exclusions, created_at, ) + report_fragments = build_report_fragments( + root, + run_dir, + run_id, + plan, + evidence, + findings, + mapping_records, + assessment_package, + ) + assessment_package["report_fragments"] = report_fragments assert_valid(assessment_package, "assessment-package") + export_manifest = build_export_manifest(assessment_package) run_metadata = { "id": run_id, @@ -85,6 +99,7 @@ def run_assessment( findings, mapping_records, assessment_package, + export_manifest, retention_summary, ) return { @@ -94,6 +109,7 @@ def run_assessment( "assessment_package": str(run_dir / "reports" / "assessment-package.json"), "report": str(run_dir / "reports" / "report.md"), "submission_package": str(run_dir / "reports" / "submission-package.json"), + "export_manifest": str(run_dir / "exports" / "export-manifest.json"), "retention_summary": str(run_dir / "retention-summary.json"), } @@ -419,6 +435,7 @@ def _assessment_package( "waivers": applied_waivers, "challenges": applied_challenges, "exclusions": applied_exclusions, + "report_fragments": [], "certification_boundary": "Guide Board produces preparation evidence only and does not issue certifications or audit assurance.", "created_at": created_at, } @@ -432,6 +449,7 @@ def _write_run_directory( findings: list[dict[str, Any]], mapping_records: list[dict[str, Any]], assessment_package: dict[str, Any], + export_manifest: dict[str, Any], retention_summary: dict[str, Any], ) -> None: write_json(run_dir / "run.json", run_metadata) @@ -447,6 +465,8 @@ def _write_run_directory( write_json(run_dir / "normalized" / "findings.json", {"findings": findings}) write_json(run_dir / "normalized" / "mappings.json", {"mappings": mapping_records}) write_json(run_dir / "reports" / "assessment-package.json", assessment_package) + write_json(run_dir / "reports" / "fragments.json", {"fragments": assessment_package["report_fragments"]}) + write_json(run_dir / "exports" / "export-manifest.json", export_manifest) (run_dir / "reports").mkdir(parents=True, exist_ok=True) (run_dir / "reports" / "report.md").write_text( _markdown_report(run_metadata, assessment_package), @@ -471,6 +491,7 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s mapping_lines = _mapping_summary_lines(package) policy_lines = _policy_summary_lines(package) review_lines = _review_summary_lines(package) + fragment_lines = markdown_for_fragments(package.get("report_fragments", [])) return "\n".join( [ @@ -496,6 +517,10 @@ def _markdown_report(run_metadata: dict[str, Any], package: dict[str, Any]) -> s "", review_lines, "", + "## Extension Fragments", + "", + fragment_lines, + "", "## Boundary", "", package["certification_boundary"], diff --git a/src/guide_board/exports.py b/src/guide_board/exports.py new file mode 100644 index 0000000..88a5dc2 --- /dev/null +++ b/src/guide_board/exports.py @@ -0,0 +1,50 @@ +"""Portable export builders derived from assessment packages.""" + +from __future__ import annotations + +from datetime import datetime, timezone +from typing import Any + +from guide_board.schema import assert_valid + + +def build_export_manifest(assessment_package: dict[str, Any]) -> dict[str, Any]: + manifest = { + "id": f"export-manifest:{assessment_package['run_id']}", + "schema_version": "guide-board.export-manifest.v1", + "export_type": "guide-board.generic-json.v1", + "run_id": assessment_package["run_id"], + "created_at": datetime.now(timezone.utc).isoformat(), + "source_package_ref": "reports/assessment-package.json", + "source_lock_ref": "sources.lock.json", + "summary": assessment_package.get("summary", {}), + "policy_summary": assessment_package.get("policy_summary", {}), + "mapping_summary": assessment_package.get("mapping_summary", {}), + "report_fragments": [ + _export_fragment(fragment) + for fragment in assessment_package.get("report_fragments", []) + if isinstance(fragment, dict) + ], + "counts": { + "evidence_refs": len(assessment_package.get("evidence_refs", [])), + "findings": len(assessment_package.get("findings", [])), + "artifacts": len(assessment_package.get("artifact_manifest", [])), + "waivers": len(assessment_package.get("waivers", [])), + "challenges": len(assessment_package.get("challenges", [])), + "exclusions": len(assessment_package.get("exclusions", [])), + "report_fragments": len(assessment_package.get("report_fragments", [])), + }, + "certification_boundary": assessment_package["certification_boundary"], + } + assert_valid(manifest, "export-manifest") + return manifest + + +def _export_fragment(fragment: dict[str, Any]) -> dict[str, Any]: + return { + "id": fragment.get("id"), + "extension_id": fragment.get("extension_id"), + "title": fragment.get("title"), + "kind": fragment.get("kind"), + "structured": fragment.get("structured", {}), + } diff --git a/src/guide_board/reports.py b/src/guide_board/reports.py new file mode 100644 index 0000000..1c5dc4d --- /dev/null +++ b/src/guide_board/reports.py @@ -0,0 +1,204 @@ +"""Report fragment loading for extension-contributed report content.""" + +from __future__ import annotations + +import importlib.util +from pathlib import Path +from types import ModuleType +from typing import Any + +from guide_board.errors import ValidationError +from guide_board.io import load_json + + +def build_report_fragments( + root: Path, + run_dir: Path, + run_id: str, + plan: dict[str, Any], + evidence: list[dict[str, Any]], + findings: list[dict[str, Any]], + mapping_records: list[dict[str, Any]], + assessment_package: dict[str, Any], +) -> list[dict[str, Any]]: + fragments: list[dict[str, Any]] = [] + for extension in plan["extension_snapshots"]: + extension_path = _snapshot_path(root, extension) + manifest = load_json(extension_path / "extension.json") + for descriptor in manifest.get("report_fragments", []): + fragment = _load_fragment( + root, + run_dir, + run_id, + plan, + evidence, + findings, + mapping_records, + assessment_package, + extension, + extension_path, + descriptor, + ) + if fragment is not None: + fragments.append(fragment) + return fragments + + +def markdown_for_fragments(fragments: list[dict[str, Any]]) -> str: + markdown_blocks = [ + fragment.get("markdown", "") + for fragment in fragments + if isinstance(fragment.get("markdown"), str) and fragment.get("markdown") + ] + if not markdown_blocks: + return "- no extension report fragments" + return "\n\n".join(markdown_blocks) + + +def _load_fragment( + root: Path, + run_dir: Path, + run_id: str, + plan: dict[str, Any], + evidence: list[dict[str, Any]], + findings: list[dict[str, Any]], + mapping_records: list[dict[str, Any]], + assessment_package: dict[str, Any], + extension: dict[str, Any], + extension_path: Path, + descriptor: Any, +) -> dict[str, Any] | None: + if isinstance(descriptor, str): + descriptor = { + "id": descriptor, + "kind": "markdown_file", + "path": f"reports/{descriptor}.md", + "title": descriptor, + } + if not isinstance(descriptor, dict): + return None + + if descriptor["kind"] == "markdown_file": + markdown = _load_markdown_fragment(extension_path, descriptor) + return _fragment_record(extension["id"], descriptor, markdown, {}) + if descriptor["kind"] == "python_module": + result = _run_python_fragment( + root, + run_dir, + run_id, + plan, + evidence, + findings, + mapping_records, + assessment_package, + extension_path, + descriptor, + ) + return _fragment_record( + extension["id"], + descriptor, + result.get("markdown", ""), + _object_or_empty(result.get("structured")), + ) + raise ValidationError(f"{descriptor['id']}: unsupported report fragment kind") + + +def _load_markdown_fragment(extension_path: Path, descriptor: dict[str, Any]) -> str: + raw_path = descriptor.get("path") or f"reports/{descriptor['id']}.md" + fragment_path = _safe_extension_path(extension_path, raw_path, descriptor["id"]) + if not fragment_path.is_file(): + raise ValidationError(f"{descriptor['id']}: report fragment not found: {raw_path}") + return fragment_path.read_text(encoding="utf-8") + + +def _run_python_fragment( + root: Path, + run_dir: Path, + run_id: str, + plan: dict[str, Any], + evidence: list[dict[str, Any]], + findings: list[dict[str, Any]], + mapping_records: list[dict[str, Any]], + assessment_package: dict[str, Any], + extension_path: Path, + descriptor: dict[str, Any], +) -> dict[str, Any]: + module_path = descriptor.get("module_path") + callable_name = descriptor.get("callable") + if not module_path or not callable_name: + raise ValidationError( + f"{descriptor['id']}: python_module report fragments need module_path and callable" + ) + + module_file = _safe_extension_path(extension_path, module_path, descriptor["id"]) + module = _load_module(module_file, descriptor["id"]) + fragment_callable = getattr(module, callable_name, None) + if not callable(fragment_callable): + raise ValidationError(f"{descriptor['id']}: callable {callable_name!r} was not found") + + context = { + "root": str(root), + "run_dir": str(run_dir), + "run_id": run_id, + "plan": plan, + "evidence": evidence, + "findings": findings, + "mappings": mapping_records, + "assessment_package": assessment_package, + "policy_summary": assessment_package.get("policy_summary", {}), + "source_lock": assessment_package.get("source_lock", {}), + "extension_path": str(extension_path), + "report_fragment": descriptor, + } + result = fragment_callable(context) + if not isinstance(result, dict): + raise ValidationError(f"{descriptor['id']}: report fragment must return an object") + return result + + +def _fragment_record( + extension_id: str, + descriptor: dict[str, Any], + markdown: str, + structured: dict[str, Any], +) -> dict[str, Any]: + return { + "id": descriptor["id"], + "extension_id": extension_id, + "title": descriptor.get("title") or descriptor["id"], + "kind": descriptor["kind"], + "markdown": markdown if isinstance(markdown, str) else "", + "structured": structured, + } + + +def _safe_extension_path(extension_path: Path, raw_path: str, fragment_id: str) -> Path: + path = (extension_path / raw_path).resolve() + try: + path.relative_to(extension_path.resolve()) + except ValueError as exc: + raise ValidationError( + f"{fragment_id}: report fragment path must stay inside the extension directory" + ) from exc + return path + + +def _load_module(path: Path, fragment_id: str) -> ModuleType: + if not path.exists(): + raise ValidationError(f"{fragment_id}: module not found: {path}") + module_name = f"_guide_board_report_fragment_{fragment_id.replace('-', '_')}" + spec = importlib.util.spec_from_file_location(module_name, path) + if spec is None or spec.loader is None: + raise ValidationError(f"{fragment_id}: unable to load module from {path}") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def _snapshot_path(root: Path, extension: dict[str, Any]) -> Path: + path = Path(extension["path"]) + return path if path.is_absolute() else root / path + + +def _object_or_empty(value: Any) -> dict[str, Any]: + return value if isinstance(value, dict) else {} diff --git a/src/guide_board/retention.py b/src/guide_board/retention.py index 8b980bd..57b2f9c 100644 --- a/src/guide_board/retention.py +++ b/src/guide_board/retention.py @@ -49,8 +49,12 @@ def build_retention_summary( "report_refs": [ "reports/assessment-package.json", "reports/report.md", + "reports/fragments.json", "reports/submission-package.json", ], + "export_refs": [ + "exports/export-manifest.json", + ], "artifact_retention": { "policy": plan["assessment_profile_snapshot"].get("retention_policy", {}), "output_artifact_retention": plan["assessment_profile_snapshot"] @@ -200,6 +204,7 @@ def _run_projection(run: dict[str, Any]) -> dict[str, Any]: "status": summary.get("status", "unknown"), "unexpected_findings": _summary_int(summary, "unexpected_findings"), "finding_count": _summary_int(summary, "finding_count"), + "mapping_target_count": _summary_int(summary, "mapping_target_count"), "artifact_count": _summary_int(summary, "artifact_count"), "challenged_findings": _summary_int(summary, "challenged_findings"), "authority_exclusions": _summary_int(summary, "authority_exclusions"), @@ -217,12 +222,18 @@ def _trend_between( return { "direction": "insufficient-history", "status_changed": False, + "status_change": { + "from": None, + "to": _status_for(latest), + }, "unexpected_findings_delta": 0, "finding_count_delta": 0, - "artifact_count_delta": 0, - "unresolved_review_items_delta": 0, - "evidence_result_deltas": {}, - } + "artifact_count_delta": 0, + "unresolved_review_items_delta": 0, + "mapping_target_count_delta": 0, + "evidence_result_deltas": {}, + "summary_text": "No previous retained run is available for comparison.", + } previous_summary = previous.get("summary", {}) latest_summary = latest.get("summary", {}) @@ -242,17 +253,34 @@ def _trend_between( review_delta = _summary_int(latest_summary, "unresolved_review_items") - _summary_int( previous_summary, "unresolved_review_items" ) + mapping_target_delta = _summary_int(latest_summary, "mapping_target_count") - _summary_int( + previous_summary, "mapping_target_count" + ) previous_status = _status_for(previous) latest_status = _status_for(latest) + direction = _trend_direction(previous_status, latest_status, unexpected_delta) return { - "direction": _trend_direction(previous_status, latest_status, unexpected_delta), + "direction": direction, "status_changed": previous_status != latest_status, + "status_change": { + "from": previous_status, + "to": latest_status, + }, "unexpected_findings_delta": unexpected_delta, "finding_count_delta": finding_delta, "artifact_count_delta": artifact_delta, "unresolved_review_items_delta": review_delta, + "mapping_target_count_delta": mapping_target_delta, "evidence_result_deltas": evidence_deltas, + "summary_text": _trend_summary_text( + direction, + previous_status, + latest_status, + unexpected_delta, + review_delta, + mapping_target_delta, + ), } @@ -294,6 +322,24 @@ def _summary_int(summary: dict[str, Any], key: str) -> int: return value if isinstance(value, int) and not isinstance(value, bool) else 0 +def _trend_summary_text( + direction: str, + previous_status: str, + latest_status: str, + unexpected_delta: int, + review_delta: int, + mapping_target_delta: int, +) -> str: + parts = [ + f"Trend {direction}", + f"status {previous_status} -> {latest_status}", + f"unexpected findings delta {unexpected_delta}", + f"unresolved review delta {review_delta}", + f"mapping target delta {mapping_target_delta}", + ] + return "; ".join(parts) + "." + + def _dict_deltas(previous: Any, latest: Any) -> dict[str, int]: previous_dict = previous if isinstance(previous, dict) else {} latest_dict = latest if isinstance(latest, dict) else {} diff --git a/src/guide_board/service.py b/src/guide_board/service.py index a993cf0..32dece3 100644 --- a/src/guide_board/service.py +++ b/src/guide_board/service.py @@ -285,6 +285,12 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): if isinstance(submission_value, str) and submission_value else None ) + export_value = result.get("export_manifest") + export_path = ( + Path(export_value) + if isinstance(export_value, str) and export_value + else None + ) try: report_markdown = report_path.read_text(encoding="utf-8") assessment_package = load_json(package_path) @@ -294,6 +300,11 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): if submission_path is not None and submission_path.is_file() else None ) + export_manifest = ( + load_json(export_path) + if export_path is not None and export_path.is_file() + else None + ) except OSError as exc: raise HttpProblem(404, f"run report artifact is missing: {exc}") from exc @@ -307,6 +318,7 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): "assessment_package": str(package_path), "retention_summary": str(retention_path), "submission_package": str(submission_path) if submission_path is not None else None, + "export_manifest": str(export_path) if export_path is not None else None, }, "report": { "path": str(report_path), @@ -324,6 +336,10 @@ class GuideBoardRequestHandler(BaseHTTPRequestHandler): "path": str(submission_path) if submission_package else None, "json": submission_package, }, + "export_manifest": { + "path": str(export_path) if export_manifest else None, + "json": export_manifest, + }, } def _retained_runs(self, query: dict[str, str]) -> dict[str, Any]: diff --git a/tests/golden/export-manifest-shape.json b/tests/golden/export-manifest-shape.json new file mode 100644 index 0000000..861afd8 --- /dev/null +++ b/tests/golden/export-manifest-shape.json @@ -0,0 +1,17 @@ +{ + "top_level_keys": [ + "certification_boundary", + "counts", + "created_at", + "export_type", + "id", + "mapping_summary", + "policy_summary", + "report_fragments", + "run_id", + "schema_version", + "source_lock_ref", + "source_package_ref", + "summary" + ] +} diff --git a/tests/golden/sdk-fixture-report-fragment.md b/tests/golden/sdk-fixture-report-fragment.md new file mode 100644 index 0000000..0381bcb --- /dev/null +++ b/tests/golden/sdk-fixture-report-fragment.md @@ -0,0 +1,5 @@ +### SDK Fixture Summary + +- evidence items: 2 +- findings: 0 +- source lock: source-lock:sdk-fixture-assessment:sdk-fixture-target diff --git a/tests/test_core.py b/tests/test_core.py index 7ac412d..b71e064 100644 --- a/tests/test_core.py +++ b/tests/test_core.py @@ -230,6 +230,10 @@ class CoreArchitectureTests(unittest.TestCase): assessment_package = json.loads( (run_dir / "reports" / "assessment-package.json").read_text(encoding="utf-8") ) + report = (run_dir / "reports" / "report.md").read_text(encoding="utf-8") + export_manifest = json.loads( + (run_dir / "exports" / "export-manifest.json").read_text(encoding="utf-8") + ) self.assertEqual(result["status"], "completed") self.assertEqual(plan["extension_snapshots"][0]["source"], "external") @@ -259,6 +263,21 @@ class CoreArchitectureTests(unittest.TestCase): ) self.assertEqual(mappings[0]["target_id"], "normalizer-plugin") self.assertEqual(assessment_package["summary"], {"pass": 1, "skipped": 1}) + self.assertEqual( + assessment_package["report_fragments"][0]["markdown"], + (ROOT / "tests" / "golden" / "sdk-fixture-report-fragment.md") + .read_text(encoding="utf-8") + .rstrip(), + ) + self.assertIn("### SDK Fixture Summary", report) + assert_valid(export_manifest, "export-manifest") + export_shape = load_json(ROOT / "tests" / "golden" / "export-manifest-shape.json") + self.assertEqual(sorted(export_manifest), export_shape["top_level_keys"]) + self.assertEqual(export_manifest["counts"]["report_fragments"], 1) + self.assertEqual( + export_manifest["report_fragments"][0]["structured"]["evidence_count"], + 2, + ) self.assertEqual( assessment_package["source_lock"]["metadata_hooks"]["runner_entrypoints"][0][ "metadata" @@ -297,7 +316,9 @@ class CoreArchitectureTests(unittest.TestCase): self.assertTrue((run_dir / "normalized" / "evidence.json").exists()) self.assertTrue((run_dir / "reports" / "assessment-package.json").exists()) self.assertTrue((run_dir / "reports" / "report.md").exists()) + self.assertTrue((run_dir / "reports" / "fragments.json").exists()) self.assertTrue((run_dir / "reports" / "submission-package.json").exists()) + self.assertTrue((run_dir / "exports" / "export-manifest.json").exists()) retention = json.loads( (run_dir / "retention-summary.json").read_text(encoding="utf-8") ) @@ -309,9 +330,14 @@ class CoreArchitectureTests(unittest.TestCase): result["submission_package"], str(run_dir / "reports" / "submission-package.json"), ) + self.assertEqual( + result["export_manifest"], + str(run_dir / "exports" / "export-manifest.json"), + ) self.assertEqual(retention["summary"]["status"], "completed") self.assertEqual(retention["summary"]["artifact_count"], 0) self.assertIn("reports/submission-package.json", retention["report_refs"]) + self.assertIn("exports/export-manifest.json", retention["export_refs"]) self.assertEqual( retention["artifact_retention"]["policy"], {"raw_artifact_days": 0, "summary_days": 365}, @@ -463,6 +489,10 @@ class CoreArchitectureTests(unittest.TestCase): reports["submission_package"]["json"]["run_id"], status["result"]["run_id"], ) + self.assertEqual( + reports["export_manifest"]["json"]["run_id"], + status["result"]["run_id"], + ) finally: service.stop() @@ -552,7 +582,13 @@ class CoreArchitectureTests(unittest.TestCase): self.assertEqual(group["previous_run"]["run_id"], "run-old") self.assertEqual(group["trend"]["direction"], "improved") self.assertTrue(group["trend"]["status_changed"]) + self.assertEqual( + group["trend"]["status_change"], + {"from": "blocked", "to": "completed"}, + ) self.assertEqual(group["trend"]["unexpected_findings_delta"], -1) + self.assertEqual(group["trend"]["mapping_target_count_delta"], 0) + self.assertIn("Trend improved", group["trend"]["summary_text"]) self.assertEqual( group["trend"]["evidence_result_deltas"], {"blocked": -1, "manual": 1, "skipped": 1}, diff --git a/workplans/GUIDE-BOARD-WP-0007-report-and-export-maturity.md b/workplans/GUIDE-BOARD-WP-0007-report-and-export-maturity.md index 771fd86..eb7fb72 100644 --- a/workplans/GUIDE-BOARD-WP-0007-report-and-export-maturity.md +++ b/workplans/GUIDE-BOARD-WP-0007-report-and-export-maturity.md @@ -4,12 +4,12 @@ type: workplan title: "Report And Export Maturity" repo: guide-board domain: markitect -status: active +status: completed owner: codex planning_priority: medium planning_order: 7 created: "2026-05-15" -updated: "2026-05-15" +updated: "2026-05-16" state_hub_workstream_id: "ef9351d2-e99c-470e-aeec-f17aa51eae14" --- @@ -41,7 +41,7 @@ extension-provided fragments or exporters. ```task id: GUIDE-BOARD-WP-0007-T001 -status: todo +status: done priority: high state_hub_task_id: "bf3fe163-b06d-4c2e-9b45-31721864e1f2" ``` @@ -55,11 +55,19 @@ Acceptance: summary, and source lock data. - Add a fixture fragment and tests. +Progress: + +- Added Markdown file and Python module report fragment descriptors. +- Loaded fragment paths safely from extension roots. +- Added fragment context for assessment package, evidence, findings, mappings, + policy summary, and source lock data. +- Added an SDK fixture Python report fragment and focused tests. + ## D7.2 - Portable Export Formats ```task id: GUIDE-BOARD-WP-0007-T002 -status: todo +status: done priority: high state_hub_task_id: "fda51e62-98aa-408e-a057-4db40fe7c644" ``` @@ -72,11 +80,20 @@ Acceptance: - Preserve certification boundary and source lock references in each export. - Document which exports are generic and which must remain extension-owned. +Progress: + +- Added `docs/schemas/export-manifest.schema.json`. +- Wrote `exports/export-manifest.json` for each run. +- Included source package refs, source lock refs, summaries, policy summaries, + mapping summaries, report fragments, counts, and certification boundary. +- Documented the generic export boundary; authority-specific formats remain + extension-owned. + ## D7.3 - Trend And Gate Report Improvements ```task id: GUIDE-BOARD-WP-0007-T003 -status: todo +status: done priority: medium state_hub_task_id: "33c3089a-9d5e-4605-89c4-a1e070bc12ad" ``` @@ -89,11 +106,19 @@ Acceptance: - Keep machine-readable gate summaries stable for automation. - Add CLI report helpers or Markdown summaries where useful. +Progress: + +- Added trend status-change details, unresolved review deltas, mapping target + deltas, and compact summary text. +- Kept existing gate summary shape stable while improving trend inputs for + human review. +- Added assertions for the richer trend output. + ## D7.4 - Golden Fixtures And Documentation ```task id: GUIDE-BOARD-WP-0007-T004 -status: todo +status: done priority: medium state_hub_task_id: "66669f68-6728-4484-9ec9-267ffe025027" ``` @@ -106,6 +131,15 @@ Acceptance: - Ensure report text remains clear about preparation evidence versus formal certification. +Progress: + +- Added golden fixture expectations for report fragment text and export manifest + top-level shape. +- Updated extension SDK, assessment operations, architecture, service, container, + and README docs. +- Kept report and export boundary language tied to preparation evidence, not + formal certification. + ## Definition Of Done - Extensions can contribute report fragments through a documented contract.